ponylang / pony-stable Goto Github PK

Folks are confusing stable with thinking its the stable release of Pony. We initially discussed renaming to stable but @jemc pointed out that could be confusing as well.

We would need to rename the repo, and any packages such as brew etc as well as check all documentation.

Please submit your thoughts on names below and we can vote on them. If you like a particular name please use the 👍 reaction.

I believe this is mostly adding in order as we verify everything:

1- PR support for FreeBSD
2- Nightly builds
3- Release builds

Because GitHub actions don't have FreeBSD support, we'll need to use CirrusCI.

I did there "trigger on tag" option

https://github.com/ponylang/pony-stable/blob/master/.appveyor.yml#L7

But it didnt work for 0.2.2. Need to figure out in a test repo before next release.

Right now pony-stable still has no concept of versions.

I'd like pony-stable to follow the model of Rust's Cargo but we can avoid version ranges for now.

A basic package manager needs to do implement these rough functions.

1. Define the dependencies inside a manifest.

2. Optional: If version ranges will be supported the package manager has to use constraint solving to generate the current dependency graph and save it in a lockfile. The lockfile has to be commited to the repository to ensure reproducible builds.

3. Fetch the dependencies and store them in this flat path structure:

<package-name>/<version>/<package-contents>

(Let's avoid the recursive mess that NPM used to create before version 3.)

4. The last step is to give the compiler information about the location of the packages.

pony-stable already does 1., 3., 4. partially but it's still missing features such as version information.

What I'd like to see is the ability to specify the version in the manifest.

Without version ranges there are two primary version types in this scenario:
static/pinned versions like 2.5.3 which could be denoted '=2.5.3' (in line with NPM and Cargo)
moving versions like latest or master that follow a branch which could be denoted as ':latest' (afaik no package manager uses colons as a prefix)

Side Note: These symbols were chosen to leave room for version ranges if the need arises.

Step 1., 2., 3. can be implemented without modifying ponyc.

Step 4. is problematic when there is more than one version of a single package.

Here is an example dependency graph:

A version 1.0.1
    B version 1.3.5
       C version 2.3.0
    D version 1.0.9
       C version 1.4.0

Right now there can only be one version of C.
When B uses "C" the compiler looks for .deps/C rather than .deps/C/2.3.0
and D uses "C" it also chooses .deps/C rather than .deps/C/1.4.0.

Ponyc has to be changed to support some sort of "package redirection" for specific packages.

Alternatively we could also use pony_packages and redirect from pony_packages/C to .deps/C/2.3.0 on the filesystem layer via symlinks.

I'm still not sure how to pass on the redirection information to the compiler.

Here are the two approaches I've figured out so far.

Package specific PONYPATHs via command line parameters. The path of the package is specified and the paths of the dependency packages. The command may become very large and hit limits of the CLI.

ponyc --package-path /path/to/package/D/1.0.9=/path/to/package/C/1.4.0 --package-path /path/to/package/B/1.3.5=/path/to/package/C/2.3.0

The alternative is that pony-stable generates a json file that contains this information and the compiler reads the redirect information from it.

{
    path: "/path/to/project"
    dependencies: {
        "B": {
            path: "/path/to/project/.deps/B/1.3.5",
            dependencies: {
                "C": {
                    path: "/path/to/project/.deps/C/2.3.0",
                    dependencies: {}
                }
            }
        },
        "D": {
            path: "/path/to/project/.deps/D/1.0.9",
            dependencies: {
                "C": {
                    path: "/path/to/project/.deps/C/1.4.0",
                    dependencies: {}
                }
            }
        }
    }
}

I tried to run stable add local-path a couple of times, as per the README instructions (fixed by #20), and nothing would happen.

stable should print an error instead in this case

That prints the version ala ponyc.

➜  sean ponyc --version
0.19.1-021e3d4b [release]
compiled with: llvm 3.9.1 -- Apple LLVM version 8.0.0 (clang-800.0.42.1)

Summary

If one of your dependencies points to a git tag/commit rather than master then running stable a second time can result in merge conflicts which can then cause the checkout of the branch to fail, leaving the .deps folder in a bad state.

Steps to Reproduce

1. Create a bundle.json file that uses a tag as the version. I'm using this:

{
"deps": [
  {
    "type": "local",
    "local-path": "../../lib"
  },
  {
    "type": "local",
    "local-path": "lib"
  },
  {
    "type": "local",
    "local-path": "../../go_api/pony"
  },
  {
    "type": "github",
    "repo": "WallarooLabs/pony-kafka",
    "tag": "ee70576"
  }
]
}

2. Run stable fetch twice.

Expected Behavior

Stable fetches the correct version of the library (pony-kafka in the case of the example above).

Actual Behavior

There is a merge conflict when pulling in the repo and so the git checkout command fails, leaving the .deps directory in a bad state.

Notes

I talked to @SeanTAllen and @jemc about it and we think the right thing to do for now is to call git -C [root] fetch instead of git -C [root] pull [url] (here)[https://github.com/ponylang/pony-stable/blob/master/stable/dep.pony#L54]. There's probably a more efficient way to do this in the future, but for now that's probably the best solution.

This is a rather simple bug: when a repository is added through stable, its git submodules do not get initialized.

This, of course, causes issues with building the code afterwards, as some of the libraries/code will be missing.

Example: (using any repository with submodules work)

stable add github adri326/pony-sdl-mixer
# should be populated off https://github.com/LambdAurora/lcmm
ls .deps/adri326/pony-sdl-mixer/cmake/

The current workaround is to simply cd to the different libraries's folders and to run git submodules update --init --recursive in them

We need to switch to setting the version via a -D define rather than text replacement.

Currently if you build stable, change the version, and build again, you won't pick up the version change. Using sed to change the string is error prone and not what we want to be doing.

We currently have some kind of template for setting up a new project: https://github.com/ponylang/library-project-starter but setting this up is a tiny bit cumbersome as one has to replace all variables defined therein somehow manually or with a custom sed script or the like.

My suggestion is somehow inspired by giter8 and sbt which exist in the scala world. With these tools it is possible to define a template as a github repository containing variable placeholders (and default values for some or all of these). And then just call sbt new <github-org>/<repo> to populate a directory with all the boilerplate for some kind of setup. There can be any kind of templates for different setups and different tastes. Creating them is actually super-easy.

It would be very awesome to have something similar for ponylang. This entails the following moving parts:

• a very simple templating engine (just replacing variables in files from a given set of values in a Map)
• a new stable subcommand (new would be my suggestion) that uses the template repository to template-replace all files into a given location
  • this should ask for values for every found variable in the template first before actually replacing them, or ask for input for them on hitting them in the process
• maybe this should honour a special file containing default values (e.g. .stable-new-defaults.ini)
• rework the https://github.com/ponylang/library-project-starter to be compatible with this setup

Any thoughts on this?

We should be directing Linux users to use ponyup to install stable. Should include a link to installation instructions for ponyup in the ponyup repository.

Currently the dependencies are listed in an array. With this its possible to list the same dependency twice, even with different tags.

Dependencies should be unique with a set of properties, so an object would be better.

Running commands using pony stable doesn't seem to do anything on OSX.

I have a bundle.json

{
  "type": "github",
  "repo": "oraoto/pony-websocket"
}

When running stable fetch nothing happens.

I then tried emptying bundle.json then running stable add github oraoto/pony-websocket. Nothing happened again, bundle.json was still empty.

I then tried stable add github jemc/pony-inspect as suggested in the pony-stable help command, still no effect.

> stable version
0.1.1 [release]

With the closing of #101, we should do a new release.

Currently neither the readme nor pony-stable provide/explain a way to initialise a project.

I had to look at other repos and projects in pony to find how to structure the bundle.json. It's not explained in the readme how to get started.

A common solution is to add an init command to pony-stable to initialise the config file as seen in other package managers.

Let's say that stable first loads bundle.json and then it overrides the json object (dict merge) with the content of bundle.*.json (taken in alphabetical order, the last one has precedence). With this mechanism one can commit bundle.json for their user to use and use local repos while they develop the package.

This can be deactivated by a switch in the stable command line so we are sure that the build passes before a release or a git push to origin.

A further refinement would be a command that updates bundle.json with the versions of the repos defined in bundle.local.json before the release.

I can come up with a PR if the idea sounds good to you.

Going to give this a go.

Will make it easier for folks to set up their own CI jobs that need stable, if prebuilt stable versions are available.

• available via Bintray for Windows
• available via Bintray for RPM
• available via Bintray for Deb
• available via Homebrew for OSX

Needs #23

/> tree
.
├── pony-bar
│   ├── bar
│   │   └── bar.pony
│   └── bundle.json
└── pony-foo
    └── foo
        └── foo.pony
/> cd pony-bar
/pony-bar> stable add local ../pony-foo
/pony-bar> cd bar
/pony-bar/bar> stable env ponyc -d
Building builtin -> /usr/local/Cellar/ponyc/0.15.0/packages/builtin
Building . -> /pony-bar/bar
Error:
foo: couldn't locate this path
Error:
/pony-bar/bar/bar.pony:1:1: can't load package 'foo'
use "foo"
^

Because I'm in the /pony-bar/bar folder, the ../pony-foo path refers to /pony-bar/pony-foo, which obviously doesn't exist. This seems unintuitive to me. I would expect all paths to be relative to the bundle.json file.

This probably affects local-git dependencies as well.

Temporily turned off until we are providing .tar.gz files for download rather than packages. Once that is done, we can more easily create environments where the previous version of stable is available so we can fetch the regex dependency to build the current stable.

See #108.

The "integration.Env(invalid bundle.json in nested dir)" periodically fails. So far twice since it was merged. Here's an example:

https://circleci.com/gh/ponylang/pony-stable/204

Need to figure out why and address.

cc @srenatus

That reference bintray and homebrew.

I'm attempting to install this on Windows 10 in order to start working with Pony, but when I run make.bat as specified in the installation instructions, somewhere around 2 dozen syntax errors are spat out, mostly errors relating to it apparently not recognizing partial function calls as valid syntax, a la args(0)?.

I have followed the installation instructions exactly for both Pony and Stable (aside from the C++ build tools and Windows SDK, which I already had installed), and am running the most recent version of Pony as of this writing. File permissions are not a concern as I'm running make.bat with administrator permissions. I've included the full make.bat log below.

Compiling: ponyc stable -o build\release
Building builtin -> C:\Program Files (x86)\ponylang\packages\builtin
Building stable -> C:\Program Files (x86)\ponylang\pony-stable\stable
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\add.pony:6:23: syntax error: unexpected token ? after type, interface, trait, primitive, class or actor definition
    let kind = args(0)?
                      ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\add.pony:27:32: syntax error: unexpected token ? after type, interface, trait, primitive, class or actor definition
    info.data("repo") = args(0)?
                               ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\add.pony:40:38: syntax error: unexpected token ? after type, interface, trait, primitive, class or actor definition
    info.data("local-path") = args(0)?
                                     ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\add.pony:52:38: syntax error: unexpected token ? after type, interface, trait, primitive, class or actor definition
    info.data("local-path") = args(0)?
                                     ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\bundle.pony:17:47: syntax error: unexpected token ? after type, interface, trait, primitive, class or actor definition
    let bundle_path = path.join("bundle.json")?
                                              ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:11:5: syntax error: unterminated match expression
    match info.data("type")?
    ^
    Info:
    C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:11:28: expected terminating end before here
        match info.data("type")?
                               ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:32:7: syntax error: unterminated try expression
      try info.data("repo")? as String
      ^
    Info:
    C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:32:28: expected terminating end before here
          try info.data("repo")? as String
                               ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:76:7: syntax error: unterminated try expression
      try info.data("local-path")? as String
      ^
    Info:
    C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:76:34: expected terminating end before here
          try info.data("local-path")? as String
                                     ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:119:7: syntax error: unterminated try expression
      try info.data("local-path")? as String
      ^
    Info:
    C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:119:34: expected terminating end before here
          try info.data("local-path")? as String
                                     ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:136:5: syntax error: unterminated match expression
    match git_tag
    ^
    Info:
    C:\Program Files (x86)\ponylang\pony-stable\stable\dep.pony:138:59: expected terminating end before here
          Shell("cd " + root_path + " && git checkout " + str)?
                                                              ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\main.pony:11:13: syntax error: unterminated try expression
    command(try env.args(1)? else "" end, env.args.slice(2))
            ^
    Info:
    C:\Program Files (x86)\ponylang\pony-stable\stable\main.pony:11:28: expected terminating end before here
        command(try env.args(1)? else "" end, env.args.slice(2))
                               ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\shell.pony:26:41: syntax error: unexpected token ? after type, interface, trait, primitive, class or actor definition
    apply(consume command, exit_code_fn)?
                                        ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\_bundle_locator.pony:7:23: syntax error: unterminated try expression
      let candidate = try
                      ^
    Info:
    C:\Program Files (x86)\ponylang\pony-stable\stable\_bundle_locator.pony:8:50: expected terminating end before here
              FilePath(env.root as AmbientAuth, path)?.join("bundle.json")?
                                                     ^
Error:
C:\Program Files (x86)\ponylang\pony-stable\stable\_subdir_name_generator.pony:14:39: syntax error: expected then after condition expression
            elseif acc(acc.size() - 1)? != dash_code then

Hello, just out of curiosity. Why isn't pony-stable bundled with the language itself?

We have no documentation currently for how to use stable and bundle.json.

For updated release process to test out COPR/PPA packaging.

Incorporate arch fix

stable != staple, and stable is a bit misleading. now it's still time, later might be too late

We want to release 0.3.0 once we have merged #95 and #91.

This is my thought on approach. Let's do:

• release related portions of rename first in particular, dockerfiles etc. (including renaming this repo)

by doing this we won't break existing stable usage and have a weird "if i use master if is one thing, if i use release it is another". that would be a pain to deal with.

• merge #95

• update #91 to make additional changes related to #95

• merge #91

Currently we use the deprecated options package.
We should port stable/tack to cli instead.

This will resolve the problem that currently stable is unable to show a help dialogue for the subcommands, which makes it a little harder to use right now.

For Ponyc 0.22.x compat + bug fix from @aturley

As demonstrated in this demo app, running
stable env ponyc in a project that contains cyclic dependencies creates a process that never finishes, consuming more and more memory until it's killed.

Bug report

Current behavior

Cyclic dependencies cause stable env ponyc to fall into an infinite loop.

Expected behavior

Cyclic dependencies get compiled, as happens when using 'plain' ponyc.

Software versions where bug appears:

• OS: Linux tmr.local 4.19.20_1 #1 SMP PREEMPT Thu Feb 7 06:55:19 UTC 2019 x86_64 GNU/Linux
• Ponyc: 0.25.0-232a0abe [release] compiled with: llvm 3.9.1 -- cc (GCC) 8.2.0 Defaults: pic=true ssl=openssl_0.9.0
• Stable: 0.1.3-353733f [release]

Platform: Manjaro amd64
CPU: Intel i5-2450M (a very old one)

pony-stable was installed (version 0.2.2-2) from community repo.

It can not run 😢, just says: "[1] PID illegal hardware instruction (core dumped) stable".
That's super weird, I have never seen this when using other pony applications.

I guess this should be the problem of ponyc.

The regex library has been removed from ponyc and now lives in its
own library. This causes a bit of a problem for usage with stable.

How would we install that library so we can test stable? With the
stable we just built? Perhaps. Regex is only used in an integration
test and it can probably be accomplished in another fashion.

This issue is to reactivate the small amount of code that is currently commented out in the integration tests that uses regex.

@srenatus, do you have any thoughts on how to address?

See #108 for the code in question.

Let's get that gitlab support out there for the world and all those folks #movingtogitlab.

The library starter project has a new release script (first version). It's similar to the ponyc one but takes 2 extra args and posts the release notes on GitHub.

I'm tracking an error that makes pony-stable dump core in certain cases.
To reproduce, just install pony-stable in a machine and run "stable version" would be enough to reproduce it. It'll error as: "Illegal instruction (core dump)"

Errors in: i5-3320M, i5-2520M
Works fine with i7-4510U.

Arch Linux Forum thread:
https://bbs.archlinux.org/viewtopic.php?id=251577

Arch pony-stable package bug report:
https://bugs.archlinux.org/task/64934

Dockerfiles helping triage the problem:
https://gist.github.com/superherointj/fe031e0255d86fc40fc333a7e14de310

It happens in both of these kernels:
Linux x220-arch 4.19.91-1-lts #1 SMP Sat, 21 Dec 2019 16:34:46 +0000 x86_64 GNU/Linux
Linux x220-arch 5.4.6-arch1-1 #1 SMP PREEMPT Sat, 21 Dec 2019 16:34:41 +0000 x86_64 GNU/Linux

I'll add more information here as soon as I have more time to do it.

After the change @dipinhora made to allow for the use of stable with projects that don't define a bundle.json, I switched my emacs flycheck to always use stable if it is installed. This has an interesting effect that when used in a project with no bundle.json that we get an error due to a non-zero exit code.

If not having a bundle.json is ok, should the exit code be 0 instead?

This release is mainly to test out the new bintray deb packaging process.

We should set up Travis and Appveyor to verify that pony-stable builds. From there, we can move to potentially adding tests.

• Travis
• Appveyor

At the moment, the best we can do when checking a PR is to verify that stable builds. We have no automated tests of basic functionality. I'm opening this issue to collect and track thoughts on what tests we should add.

There are a couple in the pony release image usage that can be removed on the next pony release.
Same for the changelog-tool but, that requires a new changelog-tool release.

When executing

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys "D401AB61 DBE1D0A2"

I get the following result:

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys "D401AB61 DBE1D0A2"
Executing: /tmp/apt-key-gpghome.B3RuvWpOpZ/gpg.1.sh --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys D401AB61 DBE1D0A2
gpg: "D401AB61 DBE1D0A2" not a key ID: skipping