distribution-scripts's People
Forkers
bcardiff matiasgarciaisaia rx14 j8r txdv straight-shoota eyusupov oprypin daniel-worrall blacksmoke16 f4tq maxfierke efeali44 hovsater caspiano beta-ziliani electrocucaracha priya1puresoftware isabella232 initdc chriscz seanpm2001 mirydistribution-scripts's Issues
Automate testing of changes to distribution-scripts
After every change to distribution-scripts, that affect the release packaging process, we need to test if they work in the CI workflow of https://github.com/crystal-lang/crystal/.
This could be automated. For every PR, a corresponding test branch needs to be created at https://github.com/crystal-lang/crystal/. The branch name must start with ci/
to trigger a maintenance build. This could just be prefixed to the branch name in https://github.com/crystal-lang/distribution-scripts.
That test branch must contain a commit which updates the commit SHA in https://github.com/crystal-lang/crystal/blob/5663eb00613114a7d6a784428f1d0af030a2aa09/.circleci/config.yml#L147 to the HEAD of the distribution-scripts branch. For this purpose we should probably extract that reference to an easily modifyable value.
For every subsequent commit to the distribution-scripts branch, there should be a respective update to the test branch.
A comment should be posted to the PR with a link to the test branch and the corresponding workflow run at circleCI.
Vagrant box
There is official box provision located at vagrantup, but the last update is quite far from what we have right now.
As development on Docker for MacOS is barely usable, some of us would appreciate an update to above box with latest crystal-lang
releases.
Thanks!
RHEL 8+ is unsupported
Installation script does not work with Red Hat Enterprise Linux of version 8 and above (current 8.5) because of OBS not set up for any above 7, current workaround is to use CentOS 8 package
Missing VERSION file in Debian and Arch packages
The VERSION file that is used by compiler/crystal/config.cr is missing from both Debian and Archlinux packaging.
This leads to errors when using the Crystal::Config
package.
To reproduce (full log):
$ docker run --rm crystallang/crystal:0.28.0 sh -c 'apt-get update && apt-get install -y llvm-dev && crystal eval '"'"'require "compiler/crystal/**"; puts Crystal::Config.default_config'"'"''
...
in usr/share/crystal/src/compiler/crystal/config.cr:11: error executing command: cat /usr/share/crystal/src/compiler/crystal/../../../VERSION, got exit status 1
{{ `cat #{__DIR__}/../../../VERSION`.stringify.chomp }}
There is no more issue when creating the missing VERSION file on the two distributions.
However, for Archlinux, this file is located at /usr/lib/VERSION which might be problematic :-)
I'm not really sure if this is a packaging or code issue. Please just let me know if I should open an issue in the core's repo.
More info about the packages:
Debian
$ pacman -Ql crystal | grep -q VERSION && echo OK || echo KO
KO
$ pacman -Qi crystal
Name : crystal
Version : 0.28.0-1
Description : The Crystal Programming Language
Architecture : x86_64
URL : http://crystal-lang.org
Licenses : Apache
Groups : None
Provides : None
Depends On : gc libatomic_ops pcre libevent llvm6-libs
Optional Deps : shards: crystal language package manager [installed]
libyaml: For YAML support [installed]
gmp: For BigInt support [installed]
libxml2: For XML support [installed]
Required By : None
Optional For : None
Conflicts With : None
Replaces : None
Installed Size : 95.62 MiB
Packager : Anatol Pomozov <[email protected]>
Build Date : Wed 17 Apr 2019 07:41:48 PM CEST
Install Date : Sun 28 Apr 2019 08:04:36 PM CEST
Install Reason : Explicitly installed
Install Script : No
Validated By : Signature
Archlinux
$ dpkg -L crystal | grep -q VERSION && echo OK || echo KO
KO
$ apt-cache show crystal
Package: crystal
Status: install ok installed
Priority: extra
Section: default
Installed-Size: 88303
Maintainer: Crystal Team <[email protected]>
Architecture: amd64
Version: 0.28.0-1
Depends: gcc, pkg-config, libpcre3-dev, libevent-dev
Recommends: git, libssl-dev, libz-dev
Suggests: libxml2-dev, libgmp-dev, libyaml-dev, libreadline-dev
Description: no description given
Description-md5: c0af8b65ef8df63b3bfb124d96da1778
License: APACHE-2.0
Vendor: Crystal Team <[email protected]>
Homepage: https://crystal-lang.org/
(I only tested with Crystal 0.28.0 on Debian Buster and an up to date Archlinux)
Run ecosystem test after a mainteinance_release
Perhaps we need to split the workflow maintenance_release
so it only runs this step after a release/
branch. Now it also runs on a ci
branch.
Add resource_class to CircleCI jobs
CircleCI has a field to specify the size of the computer. It's not required, but probably a good idea to set. Probably we could stick with medium+
(that's 6GB RAM).
Release Crystal 1.3.0
Release preparation
- Announce expected release date and time span for feature freeze (https://forum.crystal-lang.org/t/crystal-1-3-lets-see-what-the-three-wise-men-bring-us/4125)
- Feature freeze is about two weeks before release
- Set date on milestone
- Start preparing changelog and release notes
- Start feature freeze period
- Either no merging of features into
master
or split off release branch for backporting bugfixes.
- Either no merging of features into
- Publish release PR draft (crystal-lang/crystal#11599)
- It should contain the expected date of the release (~two weeks after the PR is issued).
- It should be populated with updates to
CHANGELOG.md
andVERSION
.
- Publish release notes PR (crystal-lang/crystal-website#219)
- Ensure documentation for language and compiler changes and other relevant changes is up to date.
- Crystal Book
- Update language specification
- Update compiler manual
- Add or update guides / tutorials?
- Crystal Book
Release process
Source release
- Finalize the release PR
- Make sure all changes are mentioned in the changelog
- Check release date
- Un-draft the PR
- Split off release branch (
release/x.y
) - Verify Maintenance CI workflow succeeds on the HEAD of the release branch (https://app.circleci.com/pipelines/github/crystal-lang/crystal/7972/workflows/72a0bc71-41ce-4dfe-86fd-05845a19ec4c)
- Smoke test with test-ecosystem
- Run *Test Crystal & Shards Workflow with the release branch as
crystal_branch
. (https://github.com/crystal-lang/test-ecosystem/actions/runs/1662578953)
- Run *Test Crystal & Shards Workflow with the release branch as
- Merge the release PR
- Tag & annotate the commit with the changelog using
<M.m.p>
pattern as {version} (as a pre-release directly in GH?)git tag -s -a -m $VERSION $VERSION
- Publish Github release
- Copy the changelog section as description
- Binaries are added later
- Close milestone
Publish documentation for the release
- Publish API docs
- Have
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
env variables defined- Keys can be generated at https://console.aws.amazon.com/iam/home#/security_credentials (contact a Manas admin if you don't have access).
- Run
make -C docs publish_docs CRYSTAL_VERSION=${VERSION}
to publish docs toapi/${VERSION}
- Run
make -C docs dist-redirect_latest CRYSTAL_VERSION=${VERSION}
to apply redirect fromapi/latest
toapi/${VERSION}
- Have
- Publish language reference (TBD)
Binary releases
- Wait for the release build in circle CI
- Smoke test with test-ecosystem (again) (https://github.com/crystal-lang/test-ecosystem/actions/runs/1663249585)
- Attach build artifacts to Github release
crystal-*-darwin-*.tar.gz
crystal-*-linux-*.tar.gz
crystal-*.pkg
crystal-*-docs.tar.gz
- Push changes to OBS for building linux packages
- Checkout https://github.com/crystal-lang/distribution-scripts and go to
./packages
- Configure build.opensuse.org credentials in environment variables:
export OBS_USER=
export OBS_PASSWORD=
- Run
./obs-release.sh devel:languages:crystal crystal $VERSION
- Uses the docker image
crystallang/osc
to run the CLI client for OBS. - The script creates a branch in you home project, updates the version and pushes it back to OBS.
- You can also run the commands from that file manually and check build locally with
osc build xUbuntu_20.04 x86_64
osc build Fedora_Rawhide x86_64
- Uses the docker image
- Now OBS builds the packages. It’s best to follow the build status in the browser:
open https://build.opensuse.org/project/show/home:$OBS_USER:branches:devel:langauges:crystal/crystal
- Wait for all package build jobs to finish and succeed
- Verify package installation
OBS_PROJECT=home:$OBS_USER:branches:devel:languages:crystal bats test
- When everything is green, create a submit request against the original package (Submit package link in the menu bar on the package in your branch)
- Checkout https://github.com/crystal-lang/distribution-scripts and go to
- Tag
latest
docker images- Versioned docker images have been pushed to dockerhub.
- Now just assign the
latest
tags: $ ./docker/apply-latest-tags.sh {version}
- Publish snap package
- You need to logged in via
$ snapcraft login
- Recent tagged release is published directly to edge channel. The CI logs the snap revision number. Otherwise the .snap file is in the artifacts.
- Check the current status to find the revision of the tagged release otherwise:
$ snapcraft status crystal
$ snapcraft release crystal <revision-number> beta
$ snapcraft release crystal <revision-number> stable
- You need to logged in via
- Submit a PR to update the homebrew formula in https://github.com/Homebrew/homebrew-core/blob/master/Formula/crystal.rb .
- Update the previous and new version (with their respective hashes).
- Try locally
$ brew install --build-from-source <source of formula>
- Create PR
Release announcements
- Publish release notes on the website
- Post announcement in https://forum.crystal-lang.org/c/news/official
- Tweet about the release
- Post in Reddit
- Update https://github.com/crystal-lang/crystal-book/blob/master/crystal-version.txt
Post-release
- Update crystal
master
branch to use released version- Edit PREVIOUS_CRYSTAL_BASE_URL in
.circleci/config.yml
- Edit DOCKER_TEST_PREFIX in
bin/ci
- Edit
prepare_build
on_osx download package and folder - Edit
.github/workflows/*.yml
to point to docker image - Edit
shell.nix
latestCrystalBinary
usingnix-prefetch-url --unpack <url>
- Edit PREVIOUS_CRYSTAL_BASE_URL in
- Increment VERSION file to the next minor and -dev suffix
- Perform uncomment/todos left in the repo
Integrate aarch64 into the building /release process
Missing man pages
Missing man pages for crystal, shards, shard.yml (5).
Steps to Reproduce
$ man shards
No manual entry for shards
$ man shard.yml
No manual entry for shard.yml
$ man 5 shard.yml
No manual entry for shard.yml in section 5
Version Info
$ which crystal
/var/lib/snapd/snap/bin/crystal
$ which shards
/var/lib/snapd/snap/bin/shards
$ snap info crystal | grep installed
installed: 0.36.1 (600) 23MB classic
$ snap version
snap 2.49-2.fc32
snapd 2.49-2.fc32
series 16
fedora 32
Sometimes packages are built without static linking
0.28.0 32 bits .deb packages went out without static linking.
Yet nightlies worked fine https://circleci.com/gh/crystal-lang/crystal/21793#artifacts/containers/0
The tagged one are not statically linked https://circleci.com/gh/crystal-lang/crystal/21811#artifacts/containers/0
The non-static link also happened in maintenance release ci/preview.
But since last nightlies, before 0.28 worked I thought tagged version would work.
Publish a Windows build (crystal.exe)
Right now it is kinda-possible to get a pre-built Crystal (.exe) for Windows, by clicking into one of the GitHub Actions runs. Unfortunately, this requires being logged in to GitHub Web (yes, no anonymous access through web and no access through APIs at all).
Of course, the end game will be to have a Crystal build for Windows as part of the Releases, but right now there isn't yet such a release to speak of (0.34.0 didn't have essential fixes for Windows yet).
My main point here is that we should be ready for that when the release comes. It would simplify a lot of workflows (including Crystal's own CI), and I'm personally very excited about it.
The minimal thing to publish might even be the artifact archive that the workflow produces (at the commit corresponding to the release tag); for start it would suffice to manually download it from there and upload it under the release. (We could also tell GitHub Actions to do that automatically when a tag appears, but it requires the release to also already be there, which might not fit with the current release workflow.)
Of course, the contents of the artifact (just crystal.exe and static libs) aren't exactly in line with the other systems' releases, which have share/crystal/src (source code) and bin/crystal (shell wrapper, to which there is currently no Windows equivalent). And there have been no discussions at all about the directory structure that Crystal should be installed as. But I would strongly prefer having some download even if these details aren't fleshed out.
Publishing the nightlies themselves somewhere would also be nice. I would appreciate having an official download URL for any crystal.exe even before the release happens, but maybe that's asking too much right now.
And I hear that the story with Mac nightlies also isn't so great as it requires hoops through CircleCI APIs, so maybe there's some common solution here.
Release Crystal 1.3.2
Release preparation
- Start preparing changelog and release notes (crystal-lang/crystal-website#252)
- Publish release PR draft (crystal-lang/crystal#11751)
- It should be populated with updates to
CHANGELOG.md
andVERSION
.
- It should be populated with updates to
Release process
Source release
- Finalize the release PR
- Make sure all changes are mentioned in the changelog
- Check release date
- Un-draft the PR
- Verify Maintenance CI workflow succeeds on the HEAD of the release branch
- Smoke test with test-ecosystem
- Run *Test Crystal & Shards Workflow with the release branch as
crystal_branch
.
- Run *Test Crystal & Shards Workflow with the release branch as
- Merge the release PR
- Tag & annotate the commit with the changelog using
<M.m.p>
pattern as {version} (as a pre-release directly in GH?)git tag -s -a -m $VERSION $VERSION
- Publish Github release
- Copy the changelog section as description
- Binaries are added later
- Close milestone
Publish documentation for the release
- Publish API docs
- Have
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
env variables defined- Keys can be generated at https://console.aws.amazon.com/iam/home#/security_credentials (contact a Manas admin if you don't have access).
- Run
make -C docs publish_docs CRYSTAL_VERSION=${VERSION}
to publish docs toapi/${VERSION}
- Run
make -C docs dist-redirect_latest CRYSTAL_VERSION=${VERSION}
to apply redirect fromapi/latest
toapi/${VERSION}
- Have
Binary releases
- Wait for the release build in circle CI
- Smoke test with test-ecosystem (again)
- Attach build artifacts from circleci and GitHub Actions (Windows) to GitHub release
crystal-*-darwin-*.tar.gz
crystal-*-linux-*.tar.gz
crystal-*.pkg
crystal-*-docs.tar.gz
crystal.zip
(GHA) ->crystal-$VERSION-windows-x86_64-msvc-unsupported.zip
- Push changes to OBS for building linux packages
- Checkout https://github.com/crystal-lang/distribution-scripts and go to
./packages
- Configure build.opensuse.org credentials in environment variables:
export OBS_USER=
export OBS_PASSWORD=
- Run
./obs-release.sh devel:languages:crystal crystal $VERSION
- Uses the docker image
crystallang/osc
to run the CLI client for OBS. - The script creates a branch in you home project, updates the version and pushes it back to OBS.
- You can also run the commands from that file manually and check build locally with
osc build xUbuntu_20.04 x86_64
osc build Fedora_Rawhide x86_64
- Uses the docker image
- Run
./obs-release.sh devel:languages:crystal crystal${VERSION%.*} $VERSION
- Now OBS builds the packages. It’s best to follow the build status in the browser:
open https://build.opensuse.org/project/show/home:$OBS_USER:branches:devel:langauges:crystal/crystal
- Wait for all package build jobs to finish and succeed
- Verify package installation
OBS_PROJECT=home:$OBS_USER:branches:devel:languages:crystal bats test
- When everything is green, create a submit request against the original package (Submit package link in the menu bar on the package in your branch)
- Checkout https://github.com/crystal-lang/distribution-scripts and go to
- Tag
latest
docker images- Versioned docker images have been pushed to dockerhub.
- Now just assign the
latest
tags: $ ./docker/apply-latest-tags.sh {version}
- Publish snap package
- You need to logged in via
$ snapcraft login
- Recent tagged release is published directly to edge channel. The CI logs the snap revision number. Otherwise the .snap file is in the artifacts.
- Check the current status to find the revision of the tagged release otherwise:
$ snapcraft status crystal
$ snapcraft release crystal <revision-number> beta
$ snapcraft release crystal <revision-number> stable
- You need to logged in via
- Submit a PR to update the homebrew formula in https://github.com/Homebrew/homebrew-core/blob/master/Formula/crystal.rb .
- Update the previous and new version (with their respective hashes).
- Try locally
$ brew install --build-from-source <source of formula>
- Create PR
Release announcements
- Publish release notes on the website
- Post announcement in https://forum.crystal-lang.org/c/news/official
- Tweet about the release
- Post in Reddit
- Update https://github.com/crystal-lang/crystal-book/blob/master/crystal-version.txt
Post-release
- Update crystal
master
branch to use released version- Edit PREVIOUS_CRYSTAL_BASE_URL in
.circleci/config.yml
- Edit DOCKER_TEST_PREFIX in
bin/ci
- Edit
prepare_build
on_osx download package and folder - Edit
.github/workflows/*.yml
to point to docker image - Edit
shell.nix
latestCrystalBinary
usingnix-prefetch-url --unpack <url>
- Edit PREVIOUS_CRYSTAL_BASE_URL in
- Update default base version in test-ecosystem
Release Crystal 1.3.1
Release preparation
- Announce expected release date and time span for feature freeze
- Feature freeze is about two weeks before release
- Set date on milestone
- Start preparing changelog and release notes (crystal-lang/crystal-website#248)
- Start feature freeze period
- Either no merging of features into
master
or split off release branch for backporting bugfixes.
- Either no merging of features into
- Publish release PR draft (crystal-lang/crystal#11744)
- It should contain the expected date of the release (~two weeks after the PR is issued).
- It should be populated with updates to
CHANGELOG.md
andVERSION
.
- Ensure documentation for language and compiler changes and other relevant changes is up to date.
- Crystal Book
- Update language specification
- Update compiler manual
- Add or update guides / tutorials?
- Crystal Book
Release process
Source release
- Finalize the release PR
- Make sure all changes are mentioned in the changelog
- Check release date
- Un-draft the PR
- Split off release branch (
release/x.y
) - Verify Maintenance CI workflow succeeds on the HEAD of the release branch (https://app.circleci.com/pipelines/github/crystal-lang/crystal/8041/workflows/7425f8bb-f05d-433d-9740-21bbb3c34900)
- Smoke test with test-ecosystem
- Run *Test Crystal & Shards Workflow with the release branch as
crystal_branch
. (https://github.com/crystal-lang/test-ecosystem/runs/4806030075)
- Run *Test Crystal & Shards Workflow with the release branch as
- Merge the release PR
- Tag & annotate the commit with the changelog using
<M.m.p>
pattern as {version} (as a pre-release directly in GH?)git tag -s -a -m $VERSION $VERSION
- Publish Github release
- Copy the changelog section as description
- Binaries are added later
- Close milestone
Publish documentation for the release
- Publish API docs
- Have
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
env variables defined- Keys can be generated at https://console.aws.amazon.com/iam/home#/security_credentials (contact a Manas admin if you don't have access).
- Run
make -C docs publish_docs CRYSTAL_VERSION=${VERSION}
to publish docs toapi/${VERSION}
- Run
make -C docs dist-redirect_latest CRYSTAL_VERSION=${VERSION}
to apply redirect fromapi/latest
toapi/${VERSION}
- Have
- Publish language reference (TBD)
Binary releases
- Wait for the release build in circle CI
- Smoke test with test-ecosystem (again)
- Attach build artifacts to Github release
crystal-*-darwin-*.tar.gz
crystal-*-linux-*.tar.gz
crystal-*.pkg
crystal-*-docs.tar.gz
- Push changes to OBS for building linux packages
- Checkout https://github.com/crystal-lang/distribution-scripts and go to
./packages
- Configure build.opensuse.org credentials in environment variables:
export OBS_USER=
export OBS_PASSWORD=
- Run
./obs-release.sh devel:languages:crystal crystal $VERSION
- Uses the docker image
crystallang/osc
to run the CLI client for OBS. - The script creates a branch in you home project, updates the version and pushes it back to OBS.
- You can also run the commands from that file manually and check build locally with
osc build xUbuntu_20.04 x86_64
osc build Fedora_Rawhide x86_64
- Uses the docker image
- Now OBS builds the packages. It’s best to follow the build status in the browser:
open https://build.opensuse.org/project/show/home:$OBS_USER:branches:devel:langauges:crystal/crystal
- Wait for all package build jobs to finish and succeed
- Verify package installation
OBS_PROJECT=home:$OBS_USER:branches:devel:languages:crystal bats test
- When everything is green, create a submit request against the original package (Submit package link in the menu bar on the package in your branch)
- Checkout https://github.com/crystal-lang/distribution-scripts and go to
- Tag
latest
docker images- Versioned docker images have been pushed to dockerhub.
- Now just assign the
latest
tags: $ ./docker/apply-latest-tags.sh {version}
- Publish snap package
- You need to logged in via
$ snapcraft login
- Recent tagged release is published directly to edge channel. The CI logs the snap revision number. Otherwise the .snap file is in the artifacts.
- Check the current status to find the revision of the tagged release otherwise:
$ snapcraft status crystal
$ snapcraft release crystal <revision-number> beta
$ snapcraft release crystal <revision-number> stable
- You need to logged in via
- Submit a PR to update the homebrew formula in https://github.com/Homebrew/homebrew-core/blob/master/Formula/crystal.rb (Homebrew/homebrew-core#93105)
- Update the previous and new version (with their respective hashes).
- Try locally
$ brew install --build-from-source <source of formula>
- Create PR
Release announcements
- Publish release notes on the website
- Post announcement in https://forum.crystal-lang.org/c/news/official
- Tweet about the release
- Post in Reddit
- Update https://github.com/crystal-lang/crystal-book/blob/master/crystal-version.txt
Post-release
- Update crystal
master
branch to use released version- Edit PREVIOUS_CRYSTAL_BASE_URL in
.circleci/config.yml
- Edit DOCKER_TEST_PREFIX in
bin/ci
- Edit
prepare_build
on_osx download package and folder - Edit
.github/workflows/*.yml
to point to docker image - Edit
shell.nix
latestCrystalBinary
usingnix-prefetch-url --unpack <url>
- Edit PREVIOUS_CRYSTAL_BASE_URL in
- Increment VERSION file to the next minor and -dev suffix
- Perform uncomment/todos left in the repo
Release Shards 0.17.0
Release preparation
- Start preparing changelog and release notes
- Publish release PR draft (crystal-lang/shards#549)
- It should be populated with updates to
CHANGELOG.md
,VERSION
, andshard.yml
.
- It should be populated with updates to
Release process
- Finalize the release PR
- Make sure all changes are mentioned in the changelog
- Check release date
- Build man files with the release date:
$ make clean docs SOURCE_DATE_EPOCH=$(gdate -d "YYYY-MM-DD" +"%s")
- Un-draft the PR
- Smoke test with test-ecosystem
- Run Test Crystal & Shards Workflow with the release branch as
shards_branch
.
- Run Test Crystal & Shards Workflow with the release branch as
- Merge the release PR
- Tag & annotate the commit with the changelog using v
<M.m.p>
pattern as {version}git tag -s -a -m v$VERSION v$VERSION
- Publish Github release (https://github.com/crystal-lang/shards/releases/new)
- Copy the changelog section as description
- Close milestone (https://github.com/crystal-lang/shards/milestones)
Post-release
- Increment VERSION file to the next minor and -dev suffix
- Update distribution-scripts (#184)
- Update https://github.com/crystal-lang/crystal (crystal-lang/crystal#11930)
- Submit a PR to update the homebrew formula in https://github.com/Homebrew/homebrew-core/blob/master/Formula/crystal.rb . Or do it on Crystal release.
- Perform uncomment/todos left in the repo
Bumping LLVM version on macOS?
My understanding from this line:
is that LLVM for the binary distribution uses LLVM 6. Is there a particular reason for holding this back on LLVM 6?Would it be possible to bump this to something much newer, perhaps LLVM 10 (or ideally LLVM 11+)?
The reason I ask is that I believe either LLVM 10 or 11 is needed to target AArch64 (ARM64) on Darwin/macOS. If were to use a newer LLVM and also set -DLLVM_TARGETS_TO_BUILD=X86;AArch64
for the builds of LLVM used to build the distribution releases, we could use the official x86_64 binary release to bootstrap a ARM64 Homebrew build
Provide macOS PKG Welcome Message & License
Bug Report
When installing Crystal on macOS via the .pkg distributable:
This will install Crystal on your Mac.
This is a generic Omnibus welcome message. If you are building Mac OS X packages, you should generate a custom welcome file to suit your needs.
If you are not the creator of Crystal, the presence of this message is likely a bug and should be reported to the creator.
Similarly for the package's license:
All Rights Reserved
This is a generic Omnibus license file for Crystal. If you are building Mac OS X packages, you should generate a custom license file and description to suit your needs.
Missing bash and zsh completion in linux packages
The /etc/*
files in the crystal repo are not bundled
Snapcraft support `armhf` architecture?
Kudos for adding snapcraft support. 🎉 👏
Any chance armhf
architecture will be supported?
$sudo snap install crystal
error: snap "crystal" is not available on stable for this architecture (armhf)
but exists on other architectures (amd64).
Debian packages still have the homepage as http://example.com/no-uri-given
Uninstalling rpm package does not clear directories
It seems we need to use --auto-add-directories
or --directories
to the fpm call to build rpm (64 and 32 bits) in https://github.com/crystal-lang/distribution-scripts/blob/master/linux/Makefile
Extracted from crystal-lang/crystal#7421
Automate the OBS process
and make it part of the building process
Create aarch64-musl docker images
We can use alpine edge, since it has a reasonably updated version of crystal.
As ref: currently images are built using crystal-infrastructure.
Remove i386 build recipes
Since i386-linux-gnu
has been demoted to a tier 2 supported platform and builds are occassionally failing, I don't think we should continue to build packages for this architecture.
We have already announced that we won't publish 32-bit binary packages for Crystal 1.2 (https://crystal-lang.org/2021/09/30/preparing-1.2.html).
Add OpenSUSE support in BinTray install.sh
The new BinTray install.sh
script fails on OpenSUSE Tumbleweed 20200817 (and, I suspect, most OpenSUSE distributions).
In _discover_distro_type
, execution continues until if hash apt-get 2>/dev/null
, which evaluates as true and sets DISTRO_TYPE
to "deb"
, when, by default, OpenSUSE uses Zypper, which is RPM-based. Actually, the reason for that if
evaluating to true is that, for some reason, apt-get
"forwards" to Zypper by default on OpenSUSE instead of just not existing. DISTRO_TYPE
equal to "deb"
later causes apt-key
to fail because there is no such command.
That is, of course, only the first issue. The script as it stands seems to support only Yum and Apt, whereas OpenSUSE support would likely require the use of Zypper. I hope to put in a PR for this myself, but at least this issue will exist if I don't get around to it.
CI fails for newest nightly docker image
The newest nightly docker image of crystallang/crystal seems to have an issue.
My CI currently fails with the error message Could not execute 'crystal': Error executing process: 'crystal': No such file or directory
.
Trying this on my local machine gives the following output:
[blobcodes@david-pc ~]$ podman run crystallang/crystal:nightly shards --version
Shards 0.16.0 [ff94dd2] (2021-10-06)
[blobcodes@david-pc ~]$ podman run crystallang/crystal:nightly crystal --version
{"msg":"exec container process (missing dynamic library?) `/usr/bin/crystal`: No such file or directory","level":"error","time":"2021-11-30T10:29:58.000185975Z"}
[blobcodes@david-pc ~]$ podman run crystallang/crystal:nightly nonexistent --version
Error: executable file `nonexistent` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
So there seems to be a crystal binary, but some library seems to be missing.
Trigger `test_ecosystem` after the build in ci
Also, making sure it uses the binaries/images generated from there.
Clean up build artifacts
The following files are currently produced as build artifacts of the nightly release workflow on circleci (see https://app.circleci.com/pipelines/github/crystal-lang/crystal/8502/workflows/77e21f43-14ad-4840-91c7-53a30649ca20/jobs/68065).
crystal-1.4.0-dev-1-darwin-universal.tar.gz (46 MB)
crystal-1.4.0-dev-1-linux-x86_64-bundled.tar (95 MB) ❌
crystal-1.4.0-dev-1-linux-x86_64-bundled.tar.gz (30 MB)
crystal-1.4.0-dev-1-linux-x86_64.tar (94 MB) ❌
crystal-1.4.0-dev-1-linux-x86_64.tar.gz (29 MB)
crystal-1.4.0-dev-1-linux-x86_64.tar.xz (19 MB) ❓
crystal-1.4.0-dev-1.universal.pkg (46 MB)
crystal-1.4.0-dev-docs.tar.gz (13 MB)
crystal_1.4.0-dev_amd64.snap (24 MB) ❓
docker-1.4.0-dev-alpine-build.tar.gz (262 MB) ❓
docker-1.4.0-dev-alpine.tar.gz (173 MB) ❓
docker-1.4.0-dev-ubuntu-x86_64-build.tar.gz (370 MB) ❓
docker-1.4.0-dev-ubuntu-x86_64.tar.gz (208 MB) ❓
versions-1.4.0-dev.json (2.1 kB) ❌
I'm wondering about the purpose of some of them. I suppose some are just intermediary steps that just happen do end up being published.
- ❌ The uncompressed
.tar
files are redundant and more wasteful than the compressed counterparts. - ❓ Compressed packages are usually
tar.gz
, but the linux build also has atar.xz
. Is there a particular reason for this? The xc compression seems quite more efficient than gz. But it's still a question whether the duplication makes sense. Or if we should consider xc compressed files for other artifacts as well. - ❓ The docker images are already published to docker hub. The snap package is uploaded to snapcraft. Do they even need to be available as directly downloadable files?
- ❌ The
json
file is completely unnecessary. That's just a temporary artifact from the API docs build process.
This initiative is related to crystal-lang/crystal#11902.
Some of these artifacts probably don't need to be pushed at all, others may be useful for debugging but don't need to be made publicly available.
Move library build steps from docker image building
The bundled libraries in the linux tar.gz (currently only libgc
) depend on GNU libc. They don't work with musl libc. Thus, for the alpine docker images, we need a different libgc build that works with musl libc. This is currently done in docker/alpine.Dockerfile
:
distribution-scripts/docker/alpine.Dockerfile
Lines 14 to 29 in 6133116
This is subpar because it's mixing concerns. The build process of binary artifacts should be enclosed in the linux
stage, docker
should just be responsible for putting them in a docker image.
And it's actually a duplication: We're already building a musl-compatible libgc in the build process of the tar.gz.
So I believe we should have an additional tar.gz with musl-compatible libraries as an intermediary product. Maybe even publish that, but that's an optional enhancement. Main reason is to streamline alpine image generation.
Consider automating the `homebrew` publishing process
It requires querying the binaries of the last two versions and its hashes; not 🚀 science. It should be part of a tagged_release
.
Publish crystal images in Docker official
Ref: #120 (comment)
Snap package should use snap configuration to set env variables
The following environment variables are used in shards and crystal for some workflows.
- SHARDS_OPTS
- SHARDS_OVERRIDE
- SHARDS_CACHE_PATH
- CRYSTAL_OPTS
- CRYSTAL_PATH
- CRYSTAL_LIBRARY_PATH
- CRYSTAL_CACHE_DIR
The snap package does not support them because of its sandboxing.
The right way to support them is to use snap configuration, and to use snapctl get
inside the snap wrapper scripts for crystal and shards.
The publish of docs should be performed within CI
Also, it should be part of the tagged_release
workflow
Crystal release 1.2.2
Release preparation
- Announce expected release date and time span for feature freeze
- Feature freeze is about two weeks before release
- Start preparing changelog and release post
- Start feature freeze period
- Either no merging of features into
master
or split off release branch for backporting bugfixes.
- Either no merging of features into
- Publish release PR draft (crystal-lang/crystal#11391)
- It should contain the expected date of the release (~two weeks after the PR is issued).
- It should be populated with updates to
CHANGELOG.md
andVERSION
.
Release process
Source release
- Finalize the release PR
- Make sure all changes are mentioned in the changelog
- Check release date
- Un-draft the PR
- Split off release branch (
release/x.y
) - Verify Maintenance CI workflow succeeds on the HEAD of the release branch
- Smoke test with test-ecosystem
- Run *Test Crystal & Shards Workflow with the release branch as
crystal_branch
.
- Run *Test Crystal & Shards Workflow with the release branch as
- Merge the release PR
- Tag & annotate the commit with the changelog using
<M.m.p>
pattern as {version} (as a pre-release directly in GH?) - Publish Github release
- Copy the changelog section as description
- Binaries are added later
Binary releases
- Wait for the release build in circle CI
- Smoke test with test-ecosystem (again)
- Attach build artifacts to Github release
crystal-*-darwin-*.tar.gz
crystal-*-linux-*.tar.gz
crystal-*.pkg
crystal-*-docs.tar.gz
- Publish docs
- Have
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
env variables defined- Keys can be generated at https://console.aws.amazon.com/iam/home#/security_credentials (contact a Manas admin if you don't have access).
- Run
make -C docs publish_docs CRYSTAL_VERSION=${VERSION}
to publish docs toapi/${VERSION}
- Run
make -C docs dist-redirect_latest CRYSTAL_VERSION=${VERSION}
to apply redirect fromapi/latest
toapi/${VERSION}
- Have
- Push changes to OBS for building linux packages
- Checkout https://github.com/crystal-lang/distribution-scripts and go to
./packages
- Configure build.opensuse.org credentials in environment variables:
export OBS_USER=
export OBS_PASSWORD=
- Run
./obs-release.sh devel:languages:crystal crystal $VERSION
- Uses the docker image
crystallang/osc
to run the CLI client for OBS. - The script creates a branch in you home project, updates the version and pushes it back to OBS.
- You can also run the commands from that file manually and check build locally with
osc build xUbuntu_20.04 x86_64
osc build Fedora_Rawhide x86_64
- Uses the docker image
- Now OBS builds the packages. It’s best to follow the build status in the browser:
open https://build.opensuse.org/project/show/home:$OBS_USER:branches:devel:langauges:crystal/crystal
- Wait for all package build jobs to finish and succeed
- Verify package installation
OBS_PROJECT=home:$OBS_USER:branches:devel:languages:crystal bats test
- When everything is green, create a submit request against the original package (Submit package link in the menu bar on the package in your branch)
- Checkout https://github.com/crystal-lang/distribution-scripts and go to
- Tag
latest
docker images- Versioned docker images have been pushed to dockerhub.
- Now just assign the
latest
tags: $ ./docker/apply-latest-tags.sh {version}
- Publish snap package
- You need to logged in via
$ snapcraft login
- Recent tagged release is published directly to edge channel. The CI logs the snap revision number. Otherwise the .snap file is in the artifacts.
- Check the current status to find the revision of the tagged release otherwise:
$ snapcraft status crystal
$ snapcraft release crystal <revision-number> beta
$ snapcraft release crystal <revision-number> stable
- You need to logged in via
- Submit a PR to update the homebrew formula in https://github.com/Homebrew/homebrew-core/blob/master/Formula/crystal.rb .
- Update the previous and new version (with their respective hashes).
- Try locally
$ brew install --build-from-source <source of formula>
- Create PR (Homebrew/homebrew-core#89149)
Release announcements
- Update crystal-website (crystal-lang/crystal-website#208)
- Post a release notes in the website
- Update information of
latest_release
in https://github.com/crystal-lang/crystal-website/blob/master/_config.yml - Update link to blog in https://github.com/crystal-lang/crystal-website/blob/5905e5644fcb4863976d0ca69856edbd74091370/index.html#L26
- Post announcement in https://forum.crystal-lang.org/c/news/official
- Tweet about the release
- Post in Reddit
- Update https://github.com/crystal-lang/crystal-book/blob/master/crystal-version.txt (https://github.com/crystal-lang/crystal-book/compare/master...straight-shoota:feature/crystal-1.2.2?expand=1)
Post-release
- Update crystal
master
branch to use released version (crystal-lang/crystal#11430)- Edit PREVIOUS_CRYSTAL_BASE_URL in
.circleci/config.yml
- Edit DOCKER_TEST_PREFIX in
bin/ci
- Edit
prepare_build
on_osx download package and folder - Edit
.github/workflows/win.yml
to point to docker image - Edit
shell.nix
latestCrystalBinary
usingnix-prefetch-url --unpack <url>
- Edit PREVIOUS_CRYSTAL_BASE_URL in
- Increment VERSION file to the next minor and -dev suffix
- Perform uncomment/todos left in the repo
- Update carc.in / play.crystal-lang.org (by jhass)
Note: This is done after Arch repos are updated
Build fails on latest Crystal nightly image
docker run --rm crystallang/crystal:nightly-alpine crystal eval 'puts "foo"'
Produces:
/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/bin/../lib/crystal/libgc.a(pthread_support.o): in function `GC_thr_init':
pthread_support.c:(.text+0xe39): undefined reference to `gnu_get_libc_version'
/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/bin/../lib/crystal/libgc.a(mach_dep.o): in function `GC_with_callee_saves_pushed':
mach_dep.c:(.text+0x3b): undefined reference to `getcontext'
/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/bin/../lib/crystal/libgc.a(pthread_start.o): in function `GC_inner_start_routine':
pthread_start.c:(.text+0x44): undefined reference to `__pthread_register_cancel'
/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: pthread_start.c:(.text+0x6b): undefined reference to `__pthread_unregister_cancel'
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc "${@}" -o /root/.cache/crystal/crystal-run-eval.tmp -rdynamic -L/usr/bin/../lib/crystal -lpcre -lm -lgc -lpthread -levent -lrt`
First noticed via https://github.com/Blacksmoke16/oq/runs/3875659755?check_suite_focus=true.
`test_ecosystem` should record the last working shards
Right now it just runs on the latest release of shards, but that doesn't guarantee that if something fails it's our fault.
Alpine Docker image can't build with preview_mt, missing GC patch
$ docker run --rm -ti crystallang/crystal:0.33.0-alpine
/ # echo 'puts "hello"' > a.cr
/ # crystal build -Dpreview_mt a.cr
/usr/lib/gcc/x86_64-alpine-linux-musl/9.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: G-C-.o: in function `set_stackbottom':
/usr/share/crystal/src/gc/boehm.cr:245: undefined reference to `GC_set_stackbottom'
/usr/lib/gcc/x86_64-alpine-linux-musl/9.2.0/../../../../x86_64-alpine-linux-musl/bin/ld: G-C-.o: in function `current_thread_stack_bottom':
/usr/share/crystal/src/gc/boehm.cr:236: undefined reference to `GC_get_my_stackbottom'
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc "${@}" -o '/a' -rdynamic -lpcre -lm /usr/lib/libgc.a -lpthread /usr/share/crystal/src/ext/libcrystal.a -levent_pthreads -levent -lrt -L/usr/bin/../lib/crystal/lib -L/usr/lib -L/usr/local/lib`
The alpine version uses libgc 8.0.4 (latest release), but it doesn't include de needed patch.
Upgrade to LLVM 13
The latest LLVM version 13.0.0 has been released yesterday. It includes relevant fixes we've been waiting for.
A regression in LLVM 11 prevented us from upgrading to LLVM 11 or 12 (crystal-lang/crystal#10359). The fix should be available with LLVM 13.
It has been very recently discovered that LLVM 13 should also have a fix for crystal-lang/crystal#11047 (although that's less relevant here because we don't ship Windows builds yet).
A challenge for that is that LLVM 13 is not yet available in aports edge and I'm not sure when it will be. I don't think it would be backported to Alpine 3.12, which we're still stuck with for now (#127, crystal-lang/crystal#10366). We still don't know the reasons for the issues experienced on Alpine > 3.12.
We can either pull in the llvm-13
packages from Alpine edge whenever they are available. Or we need to build LLVM 13 manually.
If we want to link the upcoming Crystal 1.2 builds with LLVM 13, we'll probably have to resort to the latter. We want to ship them next week.
This is obviously more effort. So maybe it's not worth it considering that we can stay with LLVM 10 for now to avoid crystal-lang/crystal#10359.
The most relevant fix in LLVM 13 at this point is really for Windows and we can already use that in CI.
Requires crystal-lang/crystal#11277
Update readme on docker hub
The readme at https://hub.docker.com/r/crystallang/crystal is outdated.
crystal-lang/crystal-dist#14 suggested an automation to update the readme from the Crystal repository. We could to this and integrate it as a step in docker/apply-latest-tags.sh
.
However, I think it would be better to provide a readme specific to the docker repository, which explains the available image types. We might still want to have that automated to always show the latest tags, though.
Update image for crystallang/crystal:1.1.1 (because LE certificate expired) - consider autobuild of all images regularly?
Hi, as you may know, Let's Encrypt Root certificate has expired.
Because of that, ca-certificate package has to be updated.
I've run into an issue, where running docker build in my CI pipeline fails, because of that expired certificate.
dev:~# docker pull crystallang/crystal:1.1.1
1.1.1: Pulling from crystallang/crystal
Digest: sha256:ae8f53cc10b9cfdd786682cc420cce1097a06fc2e0db0ab4647965df64f25951
Status: Image is up to date for crystallang/crystal:1.1.1
docker.io/crystallang/crystal:1.1.1
dev:~# docker run --rm -ti crystallang/crystal:1.1.1 git clone https://my-le-domain.com/test/test.git
Cloning into 'test'...
fatal: unable to access 'https://my-le-domain.com/test/test.git/': server certificate verification failed. CAfile: none CRLfile: none
(If you do this via shards install, you just get:
$ shards install
Resolving dependencies
Fetching my-le-domain.com/test/test.git
Failed to clone my-le-domain.com/test/test.git
with no explanation why... I had to do manual git clone
in CI config to find out the real cause.)
I have to run apt update && apt install ca-certificate for this to work.
Interestingly, curl on the same domain works (don't know why).
Updating ca-certificates package removes one CA certificate and updates none. Maybe it has to do something with that.
Anyway, the current image for current crystal/crystallang:1.1.1 is two months old now.
Are there any plans to update all (at least current, not EOL) versions of crystal images automatically?
Patches in upstream ubuntu:focal container maybe should be propagated downstream to your image... what do you think?
Base ubuntu:focal image doesn't include ca-certificates package, nor git-core package, so you install that yourself. Autobuilding your images could aleviate such problems in the future...
Install llvm-dev in the Docker image
It would be nice to not have to install llvm-8-dev when working on Crystal itself in the docker image.
E.g. eliminate this installation step here
https://github.com/crystal-lang/crystal/runs/597246816#step:4:1
https://github.com/crystal-lang/crystal/actions/runs/81290075/workflow#L12
Only llvm-8 is provided, though:
The -dev package should be a small difference.
Push docker images directly from CI
Right now nightlies images are being pushed already, the difference lies in that the images of releases (apparently) uses OBS packages instead of hard-coded binaries. The intention as we understand it is to be able to deliver updateable images. However, that doesn't seem to be super important: those willing to have such images should simply add the steps to install crystal from a base image.
macOS builds should not depend on clock_gettime
clock_gettime
doesn't exist on macOS versions previous to Sierra (10.12).
Crystal's codebase explicitly tries to avoid it, but the builds done by the CI still depend on it, failing to run on OS X El Capitan (10.11).
Local builds (even in Sierra) don't depend on them, so there must be something odd in the omnibus script or something.
Provide newer Git version in the Docker image
The current version (which I guess is 2.17.1 from Bionic) causes GitHub Actions to not actually expose the Git repo as a repo but just check out the bare files, which can be a hard-to-debug surprise when people rely on the image in GitHub Actions.
The repository will be downloaded using the GitHub REST API
To create a local Git repository instead, add Git 2.18 or higher to the PATH
https://github.com/crystal-lang/crystal/runs/597246816?check_suite_focus=true#step:3:20
I've had to work around this by not using the image and instead installing Crystal with something like snap
Why not use obs https://openbuildservice.org/ distro packaging
The following build targets are supported by OBS:
https://en.opensuse.org/openSUSE:Build_Service_supported_build_targets
Release a `snap` with a `tagged_release`
Currently snap
packages are published, but the release is performed manually. Automate this step for every tagged_release
.
crystal:0.36.1-alpine missing yaml-static
A new Alpine image for 0.36.1 made it's way to docker hub last night. This is unable to statically compile crystal programs that use YAML.
Reduced example: https://gist.github.com/kimburgess/2a740af0f2c8bc5b217a9fa47638df0e.
Cause appears to be bdff883
Build output:
Sending build context to Docker daemon 3.072kB
Step 1/3 : FROM crystallang/crystal@sha256:ee97e1ef8b609cd80a8d504fdf8c5c09f7ac2fc520d360a7b10c8d50aa21f4b2
---> 32474966bd9f
Step 2/3 : COPY . .
---> 7e65d2efd8e7
Step 3/3 : RUN crystal build --static --error-trace yaml.cr
---> Running in cffb373d5aed
/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lyaml (this usually means you need to install the development package for libyaml)
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc "${@}" -o /yaml -rdynamic -static -L/usr/bin/../lib/crystal/lib -lyaml -lpcre -lm -lgc -lpthread /usr/share/crystal/src/ext/libcrystal.a -levent -lrt`
The command '/bin/sh -c crystal build --static --error-trace yaml.cr' returned a non-zero code: 1
Automate scripting of changelog
Right now the changelog is built using a script per PR. Ideally, the changelog shuld be generated with a button, and then we will manually edit it to polish it.
Create aarch64 gnu docker images
One option is to use Arch linux for this.
In the release process, create a docker image to distribute using the latest binaries generated from CI.
Related #123
Codesigning and/or notarizing macOS builds
Not sure if this is on the team's radar or not, but increasingly a nice-to-have, would be for the official macOS builds to receive notarization and codesigning to ensure:
- the build hasn't been modified
- to satisfy the strict enforcement of notarization present in macOS 10.15+.
While it's relatively easy to workaround it (either removing the quarantine xattr
manually, going through System Preferences to allow crystal
and shards
, or downloading it via something that does not set the quarantine xattr
), it's not particularly convenient and it's not immediately clear to a new user that they need to do that (they'll just get the "Move to Trash" or "Cancel" options when running). You can use Homebrew builds and it won't be an issue, but not everyone uses Crystal via Homebrew on macOS
For my own Crystal projects, I've taken to using gon for automating the notarization and codesigning in CI (see example here), which works pretty smoothly and (it or something like it) could probably be integrated into release process. The tricky piece would be credential management for macOS developer certs, though it sounds like CircleCI recommends something called Fastlane Match for this.
Remove stackbottom patch
Proposing feature-thread-stackbottom.patch to https://github.com/ivmai/bdwgc would be a win for everybody:
Crystal core developers won't have to maintain this patchCrystal developers won't have to recompile bdwgc to have this feature, it will be directly available in the OS packagebdwgc users will be able to use it
Edit: the patch is already merged with ivmai/bdwgc@5668de7
Remains to remove the patch when a new release of bdwgc will be out.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.