Build and release process about qsharp HOT 17 OPEN

OK. Here's the proposal.

• For our release artifacts (which initially are a VS Code extension and some pip packages), we'll ship to 3 separate channels. A <name>-nightly, a <name>-preview, and a <name> channel, which we'll refer to as the nightly, preview, and stable channels, respectively.
• Nightly will get published every night from the 'main' branch. The expectation is that engineering will be using this branch for daily work to catch regressions early, or partners/customers can use it after a feature is merged to verify early.
• As we start to stabilize an upcoming release, we'll branch for it and push releases to the preview channel for testing and early adopters.
• When a release is ready, we'll push to the stable channel, which is where the vast majority of customers will pick it up.

We'll assume a monthly release schedule, aiming for roughly the middle of each month (this avoids most holidays which are typically start or end of month). To stabilize once features are done, we'll create a branch named release/YYMM for the target release month, e.g. for September 2023 release we'll create release/2309. Only fixes should go in to a release branch, and be ported to 'main'.

We'll avoid using pre-release semantics in versions (e.g. 1.10.1-beta). If you install from a pre-release channel, that should be obvious and intended. We'll simply version with the major version being 1 (only incrementing when we make changes worthy of a major version bump - we can argue about semver later :-) ), the minor version being the YYMMDD date, and the patch being at 0 unless we need to push to a channel repeatedly in a day (which I expect would be only if fixes are needed).

Having a version such as 1.230915.0 makes it easy to see when the release came out, and it naturally increments over time. If we make fixes to that channel that ships a week later, it'll just be version 1.230922.0. We won't support patching old (minor) versions on a channel. Pick up the latest release if you want the latest fixes.

For releases pushed to the preview and stable channels, we'll tag the release branch with a tag that matches version number for easy correlation when investigating issues. We'll not worry about tagging for nightly releases, as that seems like more tags/noise than value.

from qsharp.

Note: I just had a request from an "air gapped" customer how to download and install the current QDK. This is tricky with the VS Code extension automatically trying to fetch stuff, and various dependency graphs on some packages. It may be something we want to consider when designing the new release artifacts and install scenarios.

from qsharp.

Also, it might go without saying, but just in case, I don’t think it’s worth forcing a release every month. It should be ok to skip a month if the ongoing work is not ready or there isn’t accumulated value to users in a given month.

from qsharp.

OK, here's the latest proposal, with some background as to why.

Constraints

We have to satisfy 3 different and incompatible versioning schemes.

• Python, which follows PEP440 (https://peps.python.org/pep-0440/)
  • Allows for major.minor.patch, but has different version string formats and semantics for pre-release or dev builds.
• Npm, which follows SemVer (https://semver.org/)
  • Allows for major.minor.patch, but has very flexible pre-release semantics with different syntax to PEP440.
  • Also supports tags as a common way to install, e.g., npm install foo by default installs foo@latest (https://docs.npmjs.com/adding-dist-tags-to-packages)
• VSCode, which is a subset of semver
  • Allows for major.minor.patch, but has no notion of pre-release components in the version
  • It does allow for a version to be marked as a 'pre-release' via a flag when publishing, which a user can then explicitly opt-in to installing.

Goals

We'd like to be able to publish:

• Stable versions that are the default when installing.
• The ability for users to opt-in to 'pre-release' versions and get RC builds.
• The ability to install 'dev' builds for early testing & validation (e.g. nightly builds from main).

To satisfy this for VS Code, we will publish two separate extensions

• The 'stable' extension, which is what users will install by default. We will also push '--pre-release' marked RC builds to this extension, so users can opt in to getting RC builds for upcoming releases.
• The 'dev' extension, which is published to regularly from main, or when we have a specific build we want to test in the marketplace. This would be considered (and documented in the readme as) 'unstable' and not for general usage.

For the VS Code 'dev' channel, we may want to push multiple releases a day, so just using the date doesn't really work (e.g. 1.0.20230609). (PEP440 also doesn't allow for multiple parts to the pre-release component as semver does)

For the VS Code 'stable' channel, likewise if we have both RC and stable builds on it, they'll need to use a common incrementing version, e.g. published builds of 'rc', 'rc', 'stable' builds might be versioned '1.0.0', '1.0.1', '1.0.2'.

The dev, rc, and stable builds can be merged into the same package for npm and pypi by using the pre-release markers, e.g. for pip: 1.0.0.dev, 1.0.1.dev, 1.0.2.dev, 1.0.0rc, 1.0.1rc, 1.0.2. And for npm: 1.0.0-dev, 1.0.1-dev, 1.0.2-dev, 1.0.0-rc, 1.0.1rc, 1.0.2

Proposal

A simple and intuitive option is just to have a constantly incrementing 'patch' for any published build (dev, rc, or stable) for a major.minor version, e.g. 1.0.0-dev, 1.0.1-dev, ..., 1.0.25-dev, 1.0.26-rc, 1.0.27-dev, 1.0.28-rc, 1.0.29, 1.1.0-dev, etc.

Under this proposal:

• We will regularly (approx. monthly) ship a new major.minor version, e.g. 1.0, 1.1, 1.2, 2.0, ...
• Every published build for that version will increment the patch number, even if a dev build or an rc build, e.g. 1.0.0-dev, 1.0.1-dev, 1.0.2-dev,... 1.0.22-dev, 1.0.23-rc, 1.0.24-rc, 1.0.25
• We will push dev builds to a distinct 'development' VS Code extension for dogfooding and early feature testing
• RC and stable builds will go to the 'official' VS Code extension, with 'rc' builds being marked '--pre-release' (so you need to opt in to pre-release builds to get them).
• All builds will be pushed to the Python 'qsharp-lang' and 'qsharp-jupyterlab' package using 'developmental' or 'pre-release' markers as appropriate per PER 440, e.g. 1.0.0.dev, 1.0.24rc, 1.0.25
• All builds will be pushed to the npm 'qsharp-lang' package using pre-release components per semver, e.g. 1.0.0-dev, 1.0.1-dev, 1.0.24-rc, 1.0.25
• The most recent dev, rc, or stable builds published to npm will also be tagged as 'dev', 'rc', or 'latest', respectively.

Side-note: As we use an ADO DevOps pipeline to publish builds, it does have support for incrementing a counter on each run with a common prefix (e.g. "1.1"), so this could be used to easily implement the patch version increment per build. See https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops#counter

from qsharp.

Something simple for branching to start with. Basically all work happens in feature branches that get merged into main once the feature is complete. "main" should always be in a "complete' state, never with features half working or broken. (Long running feature development may be in 'main' behind an off-by-default flag until ready to ship).

When we want to release we branch main with "release/yymm" where yymm is year and month (assuming we continue to ship on a monthly basis, this seems like an intuitive versioning scheme). When the branch is ready to ship - be it initial release for that milestone or a fix release, we tag the commit that ships with the release number. (Fixes should be cherry-picked into main also if needed).

It'll look something like:

---
title: Git branching and release
---
gitGraph
   commit
   commit
   branch "release/2303"
   checkout "release/2303"
   commit
   commit tag: "2303.0.0"
   commit
   commit tag: "2303.0.1"
   checkout main
   commit
   commit
   branch "release/2304"
   checkout "release/2304"
   commit
   commit tag: "2304.0.0"
   checkout main
   commit

from qsharp.

Note on the above: If we wanted to stick with "semver" versioning (https://semver.org/), which we probably do, major number could be reserved for "breaking changes", i.e. only bumped when its a release you shouldn't auto-update to. Minor version could be YYMM (which will increment with each release), and patch for fixes on that. Thus assuming the April 2023 release has some breaking changes and then needed a fix release, versions would be 1.2302.0 -> 1.2303.0-> 2.2304.0 -> 2.2304.1 etc.

from qsharp.

Cargo https://doc.rust-lang.org/cargo/reference/resolver.html#pre-releases Python https://peps.python.org/pep-0440/#pre-release-separators and npm all support the notion of "prerelease" versions or tags (latest, beta, nightly,...). We should come up with a strategy for how we also publish and enable all users to easily install from these "channels".

from qsharp.

I definitely agree with semver and like the approach of using yymm for minor version. I think we should keep the major version at 0 for a while longer until we are confident enough to call the language a 1.0 release, and then yes, major version updates are only for breaking changes.

One thing to consider: do we want CI builds to somehow increment a build number in the version? I would say no, because cargo versioning is not well suited to build injected versions. But we need to think through the ergonomics of every CI or local build having the same vNext version until release. Plenty of projects do that, it is just not what everyone is accustomed to.

from qsharp.

Coming from TypeScript I feel obligated to represent the point of view that semver can be somewhat meaningless for languages/compilers. How do we define breaking changes to the API? How do we define the "API" for a language? If a breaking change is defined as "code that used to compile clean continues to compile clean", this is an extremely difficult bar to meet, and even Rust itself does not seem to stick to it (reviewing the Compatibility Notes for the latest minor version, a lot of the items sound like breaking changes to me). It doesn't seem like they have any pretense of following semver anyway, so that might be a good lead to follow.

I'm also guessing (I may be wrong?) that the QDK that we build out of this repo will be a collection of many packages & tools, not just the compiler - and each will have its own use case and definition of "breaking change". Would we version each one independently, or share a version and make sure we bump major when any tool introduces a breaking change? That may decrease the usefulness of the version number.

Lastly, Semver's strict definition of "minor" is also incompatible with a yymm scheme, since each monthly release may not be to "add functionality".

Pedantry aside, I do agree x.yymm.y is a good scheme. As long as we carefully document compat issues each release, and occasionally make subjective calls about what constitutes a big enough change for a major version bump, I think we're good?

do we want CI builds to somehow increment a build number in the version?

Is there some kind of internal registry we'd be publishing nightlies to? If we don't bump, wouldn't we get collisions in the version number? If I'm, say, installing nightlies for testing, and I find a bug, how do I quickly track down which build it came from? Not having unique version strings for each build feels very odd to me, but if this is already a well-established convention elsewhere, then I'm sure people have found ways to work with it - I can get over my distaste :)

At the very least, embedding the git commit in the version metadata seems sensible.

from qsharp.

At the very least, embedding the git commit in the version metadata seems sensible.

That's a great idea... I wonder if there is a reasonable way to have that commit included in the output from --version in the exe and/or library metadata. I'll look around for options or examples.

from qsharp.

Version metadata (i.e. the part after the + in x.y.z+abc) is defined in Semver and is commonly used to embed git commit info. Cargo does seem to have some awareness of it since it comes up in documentation.

from qsharp.

We'll avoid using pre-release semantics in versions (e.g. 1.10.1-beta). If you install from a pre-release channel, that should be obvious and intended.

From context above, I understand "channel" to mean an entirely different package. Is there any precedence for this for pip packages? This seems like swimming against the current of established conventions, and I'm not sure I see a compelling enough reason to do it. As for some drawbacks - a consumer would be able to install the preview/nightly/stable packages side by side, which may create confusion. If a consumer wants to test their code against the upcoming preview, that's a code change on their part (rather than just a version bump). But minor drawbacks aside, why not follow convention?

from qsharp.

Is there any precedence for this for pip packages?

I'm not super familiar with that many Python packages, so just picked a few I know of at random and see the below examples:

• https://pypi.org/project/torchx-nightly/
• https://pypi.org/project/keras-nightly/
• https://pypi.org/project/tf-nightly/#description

That said, happy to discuss alternate proposals. What do you have in mind?

from qsharp.

If a consumer wants to test their code against the upcoming preview, that's a code change on their part (rather than just a version bump)

It's a change to 'pip install', but there code could remain the same, no? I believe any of the 3 pip packages could install the 'qsharp' module that the code would import from.

from qsharp.

just picked a few I know of at random and see the below examples:

The fact that there's precedence and an example we can follow makes me feel much better about this, thanks!

What do you have in mind?

Oh what I had in mind was just the semver convention that I'm used to from other packages (e.g. most npm packages) - having only one package name, and using prerelease version tags for nightly and preview. This seems intuitive to me - what's the downside?

I believe any of the 3 pip packages could install the 'qsharp' module that the code would import from.

I believe you're correct. I was mistakenly assuming they'd declare different module names, I guess. In that case, I don't know what happens if the user accidentally installs both qsharp-preview and qsharp side by side, but it's probably not catastrophic.

from qsharp.

I think this makes sense. While the date format is easy to read, it is not portable across all these requirements. We should also ensure that commit id is easily gettable from the published bits as an alternative way to identify a build, which already works for the playground but I don't think has been plumbed through the other packages yet.

from qsharp.

@billti is this still a relevant issue, given that we have shipped/named v1.0.0? Perhaps we can canonicalize this into a document and close the issue?

from qsharp.