Augmented Compatibility Annotation Scheme (version 2; WIP)
The Problem
A significant obstacle to being able to automate relaxation of PVP-style upper bounds is that major version increments conflate/overlay two different signals into a single one.
A related problem is that while version numbers are almost always properly assigned according to the PVP rules, mistakes can still happen.
Signal conflation in major version increment signalling
There are two categories of breaking changes that get signaled by the same mechanism:
- Breaking changes that can be detected/inferred statically by the exposed API type signature
- Breaking changes that can not be detected/inferred statically by the exposed API type signature
Major version increments that fall into the first category have the important property:
- Given a package which is known to be correct,
when updating its dependee package to the subsequent new major version (of cat. 1), and it still compiles (in pedantic mode), its preexisting correctness is preserved.
IOW, in this case, we can infer statically and automatically that an upper bound relaxation to the next major version is indeed safe.
These harmless first category major version increments are likely the most frequent ones occurring on Hackage.
For the less frequent second category, however, we don't have any such properties. Even unittests won't give enough confidence (except for trivial enough cases where it's possible to encode all semantic properties a package relies on; but it's unrealistic to assume that authors will invest the required effort and discipline).
The ability to signal this kind of second category is however extremely important for our ecosystem, as otherwise we'd be unable to fix or improve our APIs at the semantic level, and would have to introduce new names each time we want to change or introduce new semantics for existing entities. This tradeoff is obviously undesirable.
In other words, the second category of major version increments is the one which cannot be handled automatically and therefore needs to be detected to avoid allowing incorrect package-build-plans which would be hard to detect and even harder to fix, especially if a massive automation had whitelisted them already.
Misattribution of version increment signalling
As aforementioned, the other problem relevant to automating management of version constraints of dependency are the (infrequently) occurring mistakes in version assignment. We should improve the tooling to help with reducing the risk of mistakes during version assignment; but there will always remain an element of risk.
Scenarios that can occur (listed in order of increasing danger):
- false positive: Signalling an incompatible change when there isn't one
- e.g.
text-1.0 signalled breaking compat even though there wasn't.
- Other examples include packages which follow SemVer rules for >=1.0.0 (SemVer minor-ver-increments are seen as major-ver-increments in PVP)
- false negative: Failing to signal a backward incompatible change
- cat. 1: e.g.
Cabal-1.24.1 accidentally introduced a backward incompatible typesignature change (i.e. cat. 1) (reverted in Cabal-1.24.2; unfortunately, Stackage LTS had already picked up the broken 1.24.1 release)
- cat. 2: e.g.
directory-1.2.3 inadvertently introduced a semantic change (i.e. cat. 2) which went undetected for long time as it shipped with GHC 8.0.1 (and got reverted in directory-1.3; to be shipped with GHC 8.0.2)
- Other examples include packages which follow SemVer (non-)rules for 0.y.z versions ("Anything may change at any time. The public API should not be considered stable.")
A different scenario (and actually not a case of misattribution) is when a package does a backward incompatible change, and signals that properly, but then reverts back to the original semantics (while again signalling this properly). This happened for aeson-0.9.* -> aeson-0.10.* -> aeson-0.11.*. So aeson-0.9 and aeson-0.11 are semantically compatible to each other.
The Solution
In order to address the problems stated, we need to
- detect whether a major version increment is purely of the first category,
- annotate the compatibility relation to non-directly-preceding major versions, and
- be able to override PVP compatibility semantics retroactively to correct mistakes in the meta-data.
To this end, we can simply introduce a new field for .cabal files which can be interpreted by the build-bot infrastructure:
x-compatibility: <version-lhs> <compat-relation> <version-rhs>
By simply express this meta-data inside .cabal files, we can leverage the existing Hackage .cabal revision mechanism to add missing annotations retroactively, or to fix incorrectly annotated major versions.
Moreover, we make the standard case simple (i.e. no annotations are needed when the PVP has been applied correctly), and we make it possible to handle exceptional cases (annotations can be added in case of mistakes or deliberate deviations from the PVP).
The valid <compat-relation>s are
compatible-withincompatible-withsemantically-incompatible-with
version-rhs needs to refer to a single version (e.g. 1.2.3) or a version-constraint expression (e.g. >= 1.2 && < 1.3 or == 1.2.*); it must always refer to versions prior to the current package version.
version-lhs needs to be the same version as specified in the version: field. This is a safe-guard against leaving a stale compatibility annotation in your .cabal file, as each time you update the package version:, you'll get an error if you forgot to remove (or update) any preexisting x-compatibility: annotation.
By default, the following implicit x-compatibility annotations are inferred from the version: field (NB: obviously, for single-part version numbers, i.e. only A, a slightly different implicit default is constructed):
version: A.B.C.Ds
x-compatibility: A.B.C.Ds semantically-incompatible-with <A.B
x-compatibility: A.B.C.Ds incompatible-with >=A.B && <A.B.C
TODO define shadowing/combining/overriding rules in case of conflicting annotations
TODO reconsider whether we can reduce the power of <version-rhs> by allowing only single versions
TODO There doesn't seem to be any use-case for compatible-with; after all, the only reason we need to differentiate between major/minor version incs is for the purpose of efficiently inferring PVP-style upper bounds; but from the POV of empiric testing we only need to differentiate between semantic/non-semantic changes
Examples
directory
Annotations are only needed in directory-1.2.3,
name: directory
version: 1.2.3
-- retroactively revised via .cabal edit
x-compatibility: 1.2.3 semantically-incompatible-with <1.2.3
as well as directory-1.3.0 (releases inbetween don't need any annotation, as the property is transitive)
name: directory
version: 1.3.0
-- annotation included in 1.3.0 release (assuming `x-compatibility` was already known then)
x-compatibility: 1.3.0 incompatible-with >=1.2 && <1.2.3
aeson
Under the assumption that aeson-0.9 and aeson-0.11 have only a cat.1 incompatibility:
name: aeson
version: 0.11.0.0
x-compatibility: 0.11.0.0 incompatible-with ==0.9.*
time
We just need to reduce the severity of the major version increment that occured with 1.0.0 which was done mostly for marketing text as mature/stable, than for actual breaking changes.
name: text
version: 1.0.0
x-compatibility: 1.0.0 incompatible-with ==0.12.*
Cabal
The breaking change in 1.24.1 was a cat.1 one; we only need two annotations, one in
name: Cabal
version: 1.24.1.0
-- retroactively revised via .cabal edit
x-compatibility: 1.24.1.0 incompatible-with ==1.24.0.*
and another one in
name: Cabal
version: 1.24.2.0
x-compatibility: 1.24.2.0 incompatible-with ==1.24.1.*
Augmented Compatibility Annotation Scheme (version 1; superseeded)
The Problem
A significant obstacle to being able to automate relaxation of PVP-style upper bounds is that major version increments conflate/overlay two different signals into a single one.
There are two categories of breaking changes that get signaled by the same mechanism:
- Breaking changes that can be detected/inferred statically by the exposed API type signature
- Breaking changes that can not be detected/inferred statically by the exposed API type signature
Major version increments that fall into the first category have the important property:
- Given a package which is known to be correct,
when updating its dependee package to the subsequent new major version (of cat. 1), and it still compiles (in pedantic mode), its preexisting correctness is preserved.
IOW, in this case, we can infer statically and automatically that an upper bound relaxation to the next major version is indeed safe.
These harmless first category major version increments are likely the most frequent ones occuring on Hackage.
For the less frequent second category, however, we don't have any such properties. Even unittests won't give enough confidence (except for trivial enough cases where it's possible to encode all semantic properties a package relies on; but it's unrealistic to assume that authors will invest the required effort and discipline).
The ability to signal this kind of second category is however extremely important for our ecosystem, as otherwise we'd be unable to fix or improve our APIs at the semantic level, and would have to introduce new names each time we want to change or introduce new semantics for existing entities. This tradeoff is obviously undesirable.
In other words, the second category of major version increments is the one which cannot be handled automatically and therefore needs to be detected to avoid allowing incorrect package-build-plans which would be hard to detect and even harder to fix, especially if a massive automation had whitelisted them already.
The Solution
Consequently, we need a way to detect whether the major version increment is purely of the first category.
All we need is literally a single bit of additional information, besides the major version increment.
To this end, we introduce a new convention, by defining a new optional annotation for .cabal files:
x-major-version-kind: <version> <kind>
where currently only the safe token is defined for <kind>.
Without any such annotation recorded for the current major version series we have no knowledge and therefore have to assume pessimistically that the current major version was a result of a category 2 major version increment. This default is also safe-guard, as otherwise we'd risk to err towards incorrectness which is hard to detect, which is what we want to avoid in the first place.
An example would be:
name: foobar
version: 1.2.0
x-major-version-kind: 1.2.0 safe
The version number must match the one declared in the version: field.
This is a safe-guard against leaving a stale x-major-version-kind: annotation in your .cabal file, as each time you update the package version:, you'll get an error if you forgot to remove (or update) any preexisting x-major-version-kind: annotation.
Moreover, since we simply express this meta-data inside .cabal files, we can leverage the existing Hackage .cabal revision mechanism to add missing annotations retroactively, or to fix incorrectly annotated major versions.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent
