Comments (5)
I think solving #539 will give us a better sense of the problem space of components, since we'd have to deal with them more explicitly in haskellPackages.mkDerivation
and cabal2nix
in order to solve it—without splitting anything up at that point.
Then, for me, the logical next step would be to work on having haskellPackages.mkDerivation
or rather a mkDerivation-ng
support per-component builds in some way. This would be a relatively easy first step and would give us a better sense of what would need to change in cabal2nix
and hackage2nix
.
One difficulty seems to indeed be how to depend on specific components of another package without breaking splicing, i.e. support for cross-compilation.
The biggest question is: Do we generally want to make a package usable without depending on its test. The general design in nixpkgs is to not do it, I think? I am not aware of any language builder which separates tests into an extra derivation. The danger is very large, that we simply don’t run tests if we can get away with not doing it.
It seems to me that this is a big concern. If test suites can't fail the derivation, they become pointless. It seems to me that we need wrapping derivations
- around each component, that depends on the tests,
- around all components, that depends on the tests and re-instates the package unity (e.g. a
pandoc
wrapping derivation that unitespandoc:lib
andpandoc:exe:*
again).
These wrapping derivations would be fairly trivial to build, ideally just symlinking a bunch of stuff and ensuring tests succeeded.
There'd be an explosion in the number of derivations definitely. Another question is how to go about tests: Should the test component derivation directly run it? Probably, to avoid it having to be executed multiple times…
from cabal2nix.
This sounds like a pretty big idea. I am generally very much in favor of improvements to cabal2nix, but the constraints of living in nixpkgs are tough.
Before you take on such a big undertaking, I wonder what use cases you are envisioning and if this approach actually helps.
from cabal2nix.
That's a really good point!
Currently I'm switching my work CI over from using haskell.nix to the nixpkgs Haskell infra. The main thing I'm missing is component based builds.
While this isn't blocking me from switching, I think it would be good to try to implement this to make things work better (faster builds, failing builds/tests blocking less of the graph).
Since this is quite a big thing, I hope to whittle away at it when I get some time to improve CI.
I'm also keeping in mind that the maintainers have limited time, so I want to approach this in whatever way y'all think is best, including dropping this if y'all think it's too much complexity.
from cabal2nix.
If we can find a good design for the features in question, I think it would be worth the effort. At least from my personal maintainer perspective.
I wonder if it’d make sense to have a call about this, because it is really not clear where we want to go with this.
- The technically hardest part is probably using the outputs of one component as input to another component.
- One question I have: Can we trim inputs for certain component builds in a way so that we can get at least sometimes recompilation avoidance. (Mainly thinking about fixing tests without recompiling the lib.) This is not a must, but would be super cool.
- The biggest question is: Do we generally want to make a package usable without depending on its test. The general design in nixpkgs is to not do it, I think? I am not aware of any language builder which separates tests into an extra derivation. The danger is very large, that we simply don’t run tests if we can get away with not doing it.
On the other hand, making the hot path for building shorter sounds really intriguing.
from cabal2nix.
A call definitely sounds like a good shout.
One question I have: Can we trim inputs for certain component builds in a way so that we can get at least sometimes recompilation avoidance. (Mainly thinking about fixing tests without recompiling the lib.) This is not a must, but would be super cool.
I would definitely like to have this as well!
In terms of the other points. I think you are right that these are going to be tricky. I think it should be doable to ensure that tests get run. One possibility is that the derivation that we make available to users is a package level summary of the components and so includes checks that the test suite and the test suites of dependencies have run.
from cabal2nix.
Related Issues (20)
- Map `extra-libraries: FLAC` to pkgs.flac HOT 4
- latest library doesn't pull in other latest library
- do not encourage use of `nix-env` HOT 1
- Confusing output when using --subpath in conjunction with a source that needs to be unpacked
- Improve tooling for making a new release
- Add --disable-library-profiling option HOT 2
- `posix_spawnp: illegal operation` HOT 1
- Support for GitHub/GitLab PR URLs HOT 1
- Add warning redirecting to nixpkgs manual at the top of every readthedocs page
- Get pkg-config dependencies from `pkg-configPackages` HOT 5
- Don't set mainProgram when it matches pname HOT 1
- Removing `configureCabalFlags` completely HOT 1
- Invalid Nix is generated for a .cabal file that has the `if` package as a dependency HOT 1
- Generate `meta.longDescription` HOT 2
- Generate version alias info to streamline updates HOT 3
- Support single-file Haskell script HOT 3
- Pulling from darcs zip file results in unpacking failure. HOT 1
- Status of the repository? HOT 2
- Adding pkgconfig-depends: hdf5 pulls in Haskell's hdf5, not system hdf5 HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cabal2nix.