Comments (2)
Hi Niklas,
It appears that your second option is really the same as the first one just implemented further upstream. I suspect protoc maintainers will be hesitant to make such a change as their primary use case is building into a clean tree (that's the way bazel works). So, realistically doing an external workaround is the easier way to go about it.
For my edification, which scenario do you have in mind, ghc --make? Or when running it from stack/cabal? In my dream world we would be pushing fixes where they belong: the tools that rely on mtime shouldn't really do that.
from proto-lens.
Hi Greg,
So, realistically doing an external workaround is the easier way to go about it.
OK, I guessed right about that then. I have implemented the first approach in PR #177.
For my edification, which scenario do you have in mind, ghc --make? Or when running it from stack/cabal?
Both. Since cabal-the-library uses ghc --make
, and both cabal-install and stack use cabal-the-library, in all possible scenarios does ghc --make
get invoked. As a result, in my project where I use proto-lens-protoc
, the .hs modules that are generated by a proto are touched on every build, and thus recompiled on every incremental build (that is, if you change a bit of code and run e.g. stack build
again).
This problem gets intensified by "the [TH]
check problem", as I call it: Right now in GHC, if a module A
gets changed (in this case our proto-generated module), and a module B
improts it and uses TemplateHaskell splices, then B
needs to get recompiled (with [TH]
shown as the reason in ghc --make
), because values in A
might have changed that might get reified in TH splices in B
. Further, if a module C
imports B
and uses TH, then C
also gets recompiled for the same reason, and so on. In projects where TH is used a lot (e.g. for logging in every module), this means a single file change (even due to touch
) can result in the recompilation of large parts of the hole projects. A mini example of that can be seen here.
(I'm working on making this problem less severe but it will take a lot of effort to be fully fixed, and until then not touching files unnecessarily is a very effective cure.)
In my dream world we would be pushing fixes where they belong: the tools that rely on mtime shouldn't really do that.
I agree in part: Yes, any build system should check for content changes instead of relying only on mtime changes, and GHC should be improved to do so.
However, an mtime check before a contents check (as shake
does) is a very effective optimisation: In the good (file unchanged) case, it reduces the cost, from reading the file to a single stat()
, making no-op/incremental builds O(number of files in the project) vs O(size of source code in the project), and touching cached FS meta information only.
As a result, I think that wherever possible, file generators should not update the mtime of files when they can, and that pushing this decision to happen as early as possible is the most effective way to get fast incremental builds.
Let me know if this makes sense!
from proto-lens.
Related Issues (20)
- Too strict bounds on profunctors on hackage HOT 2
- Release for Stackage LTS-18.0 HOT 5
- Support GHC 9
- `readMessage` from `TextFormat` chokes on unicode in strings
- Re-add blackgnezdo HOT 2
- Increment upper bounds for some dependencies HOT 2
- New Hackage release HOT 3
- Support lazy construction for byte fields HOT 2
- Not compatible with mtl-2.3 and transformers-0.6 HOT 3
- google/protobuf/struct.proto not included in protobuf types HOT 2
- Show instance not working for first/default enum values
- Show instance not working for first/default enum values
- Support GHC 9.4 HOT 10
- Compilation hangs after some `default-extensions` added. HOT 1
- Replace deprecated `set-output` command with environment file
- Support deepseq-1.5 and bytestring-0.12
- Bump upper bounds for base, containers, ghc-prim, primitive
- Any way to generate protobuf schema from haskell types? HOT 2
- Please issue new Hackage release of proto-lens-protobuf-types HOT 2
- Support "Protobuf Editions"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from proto-lens.