GHC 9.4 included a patch by Harry Garrood which changed how source files are tested for modification. Formerly it was based on modification time, now it’s based on the content hash.
For teams who manage their CI builds using Nix with the usual Nixpkgs infrastructure, this change opens a way towards incremental builds.
The idea is to expose the dist folder of each Cabal compilation as another derivation output. Builds could then depend of the dist folder of the previous build to avoid unnecesssary recompilation, potentially speeding up CI for large projects.
One option is to use the normal Haskell nixpkgs infrastructure for building Haskell packages, but modify it slightly to allow producing build products as an output alongside the other outputs, and then to also allow passing in build products from a previous build. This approach does require GHC 9.4 as we’re using GHC’s build system. The idea for this came from Jonas Chevalier at NixCon 2022.
The devil is in the details though. We would need to keep track of which stored dist is the best one to use for a given CI run. We would need to take Git feature branches into account. And remember to clean up old obsolete dists from the Nix cache.
So I was wondering if some team already has something like this working in production. I would love to know about their experiences with it.