Request for comment: `cabal freeze` doesn't produce a lock file

1

Hello all,

I have recently been made aware that the file produced by cabal freeze is not a true lock file.

A lock file is a file containing an exhaustive set of pinned dependency versions (and even sometimes with the hash of the source). The key property here is that a lock file is exhaustive: it contains the pinned version of every direct and indirect dependency.

You might want to use a lock file in situations where every direct and indirect dependency must be audited, and bringing in new dependencies (directly or indirectly) should only be done deliberately.

The cabal freeze command can be used to create a file which looks very much like a lock file, with one caveat: the pinned dependency versions are not exhaustive.

Consider this example: a simple cabal project with a single cabal file, that looks like:

executable myexe:
    build-depends: base, text

If I run cabal freeze, the freeze file might look like this:

constraints: any.base ==4.20.0.0,
             any.text ==2.1.1

If I edit my cabal file to this:

executable myexe:
    build-depends: base, text, containers >=0.6

cabal-install will happily build my package, even though containers was not part of the freeze file. [1].

My question for the community is twofold:

  1. Did you know about this, or did you assume (like me) that such a situation would result in a build error?
  2. Is there a situation where it is desirable for the cabal freeze output to be non-exhaustive, or should we work towards plugging this hole?

  1. Even worse, cabal build --reject-unconstrained-dependencies=all will not reject this build either, since the new dependency (containers) has a constraint. ā†©ļøŽ

7 Likes
2

I remember having produced freeze files from which Iā€™d strip some dependencies like unix in order for contributors on Windows to be able to build and run the software. Similarly, I know that at work we strip out the boot dependencies of GHC from the freeze file in order for the project to be buildable with two versions of the compiler. If there are ways to produce freeze files that can be flexible enough so that this is unnecessary, Iā€™m all ears! :slight_smile:

5 Likes
3

I see no reason why someone would want this to be an error. I.e. I donā€™t think it is a hole that needs to be plugged.

4

I would be quite keen to have lock files as a separate feature of cabal-install.
I think freeze files as they stand are quite far from lock files.

Other things lacking from freeze files that I would expect from lock files:

  • they donā€™t record source hashes for non-Hackage sources
  • they donā€™t deal with Hackage revisions (though maybe this isnā€™t a huge problem)
  • they arenā€™t in an easily machine readable format like JSON
  • they canā€™t account for setup components requiring distinct versions of packages, and instead all the constraints apply to every scope. This might be a bigger problem in the future if we ever split out plugin/TH dependencies into its own scope
12 Likes
5

One can create a custom 00-index.tar.gz (e. g., from a Stackage snapshot) and set it as a local hackage repo in cabal.config. This gives you very solid reproducibility: other versions of packages are simply not there, revisions are not there. You can include platform-dependent packages (e. g., both Win32 and unix) and you can include multiple versions of the same package if both were audited.

4 Likes
6

Did you know about this, or did you assume (like me) that such a situation would result in a build error?

I knew about this, because Iā€™d peeked inside these files and seen that they actually just use cabal.project syntax, which in itself was a bit of a surprise. Like @TeofilC, Iā€™m not convinced this is ideal for supporting other features we should want as part of freezing/locking.

I see no reason why someone would want this to be an error. I.e. I donā€™t think it is a hole that needs to be plugged.

One might reasonably assume that the presence of a freeze file means that all dependency versions are locked down. This would be consistent with how lock files work with other tools, e.g. NPM or Nix flakes. So it would be surprising that one could then switch between containers-0.6 and containers-0.7 without Cabal either complaining or automatically updating cabal.project.freeze.

Anyway, I think this confirms a bigger issue that Iā€™d suspected, which is that there isnā€™t widespread agreement on what exactly freeze files are for.

7 Likes
7

thanks for posting this. iā€™m just getting into haskell after nearly two decades in ruby on rails, go, and most recently typescript/node. i just took lock files for granted. iā€™ve also spent a lot of time arguing that version constraints are (generally) an anti-pattern.

in any case, i tried the freeze file, but then it wouldnā€™t let me change dependency versions. really kind of astonishing given the level of rigor iā€™m seeing within the haskell language.

3 Likes
8

having a lock file is absolutely essential for reproducible builds. itā€™s just dependency management 101 in every other language iā€™ve seen.

2 Likes
9

wow, thanks for the added detail.

10

I see this comment added to a very recent cabal docs MR:

 Please also mention that while cabal.project.freeze 
 restricts specified dependency versions, it does NOT 
 prevent from including future dependencies unless 
 reject-unconstrained-dependencies=all is specified

Perhaps this is the switch you are looking for?

1 Like
11

Unfortunately that flag is not enough to ensure exact dependencies. See this comment: Add a `--lock` flag to `cabal freeze` to promote a freeze file to a lock file by LaurentRDC Ā· Pull Request #10785 Ā· haskell/cabal Ā· GitHub

1 Like
12

One doesnā€™t even need to create a custom .tar.gz ā€“ a directory of packages itself suffices to be a local no-index repository: 3.1. Configuration ā€” Cabal 3.6.0.0 User's Guide

For fully locked down build structures (for compliance reasons) ā€“ the only reason i can imagine people would want a ā€œtrueā€ lockfile ā€“ this seems like a more thorough and complete solution than any lockfile would be, no matter what semantics it is given.

3 Likes
13

The ā€œfreezeā€ files cabal gives absolutely suffice for reproducible builds ā€“ if the package does not change, and if one also locks the index-state, then every build is deterministically the same.

The request is that the package be able to be changed and that change be checked to ensure that it doesnā€™t draw in new dependencies.

But of course, when you change the package, your build will never reproduce the prior package, because the thing you are building has changed!

So clearly there is some other desire at play beyond ā€œreproducible buildsā€?

14

My initial use-case is for compliance reasons indeed, not for reproducible builds.

I like @Kleidukos 's explanation that sometimes you might want to constrain 90%+ of your dependencies, but still allow some flexibility to account for e.g. different platforms. This means that thereā€™s a clear place for freeze files, and that lock files would be a separate concept

1 Like
15

That doesnā€™t work across platforms. Youā€™d need freeze files for every OS and architecture.

Additionally, you have no actual hash protection for build inputs. Not that it matters in practice.

But I agreeā€¦ if this is about compliance, use repos.

16

That doesnā€™t work across platforms. Youā€™d need freeze files for every OS and architecture.

Well yes, but thereā€™s no precise ā€œmeaningā€ to reproducing a build between linux and windows, the builds are definitionally different.

17

if you canā€™t change the freeze file, how is it useful? i donā€™t want to just release one time.

huh? what do you mean reproduce the prior package?

1 Like
18

You create a new freeze file based on your changed package description. I think this is pretty consistent with how everyone else does it.

19

I mean if I have a package repo at state A, then a ā€œreproducible buildā€ to me is something that turns that repo reproducibly (i.e. over multiple runs on multiple machines) into a binary B. If I change that repo to state A1 by changing the code, then the binary produced will be B1 ā€“ different code produces a different binary, thatā€™s the point!

So to ask that we have a ā€œreproducible buildā€ between A and A1 makes no sense to me ā€“ both states should produce different binaries, it would be a bug if one build ā€œreproducedā€ the other.

As MangoIV says above, a freeze file is specific to a fixed state of code. When the code changes, the freeze file should too.

20

I think you missed the issue: state A1 can create binaries B1, B2, B3, etc. from different runs, even with a freeze file in the repo. If the tooling neither warns about nor resolves said discrepancy, thatā€™s not a lock file.

At that point, why even have a freeze file?

3 Likes