Does cabal update force recompilation of everything?

#15

https://cabal.readthedocs.io/en/stable/nix-local-build.html#how-can-i-have-a-reproducible-set-of-versions-for-my-dependencies

3 Likes
#16

Unfortunately, I think what you would like is optimizing for you, rather than your users. I think I would expect users to be, in general, more up to date. If Cabal prioritises a plan that suits you by using old dependencies, I think you risk the scenario where the bounds are subtly wrong, but it works for you.

#17

Alright, so then the last test before making a new release would be to then update dependencies, ideally on a separate copy of the project. Changes in dependencies can be dealt with at that time, rather than being a continual distraction from regular developmental work.

#18

I think it’s really just a preference. I personally would always prefer course-correcting as soon as possible, rather than right at the end.

#19

That might work if the project only has a small number of dependencies. But it just doesn’t scale up - lots of dependencies means more “course-correcting” (boring!) and less developing (interesting, usually the rasion d’être for being involved in the project!)

1 Like
#20

This is the reason I don’t run cabal update so often. You have to wait for both cabal and hls to fully build your project after it… It makes developing not fun… which is the main reason I programm in Haskell. Of course, cabal should not aim for fun but for stability and reproducibility, so I guess the correct behaviour is the current.

Nevertheless, I could imagine cabal build --develop to be a “just take the plan for whatever is downloaded or cached.” But now, we are approaching again to this other thread

1 Like
#21

I think what you would like is optimizing for you, rather than your users.

Oh, I don’t think it should be the only possible behaviour! I think it should one of the possibilities supported by configuration.

2 Likes
#22

While I agree this is annoying, Cabal and HLS should re-use the same dependencies. If you’re seeing everything get built twice then there’s a serious bug somewhere.

#23

Oh that’s true. hls should re-use downloaded dependencies… Maybe I had vscode opened at the same time I run cabal build in a separated terminal causing some weird behaviour.

Now, given that hls uses its own “cabal working directory” (under ~/.cache/hie-bios in ubuntu), Is it completely free for hls to build the project after you run cabal build on external terminal? I think it has to do some work still.

#24

It will till build the project itself (as opposed to dependencies) separately.

Admittedly I’m not totally sure what happens in this scenario. I’m pretty sure Cabal is smart enough that there are no race conditions, but I’m not sure whether some work might be duplicated.

2 Likes
#25

I’m writing an announcement post for tooling that does this. Updo can generate Stack and Cabal projects that work with stackage package sets. If you have a project that needs this, I could look at adding yours to the examples of Updo conversions.

4 Likes
#26

My POV is that cabal-install does a lot good things which are often desiderable but it must give better control to the user.

There are ways to bend it to do what you want but without understanding (or often distinguishing) the use cases, things end up becoming over complicated or worse broken.

E.g. the freeze file these days includes the index-state so with a freeze file you’d been surprise to see cabal still picking the old vty (even after removing the constraint on vty)

Lots of good stuff in there (on a cursory look). I am happy to see work in this space (composable project configuration).

5 Likes
#27

The post is up, Announcing Updo.

1 Like
#28

Sorry to bring this thread to live again but… How the **** does cabal work? I am working in some project with two branches. Between both, there is bearly no changes in the cabal file. Something like this:

master
|
HEAD
    \     feature branch
     |       
     commit-1
     |
     commit-2; adds some modules to .cabal but not dependencies
     |
     HEAD

I would expect that if I am in feature branch and build the project, then If I change to master the only thing that needs to be build is the application itself, but not the dependencies… I am still waiting for many dependencies to be downloaded (just a few of them) and build (many of them).

So how does cabal internally works? is there any sort of whole-programm-compilation? Is there any way to avoid building dependencies which have been build, literally, one hour ago?

Debug why Cabal is choosing a version
#29

Cabal caches all Hackage dependencies, so it will only rebuild things if it resolves your dependencies to different versions. But cabal is stateful so that could happen from time to time, for example if you run cabal update (and perhaps changing the cabal file triggers the dependency resolution again). You can make sure cabal always resolves your dependencies to the same versions by using cabal freeze.

#30

Don’t know… I did something equivalent to:

git pull --all
git checkout feature-branch
cabal update
cabal build all # took 40 min. It's ok because of hackage update

# make some changes. Go to work, and come back three hours later

cabal run            # run ok building only last changes within a few seconds
git checkout master
cabal run            # I'd expect this to just compile the difference between branches
                     # but instead It took 36 min to rebuild the whole thing.

to me it seems like cabal (or maybe ghc) are always rebuilding more than necessary, but I guess there are all sort of complicated corner cases… Maybe I just ranted a little bit.

#31

  • The reason my local Hackage index was so old is precisely to avoid this situation which happened already a few months ago. Is there any cabal build --disk-space-isn't-that-f******-cheap flag?

  • Sorry to bring this thread to live again but… How the **** does cabal work?

…the solver-based approach Cabal uses - does any other programming language’s package manager use it, or something similar to it?

#32

I think Rust’s Cargo also uses a constraint solver to determine which package (crate) versions are used. And people seem to love admire* Cargo (e.g. see the stackoverflow survey).

* the proportion of users that have used Cargo in the past year and want to continue using it

#33

If so, then what is Cargo doing right (and Cabal apparently doing wrong)?

#34

iirc, Cargo’s default workflow is to use a lock file. Whereas the use of a cabal freeze file isn’t as common partially because it’s an extra command you have to use. If OP had used a freeze file, this issue wouldn’t have happened.

Which makes sense. A lock file is how you minimize rebuilds when adding/bumping deps. The “downside” is you have to manage locked versions explicitly.

EDIT: Apparently I already commented in this thread about this very thing back in November :laughing: Does cabal update force recompilation of everything? - #2 by Ambrose

1 Like