Does cabal update force recompilation of everything?

#21

I think what you would like is optimizing for you, rather than your users.

Oh, I don’t think it should be the only possible behaviour! I think it should one of the possibilities supported by configuration.

2 Likes
#22

While I agree this is annoying, Cabal and HLS should re-use the same dependencies. If you’re seeing everything get built twice then there’s a serious bug somewhere.

#23

Oh that’s true. hls should re-use downloaded dependencies… Maybe I had vscode opened at the same time I run cabal build in a separated terminal causing some weird behaviour.

Now, given that hls uses its own “cabal working directory” (under ~/.cache/hie-bios in ubuntu), Is it completely free for hls to build the project after you run cabal build on external terminal? I think it has to do some work still.

#24

It will till build the project itself (as opposed to dependencies) separately.

Admittedly I’m not totally sure what happens in this scenario. I’m pretty sure Cabal is smart enough that there are no race conditions, but I’m not sure whether some work might be duplicated.

2 Likes
#25

I’m writing an announcement post for tooling that does this. Updo can generate Stack and Cabal projects that work with stackage package sets. If you have a project that needs this, I could look at adding yours to the examples of Updo conversions.

4 Likes
#26

My POV is that cabal-install does a lot good things which are often desiderable but it must give better control to the user.

There are ways to bend it to do what you want but without understanding (or often distinguishing) the use cases, things end up becoming over complicated or worse broken.

E.g. the freeze file these days includes the index-state so with a freeze file you’d been surprise to see cabal still picking the old vty (even after removing the constraint on vty)

Lots of good stuff in there (on a cursory look). I am happy to see work in this space (composable project configuration).

5 Likes
#27

The post is up, Announcing Updo.

1 Like
#28

Sorry to bring this thread to live again but… How the **** does cabal work? I am working in some project with two branches. Between both, there is bearly no changes in the cabal file. Something like this:

master
|
HEAD
    \     feature branch
     |       
     commit-1
     |
     commit-2; adds some modules to .cabal but not dependencies
     |
     HEAD

I would expect that if I am in feature branch and build the project, then If I change to master the only thing that needs to be build is the application itself, but not the dependencies… I am still waiting for many dependencies to be downloaded (just a few of them) and build (many of them).

So how does cabal internally works? is there any sort of whole-programm-compilation? Is there any way to avoid building dependencies which have been build, literally, one hour ago?

Debug why Cabal is choosing a version
#29

Cabal caches all Hackage dependencies, so it will only rebuild things if it resolves your dependencies to different versions. But cabal is stateful so that could happen from time to time, for example if you run cabal update (and perhaps changing the cabal file triggers the dependency resolution again). You can make sure cabal always resolves your dependencies to the same versions by using cabal freeze.

#30

Don’t know… I did something equivalent to:

git pull --all
git checkout feature-branch
cabal update
cabal build all # took 40 min. It's ok because of hackage update

# make some changes. Go to work, and come back three hours later

cabal run            # run ok building only last changes within a few seconds
git checkout master
cabal run            # I'd expect this to just compile the difference between branches
                     # but instead It took 36 min to rebuild the whole thing.

to me it seems like cabal (or maybe ghc) are always rebuilding more than necessary, but I guess there are all sort of complicated corner cases… Maybe I just ranted a little bit.

#31

  • The reason my local Hackage index was so old is precisely to avoid this situation which happened already a few months ago. Is there any cabal build --disk-space-isn't-that-f******-cheap flag?

  • Sorry to bring this thread to live again but… How the **** does cabal work?

…the solver-based approach Cabal uses - does any other programming language’s package manager use it, or something similar to it?

#32

I think Rust’s Cargo also uses a constraint solver to determine which package (crate) versions are used. And people seem to love admire* Cargo (e.g. see the stackoverflow survey).

* the proportion of users that have used Cargo in the past year and want to continue using it

#33

If so, then what is Cargo doing right (and Cabal apparently doing wrong)?

#34

iirc, Cargo’s default workflow is to use a lock file. Whereas the use of a cabal freeze file isn’t as common partially because it’s an extra command you have to use. If OP had used a freeze file, this issue wouldn’t have happened.

Which makes sense. A lock file is how you minimize rebuilds when adding/bumping deps. The “downside” is you have to manage locked versions explicitly.

EDIT: Apparently I already commented in this thread about this very thing back in November :laughing: Does cabal update force recompilation of everything? - #2 by Ambrose

1 Like
#35

(…your old 2023-Nov comment only suggests using a freeze file - no comparison to Cargo there.)

Assuming your recollection is accurate…then solving this particular problem (unexpected/unwanted rebuilds) is a simple matter of switching Cabal to use freeze files by default, with something like a refresh command (similar to apt update or apk update) to only update the dependency DB (I’ve never used it myself, but presumably when one runs emerge --sync on Gentoo, it doesn’t cause the mass rebuilding of installed packages on the OS).

But it surely couldn’t be that simple - there must be some other major points of difference between Cargo and Cabal…

#36

I guess personal preferences. On pretty much every Rust video there are jokes about cargo being very slow and downloading way too many dependencies (which looks exactly like my original complaint with cabal)

I just wish there would be a command like cabal build --develop which picks the plan which minimizes rebuilds/downloads. This is a minimal use case (and apparently solved by using a freeze file), at this point many user would prefer “pick latest” and others “pick fastest”, but again: it is a matter of personal preference.

I will try using freeze files and see how good/bad they are. Probably I’ll still have problems between projects. Example: project A and project B uses same dependency set, but cabal picks different versions on each project for whatever reason.

I think there is no point to keep the discusion open without proposing any real solution. So I am asking to close it if that’s ok. @f-a may you close it?

1 Like
Specify freeze file to hls
#37