Why is cabal install foo --lib frowned upon?

Liamzy · August 27, 2023, 4:32pm

I know it ties up the global namespace by importing the library into GHC, but I typically use it to initialize a package database.

For instance, I just set up a new 8.4.4 installation, I tried cabal install acme-dont, then tried to ghci and :set -package acme-dont.

There is no package environment visible to ghci, so it returns an error message.

So, I then cabal install acme-dont --lib, and thus set up a package environment.

Everything else is then installed without --lib, only being made visible via :set -package acme-dont.

If this isn’t a smelly practice, is there then a space for a nullary package in Hackage? From what I’ve seen, it’s possible to get cabal to build a library with NO modules at all specified.

i.e, hackage-debug-nullary-package would then be specified such that it has no modules at all, and can be used to spawn a package environment via cabal.

Has this been done before?

brandonchinn178 · August 27, 2023, 7:27pm

What’s the actual use case for this? Typically, I would do stack ghci --package PKG to get into a quick shell with access to a dep. I’m sure cabal has similar functionality

tomjaguarpaw · August 27, 2023, 7:41pm

Yup, cabal repl --build-depends PKG.

Liamzy · August 27, 2023, 9:27pm

I think this comes down to whether or not you want a package environment file.

The advantages of having a package environment file:

-Directly starting ghci is much faster than cabal repl --build-depends foo.

-Ability to load packages via :set -package foo; for instance, let’s say you’re using twain, and all of a sudden you decide to recycle the ghci window for flatparse, not losing any draft functions in ghci. With cabal repl --build-depends foo, you don’t have an environment file and therefore cannot :set -package flatparse.

In actuality, so far, I think the better way to do this might simply be:

cabal install base --lib.

Since base is already installed by default, this amounts to no occupation of the package or module namespace whatsoever, takes no download time, etc.

The only concern I might have is that there might still yet be a better way to spawn a package environment, an actually canonical one. I’m going through the cabal docs right now for such.

hellwolf · August 27, 2023, 9:56pm

Personally I find anything polluting global environmental objectionable.

What do you think of cabal script style of declaring dependency in the same file: 6. cabal-install Commands — Cabal 3.4.0.0 User's Guide

It’s a different workflow: you create a scratch pad file → declare some dependency and cabal flags in it → :!cabal repl %. It’s some upfront “setup cost”, but at least your repl session is more versatile with stuff you need always loaded even after rebooting.

Ambrose · August 27, 2023, 10:05pm

It’s mostly frowned upon because it can cause cabal hell issues if it’s all you use for package management. So in a way, it’s historical more than an ironclad, proven rule imo.

I honestly don’t think there’s anything wrong with globally installing libraries you commonly use with ghci.

Now, if you’re on NixOS, you might as well just install a ghcWithPackages and maintain it that way. Compared to the other options in this thread, this is the only “more pure” one I know of that actually matches globally installing packages (e.g. it’s straight ghci, you don’t have to build packages periodically).

artem · August 28, 2023, 1:50am

If you always use cabal install --lib --package-env=. to employ a directory-local environment instead of the global one, you may avoid the main issue (the cabal hell) with polluting the global one.

By the way, the upcoming cabal 3.12 install --lib will feature some improvements that should accommodate a number of common use cases like bumping the version of a package that has already been added to the environment.

Still, install --lib has a number of limitations. The most prominent imo is inability to use profiling versions of libraries.

andreabedini · August 28, 2023, 3:20am

cabal-install’s solver give prefence to already installled packages. Therefore changing the global packages database can potentially change the build plan of all projects you build.

TBH this is a reproducibility failure mode I had not thought about before.

Ambrose · August 28, 2023, 4:06am

I’m surprised v2 builds behave this way - feels contradictory to…their whole sales pitch?

andreabedini · August 28, 2023, 10:26am

¯_(ツ)_/¯

I didn’t have time to test this today but AFAIK the global package db is always in scope (because that’s where you get base, for example)

Ninja edit: nix style builds are not nix builds though There’s a bunch of system stuff that is not (and can not) be captured in cabal-hash.txt

hellwolf · August 28, 2023, 11:15am

sounds disturbing… I will give it a try later

tomjaguarpaw · August 28, 2023, 8:04pm

I’d be interested to know your conclusion. You may also be interested in How can environment file get into invalid state? · Issue #8473 · haskell/cabal · GitHub.

hellwolf · August 28, 2023, 10:10pm

Here are some of my findings:

$ cabal install --lib tuple
$ ghci
Loaded package environment from /home/hellwolf/.ghc/x86_64-linux-9.6.2/environments/default
GHCi, version 9.6.2: https://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/hellwolf/Applications/hwutils.repo/hw.dotfiles/files/ghci
λ :set -package tuple
package flags have changed, resetting and loading new packages...
λ import Data.Tuple.Curry
...
$ cat /home/hellwolf/.ghc/x86_64-linux-9.6.2/environments/default
clear-package-db
global-package-db
package-db /home/hellwolf/.cabal/store/ghc-9.6.2/package.db
package-id tuple-0.3.0.2-62430523e24213173449b37742193be893eca90faab2241d948f3c006f50683a
$ # errrrr, how to delete package ???
$ rm -f /home/hellwolf/.ghc/x86_64-linux-9.6.2/environments/default
$ ghc-pkg --package-db /home/hellwolf/.cabal/store/ghc-9.6.2/package.db list
... # this is fun too

Anyways, it doesn’t seem to be too magical, good to know how it works. But I don’t think I would ever use or rely on this.

References:

andreabedini · August 29, 2023, 8:01am

tl;dr: yeah but nah, cabal install --lib does not install packages in the global packagedb; this is what Setup.hs and cabal v1-install used to do.

Let’s give this a go.

I am using GHC 9.4.6, and the global packagedb has mtl-2.2.2 preinstalled.

❯ ghc-pkg list mtl
/home/andrea/.ghcup/ghc/9.4.6/lib64/ghc-9.4.6/lib/package.conf.d
    mtl-2.2.2

Note that a newer version, mtl-2.3.1, is available from Hackage. Let’s make a package that declares a dependency on mtl>=2.2.2.

❯ cabal init --non-interactive --minimal --depend "base,mtl>=2.2.2"
[Log] Using cabal specification: 3.0
[Warning] unknown license type, you must put a copy in LICENSE yourself.
[Log] Creating fresh file CHANGELOG.md...
[Log] Creating fresh directory ./app...
[Log] Creating fresh file app/Main.hs...
[Log] Creating fresh file cabal-global-pkg-db.cabal...
[Warning] No synopsis given. You should edit the .cabal file and add one.
[Info] You may want to edit the .cabal file and add a Description field.

The solver prefers the installed version of mtl-2.2.2.

❯ cabal build --dry-run -v3 | grep '^\['
["/home/andrea/.ghcup/ghc/9.4.6/bin/ghc-pkg-ghc-9.4.6","/home/andrea/.ghcup/ghc/9.4.6/bin/ghc-pkg-9.4.6","/home/andrea/.ghcup/ghc/9.4.6/bin/ghc-pkg","/home/andrea/.ghcup/bin/ghc-pkg"]
[__0] trying: cabal-global-pkg-db-0.1.0.0 (user goal)
[__1] trying: base-4.17.2.0/installed-4.17.2.0 (dependency of cabal-global-pkg-db)
[__2] trying: rts-1.0.2/installed-1.0.2 (dependency of base)
[__3] trying: ghc-prim-0.9.1/installed-0.9.1 (dependency of base)
[__4] trying: ghc-bignum-1.3/installed-1.3 (dependency of base)
[__5] trying: mtl-2.2.2/installed-2.2.2 (dependency of cabal-global-pkg-db)
[__6] next goal: transformers (dependency of mtl)
[__6] trying: transformers-0.5.6.2/installed-0.5.6.2
[__7] done

The solver knows that mtl-2.3.1 is available and we can explicictly ask for it.

❯ cabal build --dry-run -v3 --constraint 'mtl==2.3.1' | grep '^\['
[__0] trying: cabal-global-pkg-db-0.1.0.0 (user goal)
[__1] trying: base-4.17.2.0/installed-4.17.2.0 (dependency of cabal-global-pkg-db)
[__2] trying: rts-1.0.2/installed-1.0.2 (dependency of base)
[__3] trying: ghc-prim-0.9.1/installed-0.9.1 (dependency of base)
[__4] trying: ghc-bignum-1.3/installed-1.3 (dependency of base)
[__5] next goal: mtl (dependency of cabal-global-pkg-db)
[__5] rejecting: mtl-2.2.2/installed-2.2.2 (constraint from command line flag requires ==2.3.1)
[__5] trying: mtl-2.3.1
[__6] next goal: transformers (dependency of mtl)
[__6] trying: transformers-0.5.6.2/installed-0.5.6.2
[__7] done

Now, if I do cabal install --lib mtl-2.3.1, what happens?

❯ cabal install --lib mtl-2.3.1
Resolving dependencies...
❯ ghc-pkg list mtl
/home/andrea/.ghcup/ghc/9.4.6/lib64/ghc-9.4.6/lib/package.conf.d
    mtl-2.2.2

Where did it go!?

It turns out I have been wrong all along. cabal install --lib does not install packages in the global packagedb!

./Setup.hs install would register packages in the global packagedb by default:
cabal v1-install would register packages in the user packagedb (by default, I think you can pass --global)
and cabal install --lib registers packages in a ghc environment

fgaz · August 29, 2023, 2:02pm

As far as I know this only happened with v1- commands. v2- will only use installed packages if they happen to match the version in the build plan, which is produced independently from the package database

andreabedini · August 30, 2023, 4:58am

No reason to panic, cabal install --lib does not touch the global packagedb.

I am afraid that’s not really how it works. The solver does take into account the pre-installed packages in the global package db. That’s where the familiar rejecting: base-4.18.0.0/installed-4.18.0.0 comes from!

Don’t get me wrong, the solver will still pick a source package if it is needed to satisfy a constraint. In the example above, cabal-global-pkg-db.cabal says it is ok with mtl>=2.2.2 so the solver thinks “if the installed mtl-2.2.2 is good why bother downloading and compiling a different (newer in this case) version”.

Let’s test this by messing up the global packagedb. I used GHCup to get an extra installation of GHC so not to mess up with the one I use everyday.

❯ ghcup install ghc 9.4.7 --isolate ~/Scratchpad/ghc-9.4.7-temp
...
[ Info  ] GHC installation successful

As before:

❯ /home/andrea/Scratchpad/ghc-9.4.7-temp/bin/ghc-pkg list mtl
/home/andrea/Scratchpad/ghc-9.4.7-temp/lib64/ghc-9.4.7/lib/package.conf.d
    mtl-2.2.2

❯ cabal build --dry-run --with-compiler /home/andrea/Scratchpad/ghc-9.4.7-temp/bin/ghc -v3 | grep '^\[_'
[__0] trying: cabal-global-pkg-db-0.1.0.0 (user goal)
[__1] trying: base-4.17.2.0/installed-4.17.2.0 (dependency of cabal-global-pkg-db)
[__2] trying: rts-1.0.2/installed-1.0.2 (dependency of base)
[__3] trying: ghc-prim-0.9.1/installed-0.9.1 (dependency of base)
[__4] trying: ghc-bignum-1.3/installed-1.3 (dependency of base)
[__5] trying: mtl-2.2.2/installed-2.2.2 (dependency of cabal-global-pkg-db)
[__6] next goal: transformers (dependency of mtl)
[__6] trying: transformers-0.5.6.2/installed-0.5.6.2
[__7] done

Now let’s install mtl-2.3.1 in the global packagedb (v1-install doesn’t use cabal.project so I used --with-compiler for both v1 and v2 commands and --global will cause install to use /usr/local as a prefix so I had to specify something else).

❯ cabal v1-install --global --with-compiler /home/andrea/Scratchpad/ghc-9.4.7-temp/bin/ghc --prefix=/home/andrea/Scratchpad/ghc-9.4.7-temp-prefix mtl-2.3.1
...
Completed    mtl-2.3.1

❯ /home/andrea/Scratchpad/ghc-9.4.7-temp/bin/ghc-pkg list mtl
/home/andrea/Scratchpad/ghc-9.4.7-temp/lib64/ghc-9.4.7/lib/package.conf.d
    mtl-2.2.2
    mtl-2.3.1

And indeed:

❯ cabal clean && cabal build --dry-run --with-compiler /home/andrea/Scratchpad/ghc-9.4.7-temp/bin/ghc -v3 | grep '^\[_'
[__0] trying: cabal-global-pkg-db-0.1.0.0 (user goal)
[__1] trying: base-4.17.2.0/installed-4.17.2.0 (dependency of cabal-global-pkg-db)
[__2] trying: rts-1.0.2/installed-1.0.2 (dependency of base)
[__3] trying: ghc-prim-0.9.1/installed-0.9.1 (dependency of base)
[__4] trying: ghc-bignum-1.3/installed-1.3 (dependency of base)
[__5] trying: mtl-2.3.1/installed-C6BZ4nXXpU71PMyBnd8tMy (dependency of cabal-global-pkg-db)
[__6] next goal: transformers (dependency of mtl)
[__6] trying: transformers-0.5.6.2/installed-0.5.6.2
[__7] done

Liamzy · August 31, 2023, 5:35am

So, in general, I can use cabal install base --lib to generate an environment file (unless there’s a safer way to do so), then use cabal install foo without --lib to populate it?

Not only am I calling ghci, I’m also calling ghc directly to compile files, with --package flags, for simple and dumb little scripts that in my view, don’t warrant the ceremony of cabal or stack init and setting up the foo.cabal or package.yaml, and the associated cabal.project etc files.

brandonchinn178 · August 31, 2023, 2:31pm

Why can’t you use cabal/stack script? You just add a shebang and a comment to the file and executing the file will run it through cabal/stack with the given deps

Liamzy · August 31, 2023, 3:07pm

Cabal actually has an appreciable startup time on Arch, at least with my tooling. There’s also the appeal of minimalism; i.e, doing it with minimal tooling.

brandonchinn178 · August 31, 2023, 4:01pm

That’s fair. To your first point, I would hope that cabal script would cache the eecutable and reuse it. Does it not?