Hidden package — Could not load — how to :set

Learn
#1

Some programs that once-upon-a-time were buildable with just something like this:

~/.ghcup/ghc/9.4.4/bin/ghc --make prog.hs

now fail with messages like this:

Could not load module ‘GHC.Compact’
It is a member of the hidden package ‘ghc-compact-0.1.0.0’.
You can run ‘:set -package ghc-compact’ to expose it.

Is :set -package ghc-compact somethings that’s supposed to be on the command-line? Like:

~/.ghcup/ghc/9.4.4/bin/ghc --make :set -package ghc-compact prog.hs

Examples:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/binarytrees-ghc-8.html#log

https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/knucleotide-ghc-2.html#log

#2

Is :set -package ghc-compact somethings that’s supposed to be on the command-line?

Hmm, no, it’s not. It’s something you give write at a ghci prompt. Very strange to see it mentioned from ghc.

#3

Those are my programs. The original build instructions I gave for binarytrees were:

$ cabal install -O2 --lib --package-env . ghc-compact parallel
$ ghc -package-env .ghc.environment.x86_64-linux-8.10.1 -O2 -threaded -rtsopts -fno-cse -fllvm -XBangPatterns HaskellGHC8.hs

But that is specific to GHC 8.10.1 so it might be easier to use this (this will create a file called env):

$ cabal install -O2 --lib --package-env ./env ghc-compact parallel
$ ghc -package-env ./env -O2 -threaded -rtsopts -fno-cse -fllvm -XBangPatterns HaskellGHC8.hs
#4

Are you sure you get that message on 9.4? I get it on 8.10 but on 9.0 and above I get a correct message:

% ghc-8.10 --make test17.hs
    Could not load module ‘Opaleye’
    It is a member of the hidden package ‘opaleye-0.9.6.1’.
    You can run ‘:set -package opaleye’ to expose it.

% ghc-9.0 --make test17.hs
test17.hs:6:1: error:
    Could not find module ‘Opaleye’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.

(I used Opaleye rather than GHC.Compact since it seems that GHC.Compact is wired-in in 9.0 and above, at least ghc didn’t have any problems finding it.)

#5

Are you sure you get that message on 9.4?

I’m just copying text from the terminal window into a web page input field.

If it says ~/.ghcup/ghc/9.4.4/bin/ghc that’s what it is.

$ which ghc
/home/dunham/.ghcup/ghc/9.4.4/bin/ghc

at least ghc didn’t have any problems finding it.

Thank you for trying to be helpful.

Maybe if you tried what caused problems for me.

#7

Oh, I beg your pardon. The difference was because I didn’t have any installed version of opaleye under the other GHCs. I now see what you see. This seems like a GHC bug to me.

#8

I filed an issue: ghc gives suggestion as though it is ghci (#22884) · Issues · Glasgow Haskell Compiler / GHC · GitLab

2 Likes
#9

Seems simpler for me to work around the hidden package stuff with this option I poached from tomjaguarpaw’s bug report:

ghc -package ghc-compact --make haskell-test.hs

Would that be sufficient?

#10

Seems like that works, it just becomes ugly with many packages.

Only 3 of those old programs fail now.

#11

Using ghc ... -package somepackage only works if somepackage has already been installed. So for packages that are not distributed with GHC you would need to run some form of cabal ... command every time you switch to a new GHC version.

I wonder how other ecosystems deal with this. I believe Rust doesn’t even allow using dependencies without making a crate (the Haskell equivalent would be a cabal package). But maybe other languages generally don’t use as many dependencies as Haskell.

#12

every time you switch to a new GHC version

Once a year maybe, so that’s OK for the benchmarks game.

other ecosystems

For Rust someone showed me cargo commands to download and build the latest versions of a programs dependencies. That expanded into a shell script and I jump through hoops making it work.

For Perl I’ve given up on a couple of benchmarks game programs because they pull in a build system with a seemingly bottomless pit of dependencies.

Of course, these things may work very well for their intended audience.

Meanwhile is:

Could not find module ‘Control.Concurrent.ParallelIO.Global’

an example where I need to run cabal ?

#13

Yeah, if you see such an error without the suggestion to use -package ... then you will need to run cabal.

For your specific example you can search for the module on hoogle: Control.Concurrent.ParallelIO.Global - Hoogle. Then you’ll see all the results are from the parallel-io package.

One complication now is that as far as I know there is no cabal command to install a package without exposing it in a package environment file. So you can run cabal install --lib --package-env ./env parallel-io and then just discard the generated ./env file and use ghc -package parallel-io ... instead.

You could also save those env files for every benchmark somewhere and just use ghc -package-env ./env ... to compile. That avoids the ugliness with multiple packages, but it requires some extra management of those files.

Finally, one other alternative is to just run cabal install --lib parallel-io, which installs it into an environment that is always read by default. This avoids the ugly command line arguments and does not require you to manage environment files. However, this can lead to very confusing error messages if you are trying to install packages that conflict with packages that were installed earlier. So, it is generally not recommended, but maybe that isn’t so much of a problem in this case.

#14 
$ cabal install --lib parallel-io
Resolving dependencies...
…
…
Building     parallel-io-0.3.5 (lib)
Installing   parallel-io-0.3.5 (lib)
Completed    parallel-io-0.3.5 (lib)

$ cabal install --lib vector-algorithms
Resolving dependencies...
…
…
Building     vector-algorithms-0.9.0.1 (lib)
Installing   vector-algorithms-0.9.0.1 (lib)
Completed    vector-algorithms-0.9.0.1 (lib)

$ cabal install --lib text
Resolving dependencies...
Up to date

But

knucleotide.ghc-3.hs:33:1: error:
    Could not find module ‘Text.Builder’
    Use -v (or `:set -v` in ghci) to see a list of the files searched for.
   |
33 | import qualified Text.Builder                         as Builder
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Also I was trying to avoid being online when time measurements were being made, by downloading required libraries beforehand.

#15

We should really (as a community) stop recommending cabal install --lib. It’s horrendously flaky.

#16

That seems to be an aside — in this actual example what do you recommend?

#17

Well, do @jaror’s original instructions work? Hidden package — Could not load — how to :set - #3 by jaror

#18

Now I’m confused. Those instructions seem to use cabal install --lib which you say should not be recommended and is horrendously flaky?

"Is cabal-install stable enough yet that GHC should be inflicting it on newbies?"
#19

That module seems to come from the text-builder package. It is not from text.

#20

Hmm what on Hoogle told you that ?

Only 2 broken GHC programs now.

#21

Hmm what on Hoogle told you that ?

It’s the 9th result on that Hoogle page.