Is every hackage package buildable?

Learn
#1

It’s clearly not the case, otherwise nix store wouldn’t have so many broken packages in haskellPackages.

so were hackage packages once buildable when they first released?

how does my personal cabal project work if it depends on a non-buildable hackage library?

In my case, my project depends on ghc-9.2.8. I fetched the source code in a tar ball file.

The first question encountered is cabal wouldn’t allow you to build a package with the name ghc, from my understanding. This can be workaround by changing the project name anything other than ghc.

The next question is that the project depends on files that not inside the source code.
Example:

> cabal build all 
....
GHC/Builtin/Names.hs:2073:2: error:
     fatal error：primop-vector-uniques.hs-incl：No such file or directory
     2073 |
          |  ^
     |
2073 |
     |  ^
Compilation Interrupted


...

It begs the question.

Assume every hackage package was once buildable.
If the source code was missing in the first place, how was ghc-9.2.8 built the first time?

and why my current project run smoothly if it depends on some non-buildable packages?

#2

But haskellPackages are built with a specific version of GHC, aren’t they? I would say pretty much every package was buildable with at least one version of GHC when it was uploaded.

and why my current project run smoothly if it depends on some non-buildable packages?

It probably doesn’t depend on non-buildable packages.

The first question encountered is cabal wouldn’t allow you to build a package with the name ghc, from my understanding.

Maybe. I’d be surprised, since you can depend on the ghc package in cabal files! The link you shared refers to base, not ghc, so I’m not really sure what you’re saying here.

If the source code was missing in the first place, how was ghc-9.2.8 built the first time?

I don’t know why you can’t build ghc-9.2.8 from the source distribution but this works, so it must be buildable in some form:

cabal repl -w ghc-9.6.2 --build-depends='ghc==9.6.2'

(I’m using 9.6.2 because that’s the version of GHC I have. I imagine 9.6.8 works the same.)

#3

I think that Hackage only does some checks on the cabal file before you can upload a package. These checks correspond roughly to what cabal check does on the command line.
Hackage does have a builder which tries to build the package and upload the generated docs, but this build can fail. So, to my knowledge, Hackage does not prevent you from uploading packages which are completely unbuildable.

Adding build-matrices for every package on Hackage is probably unfeasible right now, but maybe Hackage could start displaying the tested-with field from cabal files. There would be no formal guarantee that the advertised GHC versions are actually tested and working, but it could encourage people to use this field. Flora.pm already displays this field, e.g in @hackage/prettyprinter

2 Likes
#4

Note that Flora is not always showing Tested-With. For example for @hackage/SDL . But you can still look in the Cabal file: https://hackage.haskell.org/package/SDL-0.5.9/SDL.cabal

#5

Once upon a time we had https://matrix.hackage.haskell.org/ which did a build of every hackage package, with all current compilers, and reported the results in a matrix.

But alas it is no more.

2 Likes
#6

Hmm, I found this Github discussion and this reddit thread. It is just a bit over a year that the builds are no longer running. From what I gather there, it doesn’t look completely impossible to get matrix builds running again.

1 Like
#7

Note also that Nix’s packages are “fragile” in the sense that:

  1. They require all test suites to pass (which may not work, especially with nix).
  2. They are more likely to fail version bound checks, given that nixpkgs only has 1-2 versions of a given package at any time.

I would guess that the ratio of

         buildable (haskellPackages + dontCheck + doJailbreak)
------------------------------------------------------------------------
                       buildable haskellPackages

is quite high.

#8

Hackage is just a package dump. It does not enforce that packages are maintained, and since the break of matrix.hackage it cannot even tell whether packages still build.

Stackage selects (currently around 3000) packages from Hackage that are guaranteed to build and kept up-to-date. Any package can join Stackage as long as the maintainer is willing to do their job.

I recommend to limit dependencies to those that are on Stackage.

2 Likes
#9

I don’t think anyone has explicitly said this yet: GHC itself—the Haskell compiler—is not buildable with cabal, and does not even really live on Hackage. There’s a ghc package, but unfortunately that’s not ghc-the-compiler and it’s mostly just there so you can browse the Haddocks for the internal libraries that you can access in your program when you compile it with GHC.

Although there’s a ton of Haskell inside GHC, it’s not really a “Haskell package” like everything else on Hackage. So that’s at least one package on Hackage that is not buildable.

1 Like
#10

I’m not sure that’s accurate. HLS uses that package to be a compiler, afaiu.

The package just doesn’t have an executable component.

In terms of “can you build it?” the answer is similar to ‘base’ I guess: it’s shipped with GHC and non-upgradable.

1 Like
#11

This is definitely not the case; and I know because I myself have accidentally uploaded packages that are unbuildable (they could only be built with other dependencies fetched via stack, not cabal).

To be honest it’s a bit of an annoying situation, as oftentimes it’s not possible to get a buildable hackage package because of some stale dep that hasn’t been updated by the maintainer.

Probably this will become a bigger issue as time goes on and more projects go unmaintained.

1 Like
#12

good point. I think haskellPackages are automatically managed about callHaskellPackage, but which specific version of ghc the function invokes is out of my knowledge.

Intuitively this makes sense. However, when cabal build a brand new local project, assuming no local cache exist, cabal fetches all dependencies listed in .cabal from hackage and build locally, right? If there are no sufficient files for ghc-9.2.8 in the tarball file, how can cabal build locally?

The same error message, rejecting: ghc-9.6.2 (only already installed instances can be used) occurred to me, when i was trying to build ghc-9.6.2. The workaround still applies here. Im surprised that this didn’t happen to you.

I checked the source code for ghc-9.6.2. It clearly contains more source files than ghc-9.2.8, e.g. most *.h files are not presented in ghc-9.2.8.

any hints for fixing this error messages?

> cabal repl -w ghc-9.6.2 --build-depends='ghc==9.6.2'
Resolving dependencies...
Build profile: -w ghc-9.6.2 -O1
In order, the following will be built (use -v for more details):
 - ghc-compiler-9.6.2 (first run)
Configuring ghc-compiler-9.6.2...
genprimopcode: readCreateProcess: posix_spawnp: does not exist (No such file or directory)
Error: cabal: Failed to build ghc-compiler-9.6.2. The failure occurred during
#13

Didn’t know such site exists, looks well-polished, cheers

#14

Ye, True. I forgot the version bound checks. Thx for reminding me.

#15

The ghc package is functional, we have a working prototype depends on it, calling runGhcT and doing program transformation stuff.

huh, i hope that’s not the case. using ghc-as-a-lib requires some functions that are not exported, i want to use some record functions such that i can peek into some data type and pretty print their value.

It just occurs to me, i can build ghc locally, exporting all the functions i need within. And build my project with it.

#16

I didn’t know that other than formalization, you devote some of your time to package maintaining. :grinning_face_with_smiling_eyes:

#17

The same error message, rejecting: ghc-9.6.2 (only already installed instances can be used) occurred to me, when i was trying to build ghc-9.6.2. The workaround still applies here. Im surprised that this didn’t happen to you.

Ah, this is interesting! Perhaps this is because the ghc package very specifically exposes parts of GHC itself and therefore can’t be built separately from a binary build of GHC.

However, there will be very few packages of this nature. ghc-prim might be another, and the forthcoming ghc-internal and ghc-experimental.

#18 
$ cabal repl -w ghc-9.4.8 -b ghc --constraint 'ghc == 9.8.1'                                                                              Wed 15:56
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] trying: fake-package-0 (user goal)
[__1] next goal: ghc (dependency of fake-package)
[__1] rejecting: ghc-9.4.8/installed-9.4.8 (constraint from command line flag
requires ==9.8.1)
[__1] rejecting: ghc-9.8.1 (constraint from non-upgradeable package requires
installed instance)
[__1] rejecting: ghc-9.6.3, ghc-9.6.1, ghc-9.4.7, ghc-9.4.6, ghc-9.4.5,
ghc-9.4.4, ghc-9.4.3, ghc-9.4.2, ghc-9.4.1, ghc-9.2.8, ghc-9.2.7, ghc-9.2.6,
ghc-9.2.5, ghc-9.2.4, ghc-9.2.3, ghc-9.2.2, ghc-9.2.1, ghc-9.0.2, ghc-8.10.7,
ghc-8.10.2, ghc-8.10.1, ghc-8.8.3, ghc-8.8.1, ghc-8.6.5, ghc-8.6.4, ghc-8.6.1,
ghc-8.4.4, ghc-8.4.3, ghc-8.4.1, ghc-8.2.2, ghc-8.2.1, ghc-9.2.3.20220620
(constraint from command line flag requires ==9.8.1)
[__1] fail (backjumping, conflict set: fake-package, ghc)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: ghc, fake-package
#19

There is a combinator called dontCheck which can be used to disable tests. As you can see, there are many packages for which tests have been disabled: configuration-common.nix

#20

Exactly. It looks like a real package, but I don’t think you can download it from Hackage and use it independently. It walks like a duck and quacks like a duck, but it’s actually part of the GHC installation and there is no duck. To me that means it’s part of GHC and not a “buildable Hackage package” :slight_smile: But maybe my definition is too narrow. After all, it is on Hackage and you can use it. You just have to use exactly the same one that’s installed with the GHC you happen to be using.

2 Likes