Pre-HFTP: GHC nightly releases

Haskell Foundation
#1

I’m definitely not the first person to think of this, but I can’t find any discussion of this in ghc-proposals, tech-proposals, or this Discourse board, so I’ll just put it out here.

Why don’t we set up GHC to release untested nightly builds? This solves a similar problem to the GHC.X.hackage proposal, but in a slightly different way. I personally don’t mind the percolation of adoption through package dependencies; the only problem is that it takes time, and that time (for everyone) starts when a version is released. But if we had nightly GHC releases, package maintainers who cared could set up CI to run tests against the nightly version of GHC and fix issues that come up, so that when the stable version of GHC is released, packages are either already working or are nearly close to working with it.

What I’d imagine this would involve:

  • GHC adds a nightly pipeline (or co-opts an existing pipeline) that builds the binaries for all OSes + uploads to https://downloads.haskell.org/~ghc/nightly/YYYY-MM-DD/ghc-YYYY.MM.DD-...
  • Also symlinks https://downloads.haskell.org/~ghc/nightly/latest/ to the latest YYYY-MM-DD/ directory
  • Update GHCup + Stack (+ HLS?) to allow specifying nightly GHC versions (I believe right now it’s a static list that gets updated with each release? This would perhaps involve allowing for GHC versions not on the list)

I’m not trying to say this would be a quick + easy change, but I think this is a conceptually simpler change to the ecosystem than GHC.X.hackage (while not precluding that change either).

4 Likes
#2

Isn’t this already in place? At least haskell-ci has an option --ghc-head to include a job that builds against GHC HEAD. (Didn’t look up where these nightly releases are taken from.)

1 Like
#3

seems like it’s out of date?

Regardless, it would be nice to be able to pin to GHC nightly 2022-09-23, in addition to getting the latest nightly. I’d imagine there are some use-cases for that.

#4

I did not know, thanks!

Is Making use of GHC bindists built by GitLab CI maybe helpful?

#5

GHCup is also capable of installing head bindists:

ghcup install ghc -u 'https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-linux-fedora33-release.tar.xz?job=x86_64-linux-fedora33-release' head

1 Like
#6

@nomeata that article seems applicable only to Nix

@jaror thanks, that’s good to know. However, I still think it would be nice to have nightly releases in the official downloads location, the same way Rust has nightly and stable in the same location. Generally, I’m pushing for the ease of switching between nightly/stable versions on GHC, the way it’s currently easy to switch between stable versions of GHC. I also want to push for having a “GHC version 2022-09-23” release, instead of just a floating “GHC whatevers-on-master” release.

All this, plus support in Stack, would go a long way to make it easy for package maintainers to test. Even though this is all possible, it’s not documented and isnt very visible as an official workflow

1 Like
#7

It also opens opportunities for usecases that rely on the bleeding age. Eg in rust, I work for a company that relies on the rust nightly releases because they depend on functionality that is not yet on a stable release.

Also, it would make catching regressions happen faster. Eg GHC 9.0 was almost unusable and most people had to wait foe 9.2 before they upgraded. Such issues might have been caught before 9.0 was released if people were testing with nightly

2 Likes
#8

This is really nice! Do you know how to find out what to substitute for x86_64-linux-fedora33 in that path for various other systems? Opening https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/ in a browser doesn’t give me a directory index, unfortunately.

I ask because I’d like to have easier contribution instructions for the error message index than “check out GHC and run Hadrian yourself”. I just checked that I can get error codes on my (Fedora Linux) machine using your command, and indeed I can, with way less effort than building GHC myself. But I suspect that it wouldn’t work as well for users of other operating systems.

1 Like
#9

I don’t know what everything means, but if you look on the Schedules page there is a “nightly master build” scheduled. If you click the “Last Pipeline” (currently this pipeline) then I think you see the whole list of options in the “full build” column.

But it is strange that only the fedora build is marked “release”.

@bgamari and @hasufell probably know much more about this.

1 Like
#10

Thanks for the link! Once I dig out from under a proper pile of work, I’ll see if I can figure out how to install the others with GHCup on some VMs.

This is great!

#11

GitLab API is a disgrace. Some API calls on artifacts specifically only work when the pipeline as a whole has succeeded (which is rather rare).

I gave up trying to make that work automatically, especially because provided bindists can change swiftly.

The only way this can work reliably is if nightly bindists are uploaded properly to downloads.haskell.org with maybe mutable symlinks that ghcup can use to fetch a distro-version-arch triple.

But I’m not sure upstream will do this.

1 Like
#12

To the extent that I qualify as “upstream”, I actually would do this. But as you correctly point out, the pipeline rarely finishes successfully, which has many other annoying side-effects. Plus there are issues building binaries for all the supported arches, anyway. Let’s just say that nightly releases is one of my overarching goals, but there are many sub-goals along the way!

6 Likes