What are the next steps for reinstallable base?

Announcements
1

I’ve opened a ticket on the CLC issue tracker to start discussing the next steps for the “reinstallable base” project. If you have thoughts about where this effort should go, please join the conversation in the issue or here on Discourse:

So far, the community has made significant progress:

  • ghc-internal has been split from base: we now have separate packages for compiler internals and for the public-facing API.
  • base is no longer wired-in: it uses a normal unit-id and is no longer privileged by the compiler.
  • cabal-install can now reinstall base, tooling no longer treats it as a fixed package.

It’s exciting to think about what the next stages of this project could look like. I’m keen to hear the community’s perspective on where we should go from here!

23 Likes
2

who’se gonna be the base maintainer?

1 Like
3

Stack 3.7.1 also handles non-magic base.

For example, I have a local, patched, copy of base-4.21.0.0 (the GHC 9.12.1 and 9.12.2 boot package) with the following added to Prelude.hs:

everythingIsAwesome :: IO ()
everythingIsAwesome = putStrLn "Everything is awesome!"

I also have the toy executable:

module Main (main) where

main :: IO ()
main = everythingIsAwesome  -- Provided by the patched Prelude

Stack’s project-level configuration file is:

# stack.yaml
snapshot: nightly-2025-11-25 # GHC 9.12.2
compiler: ghc-9.12.1 # See note below

extra-deps:
- ../../Haskell/base-4.21.0.0 # Override the GHC base boot package with local patch

The reason for downgrading the compiler to GHC 9.12.1 is that GHC 9.12.2’s boot packages include base-4.21.0.0 (not magic) and ghc-internal: 9.1202.0 (magic) but currently, on Hackage, base-4.21.0.0 has build-depends: ghc-internal == 9.1201.*, and can’t be built with ghc-internal: 9.1202.0.

Then (extracts):

> stack build
...
base         > build (lib) with ghc-9.12.1
base         > Preprocessing library for base-4.21.0.0...
base         > Building library for base-4.21.0.0...
...
Installing executable testNonMagicBase ...
...

> stack exec testNonMagicBase
Everything is awesome!
2 Likes
4

To me the Exciting Possiblity is:

  • You can upgrade from ghc 9.14 to (say) GHC 10.0 while keeping the base API unchanged, precisely as it was for 9.19.

So all your code can compile unchanged with GHC 10.0, not even library version bumps.

Thait’s a bit aspirational – libraries that make complex use of Template Haskell are a sticking point for example – but that’s the payoff we are all seeking I think.

What I don’t know is this: given all the progress you describe, what stands between us and the Exciting Possibility? In what version of GHC does EP become possible? For some libraries EP may be do-able right now? But what are the limits?

9 Likes
5

I would love if we could start a collective effort for working on “base 5.0” which I imagine to be the Haskell standard library of the 2020s, that is

  • a sensible numeric hierarchy
  • a sensible default string type
  • an integration of modern language features (unlifted and unboxed types)
  • useful algorithms, idiomatically implemented (let’s not unbounded TQueue with worst case O(n) readTQueue)
  • great developer tools
  • acknowledge the reality of a distinction between values and computation - great support for non-lazy algorithms and types
  • wayyyyyy less footguns (oops, lazy IO)
  • great documentation and versioning from the get-go (no missing @since)
  • your idea for what makes a great standard library

This has some clear benefits

  1. It makes the language feel fresh and interesting and will do away with frustration that people coming from more modern ecosystems feel. I hope this will also improve the learning experience - It is my impression that Haskell is not being taught as “this is a language you can use” but as “look at the funny tricks we can use with laziness” (at least that’s how it was in my Uni) but that’s just not true!
  2. It allows industry users to jump start their projects quicker, ultimately simplifying on-boarding into the project and making Haskell as a modern, easy to use language more attractive
  3. It will allow people to use features of the (sorry we need to acknowledge microhs, “a”) compiler that would be of great use in many settings but are currently considered “expert” features, like unboxed and unlifted types and linear types. Unboxed and unlifted types for example really aren’t anything exotic and people coming from various ecosystems already probably know how to use them and have an intuition for them but the weak support in the Haskell ecosystem makes them seem obscure and arcane.

I think that there are a lot of really great places to learn from, e.g.

  • rusts std
  • purescript
  • even other Haskell libraries that did things better, like unagi-chan

I imagine this to work as follows: No one person can design this, so we probably want to split things up - each area, e.g. concurrency, numeric hierarchy, string types, etc gets assigned a small team, each team consisting of at least one industry user and someone who teache{d,s} Haskell to work out a design. The design gets reviewed by other teams and verified by a community request for comments and then implemented.

I think this approach can work because I have experienced a lot of people having specific pain points in the current ecosystem - all of those people already have great ideas on how to improve things, many of which aren’t even that controversial!

Afterwards, a long phase of maintenance of two versions of base would follow

  • the old version would basically be continued to maintained as usual
  • the new version would probably be maintained a bit more liberally with the main “constraint” being the PVP - I think it’s a good idea to allow clear improvements that contain breaking changes, at least in the beginning - this has basically become impossible in base and it is clear why, but I think it’s good to change this up.

What do you think about this?

18 Likes
6

Good write-up, thank you for putting into word my feelings about the state of things.

I may even throw in the Elixir’s standard library for the benefit of what a user-oriented documentation looks like, with the ethos of the Joy of Programming brought by the Ruby community: Kernel — Elixir v1.19.3

4 Likes
7

Here’s a problem that immediately comes to mind that I don’t see people discussing. Presently people use the correspondence between base versions and GHC versions to restrict which compiler versions the package supports. The reasons for such restrictions more often than not go beyond the nominal base API, rather usually this is done to request a sufficiently new compiler that supports an extension, or improves/changes typechecking in a way that the package relies on.

The first thing that comes to mind is to replace

build-depends: base >=4.21

with

build-depends: base
if impl(ghc < 9.12)
  buildable: False

This however requires Cabal >=2.2 for buildable support. You can fix this (build-depends: base <0 or similar), but what this means is that the first solution that comes to mind isn’t actually perfect, and that package maintainers will need to be aware of this, what the correct incantation is, etc.

12 Likes
8

Maybe cabal can take a hint from tested-with to determine if the compiler in scope is valid?

9

the issue is that the cabal stanza version and reinstallabe base are at odds - either you allow reinstallable base or old cabal versions!

1 Like
10

It was pointed out elsewhere by @phadej that the actual mechanism we should have used to constrain GHC versions (instead of (or rather, additionally to) base versions) is other-extensions which is a cabal solver constraint - that is also compatible across compilers (which tested-with or impl(ghc) buildable isn’t.

Edit: the problem with that is that GHC sometimes changes language without adding extensions, examples are \cases or ~

4 Likes
11

Yeah, this crossed my mind as well (I came here to make the same comment as @mniip), but as far I’m aware it’s rarely used unfortunately. Also it really only works in an ideal world where all behaviour is controlled by extensions and extension behaviour never changes. Still, maybe this along with impl(ghc) is good enough in practice. But I expect there’ll be a lot of need for Hackage admins to fix things up initially.

2 Likes
12

I remember there were some restrictions about which parts of a cabal file can be revised, to the point where you couldn’t add new dependencies. I suspect adding an entire if impl block might be an problem here.

3 Likes
13

Ah, yes, I’m sure you’re right.

1 Like
14

Thanks everyone for your comments, they have been really useful to get a handle on some hidden issues which I hadn’t considered deeply. It has been very helpful to make the project concrete. I’ve tried to summarise the four main issues and sticking points which people have raised so far.

We will do some work at Well-Typed to scope out these design points and propose some way to make progress. If there is anyone interested in collaborating then let me know!

We currently rely on base bounds to signal which GHC versions a package supports

For better or worse, build-depends: base >= … has become the ecosystem’s unofficial way of expressing compiler compatibility.
If base no longer tracks GHC releases, we lose a reliable way to express that a package only builds with a specific version of GHC. Thanks to @mniip and @george.fst for bringing this one up.

Cabal doesn’t yet provide a good way to specify compiler requirements

Mechanisms like if impl(ghc) or other-extensions either don’t work well, aren’t revision-safe, or don’t capture upper bounds on GHC.
Before reinstallability is safe, Cabal needs a proper, stable way for packages to say which versions of GHC or other compilers they work with. We need to do some more work to remove internal APIs from the base package.

A question which has been raised is whether it is better to have reinstallable base if packages will still have a tight bound on ghc-version. Packages will still need to be updated when a new compiler version comes out.

base and ghc-internal are still too intertwined

A lot of base’s implementation lives in ghc-internal.
That makes it hard to maintain base independently, because changes in ghc-internal can affect it, and contributors still have to go through the GHC repo.
Independent releases might only make sense once this boundary is clearer.

One concrete way to make progress would be deprecate some unstable modules - Formally deprecate modules informally marked unstable in #146 · Issue #299 · haskell/core-libraries-committee · GitHub

There is also a spreadsheet which explains the status of all the modules in base library. Base stability - Google Sheets

There will be an increased maintainership and coordination burden

If base becomes a normal, independently released library, we’ll need clearer boundaries and stronger coordination between GHC maintainers, CLC, and a new base-maintainers group.
It seems like there is desire in the community to go in this direction but it’s more work for someone.

It seems that the CLC is interested in this direction, but it would still need to be made more concrete.

7 Likes
15

Good summary, thank you Matthew.

Just to check

  • The first two items “base bounds signal GHC version” and “good way to specify compiler” are really the same aren’t they? That it, if you want to say “I rely on GHC 9.6 or later” you want a way to say that, not a proxy “base > xx.yy”. Is that right?

    Actually it might not need a whole new mechanism. We could declare that the version number of ghc-internal always tracks GHC itself, so GHC 10.2 comes with ghc-internal-10.2. Back to proxying I know, but if the proxy is explicitly declared policy, that might be fine. And maybe easier than Cabal implementing a whole new mechanism

    Or we could have a special proxy package ghc-version with no content, but just a version number. Now you’d say build-depends: ghc-version > 9.6 which reads rather nicely.

  • " We need to do some more work to remove internal APIs from the base package." Absolutely. But it is quite a separate task from “good way to specify compiler reqts”. It’s really part of the next item “still too intertwined”.

  • “A question which has been raised is whether it is better to have reinstallable base if packages will still have a tight bound on ghc-version”. If we declare a clear intent that GHC Y should compile anything that GHC X does (where Y>X), and allow packages to depend on a base API that does not change when adopting GHC Y, the incentive to make a hard upper bound on GHC’s version becomes much smaller.

8 Likes
16

In that case, maybe it would be a good idea to think about making it a hard requirement that language extensions are not only backwards but also forwards compatible, that is, making other-extensions the declarative source of truth for which compilers the package is compatible with. The upside being that this is portable across different compilers!

4 Likes
17

Language extensions have not been very stable historically, for example the recent additions to lambdacase. Even some extensions in GHC2021 like ScopedTypeVariables are controversial and probably will change.

So perhaps we need to add version numbers to language extensions…Although, I feel like GHC is already a laughingstock due to the proliferation of extensions.

3 Likes
18

That is true, but maybe this can be changed!

3 Likes
19

For the “constraining the version of the compiler” problem, see also How should a package represent the fact that it does not support a particular GHC version? · Issue #9648 · haskell/cabal · GitHub .

IMO the status quo is pretty unsatisfactory, and it would be great to have a better mechanism. So I think it would be worth it even aside from making reinstallable base easier.

3 Likes
20

Can haddock be run on this separated base? More generally, can haddock haddock a base different to the installed one?