# A different approach to emancipating \`base\`

**URL:** <https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695>\
**Category:** Uncategorized\
**Created:** [January 30, 2023, 1:15pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695 "2023-01-30T13:15:50Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![rae](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/rae/32/888_2.png) [@rae](https://discourse.haskell.org/u/rae)\
**Post date:** [January 30, 2023, 1:15pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/1 "2023-01-30T13:15:50Z")

</div>

There has been much talk of the challenge of emancipating `base` from the GHC release process over the years. Challenges have been listed. I have not followed these conversations deeply, but I respect those that have written how hard it would be. In this post, I describe a different way to structure the relationship between GHC and `base` than we have today. I don’t have an opinion on whether this is a _better_ way than the status quo, and I’m not advocating for change here; instead, I’m laying out an alternative design so that more-informed others can consider it and decide whether it is in fact better than the status quo.

### Assumption

The key challenge of emancipating `base` is that GHC has special treatment of some of the definitions in `base`, notably `Monad` and friends (which GHC must know about in order to compile `do` blocks) and `Num` and friends (which GHC must know about in order to compile numeric literals). Because GHC knows about these definitions, the modules that define them must be shipped with GHC itself, causing a key challenge in emancipating (the rest of) `base`. This post is all about defining a different potential relationship between e.g. the definition of `Monad` and GHC.

### Background

GHC currently is aware of at least three categories of definitions in libraries:

1. _Primitive types_. There are a number of type definitions that have no phrasing in Haskell. A good example is `Int#`. GHC gives these types meaning with a little magic in the compiler (that is, they are treated in a different way from ordinary types), and they are exported from the magic module `GHC.Prim`. The `GHC.Prim` module has no source code (in its place is a stub just used for creating Haddock output) and lives in the special package `ghc-prim`. GHC knows that when `GHC.Prim.Int#` is looked up, it should provide its internal magical type definition; the type is identified by the name `GHC.Prim.Int#`.

2. _Primitive values_. There are a number of primitive values, like `negateInt#`, that cannot be written in Haskell. For the purposes of this post, these are treated identically to primitive types, in that they are exported from `GHC.Prim` and are identified by their name (e.g. `GHC.Prim.negateInt#`). The GHC implementation routes these through quite a different path than primitive types, but the differences don’t matter here.

3. _Wired-in types_. There are a number of Haskell type definitions that GHC knows intimately. A good example is `Bool`. GHC knows where `Bool` is declared, that it has exactly two constructors `False` and `True`, and what order these constructors are declared in. It must know this in order to compile guards and `if` expressions. Most of these live in the module `GHC.Types` in the `ghc-prim` package; some live elsewhere in the `ghc-prim` package. None lives in `base`. When GHC compiles the actual definition of these types (`GHC.Types` _does_ have [source code](https://gitlab.haskell.org/ghc/ghc/-/blob/master/libraries/ghc-prim/GHC/Types.hs)), it essentially ignores the definition provided and uses its own internal definition instead. These types are identified by their name (e.g. `GHC.Types.Bool`).

4. _Known-key types and values_. There are a number of Haskell type and value definitions that GHC knows how to find. A good example is `Monad`. GHC knows the package, module, and name of `Monad` (to wit, `base`, `GHC.Base`, and `Monad`). When it needs to emit, say, a `Monad m` constraint, it looks up the type with that name from that module and proceeds. These definitions might appear outside of `ghc-prim` and pose the challenge: we cannot today have it so that these types can get released separately from GHC.

The challenge is all around these known-key definitions. I don’t think anyone is talking about making changes to the treatment or packaging of any of the first three cases – just known-key types cause pain.

### “Don’t call us; we’ll call you!”

The key challenge here is simply that GHC knows exactly where the known-key definitions are declared, and thus the package including these must be released with GHC. My idea here is simply not to do this, but instead to have the types declare themselves to be e.g. the “real” `Monad`. This is what Agda does, for example, with its `BUILTIN` directive and how OCaml works with its `external` declarations.

Concretely, we might imagine a declaration like

```haskell
class {-# BUILTIN Monad #-} Applicative m => Monad m where ...

```

where the pragma tells GHC that this definition is _the_ `Monad` definition. When GHC reads this definition, it remembers that the class being defined is _the_ `Monad`, and so when it needs to produce a `Monad m` constraint, it knows where to go. This approach means that `base` is no longer special – instead, it just means that there must be one class definition labeled as `Monad` loaded when compiling any module that uses `do`. `base` can evolve independently from GHC – or community members could write completely alternative standard libraries, as long as they have a `Monad` class.

We could imagine an even-more Haskelly approach of having `class Builtin (name :: Symbol) (ty :: k)` and the definition of `Monad` would come with `instance Builtin "Monad" Monad`. The class-instance solver would then be used to find the `Monad` class. This would likely be considerably less performant.

This route also interacts nicely with `-XRebindableSyntax`: instead of looking for one definition with `BUILTIN Monad`, GHC could look for the in-scope type with `BUILTIN REBINDABLE Monad`. This would likely be considerably easier to configure than the current `-XRebindableSyntax`.

Left unsolved: what to do with a `NoImplicitPrelude` module that doesn’t import anything. In this case, no other modules are loaded, and so GHC doesn’t have a chance to find the `BUILTIN Monad`. I actually think it’s reasonable to error in such a file if it uses a `do`, but others may feel differently (and this would not be backward compatible). We could alternatively say that `NoImplicitPrelude` really means `import "base" GHC.Required ()` or something, which looks for a package named `base` and a module named `GHC.Required`, loading the definitions therein. Then we would make sure that `base` had such a module, and that it depended on all the modules with `BUILTIN`s. But the `base` could still evolve independently from GHC.

### Conclusion

Perhaps I’m addressing the wrong problems here, and perhaps there are other unseen challenges in this approach. But it might be that this idea – changing GHC instead of `base` – allows things to move forward in this space.

---

<div class="post-metadata">

**Author:** ![david-christiansen](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/david-christiansen/32/2404_2.png) [@david-christiansen](https://discourse.haskell.org/u/david-christiansen)\
**Post date:** [January 30, 2023, 1:34pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/2 "2023-01-30T13:34:27Z")

</div>

I don’t know enough about the code to evaluate any concrete approach.

But what about other parts of `base` that are strongly coupled to GHC? Here I’m thinking of things like `GHC.Exts`, which are somewhat stable and could use this at the cost of there being lots and lots of `BUILTIN` options, but even more so about things like `GHC.Stats.RTSStats`, which seem to me to be more likely to change from release to release.

I suspect that the answer is to put those into a package that does ship with GHC, rather than `base`, and deprecate their current positions in `base`. But it seems to me that the real decoupling work will spend a short time on `Monad`, and a very long time figuring out which of these definitions remain strongly coupled and which can be loosely coupled, all with a good deprecation story. So I worry that this proposal (while elegant) is mostly solving the easy problem rather than the hard one, unfortunately.

---

<div class="post-metadata">

**Author:** ![adamgundry](https://avatars.discourse-cdn.com/v4/letter/a/34f0e0/32.png) [@adamgundry](https://discourse.haskell.org/u/adamgundry)\
**Post date:** [January 30, 2023, 1:48pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/3 "2023-01-30T13:48:46Z")

</div>

I still don’t understand why we can’t “just” rename `base` to `ghc-base`, ship the latter with GHC, and make a new `base` package that re-exports modules/definitions from `ghc-base`. Crucially, that means that GHC devs can make changes to `ghc-base` without needing CLC approval (since the CLC is now claiming to have authority over every change to the exposed API of `base`). See [Specification of the stable API of `base` · Issue #105 · haskell/core-libraries-committee · GitHub](https://github.com/haskell/core-libraries-committee/issues/105) for a recent related discussion.

Of course this doesn’t completely decouple everything, because `ghc-base` will still be a big tangle, and known-key types have to live in `ghc-base` and be re-exported by `base`, so they can’t change freely. But it’s a smallish step in the right direction. Or am I missing a reason this is fundamentally difficult?

---

<div class="post-metadata">

**Author:** ![nomeata](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/nomeata/32/1188_2.png) [@nomeata](https://discourse.haskell.org/u/nomeata)\
**Post date:** [January 30, 2023, 2:35pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/4 "2023-01-30T14:35:56Z")

</div>

It’s not just known-key types, but also known-key values (e.g. the Monad members) isn’t it?

Would Richard’s proposed change would make it that `base` can be built or rebuild normally by Cabal, and thus be fetched from hackage and built independently, right? I think this used to be discussed under the term “upgradeable base”, but I am not sure of the state of affairs. I vaguely recall that this already works to some degree?

It would still require there to be exactly one `BUILTIN Monad` pragma visible to GHC, right? The pragma approach would create new error conditions, such as multiple such pragmas visible.

BTW, GHC also has [known-key types in other packages](https://hackage.haskell.org/package/ghc-9.4.4/docs/src/GHC.Unit.Types.html#wiredInUnitIds): ghc-bignum, base and template-haskell.

---

<div class="post-metadata">

**Author:** ![TerrorJack](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/terrorjack/32/5155_2.png) [@TerrorJack](https://discourse.haskell.org/u/TerrorJack)\
**Post date:** [January 30, 2023, 6:32pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/5 "2023-01-30T18:32:46Z")

</div>

> I think this used to be discussed under the term “upgradeable base”, but I am not sure of the state of affairs. I vaguely recall that this already works to some degree?

`cabal` or `stack` doesn’t like the idea of rebuilding `base`, but technically it’s possible to create a tarball for `base` and invoke `Setup.hs` to build it and install it to other package databases.

The real problem is: that tarball would need to contain certain files auto-generated by configure/hadrian. And those are target-specific. So you can’t really have a single `base` package uploaded to Hackage and expect it to work for all targets.

---

<div class="post-metadata">

**Author:** ![nomeata](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/nomeata/32/1188_2.png) [@nomeata](https://discourse.haskell.org/u/nomeata)\
**Post date:** [January 30, 2023, 6:57pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/6 "2023-01-30T18:57:02Z")

</div>

Thanks for the reminder, @TerrorJack! So while Richard’s idea sounds reasonable, I’m not sure what exactly we’d gain.

Could we move the target-specific files to `ghc-prim`?

---

<div class="post-metadata">

**Author:** ![TerrorJack](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/terrorjack/32/5155_2.png) [@TerrorJack](https://discourse.haskell.org/u/TerrorJack)\
**Post date:** [January 30, 2023, 7:09pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/7 "2023-01-30T19:09:36Z")

</div>

Or spin out `ghc-base` from `base`. I think @Ericson2314 is working on this.

---

<div class="post-metadata">

**Author:** ![bgamari](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bgamari/32/3599_2.png) [@bgamari](https://discourse.haskell.org/u/bgamari)\
**Post date:** [January 30, 2023, 9:48pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/8 "2023-01-30T21:48:14Z")

</div>

> Concretely, we might imagine a declaration like
> 
> ```haskell
> class {-# BUILTIN Monad #-} Applicative m => Monad m where ...
> 
> ```

For what it’s worth, this is also very similar to how `rustc` solves this problem. Concretely, they have so-called [`lang-items`](https://doc.rust-lang.org/unstable-book/language-features/lang-items.html), which are declarations marked with a `lang` pragma which identifies the declaration as somehow magical to the compiler.

It’s a bit unclear to me where/how these `BUILTIN` pragmas are recorded. In principle we could record them in much the same way we record orphan information: every interface file will contain all of the `BUILTIN` things visible in its transitive imports. However, this it seems like doing this for every module would be rather costly. One advantage of the status quo is that we know precisely which interface we need to load for a particular builtin declaration.

---

<div class="post-metadata">

**Author:** ![simonpj](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/simonpj/32/890_2.png) [@simonpj](https://discourse.haskell.org/u/simonpj)\
**Post date:** [January 30, 2023, 9:51pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/9 "2023-01-30T21:51:23Z")

</div>

I believe that you could upgrade/replace `base` _today_ provided:

- Known-key things and wired things remain defined in the same module, with the same definition. E.g. `Show` is a known-key thing (not a wired-in thing), but if you made it into a multi-parameter type class, `deriving( Show )` would fail badly. (Precisely what GHC relies on about the definition of each known-key thing will vary from one to another.)

I think that’s all; keep that stuff stable and you can upgrade `base`. I don’t know how hard it would be to actually do such an upgrade, but if it was desirable, I bet we could do it.

Is keeping the above known-key/wired-in things stable a major impediment on those who want to upgrade `base`? I doubt it. Richard’s post suggests the ability to move the definition of a known-key thing from one module to another. Maybe we could do that, with work, but the thing you moved would still have to be defined in the same way, so the gain would be extremely modest. I don’t think this is the problem we need to solve.

To my mind, a far bigger impediment is that, if you want the upgrade to be backward compatible, you need to support the API of `base`; and that API currently consists of every export of every exposed module, which is far too much.

That lies as the root of Adam’s suggestion:

> I still don’t understand why we can’t “just” rename base to ghc-base, ship the latter with GHC, and make a new base package that re-exports modules/definitions from ghc-base. Crucially, that means that GHC devs can make changes to ghc-base without needing CLC approval

Nothing would be gained here if the new `base` had modules corresponding 1-1 with `ghc-base`, and exported precisely the same things. But what does that really gain compared to simply

- defining the (narrower) API we want `base` to have, and
- giving it that API

Then the exposed modules are the API of `base` (under the purview of CLC) and the non-exposed ones are GHC-internals (and not under CLC’s purview). That is, the same as today! But the API is narrower.

Well, two things would be gained:

1. Psychological. Defining a new package may make it easier to have the conversation about the API we want.
2. Access to internals. It seems _extremely_ likely that some users will want to reach inside GHC’s implementation, and exploit somethign about the way it works, perhaps to improve performance. They may fully understand that in doing so they are exposing themselves to changes that GHC implementors may make. But if the GHC-internals are in un-exposed modules, they simply can’t import them. So it would be helpful to have them as the _exposed_ modules of a GHC-internal package `ghc-base`.

So thumbs up to Adam’s suggestion.

The big challenge would be that the API of `base` would change (at least by shrinking, substantially) and that risks breakage. Of course, breakage in service of less future breakage, but still breakage.

---

<div class="post-metadata">

**Author:** ![bgamari](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/bgamari/32/3599_2.png) [@bgamari](https://discourse.haskell.org/u/bgamari)\
**Post date:** [January 30, 2023, 10:19pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/10 "2023-01-30T22:19:13Z")

</div>

> [@simonpj](#):
>
> Is keeping the above known-key/wired-in things stable a major impediment on those who want to upgrade `base`? I doubt it.

In my opinion, this requirement is the biggest issue standing between us and a decoupled `base` and I’m not sure we can so easily sweep it under the rug. As far as I can tell, most of the high-impact CLC proposals in the last several years have broken this:

- the `Applicative/Monad` proposal added an `Applicative` superclass to `Monad`
- the `Semigroup/Monoid` proposal added a `Semigroup` superclass to `Monoid`
- the `MonadFail` proposal split `fail` into a new `MonadFail` class

Perhaps the fact that they changed these declarations were a sign that they were too impactful and should not have been made to begin with. However, there are plenty of other less impactful changes that similarly affected builtin declarations:

- the type-indexed `Typeable` rework required significant surgery on a good number of known-key typeable-related things
- constant-folding fixes often require the addition/modification of known-key combinators (c.f. see [Fix "build/elem" RULE. (f10d11fa) · Commits · Glasgow Haskell Compiler / GHC · GitLab](https://gitlab.haskell.org/ghc/ghc/-/commit/f10d11fa49fa9a7a506c4fdbdf86521c2a8d3495))
- not infrequently, compiler changes with are otherwise largely invisible to the user require changes in known-key things (c.f. [`unsafeEqualityProof`](https://gitlab.haskell.org/ghc/ghc/-/merge_requests/1869))
- there are plenty of small bugfixes which have required user-facing changes to `base` (e.g. the recent [`ConstPtr` change](https://github.com/haskell/core-libraries-committee/issues/118))

---

<div class="post-metadata">

**Author:** ![simonpj](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/simonpj/32/890_2.png) [@simonpj](https://discourse.haskell.org/u/simonpj)\
**Post date:** [January 30, 2023, 10:37pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/11 "2023-01-30T22:37:40Z")

</div>

> [@bgamari](#):
>
> the `Applicative/Monad` proposal added an `Applicative` superclass to `Monad`

But that implies changing the _definition_ of Monad; it does not imply moving the declaration from one module to another (which is what Richard wants to support). Now, it _might_ be that adding a superclass to a known-key class is fine – but we would need to check that on a case-by-case basis. Some changes are OK; others are not; and it’ll probably vary from one known-key thing to another.

It’s undeniable that some of the user-facing API of `base` is baked into GHC. But I don’t think that coupling is easily removed. The ability to move from one module to another doesn’t seem to me to contribute much.

---

<div class="post-metadata">

**Author:** ![romes](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/romes/32/2912_2.png) [@romes](https://discourse.haskell.org/u/romes)\
**Post date:** [January 31, 2023, 8:19am UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/12 "2023-01-31T08:19:42Z")

</div>

A bit off topic, but being able to define the built ins could be very good regardless of base:

If we can define the built ins as actual built ins rather than as simple data types and functions in an alternative prelude, the alternative preludes could become much more useful and possibly even help in avoiding breakage (by e.g. implementing upcoming changes ahead of time)

Consider the definition of the linearity aware Monad in linear base: even if we import the linear base prelude and use NoImplicitPreude and RebindableSyntax, _every_ function that depends on base’s Monad is unusable in combination with the linear Monad because they simply aren’t the same type. However, if the linear monad was defined with BUILTIN, and only that one imported and not the one from base, wouldn’t all occurrences of Monad refer to the linear Monad?

It sounds great.

---

<div class="post-metadata">

**Author:** ![Ericson2314](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/ericson2314/32/1026_2.png) [@Ericson2314](https://discourse.haskell.org/u/Ericson2314)\
**Post date:** [February 1, 2023, 5:25pm UTC](https://discourse.haskell.org/t/a-different-approach-to-emancipating-base/5695/13 "2023-02-01T17:25:13Z")

</div>

I very much agree with @rae’s analysis. GHC currently thinks of `base` coupling in a fuzzy _undirected_ way, and the key way to tease things apart and simplify them is focus in on direction: Magic stuff like `Int#` where GHC _defines_ and base _uses_ is very different and a stronger coupling that stuff like `Bool` or `Monad` where `base` _defines_ and GHC _uses_.

@adamgundry We can do `ghc-base` first and I think we should. That is simpler. But I think things like this are still very good follow up and can and should do.

@rae to flip the above around, there is also tons of code in `base` that has nothing to do GHC integration — it is simply there because of the “social purpose” of `base` being where standardized things go, as opposed to any technical reason. So the `ghc-base` thing is just a way to start plucking that lowest hanging fruit.