The evolution of GHC

[ketzacoatl]

@atravers

So the options are:

• several “small” interruptions, one for each small change;
• or one “large” interruption as a result of on big breaking change.

Either way, people are being interrupted while attempting to do what interests them, which is probably why they chose to use Haskell to begin with:

That’s not quite what I was describing.

The smaller changes are not each interruptions in their own - the large interruption is a result of making too much change at once, and in a way that introduces the interruption. Conversely, the smaller changes provide a graceful experience by bridging across the gaps so there are not interruptions. Using the database-backed application as an example, both the app and the database schema support two versions of each other in a way such that there is not a version of the app running which does not support the current state of the database, and the database is not updated in a way that breaks one of the two versions that may be running.

[atravers]

@ketzacoatl

Using the database-backed application as an example, both the app and the database schema support two versions of each other in a way such that there is not a version of the app running which does not support the current state of the database, and the database is not updated in a way that breaks one of the two versions that may be running

…which I thought Hackage already enables - the previous version of a package is retained, not deleted, whenever a new version is uploaded. That being the case, users of that package can then remain with the superseded version until such time they’re ready to update their dependencies in order to switch to the latest version.

Again, it could be just a lack of imagination on my part: I’m not quite sure what feature/s Hackage needs to make that process “graceful”…

[rubenmoor]

There is something noteworthy about GHCs language extensions. I was never bothered by adding one when it seemed somewhat useful. By no means to I understand the full implications of those (they also differ substantially in theoretical complexity).

The advances towards simple Haskell do make sense to me – thinking of bigger projects for bigger teams --, the concrete idea of going back to Haskell 2010 or Haskell 98 not so much.

Some sort of consensus on a set of language extensions that seems proven at least reduces potential confusion for a newcomer and helps creating some sort of standard.

What’s missing in the picture is a bold design choice: “This is what we think you should be able to do
with our compiler. This is feature is meant for use in production, this is research.”

“Duh”, you’ll say, “no-one can make an informed decision on that!” This is why GHC keeps it flexible in both directions (you can activate any extension, but none is activated by default).

It might be that any bold design decision is better than none, at the moment, even though no-one knows what the “optimal” direction of GHC really is. And really, the language extensions are just one aspect of that.

[ketzacoatl]

While the broader package ecosystem is relevant, and any solution needs to consider that aspect of our experience, I think the problems and challenges we’ve been discussing primarily start with, and are more relevant to, GHC and base, then core libraries, then the rest of the hackage ecosystem (in that order).

[atravers]

@ketzacoatl

I think the problems and challenges we’ve been discussing primarily start with, and are more relevant to, GHC and base, then core libraries, then the rest of the hackage ecosystem (in that order).

It appears that the sources for both ghc and base are already in Hackage. If that’s the case, then given my previous point regarding Hackage’s retention of earlier package versions, what is stopping us from making full use of that capacity to achieve the graceful migration towards a “decentralised” base?

I think we need the help of an Hackage specialist…

[atravers]

@rubenmoor

What’s missing in the picture is a bold design choice […]

Would this be bold enough?

What Would It Take To Make "Elm Embedded In Haskell" Available In GHC?

Haskell lacks what Elm has and Elm lacks what Haskell has. Both are arguably “best in class”, amazing technologies: Elm for the frontend and Haskell for the backend. If they were a little more unified, Haskell and Elm together could offer a remarkably strong value proposition. If some differences were alleviated they could be a uniquely powerful combination. Haskell and Elm are just different enough to derail a powerful synergy that could strongly motivate a much wider adoption of both techn…

…that discussion concluded with me suggesting that it probably probably would be easier to make Elm non-strict by default rather than “embedding” it in Haskell.

So here’s another [attempt at a] bold idea: let’s take an existing statically-typed strict language and make it non-strict:

• if it fails, the products of the effort can be studied and the results used by other projects;
• but if it succeeds, there’s the potential to replicate the success of Haskell through the formation of a whole new community, as interested users of that existing language exercise their curiosity.

Before Haskell, there used to be a language called Lazy ML - anyone for Lazy Standard ML '97 Non-Strict ML 2027 …?

[ketzacoatl]

I am not experienced enough to say this with confidence, but AFAICT, the problem is with how we roll out change in the GHC/base source, not the hosting or availability of different versions of those packages.

[hasufell]

@simonpj

In GHC’s case, “don’t break code” is clearly a desirable goal, but we need ways to reconcile that goal with maintaining dynamism.

There are other languages/compilers that do not break code and would rather create a new Language instead of breaking existing code. An Example is “Go”.

So I think that goal is absolutely actionable. But that would boil down to maintaining a stable fork of GHC (e.g. 8.10.7) and observe its popularity.

[atravers]

@hasufell

But that would boil down to maintaining a stable fork of GHC (e.g. 8.10.7) […]

A alternate name e.g. ghc-2021-08.r0 would make it clear that this is a stable fork (snapshot?), and as bugs are found and removed, only the “release number” .rN would be changed.

If this works, the same could be done for the last of the ghc-9 series (upon the arrival of ghc-10); only then would ghc-2021-08 stop receiving bug fixes - that activity switches to the renamed “end-version” of ghc-9.

All going smoothly (!!!), this gives people the choice of:

• relative stability with the renamed, maintenance-only end-version of the previous release series (currently ghc-2021-08 a.k.a ghc-8.10.7);
• or the current release series (ghc-9.2.1 or younger if needed: ghc-HEAD), for adding and testing language and other extensions.

Of course, this presumes there are enough Haskellers left to maintain both versions of GHC. As for having enough resources, someone with previous experience in the upkeep of GHC can let all of us know if any of this is actually possible…

[Ericson2314]

Of course, this presumes there are enough Haskellers left to maintain both versions of GHC. As for having enough resources, someone with previous experience in the upkeep of GHC can let all of us know if any of this is actually possible…

Yes for me it’s really key to make things more efficient so we can do more with the limited people we do have. And extremely modular standard library implementation with minimal coupling to the compiler is not something other languages/implementations have really nailed yet, and could be a big advantage to allow us to do more with less.

In generally I am trying to make it more efficient for existing contributors, and also entice knew ones as modularity also lowers the barrier to entry, as @doyougnu pointed out in the other thread. These are very nice multipliers. We should pluck just enough low hanging end-user-benefit fruit to where we don’t feel guilty working on them :), and then switch to working on them.

[atravers]

@Ericson2314

An extremely modular standard library implementation with minimal coupling to the compiler is not something other languages/implementations have really nailed yet, and could be a big advantage to allow us to do more with less.

Alright, here’s a wild idea: instead of trying to split up base, just rename that giant pile to reflect its seemingly-intractable entanglement with GHC e.g. ghc-base. Then you can introduce an all-new std-base, learning from the mistakes experience of writing ghc-base.

Over time, more code will appear in std-base which would make code redundant in ghc-base. That old code can then be removed, resulting in a smaller and smaller ghc-base.

Assuming ghc-base is sufficiently reduced, GHC itself can be switched over to std-base. The code in ghc-base not made redundant by the switchover would then be added to GHC’s code base. We can then put ghc-base out of our collective misery…

So no splitting, just a slow process of attrition in the old ghc-base as it is slowly superseded by the new std-base. Instead of trying to carve up the pile that is currently base, people can start writing clean new code for its successor.

[Ericson2314]

That was the first step of the plan in the original thread. The only trick is making std-base work with multiple ghc-base without erring to much on the side of stuff being in ghc-base

[ketzacoatl]

I’m so glad we’ve come full-circle and are back at that original topic!

[atravers]

Me:

So no splitting […]

---

@Ericson2314

That was the first step of the plan in the original thread.

o_O …it was - in which thread?

• From Pre-Pre-HFTP: Decoupling base and GHC

  1. Cleave base in two

  […]

  2. Prove the split works at all

  […]

• If it’s this thread: Modularizing GHC, do we want an HFTP? …it isn’t immediately obvious, at least at a glance.

Is there (yet!) another Discourse thread somewhere, about this topic, that I’ve failed to notice? It could have saved all of us going on this long, winding trip back to nowhere:

@ketzacoatl

I’m so glad we’ve come full-circle and are back at that original topic!

[Ericson2314]

Yes it might not be so obvious form the headings, but the idea is that the first “cleave in two” might not really cutting along the right “plane”, and then we move things across the boundary after.

“cleaving” is a fun dramatic word, but it’s kinda the wrong metaphor, come to think of it. Really, it’s more like pulling two sides a part, and letting it rip “just happen” wherever the bonds are weakest. “Rip base in two” would be better, as paper rips are expected to by messy and not necessarily along the path desired a priori.

[atravers]

If it was just a matter of “cleaving splitting tearing slicing cutting dividing chopping ripping up base into two or more pieces”:

• It should have been attempted at least once by now;
• It would have been easier to do back when base was smaller e.g. back in 2010.

It seems to me that the simplest option is to just move the morass of code in base to a package more ghc-centric, then starting afresh, either in an empty base or an all-new package under a different name. This approach provides the luxury of ignorance: you can just start writing new code almost immediately instead of trying to pick a path of least resistance.

[AntC2]

@atravers

long, winding trip back to nowhere:

@ketzacoatl

I’m so glad we’ve come full-circle and are back at that original topic!

If you’d wanted all along to talk only about base libs, your topic title should say so. “The evolution of GHC” seems like it’s throwing in the kitchen sink. I only got involved after the discussion moved to FP in-general and history of Haskell in-general.

I suggest you start a new topic with a carefully circumscribed title. This one has gotten beyond useful.

[jaror]

I had to come up with a title when splitting off this thread. I based the current title of the quotes in the first post which discuss the speed at which GHC should evolve. I don’t know if @atravers can actually change it.

[Ericson2314]

@atravers yes, base reexports ghc-base, and we move over to base proper just what is convenient.

The only difference is I think @Kleidukos’s heuristic of "just non - GHC.* modules is a better first shot at doing the rip, and I emphasize moving code not coppying code.

All that, though, just reflects on the first few hours of the attempt :). After that, I think it’s exactly the same.

[atravers]

@AntC2

I suggest you start a new topic with a carefully circumscribed title. This one has gotten beyond useful.

…better yet: since we’re back on this topic again, instead of trying to find consensus on how to balance stability with evolution in Haskell (and therefore, for the moment, GHC), let’s now all head back to “Pre-Pre-HFTP: Decoupling base and GHC”.

See you there!