[Dream] Toward another compiler?

Learn
#1

Disclaimers: I’m very much not an experienced compiler engineer, I’m not funded for anything beyond weekend-scale efforts, and I’m very explicit in my desire to not pull any effort off of maintaining, improving, and evolving GHC.

With those bits in mind, this is perhaps most thoroughly directed at those who are not currently contributing to GHC for whatever reason, those who are but might do things differently if they were starting from a cleaner slate (and would be willing to expound on those things), and those who hold strong opinions (possibly even well-founded ones) on what a Haskell compiler could be doing to advance the standardization of the language without the very real maintenance and support burden that an industrial-strength, battle-tested compiler like GHC must grapple with to consider such a thing.

I want to start a conversation with such people about how to build another open source compiler for Haskell. We have the great fortune that the bootstrapping problem is behind us, and we have the right venerable GHC, compiler of a thousand compilers, to get over that hump.

That’s all there really is to the pitch, for the moment. I would like to collect some feedback, do some thinking, and figure out what a plan for such a project would even look like.

For interested parties (implementation, orchestration, or peanut gallery), I guess I’d be interested to understand what a new Haskell compiler would mean to you, what avenues it would open that might otherwise be difficult to pursue, and what expectations of compatibility with existing Haskell-by-the-report could or should exist. This last point I recognize as a bit vexing, since it’s not technically a Haskell compiler if it doesn’t implement Haskell-by-the-report, but dream with me a little and envision the reference implementation for Haskell’.

It’s also potentially valuable to say “there is simply nothing to be gained from this endeavor” if you’ve a strong point to make there.

I have no performance goals and no timetable, so unreasonable dreams like “TRex 2” or “first class Backpack/other module system” or what have you could be discussed. I like algebraic effects and dependent types, myself, and I’d be interested to see where those could be pushed with less critically important architecture around. Could we solve “the IO problem” differently? Could Haskell-the-language evolve from declarative to denotative? Could we start from FunctorOf? There’s been some previous discussion on this board of fitting GRIN or HVM onto GHC’s pipeline (there’s existing work on the former and some gesticulation on the latter); would that be easier with a less practical and established compiler?

NB: I’m less interested in hearing the nth argument for “you should just work on GHC instead”, since this is particularly a thread to centralize and chew on some other points in the space, and how I choose to burn the back 9 of my creative life is likely to include more contributions to GHC, anyway. You’re welcome to write the argument in this thread, of course.

4 Likes
What’s needed to bootstrap GHC with hugs?
Introducing NeoHaskell: A beacon of joy in a greyed tech world
#2

What do you mean? How do you bootstrap GHC without binary sources?

I’d be interested to understand what a new Haskell compiler would mean

I would hope that a new Haskell compiler would be written in an already bootstrapped language, and if it were able to compile a version of GHC that would transitively compile a modern GHC, we’d have the bootstrapping problem behind us. I would hope that it would be focused on this goal so that it wouldn’t amass extensions and become harder to understand.

If you want an evolution of Haskell, I personally regard Idris as functional and I don’t see enough laziness in my day-to-day work for me to consider it an intrinsic part of Haskell. It’s not like Idris doesn’t have laziness, it just doesn’t have it by default.

3 Likes
#3

There was an amazing talk by Ben Lynn at ZuriHac 2023. Hopefully the recordings will be published soon. But the gist is that he has developed a relatively simple compiler for an impressive subset of Haskell. He can even bootstrap his compiler from C.

10 Likes
#4

My alternative dream would be preserving and enhancing further Haskell type system while using lisp syntax instead (not sure what’s the best way to express strongly typed language in lisp, to be fair). This way we have a easier meta programming foundation, and say goodbye to the dreaded TH. Shall we call it Hakellisp or Halisp? And hopefully it would be trivial to convert current Haskell syntax to that Halisp syntax…

Sorry if I digress, but since it’s tagged DREAM I felt whynot.

1 Like
#5

It’s called Hackett.

But the real solution is not LISP, but rather dependent types: [2209.09729] Staged Compilation with Two-Level Type Theory (to be fair staging is only a subset of metaprogramming).

9 Likes
#6

Right. I guess I should have emphasized the surface syntax conversaions between Haskell and the new language, while keeping the inner system staying the same. This way we wouldn’t need to bootstrap a new ecosystem. ← I guess I meant a transpiler.

#7

But the real solution is […] dependent types […]

Is that really the case?

#8

If I try to infer OP’s intent: it is the desire to contribute to Haskell ecosystem especially to have another compiler to complement what GHC does to the ecosystem; but at the same time, the fear of the difficulties and slowness for its a old and giant project?

I know I should not provide the n+1th argument for why we should contribute to GHC, but I just wanted to see if I got this correctly.

#9

Yeah. The really short version is that I’m hopeful that another actively developed compiler could help resolve the tension of standardization in the same way that, say, the Dhall standard doesn’t change until multiple implementations implement this or that proposal.

#10

That’s awesome. I’ll look forward to the recording!

1 Like
#11

I really, really hope this is not the case. DH has sucked all of the air out of the room and delivered … nothing. In fact, I now realize GHC has been preparing for DH since about 2016 and delivering nothing.

So if there’s to be another Haskell, I want it to take the language as at 2016 (Pattern Synonyms) and continue all the half-finished other stuff – like FunDeps with Overlapping, a records system, Datatype contexts done per GHC before ~1999 (to get an industrial-strength approach in the same space as GADTs), …

I see no merit in a competing compiler merely mimicking the language GHC supports, whatever its underlying technology.

2 Likes
Introducing NeoHaskell: A beacon of joy in a greyed tech world
#12

…that depends on what is meant by “denotative” here:

  • more “mathematical”: then according to this article at least, it seems laziness ought to be a second-class feature - Haskell should be eager by default.

  • no effects: I frequently see articles which refer to non-termination as being an (undesirable) effect - that being the case, a denotative Haskell would also be a total Haskell.
    (which then also has consequences for dealing with I/O…)

Having referred to the monadic IO type as being “lawless”, Ben Lynn’s Zurihac 2023 presentation is also of interest to me…

#13

I’ve given some thought to the questions you raise. I’ve been slowly developing another parser for Haskell with GHC extensions, and it’s at a point where I have to consider whether to extend it toward a proper compiler or alternatively to direct my efforts toward tools for tinkering with the source code - pretty-printer, linter, refactoring etc.

For anybody who seriously wants to develop an alternative Haskell compiler, the first step should probably be to decide which market niche they’d target that’s not well served by GHC. Here are some ideas:

  • smaller and simpler compiler to tinker with
  • new language extensions not likely to to be supported by GHC for some reason
  • better JavaScript back-end, with full Haskell 2010 compatibility but no GHC RTS support
  • JVM or CLR interoperability – though the fate of previous efforts does not inspire confidence
  • completely new runtime supporting massively parallel or distributed execution
  • bottom-up design for integration with novel development tools

Unless you have no higher goal than to have fun – and I’m in full agreement compiler developent can be fun – you should really come up with a goal that’s not attainable by tweaking GHC. Having two viable Haskell compilers would hopefully have a pleasant effect of finally providing enough reason to update the Haskell Report, but first you need to have another viable compiler with actual users.

6 Likes
#14

I think this is probably the most immediately compelling niche (at least for my cross-section of interests and the state of the art), though a more hackable Haskell compiler (i.e., the first bullet) could pay dividends in lots of directions.

3 Likes
#15

You would have to say first, what Haskell means for you. Haskell 98? 2011? 2021? 2011 plus your favourite extensions? Whose favourite extensions? Choose wisely so you will be able to use some libraries in order to bootstrap your compiler.

I think your endeavour stands and falls with a good reason or motivation to develop a compiler.

Do you want to give the Haskell ecosystem an alternative to GHC? Good luck, you will have to allocate developer decades of work for that. I don’t believe the need for that is high enough to make such an investment. GHC has no problems that are that pressing to develop an alternative.

Do you want an academic project? Fair enough, should be possible, there are certainly many ideas that could be explored in a small, newer compiler. If you have a great idea, it might be worth the cost. But many great ideas have already taken that route and established a Haskell dialect exploring it. Better Javascript support? Purescript. Logic programming? Curry. Dependent types? Agda, Idris. The list goes on. You would have to have a great idea like “what if Haskell could do X”, and then justify why not to implement it as a GHC language plugin instead in order to motivate people to do this.

2 Likes
#16

…and now someone seems to have done so: thank you, @turion.

A proverbial “wildcard”: program verification similar to what the PureCake Haskell-subset language implementation uses (by being based on CakeML)…

#17

I haven’t done so at all, I’m wondering where the misunderstanding lies.

#18

@jhenahan @blamario
Regarding a distributed Haskell runtime, I have a wacky idea, but I’m still in the stages of making a toy example to prove whether or not it is brilliance or idiocy, and I hesitate to make bold claims without evidence to back them up. Suffice to say that distributed Haskell is of great interest to me, and I have a teeny lambda calculus with a module system that I’m trying to apply said wacky idea to.

@jaror
Ben Lynn’s Compiler Quest series has held a permanent place in my bookmarks for several years now, his focus on bootstrapping with minimal information-complexity reminds me heavily of efforts like the java4k game programming contest or .kkrieger. Naturally, I am extremely interested in hearing what he had to say at ZuriHac.

3 Likes
#19

…or to use the terminology from Claus Reinke’s thesis, you already have “functions” and “frames” - perhaps his work could be of use to you.

1 Like
#20

Ooh, I love a good solid paper - this one is quite long. It’ll take me a bit to go through it in detail, of course, but I gave it a good enough skim to know it not only be helpful, but also to know that I have I have my own answers to the two major problems posed - external interaction, and managing complexity - and so it will be interesting to see how they mesh with the results of the paper.