What’s needed to bootstrap GHC with hugs?

[AntC2]

@nomeata

If someone demonstrates that we can bootstrap some version of GHC using Hugs if only Hugs supported recursive modules,

That’s rather chicken-and-egg. (Having looked at the Hugs code, it’s beyond this “expert” to figure out how to implement support in Hugs.)

Here’s a laborious tactic:

1. Try to load ghc into Hugs. It’ll give you two (more or less at random) modules that are mutually recursive.
2. Trawl through those two to find the datatypes that are mentioned in both and declared in only one.
3. Create an extra module containing those data decls.
4. Comment out the data decls in the clashing modules; put an import in both for the extra module.
5. Rinse and repeat.
6. You can get a large hint by looking inside Hugs.Prelude. But beware that gets maked individually by the Hugs installer – so we have to include make within the cordon sanitaire?

[AntC2]

@nomeata

But it seems that Hugs doesn’t even have boolean guards, right?

Wrong: Hugs has Boolean guards, per the H98 standard. That is, a single Boolean expression following the |. So (f x | p x = y ) is accepted – presuming x :: a; p :: a -> Bool.

[nomeata]

Oh, what am I doing wrong:

Hugs> f x | True = 1
ERROR - Syntax error in input (unexpected `|')

Ah, I can’t write definitions there! My bad.

That’s great, maybe hugs can compile ghc-4.08.2.

I should try to run my hs-all-in-one on ghc-4.08.2 and see how far I get.

[AntC2]

@nomeata

I can’t write definitions there!

Correct: Hugs prompt is an expression evaluator only; can’t declare functions as in GHCi. This works:

Hugs> let f x | True = 1 in f 5
1
Hugs> let { f x | p x = y; p = (> 3); y = 1 } in f 5
1

[atravers]

@nomeata

…isn’t satisfactory unless your chosen compiler for that ol’ warhorse is itself trustworthy.

It is! Just yesterday […]

…by “trustworthy”, I mean something more like the CompCert compiler for C. Since that Guix article briefly mentioned it, Ken Thompsons “trusting trust” attack involves the surreptitious addition of self-propagating code during the compilation process. Using something like CompCert would seem to make such an activity vastly more difficult.

The only other mitigation technique I’m aware of is described by Yrjan Skrimstad in his thesis, which involves the use of multiple compilers…an approach not currently applicable to Haskell, as there’s currently only one widely-supported compiler.

One detail that seems to have been overlooked in this endeavour is the ongoing maintenance required to keep it working - it’s next to useless to have this work only once. I would have thought the rasion d’être of bootstrapping is for someone in the future to be able to repeat the process, in order to assure themselves that Haskell is still “soundly-sourced”…any volunteers for maintaining Hugs again after all these years?

At the risk of being repetitive: I strongly suggest contacting the bootstrappable project about the “acceptability” of using Hugs in this way before investing any serious effort into doing so. Assuming they approve: all the best with this endeavour - it would certainly be an interesting one to watch!

[jaror]

@atravers

I strongly suggest contacting the bootstrappable project about the “acceptability” of using Hugs in this way before investing any serious effort into doing so.

Bootstrappable already lists Hugs as a potential way of bootstrapping GHC:

Bootstrapping GHC with Hugs

The Glasgow Haskell Compiler (GHC) is the most popular Haskell compiler. It is written in Haskell and since the first public release requires GHC to build newer versions of the compiler. It might be possible to build a first GHC from source with the Hugs interpreter and an older version of GHC. This blog post describes a first attempt of this project.

@atravers

One detail that seems to have been overlooked in this endeavour is the ongoing maintenance required to keep it working

They also acknowledge the maintenance cost:

When there is only one implementation

If your compiler targets a language for which no other implementation exists, then please consider maintaining a (minimal) implementation of the language written in a different language. Most likely this implementation exists, or existed at the point the programming language was created. Maintaining this alternate implementation has a cost; however, this cost should be minimal if this alternate implementation is used routinely to build the compiler, and if this implementation is kept simple—it does not need to be optimized.

[atravers]

It might be possible to build a first GHC from source with the Hugs interpreter and an older version of GHC.

…not exactly a resounding endorsement (especially considering how FFI calls work in Hugs), but it is from their website.

Regarding recursive module imports in Hugs:

(Hugs’ support for modules was added long after the original Hugs design - it
was hard getting it to do as much as it does. At one point, I almost had it
supporting mutually recursive modules but I couldn’t quite get all the
dependencies in the typechecker under control so I settled for just
supporting module namespaces and qualifiers.)

Alistair Reid.

The people from the old York Haskell Compiler project could also be helpful here:

• YHC was a GPL-licenced fork of the old NHC98 implementation;

• I believe they also did (or tried to) use Hugs for bootstrapping purposes, having made some small modifications to it for that role.

[AntC2]

@jaror

Bootstrappable already lists Hugs as a way of bootstrapping GHC

But that’s just fairy tales, and we’re going round in circles. That ref actually doesn’t describe using Hugs for bootstrapping, it tries using nhc hosted inside Hugs, and @nomeata already ref’d it in the o.p.

If Alistair Reid couldn’t get mutually recursive modules going (and that was in a late version of Hugs), there’s no hope for me.

[hellwolf]

Fascinating topic about bootstrapping, I learned a lot, thanks.

I have an outside-of-box thought, please correct me if I am missing the point:

1. compile one of the early version of ghc to its early version of ghc-core or c-- output call it ghc-redux source.
2. a bootstrappble implementation of compiler core or c-- output: ghc-core-redux or cminusminus-redux
3. Bootstrap by compiling (1) using (2)

The assumptions are

a. by having an open sourced intermediary output of an old version or ghc, human may still be able to verify the trust worthiness of it. It is more so with ghc-core than c-- outputs considering the difficulties in reading the latter.
b. (2) is much easier done than fixing hugs.

[jaror]

@AntC2

But that’s just fairy tales, and we’re going round in circles. That ref actually doesn’t describe using Hugs for bootstrapping, it tries using nhc hosted inside Hugs, and @nomeata already ref’d it in the o.p.

I mean that there’s no need to contact bootstrappable about the acceptability of using Hugs if they already point to it as a potential approach on their website. I’ve edited my comment to make it clearer that it is a direct reply to the comment above it and not to the whole topic.

@hellwolf

compile one of the early version of ghc to its early version of ghc-core or c-- output call it ghc-redux source.

The main reason for bootstrapping is for trust. If you use untrusted binaries to convert an early version of GHC to a non-human readable/understandable format like GHC Core or C-- then I think there is little gain.

[thma]

Hi Joachim,

over the weekend I watched Ben Lynn’s ZuriHac23 presentation (Ben Lynn on "MacGyver's Haskell Compiler" @ZuriHac2023 - YouTube) of his efforts to bootstrap a Haskell compiler. He starts with a tiny combinator reduction VM written in some 600 lines of C.

This VM also includes a first incarnation of a “Proto-Haskell” compiler (written in som 55 lines of “Proto-Haskell” and also contained as about 110 lines of human readable “combinator bytecode”. (compiler/vm.c at master · blynn/compiler · GitHub)

Starting from that he builds up a chain of self-hosting compilers that provide increasingly richer and more complete versions of Haskell compilers. Including a GHCi like Repl…

I don’t know whether his final compiler would be a better starting point for bootrapping GHC than Hugs from a practical perspective.
But as Ben’s approach preserves the whole bootstrapping chain, I think this would be an ideal candidate for a bootstrapping exercise in the sense of bootstrapple.org.

In his talk Ben was asked if his compiler could be used for bootstrapping GHC. He mentioned the issue of recursive module imports which are as of now not supported by his compiler. But he stated that is would be a feasible task.

[nomeata]

That was indeed a fun talk!

I am doubtful if writing a compiler from scratch will be an easier path. The surface syntax of Haskell alone is just very big, but mostly boring to implement, and most of it already exists in Hugs. So – unless you have a reason to write that new compiler anyways – the question is whether it is less (or more fun!) work to implement a lot in Haskell, or to implement a few remaining bits in C.

Now, if someone would anyways pursue the goal of writing an independent compiler for other reasons, and if one would pay a bit attention to keeping it bootstrappable, and that eventually is good enough to bootstrap GHC, maybe that would be most realistic path forwards?.

[atravers]

@nomeata

The surface syntax of Haskell alone is just very big, but mostly boring to implement […]

…so to make that task more interesting, perhaps Ben Lynn’s approach could be used to bootstrap e.g. an Agda compiler: a new interpreter for Glasgow Haskell could then be written in Agda for running GHC.

[barracuda156]

We have both nhc98 and hugs98 in Macports building from source (a caveat: very new macOS are not supported for those), could try if those are of any use?

I am very much interested in the topic, since I wanna get modern GHC working on PowerPC. Which leads to the same issue: the last version I got to work on PowerPC was 7.6.3, which is too old to build anything, and bootstrapping from source is currently impossible.

[jaror]

@barracuda156

the last version I got to work on PowerPC was 7.6.3, which is too old to build anything

You can use it to build newer versions of GHC. Perhaps you need a few intermediate steps to get to the latest versions of GHC, but to me that seems far easier than any of the other options.

[barracuda156]

I tried, but apparently there was some bug introduced after 7.6 and fixed around 8.x (I had a discussion with upstream, but relying on memory now). So I could not get 7.7 or 7.8 to build, and then either 7.10 or 7.12 already wanted a newer version.

Alternative may be cross-compiling from x86, but that’s a separate hell

[atravers]

There was a recent update-of-sorts regarding bootstrapping for matters of provenance:

Interface files and recursive modules with nhc98

I am still trying to get ghc-0.29 compiling on nhc98, but I’m stuck on this interface file that I can’t seem to get right. What might be the problem? [image] I am writing the interface file by hand because these modules are recursive. The example in the screenshot was simplified before screenshotting. Simpler examples work fine, where only functions are mutually recursive, but not data types. I still don’t understand what the NEED annotation does. Should be some dependency declaration, but …

but I’ve also been informed that it’s somewhat different from bootstrapping for cross-compilation.

---

[…] apparently there was some bug introduced after 7.6 and fixed around 8.x […] So I could not get 7.7 or 7.8 to build, and then either 7.10 or 7.12 already wanted a newer version.

GHC still has the old -fvia-c option (strictly for this purpose only, not intended for regular use) but it’s been a while since I’ve seen anyone try using it - I suggest contacting the GHC devs before attempting to use it…

[nomeata]

@barracuda156

I am very much interested in the topic, since I wanna get modern GHC working on PowerPC. Which leads to the same issue: the last version I got to work on PowerPC was 7.6.3, which is too old to build anything, and bootstrapping from source is currently impossible.

At the moment I’d say that all the attempts at bootstrapping in the sense of https://bootstrappable.org/ (i.e. absolutely not using any existing non-source artifacts like binaries or precompiled C files) are strictly less promising than all the alternatives when you goal is to “just” port to a new architecture.

It seems that Debian has GHC builds for powerpc 64: Debian -- Details of package ghc in sid. Does that help you?

Else using -fvia-C or crosscompiling are probably more plausible.

[linj]

Can MicroHs replace Hugs in the bootstrap chain of GHC?
Notably, MicroHs allows mutually recursive modules and seems bootstrapable according to their readme.

[nomeata]

Give it a try! I expect two problems

• MicroHS isn’t bootstrappable itself (in the sense of https://bootstrappable.org/) – it builds off he pre-generated C files in generated. For proper bootstrapping with MicroHS, you first have to find out how to bootstrap MicroHS. Maybe Hugs can do that?

• Then you’ll run into a bunch of modules in the GHC source code that use GHC-specific low-level features (primops, unboxed data types). Here the hope would be that they can be replaced with slow, but pure equivalents, but nobody has achieved that yet, I think.