Pre-pre-HFTP: Can the HF support GHCJS or Asterius?

Haskell Foundation
#1

I’m sort of following the process from the proposed https://github.com/haskellfoundation/tech-proposals/blob/feat/new-proposal-process/proposals/PROPOSALS.md as of right now, but bringing up a general area of concern to gather input from different stakeholders and the community.

I am very, very worried about the state of JavaScript and WebASM in Haskell. GHCJS has been a phenomenal tool, and one that I’ve bragged about widely. But the latest reliable version seems (maybe I’m wrong?) to be based on GHC 8.6.5, and work on later GHC versions appears to be slow. Ultimately, it comes down to much of the project having been driven by one person. Asterius is exciting, but ultimately in the same boat, where all development recently halted because the primary maintainer took a hiatus from the project. If the Haskell Foundation (as Andrew hinted at) has some extra funds and is looking for a high-impact way to invest in the community, I wonder what these funds might be able to do to support either or both of GHCJS and Asterius.

Personally, I’d expect the goals of such a project to be (in order):

  1. First, upstreaming any changes to GHC into the mainline compiler, including tests so that they do not bitrot. Also, integrating any tools changes into mainline Haskell tools so that build processes for the web are not separate from other targets.
  2. Second, maintaining focus on compatibility with as much of the Haskell ecosystem (Hackage, widely used libraries, etc.) as possible.
  3. Third, improving both build-time and runtime performance issues.

I don’t know much about the internal structure or work on these projects, so I don’t know to what extent financial investment in the projects could help. But there is reason for hope: IIRC, Well-Typed is already contracting with HF on GHC work, so additional hours there may help with upstreaming compiler changes, and Tweag is already a consulting company and maintains Asterius, so may be interested in a contract for development there that would devote paid hours to the project. Obsidian is another Haskell consulting company that has recently contributed to GHCJS and might be interested in a contract for it.

5 Likes
Beginner Question: Attempting to create copy of GHC
#2

GHCJS is actively worked on by IOHK. Asterius is actively developed by Tweag. Asterius has recently been updated to 8.10. GHCJS is currently being aligned with GHC. This is a lot of work. The linker has been separated out, and GHCJS is starting to use GHCs build system. The intention is to merge GHCJS into GHC. There is also work at Tweag to investigate whether or not a better intermediate language, most likely between STG and Cmm can be found that makes better sense for e.g. compiler like Asterius.

Haskell.nix has very good support for ghcjs, and we intend to also add Asterius to it.

Both projects are far from dead.

8 Likes
#3

Regarding ghcjs; I have recently been making an effort to upgrade it in the nixpkgs and the reflex-platform from 8.6 to 8.10. And I have hit a few bugs (in ghcjs, vector, cabal-install, etc) along the way which is slowing down this progress.

The bugs in the GHCJS were a bit surprising as they should have been discovered easily if the tests of hackage packages were running. (Note: the haskell.nix does not run the tests of deps packages, as I could compile even reflex-dom apps without hitting any failures.)
I hope the addition of GHCJS back to nixpkgs should resolve these problems as well.

For the WebASM, I would like to point out that there exists WebGHC as well, which in my opinion is a better approach to a maintainable cross-compiler. It uses clang to cross compilation the rts, and uses the GHC-unregistered-build’s C code. This approach requires minimal changes to the GHC, and is trivial to upstream.

In fact WebGHC already “works” and is available to use via reflex-platform (to provide TH support) (https://github.com/WebGHC/example-reflex-project), though unfortunately there has not been work done for more than a year.

There are many things which need to be fixed in it, like using a standard C lib like wasi instead of the currently used musl fork.
And the runtime fixes to improve performance and stability (not the Haskell rts, but the jsaddle-warp package which provides the necessary JS to wasm interface)

3 Likes
#4

Thanks for your kind thoughts. Regarding the priorities you listed, the real world priorities are in an opposite direction:

  1. First we need to address build-time and runtime issues, including correctness, performance, etc. Increase the usefulness of the toolchain, so to speak.
  2. Compatability with existing Haskell ecosystem, that’s a part of 1.
  3. Upstreaming bits to GHC HQ with nice tests to prevent bitroting. Yes, having JS/WASM as a native NCG in GHC is a long term vision, but much much polishing and rearchitecting needs to happen before that.

Financial investment is welcome, but it’s really not the bottleneck at this moment.

3 Likes
#5

Let me elaborate on the situation with Asterius. Asterius is currently a project internally funded by Tweag. We believe, like you, that good support for WASM is very important to the future of Haskell. That is why we are investing a rather significant amount of paid engineering time into it.

However, our resources are obviously limited, which is why there can be pauses in Asterius development (as you observed) while we have to focus on other projects. External funding could in fact eliminate these delays and could even accelerate development, as it would allow us to devote more engineering resources to the project.

If the HF would be interested in exploring the option of additional funding for Asterius, we would of course be more than happy to work with the HF on this.

6 Likes
#6

really good news, what about windows support :wink:? I guess it will be solved if merged upstream

#7

I would hope so. The most recent work can be found here: https://github.com/ghcjs/ghcjs/tree/wip-ghc-8.10.7, notably this is WIP.

#8

Thanks, everyone, for the input so far. I’ll commit to writing a more complete proposal as soon as the new proposal process is finalized. So please keep the input coming! To write a good proposal, I’ll need to understand the strands of work that are going on, and what level of effort is involved in each of them. I will do this research myself as much as possible, but would love to hear more here or in individual conversations with anyone involved.

@dfordivam Thanks for mentioning WebGHC. I’ll admit that I’d forgotten all about it. I’ll take the time to look into it, as well, as part of the process.

@emilypi Is this the kind of thing you had in mind for the proposal process? I am not proposing specific changes to the current direction, so much as a set of priorities that the HF might or might not adopt as a direction for investment.

3 Likes
#9

Exactly what I had in mind :slight_smile:

This is good discussion!

#10

Hi all,

I’ve seen a number of people hint that ghcjs and/or asterius are limping along, followed by people from Tweag or IOHK ensuring that they are very much alive.

From my perspective, they also look dead, but I guess the reason for the disconnect is that they’re effectively internal projects that happen to get their code published to github? (I assume?)

One way HF funding could perhaps help is in helping open up the community aspect of these projects. As long as the projects slow down as soon as the internal priorities change at whatever company is providing them, it puts the community in a delicate position.

3 Likes
#11

Maybe (sure!) it is out of topic but i want to mention that a jvm backend for haskell would open lot of doors for its use, specially in enterprisey environments.
Once upont a time threre was a hard fork of ghc (between 7.10.3 and 8.*) which allowed you to compile ghc compliant haskell code to jvm bytecode: eta-lang. Unfortunately it was a private effort and the company behind it ran out of founds so the project is dead. Also quite unfortunately there were not much interactions with the haskell community (and some technical reasons too) and ghc so upstream was never took in account.

With eta you could run wai applications in a j2ee server or create android applications using mainly pure haskell.

Not sure how could we bring back that backend but in my honest opinion it would help to increase haskell adoption dramatically.

3 Likes
#12

Have JVM (and CLR) targets would be very nice. However I do think GHCJS (to some degree) and Eta, do show that anything trying to be a fork of GHC will just eventually have a really hard time. WebGHC does the right thing here and I agree with @dfordivam that it’s a good approach. I do however also see the value of a custom tailored RTS to the platform and a custom codegen, which both Asterius and GHCJS provide. In the end I hope we’ll end up multiple working targets.

IOHK is also working on GHC to make it a properly multi-target capable compiler, which would remove the need for custom cross compilers for each target; and is effectively required if we want to have a pleasant story around plugins.

There is ongoing work to figure out if a better intermediate language could help here.

4 Likes
#13

Are there any papers/blog posts about what such a new intermediate language would look like?

#14

this was written up a while ago, and will likely serve as a basis for discussion.

2 Likes