GHCup state report and 2023 resolutions

Since GHCup is becoming a central tool of development, I want to share some thoughts about the current state, recent work, certain problems and upcoming ideas.

Platform support and the move GitHub

GHCup supports all major platforms: Linux, windows, mac and FreeBSD. Since GHC has ceased to support FreeBSD actively, it has been problematic to keep up with FreeBSD support through GitLab CI.

In an effort to improve this situation, but also stop relying on gitlab due to other issues, I decided to move back to GitHub. @angerman has been providing self-hosted runners for Mac M1 and Linux aarch64.

Weā€™re using Cirrus CI for FreeBSD support, which works flawlessly and apparently also supports OpenBSD and NetBSD: Add OpenBSD CI job by Bodigrim Ā· Pull Request #262 Ā· haskell/unix Ā· GitHub

If anyone is interested in providing unofficial bindists for GHC for minor platforms: Iā€™m willing to add them to GHCup.

As a side-note: this effort may also be relevant to other projects under the GitHub haskell umbrella (like HLS or core libraries), since weā€™re ultimately planning to increase platform CI coverage for all of them.

This also mean GitHub is now the primary development platform and issues and pull requests go here: https://github.com/haskell/ghcup-hs

Adding more binaries (and CI caching)?

During the move to GitHub I started using cabal-cache with BackBlaze S3 storage to speed up the CI roundtrip times. Although there are some difficulties, it worked out very well in the end. The main issue was missing binaries for esoteric platforms like armv7. Iā€™ve now built binaries for all of them: https://downloads.haskell.org/~ghcup/unofficial-bindists/cabal-cache/1.0.5.1/

@JohnKy has been providing some excellent support. So Iā€™ve been thinking about the idea to add cabal-cache to ghcup, primarily to make it more useful and seamless in CI settings.

Now, cabal-cache can also work without S3 and just sync to a file location, which can than be backed up by other means. In general, the GitHub cache is just too small for this in my opinion and S3 storage is fairly cheap these days.

Iā€™m still thinking about this and whether it needs more thought on the cli interface, which is already clogged.

More TUIs/GUIs

GHCup TUI is implemented in brick, which does not support windows (yet). I have finished a prototype of a windows TUI regardless: Ansi tui by hasufell Ā· Pull Request #702 Ā· haskell/ghcup-hs Ā· GitHub

It is based on ansi-terminal-game and was pretty rough to implement (especially scrolling, lists and resizing behavior), because a lot of the abstractions that exist in brick (which is used for linux TUI) are missing.

It works, but is a fair amount slower/less responsive than brick.

@Kleidukos has been silently working on a GTK+ GUI for GHCup, which could prove as an option for Windows too: HĆ©cate Moonlight: "Those who prayed for my downfall haven't prayed Hā€¦" - Merveilles
The main challenge I see here is how to ship it (linking GTK+ statically?)

Iā€™m inclined to drop the idea of a windows TUI if the GTK+ GUI proves to be a viable path.

More contributors (call for help)

The Haskell Foundation has been engaging in looking for contributors to GHCup, since Iā€™m still mostly the only one actively working on it and my possible time commitment isnā€™t what it used to be due to health reasons and life reasons. This is a sub-optimal situation since more than half of the community uses GHCup and itā€™s pre-installed on GitHubs runner images. Itā€™s great to know that doing even smaller mistakes sometimes makes GitHub staff ping me in private.

More eyes, better testing and CI is the way forward here.

Although Developer Guide - GHCup is already a starting point, Iā€™m also planning to maintain a proper CONTRIBUTING.md document in the repo with current information of tickets that can be picked up, what the main goals are etc.

Goals for 2023

  1. improve platform coverage
  2. improve windows support (there are still a lot of rough edges and not enough people are knowledgeable or active here)
  3. improve UI experience (GTK+ GUI or feature requests like compiling HLS from source via TUI)
  4. improve testing (itā€™s really rudimentary and a little embarrassing)
  5. get more contributors (weā€™re on github nowā€¦ cool, right?)
  6. documentation, documentation, documentation (e.g. the new stack integration via hooks is maybe not that widely known yet and needs to be promoted more aggressively)

But the most important goal for 2023 is another: improve the holistic tooling experince.
Many users have reported sadness due to GHC, ghcup and HLS not really working in lock-step. The latest cause was the bugfix 9.2.5 release, which still does not have HLS binaries. This is also largely a communication and coordination issue that needs to be resolved between those 3 teams. There are some technical reasons too, but it is my opinion that we need much better communication. The Haskell Foundation is aware of this and actively trying to support these concerns.

Final reflections

I think overall, we have been doing well, given that GHCup has taken the lead of installation methods in the latest Haskell 2022 survey. I thank all contributors, supporters, backers, the GHC team, the Haskell Foundation, the Stack team, the Cabal team, the HLS team and of course the community for their trust.

41 Likes

Thanks for your work and efforts, @hasufell! You are making Haskell world a better place to live in.

7 Likes

Thanks for this awesome tool! Iā€™d love to contibute somehowā€¦ probably with some easy/documentation issues since many of the goals seem related with CI or platform support which is a little bit of a headache to me.

2 Likes

There are a lot of tickets that just need some investigation/discussion or where the design is not clear yet, e.g.:

Then thereā€™s the Haskell Foundation error project, which is quite exciting: Integration with errors.haskell.org? Ā· Issue #434 Ā· haskell/ghcup-hs Ā· GitHub

Documentation tickets can be filtered: Issues Ā· haskell/ghcup-hs Ā· GitHub

A good way to look for coding tickets is to filter by ā€œtype: improvementā€ label: Issues Ā· haskell/ghcup-hs Ā· GitHub

But the main thing in terms of coding that I need help with right now is indeed testing. Our unit tests are very small. Integration tests are a mess of a shell script. Iā€™ve outlined the steps here: Write proper integration tests Ā· Issue #64 Ā· haskell/ghcup-hs Ā· GitHub

Another related ticket is implementing Dhall support for the ghcup metadata for which some code already exists: Use Dhall for configuration Ā· Issue #60 Ā· haskell/ghcup-hs Ā· GitHub

If youā€™re good at GTK+, consider contacting @Kleidukos.

3 Likes

Iā€™d like to say a big holiday thank-you to you, Julian. You have made (and continue to make) a huge contribution to our community with ghcup. I am super-grateful to you, and I know that feeling is widely shared.

Enjoy the holiday break.

Simon

15 Likes

Thank you, Julian, for the great work on ghcup.
Iā€™m using it quite often to build latest hls from source and to manage multiple ghc installations and it works great.
Iā€™m gonna double my monthly contribution through ghcup - Open Collective
I know itā€™s not much, but maybe it will inspire few other happy user(s) to contribute as well :smiley:

4 Likes

Thanks all for the support and kind words! Iā€™m happy to see youā€™re enjoying ghcup as well @simonpj

Wrt donations: itā€™s very likely they will be used in the near future to boost CI, so will have a real impact.

7 Likes