DevOps Weekly Log, 2024-01-17

Haskell Foundation
#1

Hello again! In this log I also want to put out two calls-to-action.

  1. Although the hpc library ships with GHC, it effectively has no maintainer. The paid maintainers of GHC don’t have enough of a time budget to give it attention, since they are fully occupied with GHC itself. That means issues like #24330: hpc complains that record field referencing an existential is never executed · Issues · Glasgow Haskell Compiler / GHC · GitLab don’t have a high chance of getting resolved. I believe that, similar to hpc / hpc-bin · GitLab, there is an opportunity for an interested party to show some leadership and pitch in here. I know the power structures at this strata of Haskell are a bit cloudy, so I’m explicitly pointing out that you, yes you, could be the difference for HPC :slight_smile:
  2. Same, but for ghc-pkg. This is a very high impact tool, used by every Haskeller every day (under the hood). Potential improvements like !11142: Fix #16324 More efficiently bulk unregister with ghc-pkg · Merge requests · Glasgow Haskell Compiler / GHC · GitLab will be very hard to get merged unless/until somebody decides to get involved and build the baseline expertise needed to provide useful reviews.

If you’re looking for inspiration, I think that hpc-bin (linked above), cabal, and stack have all had a lot of people step up in the last couple years and start building the expertise that allows them to be leaders and facilitate more contributions. It is possible! (I might say something sentimental about how Haskell is a language that makes it possible to get involved in huge legacy projects without despair thanks to good abstractions that come from pure functional programming, but that wouldn’t be very scientific of me…)

Ok, on with the log!

I’ll try to keep it short, although I’ve been hard at work. I am still working on taking ownership of Stackage.org. (Another case where someone new is becoming the expert, although this one is very difference since it’s official Haskell Foundation work.) There have been a few hiccups related to generating and displaying the haddocks for snapshot packages. I have had to extend three different processes to handle the new haddock storage bucket, and I had to work around one Cloudflare bug. As always, I think I’m “almost done” now. :grinning_face_with_smiling_eyes:

In the last week, I also performed a critical security upgrade to gitlab.haskell.org and (unrelatedly) dealt with another one of its disk space outages. Proactively preventing these things is on my todo list…

Finally, here’s the issue summary from the most recent GHC issue triage!

GHC issue triage notes
5 Likes
#2

…and the associated !10987: Make pseq robust by using seq# · Merge requests · Glasgow Haskell Compiler / GHC · GitLab ? It’s a “recurring theme” :

In addition:

1 Like
Haskell Foundation December 2023|January 2024 Update
#3

How so? hpc / hpc · GitLab seems active to me. CC @DavidB

1 Like
#4

Yup, part-time hpc maintainer here :slight_smile: I usually have enough time to make new releases and respond to urgent stuff when bounds have to be bumped or something similar comes up. I have also tried to improve the quality of the code base, improve test coverage and do some other small improvements. I have recently added a few RTS options to improve how HPC can be configured on the command line, and have one ongoing GHC proposal to add another RTS option. I think the library is in good shape, and doesn’t need a lot of maintenance. The problem with the issue that you cite is unrelated to the hpc library and concerns the code instrumentation pass inside GHC which adds tick boxes around expressions. Unfortunately I am super busy at the moment, since I am about to hand in my PhD and need to proofread my last revisions. I therefore currently don’t have the time to hunt down bugs in the hpc instrumentation pass. But I would also welcome any other person who would like to work with me on hpc :+1:

Edit: Also Rodrigo @romes has been working on cabal, improving the support for hpc there.

5 Likes
#5

Ah that’s great to know! I will pass that info back to the ghc team so they can also know. :joy: I knew you had split out hpc-bin, but based on the discussion during triage I had assumed the library itself was still under-maintained.

#6

The hpc library itself really doesn’t do anything interesting. It just provides the source of truth for the tix and mix file formats, which are unlikely to change, and helper functions to serialize and deserialize tix and mix files. The interesting part which probably could use a bit of love is the code instrumentation pass in compiler/GHC/HsToCore/Ticks.hs · master · Glasgow Haskell Compiler / GHC · GitLab I think that the code is correct for Haskell98, but for some advanced features it might generate too many or too few tick boxes. There aren’t really any tests in the testsuite which check this. #24330: hpc complains that record field referencing an existential is never executed · Issues · Glasgow Haskell Compiler / GHC · GitLab is such a case where the logic for instrumenting code using existential quantification probably isn’t correct.

3 Likes
#7

I don’t understand this. Are you saying there’s currently nobody in the world experienced and interested enough to review that MR? Or that such people exist but they’re busy with more important things in GHC land? In the latter case, I’d love an explanation of what work they’re doing that’s more important and productive than to review an existing code contribution gifted by a volunteer.

Open-source projects live and die by their attitude to outside contributors. Why are you even asking for help if you’ll let their MRs wither with no review?

1 Like
#8

That is a great question. It is terribly demoralising to a contributor if their MR appears to be disregarded.

But take me as a case in point. I am Utterly Unqualified to have an opinion about the design, implementation, appropriateness, or quality of a patch to the hpc infrastructure. And that applies to many other contributors to GHC.

So our challenge is this: GHC is large enough that there are significant sub-systems that have no one who understands that sub-system well, loves it, nourishes it, curates it, oversees its design supports contributors to it, etc. The original implementors may have long since moved on, quite legitimately as their lives have developed. The “ticks” in GHC Core language’s syntax are a good case in point. @DavidB has been doing a great job of hpc the libarary but as he says, that is only part of the picture. Bryan mentions ghc-pkg as another example. I would add -XArrows which need lots of love. Likewise the GHCi debugger.

In a language with a corporate sponsor, the company might say to employee X “please become responsible for (say) the GHCi debugger, figure out how it works, review patches etc”. But GHC doesn’t have a corporate sponsor. There is no “them”, there is only us!

To put it another way, who is the “you” in “you’ll let their MR’s wither…”? That “you” is actually us, all of us! GHC belongs to us, not to some company. That is a huge strength, but it is also a tricky weakness.

So what Bryan is asking is: would anyone like to “own” one of these parts of GHC? I wonder if there might be people who might be willing, or even keen, but who don’t want to appear to be trespassing in on someone else’s patch. Don’t worry about trespassing! Just offer :-). You will find a warm welcome.

Simon

PS You might wonder if GHC does, after all, have a sort of corporate sponsor, namely the Haskell Foundation. But, generous though the HF’s donors are, the HF’s resources are far too slender to support all of GHC, Cabal, Stackage, Hackage, HLS, Haddock, etc. Rather, the HF’s role is to empower, energise, and support the open-source community to make all of these projects thrive, by bridging some critical gaps in carefully targeted places. (Bryan’s recent work with Stackage is a case in point.)

12 Likes
#9

What’s the barrier to entry to maintenance? With Haskell I always feel like I don’t know enough to do anything serious but would be willing to be coached up to the code base. Is there any documentation one can read (not the function signatures and source code) to get up to speed?

#10

That you for the explanation of the current predicament Simon, I really appreciate it. And yes, I was thinking of the Haskell Foundation as the “you” in my question. I thought they’re doing it wrong if they’re directing their efforts toward development and maintenance in preference to supporting the existing voluntary contributors, but from your explanation it sounds like the HF’s priorities are correct but the funds are insufficient.

#11

Thanks for the great question and I hope Simon’s answer has been adequate.

I put out this call to action specifically because I’ve been thinking lately how the HF should work to support and enable the community. One of the things I am positioned to do is actually call out which pieces are up for grabs within the large project called “GHC”. It often understandably looks like a monolith with some kind of centralized direction and motivating force, and by dispelling that myth I hope that others can get motivated to jump in.

Plus, sometimes the thing that’s more important than reviewing an existing code contribution might very well be reviewing a different code contribution. There is a real bandwidth issue at GHC HQ right now.

Don’t discount the power of reading source code. GHC in particular in chock full of long explanations about how things work! Even if you don’t end up modifying the code you read, you learn a lot in a short time period and improve your own skills.

Haskell docs should be better, though. This work itself – improving documentation – also needs to be supported and enabled.

1 Like
#12

I very sure about the intent, to empower and support the community. But there is always room for improvement in the execution of intent. Less-than-brilliant execution can be due to lack of $$, but there are also constant judgement calls about priorities. E.g. is it more important to review a patch or to get a release out on time? The GHC HQ team juggles these questions all the time, because there is an almost infinite amount of stuff we could do. (E.g. we have literally thousands of open tickets.)

So if you see things going un-done that seem more important than things that are being done, it’s fine (politely, respectfully of course!) suggest a different priority scheme.

But to return to my theme: by far our largest resource is us, the community. HF is just the oil that lubricates the wheels of an engine whose motive force is us. We are stronger together.

3 Likes