GHC Medium-Term Priorities

[david-christiansen]

The GHC team asked me to help them gather input about their priorities for the next six months.

To ensure that the volume of feedback from this pilot process was manageable, we initially contacted 23 expert users with whom we have pre-existing relationships. In a future iteration we will look into how to expand the set, while keeping the process informative, deliberative, and manageable. The responses were analysed by separating the requests by topic, and we followed up as necessary for clarification. Then, we prioritised the tasks based on a mix of how often they were requested, how severe the issues were for the requesters, and our own judgement about resource availability and value.

1. Things we plan to do in the next six months

This section documents the user priorities that we plan to achieve in the next six months. We believe that they are all achievable, given available resources, and that they will lead to real improvements to GHC for many users. This consists of two large projects and a number of important smaller tasks.

1.1 Larger Project: GHC management

A number of respondents asked for more clarity and transparency in the management of the GHC open-source project. In the next six months, we plan to more clearly define and document the GHC open-source project’s technical leadership, membership criteria, and decision-making practices. (#22747)

1.2 Larger Project: GHC API

Today, the GHC API is not particularly well-defined. There is a GHC API module, but it does not include sufficient tools to enable clients of GHC to do what they need to do. As a result, in practical terms, GHC’s API is essentially its entire implementation. This poses a dilemma: clients need some form of API stability, but stability of implementation internals means an end to progress.

The Haskell Foundation and the GHC team will create a working group of GHC API clients, such as the HLS team and compiler plugin authors, to define the needs for a more stable API, along with a stability policy that’s satisfactory to both clients and the GHC implementers. We may also explore whether some facilities from certain API clients, particularly HLS, should in fact be part of the GHC source tree.

1.3 Smaller Tasks

• File layouts for releases. Packagers and redistributors of GHC have requested release file manifests and changelogs that describe changes in file layouts. We plan to add this to the changelog, and prevent omissions using CI going forward. (#22748)

• We received feedback that some users don’t feel that they can report bugs when they are incapable of producing a minimal failing example. This is a mis-comunication:

  • While small reproductions certainly make it much easier for us to diagnose a problem, we are able to work with larger open-source examples when it’s necessary.
  • As a matter of policy, we also accept larger reproductions, with the caveat that we may not have the resources to fix the bug, although we will try. (#22749)

• Advertising GHC releases. Some people who need to know about GHC releases for packaging and other reasons are not well-served by subscribing to the ghc-devs mailing list. There seem to be two issues

• Medium: some people would like to be emailed explicitly, not just monitor ghc-devs. We would be happy to hear from people who would like an explicit email.

• Timing: it would help to have a clear notification timetable; e.g. we will tell you our release timetable, rather than simply announce releases.

• Currently GHC has bugs on Apple’s latest M2 hardware with macOS Ventura, although macOS is a Tier 1 platform. We now have access to a suitable machine, and (with support from the HF) will invest in making sure that GHC works on it (#22750). More generally, we will refine the Tier 1 platform descriptions to be more precise about operating system version ranges. (#22751)

• With help from the HF, we will expand the Tier 1 platform table to specifically indicate which Linux distributions and versions are Tier 1; and we’ll enhance our CI to unpack an appropriate bindist on each of these distributions and check that it works. (#22751)

• We were asked to be less Unix-centric in our documentation, as GHC supports other platforms as well. We will do our best to view documentation updates through the perspective of a variety of platforms, but GHC development will primarily occur on Unix-like machines for the next six months, so specific feedback on the documentation is also welcome.

• Contributors to GHC are rightfully interested in a more reliable CI setup. The Haskell Foundation now employs a full-time DevOps engineer to work on this, and progress is being made. Please try again, and report CI failures to Bryan Richter (@chreekat) for follow-up.

• Tools such as HLS rely on ghc-exactprint, but time can pass between the release of a GHC version and release of the corresponding ghc-exactprint version. Due to its dependencies, ghc-exactprint cannot be made part of the GHC source repository, but we plan to include it and its transitive dependencies in head.hackage, which allows CI to detect the emergence of incompatibilities and anyone (not just its maintainers) to contribute patches to make it work with GHC HEAD. (#22752)

• One respondent mentioned desired improvements to HPC, including general polish, improved control over what gets annotated, and how to combine reports. A new group of contributors has resumed work on HPC, after a period of dormancy. We will attempt to support them in making their improvements, and we’re happy to connect users who are asking for improvements to HPC with this group.

• Linux distributions that have local patches to GHC have asked that we actively integrate them. However, the patches as submitted are not always suitable for inclusion and so are sometimes turned down. We are open to mentoring the submitters, or to helping them document the issues fixed by the distribution patches so that we can solve them in a way that works for our long-term maintenance responsibilities.

• The new diagnostics infrastructure, with datatypes whose constructors represent the various errors and warnings that GHC can emit, will be completed. As of now, there are 172 instances of TcRnUnknown, the constructor of not-yet-structured type checker and renamer errors. We see this as an easy (if somewhat tedious) way to bring immediate value to at least HLS and to errors.haskell.org, which relies on the numeric identifiers assigned to each constructor, and we hope that it will also benefit other clients of GHC-the-library. (#22753)

• It was requested that we run tests and benchmarks for all the boot libraries in CI, to better detect regressions. We don’t think that we can do this in GHC’s CI, because the test and benchmark suites for various boot libraries depend on broader parts of the ecosystem, and keeping all of the testing frameworks updated with GHC would be too much. Instead, we plan to use head.hackage for this, and extend it to also run its tests on aarch64. The migration of many boot libraries to tasty and tasty-bench, both of which have small dependency footprints, makes this easier and more reliable. (#22754)

• Finally, we were asked to ensure that recent improvements made to cross-compilation do not degrade. We assure you that cross-compilation is a priority for us, and we have no plans to down-prioritize it.

2. Proposals Needed

Language feature requests need to go through the GHC Steering Committee’s proposal process before being implemented in GHC. We’re happy to contribute insights regarding the ease or difficulty of implementing a proposal, but we are not in a position to drive the process forward for the following requests:

• We were asked to extend GHC to support multiple simultaneous default implementations of type class methods (e.g. in terms of other methods, and using default signatures). We are happy to comment on a proposal to the GHC Steering Committee and help develop the idea, but the corner cases of the design need to be worked out with the committee before we can just implement it.

• Sandboxing Template Haskell to control its ability to run arbitrary IO actions is an interesting and useful goal. As sandboxes are notoriously difficult to use appropriately, and any meaningful sandbox will also need to account for unsafePerformIO, we believe that a sandbox design should go through the GHC proposals process.

• It was requested that we find a way to support circular modular dependencies that’s easier to use than .hs-boot files and that has good support for incremental compilation. We don’t have a design for this sitting on the shelf, and we are happy to participate in keeping a GHC proposal grounded in the realities of the implementation, but we are not in a position to drive such a proposal forward ourselves.

3. Insufficient Resources

Here are some larger tasks that we would like to do, but we do not believe that we have sufficient development resources to achieve them in the next six months together with the other priorities here. They will remain in the queue for future iterations.

• We’d like to make it easier to test development versions of GHC in order to enable feedback prior to releases and help the package ecosystem be ready for new releases. While we regularly test changes to GHC against Hackage, many significant Haskell code bases are either internal applications or are not part of Hackage. One way that we could make it easier to get feedback from users would be to more conveniently provide nightly releases of GHC, e.g. by placing all nightlies for all platforms from the last month on a well-known server. Making it easy to install and test these nightly releases would enable others to give timely feedback about compatibility concerns.

• A longer term goal would be to assemble nightly releases of the Haskell toolchain, including Cabal, Stack, Haddock, and HLS, such that they all work together, with CI to catch any ongoing incompatibilities and warn users. We don’t expect to be able to do this in the next 3-6 months, and this effort would require coordination with other projects that we don’t presently have the capacity to lead, but we are interested in participating in such an effort.

• We were asked to produce a whole-program analysis to guide specialisation, rather than relying on manually inserted SPECIALISE pragmas or on flags like specialise-aggressively and expose-all-unfoldings that result in an explosion in compilation time and executable size.
  
  We believe that there have been recent significant improvements to specialisation that may address the needs of the requester, and we hope to bridge the gap by improving the documentation and potentially adding flags to make it easier to diagnose situations where specialisation is desired but has not yet occurred. We will initiate a dialog with the requester.

4. Requests that will Remain Unfulfilled for now

Making GHC a lot faster. We don’t believe that we have the possibility to make GHC significantly faster in the next three to six months. All of the low-hanging fruit has been harvested, and what remains require significant investments in R&D that we don’t believe are immediately feasible. We watch compile times obsessively through CI, and that ensures that there are no major regressions, at least.

Making it easier to use the GHCi debugger, especially on executable projects, would be very useful. Unfortunately, that code presently has no owner, and the team does not understand it well enough to iterate quickly on it. We really need a volunteer to take it over.

One respondent requested that GHC produce diffs in cases where a fix for a problem is readily apparent. The GHC developers would rather focus on enabling other tools (such as HLS) to do this. We endeavor to include enough information in the structured error representations to make this possible, and we invite requests for more data as needed to achieve this goal.

It was requested that GHC development use git-subtree instead of Git submodules. We do not plan to adopt this recommendation, because we think that it would trade one set of problems for another, and that it would not yield a net improvement. Relevant discussion can be found in #16642.

We would very much like to improve the ABI compatibility situation. However, we don’t see a technical way forward that would preserve cross-module optimization while still enabling Template Haskell to use compiled code. It seems that the biggest issues with ABI incompatibility are related to HLS and GHC bindists - perhaps we can ameliorate these issues some other way.

We received feedback that our release cadence is too fast, and other feedback that it is too slow. There seems to be no consensus as to how often major releases are desired by users. However, we don’t think we can release any faster than we do today, as a release is a lot of work, so that places a natural upper bound on the frequency of releases.

5. Breaking Changes

We were asked to reduce the number of breaking changes. We hear you. There are several distinct strands:

• The Haskell Language, as implemented by GHC, is the responsibility of the GHC Steering Committee. There is not much that the GHC team can do here, other than working to ensure that our own proposals are as backwards-compatible as possible.

• The base library is the responsibility of the Core Libraries Committee. When changes are initiated by the GHC team, we’ll make proposals to the CLC that include mitigations (e.g. deprecation warnings, pattern synonyms, whatever) whenever we can.

• The GHC API, which has in practice been the GHC implementation, remains a sore point. To begin with, it will be the responsibility of the new HF working group (see above). The GHC team cannot reasonably promise to keep every internal function stable. But when the working group does establish the API, we will work hard to keep it stable and to have as pleasant a migration path as possible when breaking changes are necessary.

[romes]

Thank’s for writing this up for us all

[hasufell]

@david-christiansen

In the next six months, we plan to more clearly define and document the GHC open-source project’s technical leadership, membership criteria, and decision-making practices.

I believe one of the major confusions is how GHC HQ comes to their list of priorities (as this one) and what/who is involved. What stakeholders are consulted etc.

However, transparency is about proactive communication too. And about community management.

So I hope this will not stop at a documentation effort.

@david-christiansen

Advertising GHC releases. Some people who need to know about GHC releases for packaging and other reasons are not well-served by subscribing to the ghc-devs mailing list.

I believe this is a misunderstanding. It’s not merely about advertising, but about collaboration. However, I think this has already been partly communicated. We will have to observe the results.

@david-christiansen

We received feedback that our release cadence is too fast, and other feedback that it is too slow. There seems to be no consensus as to how often major releases are desired by users. However, we don’t think we can release any faster than we do today, as a release is a lot of work, so that places a natural upper bound on the frequency of releases.

I urge you to investigate from which direction these opinions come from (GHC contributors, researchers, industry users, packagers, etc.). Otherwise we’ll never come to a conclusion why there’s a massive rift in our community wrt these expectations (which are ultimately about stability as well) and how to manage it.

[andreabedini]

@david-christiansen

we initially contacted 23 expert users with whom we have pre-existing relationships

@david-christiansen would you be able to provide a quick breakdown of how these 23 users are divided between academia or industry? Perhaps with some further subdivisions (e.g. “mainly research” / “mainly teaching” / “large company” / “small company”).

[Kleidukos]

@hasufell

I urge you to investigate from which direction these opinions come from (GHC contributors, researchers, industry users, packagers, etc.). Otherwise we’ll never come to a conclusion why there’s a massive rift in our community wrt these expectations (which are ultimately about stability as well) and how to manage it.

This is one of the points that had surprised me the most from the results. Not that I cannot fathom some people wanting more GHCs per annum, but I also had the distinct impression from my time in the ecosystem that the rest of the tooling community (HLS first in the trenches) is breaking their back to support newer GHC versions, which frequently leaves these new GHC releases without tooling or the dependencies required for the bigger projects out there.

Happy to hear opposite points of view, to be honest, but I’m also interest in the proportion of each opinion.

[Kleidukos]

But I have to say that I deeply appreciate the work that was done here, and that it definitely helps with the trust in the GHC development process as a whole, from an outsider’s perspective.

[david-christiansen]

@andreabedini

@david-christiansen would you be able to provide a quick breakdown of how these 23 users are divided between academia or industry? Perhaps with some further subdivisions (e.g. “mainly research” / “mainly teaching” / “large company” / “small company”).

I can do this breakdown, but I think it would be a bit misleading, so I’ll accompany it with what I think is a better one.

First, what you asked for:
Mainly research: 1
Mainly teaching: 0
Large company (> 300): 9
Small company: 11
Not sure: 3

But the invitations were not all sent out on the basis of someone’s employment (although some were). Many were sent to people as representatives of particular open-source projects (e.g. Cabal, Stack, HLS, Agda), and others were sent to regular GHC contributors who weren’t directly part of the planning process. Others were explicitly considered as representatives of their company, or asked to answer in a dual role, which often occurred in the context of people who contribute to GHC as part of their employment.

From the perspective of the project or organization represented in the feedback, the breakdown is:
Haskell tool developers (Haddock, Stack, Cabal, HLS, GHCup): 6
GHC contributors: 9
Core libraries committee: 1
Other open-source projects (Agda, Fedora): 2
Industrial users, big companies: 4
Industrial users, small companies: 6

The numbers add up to more than expected because of people who work at companies that both contribute to GHC and use Haskell for business, and who have both those roles at the company and are providing feedback from both perspectives.

Because this was a pilot process, part of the selection criteria was also choosing people with whom the core GHC developers have pre-existing relationships, partly to reduce uncertainty in the workload of processing replies and partly to ensure that the replies were well-informed about GHC. This summer, it’ll be time to go back and reflect on the value provided by this process and decide whether to do it again, perhaps in a broader format, and to follow up on the commitments made in the reply above.

[david-christiansen]

@Kleidukos

Happy to hear opposite points of view, to be honest, but I’m also interest in the proportion of each opinion.

I don’t want to send the proportion, because I think it would be misleading. The questions asked were:

What do you need from GHC that you aren’t getting at the moment? What are your primary pain points, or lost opportunities?

In other words, there was no specific question about release cadence. Additionally, very few respondents mentioned it. This makes the quantitative aspect of it much less likely to provide signal than noise.

[david-christiansen]

What the format of the questions lacked in potential for quantitative analysis, they made up for in qualitative descriptions of motivations and needs.

From that perspective, the release frequency question seems to be a matter of your relationship to GHC. First off, I’d like to stress that most people didn’t mention it at all, so I’m describing trends within tiny subsets of a tiny subset of a small group. Those who wanted less frequency had the same concerns that you present in your message (work required for tool and library support), and they were typically tool developers. Those who wanted more frequency typically maintain libraries and apps, and they coupled the desire for higher frequency with an assumption that each individual release would have fewer changes and be easier to adapt to, reducing the need to maintain branches for major releases that are less compatible with each other.

[simonpj]

@hasufell

So I hope this will not stop at a documentation effort.

It’s certainly intended to be more than a document! But what would you like to see happen, if you could wave a wand?

[Jappie]

Gosh, I wonder which of these requests came from Agda.
(looks at massive cirular dependency which only keeps on growing).

You’d say it’d be easier to adapt hs-boot to support incremental compilation.
Although I’ve no idea how any of that works.
It’s an interesting request though, I tried to rip out a single hs-boot file from agda, and it’s safe to say, this is practically impossible

[tomjaguarpaw]

@david-christiansen

Those who wanted more frequency typically maintain libraries and apps, and they coupled the desire for higher frequency with an assumption that each individual release would have fewer changes and be easier to adapt to, reducing the need to maintain branches for major releases that are less compatible with each other.

This is the key point, in my opinion.

I strongly recommend that people try to disentangle in their minds the concept of frequency of release from the concept of breakage per unit time. In principle it’s possible to release daily whilst breaking never (and personally I think that would be a good goal to aim for, although maybe we would necessarily fall a bit short in practice).

[david-christiansen]

@Jappie

Gosh, I wonder which of these requests came from Agda.

Agda in particular were public about their gathering of input from the project: Our feedback on GHC medium-term priorities · Issue #6274 · agda/agda · GitHub

I don’t want to share all the responses because I didn’t tell people they were writing for a large public audience. But I think that sharing links to public issue trackers is OK.

[hasufell]

@simonpj

It’s certainly intended to be more than a document! But what would you like to see happen, if you could wave a wand?

I’ll take the wand.

What I have in mind are two things.

1. More coordination with other projects

This has been partly described in my “proposal” here: Towards a better end-user experience in tooling · Issue #48 · haskellfoundation/tech-proposals · GitHub …I was kind of inspired by the monthly HLS meetings led by @michaelpj where @mpickering is already a regular. The idea is that we have a forum/space to catch up across tooling, inform each other of stuff that is relevant. This time investment may be a net positive for GHC devs, IMO, so they shouldn’t be worried about it. I talked about this with Matt too. We’ll have to see how things develop.

GHC is a massive project and very few people can keep up with all the things that are going on at the same time. I think it can be hard for people interested in only specific things or for the casual reader to get a picture about the state of GHC development.

E.g. I was told that there already is a regular meeting with specific stakeholders (e.g. from industry). I had never heard of it, nor do I know how one would get selected to attend. Would there be a way to promote these meetings for a wider audience or have a record of it? It sometimes appears GHC development is a bit isolated and a lot of things are handled “internally”. This may be due to efficiency concerns, but there is always a balance.

There may be other forms necessary to improve “community management”, such as marketing new ideas, more frequent blogs (not just for the advanced user), etc.

I’m not expert on this, though.

2. Actually looking for help, not just waiting for it

Some people may be tired of this example, but FreeBSD support in GHC was sort of discontinued with the explanation that it takes too much time afair and that no one is supporting it. Yet, I haven’t seen a public post asking the community for help on FreeBSD support.

It seems it was assumed these guys will just show up when bindists cease to exist. To make affairs worse, the infrastructure to build FreeBSD on gitlab had also been decommissioned, making the entry barrier for a potential FreeBSD contributor even higher.

Interestingly, I was contacted by a user on reddit privately a couple of days ago, who saw the GHCup update that fixed FreeBSD support and they asked when FreeBSD will become a tier1 platform again and how they can help.

So I guess… there are users willing to help. But you have to look for them and ask for help and make sure that contributing for the things you don’t have time for can still be done reasonably.

---

Both these things are about being more proactive. I think we forget sometimes between all the technicalities that simply asking for help or informing a downstream consumer of a breaking change can go a long way and give the project an even more welcoming vibe.

[simonmic]

Terrific informative high-level status report/edict, thanks!

I guess many of us have lost track of the various committees and groups, so a micro-review could be useful. I think “We” is the “GHC team”, while “GHC Steering Committee”, “Core Libraries Committee”, “new HF working group” are all separate groups ?

[david-christiansen]

@simonmic

I guess many of us have lost track of the various committees and groups, so a micro-review could be useful. I think “We” is the “GHC team”, while “GHC Steering Committee”, “Core Libraries Committee”, “new HF working group” are all separate groups ?

Sure thing.

This process was initiated by the GHC developers, who asked for administrative help from the Haskell Foundation (that being me in this case). We had a few discussions and meetings to figure out a process based on the following priorities:

• Lightweight enough that we could afford the required effort
• Deliberative and qualitative, rather than large-scale and data-driven (in part because we weren’t sure we had the right questions)
• Scoped to 6ish months of effort
• As much transparency and accountability as possible

One weakness of my post here is that “we” in the first two paragraphs is GHC+me, while later in the post it’s just GHC. Sorry about that.

The other groups you mention are:

• The GHC Steering Committee is a committee that evaluates proposals for the language implemented by GHC. Their remit does not include things like compiler internals, CI, performance, refactoring, or testing. Instead, they discuss things like new language extensions, changes to the Haskell language, etc. They’re not formally a successor of the Haskell 2010 committee, but given that GHC is now the only widely-used Haskell implementation, they play a similar role in practice.
• The Core Libraries Committtee is responsible for most of the libraries that almost every Haskell program transitively depends on. They evaluate proposed additions and changes, ensure that libraries remain maintained, and generally take care of the libraries.
• The new group that will be starting up once I get some other tasks done is intended to exist temporarily, not permanently, for the purpose of developing a recommendation of a more-stable API to the internals of GHC than exists today, along with strategies for reducing the maintenance burden on tools that depend on GHC (compiler plugins, formatters, HLS, etc) while imposing as small of a maintenance cost on GHC development itself as possible.

[Ericson2314]

We’re getting pretty good at cross compiling to FreeBSD in Nixpkgs these days. Adding back FreeBSD to CI as a mere cross target is not as good for regular FreeBSD users as actually testing on regular FreeBSD, but it could be way less work and hassle and still quite decently useful.

[Ericson2314]

@tomjaguarpaw

This is the key point, in my opinion.

I strongly recommend that people try to disentangle in their minds the concept of frequency of release from the concept of breakage per unit time.

I would like to second this too. The only way to stop doing accidental breakage is to release more frequently. It’s like the only way to keep your room from being dusty is to clean it more often. The release chores are healthy, and to the extent they are painful they can be further automated.

[angerman]

I agree that release frequency and breakage are should not be synonymous. That they are with GHC is a major failure. The fact that you can not trivially use a codebase that compiles perfectly fine with 8.10.7, and swap out the compiler for 9.2 is just ridiculous.

We could have nightly builds (in fact CI produces builds for each MR), but most of us won’t be able to use them. Hell, moving a codebase for 8.10 to 9.2 can take long already, how am I even supposed to try 9.6 on this?

And this has so many knock on problems. I can test alphas, and report back bugs and regressions? How am I supposed to do this, the compiler doesn’t accept the code it used to accept just fine.

So yes, we could have higher frequency and focus on automating the release process. But the much bigger elephant in the room is that GHC keep breaking the god damn surface language with almost every release. This insanity has to stop!

[david-christiansen]

There are indeed incompatible changes connected with GHC releases. I think they come in three categories:

• Changes to the ordinary, “public” part of base and other standard libraries like mtl or bytestring - this is really a topic that the CLC needs your feedback on, and the GHC team is bound by their decisions
• Changes to the part of base that are really more “exposed GHC guts” (e.g. GHC.Exts)- this is the most difficult thing, in my opinion, and we don’t have a good story about it. It would be good at the very least to make it more clear to users when they’re depending on this kind of thing, but we’ll likely still all end up transitively depending on things like the details of word types. See also CLC#105.
• Changes to the language itself - these should all go through the GHC Steering Committee. While these certainly are problems for compatibility sometimes (I’m looking at you, simplified subsumption), my own anecdotal Haskell experience points to changes in libraries more often causing incompatibilities.

A consequence of the way that the Haskell community is organized today is that there really isn’t a single group or person where the compatibility buck stops. Given this organization, I think it’s unfair for the GHC developers to get the full pummeling when things change incompatibly, even though we do typically encounter the pain upon upgrading (because new GHC means new base, which means all the changes come as a bundle).