GHC Medium-Term Priorities

I believe those two things are orthogonal. The real question is how to have higher quality releases. Higher release frequency CAN improve that if the bottleneck for release quality is the end-user feedback loop. I don’t think that’s the case for GHC at all.

This has partly been communicated in Towards a better end-user experience in tooling · Issue #48 · haskellfoundation/tech-proposals · GitHub

Wrt HLS, this won’t be easily possible, because supporting newer GHC versions may need actual code changes. I don’t think we can make GHC wait for this. For point releases that change no API (given that’s actually true) this could be done, given that GHC trusts the HLS developers to be swift on that.

2 Likes

We need to reduce the costs of making a new release. Why isn’t it fully automated? It should be a button press away, and we should ensure that every commit is release-quality using CI.

If anyone wants more frequent releases, start there.

7 Likes

There are a few things going on here. To zoom out a bit and make a high-level summary:

My main interest is to improve overall Haskell ecosystem quality. I have noticed from many esteemed individuals over the last 12 months or so that there is an assumption that slowing down GHC releases is essential to improving quality, because that way there will be less breakage to deal with. Whilst I sympathise with the frustrations of dealing with breakage[1] I find this point of view mistaken. The correct way to measure breakage is not in terms of releases frequency but in terms of breakage per unit time! That is to say, GHC could release daily but never break[2]. I urge people to disentangle in their minds the concepts of “frequency of release” and “breakage per unit time”. They need not be strongly connected.

Before we get to frequency, I want to make the case that decreased release latency would definitely help mitigate the costs of breakage. Consider this: the earliest commit that goes into a GHC release is made roughly 9 months before the release[3]. Many companies and individuals are waiting 12-24 months before upgrading to a new version of GHC, because their entire suite of package dependencies also has to catch up, amongst other reasons. Let’s say that, conservatively, a lot of GHC commits are waiting at least 24 months to get into the hands of users. There’s a lot of latency in that pipeline! It’s very easy for breakage to go completely unaddressed because so much additional work has been done on top of the breaking change in the intervening two years, and it’s become too hard to do anything about it.

Consider an alternative fantasy world: commits get into the hands of users within 24 hours. Then users are aware of breakage much more quickly, can notify upstream about it much more quickly, and there’s a chance the breakage can be reverted or easily mitigated.

A real world example of this is the DeepSubsumption reintroduction. There was about an 18 month lag between the 9.0.1 that removed deep subsumption, and the compatibility flag being reintroduced in 9.2.4 (even longer than 18 month between the commits that removed it and the compatibility flag). There’s a whole compiler version (9.0 series) that some users can never use! And this is perhaps the best of example of where the costs of GHC breakage were mitigated. Mostly breakage is never satisfactorily dealt with. The ecosystem must simply cope with it, somehow.

Where does frequency of release come in? Well, as touched on above, I think that reduced latency would be an important factor in improving quality. I also think it is unlikely that we can reduce latency without also increasing release frequency. A system gets good at doing what it does, and bad at doing what it doesn’t do. If releases continue to happen at the rate of only two per year then the system will never really have enough opportunity to exercise the parts of itself that need to improve in order to reduce latency. As @int-index says, automation has a vital role to play in this. But something that happens only twice a year can never really satisfactorily be automated. There simply isn’t enough selective pressure to make that automation actually good.

But, I am not saying that increasing frequency of releases is definitely going to improve Haskell ecosystem quality. It might even make it worse. It definitely will make it worse if done in the wrong way. I am not making a proposal of any sort. I am not asking anyone to do any additional work, and there is no need for anyone to feel “chills” about this topic being discussed. Furthermore, I think there is a lot of lower hanging fruit when it comes to quality. I don’t suggest that release frequency should be our top priority.

I am merely inviting people to consider the notion of GHC release frequency from a different perspective. That perspective is only possible once the notion of release frequency has become disentangled from the notion of breakage per unit time. Maybe nothing will come of this line of thinking, but I think it’s interesting and promising, which is why I’m inviting everyone to think it through with me.

[1] and I personally invest time in this, volunteering with the stability working group to determine ways to reduce breakage and mitigate its costs

[2] in an ideal world – in the real world we would fall short, but we could fall much less short that we currently are, in my assessment

[3] certainly more than 6, probably not as much as 12, but I’m not an expert in these matters, so let’s say 9 as a rough figure

5 Likes

Well, I can at least say that I did not suggest this. I think I’ve outlined many times that improving release quality consists of:

  • better release coordination
  • better communication with stakeholders
  • sourcing opinions and providing pre-releases that can be widely tested
  • improving the bus factor of GHC/tooling development

GHC developers in fact told me that slowing down releases will be more work for them.

What I wished for are higher quality releases and less of them. There may be arguments why less releases could improve release quality, but that also depends on many factors.

The main reason some of us wish for less releases is not so much the quality, but the toll it takes on tooling. The end user can simply ignore a couple of broken point releases (there are a lot in the 8.10 series).

From the outside these things (tooling) may not look like a lot of work. Or it may seem all this can be easily automated in some way. But that is not the case.

So, I guess we agree we want to improve release quality.

There are ways to solve this: Nightlies. These will only have rudimentary tooling support (e.g. no curated bindists, no prebuilt HLS binaries, etc.).

In an alternative fantasy world… this would have been communicated to the community earlier, because it was known to GHC developers that it’s a (non-trivial) breaking change. A couple of key people would have been enough to get the feedback “no no”. It did not have to get exposed to the entire “world” to figure out it wasn’t a good idea.

This is what I’ve tried to describe earlier in this thread: community management, involving of relevant stakeholders (during development and release), managing expectations, etc.

All of this takes time and effort of course, but that’s the cost you pay for improving release quality.

I’m worried there may even be less communication and more isolation with higher frequency, because you’ll get feedback anyway post release and can just go and revert.

Do we really need to expose the code to ALL users to get feedback? I don’t think so.

Nightlies are a great way to solve this balancing act of differing requirements.

So what’s the main problem with Nightlies? I guess it’s the fact that old code stops compiling with newer compilers all the time and people can’t reasonably test it on a real world project.

So we’re back to the stability and backwards compatibility discussion.

2 Likes

FWIW I was not referring to you.

So, I guess we agree we want to improve release quality.

Yes, I think so.

Nightlies are a great way to solve this balancing act of differing requirements.

Yes, I think so too, as long as there’s a way to consume them that is not significantly harder than acutal releases, otherwise very few people will do it.

Anyway, I take your point of view seriously under advisement. You are an expert in distribution and release management, and I am not.

1 Like

That is the idea behind the GHC Steering Committee, whose members are meant to represent various interests: education, research, industry. And the simplified subsumption was accepted (#287), so apparently the key people you are referring to have not nominated themselves to join the committee for one reason or another, where their voices would have been heard.

This mistake can be corrected. The committee seeks new members regularly.

3 Likes

Thanks @tomjaguarpaw and @hasufell for your detailed thoughts as always!

On the second point:

  • Currently, are GHC releases held back at release candidate stage until they have tooling support (as far as, say, HLS), notwithstanding issues like 9.2.5? Is this something that could be done?

In my view, likely a minority view [edit: maybe not], a working HLS is a prerequisite for a functioning ecosystem to be built around a version of GHC. I agree that it will often (always?) require code changes to HLS to support updated GHCs. However, if that can be done in the space of days, it seems wise from an ecosystem perspective to leave GHC at RC stage (or maybe with some “bleeding-edge” label on it) for a few days until those code changes have been made.

6 Likes

FWIW, I share this view.

4 Likes

That’s also my opinion.

I find this statement a bit sly: there are plenty people in the Haskell community (me included), who bear no interest in programming language design, dependent types or type systems in general, or endless syntactic sugar debates. Such people would not find themselves comfortable within GHC Steering Committee and will be necessarily underrepresented. The public outcry with regards to the simplified subsumption was a clear indicator how detached GHC Steering Committee is and how little involvement the general public has with its proceedings. And this is not a fault of the public, because the community is never wrong.

Upd.: Detachment from users is an expected property of a committee (e. g., I’m not saying that CLC is any better). This is unlikely something that can be truly fixed, just something to bear in mind and acknowledge, not dismiss under a pretense that anyone can be elected.

9 Likes

It is an invitation, not a dismissal. The committee is running a call for nominations as we speak (until February 11th), see the announcement. EDIT: I got confused about the date, that was in 2022.

The community is not uniform. Some people are fine with breaking changes, some are not. The public outcry comes from the latter group, and I am trying to offer a solution. My apologies that the solution isn’t all rainbows and unicorns. I, for one, am perfectly fine with breaking changes such as simplified subsumption, but people who aren’t need to speak up during the decision-making process, not after.

1 Like

I’m very much with @Bodigrim here. If I was hypothetically on the steering committee, I would not care about any of the topics discussed much at all. I would ask for every proposal only: Does this break existing code? Does it stop GHC accepting code it accepted before? If the answer is yes, I would demand it has to be behind a language flag to be enabled. Never by default.

Applying to the steering committee as someone who was near no interest in new language features in GHC, seems … wrong? Most of the time trying to read the proposal would be a chore I would not enjoy at all, not likely understand or even understand the implications. And again, I wouldn’t really care about the proposal outside of “does it make the compiler stop accepting code it accepted before?” and if so demanding to to be behind a language flag.

So the Steering Committee might need to consider there to be a strong selection bias.

6 Likes

So I guess I should just apply to make sure my voice is heard?

However the announcement explicitly tells me I should not apply as I most likely wouldn’t be able to:

  • to understand such language extension proposals

And I also do not

  • have expertise in language design and implementation, in either Haskell or related languages, which they can share with us.

And as I mentioned before my contribution to each and every proposal would be identical: does it break existing code? If so, it must be behind a language pragma.

@int-index would you suggest that I apply?

2 Likes

It takes effort to understand proposals that are outside of your area of expertise. For example, it took me a while to figure out what was going on in the proposal about exception backtraces. But don’t sell yourself short (I’ve seen your GHC contributions). Besides, the onus of writing a well-motivated proposal with a clear specification lies with the proposal author.

That is one of the possible qualifying properties, the announcement lists other. It’s ||, not &&.

Apparently, this is exactly what had to be said to prevent the situation with simplified subsumption. Now it’s guarded behind NoDeepSubsumption. So, yes, it would have been valuable if you said it and insisted on it.

It is your decision. But you would do the Haskell community a service if you did.

8 Likes

Well then. I have applied.

15 Likes

I think that some clarification is perhaps in order. The document contains the following:

Many commenters have rightly focused on the fact that reducing the amount of work required for a releases is important (e.g. by automating as much as possible). However, I think that this text did not communicate an important aspect of the work necessary for each release, which is that backports of bug fixes are time-consuming and difficult. Right now, there are four branches: the main development branch, the 9.6 pre-release branch, the 9.4 maintenance branch, and the 9.2 maintenance branch. Keeping track of which bugs apply to which, and ensuring that fixes are correctly backported even when the internals have changed a fair bit, is difficult and expensive, and that doesn’t seem to be easily solvable by release automation.

Of course, none of these caveats apply to pre-releases and nightlies - those would be much easier to do more frequently and more prominently, and together with head.hackage, they can also serve as a basis for evaluating language proposals against large non-open-source codebases. But making that feasible for most people will certainly require some tool development and UX improvements.

5 Likes

I think we will be able to cover a lot of the requirements from folks who desire more frequent releases via nightlies, indeed.

I should have a discussion with GHC developers about this topic and figure out if we can flesh out a proposal. Maybe not in the short term, but somewhen not too far away.

Maybe that will make some of the discussed ideas about changing release cycle obsolete and allow us to satisfy both parties.

3 Likes

Perhaps a silly suggestion, but is it more tractable to put less energy into backporting and more energy into automating upgrades with retrie?

In a fantasy world someone who wants an update in GHC 9.4 that is on 9.0 would be able to seamlessly upgrade to 9.4.

That is perhaps an even more difficult problem and not even possible, but there is room for adjusting the levers.

As an example without too much thought behind it: reducing how many versions receive backports in exchange for ensuring upgrades between major versions are 90% automated.

However my memory of the stability working groups goals (if it’s still a thing) tell me this wouldn’t be a very popular idea.

1 Like

…you mean something like:

Also, and since this is a fantasy, to “seamlessly upgrade” GHC tends to suggest that its documentation will be upgraded in similar fashion - maybe one can adapt the ol’ Commentator for use with Glasgow Haskell…

I don’t think so. The fact that GHC and base are tightly coupled right now, combined with the fact that some libraries couple compatibility with new versions of base to their own major releases, means that automating upgrades is probably infeasible in general unless those underlying conditions change. And code compatibility is not everything - a performance regression in a new compiler, or support for a platform that exists in an older version but was dropped in a new version, can make an older branch attractive for a while. Bugfixes on those branches will continue to be useful, I suspect.

It’s definitely still a thing! We’re interested in any technique that has promise for making maintenance of code easier over time.