Call for opinions: shortening the HLS support window

#1

Currently, HLS’s GHC support policy is as follows:

  • We designate the GHC major version that is used by the latest Stackage LTS as “stable”
  • We stop supporting a GHC major version once it is three major versions behind the stable major version

For example, at the moment Stackage LTS is on 9.6, so we support everything from 9.2 onwards.

We have been wondering about shortening our support window. Supporting more versions of GHC is generally fairly costly for us, as it requires quite a lot of CPP and CI time. The main cost to users of a shorter support window would be that users on older versions wouldn’t get new versions of HLS (the old versions, of course, would continue to work). However, these days HLS is changing more slowly, so the cost of not having the latest and greatest is comparatively lower.

There are a lot of plausible options for the support window. I think our current method of using Stackage LTS as a measure for when the ecosystem has sufficiently adopted a GHC version is reasonable, so the main choice is “how many GHC version before that do we support?”, specifically:

  • 0
  • 1
  • 2 (the current situation)

Opinions welcome!

8 Likes
#2

Doesn’t this also mean that these days it’s actually easier to support a wider version range, since less changes and so less is likely to break?

#3

Not really, no. There are fewer user-facing changes, but that doesn’t mean that the amount of work we need to do internally to support new GHC versions is less. That’s pretty constant.

For example, right now we’re working to support 9.10, which is resulting in a lot of work to adapt to the exactprint changes. We’re going to end up with substantially different code for 9.10, and that all has to be gated behind CPP. It’s hard to avoid this, since we depend on lots of truly internal bits of GHC where it would be unreasonable to expect stability. Perhaps one day at least exactprint will be more stable!

Similarly, the overhead of running all of our CI jobs for an additional GHC version is just a constant overhead on everything.

6 Likes
#4

More generally, fairly often we want to take advantage of something new in GHC (maybe something we implemented or asked for!). Suppose this is introduced in GHC major version X. That inevitably means CPP for the before-X/after-X case, which will persist until we no longer support versions before X.

#5

The other option is to have separate branches per supported GHC and do backporting and separate releases.

That would somewhat change how we do packaging. But it is worth consideration.

#6

Yes, we’ve talked about this. My current feeling is that this would be worse - our current state would be the same as maintaining five active release branches and backporting everything to each of them. That would really suck. If we were only backporting bug fixes it would be better, but still quite a lot of work (and fewer updates for old versions).

#7

Well, HLS is effectively a compiler. Since GHC API is ad-hoc and has no stability guarantees, you could also make an argument it should be shipped with GHC.

On the other hand, the decoupling feels cleaner and is also a good argument to actually have a stable GHC API (for this and other reasons).

I’m just thinking out loud.

2 Likes
#8

Just to throw in my two cents; I’m a very happy user of HLS + VS code (so a billion thanks for all your efforts!). As long as my VS code Haskell extension keeps working with some version of HLS, I don’t mind not getting the cutting edge features for those projects that happen to lag behind in their GHC version. That said, I don’t know whether the VScode Haskell extension is capable of picking an HLS version based on the project’s GHC version.

#9

It is :slight_smile:

It uses the following file to determine the latest HLS release supporting your project GHC: ghcup-metadata/hls-metadata-0.0.1.json at develop · haskell/ghcup-metadata · GitHub

(This feature is exclusive to VSCode haskell extension)

2 Likes
#10

I feel like rather than basing it off stackage LTS, we should be basing things off the GHCs supported by GHC HQ: GHC Status · Wiki · Glasgow Haskell Compiler / GHC · GitLab
Practically I think the green releases would be good to stick with: 9.4+ inclusive at the moment.
I also feel like each release, it’s easier and easier to upgrade GHC, so maybe in a year or two the support window could be cut by another release.

I think this reflects stability of releases better, and it means that if a user runs into a GHC bug while using a version supported by HLS, they can make a GHC bug report without being told their version is no longer supported.

The other thing is I think Stackage bumps tend to reflect more the capacity of the Stackage maintainers rather than the stability of releases. Or at least that’s my impression as an outsider – I might be wrong!

4 Likes
#11

That depends on what you mean with “stable”. Historically, the burden of figuring out what is actually stable lies on the distributors shoulders.

The linked page is about upstream support windows.

Also see #19901: Advertise "latest" vs "recommended" GHC versions · Issues · Glasgow Haskell Compiler / GHC · GitLab

2 Likes
#12

Yeah fair enough. On reflection I think you are right. “stability” was a bad choice of words.

The main point is just that I don’t think it makes much sense to support many versions of GHC that are no longer getting bug fixes. I think basing off Stackage LTS will lead to a picture that lags behind a bit.

#13

I’m not sure. I’d much rather see HLS follow GHCup and support only two GHC versions at a given time:

  • latest
  • recommended

The reality is that large parts of the ecosystem and companies are still using GHC versions that don’t receive active bugfixing anymore.

I think it would be quite frustrating if the recommended GHC version in GHCup is not supported by the latest HLS.

This would also require a bit more coordination, possibly.

4 Likes
#14

Supporting at least the HLS recommended GHC sounds like a good policy IMO.

Yeah I agree. But I feel like we should be spending our limited energy as a community to try to avoid this situation, ie, by making upgrades easier. Though of course there’s a balance

#15

Anecdotally, work only just moved to GHC 9.6, and we’re in no particular hurry to move to any other GHC (we do dependency checks on a regular cadence, and upgrade GHC if it’s convenient to do so). I think my colleagues would be sad if HLS support was only intermittently available (beyond nix-induced weirdness, I mean).

I’d also endorse the suggestions that you maintain support for at least the GHCup-recommended GHC, and the latest one GHCup knows about (if possible). That seems like an important part of the new Haskeller experience.

1 Like
#16

I feel like rather than basing it off stackage LTS, we should be basing things off the GHCs supported by GHC HQ… I think Stackage bumps tend to reflect more the capacity of the Stackage maintainers rather than the stability of releases

Sorry, I think the “stability” thing was a red herring. We care more about which compilers people are using rather than which ones are considered stable by the GHC team. Then we’re looking at a series of proxies:

  • My impression is that the Stackage LTS only moves when a significant enough portion of the Stackage packages work on that compiler version, and that’s a decent proxy for “the Haskell ecosystem as a whole works with that compiler version”
  • If the ecosystem has moved on, that’s a decent proxy for both “people can use this version for real (because their dependencies now work)” and also “people (package maintainers) are using this version for real”

Ideally, we would have some kind of usage metrics (downloads from haskell.org or something) that we could use, but at the moment the best direct metric we have is the annual state of Haskell survey, which is infrequent and not super complete.

GHCup’s “recommended” version could work as a proxy. I think that’s mostly about whether the compiler is in a good state, but it does probably cause people to use that version! I think it could be reasonable to pick a “oldest popular version” as min(stackage_lts, ghcup_recommended).

So here’s a policy proposal: we support up to the “oldest popular version” as defined above. That would mean 9.4 at the moment, since that’s where GHCUp is currently sitting.

I think my colleagues would be sad if HLS support was only intermittently available… I’d also endorse the suggestions that you maintain support for at least the GHCup-recommended GHC … That seems like an important part of the new Haskeller experience.

I just want to re-emphasize that nothing here means that HLS will be unavailable for old GHCs. The old HLS versions will continue to be available and work fine, just as they did when they were released! The only difference is that if you are on an old version of GHC you might not be able to get stuff from the newer versions of HLS.

4 Likes
#17

I think that it is now settling into its final form. Any future changes should be minor tweaks

2 Likes
#18

Giving my two cents as to how we handle GHC versions and HLS at work:

  • We look at Stackage for our GHC update cadence (not strictly, it just makes it easier)
  • We’re currently on GHC 9.2.8
  • We update a jump of 2 when the target version stops being the newest LTS.
  • So we’ll update to GHC 9.6.* when the newest LTS starts using GHC 9.8.*

We’re happy that HLS still supports GHC 9.2.* in the newest versions, but we’d be totally fine with the features we have now, and only getting new stuff when we update to GHC 9.6.*.

The only thing I’m “worried” about is that the VS Code extension might start being annoying about “having to upgrade” when the newest HLS wouldn’t work with our current GHC version, and then forcing us to stop using GHCup to manage the HLS and having to fiddle around with the settings to make sure the right HLS is in the $PATH, and stuff like that.

So my plee is “No issues (Developer Experience-wise) when using older GHC versions with older HLS versions, then we’ll have no problem with the newest HLS only working for the newest GHC versions” :+1: