This is great news, @hasufell!
Out of curiosity, what will this mean for users of Stack, ghcup, and HLS? Will they have an extra configuration step to get Stack to use ghcup install a GHC that’s ABI-compatible with the HLS on their path, or will that be taken care of in some other way?
I would add, there is current demand for the release for a new version of Stack (https://github.com/commercialhaskell/stack/issues/5733), which would include this ‘hooks’ and other enhancements.
I have never done a release myself, and find the guide (Releases - The Haskell Tool Stack) (comprehensive, but not necessarily fully up to date), daunting. I am also only a Windows user, worried that I might accidently break something for Linux users.
Stack’s release guide says ‘Ensure CI matrices in docs (travis-complex, appveyor, azure) have current stackage snapshots and GHC versions’, I and Sibi Prabakaran have been trying to move Stack to GHC 9.0.2 or GHC 9.2.3 - the latter overtaking the former. The sticking point is two macOS-specific bugs in process, one fixed but not yet released on Hackage (https://github.com/haskell/process/pull/250) and one identified but not yet diagnosed (https://github.com/haskell/process/issues/251). The latter has the kind attention of @bgamari, but if any other macOS experts out there could help pin down what is the cause, I am sure that would be appreciated. As Stack aims to support fully Linux, macOS and Windows (Contributors guide - The Haskell Tool Stack), it seems reasonable to hold off a release until the macOS things are sorted and published.
On GHC ABI (application binary interface), my understanding of the problem is this:
For some Linux users (Debian 9, Fedora 27), the version of GHC 9.0.2 linked on GHC’s download web page is broken. However, elsewhere, ‘9.0.2a’. versions are now available for ghc-9.0.2a-i386-deb9-linux.tar.xz, and ghc-9.0.2a-x86_64-fedora27-linux.tar.xz.
Very recently, Stack’s download of GHC 9.0.2 for linux64-tinfo6 has changed to reference the 9.0.2a version. If Stack has already installed GHC 9.0.2, a user will need to delete GHC 9.0.2 from the stack path --programs directory, to cause Stack to reinstall the GHC 9.0.2a version.
Stack should be able to distinguish things to be built using the GHC 9.0.2a version from things built, and cached, using the GHC 9.0.2 version.
That won’t fix all possible issues though.
HLS itself is built for 10 or so platforms. GHCup is used to build it, which means HLS ends up having the exact same ABI of the bindist that GHCup picks. Additionally, GHCup is used to pick the corresponding HLS bindist for installation.
GHCup and stack have slightly different strategies on how to pick a bindist. They often end up the same, but it’s not guaranteed. GHCup also ships some unofficial bindists and seems to have a larger set (e.g. stack doesn’t install centos bindists for modern versions of GHC).
One alternative would be to build more permutations of HLS bindist releases for a single GHC version and then make hls-wrapper ABI aware, but that’s a lot of work. Discussed here: https://github.com/haskell/haskell-language-server/issues/2865
But I think these issues might not be as common as the 9.0.2 debacle.
I think for the first version it would be:
Then we’ll need to gather user feedback about this feature.
Thanks for the explanation!
To make sure I’m understanding you right, is the bootstrap script what’s found at https://get-ghcup.haskell.org? Is there any mechanism to ask the question when users upgrade existing ghcup installs? What about users who answer “no” but then later want an easy way to change their answer to “yes”?
Yes, this script can be found in the repo:
Not for the normal cli interface, which is non-interactive and will always be.
The TUI however could theoretically achieve that. I haven’t had time to improve it. The last plan was to migrate away from brick to support windows (I made a prototype during zurihac: TUI on windows? (#198) · Issues · Haskell / ghcup-hs · GitLab ). It would probably need a lot of work.
The bootstrap script is idempotent. You can just rerun it any time.
I guess for now the ghcup documentation will have to do.
I thought about adding a command ghcup stack hook install, because at some point, cabal should also get installation hooks: https://github.com/haskell/cabal/issues/7394
That might be a reason to provide a less ad-hoc way to install hooks. But I think this is still in flux and needs further thought.
The discussion under this topic has become wide-ranging, but apologies if this is off-topic. What are the barriers to: [1] GHC/Stack and [2] HLS/GHCup being better coordinated when it comes to referring to the same binary distributions of GHC versions? I say ‘GHC/Stack’ here because Stack says it mirrors GHC’s Index of /ghc/ (and has done so for a while). For people who do not know, by default (it is fully configurable by the user), Stack looks to this YAML file to know where to look for a GHC version (and other tools), organised by ‘operating system’, ‘architecture’ and (optional) ‘variant’.
(For example, taking GHC 8.10.7 as an example, Stack’s default YAML provides sources for linux32, linux32-musl, linux64, linux64-nopie, linux64-musl, linux64-tinfo6, macosx, macosx-aarch64, windows64-integersimple, windows64, freebsd32, freebsd32-ino64, freebsd64, freebsd64-ino64, freebsd-aarch64-ino64, linux-armv7, and linux-aarch64.)
Edit: The reason I ask is to see if there is anything that could be done by me (or somebody like me) at the Stack end to help.
Stack has a different logic. It looks at system libraries (tinfo and libc) to pick the bindist.
GHCup does not. It picks based on distro and distro version: https://github.com/haskell/ghcup-metadata/blob/master/ghcup-0.0.7.yaml#L1807
One difference here is that it picks centos bindist for centos. Stack does not from what I see. There may be many other differences.
Thanks for directing me to that GHCup YAML. I think it would be pretty easy to allow a Stack user, who knows the GHCup Linux ‘taxonomy’, to use that same taxonomy with Stack if they wanted to. I distiguish Stack (the application) from the default Stack setup-info dictionary. Stack (the app) already includes the --ghc-variant option for the very purpose to allow people to distinguish Linux distributions. I can’t see any barrier to the default dictionary containing more entries than it currently has. I’ll think about this further, as an issue on the Stack repository.
GHCup also provides unofficial bindists not in stack. Synchronizing both is difficult, since those are basically separate distribution channels.
We try to use the upstream bindists as much as possible for the purpose of building HLS and not breaking ABI with stack. But ultimately the best solution for HLS stack users is to let ghcup manage things.
In respect of the ‘unofficial’ binary distributions of GHC (the compiler) used by HLS but not published by GHC (the team), are they (a) supplementing what is published by GHC; or (b) replacing something published by GHC?
If they are supplementing, I can’t see any high barrier to Stack’s default dictionary containing additional entries.
If they are replacing, that seems to me to be more complex. It raises the question in my mind: Why has HLS (the team) wanted/needed to replace what GHC (the team) has published? Is there something I can read to understand the typical cause(s)?
Both.
E.g. official alpine 8.10.7 bindist is not usable because it forces integer-simple, which created issues with some packages, so I built one against gmp myself. HLS for alpine is built against the same.
I may also delay official releases in ghcup and fix packaging errors myself, even if they cause ABI breakage. The 9.0.2a release is one example, although it is in fact official. But GHC devs have signalled they won’t do those type of packaging bugfix releases anymore, probably because of the unexpected ABI fallout with hls and stack.
That means GHC bindists are not always reliable and may need fixing downstream. GHCup will do that. That’s the point of being a distribution channel.
I don’t see how that can easily be synchronized with stack, except for relying on ghcup to install GHC.
Again: this is a very specific issue due to HLS and ABI incompatibility. Users can just build HLS from source and use it with any bindist they like. I won’t have time to put a lot of effort into this in the near future.
I’m not intentionally advocating for any specific course of action for the HF to take. Tom’s suggestion of a meeting sounded good. I’m suggesting that to do nothing at all is a mistake. I am suggesting that the HF supports a big friendly button (BFB) on www.haskell.org somehow, because it satisfies the core mission of the foundation.
The “what happened last time” happened within a context of HF activity, is not really public, and is thus difficult to pin down when advocating for the BFB.
I don’t think much has to happen. You can clearly see common sense and comraderie happening between stack and GHCUp in this thread. There will be more if the community is allowed to come together in this, around a BFB.
I think this because of the sum total of evidence I see in our community media. There is no longer any substantive bipartisanship based on our tooling eg has there been any calls for cabal to be replaced? Instead, there is a bunch of stack and GHCUp people working towards improvement in this very thread. Time to celebrate! We are not split by our tooling anymore. That’s what’s changed.
There still exists a small minority of cabal bros, who come out and howl in the moonlight whenever stack is mentioned. They get too much attention as it is and I suggest that they are the ones that you should either directly confront or ignore.
There are also a bunch of people in our community that have put up with this howling for ever so long, and they should be acknowledged and canvassed. But it doesn’t serve them to associate this community effect with acknowledgement of GHCUp as a great solution for new users.
We can better respect them by making sure “it” doesn’t happen again, and we support the next generation of like-minded innovators, even (especially) if they disrupt us.
To be clear: as a ghcup developer, I have no opinion about cabal vs stack at all. In this function, all I care about is that ghcup can reasonably support these tools and that there’s at least some notion of unix philosophy (tooling hierarchy, do one thing, passive integration), so that my life doesn’t get more difficult.
As a haskell developer, I do have some other opinions, but I can very well ignore these and will not have them influence decision making in ghcup, which affects a large part of the community, including stack users.
…similar to the “Download Haskell” button which has been on the wiki front page since (at least) 05:38, 12 December 2007 09 Oct 2018, but only larger?
I strongly support the Big Friendly Button that points to ghcup for many of the reasons that have already been expounded upon.
If identity matters, which I hope it does not, I have been more or less exclusively a Stack user, professionally and personally, for something like 7-8 years now, and I have never used ghcup. Why this is so is irrelevant to the proposal, but I guess it nevertheless makes me a member of the “stack community”, if there is such a thing, although I would not choose to identify as anything other than a Haskell User.