RFC: brokering toolchain installations with hs

Haskell Foundation
#1

The Haskell Foundation would like to see Haskell installers installing both cabal-install and stack everywhere. That opens the issue of managing GHC installations as both ghcup and stack manage their own stash of bindists. To this end I propose a simple broker tool for reporting to the build managers (when asked) whether a given GHC bindist has been installed or not, and if so, where it is located.

According to configuration, hs can query the user if they would like to install the missing tool chain or not, requesting the preferred installation manager (currently ghcup or stack) to install it accordingly.

I have a proof-of-concept stack integration on a private branch (This patch is Unix only for now — extending this to Windows should be straightforward.)

To achieve cabal-install integration I have made up wrapper scripts that call out to hs to locate and run the required tool.

With these setup on the PATH (hs, the pathched stack and the wrapper scripts) I see a unified environment with bindists being shared and control with the developer of who carries out the GHC bindist installations as the requirement arises with the build managers.

It would be ideal if we could patch cabal-install, like stack, and do away with the wrapper scripts of course.

See this blog post for more details and let me know what you think.

3 Likes
#2

“There is no problem in computer science that can’t be solved using another level of indirection. But that usually will create another problem.”

6 Likes
#3

Why? Do the users want this?

I’m not sure I agree that this is good UX, because it adds yet another tool/layer. Looking at the README already made me confused what it really does or why I would need it.

It also seems you’re trying to make cabal-install do automatic GHC installations through those wrapper scripts? I believe this goes against cabal-install spirit, especially since it’s an interactive installation: https://github.com/cdornan/hs/blob/2fd6cb9ba4f94a715f9b6255b2af6ba1fb2ab58a/src/HS/Install.hs#L34

I’ll dig into it further.

1 Like
#4

Ok, I think I got some clarity now as to what is going on.

The motivation seems to be: whatever tool people use, they never have to install a specific GHC version twice.

To be clear, this is a secondary problem. It is worthwhile to discuss, but I do not understand why this is brought up now, while we’re still struggling to get a consistent story for windows, which should be our primary focus.

As for the proposed solution: I think it is not the right way, will require more maintenance, adds more moving parts, more things that can go wrong, more stuff for a user to debug in case of failure.

IMO, the better way would be to:

  1. have a way to tell stack to direct GHC installation to a user provided shell script (what we call a HOOK in many package managers, git and other unix tools, etc). The input would be e.g. GHC version/commit etc and the shell script would have to print the path to the GHC bin/ dir to stdout.
  2. Then the user can use ghcup in said hook, or whatever else they please
  3. ghcup could (during its bootstrapping process) pre-install a ghcup-maintained hook, that is guaranteed to work

This way stack doesn’t have to integrate actively with ghcup and also gives users much greater flexibility, without increasing maintenance load. People could even use nix to install GHC.

3 Likes
#5

One concern I have is that much air has been spent discussing implementation strategies, when we haven’t yet agreed on a specification.

@chris outlined a starting point for a user-focused specification. Before diving into implementation strategies (such as Chris’s more recent proposal), I would urge us to agree on a user-focused specification. In the end, our users care much more about how they interact with Haskell than what tools are used under the hood to make that interaction happen.

In particular, the more recent proposal seems focused on avoiding duplicate installations of the same version of GHC. While this is a nice thing to have, how much does it really matter to users? I don’t know – perhaps we make it one of our user-focused design goals (but I would argue against). Instead, I think a more user-focused design goal would be something like “quick installation” (for some definition of “quick”). If a second GHC is quick to download and set up, then maybe it’s not so bad. Furthermore, if someone switching between cabal-install and stack already has to recompile all their libraries (do they?), then the advantage of keeping just one GHC is fairly minimal. I’m not sure (truly), but I would want all of these decisions to start and end with “what’s best for users”.

Everyone: for reasons we’ll never know, the issue of installers is like The Ring to Rule Them All – note “that the bearer soon developed a strong attachment to it, becoming increasingly reluctant to relinquish it.” Somehow, as soon as someone has a concrete idea of how to implement a Haskell installer, the idea becomes precious, and the bearer soon becomes increasingly reluctant to relinquish it. I do not wish to debug this quirk of our community, just to draw awareness to it. And awareness suggests that all of us should tread lightly and try hard to be open-minded. One way I hope to encourage this open-mindedness is to set aside the issue of implementation and focus only on user experience, at least until agreement there has been reached.

7 Likes
#6

I’ll give an example scenario to why I think this is important (even though I don’t think it’s the most important thing right now):

Consider a new Haskell learner who wants to learn Haskell from a tutorial that teaches them how to build a guess the number game. This tutorial uses stack by using either stack exec --package random runghc -- guess.hs or by creating a stack project.

The user goes on to the downloads page and downloads ghcup. Say ghcup straight away installs ghc, cabal-install and stack. The learner waits for ghc to install so the can start. In places with slow internet connection this could take some time.

When everything is installed the user goes ahead and run the stack commands in the tutorial. Stack installs ghc again. The user is frustrated. Why is this doing it twice? Why does it take so long? Why does it use so much space? First impression is not very positive.

2 Likes
#7

Richard you are right to question it but actually users do care about storage overhead of Haskell installations — not as much as compile times of course, but this has been repeatedly mentioned to me (see the Reditt response to this article for example). i think to just ignore the problem is quite problematic — reflects our general lack of concern about the Haskell footprint.

i find the discussions around the installer really interesting and revealing. if we can work through the issues then we have a real shot at making real progress on the issues that i think really matter — a shared understanding of what a Haskell development should look like — for all of the community. These discussions ate definitely illuminating critical issues as i see it.

#8

We could start by asking the maintainers of the relevant tools what points have been brought forward by users on e.g. their issue trackers.

All I see right now is plans, ideas and propositions coming from the Haskell Foundation (or parts of the HF) with little evidence that these things are really as critical as is claimed.

GHCup has just added stack support. There are already user reports about it. Not many. These things take time. Pumping out proposals before we even have a clear picture about what needs to be solved just adds to the confusion.

I prefer to work with my users to improve the UX or figure out solutions with maintainers of other tools on how to approach an integration. I’m not even sure how to respond to these proposals/blog posts anymore. It’s chaotic.

1 Like
#9

This will be biased toward advanced users.

It seems like these tools and changes are targeting people who likely won’t file issues because they aren’t as invested in the language yet.

However I’m also under the impression that care will be taken not to step on the toes of advanced users.

6 Likes
#10

I can say “no” with confidence. I’ve already had a haskell newcomer (1 month experience) implementing a feature for ghcup, so they had no problem finding the issue tracker, posting several issues and getting involved in the codebase.

2 Likes
#11

A dedicated newcomer is always welcome, but I do indeed wonder if that represents the median of dedication beginners tend to have.

Asking also people who dwell on #haskell and help beginners (and similar channels) could bring more voices in and maybe address the relevant point made by @codygman.

1 Like
#12

GHCup redirects to #haskell and #haskell-ghcup libera IRC channels on the website. So far, no one has complained that they didn’t know where to post issues or where to ask questions. All the links are on one line.

I’m confident that stack maintainers have a similarly strong relationship with newcomers. Using github isn’t a high entry barrier these days.

To answer your question: I hang out a lot on IRC and I’ve never heard complaints about this specific issue. I do remember users asking why GHC itself is such a huge install (1GB or so), but very rarely.

For users, who have space constraints, we can provide documentation on how to alleviate that (namely by using stacks system-ghc feature, which works very well).

For a more flexible solution, I propose https://github.com/commercialhaskell/stack/issues/5580 but that doesn’t need to be solved now or as part of the current installer proposal.

2 Likes
#13

That’d be wonderful, but when groups of people can’t agree, some people just take action. Turning them away because we didn’t spec out the solution to everyone’s liking is a hard sell.

At the same time, I agree with you.

Personally, I think it’s silly to build solutions without first establishing requirements… but with no central authority, and with marginal success in pulling together the disparate groups, it’s not surprising that this has become the norm.

2 Likes
#14

Yes, that would be nice, particularly for those beginning with haskell and learning their way around. Once you have had some time, you’ll probably have an opinion, yet this type of “playing nice together” makes it easier for the ecosystem to evolve and produce more harmonizing results.

That’s a reasonable pain point to address. I’m appreciative of your efforts and making an effort to see it happen!

2 Likes
#15

We’re not even sure what the RFC is really solving. IMO, half of it is a duplication of ghcup, the other complicated wrapper mechanisms that need explicit integration with stack/ghcup. And this at a time where it was clear that we should limit the scope of our efforts in order to not get stalled in endless details about how integrations should work. This was worked on “in the dark” and presented at last minute on the tech track.

At the same time, a very simple specced out solution that I wrote for stack to provide GHC installation hooks that can be used in diverse ways has been shut down and the thread locked, signalling that broader discussions aren’t appreciated: https://github.com/commercialhaskell/stack/issues/5580

This to me is an indication that we might need to take a step back from this whole topic and approach it in a different way.

Of course, everyone is free to work on whatever they want. But unifying efforts requires more than that.

#16

I don’t disagree on the dysfunction here @hasufell, however, given that both of you are listed on the HF projects page as the unified installer’s leaders/advisors, opening that issue to begin with shows that as leaders/advisors for the project, you two do not seem to be working in concert, but more as individuals (at least that’s what it looks like from the outside).

1 Like
#17

I believe it’s extremely important to demarcate opinions and proposals from decisions. I was made aware that several people misread this topic as an official announcement of HF’s course of actions. This discussion is an RFC for a proposal on Chris’ personal blog. HF Tech Track discussed it last Tuesday, but did not come to a conclusion yet.

Speaking of demarcation, I believe there is a certain misunderstanding here:

The Haskell Foundation would like to see Haskell installers installing both cabal-install and stack everywhere

Just to be clear: this has been proposed indeed, but is under discussion and not approved yet.

5 Likes