Which if a specific temporary such hackage existed I might be less concerned about. But it has to be deleted when the release is out. It can not be some indefinite running thing like head.hackage.
2 Likes
For what it’s worth, I completely agree with this. This was one of the reasons why I proposed ghc.X.hackage, making this explicit. However, I also understand (and agree) that ideally we would not have as much need for head.hackage.
On the other hand, I can’t wave a magic wand and make the need disappear. Core library (e.g. text), base, and template-haskell changes happen. In some of these cases we can try to minimize/mitigate changes (e.g. #24021: Making Template Haskell (more) stable across GHC releases · Issues · Glasgow Haskell Compiler / GHC · GitLab) but ultimately some amount is (practically) unavoidable.
I think it is defensible to say that we shouldn’t actively encourage head.hackage usage by end-users (as ghc.X.hackage would have done). However, in the short term I struggle to see a world where we could reasonably release an alpha which could build a significant fraction of Hackage without something like head.hackage.
4 Likes
To be clear, that’s because it’s unrealistic for GHC to wait for patched versions of all relevant packages to be available on Hackage before the release happens, right? I.e. head.hackage allows for loose coupling between the release of GHC and its dependencies. Without it, GHC would have to wait for all its dependencies to be patched and released before GHC could make an official release.
If so, I understand. And yet, I believe in Moritz’ vision of a world where this is not necessary. I don’t know what it looks like yet, but it seems like a good problem to eventually solve.
1 Like
I’d even argue more vehemently against this, for another reason.
If I as a library maintainer get a pull request saying “this will fix compatibility with upcoming GHC release”… I will want proof and I want to be able to install said GHC version, before I accept said patch. For that to work, you need to release a pre-release. So we have a chicken and egg problem here.
So in my opinion:
- pre-releases are for the ecosystem to catch up (libraries in particular), not for application developers to conveniently test their whole stack and expect the ecosystem has already caught up
- there should be a longer period between pre-release and proper release (for new major versions at least), allowing for said catch up
- X.Y.1 will then naturally be the phase where application developers can start bringing in feedback
- GHCup will always avoid recommending X.Y.1 versions and I believe the community is aware of what this release means… I don’t see the point of doing two phases of pre-releases. It’s already hard enough to tell people to use any pre-release, so we shouldn’t make it more complicated.
7 Likes
The main reason I don’t like head.hackage it not even that.
head.hackage is a pool of patches from different authors and I as an end-user have no easy way of knowing whether those patches are:
- correct
- have been submitted upstream
- have been approved upstream
You don’t even write such information in the patch header: patches/lens-5.2.3.patch · master · Glasgow Haskell Compiler / head.hackage · GitLab
To me that is enough reason to prefer going through all the PRs and issue trackers and cooking up source-repository stanzas instead.
4 Likes
I don’t think this is quite right. None of the examples given are dependencies of GHC. It’s a set of patches for ecosystem packages broken by the new version of GHC.
2 Likes
Ah, you are right. Rereading Ryan’s post at the top of this discussion makes that clear:
But to check my understanding, does that mean that most patches are added by (or at the behest of) Hackage package maintainers?
1 Like
One thing that’s a shame is that the source-repo stanza strategy requires quite a bit of duplicated work for each early user (building, gathering failures, searching/making PRs, repeat). It would be good if we could get head.hackage or an alternative into a state where it was able to fulfil this role.
Perhaps it would be good to make some issues on the head.hackage repo?
2 Likes
AFAIK patches are mostly written by GHC devs (@RyanGlScott does the lion’s share of this) or other users of head.hackage, which can overlap with maintainers but mostly doesn’t I think.
Isn’t it the complete opposite? If Hackage package maintainers wanted changes to their packages, they’d just make them! It seems to me that the only people want to incorporate something into head.hackage are those who can’t make a change to a package directly.
2 Likes
Yes, I think the existence of head.hackage is an indication that we might need better collaboration mechanisms for small patches. E.g. a way in hackage to submit such a patch, that is then relayed to either a github PR or git email/etc.
It’s also an indication we might need a better mechanism for release candidates on hackage (GHC compat patches for unreleased GHC versions should naturally go there). Their ergonomics currently is rather awful. An end user should be able to tell cabal “consider all release candidates”.
1 Like
I have to imagine you’re right! But then, who are these non-maintainer, non-GHC-developer users of head.hackage? I feel like we’re missing their voice.
In other words, the only uses of head.hackage that I am actually familiar with are the “secondary” use cases Ryan mentions. I know head.hackage is built against GHC every night (sometimes it even succeeds! But not usually), and I know that GHC tests that require non-boot packages have been bolted on to it.
Who else uses it?
Who else uses it?
As I understand from @angerman, it’s used by developers of either libraries or applications who want to bring their codebase up-to-date with a new GHC release, but who depend on packages that require changes in head.hackage to build correctly.
I can’t tell from that if anybody is actually using it in the way Moritz describes. Yes, we want people to trial alphas… but from listening in on GHC calls, it sounds like that isn’t happening.
Ben’s proposal gives a pretty good summary of the end user use case: tech-proposals/proposals/001-ghc-x-hackage.md at ghc-x-hackage · bgamari/tech-proposals · GitHub. But as Ben said on the other thread one reason why this proposal was parked was because folks seemed to not do this too much in practice
1 Like
Well, I don’t know personally, but @angerman thinks it’s not only happening, but happening so much that it is lowering the quality of the ecosystem.
That would basically just be a centrally maintained “project file” with all those source-repository-package stanzas, that you can import:
https://cabal.readthedocs.io/en/stable/cabal-project.html#conditionals-and-imports
It’s not clear to me how this would be an improvement over the status quo. To the contrary, it seems to make your concern worse: Auditing the patch-set would require checking out dozens of distinct repositories whereas today they are found centrally in the head.hackage repository.
I wholeheartedly agree with this. With better collaboration tools I think the need for head.hackage (both social and technical) would be greatly reduced.
For various reasons:
- easier to use/configure: one import line in my cabal.project. No dealing with repositories or running cabal update.
- I can see the origin of the patches very clearly and often know just from looking at the repo url if it’s an upstream patch or not. Otherwise I can easily see on gtihub if it’s a fork and follow the the links to the upstream repo. head.hackage doesn’t bother putting any of that information in the patch header.
- I can see what packages are changed more easily. I don’t have to look up the head.hackage repo.
- it’s not a moving target
Downsides:
- extracting the exact patches for all changes is likely non-trivial
- I’m not sure how exactly in-place updates to remote project files work. I remember a discussion I had with @sclv about this on the PR that introduced imports. I think you’d have to delete the downloaded file from
dist-newstyle/
1 Like