Abandoned Haskell Packages

[enobayram]

This sounds great, thank you for sharing Updo, I’ll definitely check it out.

In the grand scheme of things, though, this sounds like building a workaround layer on top of cabal.project (which itself is a workaround layer on top of the project’s .cabal file when you think about it) and the ideal case for the whole ecosystem would be for these issues to not be there in the first place. That’s why I wanted to open this discussion here hoping to push the needle for a solution to the root cause. Us experienced Haskellers can find one way or another to get our packages to build, but what saddens me is the deteriorating state of the package ecosystem due to the bureaucracy of pushing 1-line fixes.

[atravers]

Even though it was orphaned in 2018, Hugs continues to exist in Debian largely due to non-maintainer uploads - what would be required to support a similar mechanism for Hackage?

[chreekat]

@atravers

what would be required to support a similar mechanism for Hackage?

I think it already exists and it’s exactly the process @LaurentRDC described!

1. Send PRs
2. Request Hackage revisions
3. Follow the Hackage takeover procedure

This might even be documented somewhere lol

[Bodigrim]

@enobayram

Putting aside the gloomy discussions about the decline of Haskell, I tend to think we have another problem in our hands here, because I see this happening to popular packages as well. I don’t want to name specific packages, but when this issue happens I see that somebody has opened the GHC upgrade fix PR within a few days of the release of the GHC version in question. I also see PRs from other contributors adding new features to the package, all ignored for the last few months.

@enobayram it would actually help if you were specific about packages and GHC versions in question. Stackage offers a reasonably complete snapshot for GHC 9.8, so it seems we talk either about less popular packages or about newer GHCs.

@jaror

I think you can ping @hackage-trustees on github to report version bound issues. They can make a revision on Hackage to change the bounds.

Although I see not all trustees are part of that github org, for example @Bodigrim is not listed there but he has helped me with bounds issues before.

GitHub does not relay a ping when you mention an organization. Also please do not ping me personally in such cases, but rather raise an issue at Issues · haskell-infra/hackage-trustees · GitHub

@atravers

Even though it was orphaned in 2018 , Hugs continues to exist in Debian largely due to non-maintainer uploads - what would be required to support a similar mechanism for Hackage?

Hackage supports NMUs, available for Hackage Trustees.

[atravers]

@Bodigrim

Hackage supports NMUs, available for Hackage Trustees.

Thank you. However Debian allows anyone to provide an NMU. More importantly, the “acquisition” of orphaned or dormant packages merely to remove a few bugs (rather than committing to long-term maintenance) is actively discouraged e.g:

5.12. Package Salvaging

Do not start a package salvaging process when you do not intend to maintain the package for a prolonged time. If you only want to fix certain things, but not take over the package, you must use the NMU process, even if the package would be eligible for salvaging.

As a last resort (having tried to contact the current maintainer/s, and so on), would it be possible to arrange for non-owners to contact the Hackage Trustees first for permission before the NMU?

[Bodigrim]

@atravers

As a last resort (having tried to contact the current maintainer/s, and so on), would it be possible to arrange for non-owners to contact the Hackage Trustees first for permission before the NMU?

Sorry, I don’t quite follow. If you contacted the current maintainers, but did not receive a response within a month or two, in Hackage setting you are welcome to take over the package, even if only to fix a few bugs. Hackage NMUs are largely for cases when no one volunteered to take over or a quicker than permitted by takeover procedure response is justified.

[atravers]

@Bodigrim

If you contacted the current maintainers, but did not receive a response within a month or two, in Hackage setting you are welcome to take over the package, even if only to fix a few bugs.

But for reasons seemingly unknown to anyone here, people just aren’t interested in taking over as package maintainers merely to remove a few bugs - instead they post a PR for their debugged version and move on (presumably back to their own Haskell project/s of interest). As a direct result, we’re now all here on this latest of posts about this topic.

The current situation with abandoned packages clearly is unsatisfactory, so I merely made a suggestion to try improving it: if it’s a bad one, bring forth the next suggestion…

[Bodigrim]

@atravers

The current situation with abandoned packages clearly is unsatisfactory, so I merely made a suggestion to try improving it: if it’s a bad one, bring forth the next suggestion…

As I said, I do not follow what’s your suggestion. Can one ask Hackage Trustees to make an NMU with a patch? Yes.

[atravers]

@Bodigrim

I do not follow what’s your suggestion.

To allow anyone to make an NMU, subject to certain conditions - similar to what is allowed in Debian.

---

Can one ask Hackage Trustees to make an NMU with a patch? Yes.

…this is the first time I’ve seen that option mentioned here (on Discourse).

[chreekat]

@atravers

…this is the first time I’ve seen that option mentioned here (on Discourse).

It’s also not mentioned on https://hackage.haskell.org/ or Taking over a package - HaskellWiki. There is a mention at Hackage trustees - HaskellWiki, but it requires a leap of analysis to realize that requesting a NMU is an option for any old Haskeller.

[tomjaguarpaw]

I’m concerned about this problem. It’s a difficult one because there are lots of factors in play. I don’t know what the solution should be, but here is an assorted collection of my thoughts.

Importance

I believe that keeping things building when the ecosystem changes (new GHC/base releases, dependency releases) is the highest-impact ongoing maintenance task that community members can perform. It costs so, so much time and aggravation when you receive build errors when bumping a dependency bound or GHC version. Unfortunately it’s often thankless work (if people don’t see things break they don’t know that you’ve done anything) and uninteresting work.

How to make life easier

I endeavour to swiftly respond to keep my own packages building with the latest version of GHC and dependencies. The Hackage feature that notifies you when a new major version of a dependency is released is very useful for this purpose. You can turn it on through the account management page.

Transition policy

Even though I endeavour to respond swiftly, who knows what the future may hold? To insulate my packages’ users from risk I have instigated a transition policy for Opaleye. Specifically, there is a system of backup maintainers and a specific schedule under which I pre-authorize them to take over the package should I be unresponsive. Hopefully this procedure avoids the uncertainty and delays of a Hackage takeover.

I intended to promote this style of pre-authorized transition policy amongst the community more widely, but I haven’t had time to do so.

Volunteering to help

When I ask the maintainer of a package for a version bounds bump, or a small change to keep a package compiling, I also offer to help maintain the package. I typically say “I’m willing to help maintain the package for the purpose of keeping it compiling” to reassure the maintainer that I don’t want to change or add and functionality of the package, just do basic housekeeping. Consequently, 15 of my 21 Hackage packages are other people’s packages that I am just volunteering to keep building. I’m not doing any development on them.

base breakage

The fact that GHC releases are tied to base major version bumps is a big part of the problem. It means that someone has to do the busywork of bumping base version bounds every time GHC is released, even if the new base version didn’t break the compile.

Hackage Trustees

When all that is required is to make a Hackage revision I have had a good experience asking the Hackage Trustees to make it (after receiving no response from the package maintainer).

[enobayram]

@chreekat

I think it already exists and it’s exactly the process @LaurentRDC described!

Then it seems to me like there’s room for improvement in this process, because we are still in this situation where popular packages can fall behind despite the fact that there are users who will send fixes within days.

@Bodigrim

@enobayram it would actually help if you were specific about packages and GHC versions in question.

The last package that pushed me over the edge to start this thread was beam-postgres, which isn’t in the latest Stackage LTS, so it requires tweaking to get it to compile with GHC 9.6, which I had no trouble doing on my end, but I was left with disappointment to see the package in this state.

[tomjaguarpaw]

@enobayram

The last package that pushed me over the edge to start this thread was beam-postgres, which isn’t in the latest Stackage LTS, so it requires tweaking to get it to compile with GHC 9.6

I’m going to take this opportunity to make a self-promotional aside to point out that Opaleye typically supports a new GHC within days of release (although its dependencies don’t always).

[enobayram]

I did lift an eyebrow when I read your previous message TBH

[Bodigrim]

@enobayram

The last package that pushed me over the edge to start this thread was beam-postgres, which isn’t in the latest Stackage LTS, so it requires tweaking to get it to compile with GHC 9.6, which I had no trouble doing on my end, but I was left with disappointment to see the package in this state.

Stackage LTS for GHC 9.6 is the largest snapshot ever, containing 3300+ packages. So if beam-postgres is not there, I think this package is not that popular after all, simple as that. No need for gloom and doom.

I assume we are talking about Support GHC 9.6 in beam-postgres by kmicklas · Pull Request #694 · haskell-beam/beam · GitHub? Note that the author of this PR is a maintainer of beam-postgres himself, so it’s not like the package is abandoned; things just fall through the cracks from time to time. It seems to be a simple Hackage revision, so you can raise an issue for Hackage Trustees, but I would start by simply pinging the maintainer in the PR.

[jamesbrock]

@enobayram

If it were not for the bureaucracy of these failures happening in transitive dependencies, these would be 2-minute fixes.

The Haskell Github Trust was created to help with this situation. Haskell Github Trust is for Other People’s Packages.

I sent you an ownership invitation.

[enobayram]

@jamesbrock

I sent you an ownership invitation.

Thank you! I’ve accepted it

[jkb101]

Can I suggest that we allow ownership to decay? Decay would happen if the package isn’t updated over time and if PRs aren’t processed over time.

If package ownership has decayed below some limit, then some other interested party could get access – this would require some (undefined) technical and (undefined) process implementation.

The point of my comment is that the above topic is another symptom of something we see all the time, namely a public request to take over a package. The process to take over a package takes time and is rather cumbersome. Moreover, not everyone wants to take over the package, they just want the PR to be executed, or possibly refused, but at least dealt with. Letting others have some kind of update access would simplify all this.

[hasufell]

@jkb101

Can I suggest that we allow ownership to decay? Decay would happen if the package isn’t updated over time and if PRs aren’t processed over time.

Some packages are simply done from an engineering perspective and not every package has a corresponding github repo that you can monitor.

We already have enough processes for that:

• NMU for GHC compat patches
• package takeover for new maintainers

See hackage-trustees/policy.md at master · haskell-infra/hackage-trustees · GitHub

If hackage would automatically transfer ownership of my packages, I’d probably stop publishing them there.

[tomjaguarpaw]

I like this idea, with some caveats. Firstly, I wouldn’t want it to be forced upon maintainers. Secondly, I don’t want a package to be taken over just because feature development has stopped – that is a valid state for a maintained package!

I agree that the current update procedure is cumbersome and I would prefer a more straightforward procedure that is worked out ahead of time.

This is my intention for Opaleye’s policy of backup maintainers. There is a specific criterion for when the policy is triggered (I am “unavailable or unresponsive to maintenance requests for n months”) and a specific outcome in that case (a particular person becomes entitled to full ownership of the package).

I’d be happy if other maintainers wanted to adopt a similar policy for their own packages (and perhaps they have ideas for improving the policy).