Abandoned Haskell Packages

#1

Hi Everyone, I want to discuss an issue that I’ve been running into whenever I needed to upgrade GHC versions recently. I know from past experience that some projects only ever need to deal with GHC upgrades once every few years, but with my current hat, I end up doing that quite often with little projects at work that are still actively used but not actively developed.

The recurring issue I run into is that these packages resist GHC upgrades due to very minor problems. Like version bounds that are unnecessarily restrictive, or when not getting exposed from mtl anymore, or a failure due to a minor change in type inference, requiring a very simple change to the type class context of a function etc. If it were not for the bureaucracy of these failures happening in transitive dependencies, these would be 2-minute fixes.

Every time this happens, I want to open a PR to the problematic repo, but I find out that there’s already a PR from 6 months ago that fixes the GHC upgrade, sitting there completely ignored. So I end up adding the fix commit to the cabal.project file as a source-repository-package and move on, but I feel dissatisfied every time I do so. That’s because:

  • There’s no way to share cabal.project between repos, so I have to do it in every such project
  • source-repository-package's aren’t actually that ergonomic, because they (and their dependents) are then considered local packages and that causes a variety of ergonomics issues (which I won’t go into).
  • It saddens me to see that another one of our dependencies is abandoned and we should probably move away from it.

Putting aside the gloomy discussions about the decline of Haskell, I tend to think we have another problem in our hands here, because I see this happening to popular packages as well. I don’t want to name specific packages, but when this issue happens I see that somebody has opened the GHC upgrade fix PR within a few days of the release of the GHC version in question. I also see PRs from other contributors adding new features to the package, all ignored for the last few months.

Regardless of what might or might not be happening to the Haskell ecosystem at large, what I often observe with these popular packages seems to me like a bureaucratic issue. You could justifiably tell me that I should adopt these projects myself then, and I would indeed be willing to do so, but that’s not the kind of process one is typically willing to go through in the middle of upgrading the GHC of an occasionally developed project.

What do you think is the solution here? What do other language ecosystems do? What do you think is the best way I could try to improve the situation?

17 Likes
#2

There’s no way to share cabal.project between repos

This can be done via project imports. You can put the source-repository-package stuff to be shared in its own .config and import that.

3 Likes
#3

Another option would be to import a common .dhall configuration for the repo fork and have Updo generate the project. The Updo/unison/cabal.project example does this.

2 Likes
#4

I think you can ping @hackage-trustees on github to report version bound issues. They can make a revision on Hackage to change the bounds.

Although I see not all trustees are part of that github org, for example @Bodigrim is not listed there but he has helped me with bounds issues before.

Edit: The right way to contact the trustees via GitHub is to:

3 Likes
#5

That is a tough situation indeed. The best we can do is to offer to put in the work, effectively become maintainers of packages which are important to us.

I recently encountered this situation for the distributed-process-* packages, which until recently, looked almost abandoned. I reached out to the maintainers and they were happy to onboard me so that I could help.

Similarly, you might want to reach out and offer to help maintaining the packages which are blocking you.
Personally, I would follow the following levels of escalation:

  1. Make PRs for the fixes you want;
  2. If the PRs are ignored; offer to become maintainer;
  3. As an extreme last resort, if the maintenance offer is ignored, there is a process to take over packages on Hackage. You can then maintain a fork and become the de-facto maintainer.
7 Likes
#6

Thank you, I didn’t know about this! This might come in handy in the future, especially after refactoring the cabal.project files to extract the bits that will help downstream projects. That said, this still seems like a rabbit hole since it will probably have friction with haskell.nix and I can also imagine the cabal.project file of downstream repos conflicting with each other without a lot of care. Still great to know about this.

#7

Yeah; it’s a nice comment and I’ve felt this pain.

For what it’s worth, I’ve found nix(pkgs) to be a nice storehouse of fixes of this kind; it’s not a system that works for everyone of course; but I think the general theme of fixing these things in an overlay is basically the way to go.

1 Like
#8

I developed Updo while working on an industrial project using haskell.nix, so they can work together. Updo has the option to generate everything in cabal.project without using imports. I can’t remember for sure but I think cabal project imports can be a problem if the version of cabal with nixpkgs/haskell.nix is too old.

#9

We already use Nix to build our projects, but we also want them to build with cabal using the same dependencies, so we use haskell.nix to Nixify the Haskell packages, so I prefer to keep the tweaks at the cabal.project layer.

1 Like
#10

This sounds great, thank you for sharing Updo, I’ll definitely check it out.

In the grand scheme of things, though, this sounds like building a workaround layer on top of cabal.project (which itself is a workaround layer on top of the project’s .cabal file when you think about it) and the ideal case for the whole ecosystem would be for these issues to not be there in the first place. That’s why I wanted to open this discussion here hoping to push the needle for a solution to the root cause. Us experienced Haskellers can find one way or another to get our packages to build, but what saddens me is the deteriorating state of the package ecosystem due to the bureaucracy of pushing 1-line fixes.

2 Likes
#11

Even though it was orphaned in 2018, Hugs continues to exist in Debian largely due to non-maintainer uploads - what would be required to support a similar mechanism for Hackage?

#12

I think it already exists and it’s exactly the process @LaurentRDC described!

  1. Send PRs
  2. Request Hackage revisions
  3. Follow the Hackage takeover procedure

This might even be documented somewhere lol

#13

@enobayram it would actually help if you were specific about packages and GHC versions in question. Stackage offers a reasonably complete snapshot for GHC 9.8, so it seems we talk either about less popular packages or about newer GHCs.

GitHub does not relay a ping when you mention an organization. Also please do not ping me personally in such cases, but rather raise an issue at Issues · haskell-infra/hackage-trustees · GitHub

Hackage supports NMUs, available for Hackage Trustees.

2 Likes
#14

Thank you. However Debian allows anyone to provide an NMU. More importantly, the “acquisition” of orphaned or dormant packages merely to remove a few bugs (rather than committing to long-term maintenance) is actively discouraged e.g:

As a last resort (having tried to contact the current maintainer/s, and so on), would it be possible to arrange for non-owners to contact the Hackage Trustees first for permission before the NMU?

#15

Sorry, I don’t quite follow. If you contacted the current maintainers, but did not receive a response within a month or two, in Hackage setting you are welcome to take over the package, even if only to fix a few bugs. Hackage NMUs are largely for cases when no one volunteered to take over or a quicker than permitted by takeover procedure response is justified.

1 Like
#16

But for reasons seemingly unknown to anyone here, people just aren’t interested in taking over as package maintainers merely to remove a few bugs - instead they post a PR for their debugged version and move on (presumably back to their own Haskell project/s of interest). As a direct result, we’re now all here on this latest of posts about this topic.

The current situation with abandoned packages clearly is unsatisfactory, so I merely made a suggestion to try improving it: if it’s a bad one, bring forth the next suggestion…

#17

As I said, I do not follow what’s your suggestion. Can one ask Hackage Trustees to make an NMU with a patch? Yes.

1 Like
#18

To allow anyone to make an NMU, subject to certain conditions - similar to what is allowed in Debian.

Can one ask Hackage Trustees to make an NMU with a patch? Yes.

…this is the first time I’ve seen that option mentioned here (on Discourse).

1 Like
#19

It’s also not mentioned on https://hackage.haskell.org/ or Taking over a package - HaskellWiki. There is a mention at Hackage trustees - HaskellWiki, but it requires a leap of analysis to realize that requesting a NMU is an option for any old Haskeller.

#20

I’m concerned about this problem. It’s a difficult one because there are lots of factors in play. I don’t know what the solution should be, but here is an assorted collection of my thoughts.

Importance

I believe that keeping things building when the ecosystem changes (new GHC/base releases, dependency releases) is the highest-impact ongoing maintenance task that community members can perform. It costs so, so much time and aggravation when you receive build errors when bumping a dependency bound or GHC version. Unfortunately it’s often thankless work (if people don’t see things break they don’t know that you’ve done anything) and uninteresting work.

How to make life easier

I endeavour to swiftly respond to keep my own packages building with the latest version of GHC and dependencies. The Hackage feature that notifies you when a new major version of a dependency is released is very useful for this purpose. You can turn it on through the account management page.

Transition policy

Even though I endeavour to respond swiftly, who knows what the future may hold? To insulate my packages’ users from risk I have instigated a transition policy for Opaleye. Specifically, there is a system of backup maintainers and a specific schedule under which I pre-authorize them to take over the package should I be unresponsive. Hopefully this procedure avoids the uncertainty and delays of a Hackage takeover.

I intended to promote this style of pre-authorized transition policy amongst the community more widely, but I haven’t had time to do so.

Volunteering to help

When I ask the maintainer of a package for a version bounds bump, or a small change to keep a package compiling, I also offer to help maintain the package. I typically say “I’m willing to help maintain the package for the purpose of keeping it compiling” to reassure the maintainer that I don’t want to change or add and functionality of the package, just do basic housekeeping. Consequently, 15 of my 21 Hackage packages are other people’s packages that I am just volunteering to keep building. I’m not doing any development on them.

base breakage

The fact that GHC releases are tied to base major version bumps is a big part of the problem. It means that someone has to do the busywork of bumping base version bounds every time GHC is released, even if the new base version didn’t break the compile.

Hackage Trustees

When all that is required is to make a Hackage revision I have had a good experience asking the Hackage Trustees to make it (after receiving no response from the package maintainer).

3 Likes