New Hackage Server Features

Edit: deleting this post because I just realized it was a response to a comment from over a year ago and not worth reopening.

Discourse confuses me by highlighting new-to-me discussions as “recent” just because somebody posts a new comment on it. :sweat_smile: (Maybe we shouldn’t resurrect old threads very often)

Is there any way to place the thread’s details:

at the very beginning, so one doesn’t have to scroll through the original post just to see it?

What’s the right setup for cabal-install < 2.0 now? I have some CI jobs with GHC 8.0 / Cabal 1.24 (coming with an old, 32-bit Ubuntu), which fail after the last changes to Hackage server, e. g., Update changelog for v0.11.4.0 (#534) · haskell/bytestring@0602eab · GitHub.

Upd.: After reading remove cabal > 2.0 restriction on files in the 00-index file? · Issue #1063 · haskell/hackage-server · GitHub I assume the answer is "cabal-install < 2.0 no longer works". That’s fine, I can update, just would be better to be a bit more explicit about it.

Yep, you got it. I just sort of decided that there’s no need to support cabal < 2 – even older ghcs are still supported by newer cabal versions, and the last ltses which shipped with it seem about to EOL (cabal 2.0 was released in 2017). And meanwhile the occasional confusion about the 00-index not containing all packages seems to come up far more often, especially as newer versions of cabal files become more frequent on hackage.

Bumping this thread to announce a new hackage redeploy. Some more notable changes:

  • Email notifications for revision upgrades – opt in to that and other useful notifications from your account management page linked from User accounts | Hackage
  • Bugfix for nonupdating “latest version” in package browse.
  • JSON endpoint for top downloaded packages
  • improved check for changelogs and readme files
  • uses cabal 3.10 – this should mean support for uploading multilibs is enabled!

Thanks to @andreasabel @peterbecich @Kleidukos @janus among others for the patches.

21 Likes

Please enable the new email notification (for dependencies going out of bounds) and let me know if there are any problems with it!

When the notification is enabled, the setting can be configured to not keep sending emails if the second highest version number also doesn’t accept the new dependency (see below). This should keep traffic to a minimum.

9 Likes

The email notifications for out of bounds packages is so nice. I used to rely on Stackage reports on GitHub but then I created an issue for a number of packages including one I maintained so I did not see that it went out of bounds.

1 Like

For those who prefer RSS to email, there is also https://packdeps.haskellers.com/feed?needle=Joachim%20Breitner

6 Likes

With the machinery on hackage now, we could probably calculate this there directly as well. PRs to do so would be very welcome!

Do you know of any good user-facing resources that describe multilibs? I think these have a lot of really good potential, but I think that we need to explain them well in order to get the most value.

7 Likes

@nomeata Great feature, didn’t know about it.
packdeps does not really advertise it, does it; at least not on the homepage https://packdeps.haskellers.com/ nor on the linked github page GitHub - snoyberg/packdeps: Web app to track lagging package dependencies. .
So I raised Advertise the feed for one's outdated dependencies · Issue #57 · snoyberg/packdeps · GitHub

2 Likes

7 posts were split to a new topic: How do you display images on Hackage README.me?

There is my article about them and a section in the cabal docs. PRs to improve the docs are welcome!

2 Likes

Currently, Stack does not support dependency on public sublibraries. I consider it ‘No. 1’ in the list of priorities for Stack, but it has exceeded my own capabilities to date. Other people have started on the endeavor from time to time, but not yet seen it through. I think the practical problem is that ‘a package has no more than one library’ is deep-rooted in Stack’s code, and so a lot of work is required to change that: it is not a ‘bite-sized’ problem.

EDIT: To explain my prioritisation, my concern was that - as the Cabal package description format allows for public sublibraries (from cabal-version: 3.0), and Cabal (the library) supports them - then people might start producing packages that Stack cannot handle - including packages that have a central role in the Haskell ‘ecosystem’.

1 Like

Notice that cabal-install has issues with public sub libraries too. The solver just does not see them. See Make the cabal-install solver multilibs-aware · Issue #6039 · haskell/cabal · GitHub and other issues linked there.
If boot packages start using public sub-libraries, cabal-install will always recompile them and never use the pre-installed version.

1 Like

I’m trying to think of a use case that can’t be covered by a multi-project setup via cabal.project.

There’s a minor use case of reducing code duplication across *.cabal files. Not worth the added complexity, imo.

Then there’s the use case of re-using modules in different libraries. Only interesting if they are unexposed… otherwise you can just factor out the common parts into a package and re-use them.

So, what’s the point?

2 Likes

The difference is for package authors. You share a package, not a project. So if you have multiple libraries you want to share “as one”, putting them in a single package can save some headaches. E.g. you can choose to use a single version number, rather then being forced to use a version number per each library.

2 Likes

That sounds more like a drawback to me. Now I can’t version internals independently. Proper projects seem more flexible.

2 Likes

I imagine multilibs were primarily designed with Backpack in mind, because this is where you end up with insane amount of libraries/signatures, which are tedious to split into separate Cabal files. But otherwise it’s a very niche feature and I’m not sure it is worth the incurred complexity on tooling.

1 Like

Here are three reasons why I think you might want to use multilibs:

  1. Multiple packages that are versioned together

One good test is indeed “does it make sense to version these packages independently?”. Often in a multi-package project you have a situation like this:

  • You only ever use and test the packages together at the exact same version (same commit). Therefore you never test other version combinations.
  • You have no interest in e.g. using CPP to ensure that the packages are compatible with a range of the other packages.
  • The packages are tightly coupled, e.g. each new version of A requires the features from the new version of B, etc.

In this case you often end up using the same version for everything and advancing them in lockstep because that reflects the reality of how the packages are supposed to be used. Merging them into a single package with multiple sublibs does the same thing much more cleanly.

Examples:

  • HLS and its plugins
  • amazonka
  • Many industrial codebases I’ve seen

As a side note, if you can version things separately then it’s tempting to try and actually do it properly. I have seen people drive themselves a bit insane trying to properly follow PVP for many packages in the same repo (this is part of why we switched to lockstep versioning in HLS recently).

  1. Extra functionality that incurs more dependencies

Sometimes you want to expose just a little extra thing from your library, but that incurs a new dependency that you don’t want in the main library.

Examples

  • Typeclass instances (for scale, try searching Hackage for “instances”)
  • Test utility code
  • Just little extras, e.g. the lsp-types package now has an extra public library that exposes the model used by the code generator, which isn’t even a dependency of the main library

At the moment you have to put such things in separate packages, which is quite annoying. With multiple public libraries you can just have the test-utility code in an additional library with extra deps.

Of course, in principle you might want to version these libraries separately. Maybe the Arbitrary instances don’t change very often and work with many versions of the base package. But: probably nobody cares or would gain anything from the finer-grained versioning.

  1. Discoverability

In theory, it should be easier to discover a family of related libraries if they are packaged together. Consider again e.g. amazonka. Today the Hackage UI is not there, but in the future it would be nice to be able to pull up a package and go “oh, there’s also a library for quickcheck instances, that’s useful”.


I think some version of these conditions applies for a lot of packages.

9 Likes