New Hackage Server Features

[sclv]

Bumping this thread to announce a new hackage redeploy. Some more notable changes:

• Email notifications for revision upgrades – opt in to that and other useful notifications from your account management page linked from User accounts | Hackage
• Bugfix for nonupdating “latest version” in package browse.
• JSON endpoint for top downloaded packages
• improved check for changelogs and readme files
• uses cabal 3.10 – this should mean support for uploading multilibs is enabled!

Thanks to @andreasabel @peterbecich @Kleidukos @janus among others for the patches.

[janus]

Please enable the new email notification (for dependencies going out of bounds) and let me know if there are any problems with it!

When the notification is enabled, the setting can be configured to not keep sending emails if the second highest version number also doesn’t accept the new dependency (see below). This should keep traffic to a minimum.

[mihaimaruseac]

The email notifications for out of bounds packages is so nice. I used to rely on Stackage reports on GitHub but then I created an issue for a number of packages including one I maintained so I did not see that it went out of bounds.

[nomeata]

For those who prefer RSS to email, there is also https://packdeps.haskellers.com/feed?needle=Joachim%20Breitner

[sclv]

With the machinery on hackage now, we could probably calculate this there directly as well. PRs to do so would be very welcome!

[david-christiansen]

Do you know of any good user-facing resources that describe multilibs? I think these have a lot of really good potential, but I think that we need to explain them well in order to get the most value.

[andreasabel]

@nomeata Great feature, didn’t know about it.
packdeps does not really advertise it, does it; at least not on the homepage https://packdeps.haskellers.com/ nor on the linked github page GitHub - snoyberg/packdeps: Web app to track lagging package dependencies. .
So I raised Advertise the feed for one's outdated dependencies · Issue #57 · snoyberg/packdeps · GitHub

[fgaz]

@david-christiansen

Do you know of any good user-facing resources that describe multilibs? I think these have a lot of really good potential, but I think that we need to explain them well in order to get the most value.

There is my article about them and a section in the cabal docs. PRs to improve the docs are welcome!

[mpilgrem]

Currently, Stack does not support dependency on public sublibraries. I consider it ‘No. 1’ in the list of priorities for Stack, but it has exceeded my own capabilities to date. Other people have started on the endeavor from time to time, but not yet seen it through. I think the practical problem is that ‘a package has no more than one library’ is deep-rooted in Stack’s code, and so a lot of work is required to change that: it is not a ‘bite-sized’ problem.

EDIT: To explain my prioritisation, my concern was that - as the Cabal package description format allows for public sublibraries (from cabal-version: 3.0), and Cabal (the library) supports them - then people might start producing packages that Stack cannot handle - including packages that have a central role in the Haskell ‘ecosystem’.

[andreabedini]

Notice that cabal-install has issues with public sub libraries too. The solver just does not see them. See Make the cabal-install solver multilibs-aware · Issue #6039 · haskell/cabal · GitHub and other issues linked there.
If boot packages start using public sub-libraries, cabal-install will always recompile them and never use the pre-installed version.

[hasufell]

@david-christiansen

I think these have a lot of really good potential,

I’m trying to think of a use case that can’t be covered by a multi-project setup via cabal.project.

There’s a minor use case of reducing code duplication across *.cabal files. Not worth the added complexity, imo.

Then there’s the use case of re-using modules in different libraries. Only interesting if they are unexposed… otherwise you can just factor out the common parts into a package and re-use them.

So, what’s the point?

[andreabedini]

@hasufell

I’m trying to think of a use case that can’t be covered by a multi-project setup via cabal.project.

The difference is for package authors. You share a package, not a project. So if you have multiple libraries you want to share “as one”, putting them in a single package can save some headaches. E.g. you can choose to use a single version number, rather then being forced to use a version number per each library.

[hasufell]

@andreabedini

E.g. you can choose to use a single version number, rather then being forced to use a version number per each library.

That sounds more like a drawback to me. Now I can’t version internals independently. Proper projects seem more flexible.

[Bodigrim]

I imagine multilibs were primarily designed with Backpack in mind, because this is where you end up with insane amount of libraries/signatures, which are tedious to split into separate Cabal files. But otherwise it’s a very niche feature and I’m not sure it is worth the incurred complexity on tooling.

[michaelpj]

Here are three reasons why I think you might want to use multilibs:

1. Multiple packages that are versioned together

One good test is indeed “does it make sense to version these packages independently?”. Often in a multi-package project you have a situation like this:

• You only ever use and test the packages together at the exact same version (same commit). Therefore you never test other version combinations.
• You have no interest in e.g. using CPP to ensure that the packages are compatible with a range of the other packages.
• The packages are tightly coupled, e.g. each new version of A requires the features from the new version of B, etc.

In this case you often end up using the same version for everything and advancing them in lockstep because that reflects the reality of how the packages are supposed to be used. Merging them into a single package with multiple sublibs does the same thing much more cleanly.

Examples:

• HLS and its plugins
• amazonka
• Many industrial codebases I’ve seen

As a side note, if you can version things separately then it’s tempting to try and actually do it properly. I have seen people drive themselves a bit insane trying to properly follow PVP for many packages in the same repo (this is part of why we switched to lockstep versioning in HLS recently).

2. Extra functionality that incurs more dependencies

Sometimes you want to expose just a little extra thing from your library, but that incurs a new dependency that you don’t want in the main library.

Examples

• Typeclass instances (for scale, try searching Hackage for “instances”)
• Test utility code
• Just little extras, e.g. the lsp-types package now has an extra public library that exposes the model used by the code generator, which isn’t even a dependency of the main library

At the moment you have to put such things in separate packages, which is quite annoying. With multiple public libraries you can just have the test-utility code in an additional library with extra deps.

Of course, in principle you might want to version these libraries separately. Maybe the Arbitrary instances don’t change very often and work with many versions of the base package. But: probably nobody cares or would gain anything from the finer-grained versioning.

3. Discoverability

In theory, it should be easier to discover a family of related libraries if they are packaged together. Consider again e.g. amazonka. Today the Hackage UI is not there, but in the future it would be nice to be able to pull up a package and go “oh, there’s also a library for quickcheck instances, that’s useful”.

---

I think some version of these conditions applies for a lot of packages.

[hasufell]

@michaelpj

Multiple packages that are versioned together

Yes, but you can just use == x.y.z bounds too.

@michaelpj

Extra functionality that incurs more dependencies

Right, something that you can do with real packages too.

@michaelpj

Discoverability

It seems to me this is an issue with hackage not having better grouping mechanisms and sounds like a rather odd motivation to introduce public sublibraries.

All in all, I don’t feel it solves anything significant. It seems merely a convenience feature (good), but at a high cost.

[sclv]

I would agree that in a strictly formal sense there’s no more expressiveness than with multiple packages that move in lockstep. I do think the other ergonomic benefits are quite nice, and hope this will be able to improve organization of various packages and codebases in the future…

[danidiaz]

I’m considering using public sublibraries for publishing .Internal modules.

.Internal modules clutter the package’s module listing in Hackage and make it more confusing to scan. It’s no big deal but I find it annoying (see the module list for text for example).

Also, putting them in a public sublibrary would make cabal build-depends: sections more informative, because it would be evident at a glance if another library depends on the internals or not.