New Hackage Server Features

Announcements
#18

The 01-index.tar can be incrementally downloaded because it is append only and provides incremental merkle-hashes internally – this is the design of hackage-security. I’ll grant there’s no libraries in other languages that do this, but that’s the reason behind its design.

As a more general point – the reason there’s no api for “latest recommended package versions” as such is that this isn’t a particularly natural thing to ask – there’s really no “recommended” versions, just many versions, which the solver may pick any of, and some which are deprecated, and a rare few which are “preferred”, a very strange concept that was introduced mainly as a solver hack, and only used in specialized corner cases.

Since “whole repository” listings are expensive, the hackage api provides relatively few of them, and only those that are necessary as entry points or for its functioning.

For any individual package, there’s a much more straightforwardly consumable listing of preferred/deprecated info at e.g. https://hackage.haskell.org/package/bytestring/preferred.json

#19

Yes, that tar data type can be incrementally synced between servers - is what I also was wondering. But to incrementally sync something like rsync/zsync support is needed. Or maybe/probably tar can do it by itself (it is at least what it was designed for in the first place, for magnetic tapes).

Thank you, it was pleasant having a dialog with you.

#20

One can incrementally fetch the tar with http and range requests directly – that’s what the codebase inside cabal does, if you look through it!

#21

New hackage redeploy up with some new features. Now language extensions up through ghc 9.2 should be allowed in cabal files, as well as cabal files up through version 3.4 of the cabal spec.

Most notably there’s a very slick and powerful new search/browse interface thanks to
Janus Troelsen (Rewrite search/browse pages by ysangkok · Pull Request #1021 · haskell/hackage-server · GitHub).

Additionally, Hecate finished up the longstanding PR for adding a json api to package pages (Package JSON API (replacement of #810) by Kleidukos · Pull Request #996 · haskell/hackage-server · GitHub)

And finally Andres Abel did a ton of work in just general cleanup of CI and tests as well as bringing the dependencies up to date and getting everything functional on recent ghcs.

Thanks to all who contributed, and I hope everyone enjoys the new features!

20 Likes
#22

Thank you so much to the people who keep this critical piece of infrastructure ticking over!

3 Likes
#23

Updating to announce a whole bunch of new features in the latest redeploy, with a few yet to come. This release includes a bunch of work from GSoC students, as well as a few other big-ticket items, and some various small fixes.

The biggest highlight is reverse-dependencies, which are finally here after years of attempts. Janus Troelsen re-thought the problem space to make an order-of-magnitude improvement in memory footprint, which means that we can finally provide this much requested feature.

A fun and noticeable change is that hackage now has a “night mode” color scheme enabled when your browser is set to request it. Thanks to Peter Becich for that!

Also included is a ton of GSoC work from Alias Qli and Ondřej Kubánek (though the latter’s work on a “hackage-rank” feature has not yet been merged). Notably, hackage now has an improved user account management page (reach it from User accounts | Hackage). You can now self-update your name and email address, and even more excitingly, opt into notification emails. You can now request notification when a maintainer is added to a package you maintain, a metadata revision is performed, a tag is proposed to be added, or when a docbuilder run finishes. For now, you’ll need to opt-in explicitly, since its not clear if existing hackage users will want to start getting a bunch of new emails all of a sudden.

Along with smaller bugfixes, and the usual dependency bumps and CI fixes (thanks Andreas Abel and Peter for all your help on that!) here are some other changes of note:

  • improvements to the sitemap to help guide google search (and searchbox metadata)
  • improvements to rendering of markdown
  • allows syntax from cabal 3.8
  • maintainers can disable tests being run by the docbuilder
  • display logs for docbuilder-run tests
  • Updates to accepted licenses (allowed both from osi and fsf lists)
  • better warnings on deprecated versions
  • maintainers listed on package pages
  • captcha for user registration (should reduce spam registrations we get hammered with)
  • remove filtering 00-index for cabal version < 2.0 hack (not a big deal, but makes some weird corner cases less weird)
  • Add lastVersion in browse listings
  • quickjump now works on candidate pages
  • paging enabled for recent uploads and revision pages

Other various contributors to this release are Hécate Moonlight, Matthew Pickering, bodigrim, Gautier Di Folco, and Levi Butcher

36 Likes
#24

Thanks a lot for this deployment!

2 Likes
#25

Amazing stuff! Thanks very much to everyone involved. I particularly like the reverse dependencies and night mode.

1 Like
#26

This is great stuff - thank you! I’m looking forward to giving the updated package candidates stuff a spin, and the reverse deps.

#27

Just one last thing voice

As a final note, I just realized while updating the docbuilder that while it is still somewhat janky, and the UI is lacking, I believe that it now builds documentation for candidates!

#28

Is there any way to opt out of Hackage’s dark mode? I like dark modes, all my desktop applications implement a dark mode, as are most of the websites I visit, but Hackage’s dark mode is incomplete and hinders readability in some places.

Screenshot 2023-01-06 at 16-28-28 pg-transact
Screenshot 2023-01-06 at 16-28-28 pg-transact1920×1040 110 KB

and

Screenshot 2023-01-06 at 20-48-49 colourista
Screenshot 2023-01-06 at 20-48-49 colourista1920×1040 139 KB

1 Like
#29

Chrome at least has a setting for it: How to Enable/ Disable Dark Theme on a Per-Site Basis in Google Chrome | Beebom

Also, if anyone would like to help improve the stylesheet to handle the missing cases for hackage’s dark mode, that would be very appreciated!

#30

My mistake on the CSS oversights, here are some fixes:

How do you enable Quick Jump on Hackage running locally? I am trying to test this: [WIP] Night mode quickjump fix by peterbecich · Pull Request #1163 · haskell/hackage-server · GitHub

Thanks

2 Likes
#31

Good catches! Peter jumped in and patched up the issues you found. The fixes are now live, assuming you force a reload of the css. We still will need to polish the dark mode a bit going forward, but the most problematic stuff is at least all solved.

1 Like
#32

Fantastic, my thanks to the both of you.

2 Likes
#33

Edit: deleting this post because I just realized it was a response to a comment from over a year ago and not worth reopening.

Discourse confuses me by highlighting new-to-me discussions as “recent” just because somebody posts a new comment on it. :sweat_smile: (Maybe we shouldn’t resurrect old threads very often)

#34

Is there any way to place the thread’s details:

Screenshot 2023-01-07 at 19-51-17 New Hackage Server Features - Announcements - Haskell Community
Screenshot 2023-01-07 at 19-51-17 New Hackage Server Features - Announcements - Haskell Community699×56 14.7 KB

at the very beginning, so one doesn’t have to scroll through the original post just to see it?

#35

What’s the right setup for cabal-install < 2.0 now? I have some CI jobs with GHC 8.0 / Cabal 1.24 (coming with an old, 32-bit Ubuntu), which fail after the last changes to Hackage server, e. g., Update changelog for v0.11.4.0 (#534) · haskell/bytestring@0602eab · GitHub.

Upd.: After reading remove cabal > 2.0 restriction on files in the 00-index file? · Issue #1063 · haskell/hackage-server · GitHub I assume the answer is "cabal-install < 2.0 no longer works". That’s fine, I can update, just would be better to be a bit more explicit about it.

#36

Yep, you got it. I just sort of decided that there’s no need to support cabal < 2 – even older ghcs are still supported by newer cabal versions, and the last ltses which shipped with it seem about to EOL (cabal 2.0 was released in 2017). And meanwhile the occasional confusion about the 00-index not containing all packages seems to come up far more often, especially as newer versions of cabal files become more frequent on hackage.

#37

Bumping this thread to announce a new hackage redeploy. Some more notable changes:

  • Email notifications for revision upgrades – opt in to that and other useful notifications from your account management page linked from User accounts | Hackage
  • Bugfix for nonupdating “latest version” in package browse.
  • JSON endpoint for top downloaded packages
  • improved check for changelogs and readme files
  • uses cabal 3.10 – this should mean support for uploading multilibs is enabled!

Thanks to @andreasabel @peterbecich @Kleidukos @janus among others for the patches.

21 Likes