New Hackage Server Features

After a pandemic-adjacent interregnum, hackage server is now again being maintained a bit more regularly with occasional feature updates and plans for the future.

In the current server release (which had to coincide with a migration to a new machine) there’s a number of smaller cosmetic improvements, as well as a few more noticeable features – largely stemming from deployment of the work in last summer’s GSOC. Here are a few things I wanted to highlight:

Candidates
The candidates feature still isn’t as polished as we’d like, but it is certainly in better shape – main package pages now list candidates, and users’ individual pages also include links to their package candidates. The candidates index page had been broken for some time and is now fixed. Candidates auto-delete as candidates on publish, and also the docbuilder runs on candidates! (However a nice UI for looking at candidate build-reports is still not implemented). Furthermore, the workflow for maintaining and releasing candidates is somewhat streamlined.

Builds and Badges
The docbuilder now runs tests and coverage as well when possible, and a package’s hackage page will display badges with status of tests as well as coverage and doc availability. Furthermore, now when you delete docs and reset the failure count, they will autorebuild – i.e. one need not request a manual reset of the docbuilder’s previously locally-maintained failure state.

Markdown
For rendering README.md and changelog files we now use John MacFarlane’s wonderful commonmark-extensions package to render markdown, allowing us to offer a fairly full set of so-called “github flavored markdown” including (sanitized) inline html, inline images, tables, github-style reference links, and some other niceities. Since many people author their md files first and foremost for display by github, this should be a nice improvement to existing display as well as open possibilities for package authors in the future.

Misc
For end-users (rather than bots) we also now have a better mechanism to attempt to always redirect http to https connections, which I know some people had been asking for for some time.

Final Note
The hackage server codebase is scary and there’s a plan on how to modernize it, although many details aren’t pinned down and there’s certainly no immediate timetable for when we’ll start work on it (or when it may finish). But even as is, there’s plenty of small low-hanging fruit to work on, which improves the user experience of all the many thousands of daily hackage users.

Interested people can take a look at the “good first issues” tag, or attempt to roll up their sleeves on plenty else as well. I’m around in the usual irc channels to lend a hand, and would be happy to help you get set up. Further, if you’d like to help in a more serious way, these issues are a place to just warm up a bit and get familiar.

Happy hacking, all!

18 Likes

Is the published source is the same as deployed?

I tried multiple times to hunt down that “install with cabal install” suggestion that pops up with every hackage link, but there’s nothing like that on github-published code.

Perhaps the page template was patched to include analytics tokens or something like that, but this begs the question of «what else is there we don’t have to see».

2 Likes

The published github of the central-server branch (not master) is precisely the same as deployed. This branch has certain tweaks to validation and page templates to make the server more suitable for use as the core instance.

I think the language you’re looking for is buried in a meta tag on a page template (so search doesn’t find it because there’s explicit nbsp characters instead of spaces). There’s a ticket to change it, which includes a link to the location. If you want to try to fix it up, please do! https://github.com/haskell/hackage-server/issues/912

1 Like

Ugh, it hides well from github indexer. I couldn’t find that with Install via and og:description queries.
Sorry for casting doubt.

Everything hides well from the GitHub indexer!

4 Likes

Or, that can be an effect of master being rather stale:

This branch is 145 commits ahead of master.

Master is not stale – its the active development branch. Those 145 commits are all the “tweaks” of the sort I mentioned – things which make sense for the main hackage server instance but not individual private ones. Prior to every hackage server deploy we merge master into central-server, then deploy the latter. I’m not sure if there’s a better workflow, or if we should just periodically rebase those tweaks or what (not really a git guru) but that’s the process…

1 Like

Hm… So, to patch out that cabal install (to use a concrete example) one would need to patch the “tweaks” branch?

It looks like some hackathon is in order to cherrypick all the possible tweaks into master and put in extension points for the remaining.

I think what @sclv said makes sense if you consider the fact that there is no “canonical” hackage server; you’re free to run one as you please. But for the one that sits at https://hackage.haskell.org, they may want their own set of “tweaks” to delineate that special application. That seems fine to me. Remember, Hackage Server is not a singular notion!

1 Like

But I don’t have “my” hackage server. I want to improve the one I use…

This is very cool, thank you to everyone involved in the release.

I don’t understand however why the docbuilder runs tests only … sometimes? E.g. in my rp-tree package, v 0.5 ran tests whereas 0.6 didn’t

Good question. Looks like we provide build logs for docs, but not logs for tests (though they are uploaded – it would be a nice PR to expose them). Looking on the docbuilder box, the test suite for rp-tree 0.6 failed because it could not execute hspec-discover. I suspect this is because the test suite requires cabal v2 builds and for various reasons due to some missing features in cabal, we can’t yet use v2-build for the docbuilder…