To grow the Haskell user base one area to improve is documentation. My impression is that while Haddock is great for reference documentation it often leads to scattered and hard to navigate guides. For example, one important Persistent guide page is buried in Database.Persist.Quasi.
I would rather like a “companion site” (maybe there’s a better term?) like many Rust projects have that serves as an entry point for the library, shows common developer tasks and links to the reference docs. One good example is the restructured Welcome to the Cabal User Guide — Cabal 3.13.0.0 User's Guide.
A few reasonable requirements could be
Easy to setup (ideally not more than a single static executable)
Doc source file can live next to library source code.
Markdown syntax (simple, and most widespread)
Haskell examples can be embedded and get type checked
Easy cross-references to Haddock.
Nice to have: Image and Math embeddings, Versioning
Can you recommend such tool that could work across the whole Haskell ecosystem?
This reminds me of @Kleidukos’s talk at the 2024 Haskell ecosystem workshop on documentation. There is a blog post which briefly covers the ideas discussed in the talk:
I don’t know if there are recordings of the talk itself.
I can’t remember in which package I saw it first, but I like the idea of having a Tutorial module in packages whose sole purpose is to have a user guide right on Hackage.
Here’s an example for a package of mine, javelin, which has a documentation-only module, Data.Series.Tutorial.
The advantage of doing it this way is two-fold:
It is trivial to use doctest on such a module, just like you might already be doing on your codebase, which ensures that the tutorial is always correct;
You can easily link to reference documentation right in the tutorial, since it’s based on Haddock.
I agree that having something more than reference documentation is needed. I stumbled upon this documentation structure framework a while back, and I now recognize that user guides / tutorials are extremely useful.
My fear is splitting the reference documentation (e.g. on Hackage) from the user guide (e.g. on some custom domain). In practice, I usually consult both types of documentation simultaneously.
I really like this approach since you can load it on the REPL and try stuff out like examples, and from a maintainers perspective the code in such module is always compiling.
@jaror@LaurentRDC@emlautarom1 This diataxis framework is exactly what i have in mind. I really like this documentation module(s) idea, because it ticks all the boxes except Markdown syntax for me and is ready to be used and promoted today. Do you have an opinion how to structure multiple shorter guides and tutorials such that the sections become more prominent in Haddocks content overview? I was thinking about a top-level Documentation module, e.g.
applied to Javelin
Doc
Doc.Guide
Doc.Guide.How-To-Create-Series
Doc.Guide.How-To-Access-Series-Data
Doc.Guide.How-To-Aggregate-Series-Data
Doc.Tutorial
...
...usual reference doc for real code modules
@Kleidukos What made you reach for for mdbook? The doc module approach supports versioning, which i imagine can be fiddly to setup. Do you think the Haddock maintainers would be open to add Markdown support at some point? Maybe at least a documentation top-level module could become a first-class thing (like e.g. being shown as the first content element).
Ah, i wasn’t aware. That opens up a nice migration path. First, manually call Pandoc before Haddock. Later, Haddock may support Markdown out of the box with Pandoc as a library.
A convention is what i’m rooting for here. For the many overloaded single-maintainer libraries we have in the Haskell ecosystem at the moment, the friction must be minimal to be adopted.
What i like about the Rust ecosystem is that there’s this really nice convention of having a small usage example of a library on the entrypoint page (Github Readme, crates.io) and quite often a companion page created with the easy-to-setup mdbook tool.
As i have learned from this thread the latter could be achieved with a low-hanging fruit of existing tools in Haskell, e.g. Haddock, doctest and doc modules. It may be just missing a few structuring and setup recommendation, which i am searching for here.
Regarding examples, i plan to try a simple setup out with Persistent (the maintainer is open to restructure the guide sections). I already pushed e.g. Cabal documentation to adopt this Diataxis structure that @LaurentRDC mentioned, which seems to work out great; except using Asciidoc and Readthedocs hosting in my opinion come with too much friction to be adopted ecosystem-wide.
Do you think the Haddock maintainers would be open to add Markdown support at some point?
I personally will work towards having something better than markdown, which has less footguns and enables real error messages instead of the “produce HTML at all cost” mentality of Markdown. Maybe something inspired by Djot, or more probably something inspired by Lean4’s Verso.
Maybe at least a documentation top-level module could become a first-class thing (like e.g. being shown as the first content element).
References are not guides. We shouldn’t try to fit more things inside modules. Modules are not the endgame of Haskell documentation, and things like mdbook should be doable from within Haddock.
Regarding the talk I gave at the Haskell Ecosystem Workshop of 2024, here are the slides: Haskell and its documentation. I encourage you to read them.
By and large the biggest issue with Haddock markup is that it’s not Markdown I’m not quite sure that introducing yet another markup language can solve more issues than create.
I don’t feel like we should add more modules just for documentation.
The first thing that came to my mind is the duckling package Facebook made a while ago that in itself is pretty useful, but has an entire snap dependency because one of the example executables uses it.
IIRC this did end up in my dependency stack, even though I never used it. (not sure if it’s better nowadays)
But the idea of adding more time to compilation, when documentation has nothing to do with compilation doesn’t sit well with me.
I do like the mdbook setup that @Kleidukos showed in text-display.
Haddock is already something like that, but there is also the possibility of literate Haskell. One of my projects consists entirely of .lhs files with one main LaTeX file that
imports all the modules that are exported in the Cabal file
can import even more hidden modules to give more examples
Instead of LaTeX one could possibly also use Pandoc to produce other kinds of documents, but LaTeX is effortless. Major downside: The source becomes much less readable and many editors have problems with syntax highlighting two languages in one file.
Could such modules be part of a separate compilation goal that’s off by default?
I agree it wouldn’t be very nice to get an extra huge dependency just because the examples show how easy it is to integrate with some third party library or whatever (which can be super useful in a howto guide! But you don’t want to actually compile that example on every CI build of some app where this well-documented library is perhaps deep in your dependency tree).
So if one is to have a convention of using FooLib.Tutorial modules, I guess there also has to be a strong convention of making their compilation default-off.
If not, the extra-doc-files trick looked neat. It doesn’t give the compile-check on tutorial examples, but I’m not sure that’s worth the cost if everyone has to do it on build.
I looked into adding Markdown support to Haddock once: Support Markdown syntax via `commonmark-hs` · Issue #794 · haskell/haddock · GitHub It looked to me like Haddock’s native format is richer and you’ll have to make arbitrary choices on how to map Markdown to Haddock’s IR, which felt unsatisfying. That said, I think Markdown should be supported, because it’s a reasonable expectation from anyone coming to the language today.
Thanks for the additional comments. It’s great to see a shared understanding that improving the documentation tooling is worthwhile.
Regarding documentation modules i learned that they are not the right approach, because they open the possibility to pull in unintended additional (example-) dependencies. So something like mdbook as part of Haddock seems to be the way to go.
What i am missing is consensus on the documentation format. If we were to decide to choose Markdown, initiatives like @artem may be revived and the ecosystem could start producing guides in this format. @Kleidukos do you do you have a roadmap when such a decision can be made? Do you work as a Haskell Foundation representative on this topic?