Recommendations on how to create Koans to learn a library

Learn
#1

I want to improve the range of documentation I have for my some of my libraries. One kind of documentation I’m looking into is “Koans”. As I understand it, these consist of many small consecutive problems where the learner is required to fix a piece of broken code. Koans exist for different programming languages, and also for some frameworks. I can imagine they make for a fun way to learn a language or library ecosystem.

Therefore I’m looking into creating koans for some of my libraries. I have these requirements:

  • The koans should have at least a way of ordering them linearly (ideally even hierarchically, to sort by topics)
  • A failing koan should point the learner at a specific file to inspect, with a realistic error message they might encounter in real world
  • Type errors and typed holes should be possible in a koan
  • Test failures should be possible in a koan (It is fine that these would point to the failing test case instead of at the file where the error to fix is)
  • There should be a way to run the koans, which only shows the error message of the smallest/first failing koan, and not all of them at the same time. (-freverse-errors goes some way, but not all the way)
  • Stretch goal: There should be a way to run a specific koan (ignoring all smaller ones)
  • It should be possible to provide the solutions to all the koans, and they should be checked on CI. (It should be checked that the solutions actually compile and test fine.)

Is there anything that achieves this? Tooling, a template, or even an example koan structure that I could mimick? Does anyone have ideas how to do this?

I had some vague ideas what one could do, where I’d be interested about your feedback:

  • Create a cabal package (or executable) for every koan, consecutively numbered
  • Create a cabal.project including all the koans

To run the koans, either:

  • cabal test all with -freverse-errors, and tell people to solve the last error first
  • cabal test 23-foo-koan, and make sure that the package n+1-foo-koan depends on n-bar-koan. Possibly, this dependency can be placed under a cabal flag so that one can work on n+1-foo-koan without having to solve n-bar-koan first.

To ensure that there are solutions and they work, I thought about maybe using a separate branch, and then testing that on CI. But how do I make sure that I only need to make a particular small change to get to that solution? I know, that’s kind of a fuzzy constraint.

3 Likes
#2

Some findings so far:

  • Test suites can’t depend on other test suites it seems, so my second idea with dependencies between the koans is a bit harder to realise, I need an internal library and a test suite for every koan
  • One might be able to automate this koan idea nicely by writing a custom Setup.hs, but this seems a daunting task to me
  • If the test suite succeeds it captures all IO, it seems. So I can’t display an encouraging “well done” message, or the result of the program when the learner has solved the koan. Maybe I should just use executables instead.
#3

If you want to have a look how it’s going: GitHub - turion/rhine-koans: Koans to learn rhine

Currently, I’m a bit unsure how I should go about with putting the problems (which don’t compile or don’t test) and the solutions (which compile and test) in the same repo. My current setup is having a main branch which contains the problems (so when checking out, one is immediately presented with the problems without seeing the solutions), and a solution branch where every Koan is solved. This poses a few challenges:

  • I have to lint both branches
  • I have to check out the solution branch on CI and test it
    • But a push to solution doesn’t trigger the ci, so I need to push it first, or rerun CI
  • I have to make sure (currently with a CI check) that main is always merged into solution, and that the diff between the branches is only in the koans, not anywhere else (i.e. solving the koan is possible by only touching the koan’s code, not its test or so)
  • I must not accidentally merge the solution into the main branch

I’m considering not separating solutions and problems via git branches, but instead with a cabal flag. Basically, the flag would just change main-is from the koan file to some Solution.hs.