From a developer’s perspective one of those basics I need is a watcher that reloads rapidly on any change I make on the code base and reruns all the tests. I dare to say it’s standard for many to develop in such a fashion.
Yet it was surprisingly long and difficult until I finally learned the right command:
Of course this is a special cabal repl call using ghcid.
Why isn’t this a builtin command within cabal?
What does it take to make it one?
I mean, there is value in language extensions but I feel what people need is stuff like this. Basic things that simplify their development experience.
I hold off from filing an issue of this before understanding the landscape properly. An issue should not be a discussion. It should be a complete requirement.
(Also, I was surprised to see that there is not really a category for this one this discourse).
For the long haul: Which other basic things bring high practical value to the common developer yet don’t exist for Haskell?
I mean, has anyone compiled a table of current languages and their tooling comparing it with Haskell’s?
I think it’s a good idea. The Cabal project has a Proposals process, so I’d encourage you to draft one. I’d be in favour of more integrated tooling within cabal-install, especially since it puts us less at risk of suffering from unmaintained third-party tools (like ghciwatch).
Funny you should ask. I’ve been wondering the same thing.
FWIW here’s my current default invocation, suitable(?) for multi-package projects, including some cargo-culting from a plain ‘ghcid’'s invocation of ghci:
Yeah, this is one of the more embarrassing ecosystem gaps IMO. I have so many variations of horrible scripts using ghcid and ghciwatch (both of which have their own awkward bugs and limitations), for stuff that really should be pretty simple.
There is quite a big design space, and probably a lot of fixes across various parts of the stack. Plus discussions to be had about which logic belongs in Cabal versus GHCI versus an external tool. It’s all doable. It just needs someone motivated to take it on. Adding support in Cabal itself has been proposed at least once here, so that could be the place to start.
What I personally want is to be able to go in to a big multi-component Cabal project and just run cabal watch 'some compound expression' and have that expression get re-evaluated precisely when GHC detects that its dependencies have changed. (I’d previously wanted a more REPL-based approach, with a :watch command in GHCI, but the main reason for that was not wanting to wait to reload the whole session just to change the expression I want to watch, and that concern should be becoming obsolete due to the work on serialisable bytecode and making Cabal spend less time unnecessarily reconfiguring stuff.)
FWIW, I’m currently working on a file watching tool specifically for Obelisk, with separate REPL processes for the backend and the Wasm frontend. I’ll try to upstream what I can to GHCI etc.
Haskell Foundation tech proposals are the correct place to request funding for such things (after you’ve of course cleared the details with the Cabal project)
The difficulty with a tool of this nature is that it simply does not scale. You eventually need to build only a selection of the module graph instead of the whole thing (though with the work we’ve done on improving GHC’s memory use, we’re able to load ~30k modules in GHCi, though the GC pauses and reload times are pretty punishing when you’re at 50-100GB of RAM). This is where the --repl-no-load flag can save the day.
Eval comments let you selectively run tests, but it doesn’t scale nicely due to the same GC pauses that make reloading slow also making test running slow.
I’m extremely excited to eventually be able to open source and release all of the work we’ve been doing on the buck2 build system integration with Haskell. Then the magic invocation will simply be buck2 test //... to answer what you’re looking for here - between test caching, module-level parallelism between packages, and test execution being possible before the whole package is compiled, this will significantly outperform any cabal based workflow.
I think the biggest one is that buck2 has a notion of a “target graph” (analogous to the package graph in Cabal terms) and “action graph” (analogous to the module graph in GHC terms). In buck2, we can say “This target has a test suite which depends on these target libraries,” and then it decomposes the test suite target into the actual actions - “build test module” and “run test module” are separate “actions” and you can start an action once its action dependencies are satisfied, even if the target dependencies are not complete yet.
This would allow you to begin running the test suite for a library before the library is done compiling. It also allows you to begin building a library before all of it’s dependent libraries are done.
In Cabal/GHC’s, case, we need cross-package module-level parallelism. I’ve proposed this before, but I’m unsure how far away that is from being realized.
The last Well-Typed activity report mentioned this (“Cabal modernisation” section), and the latest writeup seems to be in this GitHub issue. My reading is that it isn’t imminent, but we’re moving in the right direction. Which is exciting.
(2) just isn’t gonna happen idt. iirc the justification is that a cabal file in isolation should fully describe a haskell package. with globs, you need IO and the source in-hand
I tried tricorder and found a few showstopping bugs, but I think they have already been fixed.
Meanwhile I had an agent choose the latest and greatest repl speed-up options available with ghc >=9.10.3, and I used it to write a cabal-project-aware ghcid wrapper:
#!/usr/bin/env bash
# -T/-r evaluate an expression, which needs actual code. Only a plain
# check-loop can get away with typecheck-only.
code_opt=-fno-code
for arg in "$@"; do
case $arg in
-T*|--test*|-r*|--run*) code_opt=-fbyte-code ;;
esac
done
repl_opts=(
"$code_opt"
-fprefer-byte-code # TH splices load bytecode deps, skipping codegen
-fno-break-points # no debugger under ghcid; skip breakpoint ticks
-j"$(nproc)"
-fdefer-diagnostics # group diagnostics after the parallel-load noise
-ferror-spans
-fdiagnostics-color=always
-fwrite-if-simplified-core # restart from interfaces, not source
-fno-hide-source-paths # ghcid's output parser needs full paths
+RTS -A64m -RTS
)
# A separate builddir keeps repl artifacts out of the
# real build's dist-newstyle, so it doesn't compete with `cabal build`.
# THIS SHOULD BE DEFAULT.
cmd="cabal repl --builddir dist-newstyle-ghcid --enable-multi-repl all"
for opt in "${repl_opts[@]}"; do
cmd+=" --repl-options=$opt"
done
# Restart ghcid if any of the cabal files change.
mapfile -t cabals < <(git ls-files '*.cabal')
restart_args=()
for cabal in "${cabals[@]}"; do
restart_args+=(--restart "$cabal")
done
ghcid "${restart_args[@]}" --command "$cmd" -o errors.err "$@"
There are still bugs in GHC’s multiple-home-units that break this workflow on some projects I work on, but the future is looking better.
A single repl that has all cabal components loaded at once has been something I’ve wanted since the first month I started using Haskell professionally. Stack has provided it in a hacky form since ~2015. Excited that it might become the ecosystem-wide-default soon.
It builds libraries using ghc --make like Cabal, so it doesn’t have the cross-package parallelism that the Tweag/Mercury Buck2 integration does. But it does support enough that I was able to build Glean with it, and it’s vastly preferable to rebuilding with Cabal.