What's the current status of HLS cradle discovery?

tomjaguarpaw · December 2, 2023, 10:00am

Back in the past I used to have to write a hie.yaml to get HLS (through lsp in emacs) to work correctly. It always just contained

cradle:
  cabal:

More recently it seemed that I didn’t have to do that. Somehow it just worked out itself that the cradle should be cabal. I don’t think I’ve written a hie.yaml file by hand in at least a year. However, even more recently, HLS stopped being able to see modules from external packages (i.e. from non-boot packages). The first hit on Kagi is an HLS Github issues that explains that you have to write a hie.yaml file, but it’s from 2020.

What is the current status of this? Should I expect HLS to set up a cabal cradle for me by default? Last time I looked into this, cradle discovery had some heuristics about detecting .cabal or stack.yaml files to work out which cradle to choose automatically, but it was often flaky. Have I got the configuration wrong somewhere else?

The user experience around this is rather sad. There’s no indication at all what’s wrong. HLS just says it can’t find the modules in question, even though cabal build works without problems. I wouldn’t expect a new user to have the patience to resolve this issue. I’d expect them to just stop using HLS (or even stop using Haskell).

jaror · December 2, 2023, 10:14am

There’s an open issue about the bad error message:

github.com/haskell/haskell-language-server

Improve the error message for unknown modules

opened 06:23PM - 08 Jul 23 UTC

fendor

type: enhancement build tool: cabal

## Context When users add a new module to a cabal project, they are usually g…reeted with an uninformative `Multi Cradle: No prefixes matched` or something similar. We can identify three different cases and different behaviour: ### 1. Adding an other-module `Test.hs` to an executable component In an example project with only a `Main.hs` module, adding an other-module `Test.hs` results in the following error message: ``` Multi Cradle: No prefixes matched pwd: d:\Privat\Documents\programming\haskell\example filepath: D:\Privat\Documents\programming\haskell\example\app\Test.hs prefixes: ("app/Main.hs",Cabal {component = Just "example:exe:example"}) ``` The cause is that `implicit-hie` generates roughly this hie.yaml file ```yaml cradle: cabal: - path: ./app/Main.hs component: "example:exe:example" ``` Which essentially means, only map `./app/Main.hs` to the component `example:exe:example`. Clearly `./app/Test.hs` is not listed here, resulting in the sub-par error message from `hie-bios`. Note, `implicit-hie` doesn't handle overlapping source directories correctly in most cases, see https://github.com/haskell/haskell-language-server/issues/3606 for example. ### 2. Adding an exposed-module to a library component This generally works ok because `implicit-hie` generates `hie.yaml`s of the form: ```yaml cradle: cabal: - path: ./src component: "example:lib" ``` Essentially, maps for each source directory to the library component. So why does this work even if a hypothetical `Lib.hs` is not part of the library component? `hie-bios` doesn't care about that and just gives us the compilation options for the component `example.:lib`. Then haskell-language-server just takes the options, creates a unit, and adds `Lib.hs` to it: https://github.com/haskell/haskell-language-server/blob/feb596592de95f09cf4ee885f3e74178161919f1/ghcide/session-loader/Development/IDE/Session.hs#L841 Read the comment right above it for the justification. That's why adding modules to a library works "okayish" most of the time. ### 3. Adding a new module to a component with a simple 'hie.yaml' However, this all falls apart when we handwrite the recommended hie.yaml file: ```yaml cradle: cabal: ``` This basically runs `cabal repl src/Lib.hs` to find the compilation options. However, then `cabal` complains even in the case where it previously was working fine! ``` Failed to run ["cabal","v2-repl","src\\Lib.hs"] in directory "D:\Privat\Documents\programming\haskell\example". Consult the logs for full command and error. Failed command: cabal --builddir=C:\Users\Hugin\AppData\Local\hie-bios\dist-example-897af6579b5c0528e593e285a92d28a4 v2-repl --with-compiler C:\Users\Hugin\AppData\Local\hie-bios\wrapper-340ffcbd9b6dc8c3bed91eb5c533e4e3.exe --with-hc-pkg C:\ghcup\ghc\9.2.7\bin\ghc-pkg-9.2.7.exe src\Lib.hs Error: cabal-3.10.1.0.exe: Failed extracting script block: `{- cabal:` start marker not found ``` Naturally, if `Lib.hs` is not an exposed- or other-module of the library component, `cabal` cannot find it. In this case, the error message is terrible in particular, as it tries to interpret the target as a cabal script, which is not even close to the desired result. ## Solution We can identify at least two possible ways forward. One way, is improving `implicit-hie` to generate more lax `hie.yaml` files. I am in general against that approach. It will conceal the problem, and will just produce inconsistent views between the build-tool (cabal) and HLS. E.g. HLS reports no error or warning while `cabal build` straight up fails (also the package is rejected on hackage, etc..). The better approach is to interject these terrible error messages and provide useful, actionable diagnostics to the user. For example, for the first example, we should display a message like this: ``` Loading the module 'Test.hs' failed. It seems like it is not listed in your example.cabal file! Perhaps you need to add `Test` as an other-module to your executable named 'example' in example.cabal. If you don't know about other-module, yet, read this link <some-link> ``` It might not always be possible to provide precise information, but general information, perhaps with links to https://errors.haskell.org/, would be great already. To improve the error message further, there are a couple of hacky solutions we can think of. We can try to guess the component the module likely will be part of by looking at already loaded components, and comparing the importPaths. If the loaded component options have a unit-id (supplied by `-this-unit-id`), we can then lookup that unit-id in `dist-newstyle/cache/plan.json`, find the component and produce a more accurate error message. Fixing the issue programmatically is not something we want to do in this issue, we want to focus on purely better error messages. ## Implementation Roadmap 1. Find the location where the error message/diagnostic is shown to the user * Here hie-bios is called: https://github.com/haskell/haskell-language-server/blob/feb596592de95f09cf4ee885f3e74178161919f1/ghcide/session-loader/Development/IDE/Session.hs#L738 * That result is *eventually* turned into a [Diagnostic](https://hackage.haskell.org/package/lsp-types-2.0.0.1/docs/Language-LSP-Protocol-Types.html#t:Diagnostic) 2. Change the error message *only* if we are trying to load a 'Cabal' cradle https://github.com/haskell/hie-bios/blob/78d0cd2332e05c46b747b70ca69c0746209f440e/src/HIE/Bios/Cradle.hs#L200 3. Parse the error message, and produce a nicer diagnostics as discussed above, with mostly a high-level description of what's going wrong. * It might be sensible, to turn the generic 'CradleError' into an ADT describing the error in more details, but parsing is also fine for the first approximation 4. Try to guess the location of the module based on its proximity to already loaded components * Some relevant type https://github.com/haskell/haskell-language-server/blob/feb596592de95f09cf4ee885f3e74178161919f1/ghcide/session-loader/Development/IDE/Session.hs#L939 * `dist-newstyle/cache/plan.json` helps you map a `unit-id` to a component name. This issue tracks the effort for improving the error message.

And last time I asked @fendor wrote:

We aim to migrate away from implicit-hie, or at least generate smarter cradles in the future.

tomjaguarpaw · December 2, 2023, 10:16am

Ah, that explains it. It’s because I have a common stanza!

jaror · December 2, 2023, 10:18am

And sadly cabal now generates a file with a common stanza if you run cabal init, so a lot of beginners will run into this.

tomjaguarpaw · December 2, 2023, 10:20am

Yes, that’s exactly how that stanza got there in my case too.

fendor · December 2, 2023, 10:27am

It is kind of ironic that I am one of the people that pushed cabal init to use a common section but HLS can’t handle it by default…

It is currently the plan to fix this issue until the next HLS release.

tomjaguarpaw · December 2, 2023, 10:32am

Good to hear, thanks!

michaelpj · December 2, 2023, 1:05pm

A nice integration test would be to generate a project with cabal init and check that everything works.