[Well-Typed Blog] Making GHCi compatible with multiple home units

[adamgundry]

From GHC 9.14, GHCi will fully support Multiple Home Units, so you can load a whole multi-package Haskell project into a single GHCi session and use the REPL normally!

Making GHCi compatible with multiple home units

[chreekat]

Thanks for the implementation and the write-up! I have wanted this feature since day one. Once it’s fully integrated into the ecosystem, Haskellers will stare in amazement when we (who will be, by definition, old-timers) talk about how we used to work around the lack of this affordance.

Is there any budget for adapting Stack to benefit from multiple home units? Stack has a workaround that provides support for the same idea. Ripping that out and using these new foundations would be more robust.

From an ecosystem perspective, most Haskellers rely on Stackage. Even if they don’t use it directly or indirectly, Stackage snapshot curation is an important way for the ecosystem to adapt to new GHC/base versions. And snapshot curation, in turn, depends on Stack. (That’s the wrinkle that seems to catch people by surprise.) Including Stack when new core features are developed is the fastest way to get them in the hands of the whole ecosystem.

[mpilgrem]

stack ghci (aka stack repl) is the one part of Stack that uses GHC directly.

In principle†, I can see no barrier to: "if specified GHC <= 9.14 then ‘Stack do X’; otherwise, ‘Stack do Y’ ", as long as any changes (there may be none of note) to the user experience as between those two limbs are well-documented.

The ‘if specified GHC <= 9.14’ limb will have to stick around until Stack no longer supports GHC 9.14. (Currently, Stack supports GHC >= 8.4.)

† ‘In principle’, because, in practice, somebody has to be both willing and able to code the change. (If I am able, I am willing.) A ‘budget’ might increase the number of able people who are also willing.

EDIT: Corresponding issue created at Stack’s repository:

• Use GHCi's full support for multiple home units, from GHC 9.14.1 · Issue #6748 · commercialhaskell/stack · GitHub

[mpilgrem]

Two things that I did not follow from the blog post were:

1. This may be typographical: How does ghc --interactive -this-unit-id lib-mhu-example ... create a home unit graph with one of the home units being mhu-example-library? I was expecting that home unit to be named lib-mhu-example.

   Also, in the related diagram, the home unit is named lib:mhu-example not lib-mhu-example. I am confused: what is the name of the home unit that is being created?

   The GHC 9.12 User Guide states:

   As of GHC 8.0, unit IDs must consist solely of alphanumeric characters, dashes, underscores and periods. GHC reserves the right to interpret other characters in a special way in later releases.

   Will that change in GHC 9.14?

2. What are the arguments that need to be/may be included in each response file specified by the -unit option? In particular, how does the response file for one home unit specify that the unit has a dependency on another home unit? Is it by using the -package-id <unit-id> option? (I have seen the GHC 9.12 documentation on ‘the normal arguments’.)

I also have a third question about Cabal. The post refers to Cabal (the tool). Cabal (the library) also has a runhaskell Setup.hs repl command (although it is not documented in the Cabal User Guide). Am I correct to understand that this functionality is not coming to Cabal (the library)?

[mpilgrem]

Some other things that occured to me (which may be out of scope for the blog post):

Does it matter in what order the -unit options appear on the command line?

Related to that, if you have GHC flags/options in the various response files and other GHC falgs/options on the command line (before or after -unit options), how does that ‘work’ in terms of what gets applied to what?

[fendor]

@chreekat

Is there any budget for adapting Stack to benefit from multiple home units?

There is currently no budget planned to adapt stack to use GHCi’s multi home unit feature.
However, it would be great to teach stack about multiple home units and will try to answer any question that arises.

@mpilgrem

This may be typographical: How does ghc --interactive -this-unit-id lib-mhu-example ... create a home unit graph with one of the home units being mhu-example-library? I was expecting that home unit to be named lib-mhu-example.

Thanks for pointing this out, this name is a mistake and it should be lib-mhu-example.

@mpilgrem

Also, in the related diagram, the home unit is named lib:mhu-example not lib-mhu-example. I am confused: what is the name of the home unit that is being created?

The name of the home unit is specified by the tool, e.g. cabal invokes GHC with the argument: -this-unit-id ... which is used as the name of the home unit. In the diagram, we took the liberty to use the component name, as users are usually more familiar with components than units.
To be precise, the home unit graph would only use what is given in the -this-unit-id flag. We will not change the format of unit ID, so lib:mhu-example is not a valid unit id. It is only used as an attempt to make it more approachable.

@mpilgrem

What are the arguments that need to be/may be included in each response file specified by the -unit option? In particular, how does the response file for one home unit specify that the unit has a dependency on another home unit? Is it by using the -package-id <unit-id> option? (I have seen the GHC 9.12 documentation on ‘the normal arguments’.)

Yes, precisely, the flag -package-id <unit-id> is used to express the dependency.

@mpilgrem

I also have a third question about Cabal. The post refers to Cabal (the tool). Cabal (the library) also has a runhaskell Setup.hs repl command (although it is not documented in the Cabal User Guide). Am I correct to understand that this functionality is not coming to Cabal (the library)?

The blog post is particularly talking about GHCi support, independent of Cabal, cabal-install or stack.
I am not aware of any efforts of updating Setup.hs repl to use multiple home unit with GHCi.

@mpilgrem

Does it matter in what order the -unit options appear on the command line?

No, it does not!

@mpilgrem

Related to that, if you have GHC flags/options in the various response files and other GHC falgs/options on the command line (before or after -unit options), how does that ‘work’ in terms of what gets applied to what?

Currently, we first parse all cli arguments into DynFlags, including -unit arguments. We then use the initial DynFlags (e.g., the flags given at the top-level) and then process arguments in the -unit response files.

As an example:

ghc -fwrite-ide-info -unit @a -unit @b

then the option -fwrite-ide-info is prepended to the arguments given in @a and @b.
Perhaps counter-intuitively, the order doesn’t matter.

ghc -unit @a -unit @b -fwrite-ide-info 

is the same ghc invocation, the -fwrite-ide-info option would be prepended to the options @a and @b.

[mpilgrem]

Many thanks for those responses. May I ask another thing, as follows:

Imagine that a user has a package (my-package) that provides two executable components (my-app-A and my-app-B), the source code for each of which is in a file Main.hs (in different directories).

What does the user experience, once in GHCi, if they load the whole of my-package into GHCi using -unit for each of the components? How do they distinguish one main from the other? Does -this-package-name <unit-id> somehow come into play?

[fendor]

The GHCi UI hasn’t been 100% updated to let the user handle such ambiguities easily.
For example, you can’t use :module Main in these cases, as the ModuleName is ambiguous.
That’s a short-coming we want to resolve by letting the user specify the <unit-id>.
For example, :module <unit-id>:Main to import Main from the home unit <unit-id>.

However, as you point out, you can use PackageImports to import the modules via import "package-name" Main, so yes -this-package-name flag is one way to handle these ambiguities.