GHC's -fplugin - Any changes between 9.4.7 and 9.6.2?

There is an issue on Stack’s repository that has me stumped, so I am putting it to a wider audience. Any advice, gratefully received. I’ve not spotted anything in the GHC 9.6.1/9.6.2 or Cabal 3.10.1.0 change logs that seems to go to the following.

A single-package project has src/A.hs:

{-# LANGUAGE DataKinds #-}
module A where

import Polysemy
import Polysemy.State
import Prelude

f :: Sem '[State Int] ()
f = pure ()

and Cabal file:

cabal-version: 1.12
name:           problematic
version:        0.0.0.0
build-type:     Simple

library
  exposed-modules: A
  hs-source-dirs: src
  ghc-options: -fplugin=Polysemy.Plugin
  build-depends:
      base
    , polysemy
    , polysemy-plugin
  default-language: Haskell2010

With GHC 9.4.7/Cabal 3.8.1.0 it builds: stack --resolver lts-21.12 build.

Simply switching to GHC 9.6.2/Cabal 3.10.1.0 it fails (stack --resolver nightly-2023-09-22 build), at the point that Cabal (the tool) tries to register the library with GHC. The error is (with --ghc-options -v, for extra GHC information):

problematic> copy/register
Installing library in C:\Users\mpilgre\Documents\Code\Haskell\psymy\.stack-work\install\834933cc\lib\x86_64-windows-ghc-9.6.2\problematic-0.0.0.0-DvlZTrIQMfKBSnzzdqzDM
Error: Cabal-simple_9p6GVs8J_3.10.1.0_ghc-9.6.2.exe:
'C:\Users\mpilgre\AppData\Local\Programs\stack\x86_64-windows\ghc-9.6.2\bin\ghc-9.6.2.exe'
exited with an error:
Glasgow Haskell Compiler, Version 9.6.2, stage 2 booted by GHC version 9.4.3
*** Deleting temp files:
Deleting:
*** Deleting temp dirs:
Deleting:
<command line>: Could not find module `Polysemy.Plugin'
Locations searched:
.stack-work\dist\01de9cfc\build\Polysemy\Plugin.hi
.stack-work\dist\01de9cfc\build\Polysemy\Plugin.hi-boot

Why GHC 9.6.2 (and, I assume, not GHC 9.4.7) is looking in .stack-work (and only there) is a mystery to me. ghc-pkg 9.6.2 in the Stack environment, for example, knows that Polysemy.Plugin is to be found in the snapshot (extracts):

> stack --resolver nightly-2023-09-22 exec -- ghc-pkg find-module Polysemy.Plugin
C:\Users\mpilgre\AppData\Local\Programs\stack\x86_64-windows\ghc-9.6.2\lib\package.conf.d
    (no packages)
C:\sr\snapshots\3a7b44d0\pkgdb
    polysemy-plugin-0.4.5.1

C:\Users\mpilgre\Documents\Code\Haskell\psymy\.stack-work\install\834933cc\pkgdb
    (no packages)

Hi!

Today, I experienced a similar problem trying to switch from GHC 9.4.6 to 9.6.2 in our Bazel rules for Haskell.

Since Cabal as a build tool also calls in to GHC, I wonder if that might be related…

I found that GHC was bailing out in the HaskellLinkBinary step which is an action run after the compilation has finished and it just needs to link object files. In this phase it should not actually care about plugins at all, and indded GHC 9.4 does not care about the -fplugin= flag – I did remove it, I set it to some gibberish, it would always link the binary successfully.

GHC 9.6 on the other hand tries to find a file with a .hs, .lhs, .sighs or .lsighs extension it can load.

The workaround I came up with was that I remove all -fplugin= flags from ghc’s command line in the HaskellLinkBinary action.

I raised GHC issue #24042 in connection with this.

I think it is the same issue we saw here `register` fails if `-fplugin=Foo` is used in `ghc-options:` · Issue #9375 · haskell/cabal · GitHub

There was an old workaround for an issue with ghc-cabal (which is now gone). The workaround was to not pass -package-db to GHC when calling --abi-hash. It used to be unnecessary but with GHC 9.6 the plugin initialisation happens earlier than before, and this means we do need to pass -package-db.

We removed the workaround in Revert #3639 (Don't pass -package-db and -package flags to --abi-hash) by andreabedini · Pull Request #9384 · haskell/cabal · GitHub only two weeks ago. We could do a minor release but our “release pipeline” is a fair bit of manual work … :-/

2 Likes

Thank you for the pointer, Andrea!

I tried to also pass the appropriate -package-db and -plugin-package-id to GHC’s command line. But apparently this causes it to try to create a dynamic library instead of a binary:

*** systool:linker:
*** Linker:
bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_haskell/haskell/cc_wrapper-python \
  -Wl,--no-as-needed -lm \
  -o /tmp/ghc385754_0/libghc_1.so \
  bazel-out/k8-fastbuild/bin/tests/binary-with-plugin/_obj/binary-with-plugin2/Main.o \
  -shared -Wl,-Bsymbolic -Wl,-h,libghc_1.so \
  -L/root/.cache/bazel/_bazel_root/4e44c3743109bc8d0c5662725969652f/external/rules_haskell_ghc_linux_amd64/lib/lib/../lib/x86_64-linux-ghc-9.6.2 \
  -Xlinker -rpath \
  -Xlinker /root/.cache/bazel/_bazel_root/4e44c3743109bc8d0c5662725969652f/external/rules_haskell_ghc_linux_amd64/lib/lib/../lib/x86_64-linux-ghc-9.6.2 -lHSbase-4.18.0.0-ghc9.6.2 -lHSghc-bignum-1.3-ghc9.6.2 -lHSghc-prim-0.10.0-ghc9.6.2 -lgmp -lc -lm '-fuse-ld=gold' -Wl,-no-as-needed -Wl,-z,relro,-z,now -B/usr/bin -pass-exit-codes '-lstdc++' -lm -pthread -Wno-unused-command-line-argument

and I receive this error from the linker:

/usr/bin/ld.gold: error: 
bazel-out/k8-fastbuild/bin/tests/binary-with-plugin/_obj/binary-with-plugin2/Main.o: 
  requires dynamic R_X86_64_PC32 reloc against 'ghczmprim_GHCziCString_unpackCStringzh_closure' which may overflow at runtime; recompile with -fPIC

(with GHC 9.6.2)

It’s not clear to me if that is to be expected. How is GHC determining what type of target to build? According to the docs it should only run the linking phase when the source is an object file, so compiler plugins (that run at compile time) should have no effect at all during this phase…

Should I report this as a regression to the bug tracker?

FTR, here’s the complete set of arguments that are passed to GHC:

-pgma
bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_haskell/haskell/cc_wrapper-python
-pgmc
bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_haskell/haskell/cc_wrapper-python
-pgml
bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_haskell/haskell/cc_wrapper-python
-pgmP
"bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_haskell/haskell/cc_wrapper-python -E -undef -traditional"
-optc-fno-stack-protector
-pgml-supports-no-pie
-pgmlm
/usr/bin/ld
-optlm
-r
-optl-fuse-ld=gold
-optl-Wl,-no-as-needed
-optl-Wl,-z,relro,-z,now
-optl-B/usr/bin
-optl-pass-exit-codes
-optl-lstdc++
-optl-lm
-XStandaloneDeriving
-threaded
-DTESTS_TOOLCHAIN_COMPILER_FLAGS
-XNoOverloadedStrings
-fplugin=Plugin
-v
-optl-pthread
-o
bazel-out/k8-fastbuild/bin/tests/binary-with-plugin/binary-with-plugin2
-hide-all-packages
-fno-version-macros
-package-env
bazel-out/k8-fastbuild/bin/tests/binary-with-plugin/link-package_env-binary-with-plugin2
-hide-all-plugin-packages
-plugin-package-id
testsZSbinary-with-pluginZSplugin-lib
-package-db
bazel-out/k8-fastbuild/bin/tests/binary-with-plugin/testsZSbinary-with-pluginZSplugin-lib
-package-db
bazel-out/k8-fastbuild/bin/tests/binary-with-plugin/link-config-binary-with-plugin2
-package
link-config-binary-with-plugin2
-optc-Wno-unused-command-line-argument
-optl-Wno-unused-command-line-argument
bazel-out/k8-fastbuild/bin/tests/binary-with-plugin/_obj/binary-with-plugin2/Main.o