Cabal's foreign-library stanza

[mpilgrem]

I wanted to check that Stack can handle Cabal’s foreign-library stanza, so I tried to create a minimal example, as below. I read the Cabal guide and this blog post.

foo.cabal

cabal-version: 2.0
name:           foo
version:        0.1.0.0
build-type:     Simple

foreign-library myForeignLib
  type: native-shared
  options: standalone
  other-modules:
      Lib
  hs-source-dirs:
      src
  build-depends:
      base >=4.7 && <5
  default-language: Haskell2010

I am on Windows, so I understand options: standalone is required, to avoid a dependency on the Haskell runtime library libHSrts.

src\Lib.hs

module Lib
    ( someFunc
    ) where
someFunc :: IO ()
someFunc = putStrLn "someFunc"

When I try stack --verbose build --cabal-verbose (for full information), it builds but fails at the linking stage, with (extracts, reformatted for clarity):

2023-06-17 20:51:52.244261: [info] foo> configure
2023-06-17 20:51:52.244261: [debug] Run process within D:\Users\mike\Code\Haskell\foo\: C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_9p6GVs8J_3.6.3.0_ghc-9.2.8.exe 
--verbose=2 
--builddir=.stack-work\dist\8a54c84f 
configure 
--with-ghc=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.2.8\bin\ghc-9.2.8.exe
--with-ghc-pkg=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.2.8\bin\ghc-pkg-9.2.8.exe 
--user 
--package-db=clear 
--package-db=global 
--package-db=C:\sr\snapshots\4ae63981\pkgdb 
--package-db=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\pkgdb 
--libdir=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\lib 
--bindir=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\bin 
--datadir=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\share 
--libexecdir=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\libexec 
--sysconfdir=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\etc 
--docdir=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\doc\foo-0.1.0.0 
--htmldir=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\doc\foo-0.1.0.0 
--haddockdir=D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\doc\foo-0.1.0.0 
--dependency=base=base-4.16.4.0 
--extra-include-dirs=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20221216\mingw64\include 
--extra-lib-dirs=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20221216\mingw64\lib 
--extra-lib-dirs=C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20221216\mingw64\bin 
--exact-configuration 
--ghc-option=-fhide-source-paths
...
2023-06-17 20:51:52.737479: [info] foo> Configuring foo-0.1.0.0...
...
2023-06-17 20:51:53.556129: [info] foo> build
2023-06-17 20:51:53.556129: [debug] Run process within D:\Users\mike\Code\Haskell\foo\: C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_9p6GVs8J_3.6.3.0_ghc-9.2.8.exe 
--verbose=2 
--builddir=.stack-work\dist\8a54c84f 
build 
--ghc-options " -fdiagnostics-color=always"
2023-06-17 20:51:53.592638: [info] foo> Component build order: foreign library 'myForeignLib'
...
2023-06-17 20:51:53.619640: [info] foo> "C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.2.8\bin\ghc-9.2.8.exe" "--make" "-no-link" "-fbuilding-cabal-package" "-O" "-static" "-outputdir" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" "-odir" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" "-hidir" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" "-stubdir" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" "-i" "-i.stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" "-isrc" "-i.stack-work\dist\8a54c84f\build\myForeignLib\autogen" "-i.stack-work\dist\8a54c84f\build\global-autogen" "-I.stack-work\dist\8a54c84f\build\myForeignLib\autogen" "-I.stack-work\dist\8a54c84f\build\global-autogen" "-I.stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" "-IC:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20221216\mingw64\include" "-optP-include" "-optP.stack-work\dist\8a54c84f\build\myForeignLib\autogen\cabal_macros.h" "-hide-all-packages" "-Wmissing-home-modules" "-no-user-package-db" "-package-db" "C:\sr\snapshots\4ae63981\pkgdb" "-package-db" "D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\pkgdb" "-package-db" ".stack-work\dist\8a54c84f\package.conf.inplace" "-package-id" "base-4.16.4.0" "-XHaskell2010" "Lib" "-fhide-source-paths" "-fdiagnostics-color=always"
2023-06-17 20:51:53.670149: [info] foo> [1 of 1] Compiling Lib
2023-06-17 20:51:53.847663: [info] foo> Linking...
2023-06-17 20:51:53.848663: [info] foo> "C:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.2.8\bin\ghc-9.2.8.exe" 
"--make" 
"-fbuilding-cabal-package" 
"-O" 
"-shared" 
"-static" 
"-fPIC" 
"-outputdir" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" 
"-odir" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" 
"-hidir" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" 
"-stubdir" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" 
"-i" 
"-i.stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" 
"-isrc" 
"-i.stack-work\dist\8a54c84f\build\myForeignLib\autogen" 
"-i.stack-work\dist\8a54c84f\build\global-autogen" 
"-I.stack-work\dist\8a54c84f\build\myForeignLib\autogen" 
"-I.stack-work\dist\8a54c84f\build\global-autogen" 
"-I.stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib-tmp" 
"-IC:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20221216\mingw64\include" 
"-optP-include" 
"-optP.stack-work\dist\8a54c84f\build\myForeignLib\autogen\cabal_macros.h" 
"-lHSrts" 
"-LC:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20221216\mingw64\lib" 
"-LC:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\msys2-20221216\mingw64\bin" 
"-LC:\Users\mike\AppData\Local\Programs\stack\x86_64-windows\ghc-9.2.8\lib\x86_64-windows-ghc-9.2.8\rts-1.0.2" 
"-no-hs-main" 
"-hide-all-packages" 
"-Wmissing-home-modules" 
"-no-user-package-db" 
"-package-db" "C:\sr\snapshots\4ae63981\pkgdb" 
"-package-db" "D:\Users\mike\Code\Haskell\foo\.stack-work\install\3722b7b2\pkgdb" 
"-package-db" ".stack-work\dist\8a54c84f\package.conf.inplace" 
"-package-id" "base-4.16.4.0" 
"-XHaskell2010" 
"Lib" 
"-o" ".stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib.dll" 
"-fhide-source-paths" 
"-fdiagnostics-color=always"
2023-06-17 20:51:53.901273: [info] foo> [1 of 1] Compiling Lib [flags changed]
2023-06-17 20:51:54.057813: [info] foo> Linking .stack-work\dist\8a54c84f\build\myForeignLib\myForeignLib.dll ...
2023-06-17 20:51:54.161423: [warn] foo> C://Users//mike//AppData//Local//Programs//stack//x86_64-windows//ghc-9.2.8//mingw//bin/ld.exe: cannot find -lHSrts
2023-06-17 20:51:54.759920: [warn] foo> collect2.exe: error: ld returned 1 exit status
2023-06-17 20:51:54.768288: [warn] foo> `gcc.exe' failed in phase `Linker'. (Exit code: 1)

That is, the linker (ld.exe) is complaining that it cannot find -lHSrts and I note that Cabal (the library) is passing -lHSrts to GHC 9.2.8. I thought the options: standalone in the Cabal file was meant to make that unnecessary. I wonder is: Stack failing to pass something required to Cabal (the library) in the configure step? (Cabal-simple_9p6GVs8J_3.6.3.0_ghc-9.2.8.exe is the small exectuable that Stack uses to call Cabal (the library).)

My questions are this: (1) is there something wrong with the specification of my minimal example? If yes, I would be grateful if somebody could provide me with a minimal example known to work on Windows.(2) If the minimal example is correctly specified, can anybody give me a pointer as to why the options: standalone in foo.cabal might not be respected? Any help would be gratefully received.

[mpilgrem]

I should add, I searched issues at the Cabal repository for foreign-library and nothing jumped out at me as directly related.

[ApothecaLabs]

Last, I was trying to vend shared libraries so that I could run Haskell in Unity, and unless something has changed in the last year or so, I don’t think stack supports the foreign-library cabal stanza yet, since it is reliant on hpack, and it looks like hpack still doesn’t support it either, although there is a PR for it.

This need is partly why I have switched from stack to cabal.

[mpilgrem]

Although Hpack does not yet support foreign-library, Stack does not depend on Hpack: if there is no package.yaml, Stack will rely directly on the Cabal file.

I think Stack intends to support foreign-library - see Support for Cabal 2.0 sub and foreign libraries by snoyberg · Pull Request #3430 · commercialhaskell/stack · GitHub, which aimed to introduce support in September 2017.

If Stack does, indeed, support foreign-library, I may try to give the Hpack issue some attention.

[ApothecaLabs]

Stack does not depend on Hpack

raises eyebrow in question

What hpack supports does drive a significant portion of what stack supports, as the large majority of stack use cases do rely on it generating *.cabal files automatically with hpack - but sure, if you’ve already written out a cabal file manually, then stack will detect that it didn’t generate it, and thus will refuse to regenerate it - but then it also can’t update the cabal file to account for any changes to the package or project, either.

At that point, stack can’t do much more than call cabal anyway. It may still build successfully, but that is because stack is also designed to build non-stack-managed projects, for backwards compatibility - so I think this is more of a supporting case rather than the main one.

With that said, I do hope that you make some sort of progress on this front - it is rather ridiculous that the links both you and I cited are from 2017 and 2018, and is now at version 3.10, far beyond 2.0.

[chreekat]

@ApothecaLabs

At that point, stack can’t do much more than call cabal anyway.

The original (and arguably still most important) feature of stack was to call Cabal with very specific constraints that come from stack.yaml. I wouldn’t dismiss that feature so quickly.

[mpilgrem]

On the aside of the Hpack project and foreign-library, there is a related pull request from July 2022: Foreign Library Stanza by ropwareJB · Pull Request #518 · sol/hpack · GitHub. I have cleared the merge conflicts, but something odd now occurs with the Windows CI (only), which uses the ‘system’ GHC supplied by the GitHub hosted runner (currently GHC 9.6.2) and cabal-3.10.1.0. It dies 90% of the way in trying to build test:spec with a series of the following messages at different memory addresses:

ghc-9.6.2.exe: munmapForLinker: m32_release_page: Failed to unmap 49152 bytes at 00007ff7e07e7000: Attempt to access invalid address.

followed by:

Access violation in generated code when reading 0x20

 Attempting to reconstruct a stack trace...

   Frame	Code address
 * 0x8f3c7fd840	0x7ff7d2d06636 C:\ghcup\ghc\9.6.2\bin\ghc-9.6.2.exe+0x4286636
 * 0x8f3c7fd8c0	0x7ff7d19de05f C:\ghcup\ghc\9.6.2\bin\ghc-9.6.2.exe+0x2f5e05f
 * 0x8f3c7fd8f0	0x7ff7d19deaf8 C:\ghcup\ghc\9.6.2\bin\ghc-9.6.2.exe+0x2f5eaf8
 * 0x8f3c7fd8f8	0x7ff7d24818af C:\ghcup\ghc\9.6.2\bin\ghc-9.6.2.exe+0x3a018af
 * 0x8f3c7fd900	0x7ef4f69385d0
 * 0x8f3c7fd908	0x1ba0652d860
 * 0x8f3c7fd910	0x4ee39
 * 0x8f3c7fd918	0x7ef4f2ae7c40

Error: cabal-3.10.1.0.exe: Failed to build test:spec from hpack-0.35.2. The
build process terminated with exit code 11

which I suspect is something to do with the compiler.

[atravers]

Hmm…another m32-memory bug. Could the removal of the bug described here:

Facing "mmap 4096 bytes at (nil): Cannot allocate memory"? You're not alone!

You! yes, you! Are you facing a segfault with the following error? GHCi, version 9.2.7: https://www.haskell.org/ghc/ :? for help ghci> 40000 / 9 4444.444444444444 ghc: mmap 4096 bytes at (nil): Cannot allocate memory ghc: Try specifying an address with +RTS -xm<addr> -RTS fish: Job 1, 'ghci' terminated by signal SIGSEGV (Address boundary error) You will be pleased to know that migrating to GHC 9.4 solves this problem, as explained in #19421: internal error: m32_allocator_init: Failed to map …

…have introduced another one?

(N.B. Due to the extra information provided in the associated GHC issue, this supposition is now irrelevant.)

[mpilgrem]

I will log an issue at GHC’s GitLab repository: #23533: Windows, GHC 9.6.2, munmapForLinker: m32_release_page: Failed to unmap ... Attempt to access invalid address. · Issues · Glasgow Haskell Compiler / GHC · GitLab.