Statically-linked binary distributions for Linux

[mpilgrem]

I’m seeking some advice, as follows. Stack for Linux/x86_64 is statically linked. Historically, it has been produced by building Stack on Alpine Linux/x86_64 with a dependency on musl libc (using Stack, and a Alpine Linux version of GHC (via Nix)) and the Cabal ld-options: -static and -pthread. My problem is this, after GHC 9.2.8, problems with Alpine Linux versions of GHC (official or Nix-supplied) mean they can no longer compile Stack (they segfault at one point or another in the process). Those problems do not look like they will be resolved in the near future.

My question is: is there another way to reach the end objective of a statically linked binary? (I do not have much experience with Linux outside of maintaining CI scripts, or of ‘linking’ - and I don’t really appreciate if there is something ‘special’ about Alpine Linux/musl libc).

EDIT: Or to put the question another way, if I get Stack to build Stack on the GitHub-hosted runner ubuntu-latest and with the Stack-supplied GHC 9.4.5 (likely, the tinfo6 version) with the same Cabal ld-options, will the binary that results be - in principle - ‘just as good’ on Linux/x86_64?

[mpilgrem]

By way of experiment, I tried to build Stack locally on ubuntu (via WSL 2) with the stack:static flag (which is what turns on the ld-options) but it did not end well:

[6 of 6] Linking .stack-work/dist/x86_64-linux-tinfo6/ghc-9.4.5/build/stack/stack
/home/mpilgrem/.stack/snapshots/x86_64-linux-tinfo6/75f4f4358f0caf9e2f1bde9cf8634175872befffd316dc06a740e2c73f4e3fd6/9.4.5/lib/x86_64-linux-ghc-9.4.5/persistent-sqlite-2.13.1.1-2wjhCbfrKUwEplB6RLVjej/libHSpersistent-sqlite-2.13.1.1-2wjhCbfrKUwEplB6RLVjej.a(sqlite3.o)(.text+0x6b4d): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/snapshots/x86_64-linux-tinfo6/75f4f4358f0caf9e2f1bde9cf8634175872befffd316dc06a740e2c73f4e3fd6/9.4.5/lib/x86_64-linux-ghc-9.4.5/network-3.1.4.0-EqU1XGCyczw5BRh1gJLC4K/libHSnetwork-3.1.4.0-EqU1XGCyczw5BRh1gJLC4K.a(HsNet.o):function hsnet_getaddrinfo: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/process-1.6.16.0/libHSprocess-1.6.16.0.a(fork_exec.o):function do_spawn_fork: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/process-1.6.16.0/libHSprocess-1.6.16.0.a(fork_exec.o):function do_spawn_fork: warning: Using 'initgroups' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function unixzm2zi7zi3_SystemziPosixziUser_getAllGroupEntries6_info: warning: Using 'getgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function unixzm2zi7zi3_SystemziPosixziUser_getAllGroupEntries8_info: warning: Using 'endgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function unixzm2zi7zi3_SystemziPosixziUser_getAllGroupEntries7_info: warning: Using 'setgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function unixzm2zi7zi3_SystemziPosixziUser_getAllGroupEntries7_info: warning: Using 'endgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function unixzm2zi7zi3_SystemziPosixziUser_getAllGroupEntries3_info: warning: Using 'setgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function unixzm2zi7zi3_SystemziPosixziUser_getAllGroupEntries3_info: warning: Using 'setgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function unixzm2zi7zi3_SystemziPosixziUser_getAllGroupEntries3_info: warning: Using 'endgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function unixzm2zi7zi3_SystemziPosixziUser_getAllGroupEntries3_info: warning: Using 'endgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function ghczuwrapperZC0ZCunixzm2zi7zi3ZCSystemziPosixziUserZCendpwent: warning: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function ghczuwrapperZC1ZCunixzm2zi7zi3ZCSystemziPosixziUserZCsetpwent: warning: Using 'setpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function ghczuwrapperZC2ZCunixzm2zi7zi3ZCSystemziPosixziUserZCgetpwent: warning: Using 'getpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function ghczuwrapperZC3ZCunixzm2zi7zi3ZCSystemziPosixziUserZCgetpwnamzur: warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function ghczuwrapperZC4ZCunixzm2zi7zi3ZCSystemziPosixziUserZCgetpwuidzur: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function ghczuwrapperZC5ZCunixzm2zi7zi3ZCSystemziPosixziUserZCgetgrnamzur: warning: Using 'getgrnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(User.o):function ghczuwrapperZC6ZCunixzm2zi7zi3ZCSystemziPosixziUserZCgetgrgidzur: warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(Files.o):function ghczuwrapperZC1ZCunixzm2zi7zi3ZCSystemziPosixziFilesZCmknod: error: undefined reference to '__xmknod'
/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(ByteString.o):function ghczuwrapperZC1ZCunixzm2zi7zi3ZCSystemziPosixziFilesziByteStringZCmknod: error: undefined reference to '__xmknod'

/home/mpilgrem/code/github/commercialhaskell/stack/rts/Linker.c:601:0: error:
     warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

/home/mpilgrem/code/github/commercialhaskell/stack/rts/linker/Elf.c:2169:0: error:
     warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
collect2: error: ld returned 1 exit status
ghc-9.4.5: `gcc' failed in phase `Linker'. (Exit code: 1)

[atravers]

It appears that you’re not alone:

bradrn:

[…] on Windows I’ve been compiling it to a static library instead (with -static -optl-static -staticlib).

However, in the process of releasing a new version, today I tried recompiling it on Windows, and got a nasty error about relocation truncated to fit: R_X86_64_32. Looking this up, apparently it means my static library has gotten too big to link easily […]

This looks promising:

angerman:

I am building fully static haskell applications for linux using haskell.nix ; and even deploy them to platforms like Synology NASs.

…but:

This is arguably way more complex than anything you’d likely need though.

As for other options:

jaspervdj:

I have often distributed Haskell libraries/executables together with their shared libraries in the past. If the user (or installer) sets LD_LIBRARY_PATH correctly, is should be a pretty smooth process.

[mpilgrem]

At the promising discussion, @hasufell offered up the following - but I think that is dependent on the existence of a working ‘Alpine Linux’ version of GHC. Cabal’s --enable-executable-static flag is documented here:

Portable Haskell executables, anyone doing that (possibly using Nix)?

Use an alpine linux docker container and build your binaries with cabal build --enable-executable-static. Don’t use cabal install. That’s all there is to it. GHCup, cabal and HLS all do this to build static executables.

EDIT: On ubuntu (via WSL 2) applying in Stack’s configuration:

configure-options: # Configure Cabal's configuration options
  $locals:
  - --enable-executable-static

saw stack build end badly in the same way as my post above.

[bradrn]

@atravers

It appear that you’re not alone:

bradrn:

[…] on Windows I’ve been compiling it to a static library instead (with -static -optl-static -staticlib).

However, in the process of releasing a new version, today I tried recompiling it on Windows, and got a nasty error about relocation truncated to fit: R_X86_64_32. Looking this up, apparently it means my static library has gotten too big to link easily […]

This was a slightly different problem: I had a Haskell library which I was trying to statically link into a C++ program. Whatever the solution to that turns out to be, it’s likely different to creating a statically linked program in Haskell alone.

[mpickering]

We have recently started to provide a dynamically linked alpine bindist which can be used to create statically linked executables on alpine. (#23349: Build and distribute vanilla alpine bindists · Issues · Glasgow Haskell Compiler / GHC · GitLab)

We also modified the metadata we generate for ghcup so that if the dynamically linked bindist is available then that is the one installed on alpine.

These are currently planned to be available from 9.6.3 and 9.8.*.

The ticket tracking the original issue is #23043: Segmentation fault building package on alpine. · Issues · Glasgow Haskell Compiler / GHC · GitLab

[benz0li]

For anyone using docker images in their CI/CD pipelines: There is GHC musl (glcr.b-data.ch/ghc/ghc-musl).

These images are are multi-arch (linux/amd64, linux/arm64/v8), based on Alpine Linux and contain unofficial binary distributions of GHC (build flavour: perf+llvm+split_sections).

Image tags 9.2[.8], 9.4[.{6,7}], 9.6[.2] (latest) also include a statically linked binary of Stack 2.11.1 (unsupported build).
→ Use flags --system-ghc and --no-install-ghc with Stack to ensure that only the GHC available in the container is used.

[benz0li]

FYI Alpine Linux versions ≥ 3.17 provide package ghc for both x86_64 and aarch64.

[Bodigrim]

@mpilgrem

/home/mpilgrem/.stack/programs/x86_64-linux/ghc-tinfo6-9.4.5/lib/ghc-9.4.5/lib/../lib/x86_64-linux-ghc-9.4.5/unix-2.7.3/libHSunix-2.7.3.a(Files.o):function ghczuwrapperZC1ZCunixzm2zi7zi3ZCSystemziPosixziFilesZCmknod: error: undefined reference to '__xmknod'

The problem with __xmknod is likely because “glibc in version 2.33 removed definitions of __xstat, __xstat64, __xmknod and others from sys/stat.h”. See also Static linking on Ubuntu 22.10 does not work due to undefined __xmknod in GlibC >= 2.33 · Issue #5814 · oracle/graal · GitHub.

AFAIU the solution would be to have a GHC bindist built for Ubuntu 22.04 (currently we reuse the bindist for Ubuntu 20.04). But there are none, even for GHC 9.10 series.