Debugging Stack linker errors

Learn
#1

I maintain a rather complicated framework which is not listed on Hackage. So I was building an app using it, when discovered I needed to add a small function to the framework to make it work, so I added it. Went to build the framework, and it builds perfectly.

Now I go to bump the version of the framework in stack.yaml . And I run stack build, and it starts checking out and building the latest version, and…linker error!

Wait, linker error? I thought this wasn’t supposed to be possible in a pure Haskell program.

And yet here it is:

cubix           > [1 of 4] Compiling Examples.Multi.Syntax
cubix           >
cubix           > <no location info>: error:
cubix           >     dlopen(/private/tmp/stack-37c90e04df5ecf99/cubix-0.0.0/.stack-work/dist/x86_64-osx/Cabal-3.8.1.0/build/libHScubix-0.0.0-JlXfM6V5PObCqpL8fDmy45-ghc9.4.5.dylib, 0x0005): symbol not found in flat namespace '_compstratzm0zi1zi0zi3zm1YpMv9EB0qD2skM8X0jPr8_DataziCompziMultiziStrategyziClassification_hasAnySort_closure'

(Probably relevant: “asAnySort” is the name of the function that I just added.)

I’ve tried stack clean and rm -rf .stack-work, and it didn’t help. And I don’t know what else to try, nor what could be causing this issue.

I have pushed a small reproduction to GitHub - cubix-framework/cubix-sample-app at linker-bug-repro . Cloning this branch and merely running “stack build” should produce the issue, although I haven’t tested on another computer.

So, is this a bug in Stack, or is there some weird setting I got wrong?

P.S.: Another weird thing is that it seems to be compiling the test executables that come with my framework, when the sample app only depends on the library. Anyone know how to get it to skip the executables?

#2

This is the kind of thing that happens with cabal when you have a module in your source tree that is not specified in your .cabal file. I don’t know if stack is susceptible to the same kind of issue. Does that sound plausible? Is anySort maybe in a new file and you have forgotten to list that file in one of stack's build definition files?

#3

I Just built the repo you provided without any errors:

Screenshot 2024-01-31 at 1.36.20 in the afternoon
Screenshot 2024-01-31 at 1.36.20 in the afternoon1920×1200 175 KB

My environment:

machine: m1 mac, Sonoma 14.3
stack version: 2.11.1

#4

If by “pure” you mean “divorced from the real world”, then no - Haskell isn’t denotative; it’s capable of
I/O, hence the error message about the missing symbol in a particular file.

What happens if you revert back to the previous (without asAnySort) version of your project?

#5

After swapping the SSH references to GitHub for the corresponding HTTPS ones in your extra-deps:, I could build your reproduction on Windows 11 with Stack 2.13.1 - so, I don’t have a error that I can dig into with added verbosity. Example output:

Building all executables for cubix-sample-app once. After a successful build of
all of them, only specified executables will be rebuilt.
cubix-sample-app> configure (exe)
Configuring cubix-sample-app-0.1.0.0...
cubix-sample-app> build (exe)
Preprocessing executable 'cubix-sample-app' for cubix-sample-app-0.1.0.0..
Building executable 'cubix-sample-app' for cubix-sample-app-0.1.0.0..
[1 of 2] Compiling Main
[2 of 2] Compiling Paths_cubix_sample_app
[3 of 3] Linking .stack-work\dist\c3556505\build\cubix-sample-app\cubix-sample-app.exe
cubix-sample-app> copy/register
Installing executable cubix-sample-app in D:\Users\mike\Code\Haskell\cubix-sample-app\.stack-work\install\b2fb529a\bin

Stack is built on top of Cabal (the library) and your error message is one passed up from Cabal - not one of Stack’s.

#6

On your ‘PS’, does this assist:

This ability to specify a component applies only to a local package. With dependencies, Stack will always build the library (if present) and all executables (if any), and ignore test suites and benchmarks.

#7

I do occasionally get linking errors of this sort with stack too — most often when updating git-cloned extra-deps, that have been built already, without bumping the dependency package version number.

@jkoppel try this:

stack exec -- ghc-pkg unregister compstrat --force

in your cubix tree, followed by stack build as usual.

This will tell ghc to forget the stale build of compstrat package. You’ll see stack build recompiling it.

This isn’t happening during rm -rf .stack-work — because dependency packages (language-java, semigroupoids, profunctors, cubix-compdata, compstrat, lens, etc, in your case) are installed not under the project dir, but into a shared “package DB” under ~/.stack/snapshots/. What .stack-work holds are only the compiled modules (executables, tests…) of only this package, cubix.

Minor warning: ghc-pkg unregister will not remove the files of that build of compstrat — so you’ll leak a little bit of disk space under ~/.stack/snapshots. It’s such a hassle to locate & clean up though, not worth doing; one day .stack/snapshots will bloat enough that you’ll want to wipe it out entirely, and recompile all dependency packages from scratch, anyway.

How do we know it’s a stale build? The symbol name in the error message

_compstratzm0zi1zi0zi3zm1YpMv9EB0qD2skM8X0jPr8_DataziCompziMultiziStrategyziClassification_hasAnySort_closure

demangles to

_compstrat-0.1.0.3-1YpMv9EB0qD2skM8X0jPr8_Data.Comp.Multi.Strategy.Classification_hasAnySort_closure

which as you said, refers to hasAnySort that you’ve just added (to compstrat dependency, I assume).

Perhaps, stack doesn’t notice you’ve made changes to compstrat because its package description (cabal-file) was unchanged, therefore continues hashing to the same value. Leading to stack deciding to reuse the previous build. Leading to your program linking against a stale build of compstrat that lacks the definition of hasAnySort.

3 Likes
#8

I think Stack assumes that all extra-deps from a repository are immutable (basis: Build overview - The Haskell Tool Stack) - which is why it ends up in the write-only database (also known as the snapshot database). So, if the code of such a package is changing without its package identifier changing (that is, it is not immutable), that will be breaking Stack’s fundamental assumption.

1 Like
#9

Thanks guys! This makes a lot of sense. The ghc-pkg command did not do the trick, but rm -rf ~/.stack/snapshots did.

So now my workflow is that, every time I want to try a small improvement to Cubix in a separate project, I need to bump the version number, push to github, and update stack.yaml in the using project. Anyone have a better way to do this? Assuming I don’t want to put Cubix and the using project into the same directory and switch them to having a common stack.yaml.

#10

Is this the issue in question? Link-time error after changing a dependency · Issue #5473 · commercialhaskell/stack · GitHub

1 Like
#11

I am not sure if this helps with your workflow question, but a local extra-dep is assumed to be mutable. So, I can have a single-package foo project that depends on a single-package boo project in a neighbouring directory:

name:                foo
version:             0.1.0.0
dependencies:
- base >= 4.7 && < 5
- boo

executables:
  foo:
    main:                Main.hs
    source-dirs:         app
    ghc-options:
    - -threaded
    - -rtsopts
    - -with-rtsopts=-N

and the project-level configuration for the foo project is:

snapshot: lts-22.8
extra-deps:
- ../boo # Local, assumed to be mutable

and if I edit the code of the boo package (without changing its package identifier), building foo will see boo rebuilt.

The difference between a local project package and a local extra-dep of a project is explained here: Configuration (project and global) - The Haskell Tool Stack.