Installing a library with C dependencies on Windows

This is a short description of how I managed to install a cabal package with C library dependencies on Windows. I hope this is useful information for others and for myself in the future.

Just now I wanted to install the hackage-cli package under windows. Normally that should be as simple as opening powershell and running cabal install hackage-cli but doing that result in an error:

Resolving dependencies...
Error: cabal-3.10.2.1.exe: Could not resolve dependencies:
[__0] trying: hackage-cli-0.1.0.1 (user goal)
[__1] trying: http-io-streams-0.1.6.3 (dependency of hackage-cli)
[__2] trying: http-io-streams:+brotli
[__3] trying: brotli-streams-0.0.0.0 (dependency of http-io-streams +brotli)
[__4] next goal: brotli (dependency of brotli-streams)
[__4] rejecting: brotli-0.0.0.1, brotli-0.0.0.0 (conflict: pkg-config package
libbrotlidec-any, not found in the pkg-config database)
[__4] fail (backjumping, conflict set: brotli, brotli-streams)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, http-io-streams, brotli,
http-io-streams:brotli, brotli-streams, hackage-cli
Try running with --minimize-conflict-set to improve the error message.

This is an intimidating wall of text, but the important part is:

conflict: pkg-config package libbrotlidec-any, not found in the pkg-config database

If we dig a bit deeper by making cabal show verbose output with the cabal install hackage-cli -v option then we see an even bigger wall of text, but somewhere hidden inside it there is a more informative message:

Failed to query pkg-config, Cabal will continue without solving for pkg-config
constraints: Cannot find pkg-config program

Ah, so the pkg-config program is not even installed at all! But how do we install it? On Linux you can just install the package with the package manager of your distribution, but Windows has no standard package manager. Luckily the Haskell infrastructure on Windows uses a Linux campatibility environment called MSYS. To install packages in this environment, you can start mingw64.exe which is usually found in C:\ghcup\msys64. That gives you a shell in which you can run pacman commands to install packages.

First we need to install pkg-config. A simple search will list a bunch of options. We can use pacman -Ss pkg-config to search for packages, but it will list many results including these:

mingw64/mingw-w64-x86_64-pkg-config 0.29.2-6
    A system for managing library compile/link flags (mingw-w64)
ucrt64/mingw-w64-ucrt-x86_64-pkg-config 0.29.2-6
    A system for managing library compile/link flags (mingw-w64)
clang64/mingw-w64-clang-x86_64-pkg-config 0.29.2-6
    A system for managing library compile/link flags (mingw-w64)
msys/pkgconf 2.1.0-1
    pkg-config compatible utility which does not depend on glib

Which one should you choose? I believe the right answer is mingw-w64-x86_64-pkg-config which is what works for me. However, I don’t know exactly why.

So let’s install it:

$ pacman -S mingw64/mingw64-w64-x86_64-pkg-config

And we already know from the first error message that we also need the brotli library so let’s add that too:

$ pacman -S mingw64/mingw64-w64-x86_64-brotli

Now let’s try installing hackage-cli again. Unfortunately we see the same:

conflict: pkg-config package libbrotlidec-any, not found in the pkg-config database

And with more details again:

Failed to query pkg-config, Cabal will continue without solving for pkg-config
constraints: Cannot find pkg-config program

The reason is that cabal can’t find the pkg-config executable, even though it is installed. You can make it visible to cabal by adding it to your Path environment variable, but that is not recommended because it might interfere with other programs and toolchains. Instead you can use GHCUp to run a command with the right directories on the Path temporarily:

> ghcup run --mingw-path -- cabal install hackage-cli

When running this command it will start installing a bunch of packages, but in the end we see another error message:

Configuring HsOpenSSL-0.11.7.6...
Error: setup.exe: Missing dependencies on foreign libraries:
* Missing (or bad) header file: openssl/asn1.h
* Missing (or bad) C libraries: ssl, crypto
This problem can usually be solved by installing the system packages that
provide these libraries (you may need the "-dev" versions). If the libraries
are already installed but in a non-standard location then you can use the
flags --extra-include-dirs= and --extra-lib-dirs= to specify where they are.If
the library files do exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
If the header file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.

Error: cabal-3.10.2.1.exe: Failed to build HsOpenSSL-0.11.7.6 (which is
required by exe:hackage-cli from hackage-cli-0.1.0.1). See the build log above
for details.

Now we know how to solve this issue. The only slightly difficult part is finding the name of the package. In this case there are quite a few clues pointing us to openssl, so let’s install that:

$ pacman -S mingw64/mingw64-w64-x86_64-openssl

And now finally running cabal install hackage-cli does work and installs the hackage-cli command.

But when we try it in PowerShell we see nothing happening:

> hackage-cli --help
>

The reason this doesn’t work is that it now cannot find the dynamic linker dependencies, inspecting with ldd (which you can install in the mingw64 environment, it is in the mingw-w64-x86_64-python-mingw-ldd package) shows us these dependencies:

        libbrotlicommon.dll => /mingw64/bin/libbrotlicommon.dll (0x7ffaf2c00000)
        libbrotlidec.dll => /mingw64/bin/libbrotlidec.dll (0x7ffaf2d80000)
        libbrotlienc.dll => /mingw64/bin/libbrotlienc.dll (0x7ffa6b330000)
        libcrypto-3-x64.dll => /mingw64/bin/libcrypto-3-x64.dll (0x7ffa6b3f0000)
        libssl-3-x64.dll => /mingw64/bin/libssl-3-x64.dll (0x7ffa6b240000)

You can still run the executables built in this way under ghcup --mingw-path:

> ghcup --mingw-path -- hackage-cli --help
hackage-cli - CLI tool for Hackage

Usage: hackage-cli.exe [--version] [--verbose] [--hostname HOSTNAME] COMMAND

Available options:
  -h,--help                Show this help text
  --version                Output version information and exit.
  --verbose                Enable verbose output.
  --hostname HOSTNAME      Hackage hostname (default: "hackage.haskell.org")

Available commands:
  pull-cabal               Download .cabal files for a package.
  push-cabal               Upload revised .cabal files.
  sync-cabal               Update/sync local .cabal file with latest revision on
                           Hackage.
  push-candidate           Upload package candidate(s).
  list-versions            List versions for a package.
  check-revision           Validate revision.
  index-sha256sum          Generate sha256sum-format file.
  add-bound                Add bound to the library section of a package, unless
                           the bound is redundant. The .cabal file is edited in
                           place.

Each command has a sub-`--help` text. Hackage credentials are expected to be
stored in an `${HOME}/.netrc`-entry (or `.netrc.gpg`) for the respective Hackage
hostname. E.g. "machine hackage.haskell.org login MyUserName password TrustNo".
All interactions with Hackage occur TLS-encrypted via the HTTPS protocol.

So I’m not sure how to fix this without adding the mingw directory to your Path permanently.

4 Likes

This is not recommended, since PATH on windows affects the linker. This can cause issues for other programs/toolchains.

Instead, this directory can be added to extra-prog-path in cabal config. This is what ghcup already does.

You can also do ghcup run --mingw-path <your-binary>.


Did you check the linking of your binary? Are mingw paths hardcoded? Can you execute your binary without changed PATH in powershell? Can you ship the binary?

3 Likes

That’s good to know. I’ll do some more tests and update my post.

I saw that this had happened when I ran the verbose cabal command, but it still didn’t recognize the pkg-config program when I had installed it. Or maybe I could have gotten that wrong because I took a brief detour to try to use the clang64 environment, because I thought that GHC had moved to that. I will check this again.

These are good questions, which I’d hoped people more qualified than me would have answered already. I don’t even know that well how to check the linking, I guess I could use lld but maybe that only works on Linux?

I’ve tried your suggestions here’s what worked and what didn’t:

  • If I remove mingw from the path then cabal can no longer find pkg-config even though the right directory has been added to the extra-prog-path.
  • Building it with ghcup run --mingw-path -- cabal install hackage-cli does work, but produces an executable that only works when you run it under ghcup run --mingw-path -- ... too.

Checking the linking yields these (and a bunch more windows specific dlls with an absolute path):

        libbrotlicommon.dll => /mingw64/bin/libbrotlicommon.dll (0x7ffaf2c00000)
        libbrotlidec.dll => /mingw64/bin/libbrotlidec.dll (0x7ffaf2d80000)
        libbrotlienc.dll => /mingw64/bin/libbrotlienc.dll (0x7ffa6b330000)
        libcrypto-3-x64.dll => /mingw64/bin/libcrypto-3-x64.dll (0x7ffa6b3f0000)
        libssl-3-x64.dll => /mingw64/bin/libssl-3-x64.dll (0x7ffa6b240000)

I don’t know how to fix this.

1 Like

It seems this might be related to `cabal` doesn't inherit the path from `cabal/config` in PowerShell · Issue #9519 · haskell/cabal · GitHub

2 Likes

The issue appears to be that the dlls are added to bin, and hence they can be found while building the executable but not running it. In my experience, the typical (unfortunate) way that relocatable executables are built on windows is that the dlls are just copied into the directory of the executable itself, no?

I don’t think this is a problem with building or installing a library at all. It is simply that executables that require dlls will also require they be in the path, and you can choose to alter the path to allow this or not. And if you don’t, then you need to copy them to where they will be found regardless (the same directory).

Unless I’m missing something, there’s nothing particularly cabal or haskell specific at all?

I’m not a Windows expert, but if that’s the case then It would be nice if cabal could put the required dlls next to the executable.

1 Like