# Cross-Compiling to 32-bit arm; weird syscall behaviour

**URL:** <https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882>\
**Category:** Learn\
**Created:** [February 23, 2024, 3:03pm UTC](https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882 "2024-02-23T15:03:04Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![marmayr](https://avatars.discourse-cdn.com/v4/letter/m/8491ac/32.png) [@marmayr](https://discourse.haskell.org/u/marmayr)\
**Post date:** [February 23, 2024, 3:03pm UTC](https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882/1 "2024-02-23T15:03:04Z")

</div>

This is probably a bit of a niche use-case that is not well-supported, as I understand, but I wanted to work on a small haskell application for my e-book reader, for which I would have to target a 32-bit ARM architecture.

I was using haskell.nix for cross compilation in different variants (including fully statically linked builds against glibc and musl and a dynamically linked build where I would deploy all dynamic dependencies onto the device). I have had varying degrees of success with this. For example, computing fibonacci numbers and printing out the results was working fine. However, as soon as I introduced some code interacting with the filesystem, the stuff got a bit wonky. For example, when I am having the equivalent of `readFile "SomeExistingFile" >>= putStrLn` in my file, the output would be just the name of my executable, I guess, followed by a colon.

Originally, I aimed for making some syscalls via FFI (which work locally and also when I cross-compile a similar C program for my reader), but the syscalls return weird stuff (like `open` returning `-1`, even though it returns a valid file descriptor in the equivalent C code).

For now, I have failed to set up a cross-compiler toolchain for anything newer than GHC 9.2.8 (haven’t tested with GHC 9.4.x yet), but my basic setup is the following haskell.nix-based flake, sorry it is a bit messy:

> **My flake.nix**
>
> ````haskell
> inputs.haskellNix.url = "./haskell.nix";
> inputs.nixpkgs.follows = "haskellNix/nixpkgs-unstable";
> inputs.flake-utils.url = "github:numtide/flake-utils";
> outputs = { self, nixpkgs, flake-utils, haskellNix }:
> let
> supportedSystems = [
> "x86_64-linux"
> ];
> in
> flake-utils.lib.eachSystem supportedSystems (system:
> let
> drv' = pkgs': extraModules:
> let
> isCrossStatic = pkgs'.stdenv.hostPlatform != pkgs.stdenv.hostPlatform;
> in pkgs'.haskell-nix.project {
> src = ./.;
> compiler-nix-name = (if isCrossStatic then "ghc928" else "ghc964");
> modules = [
> {
> packages.koboworld.components.exes.koboworld =
> let p = nixpkgs.lib.strings.makeSearchPath "include" (with pkgs'; [
> linuxHeaders
> ]);
> in {
> configureFlags = ["--extra-include-dirs=${p}"];
> };
> }]
> ++ (if isCrossStatic then
> [{
> packages.koboworld.components.exes.koboworld = {
> configureFlags = [
> "--disable-executable-dynamic"
> "--disable-shared"
> "--ghc-option=-optl-pthread"
> "--ghc-option=-optl-static"
> "--ghc-option=-optl-lffi"
> "--ghc-option=-optl-lnuma"
> "--ghc-option=-optl-lgmp"
> "--ghc-option=-optl-lz"
> "--disable-executable-stripping"
> "--disable-library-stripping"
> ] ++ map (l: "--ghc-option=-optl=-L${l}/lib") (with pkgs'; [
> gmp6
> zlib
> libffi
> numactl
> ]);
> };
> }]
> else [])
> ++ extraModules;
> shell.tools = {
> cabal = {};
> } // (if isCrossStatic then {} else {
> hlint = {};
> haskell-language-server = {};
> });
> };
> drv = pkgs': drv' pkgs' [];
> pkgs = haskellNix.legacyPackages.${system};
> pkgsArm = haskellNix.legacyPackages.${system}.pkgsCross.armv7l-hf-multiplatform.pkgsMusl;
> in {
> p = (drv pkgs);
> packages = {
> "exe:koboworld" = (drv pkgs).koboworld.components.exes.koboworld;
> "armv7l-hf-multiplatform:exe:koboworld" = (drv pkgsArm).koboworld.components.exes.koboworld;
> };
> devShells = {
> default = (drv pkgs).shell;
> };
> }
> );
> }```
> 
> ````

For the musl-based build I built GHC with ld.gold, even though this is disabled in haskelll.nix, but since the referenced ld.gold bug is fixed, I still wanted to give it a try. The behaviour, I have described above, however, is the same as if I do a dynamically linked build against glibc and deploy all dependencies onto the device (or a statically linked one against glibc, I think, but I can’t confirm this easily anymore right now and this does not seem to be super-safe?).

Before I give up on this foolish undertaking, I wanted to ask whether any of you has any hints on how I could possibly proceed?

---

<div class="post-metadata">

**Author:** ![angerman](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/angerman/32/892_2.png) [@angerman](https://discourse.haskell.org/u/angerman)\
**Post date:** [February 23, 2024, 9:33pm UTC](https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882/2 "2024-02-23T21:33:35Z")

</div>

Let me have a look today. I remember something similar from the aarch64 backend.

---

<div class="post-metadata">

**Author:** ![angerman](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/angerman/32/892_2.png) [@angerman](https://discourse.haskell.org/u/angerman)\
**Post date:** [February 26, 2024, 7:36am UTC](https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882/3 "2024-02-26T07:36:43Z")

</div>

@marmayr I’ve tried to reproduce this, but was unable to.

```haskell
git clone git@github.com:angerman/nixpkgs-static-repo.git --branch angerman/discourse-32bit-arm-repro
nix build .#hydraJobs.x86_64-linux.armv7-musl.myprogram

```

this will produce `result/bin/myprogram`, using `qemu` to run it results in:

```haskell
nix-shell -p qemu
qemu-arm ./result/bin/myprogram 
foo

```

where `foo` is the content of `test.txt`

I’m afraid there must be something else going on in your code, I can’t reproduce right now. Do note that this points at a custom haskell.nix branch right now, which also sets ld.gold (though I’m thinking we might want to use `lld` when using the `llvm` toolchain anyway).

---

<div class="post-metadata">

**Author:** ![marmayr](https://avatars.discourse-cdn.com/v4/letter/m/8491ac/32.png) [@marmayr](https://discourse.haskell.org/u/marmayr)\
**Post date:** [February 26, 2024, 8:18am UTC](https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882/4 "2024-02-26T08:18:43Z")

</div>

Thanks a lot, I will try to reproduce your results with qemu and on my target device and report back. In any case, it is super helpful to have a working example, I can start from. Thank you for your help!

---

<div class="post-metadata">

**Author:** ![angerman](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/angerman/32/892_2.png) [@angerman](https://discourse.haskell.org/u/angerman)\
**Post date:** [February 26, 2024, 10:11am UTC](https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882/5 "2024-02-26T10:11:52Z")

</div>

If you can provide a broken reproducer, I’m happy to try to debug it.

---

<div class="post-metadata">

**Author:** ![marmayr](https://avatars.discourse-cdn.com/v4/letter/m/8491ac/32.png) [@marmayr](https://discourse.haskell.org/u/marmayr)\
**Post date:** [February 26, 2024, 1:47pm UTC](https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882/6 "2024-02-26T13:47:28Z")

</div>

Everything is working now, thanks!

After running through the steps in your post, I’ve basically swapped out my flake.nix file for your one and it started working immediately. I’m also very happy that I can use GHC9.6 now and won’t investigate how exactly I have messed up. I am impressed that it is actually much easier to create those binaries than I thought, couldn’t get this from reading the haskell.nix page on cross compilation.

Thanks again!

---

<div class="post-metadata">

**Author:** ![angerman](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.haskell.org/angerman/32/892_2.png) [@angerman](https://discourse.haskell.org/u/angerman)\
**Post date:** [February 27, 2024, 2:30am UTC](https://discourse.haskell.org/t/cross-compiling-to-32-bit-arm-weird-syscall-behaviour/8882/7 "2024-02-27T02:30:14Z")

</div>

Yes, I’m stretched very thin, and the documentation isn’t where I’d like it to be ☹ Any help is appreciated!