If you don’t want to depend on glibc, you are bound to find an alternative libc indeed. This could be musl, diet, or any alternative libc (even bionic in principle). You also most likely want a static one, so you can just drop and deploy them.
Using muslc, you are pretty much bound by the syscalls the kernel provides. Thus on a very very old linux, you might run into issues. There are also some issues if you use e.g. the dns package from hackage with muslc on systems that messed up their file system layout, see https://github.com/kazu-yamamoto/dns/pull/166
I am building fully static haskell applications for linux using haskell.nix; and even deploy them to platforms like Synology NASs. This is arguably way more complex than anything you’d likely need though.