Serverless Haskell with GHC WASM JSFFI + Cloudflare Workers

Preface

I was excited to know that JavaScript FFI support landed in GHC WASM (so much kudos to @TerrorJack!), and since then I have tried to put GHC-generated WASM on Cloudflare’s Serverless service Cloudflare Workers.

And here it is:

https://ghc-wasm-worker-demo.konn-san.com

Source code here:

Prior Works and many thanks to the pioneers

The idea of putting Haskell-generated WASM on Edge computing service is not my own - Stack Builders and Tweag guys have pioneering work with Asterius (the latter article is written by @TerrorJack):

These two are using Asterius, but recently @nomeata uses C-FFI of GHC WASM backend to run computation on Fastly:

Indeed, these three articles are the main motive of my work. Although these are done in slightly different settings, the fact that there already have been pioneers doing similar thing really encouraged me.

Thank you all you great pioneers!

Challenges

So, my project. @nomeata’s work on Fastly is powered by C-FFI, but Cloudflare Workers provides API in JavaScript. So I had to wait until GHC WASM backend provides JS FFI mechanism.

JS FFI is really great mechanism, but it comes only with JS and WASM backends and WASM-related modules are unavailable in other backends, e.g. the native codegen. In addition, WASM backend currently lacks an interpreter (as far as I know). These have the following implications:

  1. We cannot directly use our beloved Haskell Language Server.
    • HLS is, in essense, full-armored GHCi. So it can’t be built with WASM backend.
    • If we use plain HLS with native codegen, we cannot use JS FFI as-is.
  2. Template Haskell and compiler plugins are unavailable in the WASM backend (at the time of GHC 9.10).
    • These restriction will eventually be solved in near future, but still needs some hack for the time being.
    • As for TH, there is an effort to avoid TH-usage and we can use the tailored packages thanks to WASM backend guys.

In addition, I am mainly using Apple Silicon macOS. It seems that the recent ghc-wasm-bindist contains some bindists for ARM mac, but I couldn’t install it.

There also is a Nix flake for the WASM backend in ghc-wasm-meta, but I am not so familiar with Nix. I once tried to use the flake to install it directly on my macOS, but it fails for some unknown reason.

So, at least for my environment, I also had to fix:

  1. Prepare the build environment for WASM backend usable on (reasonablly powerful) macOS.

Tooling Challenge 1: Build Envrionment

Since it is unclear how to install GHC WASM backend on macOS, I decided to use kinda container-based approach.

This time, I decided to use Earthly as the container-based make system:

This requires BuildKit backend, so I chose OrbStack which comes with reasonablly efficient multi-arch Linux container:

Earthly is some kind of mixture of Dockerfile and Makefile. TBH, I’m not a fun of these, but Earthly is reasonablly powerful and helped the development a lot.

Here is the Earthfile I use to build project:

The docker image used in this environment can also be used with normal Docker-like containers. For those who may interested, it is available here:

This is just a x86_64 Ubuntu container with GHC WASM backend installed with GHCup. This contains GHC 9.10 prerelease (wasm32-wasi-ghc-9.10.0.20240412) - I couldn’t find the final version of GHC 9.10.1 in the cross-compiler channel in GHCup. But it seems prerelease version just works fine.

Earthfile roughly follows the build script that comes with Tweag’s GHC WASM + Miso example:

In short ,all my Earthfile does is to:

  1. Compile with WASM Backend,
  2. Extract the WASM module generated by WASM backend,
  3. Genrate JS FFI Gluecode,
  4. Optimise and pre-initialise WASM module, and
  5. Create the skeleton for Workers project and locate modules there.

Tooling Challenge 2: IDE

OK, build environment is clear.

The next challenge is to make HLS available. As described above, HLS requires interpreter so it cannot be used with WASM backend for now.

So we have to use a normal HLS that is compiled and can be used with Native Backend (as distributed with GHCup).

It just works fine until one use JSFFI and/or GHC.Wasm.Prim module in the codebase. For missing modules, we can just provide a dummy module. The compatibility module just re-exports the module with WASM backend, and export dummy module in non-WASI OS.

The problem of JSFFI is a little harder. One solution would be provide a Template Haskell macro that expands to real JS FFI declaration if available, and expands to dummy definition if unavailable. Like this:

{-# LANGUAGE TemplateHaskell #-}
javaScriptFFIImport Unsafe "$1 === null" [d|js_is_null :: JSVal -> Bool|]

{- Instead of writing:
foreign import javascript unsafe "$1 === null"
  js_is_null :: JSVal -> Bool
-}

This looks reasonable, but this solution cannot be used at the time of GHC 9.10 as WASM backend doesn’t have an appropriate external interpreter for Template Haskell as noted above.

So, I decided to use the other code-generation mechanism: GHC Source Plugin.

The GHC.Wasm.FFI.Plugin source plugin module works as follows:

  1. If the WASM backend is used, this is just a no-op.
  2. For non-WASM backend (os /= wasi), it:
    • Replaces foreign import javascript declarations with just a dummy function definition with the same type (runtime error when executed), and
    • Removes all foreign export javascript decls.

So, for example, the following module:

{-# OPTIONS_GHC -fplugin GHC.Wasm.FFI.Plugin #-}
foreign import javascript safe "$1.json()"
  js_get_body_json :: JSRequest -> IO (Promise JSON)

foreign export javascript unsafe "hs_main"
  main :: IO ()

…is silently transformed into the following content on non-WASM backends:

js_get_body_json :: JSRequest -> IO (Promise JSON)
js_get_body_json = error "js_get_body_json :: JSRequest -> IO (Promise JSON)"

-- NOTE: exported of `hs_main` is omitted. 

… and unchanged with WASM backend.

Idealy, if one put -fplugin GHC.Wasm.FFI.Plugin in cabal files or on top of the module, then everything must work fine.

However, due to the lack of interpreter in WASM backend, compiler plugins are currently unapplicable with WASM backend.

Hence, we can just enable the plugin only in non-wasi os, in cabal-level configuration, like this:

library
  if !os(wasi)
    build-depends: ghc-wasm-compat
    ghc-options: -fplugin GHC.Wasm.FFI.Plugin

With this configuration, the native GHC and cabal just get happy with codebase containing JS FFI. As running WASM backend in container is slower than native, we can also use native cabal build as a lightweight (approximation of) type-checking (this is just an approximation, as there is a restriction on what types can be appearred in JS FFI decls, and currently no check is implemented in GHC.Wasm.FFI.Plugin).

HLS also gets happy - in some case. It seems that there is some glitches in HLS with source plugin. If the module has other transitive dependent modules with JSFFI with the source plugin enabled, it emits tremendous amount of bogus errors complaining:

• The `javascript' calling convention is unsupported on this platform
• When checking declaration:
  foreign import javascript unsafe "Object()" js_new_obj
    :: IO PartialJSON

The errors usually disappear if one open the module, or make a dummy edit on them until all the error is erased. Unfortunately, in some cases, the chain-of-bogus errors continues indefinitely, and the only solution is just repeating restarting language server until HLS gets happy.

Anyway, the situation is still far better than doing things without IDE at all. I will investigate the situation, and will open the issue on HLS repository (I think there already is similar issue, but could not found).

Real Challenge: Adjusting outputs for Workers

OK, the tooling is good (enough). The real challenge is to adjust outputs form WASM backend so that it works on Cloudflare Workers.

Small culprit: wrapper must be impure

WASM backend provides the special wrapper FFI import to turn Haskell callback functions into JavaScript callbacks.

This is something like this:

type FetchHandler = WorkerRequest -> JSAny -> FetchContext -> IO WorkerResponse

-- NOTE: as described in GHC Manual, @safe/unsafe@ in WASM JS FFI just corresponds to async/sync, and have nothing with purity. Safety here means async exception safety.
foreign import javascript unsafe "wrapper"
  toJSFetchHandler :: FetchHandler -> IO JSFetchHandler

The culprit here is that return-type must be wrapped by IO and MUST NOT be pure. For example, the following declaration still typechecks, but will result in mysterious getJSVal(1233455) runtime error:

foreign import javascript unsafe "wrapper"
  toJSFetchHandler :: FetchHandler -> JSFetchHandler

getJSVal(XXX) error occurs when JavaScript object linked with the id XXX is not in the FFI table. I encountered this error at the very beginning of the awaiting on Haskell’s code, so I get really confused. It took some hours until I re-read through the User’s Guide and notice that the example of wrapper is given with impure return-type.

The error dissappears once the return type of toJSFetchHandler made impure. Hurray!

Adjusting FFI gluecode

When JSFFI is enabled, one has to generate ghc_wasm_jsffi.js with post_linker.mjs shipped with WASM backend, as described in GHC User’s Manual.

ghc_wasm_jsffi.js already contains some kind of polyfill to make codes run both on browsers and node/deno/bun, but Cloudflare Workers’ environment (deliberately) provides only a limited subset of node’s API.

In particular, it doesn’t provide MessageChannel and FinalizationRegistry.

MessageChannel is used to provide a polyfill for setImmediate. Cloudflare Workers doesn’t have setImmediate either, so we have to replace entire polyfill for setImmediate so that it runs on Workers.

So I replaced the following code:

// A simple & fast setImmediate() implementation for browsers. It's
// not a drop-in replacement for node.js setImmediate() because:
// 1. There's no clearImmediate(), and setImmediate() doesn't return
// anything
// 2. There's no guarantee that callbacks scheduled by setImmediate()
// are executed in the same order (in fact it's the opposite lol),
// but you are never supposed to rely on this assumption anyway
class SetImmediate {
  // ...snip...
}
// The actual setImmediate() to be used. This is a ESM module top
// level binding and doesn't pollute the globalThis namespace.
let setImmediate;
if (globalThis.setImmediate) {
  /// ... snip ...
}

with the following (stolen from here):

const setImmediate = (fn) => setTimeout(fn, 0);

FinalizationRegistry is used to garbage-collect Haskell Object on the JavaScript-side. As workers run only for less than 1 seconds, such GC can be disabled (after all, the objects staying inside Haskell still gets GC’d by Haskell RTS). So we can just provide a dummy “no-op” implementation for ``FinalizationRegistry`:

class FinalizationRegistry {
  constructor(_callback) {}
  register(... args) {
    return;
  }
  unregister(..._args) {
    return 1;
  }
}

I think this is similar to YOLO-mode used in Asterius + Cloudflare Worker article, so I think doing this makes sense.

Side Step: generating Web API compat layer

Cloudflare Workers build on top of Fetch API used widely in Web world. So it should just be good to have a binding to standard JS Web API to be called from WASM JSFFI.

To do this, I deviced a (incomplete) codegen tool that parses WebIDLs and generates low-level binding to call from JS FFI:

I’m aware that jsaddle-dom already parses WebIDLs and generates binding modules. But I couldn’t figure out how to use their codegen, and jsaddle uses eval under the hood (to my understanding) to be cross-platform. I want to make a direct use of JSFFI, so I decided to implement codegen on my own (or: just for fun :-)).

There seems several Haskell libraries that parses WebIDL, but I couldn’t find the one that can parse WebIDL files that comes with Rust’s web-sys, so I also write the parser for modern WebIDL:

Sadly enough, some part of WebIDL spec (especially the spec of extended attribtues) are ambiguous, and widely circulated WebIDL files are invalid strictly interpreted as the modern WebIDL. Also, some interfaces are undefined. So I had to make some hacks, but it finally works! This resulted in ~1100 auto-generated modules from this configuration file:

modulePrefix: "GHC.Wasm.Web.Generated"
outputDir: "./src/GHC/Wasm/Web/Generated"
inputDir: "./webidls"
extraImports:
- "GHC.Wasm.Web.Types"
predefinedTypes:
- ArrayBufferView
- BufferSource
- DOMTimeStamp
targets:
- Request
- Response
- WebSocket

Final Step: Launching reactor module on Cloudflare Workers

According to GHC User’s Guide, there are two types of WASM modules: command module and reactor module.

To my understanding, we have to compile Haskell code as a reactor module, as JSFFI needs to provide multiple entry points.

The section of the guide provides a really clear explanation on how to initialise with WASI module and Cloudflare Workers comes with the official JS library @cloudflare/workers-wasi to use WASI on Cloudflare Workers.

So, ideally, we can just follow it in Cloudflare Workers’ WASI context to write a wrapper script to call WASM function like this:

import { WASI } from '@cloudflare/workers-wasi';
import ghc_wasm_jsffi from './ghc_wasm_jsffi.js';
import wasm_module from './handlers.wasm';

const wasi = new WASI();
const instance_exports = {};
const instance = new WebAssembly.Instance(wasm_module, {
  wasi_snapshot_preview1: wasi.wasiImport,
  ghc_wasm_jsffi: ghc_wasm_jsffi(instance_exports),
});
Object.assign(instance_exports, instance.exports);

await wasi.initialize(instance); // !

const handlers = await instance.exports.handlers();
export default handlers;

Modulo minor divergence on how to instantiate the module, this looks quite nice.

…However, this doesn’t work for the time being (as of 2024), because @cloudflare/workers-wasi doesn’t provide wasi.initialize() function to initialise WASM reactor module!

Fortunately, we can give a dummy _start() entry point to the instance and just call wasi.start():


// Instead of await wasi.initialize(instance);
await wasi.start({ exports: { _start() {}, ...instance.exports } });

I found this hack here.

And, … Go!

With all these hacks applied, we can run GHC-generated WASM code on the Cloudflare Workers! Yay!

Although thre are some culprits and missing features in WASM backend, I’m really impressed that how mature and usable WASM backend is. Thank you all you guys make this reality and I’m looking forward to see the future of WASM and GHC.

Happy Working Haskell-Wamsing!

35 Likes

This is great work, thank you! I don’t want to seem ungrateful, but it would be nice to see some simple stats regarding the generated code size and benchmarks comparing native vs. wasm. Are we at a point where it’s practical to go serverless for at least some Haskell web apps?

5 Likes

Hi, thank you for your words and question!

Since the first post, I implemented a servant-like type-driven router that can work with GHC WASM backend and use it to provide additional endpoitns to generate random number and calculate fibonacci sequence (servant-server depends on network package, which won’t compile with GHC WASM backend).
I also switched to use effecful for composing effects, so the number reported below might be affected by these changes.

So, the performance stat.
Currently, I am using the Free Plan of Cloudflare Workers, with a binary limit (compressed) of 1MiB, 10ms/req CPU Time limit (sans idle time), and 128MiB/req memory limit.

After compression, the binary for the above example results in 851.84 KiB output, which somehow fits within the limit (but near the border). From Cloudflare Dashboard, it seems it consumes 28MiB RAM per request, so the memory consumption is reasonable.

The most important criterion is the median CPU Time. It seems fibonacci / RNG endpoints just takes <5ms, so they are not so problematic.
But the top-page generation takes 33.6 ms - which seems to exceed the Free Plan limit, but it hasn’t resulted in any error and keeps functioning for the time being. Perhaps the CPU Time limit is not enforced strictly, or the number reported on the dashboard measures a slightly different definition of CPU Time. I have another private worker written in Haskell that wraps Cloudflare’s Worker behind Cloudflare Tunnel and it has almost the same median CPU Time (30.6ms) and it has not been interrupted either.

So, from the current experience, it DOES exceed some (perhaps soft?) limit, but GHC can still generate a good enough binary that works just fine on Cloudflare Workers.

As fib/RNG take much less time, it seems that the HTML generation takes the vast majority of the CPU time. This involves many text generation and conversion, the CPU Time might still be able to be reduced if we enhance that part (personal KV middleware also involves parsing of midium-length input JSON, which obviously needs string conversion).
Currently, the implementation converts Unicode JavaScript String to Haskell String ([Char]) and then packs to Text type. This clearly involves inefficient indirection, so I expect the CPU time could be improved once we have implemented more direct conversions between textual types. More aggressive use of streaming mechanisms can be also another option.

And I wonder if there is any way to virtually turn off Haskell-side GCs with GHC WASM backend - passing -with-rtsopts at the compile time seems ignored as a consequence of -no-hs-main.

Anyway, the generated WASM binary DOES run fine on Cloudflare Workers (at least at the moment of this writing), so it is indeed usable for personal purposes. I’m not sure about the viability of their industrial use, but I think it can play a role once ecosystem gets mature enough :slight_smile:

5 Likes

You can set the GHCRTS environment variable to pass RTS options (when you use wizer for pre-initialization, you need to set it there and use --inherit-env true). For fun, I just tried How to disable Garbage Collection in GHC Haskell? - Stack Overflow on GitHub - tweag/ghc-wasm-miso-examples and compared

  • -S: Results in lots of GCs (basically one per FFI invocation)
  • -S -I0 -A1G: No GCs at all (at least for the time I interacted with the site)
2 Likes

Thank you for your imformation! It seems that specifying GHCRTS and passing --inherit-env true to wizer do the trick. Unfortunately, if one specify GHCRTS=-S -I0 -A1G, the binary size doubles and exceeds Free Plan limit (still stays within Paid Plan though). So it seems some garbage left in binary in preinitialisation phase if one (virtually) disable GC. I do not have concrete stats, but turning off GC itself seem to reduce the CPU Time at least in development mode (on my laptop, not on Cloudflare). So if the binary size problem once goes, this option can be good option I think.

1 Like

IMHO it has been for some time, at least with native-code “serverless”. Bellroy’s Haskell deployments are pretty much all on AWS Lambda. (Static linking using haskell.nix; see wai-handler-hal-example for some ideas. These days, AWS Lambda supports container images, making static linking much less necessary.) But this thread is talking about WASM/JSFFI stuff, so saying more would be off-topic.

2 Likes