Warp: hot-path performance optimizations (+101% keep-alive throughput, -51% allocation)

mpscholten · August 8, 2026, 5:19pm

warp: hot-path performance optimizations (+101% keep-alive throughput, -51% allocation) (#1090)

master ← seanparsons:perf/hot-path-optimizations

opened 11:21AM - 19 Jul 26 UTC

## Summary A series of hot-path optimizations for warp (plus small changes to… `recv` and `time-manager`), found by profiling and syscall-tracing a keep-alive hello-world server and verified end to end. Cumulative effect on the same hardware/workload (wrk -t4 -c100, server `-N4`, GHC 9.10.3): | | stock warp 3.4.15 | this branch | |---|---|---| | hello-world throughput | 35.7k req/s | **71.7k req/s (+101%)** | | allocation per request | 13.7 KB | **6.7 KB (-51%)** | | `sendResponse` (4-header builder response) | 1.58 us | **290 ns (-82%)** | | syscalls per request | recv + send + epoll_ctl + ~28 futex ops | **recv + send** | Also validated against a real production Servant app (full middleware stack, DB pool, logging): +57-80% throughput on light endpoints, +27% on a 124KB HTML page, ~40% lower latency, byte-identical response bodies. ## The two big ones **Optimistic recv (last commit).** `makeGracefulRecv` (introduced with graceful shutdown, on the hot path for every `runSettings` server) registers with the IO manager and parks in STM on *every* receive, even though under load the next request is nearly always already in the kernel buffer. The branch tries a direct non-blocking `recv(2)` first and only falls back to the STM/shutdown machinery on would-block. A syscall census went from ~1 `epoll_ctl` + ~28 `futex` per request to 186 and 725 *total* across 198k requests. **Timer traffic (commit 5).** Every request performed 3+ operations against GHC's global TimerManager (pause/resume/tickle; streaming did resume+pause per fragment). `time-manager` now caches the TimerManager in the Handle and rate-limits `tickle` to one real `updateTimeout` per min(1s, timeout/4) using a monotonic-clock check; streaming keeps the timeout armed with a rate-limited tickle per fragment. This commit alone was +45% throughput. The remaining commits: single-pass response-header handling, composing the header directly into the connection write buffer (saves a full memcpy + pinned allocation per response), strict-record header indexes on both request and response side, a cached `ForeignPtr` in `WriteBuffer`, and a lazily-built request vault. ## Correctness - Full warp spec suite passes at every commit boundary I built (and on the final tree). - Wire bytes diffed against master for content-length, chunked, and many-header responses: byte-identical including chunked framing. - Timeout behavior after the timer changes verified manually: idle, keep-alive-then-idle, and stalled-streaming connections all closed at ~3.01s with `setTimeout 3`. - The vault laziness was proven with a temporary trace probe (plus a control run). ## Behavior changes maintainers should weigh 1. **Timeout precision** (timer commit): timeouts can fire up to min(1s, timeout/4) early relative to the last suppressed tickle. 2. **Streaming timeout semantics** (timer commit): the timeout is now armed while user code runs between fragments, so a `responseStream` that emits nothing for a full timeout period is killed (previously it could stall forever between fragments). Streams that write at least once per timeout period, or send heartbeats, are unaffected. This closes a slowloris-shaped gap but is observable for e.g. SSE without heartbeats. 3. **Graceful shutdown edge** (recv commit): a request already in the kernel buffer when shutdown begins is now served (pre-graceful-shutdown behavior for bytes the client already sent). 4. **API changes**: `Connection.connAppsInProgress` and `makeGracefulRecv` change `TVar Int` -> `IORef Int` (exposed via `Internal`); `WriteBuffer` gains a `bufFPtr` field; `Header.hs` drops the now-unused array machinery. The STM->IORef commit measured neutral on its own, so I'm happy to drop it if the API churn isn't worth it. 5. `receiveNoWait` (recv) uses a direct FFI `recv(2)` because `Network.Socket.recvBufNoWait` isn't exported from `network`'s public API; on Windows it returns `Nothing` (unchanged behavior). If exporting `recvBufNoWait` upstream is preferred I can switch to it. ## Benchmarks included The first commit adds `warp:bench:response`, an end-to-end criterion benchmark of `sendResponse` with a sink `Connection` — the response side previously had no benchmark coverage at all. Each commit message carries its measured effect.

Just randomly saw this PR to warp. Great use of AI to improve performance of core libraries in the haskell ecosystem

Ambrose · August 9, 2026, 12:54am

Is the PR description AI too? Nowadays I am always suspicious. But also can never quite tell (I’m bad at detecting it.)

AI code is whatever (especially for what warp is) but having to read AI descriptions of it is a bit much for me.

I would sooner ban AI PR descriptions than AI code lolol.

hasufell · August 9, 2026, 4:15am

It’s awesome that the wai maintainers agreed to review this code.

What I noticed is that nothing in the original PR indicates whether the PR author himself understands the changes.

Ambrose · August 9, 2026, 4:29am

AI optimization PRs are insidious, aren’t they? here’s N%. why argue about code style or other concerns? it’s Faster now!

gradient descent is a dangerous drug. it can help but it can also get you stuck in a valley.

Ambrose · August 9, 2026, 4:35am

Like..I can point Claude at most popular Haskell libraries and prompt it with

Create benchmarks for various core functions and then unroll loops in their implementation so they are faster

and get infinite PRs that speed up code that many companies use.

But should our ecosystem just be low level code of unrolled abstractions? Isn’t that antithesis to the whole reason we became Haskellers to begin with?

It’s troubling..

hasufell · August 9, 2026, 5:22am

I think interesting questions are:

why did GHC not do those optimizations?
why did no one before find those hand-rolled optimizations?
why is human-driven performance debugging so cumbersome?

All those point to possible tooling improvements. It’s easy to complain about AI, but we have work to do if we want to make things fun for humans again.

arybczak · August 9, 2026, 8:55am

Right, god forbid we get faster ubiquitous libraries, because gasp EVIL COMPANIES WILL BENEFIT!

Unbelievable

I for one am having a blast with LLM-assisted development

Ambrose · August 9, 2026, 3:32pm

You seemed to focus in on this out of the context before and after it, so let me spell out my point more explicitly:

Performance optimization is often good but the real thing we are after is Quality for the end user. Performance is definitely easier to measure, which is kind of why it can tempt you off the Quality path. But Performance gains, even big ones in common libraries, can easily be barely impactful to the end user.

(As a P.S. I’d like to comment on how this is a fun full circle moment for me. Years ago when choosing an effects library, I chose cleff over effectual for precisely these reasons. Both your comments in the cleff “Advantages over effectful” issue and effectful’s API itself were a little too dripping in Performance Maximalism for my taste. In fact, your benchmark suite is what sold me on cleff’s Performance and made me feel confident using it over effectful for 60fps game logic!)

hasufell · August 10, 2026, 5:49am

Not just you, seems others are having a blast on the GHC tracker too:

And the caveat:

This ticket reports a possible bug found by an LLM-based code review (details under “Provenance” below). I, @sjakobi, have checked it for plausibility, but not verified it in depth.

So I guess the actual work has to be done by others: triage, filter, review.

FPtje · August 10, 2026, 6:07am

I think it’s perfectly fine to demand they do that first. Boundaries can be set here by placing the PRs on hold or even close them and ask them to come back once the human is fully convinced and can stand for the change.

I think this behavior falls under “Don’t be a meat proxy”. See Don't be a meat proxy

From a quick look at the Warp PR it’s a bit more unclear to me just how deeply the author understands the change. In the very quick glance I didn’t see any super obvious red flags (like the comment quoted above). I think more signals there can be gathered by asking e.g. “without using an LLM, could you tell me about the tradeoffs made in these PRs?”

That or some other open ended question. Those cost very little effort to ask, and the answers typically are a great source for green or red flags.

MangoIV · August 10, 2026, 8:32am

That’s not what they said and that’s not what I understood, either, why are you being so defensive?

AndreasPK · August 10, 2026, 9:49am

I have not come across this term before I think its great. I’ve been “meat proxied” before and it can be a very frustrating experience.

I think the new skill people need to develop here is when it’s okay to be mostly a proxy, and when it’s time to just dump a prompt into a comment and call it a day.

wiz · August 10, 2026, 10:54am

But now they won’t even have the fun implementing the fun parts