master â seanparsons:perf/hot-path-optimizations
opened 11:21AM - 19 Jul 26 UTC
## Summary
A series of hot-path optimizations for warp (plus small changes to⌠`recv` and `time-manager`), found by profiling and syscall-tracing a keep-alive hello-world server and verified end to end. Cumulative effect on the same hardware/workload (wrk -t4 -c100, server `-N4`, GHC 9.10.3):
| | stock warp 3.4.15 | this branch |
|---|---|---|
| hello-world throughput | 35.7k req/s | **71.7k req/s (+101%)** |
| allocation per request | 13.7 KB | **6.7 KB (-51%)** |
| `sendResponse` (4-header builder response) | 1.58 us | **290 ns (-82%)** |
| syscalls per request | recv + send + epoll_ctl + ~28 futex ops | **recv + send** |
Also validated against a real production Servant app (full middleware stack, DB pool, logging): +57-80% throughput on light endpoints, +27% on a 124KB HTML page, ~40% lower latency, byte-identical response bodies.
## The two big ones
**Optimistic recv (last commit).** `makeGracefulRecv` (introduced with graceful shutdown, on the hot path for every `runSettings` server) registers with the IO manager and parks in STM on *every* receive, even though under load the next request is nearly always already in the kernel buffer. The branch tries a direct non-blocking `recv(2)` first and only falls back to the STM/shutdown machinery on would-block. A syscall census went from ~1 `epoll_ctl` + ~28 `futex` per request to 186 and 725 *total* across 198k requests.
**Timer traffic (commit 5).** Every request performed 3+ operations against GHC's global TimerManager (pause/resume/tickle; streaming did resume+pause per fragment). `time-manager` now caches the TimerManager in the Handle and rate-limits `tickle` to one real `updateTimeout` per min(1s, timeout/4) using a monotonic-clock check; streaming keeps the timeout armed with a rate-limited tickle per fragment. This commit alone was +45% throughput.
The remaining commits: single-pass response-header handling, composing the header directly into the connection write buffer (saves a full memcpy + pinned allocation per response), strict-record header indexes on both request and response side, a cached `ForeignPtr` in `WriteBuffer`, and a lazily-built request vault.
## Correctness
- Full warp spec suite passes at every commit boundary I built (and on the final tree).
- Wire bytes diffed against master for content-length, chunked, and many-header responses: byte-identical including chunked framing.
- Timeout behavior after the timer changes verified manually: idle, keep-alive-then-idle, and stalled-streaming connections all closed at ~3.01s with `setTimeout 3`.
- The vault laziness was proven with a temporary trace probe (plus a control run).
## Behavior changes maintainers should weigh
1. **Timeout precision** (timer commit): timeouts can fire up to min(1s, timeout/4) early relative to the last suppressed tickle.
2. **Streaming timeout semantics** (timer commit): the timeout is now armed while user code runs between fragments, so a `responseStream` that emits nothing for a full timeout period is killed (previously it could stall forever between fragments). Streams that write at least once per timeout period, or send heartbeats, are unaffected. This closes a slowloris-shaped gap but is observable for e.g. SSE without heartbeats.
3. **Graceful shutdown edge** (recv commit): a request already in the kernel buffer when shutdown begins is now served (pre-graceful-shutdown behavior for bytes the client already sent).
4. **API changes**: `Connection.connAppsInProgress` and `makeGracefulRecv` change `TVar Int` -> `IORef Int` (exposed via `Internal`); `WriteBuffer` gains a `bufFPtr` field; `Header.hs` drops the now-unused array machinery. The STM->IORef commit measured neutral on its own, so I'm happy to drop it if the API churn isn't worth it.
5. `receiveNoWait` (recv) uses a direct FFI `recv(2)` because `Network.Socket.recvBufNoWait` isn't exported from `network`'s public API; on Windows it returns `Nothing` (unchanged behavior). If exporting `recvBufNoWait` upstream is preferred I can switch to it.
## Benchmarks included
The first commit adds `warp:bench:response`, an end-to-end criterion benchmark of `sendResponse` with a sink `Connection` â the response side previously had no benchmark coverage at all. Each commit message carries its measured effect.
Just randomly saw this PR to warp. Great use of AI to improve performance of core libraries in the haskell ecosystem
12 Likes
Is the PR description AI too? Nowadays I am always suspicious. But also can never quite tell (Iâm bad at detecting it.)
AI code is whatever (especially for what warp is) but having to read AI descriptions of it is a bit much for me.
I would sooner ban AI PR descriptions than AI code lolol.
1 Like
Itâs awesome that the wai maintainers agreed to review this code.
What I noticed is that nothing in the original PR indicates whether the PR author himself understands the changes.
3 Likes
AI optimization PRs are insidious, arenât they? hereâs N%. why argue about code style or other concerns? itâs Faster now!
gradient descent is a dangerous drug. it can help but it can also get you stuck in a valley.
1 Like
Like..I can point Claude at most popular Haskell libraries and prompt it with
Create benchmarks for various core functions and then unroll loops in their implementation so they are faster
and get infinite PRs that speed up code that many companies use.
But should our ecosystem just be low level code of unrolled abstractions? Isnât that antithesis to the whole reason we became Haskellers to begin with?
Itâs troubling..
1 Like
I think interesting questions are:
why did GHC not do those optimizations?
why did no one before find those hand-rolled optimizations?
why is human-driven performance debugging so cumbersome?
All those point to possible tooling improvements. Itâs easy to complain about AI, but we have work to do if we want to make things fun for humans again.
9 Likes
Right, god forbid we get faster ubiquitous libraries, because gasp EVIL COMPANIES WILL BENEFIT!
Unbelievable
I for one am having a blast with LLM-assisted development
6 Likes
You seemed to focus in on this out of its context before and after it, so let me spell out my point more explicitly:
Performance optimization is often good but the real thing we are after is Quality for the end user. Performance is definitely easier to measure , which is kind of why it can tempt you off the Quality path. But Performance gains, even big ones in common libraries, can easily be barely impactful to the end user .
(As a P.S. Iâd like to comment on how this is a fun full circle moment for me. Years ago when choosing an effects library, I chose cleff over effectual for precisely these reasons. Both your comments in the cleff âAdvantages over effectfulâ issue and effectfulâs API itself were a little too dripping in Performance Maximalism for my taste. In fact, your benchmark suite is what sold me on cleffâs Performance and made me feel confident using it over effectful for 60fps game logic!)
4 Likes
