Is Scotty really benchmarking better than Axum?

Haskell is still losing to Ohkami and Micro-http, but it’s impressive that Scotty is outperforming or at least keeping up with some of the most commonly used Rust servers.

That said, in theory, Haskell should be able to do at least 50% of the best Rust servers, while keeping a clear and concise syntax and maintaining better type safety.

12 Likes

i really would like to know the methodology, or maybe i am mistaken about what is being benchmarked. how can so many javascript implementations be outperforming c++, rust and pure c?

3 Likes

You can see the exact Haskell packages being tested here.

Looking at the Servant implementation, you can see that the handlers are effectively trivial, so this benchmark is measuring the overhead of the web framework routing mostly.

2 Likes

Odd benchmark indeed. Especially with all that space dedicated to tail latency :smiley:

What other large-scale comparisons of web frameworks are there? The only other one I’m aware of is TechEmpower Framework Benchmarks .

TechEmpower looks like it actually exercises the framework with some actual workloads. It would be interesting to add scotty there.

Actually, this seems strange because I recall they used to have Scotty / Warp / Servant etc in their archives, but I can’t seem to find it, it’s just Postgrest, Wizzardo-inline, and IHP these days.

I haven’t looked at this specific benchmark closely, but I wouldn’t put too much stock in benchmarks like these. This post about Servant’s performance in the TechEmpower benchmarks covers some of the problems:

https://old.reddit.com/r/haskell/comments/6b3dlt/techempower_benchmarks_14_released_with_servant/dhl500h/

7 Likes

I was thinking the same thing - some of these “javascript” implementations seem to be wrapping ridiculously optimized assembly instructions lol GitHub - just-js/just: the only javascript runtime to hit no.1 on techempower 🔥

3 Likes