Performance concerns vs hyper (~25% lower req/sec, ~8x higher latency)

[markg85]

Hi,

I'm just finding my way in Rust and the wealth of webserver-like libraries it has. With this i'm looking at benchmarks from Web Framework Benchmarks which is where my eye catches a peculiar performance difference. That strikes me as weird as Axum describes itself as a thin wrapper over Hyper.

Here's a couple screenshots of performance numbers (you can find them all in the before linked Web Framework Benchmarks).
Requests per second:

Latency

Those differences are fairly huge in my opinion.
Requests per second is about 25% lower then Hyper.
Latency in Axum is ~8x higher then Hyper.

Those are very sizable differences.
Is there perhaps a performance regression in Axum? Or is this the price to pay for the additional features it offers?

As an aside. The API and documentation of Axum are truly amazing! I'm specifically impressed by the simplicity of the router mechanism. Nicely done Axum devs, looks great to work with! 😄

[davidpdrsn]

Please do not read too much into those benchmarks. They're optimized in all sorts of unrealistic ways and rarely represent real world performance.

With that said, if they're using axum 0.7 then its probably related to #2381

[jplatte]

They're not using axum 0.7 (yet).

[markg85]

In my opinion, that makes those benchmarks perfect!
It shows the theoretical max throughput the framework can handle in ideal situations (on the hardware used for those benchmarks). And how to achieve that (in the benchmark source).

This helps massively because anything i build on top of it is going to have some impact on the ideal baseline. Ideally that impact should be as small as possible.

Obviously means that i need to benchmark with my own hardware setup to get my own baseline and go from there.