Profile-Guided Optimization (PGO) benchmarks

[zamazan4ik]

Hi!

Recently I checked optimizations like Profile-Guided Optimization (PGO) and Post-Link Optimizations (PLO) improvements on multiple projects. The results are available here. According to the tests, all these optimizations can help with achieving better performance in many cases for many applications including many network-related applications like Envoy and HAProxy. That's why I decided to test PGO on axum too.

I already did some benchmarks and want to share my results here. Hopefully, they will be helpful.

Test environment

• Fedora 39
• Linux kernel 6.7.4
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler - Rustc 1.76
• axum version: the latest for now from the main branch on commit b03f6c1184723bdc079215222fc168ef96847dea

Benchmark

For benchmark, I use cargo bench command. PGO training phase - cargo pgo bench, PGO optimization - cargo pgo optimize bench. Benchmarks are done in two modes: singe-threaded and multi-threaded. Single-threaded benchmarks are done with the same command with taskset -c 0 addition.

All PGO optimizations are done with cargo-pgo.

All measurements are done multiple times - the results are consistent enough across runs.

Results

I got the following results:

• Release (single-threaded): https://gist.github.com/zamazan4ik/62d40bb27b5a9f21c215c1e58576ea29
• PGO optimized (single-threaded): https://gist.github.com/zamazan4ik/67405fab108eaca667a18884ce01b34b
• Release (multi-threaded): https://gist.github.com/zamazan4ik/f55e558f62a3c04c40c3e98993a2c3a6
• PGO optimized (multi-threaded): https://gist.github.com/zamazan4ik/0bcfcbbab095b76e2dbf422267508b66
• (just for reference) PGO instrumentation (single-threaded): https://gist.github.com/zamazan4ik/11120b236fd84d0e27d2bcfba64d78d5

According to the results, PGO shows measurable improvement in the benchmark.

Further steps

I can recommend at least mentioning PGO somewhere in the documentation as a possible way to improve Axum's performance. Also, I can suggest continuing to test more advanced optimizations with tools like LLVM BOLT (for PLO).

I would be happy to answer questions about PGO!