Benchmarking

[hugusmaximus]

I'm trying to make a simple benchmark to a very simple code using "tiny_http" like this:

fn main(){

let server = Server::http("0.0.0.0:80");

	match server {
		Ok(_) => println!("Success"),
		Err(_) => println!("Error")
	}

	for request in server.expect("REASON").incoming_requests() {
	    let response = Response::from_string("Hello!");
		let _ = request.respond(response);
	}

}

I'm doing an extremely soft "benchmark" and getting those results:

hugo@anakin:~/hermit-rs-template$ ab -n 10 -c 3 -r "http://10.0.5.3/pruebastress"
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.0.5.3 (be patient).....done


Server Software:        tiny-http
Server Hostname:        10.0.5.3
Server Port:            80

Document Path:          /pruebastress
Document Length:        0 bytes

Concurrency Level:      3
Time taken for tests:   0.286 seconds
Complete requests:      10
Failed requests:        20
   (Connect: 0, Receive: 10, Length: 0, Exceptions: 10)
Total transferred:      156 bytes
HTML transferred:       13 bytes
Requests per second:    34.97 [#/sec] (mean)
Time per request:       85.794 [ms] (mean)
Time per request:       28.598 [ms] (mean, across all concurrent requests)
Transfer rate:          0.53 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    12   14   4.5     12      27
Waiting:        0    0   0.0      0       0
Total:         12   14   4.5     12      27

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     12
  75%     13
  80%     14
  90%     27
  95%     27
  98%     27
  99%     27
 100%     27 (longest request)

Looks like concurrent connections are not handled also with no concurrency at all I get this:

hugo@anakin:~/hermit-rs-template$ ab -n 1000 -r "http://10.0.5.3/pruebastress"
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.0.5.3 (be patient)
Completed 100 requests
Completed 200 requests
(...)
Finished 1000 requests

(...)
Document Path:          /pruebastress
Document Length:        13 bytes

Concurrency Level:      1
Time taken for tests:   213.315 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      156000 bytes
HTML transferred:       13000 bytes
Requests per second:    4.69 [#/sec] (mean)
Time per request:       213.315 [ms] (mean)
Time per request:       213.315 [ms] (mean, across all concurrent requests)
Transfer rate:          0.71 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       20   30   4.3     32      56
Processing:   146  183   6.2    185     208
Waiting:      132  166   5.9    167     189
Total:        170  213   5.9    213     256

Percentage of the requests served within a certain time (ms)
  50%    213
  66%    214
  75%    215
  80%    215
  90%    217
  95%    219
  98%    222
  99%    224
 100%    256 (longest request)

My console output (qemu-system-aarch64):

[LOADER][INFO] Loader: [0x40200000 - 0x4021f000]
[LOADER][INFO] Found ELF file with size 30194072
[LOADER][INFO] Parsing kernel from ELF at 0x48000000..0x49ccb998 (len = 0x1ccb998 B / 30194072 B)
[LOADER][INFO] Loading kernel to 0x40400000..0x40a2d228 (len = 0x62d228 B / 6476328 B)
[LOADER][INFO] TLS is at 0x408d90a8..0x408d9148 (len =  0xa0 B / 160 B)
[LOADER][INFO] Detect 1 CPU(s)
[LOADER][INFO] Detect UART at 0x9000000
[LOADER][INFO] Jumping to HermitCore Application Entry Point at 0x406b2794
[0][INFO] Welcome to Hermit 0.6.7
[0][INFO] Kernel starts at 40400000
[0][INFO] BSS starts at 0x40a2b7b0
[0][INFO] tls_info = Some(
    TlsInfo {
        start: 0x408d90a8,
        filesz: 0x20,
        memsz: 0xa0,
        align: 0x8,
    },
)
[0][INFO] RAM starts at physical address 40000000
[0][INFO] Physical address range: 16384GB
[0][INFO] Support of 4KB pages: true
[0][INFO] Support of 16KB pages: false
[0][INFO] Support of 64KB pages: true
[0][INFO] Total memory size: 500 MB
[0][INFO] Kernel region: [40400000 - 40c00000]
[0][INFO] A pure Rust application is running on top of Hermit!
[0][INFO] Heap: size 432 MB, start address 40c00000
[0][INFO] Heap is located at 0x40c00000..0x5bc00000 (0 Bytes unmapped)
[0][INFO] 
[0][INFO] ===================== PHYSICAL MEMORY FREE LIST ======================
[0][INFO] 0x0000005BCD4000 - 0x00000060000000
[0][INFO] ======================================================================
[0][INFO] 
[0][INFO] 
[0][INFO] ================== KERNEL VIRTUAL MEMORY FREE LIST ===================
[0][INFO] 0x0000005BC00000 - 0x00000100000000
[0][INFO] ======================================================================
[0][INFO] 
[0][INFO] Intialize generic interrupt controller
[0][INFO] Found GIC Distributor interface at 8000000 (size 0x10000)
[0][INFO] Found generic interrupt controller at 80a0000 (size 0xF60000)
[0][INFO] 
[0][INFO] ========================== CPU INFORMATION ===========================
[0][INFO] Processor compatiblity:  arm,cortex-a72
[0][INFO] Counter frequency:       62500000 Hz (from CNTFRQ_EL0)
[0][INFO] Run on hypervisor
[0][INFO] ======================================================================
[0][INFO] 
[0][INFO] Hermit booted on 2023-12-29 11:39:58.0 +00:00:00
[0][INFO] Mapping PCI Enhanced Configuration Space interface to virtual address 60000000 (size 0x10000000)
[0][INFO] Scanning PCI Busses 0 to 255
[0][INFO] Compiled with PCI support
[0][INFO] Compiled with ACPI support
[0][INFO] Compiled with FSGSBASE support
[0][INFO] Compiled with SMP support
[0][INFO] 
[0][INFO] ======================== PCI BUS INFORMATION =========================
[0][INFO] 00:00 Host bridge [0600]: Red Hat, Inc. QEMU PCIe Host bridge [1B36:0008]
[0][INFO] 00:01 Ethernet controller [0200]: Red Hat, Inc. Virtio network device [1AF4:1041], IRQ 4, BAR1 Memory32 { address: 0x0, size: 0x1000, prefetchable: false }, BAR4 Memory64 { address: 0x8000000000, size: 0x4000, prefetchable: true }
[0][INFO] ======================================================================
[0][INFO] 
[0][INFO] Hermit is running on common system!
[0][INFO] Found virtio network device with device id 0x1041
[0][WARN] Currently only mapping of 64 bit BAR's is supported!
[0][WARN] Currently only mapping of 64 bit BAR's is supported!
[0][INFO] Non Virtio PCI capability with id 11 found. And NOT used.
[0][ERROR] Found virtio capability whose BAR is not mapped or non existing. Capability of type 5 and id 0 for device 1041, can not be used!
[0][INFO] Feature set wanted by network driver are in conformance with specification.
[0][INFO] Feature set wanted by network driver are in conformance with specification.
[0][INFO] Driver found a subset of features for virtio device 1041. Features are: [VIRTIO_NET_F_CSUM, VIRTIO_NET_F_GUEST_CSUM, VIRTIO_NET_F_MAC, VIRTIO_NET_F_MRG_RXBUF, VIRTIO_NET_F_STATUS, VIRTIO_F_RING_INDIRECT_DESC, VIRTIO_F_VERSION_1]
[0][INFO] Features have been negotiated between virtio network device 1041 and driver.
[0][INFO] Created SplitVq: idx=0, size=256
[0][INFO] Created SplitVq: idx=1, size=256
[0][INFO] Network driver successfully initialized virtqueues.
[0][INFO] Device specific initialization for Virtio network device 1041 finished
[0][INFO] Network device with id 1041, has been initialized by driver!
[0][INFO] Virtio-net link is up after initialization.
[0][INFO] Virtio network driver initialized.
[0][INFO] Install virtio interrupt handler at line 4
[0][INFO] Trying to initialize network!
[0][INFO] MAC address 52-54-00-12-34-56
[0][INFO] Configure network interface with address 10.0.5.3/24
[0][INFO] Configure gateway with address 10.0.5.1
[0][INFO] MTU: 1514 bytes
[0][WARN] Unable to read entropy! Fallback to a naive implementation!
Success

I'm using Hermit 0.8.0. I'm missing something for sure as this low performance is not normal.... any hint?

[stlankes]

I have to check. In the past, we had some performance issue. But I thought that we have solved it. Maybe it come back with an PR. Currently, our checks do not evaluate, if a PR creates some power losses.

[stlankes]

On my system I got better performance. Naive question. Do you build the Hermit application with --release?

[hugusmaximus]

No. But I got 4/5 requests/second which is several orders of magnitude below what I would expect... how much performance are you getting? (just in case I'll try with --release...). El jue., 4 ene. 2024 19:34, Stefan Lankes ***@***.***> escribió:

…

On my system I got better performance. Naive question. Do you build the Hermit application with --release? — Reply to this email directly, view it on GitHub <#516 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQVGKU5TPEAOWFSD6SIMEVLYM3Y4DAVCNFSM6AAAAABBGQ7ZBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZXGU3TMNZYGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mkroening]

Yeah, it's expected to be slow without --release. We should add a hint to the README in https://github.com/hermit-os/hermit-rs-template.

[hugusmaximus]

Yes, with --release I got 40 req/sec. Not a rocket but it works for me. Thanks!

[mkroening]

Network performance is something that we are actively looking into, so it should get better in the future. :)

[hugusmaximus]

My apologies. I'm getting much, much more. This 40 req/sec is because an experimental KASLR I was implementing and also plus SSL offloading proxy before. Just tried your vanilla code with plain HTTP and I get 10.000 req/sec, which looks pretty good!!!

hugo@anakin:~/home/hugo$ wrk -c100 -d10000 -t 10 http://10.137.0.3:9975
Running 167m test @ http://10.137.0.3:9975
  10 threads and 100 connections
^C  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.97ms    2.27ms  25.83ms   82.25%
    Req/Sec     1.08k   323.60    10.69k    86.06%
  164378 requests in 15.32s, 38.22MB read
Requests/sec:  10731.98
Transfer/sec:      2.50MB

[mkroening]

Ah, I am relieved. Thanks for clarifying!