Throughput collapses on large object batch requests

[konradgithuup]

I measured the throughput of batches of 10_000 object read/write operations. First, the benchmarks begin approaching a throughput limit as the size of read/written segments increases, but at some point the throughput collapses and is greatly reduced from there on out. This occurs on my machine as well as on the OVGU ants cluster (ant14).

The limit itself is influenced by the I/O interface used. On the cluster, POSIX I/O approached a limit of around 6 GB, a mmap backend approached 3 GB. The collapse occurs between segment sizes of 256 and 512 KiB, regardless of the I/O interface.

POSIX sequential write (ant14 is ssd, hdd, nvme; local is local):

with linear scaling for dramatic effect:

[konradgithuup]

Local

• CPU: AMD Ryzen 5 5625U
• Memory: 16 GB
• File System: ext4
• nvme: Micron MTFDKCD512TFK

ant14

• CPU: AMD EPYC 7443 24-Core Processor
• Memory: 128 GB
• File System: ext4
• nvme: Micron_7450_MTFDKBA960TFR
• ssd: SAMSUNG MZ7LH960HAJR-00005
• hdd: ST4000LM024-2U817V