Parse report_json files with serde

[Swatinem]

Replaces the hand-written winnow-based parser with a bunch of struct definitions along with deriving serde::Deserialize.

---

The Serde definitions look very simple, and parsing performance seems to be 8-20x faster than the hand built parser, and doing a lot less less intermediate allocations.

This is a previous comparison:

pyreport              fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ serde                            │               │               │               │         │
│  ├─ complex_report  19.47 ms      │ 25.82 ms      │ 21.48 ms      │ 21.66 ms      │ 23      │ 23
│  │                  alloc:        │               │               │               │         │
│  │                    1629        │ 0             │ 1629          │ 1558          │         │
│  │                    401.6 KB    │ 0 B           │ 401.6 KB      │ 384.2 KB      │         │
│  │                  dealloc:      │               │               │               │         │
│  │                    14          │ 0             │ 14            │ 13.39         │         │
│  │                    91.82 KB    │ 0 B           │ 91.82 KB      │ 87.83 KB      │         │
│  ╰─ simple_report   2.017 µs      │ 155.4 µs      │ 2.12 µs       │ 2.4 µs        │ 182288  │ 182288
│                     alloc:        │               │               │               │         │
│                       6           │ 6             │ 6             │ 6             │         │
│                       4.056 KB    │ 4.056 KB      │ 4.056 KB      │ 4.056 KB      │         │
│                     dealloc:      │               │               │               │         │
│                       6           │ 6             │ 6             │ 6             │         │
│                       4.056 KB    │ 4.056 KB      │ 4.056 KB      │ 4.056 KB      │         │
╰─ winnow                           │               │               │               │         │
   ├─ complex_report  471 ms        │ 516.1 ms      │ 497.7 ms      │ 495.7 ms      │ 10      │ 10
   │                  alloc:        │               │               │               │         │
   │                    1020786     │ 1020786       │ 1020786       │ 1020786       │         │
   │                    81.6 MB     │ 81.6 MB       │ 81.6 MB       │ 81.6 MB       │         │
   │                  dealloc:      │               │               │               │         │
   │                    1020784     │ 1020784       │ 1020784       │ 1020784       │         │
   │                    122.4 MB    │ 122.4 MB      │ 122.4 MB      │ 122.4 MB      │         │
   │                  grow:         │               │               │               │         │
   │                    322539      │ 322539        │ 322539        │ 322539        │         │
   │                    40.89 MB    │ 40.89 MB      │ 40.89 MB      │ 40.89 MB      │         │
   ╰─ simple_report   14.31 µs      │ 168.7 µs      │ 16.18 µs      │ 17.38 µs      │ 28158   │ 28158
                      alloc:        │               │               │               │         │
                        51          │ 51            │ 51            │ 51            │         │
                        4.613 KB    │ 4.613 KB      │ 4.613 KB      │ 4.613 KB      │         │
                      dealloc:      │               │               │               │         │
                        51          │ 51            │ 51            │ 51            │         │
                        4.725 KB    │ 4.725 KB      │ 4.725 KB      │ 4.725 KB      │         │
                      grow:         │               │               │               │         │
                        12          │ 12            │ 12            │ 12            │         │
                        112 B       │ 112 B         │ 112 B         │ 112 B         │         │

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.62%. Comparing base (1b43a89) to head (05f2083).
Report is 1 commits behind head on main.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #18      +/-   ##
==========================================
- Coverage   98.69%   98.62%   -0.07%     
==========================================
  Files          20       20              
  Lines        7333     6962     -371     
==========================================
- Hits         7237     6866     -371     
  Misses         96       96              

Components	Coverage Δ
core	98.61% <100.00%> (-0.08%)	⬇️
bindings	100.00% <ø> (ø)
python	100.00% <ø> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Swatinem]

I updated this a little bit, adding divans AllocProfiler. The results are in the updated PR description.

I’m a bit surprised myself by the results. Looking more in-depth into how the report_json file actually looks like, it turns out the bulk of the file size comes from the session_totals (aka file.XXX.3).

In particular, those totals are parsed in full fidelity, but thrown away and never used, as all the totals are calculated on-demand instead:

codecov-rs/src/parsers/pyreport/report_json.rs

Lines 86 to 89 in 77b522f

	let (filename, file_summary) = delimited(ws, parse_kv, ws).parse_next(buf)?;
	let Some(chunks_index) = file_summary
	.get(0)

The serde code is using IgnoredAny to just skip over all of that.

---

Related to memory usage optimizations, I have used by very own smol_buf::Str24, which is using small string optimizations.

The memory usage is effectively:

• 24 + 8 = 32 bytes per File entry, plus HashMap overhead
• at least 8 + 288 = 296 bytes per Session entry, plus long non-inline Strings, plus HashMap overhead

We can go lower for Session still by using Str16 rather than Str24 for all the strings that tend to be short.

[matt-codecov]

thanks for taking care of this! now that python doesn't emit the session_totals bloat anymore and the p99 report json size is stable around 8mb we don't need to be paranoid about fitting everything in memory

love the perf win and the big red diff

[matt-codecov]

if i had noticed this was ready for review i'd have let you merge it before merging #27, sorry!