Performance regression in arrow_batch_points/query benchmark

[teh-cmc]

Somehow it seems that #3162 caused a serious performance regression for this specific benchmark:

(See it live here)

[emilk]

For 0.9: let's see if this has significant impact on a real point cloud example, or if we can punt on this to 0.10

[emilk]

I wonder if some optimizations got lost in #3162, e.g the stuff in this one: #2970

[emilk]

Just as a note: real-life point cloud performance has improved by 3x from 0.8.2. to 0.9: #1136 (comment)

[teh-cmc]

The regression only impacts the legacy query APIs, which do not exist anymore.

$ git co 905fcaf56  # pre-offending PR

$ cargo bench --all-features -p re_query -- 'arrow_batch_points/query'  # legacy APIs
arrow_batch_points/query
                        time:   [5.4049 µs 5.4188 µs 5.4373 µs]
                        thrpt:  [183.92 Melem/s 184.54 Melem/s 185.02 Melem/s]

$ cargo bench --all-features -p re_query -- 'arrow_batch_points2/query' # new APIs
arrow_batch_points2/query
                        time:   [7.4022 µs 7.4485 µs 7.4947 µs]
                        thrpt:  [133.43 Melem/s 134.26 Melem/s 135.09 Melem/s]

$ git co 0a2258a  # offending PR

$ cargo bench --all-features -p re_query -- 'arrow_batch_points/query'  # legacy APIs
arrow_batch_points/query
                        time:   [14.340 µs 14.359 µs 14.382 µs]
                        thrpt:  [69.531 Melem/s 69.642 Melem/s 69.734 Melem/s]
                 change:
                        time:   [+164.89% +165.74% +166.47%] (p = 0.00 < 0.05)
                        thrpt:  [-62.472% -62.369% -62.248%]
                        Performance has regressed.

$ cargo bench --all-features -p re_query -- 'arrow_batch_points2/query'  # new APIs
arrow_batch_points2/query
                        time:   [7.4420 µs 7.4699 µs 7.4966 µs]
                        thrpt:  [133.39 Melem/s 133.87 Melem/s 134.37 Melem/s]

[teh-cmc]

Oh, and for completeness, here's main:

$ git co main

$ cargo bench --all-features -p re_query -- 'arrow_batch_points2/query'
    Finished bench [optimized + debuginfo] target(s) in 0.16s
     Running benches/query_benchmark.rs (target/release/deps/query_benchmark-78136916f96ddd02)
arrow_batch_points2/query
                        time:   [2.6338 µs 2.6503 µs 2.6766 µs]
                        thrpt:  [373.60 Melem/s 377.32 Melem/s 379.67 Melem/s]
                 change:
                        time:   [-64.660% -64.496% -64.320%] (p = 0.00 < 0.05)
                        thrpt:  [+180.27% +181.66% +182.96%]
                        Performance has improved.

Which is indeed 3x faster than the legacy.