Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactor dictionary encoding in PQ writer to migrate to the new `cuco…
…::static_map` (#16541) Part of #12261. This PR refactors the dictionary encoding in Parquet writers to migrate from `cuco::legacy::static_map` to `cuco::static_map` to build the dictionaries. ### Performance Results The changes result in +0.08% average speed improvement and +16.22% average memory footprint increase (stems from the adjusted sizes by `cuco::make_window_extent` due to [prime gap](https://en.wikipedia.org/wiki/Prime_gap)) across the benchmark cases extended from #16591 Currently, we do see a roughly 8% speed improvement in map insert and find kernels which is counteracted by the map init and map collect kernels as they have to process 16.22% more slots. With a cuco bump, the average speed improvement will increase from 0.08% to +3% and the memory footprint change will go back from 16.22% to +0%. ### Hardware used for benchmarking ``` `NVIDIA RTX 5880 Ada Generation` * SM Version: 890 (PTX Version: 860) * Number of SMs: 110 * SM Default Clock Rate: 18446744071874 MHz * Global Memory: 23879 MiB Free / 48632 MiB Total * Global Memory Bus Peak: 960 GB/sec (384-bit DDR @10001MHz) * Max Shared Memory: 100 KiB/SM, 48 KiB/Block * L2 Cache Size: 98304 KiB * Maximum Active Blocks: 24/SM * Maximum Active Threads: 1536/SM, 1024/Block * Available Registers: 65536/SM, 65536/Block * ECC Enabled: No ``` Authors: - Muhammad Haseeb (https://github.com/mhaseeb123) Approvers: - Yunsong Wang (https://github.com/PointKernel) - David Wendt (https://github.com/davidwendt) URL: #16541
- Loading branch information