Optimize gathering of point cloud colors #4540

Job	Run time
Checks / Python lints (black, mypy, flake8)	7m 57s
Checks / Test Python Docs	30s
Checks / Check if running codegen would produce any changes	1m 17s
Checks / Rust lints (fmt, check, cranky, tests, doc)	2m 31s
Checks / Check Rust web build (wasm32 + wasm-bindgen)	2m 15s
Checks / Lint TOML files	27s
Checks / Rerun lints	3m 54s
Checks / Spell Check	8s
Checks / Cargo Deny	1m 34s
Checks / C++ formatting check	2m 35s
Checks / C++ tests	7m 6s
Build & Upload rerun_c (Linux x64) / Set Config (linux)	2s
Minimum Wheel / Set Config (linux)	2s
Build Web / Build Web (wasm32 + wasm-bindgen)	4m 41s
Update PR Body / Update PR body	2m 54s
Minimum Wheel / Build Wheels (ubuntu-latest-16-cores)	10m 0s
Build & Upload rerun_c (Linux x64) / Build rerun_c (ubuntu-latest-16-cores)	1m 58s
Upload Web / Upload web build to google cloud (wasm32 + wasm-bindgen)	17s
Run Notebook / Run notebook	1m 36s
Build Web Demo / Build Web Demo	4m 51s
Save PR Summary / Create HTML summary for PR	48s
Track Sizes / Track Sizes	1m 2s
Upload Web Demo / Upload Web Demo to Google Cloud	1m 5s
	59m 30s

Provide feedback