-
Notifications
You must be signed in to change notification settings - Fork 597
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: scan concurrently in RowSeqScanExecutor
#12783
Comments
Yes, multiple iterators are better than the explicit parallelism adjustment in #12741, with less change. |
Actually, this only affects the concurrency for creating the iterators, which has been confirmed (IIRC) to be CPU-intensive in most cases. For actual scanning, we're always in a single-threaded manner. risingwave/src/storage/src/table/batch_table/storage_table.rs Lines 492 to 499 in 80f1d58
|
It affects the number of iterators in the |
Oh, I get it now, So |
I'm sorry but by "single-threaded" here I mean the I/O is not concurrent as well. |
I saw #6927, where we intentionally to create fewer iterators, so I'm a little confused what to optimize for. 🤡 |
we need to scan concurrently for IO bound scan by polling multiple iterators at one time. So we need to split the range into multiple pieces. |
When no key ranges predicate on the iterator, the code will create as few as possible numbers of iterator, which could decrease the scan concurrency. So when scanning the whole table, it could be bounded by the iterator.
risingwave/src/storage/src/table/batch_table/storage_table.rs
Lines 421 to 438 in 80f1d58
The text was updated successfully, but these errors were encountered: