Various fixes to reproducible benchmarks #1800
Conversation
…in the next commit
…n-ann-bench-use-gbench
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
| }; | ||
|
|
||
| using typename ANN<T>::AnnSearchParam; | ||
| struct SearchParam : public AnnSearchParam { | ||
| int ef; | ||
| int num_threads{1}; | ||
| int num_threads = omp_get_num_procs(); |
There was a problem hiding this comment.
Does this work well with num_threads > n_queries? We've had a similar logic in raft host refinement and the performance on small batches was horrible due to overheads of managing many threads compared to the amount of work (n_queries = 1).
There was a problem hiding this comment.
Good question. This one is challenging because we don't (and shouldn't) know the number of queries when we set this argument. We could set it to something small(ish) like 8 or 16, but that would just lower the saturation point for larger batch queries. In general, I know the queries for online systems are going to be long tailed, with 1 being in the main mass and >= 100 being in the tail.
The problem is that when we run larger batch sizes, we aren't giving hnsw an fair try at all. My thinking was to take the middle ground- setting this to the number of cores. I guess I should measure the impact directly. What kind of perf difference are you seeing for, say, batch size of 10 when the thread pool contains the number of available cores?
There was a problem hiding this comment.
Maybe this explicit pool does better job than the openmp machinery that we rely upon in the refine operation. But there, I've got something like ~100x boost for a single-query batch n_queries = 1 (72 cores). I've done some other refactoring at the same time though.
|
/ok to test |
Co-authored-by: Artem M. Chirkin <[email protected]>
|
/ok to test |
|
/merge |
|
/ok to test |
|
/merge |
No description provided.