Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize zrank to avoid path comparisons #1389

Draft
wants to merge 1 commit into
base: unstable
Choose a base branch
from

Conversation

ranshid
Copy link
Member

@ranshid ranshid commented Dec 4, 2024

ZRANK is a widly used command for workloads using sorted-sets. For example, in leaderboards It enables query the specific rank of a player.
The way ZRANK is currently implemented is:

  1. locate the element in the SortedSet hashtable.
  2. take the score of the element and use it in order to locate the element in the SkipList (when listpack encoding is not used)
  3. During the ListPack scan for the elemnt we keep the path and use it in order to sum the span in each path node in order to calculate the elemnt rank

One problem with this approach is that it involves multiple compare operations in order to locate the element. Specifically string comparison can be expensive since it will require access multiple memory locations for the items the element string is compared against.
Perf analysis showed this can take up to 20% of the rank scan time. (TBD - provide the perf results for example)

We can improve the rank search by taking advantage of the fact that the element node in the skiplist is pointed by the hashtable value!
Our Skiplist implementation is using FatKeys, where each added node is assigned a randomly chosen height. Say we keep a height record for every skiplist element. In order to get an element rank we simply:

  1. locate the element in the SortedSet hashtable.
  2. we go directly to the node in the skiplist.
  3. we jump to the full height of the node and take the span value.
  4. we continue going foreward and always jump to the heighst point in each node we get to, making sure to sum all the spans.
  5. we take off the summed spans from the SkipList length and we now have the specific node rank :)

In order to test this method I creted several benchmarks. All benchmarks used the same seeds and the lists contained 1M elements. Since a very important factor is the number of scores compared to the number of elements (since small ratio means more string compares during searches) each benchmark test used different number of scores (1, 1K, 10K, 1M)
some results:

TPS

<style> </style>
Scores range non-optimized optimized gain
1 416042 605363 45.51%
10K 359776 459200 27.63%
100K 380387 459157 20.71%
1M 416059 450853 8.36%

Latency

<style> </style>
Scores range non-optimized optimized gain
1 1.191000 0.831000 -30.23%
10K 1.383000 1.095000 -20.82%
100K 1.311000 1.087000 -17.09%
1M 1.191000 1.119000 -6.05%

Memory efficiency

adding another field to each skiplist node can cause degredation in memory efficiency for large sortedsets. We use the fact that level 0 recorded span of ALL nodes can either be 1 or zero (for the last node). So we use wrappers in order to get a node span and override the span for level 0 to hold the node height.

We can support a faster zrank by walking bottom-up instead of walk-downw

Signed-off-by: Ran Shidlansik <[email protected]>
Copy link

codecov bot commented Dec 4, 2024

Codecov Report

Attention: Patch coverage is 98.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 70.88%. Comparing base (105509c) to head (25877ca).

Files with missing lines Patch % Lines
src/t_zset.c 98.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #1389      +/-   ##
============================================
+ Coverage     70.83%   70.88%   +0.05%     
============================================
  Files           118      118              
  Lines         63549    63581      +32     
============================================
+ Hits          45013    45068      +55     
+ Misses        18536    18513      -23     
Files with missing lines Coverage Δ
src/server.h 100.00% <ø> (ø)
src/t_zset.c 95.62% <98.00%> (-0.03%) ⬇️

... and 15 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant