Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up searchsorted calls #2000

Merged
merged 13 commits into from
Sep 19, 2023
Merged

Conversation

yger
Copy link
Collaborator

@yger yger commented Sep 15, 2023

Quite a lot everywhere in spikeinterface, searchsorted is called twice and thus a speedup can be gained by making a single call

@alejoe91
Copy link
Member

Thanks Pierre! Can you check why tests are failing?

@alejoe91 alejoe91 added the enhancement New feature or request label Sep 15, 2023
@h-mayorquin
Copy link
Collaborator

import numpy as np

x = np.arange(int(1e8))
first_frame = 500_000
chunk_size = 100_000
last_frame = first_frame + chunk_size

# First case: Two individual searches
def search_individual():
    left1 = np.searchsorted(x, first_frame)
    right1 = np.searchsorted(x, last_frame)
    return left1, right1

# Second case: A single search with both values
def search_together():
    left2, right2 = np.searchsorted(x, [first_frame, last_frame])
    return left2, right2

# Use the timeit magic command to compare
print("First case:")
%timeit search_individual()

print("\nSecond case:")
%timeit search_together()

Output in my machine:

First case:
2.11 µs ± 124 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

Second case:
1.93 µs ± 107 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

@h-mayorquin
Copy link
Collaborator

5 % improvement for every chunk acumulates. This looks good, @yger .

Not relevant in this PR but I wish we used better naming for the left and right bounds than i0 , i1 s0 and s1.

They are comfortable for the person that knows the code (like algebraic expressions are comfortable to operate in a mathematical theory that you already dominate) but are difficult, terse and non-obvious for someone who encounters the expression for the first time.

@h-mayorquin h-mayorquin added the performance Performance issues/improvements label Sep 15, 2023
@yger
Copy link
Collaborator Author

yger commented Sep 15, 2023

I thought it would have been larger, but a gain is a gain :-)

@yger
Copy link
Collaborator Author

yger commented Sep 19, 2023

This is ready to be merged @alejoe91 @samuelgarcia , I only handled the cases where the searchsorted were of the same type (left or right)

@samuelgarcia samuelgarcia merged commit 855a264 into SpikeInterface:main Sep 19, 2023
@yger yger deleted the searchsorted branch September 22, 2023 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance Performance issues/improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants