Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization of get_noise_levels (and thus get_random_data_chunks) #2380

Open
yger opened this issue Jan 2, 2024 · 0 comments
Open

Optimization of get_noise_levels (and thus get_random_data_chunks) #2380

yger opened this issue Jan 2, 2024 · 0 comments
Assignees
Labels
performance Performance issues/improvements

Comments

@yger
Copy link
Collaborator

yger commented Jan 2, 2024

Actually, get_noise_levels makes internal use of get_random_data_chunks that is not parallel. There is thus room for improvements, via parallelization. This could be achieved at low cost by using the run_node_pipeline machinery, assuming some patch like #2011 would be accepted. If this is the case (or we can discuss somehting in line and similar), one could simply provide a list of fake "peak" times, some temporal window, and use the WaveformDense object of the pipeline to get data in parallel. Adding an extra job_kwargs to get_noise_levels would then speed up everything. @samuelgarcia what do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance issues/improvements
Projects
None yet
Development

No branches or pull requests

1 participant