Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch to skip chunks without peaks or if enough peaks have been detected #2011

Closed
wants to merge 32 commits into from
Closed
Changes from 9 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
1720e15
Patch to skip chunks without peaks
yger Sep 19, 2023
2760279
Merge branch 'main' into skip_no_peaks
yger Sep 19, 2023
7f72b21
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 19, 2023
40b092d
Patch
yger Sep 19, 2023
c7aef09
Patch
yger Sep 19, 2023
4bee4b1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Sep 19, 2023
f4accb7
Merge branch 'main' into skip_no_peaks
yger Sep 19, 2023
226b73a
Only for Spike and PeakRetriever
yger Sep 19, 2023
68d6017
Merge branch 'main' into skip_no_peaks
yger Sep 20, 2023
b226335
Merge branch 'main' into skip_no_peaks
yger Nov 15, 2023
d564309
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Nov 24, 2023
386a23e
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Nov 29, 2023
da5f816
Merge branch 'main' into skip_no_peaks
yger Jan 2, 2024
87a6606
Skipping the loop if not peaks
yger Jan 2, 2024
5272d2b
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 3, 2024
ad54cfa
Extension to deal with max_peaks
yger Jan 3, 2024
8c589f9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 3, 2024
e9d456b
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 3, 2024
f9c3274
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 10, 2024
7250653
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 10, 2024
c4c147c
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 12, 2024
5950998
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 12, 2024
5b5a080
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 12, 2024
d4a7c0f
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 15, 2024
6d1838e
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 19, 2024
f80a1e9
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 19, 2024
d679506
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 22, 2024
971f4fd
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 29, 2024
5e2eae0
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Jan 30, 2024
03945d5
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Mar 11, 2024
54ced06
Merge branch 'SpikeInterface:main' into skip_no_peaks
yger Mar 12, 2024
6a6bcd7
Merge branch 'main' into skip_no_peaks
yger Jul 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions src/spikeinterface/core/node_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,12 @@ def __init__(self, recording, peaks):
def get_trace_margin(self):
return 0

def has_peaks(self, start_frame, end_frame, segment_index):
sl = self.segment_slices[segment_index]
peaks_in_segment = self.peaks[sl]
i0, i1 = np.searchsorted(peaks_in_segment["sample_index"], [start_frame, end_frame])
return i0 < i1

def get_dtype(self):
return base_peak_dtype

Expand Down Expand Up @@ -187,6 +193,12 @@ def __init__(
def get_trace_margin(self):
return 0

def has_peaks(self, start_frame, end_frame, segment_index):
sl = self.segment_slices[segment_index]
peaks_in_segment = self.peaks[sl]
i0, i1 = np.searchsorted(peaks_in_segment["sample_index"], [start_frame, end_frame])
return i0 < i1

def get_dtype(self):
return base_peak_dtype

Expand Down Expand Up @@ -486,9 +498,18 @@ def _compute_peak_pipeline_chunk(segment_index, start_frame, end_frame, worker_c
nodes = worker_ctx["nodes"]

recording_segment = recording._recording_segments[segment_index]
traces_chunk, left_margin, right_margin = get_chunk_with_margin(
recording_segment, start_frame, end_frame, None, max_margin, add_zeros=True
)

if isinstance(nodes[0], PeakRetriever) or isinstance(nodes[0], SpikeRetriever):
if nodes[0].has_peaks(start_frame, end_frame, segment_index):
traces_chunk, left_margin, right_margin = get_chunk_with_margin(
recording_segment, start_frame, end_frame, None, max_margin, add_zeros=True
)
else:
traces_chunk, left_margin, right_margin = None, None, None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do not have traces maybe we can also skip the node loop no ?

else:
traces_chunk, left_margin, right_margin = get_chunk_with_margin(
recording_segment, start_frame, end_frame, None, max_margin, add_zeros=True
)

# compute the graph
pipeline_outputs = {}
Expand Down