Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Savitzky-Golay filter in suppression detection #102

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

vloison
Copy link
Collaborator

@vloison vloison commented May 19, 2021

Allows the detect_suppressions function to work on recordings on which artifacts have been detected.

I'm not sure what tests you want me to add to the pull request, please let me know if I need to add stuff.

@mattbit mattbit changed the title remove savgol filter Remove Savitzky-Golay filter in suppression detection May 19, 2021
Copy link
Owner

@mattbit mattbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verify tests, check that everything makes sense, and add new tests.

@mattbit
Copy link
Owner

mattbit commented May 19, 2021

I'm not sure what tests you want me to add to the pull request, please let me know if I need to add stuff.

Start by checking the existing ones (currently failing). Make sure that the detection does not break.

Then add similar tests for signals containing artifacts, i.e. what happens when you have an artifact in the middle of a suppression? Write a test which ensures the correct behaviour.

@mattbit mattbit added the bug Something isn't working label May 19, 2021
@vloison
Copy link
Collaborator Author

vloison commented May 19, 2021

Done !
I added a function test_artifact_in_suppression in test_suppressions.py to make sure that if there is an artifact in the middle of a suppression, the suppressions don't overlap with the artefact.
I figured that we can't assume anything about what happens during an artifact. So it makes sens to make sure that we don't put any informations during an artifact. So the code is written so that detected suppressions never overlap with artifacts.

with np.errstate(invalid='ignore'):
ies_mask = envelope < threshold
ies_mask = envelope <= threshold
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this change?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because otherwise something occurs when, during the suppression, there is a single sample with value threshold, placed at a time t < min_duration after the beginning of the suppressions. In this case, the interval before the problematic sample is not considered as a suppression. This is what made the tests fail on the first commit. An AssertionError was raised when running test_detect_suppressions(3). Because the sample 532 was in the situation I mentionned before.
git pb

# The suppressions must not overlap with the artifact
sec = pd.Timedelta('1s')
assert detections.loc[0].end < 3.2 * sec
assert detections.loc[1].end > 4.8 * sec
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably start

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, sorry about that, I'll change it

@mattbit
Copy link
Owner

mattbit commented May 19, 2021

Cannot reproduce, see #79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs more info
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Problems with Savitzky–Golay filter in detection of alpha suppressions
2 participants