Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify compute_isi_violation docs and add UltraMegaSort2000 citation #3070

Merged

Conversation

chrishalcrow
Copy link
Collaborator

An attempt to close #2050

Two things:

  1. Unify the notation between isi_violation and rp_violation in the docs so that the functions are easier to compare.
  2. Get a correct citation for isi_violation: the current implementation is not actually from the Hill paper, it's a simpler function originally from UltraMegaSort2000 (https://github.com/danamics/UMS2K, https://github.com/danamics/UMS2K/blob/master/quality_measures/rpv_contamination.m) which is also written by Hill. I think citing this directly is much less confusing (see more confusion here: Contamination ratio #1973, and see https://github.com/cortex-lab/sortingQuality/blob/70c8659adc60484434be828d617e16eb83e94cca/core/ISIViolations.m#L17 for another discussion. Here our isi_violation formula is "the approximation previously used" which "wasn't accurate to the Hill paper" )

@chrishalcrow chrishalcrow added the documentation Improvements or additions to documentation label Jun 25, 2024
Copy link
Collaborator

@JoeZiminski JoeZiminski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Some small suggestions

src/spikeinterface/qualitymetrics/misc_metrics.py Outdated Show resolved Hide resolved
src/spikeinterface/qualitymetrics/misc_metrics.py Outdated Show resolved Hide resolved
doc/modules/qualitymetrics/isi_violations.rst Outdated Show resolved Hide resolved
doc/modules/qualitymetrics/isi_violations.rst Outdated Show resolved Hide resolved
doc/modules/qualitymetrics/isi_violations.rst Outdated Show resolved Hide resolved
doc/modules/qualitymetrics/isi_violations.rst Show resolved Hide resolved
doc/modules/qualitymetrics/isi_violations.rst Outdated Show resolved Hide resolved
@alejoe91 alejoe91 added this to the 0.101.0 milestone Jun 27, 2024
@JoeZiminski JoeZiminski self-requested a review June 28, 2024 16:02
Copy link
Collaborator

@JoeZiminski JoeZiminski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great stuff!

Copy link
Collaborator

@zm711 zm711 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing to stop merging, but a couple potential tweaks around the edges from my perspective.

doc/references.rst Outdated Show resolved Hide resolved
Comment on lines 270 to 272
The returned ISI violations ratio measures the approximate fraction of spikes in each
unit which are contaminted. This interpretation is good when the ratio is small, and
becomes worse as it grows. In cases of highly contaminated units, the ISI violations
Copy link
Collaborator

@zm711 zm711 Jul 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't 100% clear. Worse being what exactly... "As it grows past 0.xx it begins to diverge from xx" something like that based on llobet's comments in the issue.

Copy link
Collaborator Author

@chrishalcrow chrishalcrow Jul 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm. I was deliberately vague because the success of the approximation depends on lots of factors: when the contamination is large is it because there are LOTS spikes near the same time a few times, or many violations of where one spike violates only one other spike? The Llobet comment is on one specific example (uniformly generated random spikes) so you shouldn't take precise numbers from it.

All the models agree when $n_vT/N^2t_r$ is small (much less than one), so we could say that the interpretation only works in that situation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair. Okay so maybe something more like:

This intrepretation is good when the ratio is small, but as the ratio approaches 1 it is a less reliable indicator (or something like that? ). I know the ratio can go above one so maybe we don't even want that. But "becomes worse" is he vague part for me. ie becomes worse at what?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've realised that's also not true: you can cook up examples (where each spike is contaminated by exactly one other spike) where the ratio is a reliable indicator. So we can only really say: sometimes this is a good indicator, sometimes it isn't (like most of the quality metrics!). I suggest we refer to the Llobet paper, which discusses some of the different models? I've had a go.

src/spikeinterface/qualitymetrics/misc_metrics.py Outdated Show resolved Hide resolved
@samuelgarcia samuelgarcia merged commit 8e9bb8c into SpikeInterface:main Jul 3, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unify notation in ISI violations docs
5 participants