Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dredge lfp and dredge ap #3062

Merged
merged 37 commits into from
Jul 15, 2024
Merged

Conversation

samuelgarcia
Copy link
Member

@samuelgarcia samuelgarcia commented Jun 21, 2024

@cwindolf

Here the initial port of dredgelib into spikeinterface.
This come with some small refactoring in spikeinterface and folder reorganization.

We should be able to do:

estimate_motion(method="drege_lfp")
estimate_motion(method="drege_ap")

The come a a quite big refactoring to split methods and into files and many variable names change to align with dredge in spikeinterface and sometimes the align dredge with spikeinterface.
The entire dredge is in one unique file dredge.py

@alejoe91 alejoe91 added the motion correction Questions related to motion correction label Jun 22, 2024
Move method into separated files.
Change, refactor (and rename some) kwargs from common to specific methos.
Add dredge_lfp class.
@samuelgarcia
Copy link
Member Author

@cwindolf the last commit is a quite big refactoring in estimate motion to integrate the 'dredge_lfp' method.
Maybe you would like have a look.

@samuelgarcia
Copy link
Member Author

samuelgarcia commented Jun 26, 2024

@cwindolf I could not resists in porting also the dredge_ap and make a class.
This is not working yet because we need to fix the histogram2d that mimic all imrovements.

@samuelgarcia samuelgarcia changed the title Dredge lfp Dredge lfp and dredge ap Jun 26, 2024
@alejoe91 alejoe91 added this to the 0.101.0 milestone Jun 27, 2024
Copy link
Collaborator

@cwindolf cwindolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with stuff pretty much! Most of my "review" here is a todo for me to come back in the next PR for dredge AP.

src/spikeinterface/sortingcomponents/motion/dredge.py Outdated Show resolved Hide resolved
src/spikeinterface/sortingcomponents/motion/dredge.py Outdated Show resolved Hide resolved
src/spikeinterface/sortingcomponents/motion/dredge.py Outdated Show resolved Hide resolved
mincorr_percentile_nneighbs=None,
# raster arguments
amp_scale_fn=None, ## @Charlie this one is not used anymore
post_transform=np.log1p, ###@this one is directly transimited to weight_correlation_matrix() and so get_wieiith()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually we can probably not even transmit it. we can apply the function to the threshold in advance before passing it to weight_correlation_matrix to reduce the number of params. I can do this in my PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. I left the code like this for the moment.

src/spikeinterface/sortingcomponents/motion/dredge.py Outdated Show resolved Hide resolved
src/spikeinterface/sortingcomponents/motion/dredge.py Outdated Show resolved Hide resolved
rigid=False,
# nonrigid window construction arguments
win_shape="gaussian",
win_step_um=400,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think that these defaults are the best in typical NP1 data :)

thomas_kw, xcorr_kw, raster_kw, weights_kw
These dictionaries allow setting parameters for fine control over the registration
device : str or torch.device
What torch device to run on? E.g., "cpu" or "cuda" or "cuda:1".
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add more documentation of the other parameters in my PR.

yger
yger previously requested changes Jul 12, 2024
Copy link
Collaborator

@yger yger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! I'll test it today, and you need to fix tests also with some non appropriate imports that are still there

smooth_kernel = smooth_kernel[:, None]
motion_clean = scipy.signal.fftconvolve(motion_clean, smooth_kernel, mode="same", axes=0)

return motion_clean
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this function used? Should it be an option at the end of estimate_motion, extending the signature with motion_cleaner_kwargs as a dict, and if not None, it is launched?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not used currently, but it's an option. Let's discuss this later

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is now separated from the main estimate. I think this is better. and honestly nobody used it.

return motion_histogram, temporal_bin_edges, spatial_bin_edges


def make_3d_motion_histograms(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit of a pity that make_2d_motion_histogram and makd_3d_motion_histogram are not the same function, one being a particular case of the other. Because conceptually, they should. We could also propose some smoothing for 3d histogram over time, amplitudes and depth. What do you think? One single function could ease the code maintenance

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's make it in a follow up PR

@alejoe91 alejoe91 marked this pull request as ready for review July 12, 2024 10:02
@alejoe91
Copy link
Member

@samuelgarcia @yger

  • I cleaned up the docs, looking good now :)

  • There was a failing test due to this: the test recording probe is too short for the default win size and margin, resulting in num_windows = -1. I extended the get_spatial_windows to deal with this and switching to rigid motion in case window sizes/margins are too large for the probe depth. This produces a warning.

  • I changed contact_depth to contact_depths, since it's more accurate!

@@ -13,7 +13,7 @@ def test_estimate_and_correct_motion(create_cache_folder):
if folder.exists():
shutil.rmtree(folder)

rec_corrected = correct_motion(rec, folder=folder)
rec_corrected = correct_motion(rec, folder=folder, estimate_motion_kwargs={"win_step_um": 20})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test recording depth is not that high!

@alejoe91
Copy link
Member

One last fix needed to use numpy.broadcast_to instead of torch.broadcast_to if conv_engine is numpy

@alejoe91
Copy link
Member

In last commit: propagate API changes and docs

@alejoe91 alejoe91 merged commit c9fc8e1 into SpikeInterface:main Jul 15, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
motion correction Questions related to motion correction
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants