Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose save preprocessing in ks4 #3276

Merged

Conversation

JoeZiminski
Copy link
Collaborator

@JoeZiminski JoeZiminski commented Jul 31, 2024

This PR edits kilosort4.py to expose the save_preprocessed_copy parameter. Now, temp_wh.dat is saveed to the sorting output and params.py is edited to point phy to the temp_wh.dat. @carlacodes if you have some time maybe you could check off this fork/branch this is behaving as expected? I tested with the below code:

Test code
import numpy as np
import spikeinterface.full as si

recording, _, sorting = si.generate_drifting_recording(
    duration=10.0,
    generate_displacement_vector_kwargs=dict(
        displacement_sampling_frequency=5.0,
        drift_start_um=[0, 20],
        drift_stop_um=[0, -20],
        drift_step_um=1,
        motion_list=[
            dict(
                drift_mode="zigzag",
                non_rigid_gradient=None,
                t_start_drift=None,
                t_end_drift=None,
                period_s=200,
            ),
        ],
    ),
)

si.run_sorter("kilosort4", recording, save_preprocessed_copy=True)

At present this is not tested in #3085 but that can be extended after this PR is merged.

More generally @alejoe91, at present kilosort4.py replicates the body of run_sorter.py but as versions progress, this might prove difficult to maintain and requires a lot of testing e.g. #3085 to check nothing was missed. I wonder if long-term (assuming I am correct in thinking this is done to skip: preprocessing and drift correction), we can ask / work with KS4 team to expose options to skip preprocessing if they are interested. Then, we could just wrap the run_kilosort function directly and testing would just be checking the kwargs are passed properly (rather than having to check the outputs). Maybe there are other considerations I have overlooked though, WDYT?

@alejoe91 alejoe91 self-assigned this Jul 31, 2024
@JoeZiminski JoeZiminski marked this pull request as ready for review July 31, 2024 17:05
@alejoe91
Copy link
Member

alejoe91 commented Aug 1, 2024

@JoeZiminski this LGTM. @carlacodes were you able to test this?

Copy link

@carlacodes carlacodes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; tested on my own data with ks 4.0.12

do_CAR,
invert_sign,
device,
save_preprocesed_copy=save_preprocessed_copy, # this kwarg is correct (typo)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No way..... I have a feeling is is going to be fixed and then we will need another version just to account for this....

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zm711 you need to reviow KS PRs ahahahah

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fixed as of 4.0.13 (MouseLand/Kilosort@633ee92).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoeZiminski maybe we have to go back to positional arguments here? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think so @alejoe91 or the number of conditionals will explode, shall we merge this and then continue the discussion on how best to proceed here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do that!

@alejoe91 alejoe91 added the sorters Related to sorters module label Aug 2, 2024
@dhmjhu
Copy link

dhmjhu commented Aug 19, 2024

I've been using a locally modified version of SpikeInterface that exposes save_preprocessed_copy, but I would love to see support for this feature in an official release.

Does this PR have a chance of being merged (after the spelling is fixed)?

@alejoe91 alejoe91 merged commit 694f862 into SpikeInterface:main Aug 20, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sorters Related to sorters module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants