BF: Skip PCA projection and results storing for spike-less groups

[miketrumpis]

Pull request re: spikedetekt bug for sparsely spiking channels in phy-users link

[rossant]

do you confirm that this fixes the bug in your case?

[miketrumpis]

For me, it's good--recall that I am not doing anything downstream of spikedetekt in phy/klustaviewa.

[miketrumpis]

Scratch that -- I will take a moment in the next few days to go through the full run of klusta and then try to visualize the results.

[miketrumpis]

Ok, "next few days" got out of hand, as well as the weeks following that. Back to this now, I have two results to report:

1. phy foo.kwik seems to run (*)
2. klustaviewa foo.kwik gives me this pytables error message: NoSuchNodeError: group "/channel_groups/15" does not have a child named "clusters"

I'm tracing the lack of path error back to line 254 in klusta/launch.py that (logically) skips clustering if there are no spikes.
https://github.com/kwikteam/klusta/blob/master/klusta/launch.py#L247-L257

(*) I only seem to see spikes from one "shank", but I definitely don't know how to drive this GUI

[rossant]

I'm wondering if adding an empty child named "clusters" would work...

[miketrumpis]

Happy to try this, but I haven't dug into the KwikModel. Can I just open new empty model and save it?

`
model = KwikModel(kwik_path, channel_group=channel_group)

[miketrumpis]

... must have posted & closed with stray keystrokes there.

I was thinking I would just try to save a default KwikModel. I'll see what happens.

[miketrumpis]

I am not having success taking out the no-spikes check I mentioned. If I let the model try to write an empty cluster-ID array, an exception is thrown on the second of these two lines:

https://github.com/kwikteam/klusta/blob/master/klusta/launch.py#L282-L283

The cause is when adding the empty cluster, these lines are blocking the creation of a /channel_groups/15/clusters group, causing a path error in copy_clustering()

https://github.com/kwikteam/klusta/blob/master/klusta/kwik/model.py#L369-L379

I'm not sure how to add an empty cluster that isn't nonsensical. If you want, can you send a patch or suggestion?

[rossant]

OK, I'm wondering whether it might make more sense to slightly change this PR, so that PCA is skipped when the cluster is empty, but the loop iteration is not stopped and the cluster is nevertheless created..? So just create an empty array for the features in this case.

[miketrumpis]

No dice. I can avoid assertion errors here by making some fake feature dimensions (e.g. np.zeros( (1, 0, 0) ))
https://github.com/kwikteam/klusta/blob/master/klusta/traces/store.py#L187-L194

and create a special case for empty sets to avoid an assertion error here:
https://github.com/kwikteam/klusta/blob/master/klusta/kwik/creator.py#L35-L49

Then I run into an object size consistency problem when trying to construct a KwikModel in the clustering phase (traceback). It's not apparent how to make another "empty set" exception from here in a generalized way, and I'd be reluctant to try to fool memmap into making something of nothing.

Rather than hacking the pre-processing, perhaps it would be best to learn why phy does not break down on channels w/o clusters?

[rossant]

Okay so let's go back to the beginning. You have shanks with no spikes, right? klusta failed, and this PR fixes it? Then, phy works (you may have to test with the --channel-group option to try different shanks) but not KlustaViewa. If that's it, then I would say this is good enough. phy is meant to replace KlustaViewa anyway, and I'm guessing this particular case is relatively rare. What do you think?

[miketrumpis]

Yep, phy kwik-gui works with all valid channels groups. It's too bad that the other latent bug was too deep for me to follow, but if you're phasing out KlustaViewa anyway then no big deal.