Curation format issues

[Antho2422]

Hi all,

There is an issue with the gui when doing the curation and saving as JSON. The output JSON from the spikeinterface gui does not have the "format_version": key. This causes issues when trying to run the apply_curation function since it check the dictionnary structure.

Cheers,
Anthony

[Antho2422]

Following this issue I would like to know if there is an agreement on the curation dict format that you would like to use in the future ?

I'm encountering a lot of issues related to various formats when running the apply_curation() function.

• When having manual labels in a list like so : "manual_labels": [{"unit_id": 0, "quality": ["MUA"]}] and trying to also have a merge group, I'm encoutering this ValueError ValueError: shape mismatch: value array of shape (2,) could not be broadcast to indexing result of shape (1,) on the call of set_property(). This is triggered by line 257 of curation_format.py

if len(set(group_values)) == 1:
         # all group has the same label or empty
         sorting.set_property(key, values=group_values, ids=[new_unit_id])

group_values is a list of one element. It is pretty easy to fix just indexing the 1st element of group_values would work I guess but I do not want to make a PR because maybe you are changing the code and the format of the dict atm ?

• I tried to directly modify the dictionnary by giving manual labels as str like so : "manual_labels": [{"unit_id": 0, "quality": ["MUA"]}] but when doing that the issue comes from this check line 62 of curation_format.py :

if not isinstance(label_value, list):
    raise ValueError(f"Curation format: manual_labels {unit_id} is invalid shoudl be a list")

What should we do ?

Thanks

[Antho2422]

I also would like to mention this related issue in the call of analyzer.merge_units() in the curation_format.py file. When there is no merges to perform in the curation_dict the function raises a TypeError cannot unpack non-iterable SortingAnalyzer object.

My fix is the following :

        if len(curation_dict['merge_unit_groups']) == 0:
            new_unit_ids = analyzer.unit_ids
        else:
            analyzer, new_unit_ids = analyzer.merge_units(
                curation_dict["merge_unit_groups"],
                censor_ms=censor_ms,
                merging_mode=merging_mode,
                sparsity_overlap=sparsity_overlap,
                new_id_strategy=new_id_strategy,
                return_new_unit_ids=True,
                format="memory",
                verbose=verbose,
                **job_kwargs,
            )

But maybe you prefere modifying the merge_units() method directly in the sorting analyzer to make it consistent of the same case when the object is a sorting and not an analyzer ?

[alejoe91]

Thanks @Antho2422

We will have a round of debugging for the curation format and the integration with the GUI and merge function! Thanks for opening the issues. We'll get back to you when there is a PR yoyu can test :)