From c18e4a1f19ce2fd2331eeac4b021276fbbdbeba8 Mon Sep 17 00:00:00 2001 From: Alessio Buccino <alejoe9187@gmail.com> Date: Fri, 27 Oct 2023 15:50:56 +0200 Subject: [PATCH 1/7] Add release notes for 0.99.0 --- doc/releases/0.99.0.rst | 136 ++++++++++++++++++++++++++++++++++++++++ doc/whatisnew.rst | 8 +++ pyproject.toml | 8 +-- 3 files changed, 148 insertions(+), 4 deletions(-) create mode 100644 doc/releases/0.99.0.rst diff --git a/doc/releases/0.99.0.rst b/doc/releases/0.99.0.rst new file mode 100644 index 0000000000..a0ddf3273b --- /dev/null +++ b/doc/releases/0.99.0.rst @@ -0,0 +1,136 @@ +.. _release0.99.0: + +SpikeInterface 0.99.0 release notes +----------------------------------- + +31th October 2023 + +Main changes: + +* Refactor generate.py +* Widgets +* Docs +* New qualitymetrics and template metrics + + +core: + +* Handle edge frames in concatenated recordings (#1883) +* Use spike_vector in count_num_spikes_per_unit (#1895) +* Fix waveform extarctor crash with unfiltered wvf_extractor and sparsity (#1902) +* Restore npzfolder.py file to load previously saved sorting objects (#1903) +* Convert from samples<->times directly on BaseRecordings (#1921) +* Move peak_pipeline into core and rename it as node_pipeline (#1941) +* Implement SpikeRetriever pipeline source (#1944) +* Refactor and improve generate.py (#1948, #1970, #2098) +* Relax check_borders in InjectTemplatesRecording (#1988) +* Speed up searchsorted calls across code-base (#2000) +* Improve serialization concept : memory/json/pickle (#2027) +* Check that main_ids are str or int (#2061) +* Change some default parameters for better user experience (#2071) +* Fix NoiseGeneratorRecording : "noise_level" in kwargs (#2084) +* Add sparsity_temp_folder option to extract_waveforms (#2133: TBM) +* Re-add relative_to to dump_to_pickle (#2129: TBM) + + +extractors: + +* Fix NWB streaming: do not convert to Path if ros3 or fsspec (#1875) +* Add stream_name option to read_cbin_ibl (#1928) +* Add RecordingExtractor for NeuroExplorer (#1965) +* Fix Kilosort Phy reader docstrings (#2022) +* Add ignore_timestamps_errors to OpenEphysLegacyRecordingExtractor (#2043) +* CellExplorer: fix tests and deprecation (#2048) +* Patch with scipy for remove_artefacts when mode is linear/cubic (#2072) +* Fix grouping of OpenEphys NPIX extractors (#2102) +* Fix open ephys probe loading and unify probeinterface import syntax (#2136: TBM) + + +preprocessing: + +* Patch silence_periods for mono-segment recordings (#1946) +* Fix zero_channel_pad case when selected frames are outside of original data region (#1979: TBM) +* Improve detect_bad_channels defaults and fixes (#2003) +* Add an option to flip the order by depth (#2005) +* Update a test expression to remove two logical short circuits in remove_artifacts (#2025) +* Adjust eps for whitening in case of very small magnitude data (#2070) + + +sorters: + +* Add missing tic in ks*_master when skipping preprocessing (#1887) +* Change the signature on kilosort's delete intermediate files parameters (#1908) +* Check if recording is JSON-serializable in run_sorter (#1932) +* Spyking Circus 2 updates (#1943, #2074, #2080, #2086) +* Tridesclous 2 updates (#2076, #2087, #2104) +* Refactor sorter launcher. Deprecated run_sorters() and add run_sorter_jobs() (#1968) +* Fixes Apptainer bug (#2053) +* Use versioned kilosort options as default in skip_preprocessed_data (#2108) + +postprocessing: + +* Amplitude scaling: add collisions handling (#1940) +* Spike location with true spike channel (#1950) +* Allow to postprocess on read-only waveform folders (#1957) +* Refactor ISI calculation numpy and numba (#1958) +* Extend and refactor waveform metrics, including multi-channel metrics (#1993) +* Use default cutouts for peak-sign test (#2121) + +qualitymetrics: + +* Fix bug in pca metrics warnings (#1874) +* Implement syncrhrony metrics (#1951) +* Additional amplitude CV and firing range metrics (#1981, #2054) + +comparison: + +* Fix missing import pandas in comparison (#1872) +* Refactoring of GroundTruthStudy (#1983) +* Save/load comparison objects with pickle (#2013) +* Improved GTStudy and related widgets (#2128) +* Optimize make_match_count_matrix with numba (#2114) + +curation: + +* Allow MergeUnitsSorting to handle tuple inputs (#1995) +* Fix: Correct unit ID matching in sortingview curation (#2037) + +exporters: + +* Fix bug in export_to_phy with sparsity (#1904) +* Allow relative path for export_to_phy (#2041) + + +widgets: + +* Port legacy widgets to new API (#2016, #2068, #2115, #2124) +* For connoisseur only: add a simple "ephyviewer" backend plot_traces() (#2024) +* Improve ipywidgets backend (#2035, #2079, #2091) +* Improved GTStudy and related widgets (#2128) + + +sortingcomponents: + +* Implement proof of concept merge_clusters/split_clusters (#1996) +* Add SVD convolutions to template matching (#2045) + +docs: + +* Several typo and display fixes, mainly thanks to @zm711 (#1866, #1898, #1990, #1992, #1936, #1953, #2015, #2019, #2036, #2057, #2100, #2103, #2130) +* Update installation Tips (#1962) +* Add "how to" section on how to load matlab data (#2018, #2047) +* Add link to latest pypi version docs in readme (#2060) +* Improve assert messages (#2078, #2109, #2119) +* Unify 'defaults' and use of quotations across docsrtings (#2134: TBM) + + +ci / packaging / tests: + +* Fix the [full] install for Macs (#1955) +* Change np.in1d to np.isin as the former will be deprecated (#2002) +* Fix folder for installation test build (#2004) +* Extend common postprocessing tests to spikes at borders (#2014) +* Add SPIKEINTERFACE_DEV_PATH to aws gpu tests (#2088) +* Fix git-annex installation (#2097) +* Add actual test for check_your_install (#2112) +* Add quality_metrics specific module requirements to pyproject.toml (#2135: TBM) diff --git a/doc/whatisnew.rst b/doc/whatisnew.rst index 8b984e2510..93f61eeddc 100644 --- a/doc/whatisnew.rst +++ b/doc/whatisnew.rst @@ -8,6 +8,7 @@ Release notes .. toctree:: :maxdepth: 1 + releases/0.99.0.rst releases/0.98.2.rst releases/0.98.1.rst releases/0.98.0.rst @@ -31,6 +32,13 @@ Release notes releases/0.9.1.rst +Version 0.99.0 +============== + + + + + Version 0.98.2 ============== diff --git a/pyproject.toml b/pyproject.toml index 51efe1f585..b0ce5f98b6 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "spikeinterface" -version = "0.99.0.dev0" +version = "0.99.0" authors = [ { name="Alessio Buccino", email="alessiop.buccino@gmail.com" }, { name="Samuel Garcia", email="sam.garcia.die@gmail.com" }, @@ -25,7 +25,7 @@ dependencies = [ "joblib", "threadpoolctl", "tqdm", - "probeinterface>=0.2.17", + "probeinterface>=0.2.18", ] [build-system] @@ -139,8 +139,8 @@ test = [ # for github test : probeinterface and neo from master # for release we need pypi, so this need to be commented - "probeinterface @ git+https://github.com/SpikeInterface/probeinterface.git", - "neo @ git+https://github.com/NeuralEnsemble/python-neo.git", + # "probeinterface @ git+https://github.com/SpikeInterface/probeinterface.git", + # "neo @ git+https://github.com/NeuralEnsemble/python-neo.git", ] docs = [ From 29f7864e02ad9aa16be78820ce64d30eaddf4867 Mon Sep 17 00:00:00 2001 From: Alessio Buccino <alejoe9187@gmail.com> Date: Fri, 27 Oct 2023 15:58:37 +0200 Subject: [PATCH 2/7] Add main changes --- doc/releases/0.99.0.rst | 13 +++++++------ doc/whatisnew.rst | 8 ++++++-- 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/doc/releases/0.99.0.rst b/doc/releases/0.99.0.rst index a0ddf3273b..fb9df451e6 100644 --- a/doc/releases/0.99.0.rst +++ b/doc/releases/0.99.0.rst @@ -7,10 +7,12 @@ SpikeInterface 0.99.0 release notes Main changes: -* Refactor generate.py -* Widgets -* Docs -* New qualitymetrics and template metrics +* Refactored :code:`core.generate` module to create high-quality lazy ground-truth data +* Added DeepInterpolation training, transfer, and inference to preprocessing +* Added new quality_metrics (syncrhrony, firing_range, amplitude_cv) +* Added new template metrics (num positive/negative peaks + multi-channel metrics: velocities, spread, exp. decay) +* Ported all widgets to "new" API and dismantled legacy widgets +* Extensive clean-up of docstrings, code style, and documentation core: @@ -48,6 +50,7 @@ extractors: preprocessing: +* Deepinterpolation revived: training, transfer, and inference with more flexibility (#1804) * Patch silence_periods for mono-segment recordings (#1946) * Fix zero_channel_pad case when selected frames are outside of original data region (#1979: TBM) * Improve detect_bad_channels defaults and fixes (#2003) @@ -100,7 +103,6 @@ exporters: * Fix bug in export_to_phy with sparsity (#1904) * Allow relative path for export_to_phy (#2041) - widgets: * Port legacy widgets to new API (#2016, #2068, #2115, #2124) @@ -108,7 +110,6 @@ widgets: * Improve ipywidgets backend (#2035, #2079, #2091) * Improved GTStudy and related widgets (#2128) - sortingcomponents: * Implement proof of concept merge_clusters/split_clusters (#1996) diff --git a/doc/whatisnew.rst b/doc/whatisnew.rst index 93f61eeddc..f7466c1129 100644 --- a/doc/whatisnew.rst +++ b/doc/whatisnew.rst @@ -35,8 +35,12 @@ Release notes Version 0.99.0 ============== - - +* Refactored :code:`core.generate` module to create high-quality lazy ground-truth data +* Added DeepInterpolation training, transfer, and inference to preprocessing +* Added new quality_metrics (syncrhrony, firing_range, amplitude_cv) +* Added new template metrics (num positive/negative peaks + multi-channel metrics: velocities, spread, exp. decay) +* Ported all widgets to "new" API and dismantled legacy widgets +* Extensive clean-up of docstrings, code style, and documentation Version 0.98.2 From 4e78149fddc2bd08fe3fbaf8edd7e9d5c80b856e Mon Sep 17 00:00:00 2001 From: Alessio Buccino <alejoe9187@gmail.com> Date: Thu, 2 Nov 2023 10:20:15 +0100 Subject: [PATCH 3/7] Add latest PRs --- doc/releases/0.99.0.rst | 18 ++++++++++++------ doc/whatisnew.rst | 1 + 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/doc/releases/0.99.0.rst b/doc/releases/0.99.0.rst index fb9df451e6..c4b6e5a025 100644 --- a/doc/releases/0.99.0.rst +++ b/doc/releases/0.99.0.rst @@ -11,6 +11,7 @@ Main changes: * Added DeepInterpolation training, transfer, and inference to preprocessing * Added new quality_metrics (syncrhrony, firing_range, amplitude_cv) * Added new template metrics (num positive/negative peaks + multi-channel metrics: velocities, spread, exp. decay) +* Important improvements to internal sorters (especially Spyking Circus 2) * Ported all widgets to "new" API and dismantled legacy widgets * Extensive clean-up of docstrings, code style, and documentation @@ -31,8 +32,8 @@ core: * Check that main_ids are str or int (#2061) * Change some default parameters for better user experience (#2071) * Fix NoiseGeneratorRecording : "noise_level" in kwargs (#2084) -* Add sparsity_temp_folder option to extract_waveforms (#2133: TBM) -* Re-add relative_to to dump_to_pickle (#2129: TBM) +* Add sparsity_temp_folder option to extract_waveforms (#2133) +* Re-add relative_to to dump_to_pickle (#2129) extractors: @@ -45,14 +46,14 @@ extractors: * CellExplorer: fix tests and deprecation (#2048) * Patch with scipy for remove_artefacts when mode is linear/cubic (#2072) * Fix grouping of OpenEphys NPIX extractors (#2102) -* Fix open ephys probe loading and unify probeinterface import syntax (#2136: TBM) +* Fix open ephys probe loading and unify probeinterface import syntax (#2136) preprocessing: * Deepinterpolation revived: training, transfer, and inference with more flexibility (#1804) * Patch silence_periods for mono-segment recordings (#1946) -* Fix zero_channel_pad case when selected frames are outside of original data region (#1979: TBM) +* Fix zero_channel_pad case when selected frames are outside of original data region (#1979) * Improve detect_bad_channels defaults and fixes (#2003) * Add an option to flip the order by depth (#2005) * Update a test expression to remove two logical short circuits in remove_artifacts (#2025) @@ -69,6 +70,7 @@ sorters: * Refactor sorter launcher. Deprecated run_sorters() and add run_sorter_jobs() (#1968) * Fixes Apptainer bug (#2053) * Use versioned kilosort options as default in skip_preprocessed_data (#2108) +* Mountainsort5 version check fix (#2150) postprocessing: @@ -78,6 +80,7 @@ postprocessing: * Refactor ISI calculation numpy and numba (#1958) * Extend and refactor waveform metrics, including multi-channel metrics (#1993) * Use default cutouts for peak-sign test (#2121) +* Remove postprocessing template_tools (#2145) qualitymetrics: @@ -102,6 +105,7 @@ exporters: * Fix bug in export_to_phy with sparsity (#1904) * Allow relative path for export_to_phy (#2041) +* Change phy exporter to not save template_ind in the case of dense waveform_extractor (#2148) widgets: @@ -109,6 +113,7 @@ widgets: * For connoisseur only: add a simple "ephyviewer" backend plot_traces() (#2024) * Improve ipywidgets backend (#2035, #2079, #2091) * Improved GTStudy and related widgets (#2128) +* Add assert error in the case of unichannel data for sortingview backend + minor clarifications (#2149) sortingcomponents: @@ -122,7 +127,8 @@ docs: * Add "how to" section on how to load matlab data (#2018, #2047) * Add link to latest pypi version docs in readme (#2060) * Improve assert messages (#2078, #2109, #2119) -* Unify 'defaults' and use of quotations across docsrtings (#2134: TBM) +* Unify 'defaults' and use of quotations across docstrings (#2134) +* Add Docstring section to development docs (#2138) ci / packaging / tests: @@ -134,4 +140,4 @@ ci / packaging / tests: * Add SPIKEINTERFACE_DEV_PATH to aws gpu tests (#2088) * Fix git-annex installation (#2097) * Add actual test for check_your_install (#2112) -* Add quality_metrics specific module requirements to pyproject.toml (#2135: TBM) +* Add quality_metrics specific module requirements to pyproject.toml (#2135) diff --git a/doc/whatisnew.rst b/doc/whatisnew.rst index f7466c1129..ee52a713f0 100644 --- a/doc/whatisnew.rst +++ b/doc/whatisnew.rst @@ -39,6 +39,7 @@ Version 0.99.0 * Added DeepInterpolation training, transfer, and inference to preprocessing * Added new quality_metrics (syncrhrony, firing_range, amplitude_cv) * Added new template metrics (num positive/negative peaks + multi-channel metrics: velocities, spread, exp. decay) +* Important improvements to internal sorters (especially Spyking Circus 2) * Ported all widgets to "new" API and dismantled legacy widgets * Extensive clean-up of docstrings, code style, and documentation From f7083b9ed15d8eed23cc9c550b9373e236fb5d5f Mon Sep 17 00:00:00 2001 From: Alessio Buccino <alejoe9187@gmail.com> Date: Thu, 2 Nov 2023 12:12:05 +0100 Subject: [PATCH 4/7] Update doc/releases/0.99.0.rst Co-authored-by: Zach McKenzie <92116279+zm711@users.noreply.github.com> --- doc/releases/0.99.0.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/releases/0.99.0.rst b/doc/releases/0.99.0.rst index c4b6e5a025..a533f13a82 100644 --- a/doc/releases/0.99.0.rst +++ b/doc/releases/0.99.0.rst @@ -9,7 +9,7 @@ Main changes: * Refactored :code:`core.generate` module to create high-quality lazy ground-truth data * Added DeepInterpolation training, transfer, and inference to preprocessing -* Added new quality_metrics (syncrhrony, firing_range, amplitude_cv) +* Added new quality_metrics (synchrony, firing_range, amplitude_cv) * Added new template metrics (num positive/negative peaks + multi-channel metrics: velocities, spread, exp. decay) * Important improvements to internal sorters (especially Spyking Circus 2) * Ported all widgets to "new" API and dismantled legacy widgets From 128de556e2267b704fe381c7aca75f35914674a6 Mon Sep 17 00:00:00 2001 From: Alessio Buccino <alejoe9187@gmail.com> Date: Thu, 2 Nov 2023 12:12:13 +0100 Subject: [PATCH 5/7] Update doc/whatisnew.rst Co-authored-by: Zach McKenzie <92116279+zm711@users.noreply.github.com> --- doc/whatisnew.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/whatisnew.rst b/doc/whatisnew.rst index ee52a713f0..33735a47fd 100644 --- a/doc/whatisnew.rst +++ b/doc/whatisnew.rst @@ -37,7 +37,7 @@ Version 0.99.0 * Refactored :code:`core.generate` module to create high-quality lazy ground-truth data * Added DeepInterpolation training, transfer, and inference to preprocessing -* Added new quality_metrics (syncrhrony, firing_range, amplitude_cv) +* Added new quality_metrics (synchrony, firing_range, amplitude_cv) * Added new template metrics (num positive/negative peaks + multi-channel metrics: velocities, spread, exp. decay) * Important improvements to internal sorters (especially Spyking Circus 2) * Ported all widgets to "new" API and dismantled legacy widgets From 148de486e6eca56de1d208f00e6694815f5ea885 Mon Sep 17 00:00:00 2001 From: Alessio Buccino <alejoe9187@gmail.com> Date: Thu, 2 Nov 2023 18:24:04 +0100 Subject: [PATCH 6/7] Update probeinterface --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index b9d76d2b15..fd73a5b4f6 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -25,7 +25,7 @@ dependencies = [ "joblib", "threadpoolctl", "tqdm", - "probeinterface>=0.2.18", + "probeinterface>=0.2.19", ] [build-system] From daf226af952923c963bbf4ae10b008f9dcd1102b Mon Sep 17 00:00:00 2001 From: Alessio Buccino <alejoe9187@gmail.com> Date: Fri, 3 Nov 2023 10:10:06 +0100 Subject: [PATCH 7/7] Update doc/releases/0.99.0.rst --- doc/releases/0.99.0.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/releases/0.99.0.rst b/doc/releases/0.99.0.rst index a533f13a82..44460cb181 100644 --- a/doc/releases/0.99.0.rst +++ b/doc/releases/0.99.0.rst @@ -3,7 +3,7 @@ SpikeInterface 0.99.0 release notes ----------------------------------- -31th October 2023 +3rd November 2023 Main changes: