Changes to 'Modules Documentation' section of documentation

[JoeZiminski]

This is a repost from a discussion originally on #2778 on possible reorganisation of the 'Modules Documentation' section. This issue is a partner to #2327 which tackles organising what is a 'How To' vs. 'Tutorial' and how it is presented with sphinx gallery.

I agree the distinction between In-Depth and other documentation sections might not be clear. I think 'Module Documentation' is okay as a name but it relies on the reader to know what a 'Module' is.

I've had a look through and quick attempt and catalogue these sections (below). The information in there is super-valuable but contains a mix of different categories of documentation. I think this is in part why it is difficult to name and navigate as a user. Because of this I think it matters less what it's called now and we could have a follow-up plan to reorganise it soon. But happy to keep as 'Module Documentation' or maybe 'API Explanation' (mirroring diataxis explanation section) or 'Detailed Reference'.

Based on the below I think this section contains a mix of various documentation categories:

• General API details / explanations. These are things like the idea of the preprocessing chain concept, Quality metrics module, sorting components philosophy and API overview. I think all of this could be kept here in a 'API Explanation' / 'More Details' / 'Advanced Documentation' / 'User Guide' section. Its all extremely useful and doesn't fall into another existing category.
• Targeted API details / explanations. These are related to specific public-facing functions, in particular in the preprocessing section. These are close to the API docs, just with a little more info, and possibly risk duplicating the API section. For these I might suggest moving them to the actual function docstring, similar to a Numpy/Scipy-style approach where the public-facing docstrings contain a lot of detail / explanation. Then everything is in one place and linked closely to the code.
• There are some things that could be moved / merged with existing how-to guides and tutorials

General overview:

Core module

I would suggest the below subsections are API reference / developer docs that cover how the API is structured but aren’t linked to any specific functionality or goal. These could be kind of deep API reference / explanation (mostly for developers):

• Recording
• Sorting
• Event
• Snippets
• Recording tools
• Template tools
• LEGACY objects

These subsections are like 'How To's and could be moved / merged into the 'How To section'

• Handling Probes
• Saving, loading and compression
• Parallel processing and job kwargs
• Manipulating objects: slicing, aggregating
• Generate Toy objects
• Downloading test datasets

But Sparsity and maybe 'Saving, loading and compression' I'm not sure and are kind of general explanation and useful background.

Extractors module

The below sections could go into / be merged with an existing 'Loading data' tutorial or how-to:

• read one recording
• read one sorting
• read one event
• lazy loading

There is an important list of all supported data formats.

Preprocessing module

Contains a valuable discussion on the chain concept and dtype. This is kind of API explanation.

Then there is an Available preprocessing which has a more detail on preprocessing functions and kind of mirrors the API docs. Maybe these could be merged into the docstring of an these functions similar to Numpy.

There is a "How to implement “IBL destriping” or “SpikeGLX CatGT” in SpikeInterface¶" that could go to 'How To'.

Sorters module

It is useful to have a single 'sorting' page that links together everything related to sorting. But I think this could broken up into 3/4 separate articles:

• a general 'Sorting' tutorial introducing the 'wrapper' concept, how to run sorting generally.
• Details on running sorting in container. either a 'How To' for a kind of explanation article.
• Running several sorters in parallel / spike sorting by group / handling multi-segment recordings /installed sorters: merged into tutorials or standalone how-to
• Supported Spike Sorters (similar to the supported extractors section)
• Contributing guidelines (moved to Contributing)

Postprocessing module

This could remain as a API explanation article, or be made a tutorial.

I would suggest the sections that add more detail to the functions are moved into docstring and API docs beefed up here (available extensions)

Quality Metrics module

This is a very nice detailed deep reference for all the quality metrics.

Comparison Module,

This standalone reference article or tutorial, or maybe the most tutorial-parts taken out for a tutorial and rest left as an information article.

Exporters Module

These could be How To:

• Exporting to Phy¶
• Export a spike sorting report¶

Widgets module

This is kind of a standalone API reference on the available widget backends.

Curation module

A nice standalone overview of the curation options in SI.

Sorting Components module

A nice introduction and overview on the Sorting Components concept.

Motion/direct correction

Again nice standalone article on motion correction options.

[h-mayorquin]

There is a "How to implement “IBL destriping” or “SpikeGLX CatGT” in SpikeInterface¶" that could go to 'How To'.

Totally agree with this.

In general, I think is self-limiting to organizing the documentation around the module structure of the package. I believe we should aim ot structure the documentation around the problems that the library solves and the objects that we use them to do it. A limitation is that we might want to move things and re-organizat the pakcage and we don't want to have to re-organizat the documentation when this happens. I think this is the tendency of programmers to try to give explanations in terms of the implementation (because that is what we like!).

[JoeZiminski]

@zm711 I will add a note hear about: we should create a glossary of terms. This is a) useful for new users b) acts as a style guide for our docstrings etc. I added here as general docs discussion but ofc happy for it to be standalone issue.

[JoeZiminski]

A related issue: #1808 on NWB how-to. Another issue on deepinterpolation how-to: #1785

[JoeZiminski]

One case to discuss for the glossary is frame vs. sample, as discussed here.