Some thoughts on documentation structure

[JoeZiminski]

Hey everyone, I have a couple of small suggestions on the docs gleaned from some feedback from researchers using spikeinterface in the SWC and myself having taken a break from working with SI.

At present can be a little hard to know where to look in the docs when just getting started with spikeinterface. What do people think of the below suggestions to help orient new users to the correct place?

• Make a 'Getting Started' section on the left hand panel. In this can go the 'Installation', 'Importing SpikeInterface' and 'How to guides > How to "get started"' (and maybe 'Installing Spike Sorters').

I think this alone would be sufficient, but have some other suggestions:

• Rename 'Viewers' to 'Visualising Data' to be clearer what this is.
• Rename 'Modules example gallery' to 'Example Gallery' (it is not immediately clear what 'modules' is referring to and I only figured it out based on existing knowledge of python modules concept which many users may not be familiar with').

The 'Modules documentation' is very useful, but quite confusing for new users. These are structured more like developer docs - it is very useful to have a 1:1 mapping between code structure and documentation for developers. But for users things should be categorised according to outcome (e.g. preprocessing data, running quality metrics), rather than based on code structure. Maybe an easy solution is to rename this to indicate it is more developer-docs than outcome-based (e.g. 'Reference manual' or something) and move it a few rungs down the left-hand panel. Then it will be clearer to users to check out the 'Get started' / 'How to' / 'Examples' for outcome-based guides.

[chrishalcrow]

Hey Joe,

Lots of overlap with #2327, where there was a similar sentiment (including by past-you!) Really like the idea of a Get Started section!

I think we just need to agree a new toctree structure. Reading the old issue and this one, my proposal would be:

• Overview
• Get Started
  • Install
  • Importing si
  • How to get started
  • Install spike sorters
• Tutorials
• How to Guides
• In-depth manual* (the current Modules documentation)
• API
• Contribute
• Release notes
• Contact us

[EDIT: To tidy it up, we could add overview to the getting started bit. Or combine the overview with the homepage]

*Not sure about this name. 'References Manual' sounds like the API to me. Any other ideas, anyone??

Please someone disagree with me, or improve on my proposal!

[JoeZiminski]

Thanks @chrishalcrow! I forgot about that thread 😆 I will actually close this issue once we have finished this discussion and then repost on the other thread. Really like the toctree structure and renaming of 'References Manual' (I didn't like that either). Maybe the current 'Veiwers' could go in a 'How to visualise data'?

My only thought is not to merge How to / Examples as (my conceptualisation at least) is that 'How to' is shorter guides for specific outcomes whereas 'Examples' may be longer-form code examples (I am not sure however if this is reflected in the reality of the current how to / examples, will give the docs a proper read again ASAP!). But a 'How to' might be 'how to preprocess your data' whereas an 'Example' could be 'A complete pipeline for neuropixels data'. WDYT?

[JoeZiminski]

Based on this and the other thread, I think the key themes are splitting between long-form 'tutorials' and short form 'how to'. Currently these are mixed up between the existing 'How to guides' and 'Module example gallery'. Here is how I would categorise all below. At the moment the organisation refelcts how they are generated (sphinx gallery vs not).

How to guides

How to “get started” : TUTORIAL (Getting started)
Analyse Neuropixels datasets : TUTORIAL
Handle motion/drift with spikeinterface HOW TO
Exporting MATLAB Data to Binary & Loading in SpikeInterface HOW TO
How to Combine Recordings in SpikeInterface HOW TO
Processing a Recording by Channel Group HOW TO
From WaveformExtractor to SortingAnalyzer NEITHER?

Modules example gallery

Recording Objects TUTORIAL
Sorting Objects TUTORIAL
Handling Probe Information HOW TO
SortingAnaylzer TUTORIAL
Append and/or concatenate segments HOW TO
Handle Time Information HOW TO
Read various format into SpikeInterface] HOW TO (?) (I think this might duplicate other docs and could be deleted?)
Working with unscaled traces HOW TO
Curation Tutorial TUTORIAL (but could be a bit longer)
Explore sorters weaknesses with ground-truth comparison HOW TO

There are 4x 'Widgets' tutorials. These could be one 'visualising data' tutorial?

[zm711]

I'm fine bouncing around a bit. Whatever it takes to move this forward as long as things are linked. Maybe this could also be at least a small hackathon discussion. The only issue with modules gallery is that they are autogenerated by sphinx stuff and I'm too lazy to figure out how to move them (other than move all at once). But if someone else knows how to generate the gallery and then move individual files it would be good. Otherwise I like @chrishalcrow version of the toc tree. Maybe we have some slight movement around the edges, but as far as a move in the right direction I think it would be great.

[chrishalcrow]

Yup, sounds good @JoeZiminski
And I'm happy to figure this stuff out @zm711 !

So, we'd separate into Tutorials, then How Tos? I've updated my proposed toctree above.
From WaveformExtractor to SortingAnalyzer is a bit special. We can count it as a tutorial for now, and eventually get rid of it??

Agree about the visualisation stuff.

Happy for you to close the discussion

[zm711]

I'm a fan of keeping it (we->sa). I know we plan to deprecate->remove the mockwaveform extractor, but I'm in favor of keeping it to always provide a layer of backward compatibility (so that someone from the past isn't locked into 101-103). But we will see. Also @chrishalcrow someone complimented the detail and helpfulness of that tutorial. Wanted to make sure you knew!

[JoeZiminski]

Thanks both, I am happy with @chrishalcrow's toctree and WA->SA as a tutorial.

It is possible to have multiple sphinx gallerys in a project. The benefits of the gallerys is that they are easier to test, and you don't need to worry about saving / maintaining programmatically created figures. The downside is you loose some cool features of sphinx (e.g. dropdowns, tabs etc).

It would make sense for tutorials to be sphinx gallerys. For how-tos there is a bit of a mix, for example Handle time information would work better as a normal page. But Handle Probe Information would work better as a sphinx gallery. I am not sure if it is possible to include normal sphinx pages within a sphinx gallery.

[h-mayorquin]

• Overview

• Get Started

  • Install
  • Importing si
  • How to get started
  • Install spike sorters

• Tutorials

• How to Guides

• In-depth manual* (the current Modules documentation)

• API

• Contribute

• Release notes

• Contact us

@chrishalcrow suggestion looks great to me.

My two cents:

• Tutorials vs how-to, some heuriistics: tutorial is mean to teach you how to use the library a how-to is how to solve a specific problem. If you are trying to teach, it is a tutorial, if it you are intending for this to be copy-pasted then is a how-to. If it is too long put in the tutorial.
• Let's just pick this one and move forward with it. We can improve later but right now things are very messy. Most of the proposed designs here and in the previous thread will be an improvement.

[chrishalcrow]

Ok, great! I'll start a pull request soon to reorganise but won't change any content/names yet. And will try to figure out how to elegantly do the sphinx stuff under the hood, without further complexifying things.

[JoeZiminski]

Thanks a very nice way to conceptualise tutorials vs. howto @h-mayorquin, I think important as not obvious at first. This could even be mentioned in the docs overview.

As a plan going forward I would propose something like:

1. Finalise Reorganising documentation into Getting Started, Tutorial and How To #2778 for general reorganisation
2. Unpin from sphinx==5.1.1
3. Reorder the how to / tutorials, seeing what is best approach (e.g. two sphinx gallerys, or sphinx-gallery only for tutorial)

[JoeZiminski]

A general question, is the 'latest' docs the ones built from main? I always thought that these were the latest release. To me it seems more intuitive to have 'latest' point to the documentation for the latest released version (100.6) and the docs built from main to be 'Development Branch' or something.

I would guess the main-branch docs are going to be relevant for relatively few people and there is an underlying assumption the docs are targetted towards users rather than developers (i.e. 'latest' is latest from a user perspective not a developer). What do people think?

[h-mayorquin]

Yeah, we should probably point to stable.

[zm711]

I'm going to close this for now and we can continue these discussions on #2800!

As far as point to stable that seems right to me too.