Statistical Model for SDI - first level and second level

[venkateshness]

Closes #66

This PR contains the necessary code to perform statistical tests when there is events that require the consistency check. Tests are conducted using a 2-level-statistical model, while making use of the surrogate SDIs computed from nigsp.surrogates module.

Proposed Changes

End-to-End pipeline for a 2-level stats model;

• Non-Parametric Wilcoxon signed rank test
• Parametric tests testing for consistency across events
• Massive Univariate tests testing for consistency across subjects + Permutation-based multiple comparison correction

Change Type

• bugfix (+0.0.1)
• minor (+0.1.0)
• major (+1.0.0)
• refactoring (no version update)
• test (no version update)
• infrastructure (no version update)
• documentation (no version update)
• other

Checklist before review

• I added everything I wanted to add to this PR.
• [Code or tests only] I wrote/updated the necessary docstrings.
• [Code or tests only] I ran and passed tests locally.
• [Documentation only] I built the docs locally.
• My contribution is harmonious with the rest of the code: I'm not introducing repetitions.
• My code respects the adopted style, especially linting conventions.
• The title of this PR is explanatory on its own, enough to be understood as part of a changelog.
• I added or indicated the right labels.

• I added information regarding the timeline of completion for this PR.

• Please, comment on my PR while it's a draft and give me feedback on the development!

[venkateshness]

Hi @smoia !
I have the script ready but gonna need a bit of support infrastructure-wise. For now, I created a statistics.py
thinking that we are making a stats module (test_significance func will go there as well). Where should it be updated if we're to keep it as a separate module ?

I see a) ../operations/__init__.py b) ../objects.py ? of course the tests too. Could you advise me on that please ?

[smoia]

Hey @venkateshness, I need to check as well. For sure operations/__init__.py and __init__.py need an update. I'm not sure objects.py would (not sure though), but fairly sure workflow.py needs to be updated. All related tests files require updating too. And maybe something else, but failing tests will tell you ;)

[codecov]

Codecov Report

Merging #73 (6fcd157) into master (059532f) will increase coverage by 0.32%.
Report is 18 commits behind head on master.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #73      +/-   ##
==========================================
+ Coverage   90.61%   90.93%   +0.32%     
==========================================
  Files          14       15       +1     
  Lines        1236     1280      +44     
==========================================
+ Hits         1120     1164      +44     
  Misses        116      116              

Files Changed	Coverage Δ
nigsp/operations/statistics.py	100.00% <100.00%> (ø)

[venkateshness]

Everything looking good !

How to properly update the optional requirements😅? I added mne in the setup.cfg and the build was okay and everything working fine locally. But the RTD build fails here. It looks like it clones from the NiGSP's main git which obviously doesn't have mne. What am I missing 🧐

EDIT: All clear ! I just forced Sphinx to install packages for stats module through .readthedocs.yml and boom !

[smoia]

OHIE, EGGCITING!

I'll have a look at this later today/this week!

[smoia]

@venkateshness this is a lot of work and an awesome start!

I'm adding a few comments mainly about form and interaction with the rest of the library. I did not check the code thoroughly yet, so I can't comment on function!

Main points:

• move MNE import from top level to two_level_statistical_model
• report correctly original developers, copyrights, and licences
• adjust a few other things in the numpydocs in terms of style

See the comments in line - great job so far!

[smoia]

@tylerjereddy, I will send an email to the official scipy email too, but just to check that we're respecting the BDS-3 claused licence of scipy.

@larsoner, same for MNE-python!

[larsoner]

Both SciPy and MNE-Python seem okay to me

[venkateshness]

Ciaoooo @smoia !
We're looking into if there's a simpler and meaningful way to simplify the stats, the current procedure is a bit complex. I'll let you know when it is ready for review !