Update documentation

[pberkes]

No description provided.

[lschwetlick]

in the file index.rst it says:
'If you require additional information, or to report errors or questions please contact us: ' and then gives the contact of Felix and Heiko. Should this be changed?

[lschwetlick]

There is also an Acknowledgements section there. Should we acknowledge the members of this sprint?

[lschwetlick]

I wonder whether we should keep this style of always referencing the Matlab Version. Or whether it might be better to copy the text and adapt it rather than point out the differences. For a new user who is not migrating from matlab I imagine it would be kind of annoying to be confronted with this comparison exercise... This is particularly the case in all the user_guides.

My personal vote is to get rid of the references to the matlab version, and copy paste over all the relevant stuff. Thoughts?

Other ToDos that need some decisions before fixing:

• fix test_MATLAB_version.rst. How far are we with decisions about matlab tests?
• getThreshold and getSlope and getSlopePC as described in file docs/how_to/How-to-Get-Thresholds-and-Slopes.rst dont exist. Should we implement them or not document?
• the option to set the threshold does not exist as described in file docs/how_to/How-to-Change-the-Threshold-Percent-Correct.rst Should we implement them or not document?
• the user guides page is unlinked in the TOC on the left (https://psignifit.readthedocs.io/en/latest/user_guides/#user-guides). Is this intentional?

[otizonaizit]

I wonder whether we should keep this style of always referencing the Matlab Version. Or whether it might be better to copy the text and adapt it rather than point out the differences. For a new user who is not migrating from matlab I imagine it would be kind of annoying to be confronted with this comparison exercise... This is particularly the case in all the user_guides.
My personal vote is to get rid of the references to the matlab version, and copy paste over all the relevant stuff. Thoughts?

I agree that the references to the MATLAB version are confusing. We can refer to it right at the beginning once, linking to their docs. And if deemed necessary, we can write a page guiding the user in the transition from the MATLAB version to the Python version. But the default in my opinion should be to assume that the user is coming directly to our version.

Other ToDos that need some decisions before fixing:

* [ ]   fix `test_MATLAB_version.rst`. How far are we with decisions about matlab tests?

this may take still some time, I fear.

* [ ]  `getThreshold` and `getSlope` and `getSlopePC` as described in file `docs/how_to/How-to-Get-Thresholds-and-Slopes.rst` dont exist. Should we implement them or not document?

I thought that these are methods on the sigmoid objects, and should be already obtainable from the results object, see for example #137 ? Or am I misunderstanding something?

* [ ]  the option to set the threshold does not exist as described in file `docs/how_to/How-to-Change-the-Threshold-Percent-Correct.rst` Should we implement them or not document?

What would make sense for the user? Are these things useful? @guillermoaguilar ?

[guillermoaguilar]

I completely agree about not referencing MATLAB all the time, I think the documentation of the python package should stand for itself. One mention to the MATLAB version and documentation in the main page suffices.

Now about reading out thresholds.

• the option to set the threshold does not exist as described in file docs/how_to/How-to-Change-the-Threshold-Percent-Correct.rst Should we implement them or not document?

This should implemented. The definition of the proportion correct at which to read out a threshold is somehow arbitrary, most commonly half way between min and max. But I've also seen 3/4 between min and max. That's why it should be possible.
And the best way to do it is to parametrized the fit at a different PC, before the fitting. This also according to the MATLAB version. The reason for that are the CI, the most reliable CI are the ones you read from the parameter you fit, not extrapolated after the fitting.

I thought this was already implemented, by passing a configuration with a different value, like this psignifit(DATA, thresh_PC=0.75), but it complains and expects thresh_PC=0.5 always.

Plot should be like this one from the MATLAB version

• getThreshold and getSlope and getSlopePC as described in file docs/how_to/How-to-Get-Thresholds-and-Slopes.rst dont exist. Should we implement them or not document?

This is the alternative, and it is also needed, so that you can read out the x-values from an already fit psychometric function.

• getThreshold is now result.threshold() method and now broken, see Threshold() and slope_at_proportion_correct() broken #137
• getSlopePC is now result.slope_at_proportion_correct() and is also broken, see same issue Threshold() and slope_at_proportion_correct() broken #137
• getSlope is now result.slope() and it return similar values as in MATLAB, so all good for this one

[guillermoaguilar]

• the user guides page is unlinked in the TOC on the left (https://psignifit.readthedocs.io/en/latest/user_guides/#user-guides). Is this intentional?

This was a mistake from my side when editing the TOC, :/ They should be reinstated.

Also we should check that information is not redundant, for example there is an user guide for plotting functions, but there's also a demo in Examples with similar information.

[lschwetlick]

I need help with an executive decision. The docs are done in .rst files right now. As far as I can tell, there is no code execution upon compilation at all. It turns out we have a fair number of examples in the docs and it would be nice if the plots and code outputs were generated when the docs are compiled as opposed to having to manually copy-paste them in.

Option 1 is keeping that.

Option 2 is using markdown. The docs of Stimupy (https://stimupy.readthedocs.io/en/latest/index.html) use this method. Here the code blocks are just executed when the docs are compiled

Option 3 is using the same style as our demo.py files. The actual text is a bit hard to read and write but the code is at least runnable

Option 4 is using notebooks. To be honest for me this makes sense because it gives me a more or less WYSIWYG editor to make the plots and examples as well as for the typesetting of the text. There are extensions to compile notebooks in sphinx. I guess the only downside is that then the file is not very human-readable anymore....

[guillermoaguilar]

I strongly prefer Option 2, to switch to that format. This is done using jupyter book, which is very nice.

As you say these are executed on build, so no need to run stuff and save as images. Pages can be either pure markdown files with python codeblocks, python scripts .py with markdown interleaved, or jupyter notebooks. Or a mixture of them. So you wouldn't have to convert from notebook at all (although it is a mess for git tracking).

I can help with setting up, I've done it before.
But we have to find a way to convert the current rst to that format. I'm not sure what is the best way for that... probably manually?

[guillermoaguilar]

and by the way jupyter book supports notebooks because under the hood it uses jupytext.
This is a notebook extension that allows you to "pair" a notebooks with a .py file, in plain text format. All text is saved as comments in a certain format. So in this way you can track the .py file in git.

[otizonaizit]

I need help with an executive decision. The docs are done in .rst files right now. As far as I can tell, there is no code execution upon compilation at all. It turns out we have a fair number of examples in the docs and it would be nice if the plots and code outputs were generated when the docs are compiled as opposed to having to manually copy-paste them in.

I don't have a strong opinion. I have to comments for this discussion:

1. I find notebooks annoying as a developer, because it is difficult on GitHub to see what changed and it makes it very awkward to have a conversation about a contribution done in a notebook. For example it would have been much nicer if I could have written this comment Demo notebook for using psignifit in a log-space #149 (comment) directly in the code review interface of GitHub, instead of copying and pasting snippets of code from a rendered notebook. And if @pberkes acts on my suggestion, I have no way on GitHub to see what he changed and how. I know that locally on my laptop I can install stuff so that I can do diffs of notebooks and stuff, but the public record of the discussion still happens on GitHub, so those tools don't help for this problem. All in all from a developer/contributor perspective, I'd prefer to have a doc format in pure text, rst or md I personally don't care which.

2. Executing the code on docs-build is good, but I think it would be important to have the code tested, i.e. the docs-build should fail if a code snippet generates outputs that don't match the expected ones. Does the StimuPy set up support doc-testing?

[lschwetlick]

I actually agree 100% with the downsides of Jupyter notebooks. But I do also find it really annoying to write a thing like the md file and then I need to go and compile the entire docs (which probably involves installing 200 libraries, half of which conflict with something or other) in order to see if I am generating the right plot or output.

But I guess its the pill I have to swallow... Md then?

[otizonaizit]

But I guess its the pill I have to swallow... Md then?

yes!

[lschwetlick]

Hey, so we just did a bunch of tiny and wrong commits on main, trying to get the readthedocs configuration to work correctly. Now we have those commits on our history. Can we live with it or should we squash them?

[otizonaizit]

Hey, so we just did a bunch of tiny and wrong commits on main, trying to get the readthedocs configuration to work correctly. Now we have those commits on our history. Can we live with it or should we squash them?

well, now that the PR is merged on main rebase is not an option anymore, so we'll live with it.

[lschwetlick]

The commits i meant were not in the PR to begin with. We decided to do it directly on main because readthedocs is connected to main and there are so many tiny details with this stuff we didnt want to do a lot of effort to set it up on another branch only to have it fail again once we switch back.

[otizonaizit]

We decided to do it directly on main

then it's even more so that rebase is not an option. I think it's not such a big problem and it shouldn't happen often, it was just needed now to do the initial setup, right?

[lschwetlick]

Yup, we wanted the setup done so now we can have separate pull requests and fix the docs as we go along, instead of having wrong docs up for so long.

[guillermoaguilar]

Yep sorry about the mess.

But now Docs look much better now :) take a look

https://psignifit.readthedocs.io/en/latest/

All files are plain text markdown and executed on build (in docs/examples)

There's still tons of things to correct, formatting and content, but we're gonna coordinate this tomorrow, together with Matilda, one of Felix's students.

[otizonaizit]

How do we make comments about this? For example, I find the way of specifying options as

options = {}
options['foo'] = 'bar'

very verbose and not so pythonic... if there are two or three options only I'd do it directly in the function call, as in

psignifit(data, foo='bar')

like in some of the docs already. If there are more options, then creating the dictionary as a literal seems nicer to me:

options = {
    'foo' : 'bar',
    'baz' : 'booz',
    ....
    }

but even here I am not sure it is not better to just put the whole thing in the function call...

what do you think?

[lschwetlick]

I agree that i think the dictionary is not so nice. In most cases there should be few enough options that putting it in the function call isn't too bad, right? especially once we remove all the deprecated/dangerous options like stimulus range...

[pberkes]

Whoa such slick docs!

[guillermoaguilar]

Just a quick update: couple of weeks ago Lisa and I met Matilda, a student from Felix's lab who is going to help with the documentation. She's working on it at the moment and we're supervising. I hope in the next weeks we can have everything done.

[guillermoaguilar]

After lots of rearrangements PR #168 corrects most of the documentation.

Still to do: @lschwetlick @Matilda1206

• Priors demo:
  • change marker size, they are huge
  • check text against MATLAB wiki
  • check that it works, check that one can pass custom priors
• Pooling utility
  • migrate to md
  • migrate text from MATLAB wiki
  • test that works
• Bias analysis
  • check text against MATLAB wiki
  • check that fixing bug corrects the look of the plot plot in bias analysis to short #176
• Advanced options. Some dont work
  • 'width_alpha'
  • MAP vs mean
  • 'thresh_PC' to something different than 0.5
• How-to-Get-Thresholds-and-Slopes - broken, issue threshold method broken #172
• How-to-Get-Standard-Parameters need to be documented Get Standard Parameters #160 Implement a Sigmoid.standard_parameters method #169