move to pyproject.toml (#500)

* move to pyproject.toml * add pre-commit * fix CI * Update README.md * limit pre-commit to python, yaml, json & toml * add more pre-commit hooks * Update installation.rst * update versioning scheme * Update conf.py * Update developer_guide.rst * move some paragraphs from README.MD to sphinx docu * Update pyproject.toml * Update developer_guide.rst * Update developer_guide.rst * Update pyproject.toml * Update __init__.py * Update main.yaml * update suggested git-command for updating * add upgrade_iblrig command * some minor fixes * Update version_management.py * rescue lost changes from #502 * when the number of expected devices is undefined, do not raise but warn and exit * Copy (#503) * WIP recover from partial copy multiple devices * copier hierarchy * Update transfer_experiments.py * Update transfer_experiments.py * modify acquisition description aggregation level from copiers * add the script to copy video data / purge sessions older than 2 weeks * Fix chained protocols bug * bugfix: the criteria to end the task should not factor in the delay period --------- Co-authored-by: Florian Rau <[email protected]> --------- Co-authored-by: olivier <[email protected]>
int-brain-lab · Sep 15, 2023 · 1a70ce4 · 1a70ce4
1 parent 2473672
commit 1a70ce4
Show file tree

Hide file tree

Showing 31 changed files with 357 additions and 218 deletions.
diff --git a/.flake8 b/.flake8
@@ -5,4 +5,4 @@ exclude =
     .git,
     scratch_*.py,
     Bonsai,
-    venv*
+    venv*
diff --git a/.github/workflows/documentation.yaml b/.github/workflows/documentation.yaml
@@ -22,4 +22,4 @@ jobs:
           publish_branch: gh-pages
           github_token: ${{ secrets.GITHUB_TOKEN }}
           publish_dir: docs/build/html
-          force_orphan: true
+          force_orphan: true
diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml
@@ -17,13 +17,15 @@ jobs:
     steps:
       - name: Checkout iblrig repo
         uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
 
       - name: Setup Python
         uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.python-version }}
           cache: 'pip'
-          cache-dependency-path: '**/requirements*.txt'
+          cache-dependency-path: 'pyproject.toml'
 
       - name: flake8
         run: |
@@ -34,8 +36,7 @@ jobs:
       - name: iblrig and iblpybpod requirements
         shell: bash -l {0}
         run: |
-          pip install --editable iblrig
-          pip install -r requirements.txt
+          pip install --editable .
 
       - name: Install audio library (Ubuntu only)
         if: matrix.os == 'ubuntu-latest'

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -0,0 +1,31 @@
+repos:
+
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.4.0
+    hooks:
+    -   id: end-of-file-fixer
+        types_or: [python, yaml, json, toml]
+    -   id: trailing-whitespace
+        types_or: [python, yaml, json, toml]
+    -   id: check-case-conflict
+    -   id: check-docstring-first
+    -   id: check-ast
+    -   id: check-yaml
+    -   id: check-json
+    -   id: check-toml
+    -   id: check-xml
+    -   id: check-vcs-permalinks
+    -   id: detect-private-key
+
+-   repo: https://github.com/pre-commit/pygrep-hooks
+    rev: v1.10.0
+    hooks:
+    -   id: python-check-blanket-type-ignore
+    -   id: python-use-type-annotations
+    -   id: python-no-log-warn
+    -   id: text-unicode-replacement-char
+
+-   repo: https://github.com/pycqa/flake8
+    rev: 6.1.0
+    hooks:
+    -   id: flake8
diff --git a/LICENSE b/LICENSE
@@ -18,4 +18,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+SOFTWARE.
diff --git a/README.md b/README.md
@@ -2,31 +2,8 @@
 
 Welcome to the International Brain Laboratory decision making task implementation.
 
-The task is implemented in the Bpod system (Sanworks) and uses the Bonsai visual programming language (on Windows) for the visual stimuli and Python for all other purposes. 
+The task is implemented in the Bpod system (Sanworks) and uses the Bonsai visual programming language (on Windows) for the visual stimuli and Python for all other purposes.
 
 
 ## Documentation and installation steps here:
 https://int-brain-lab.github.io/iblrig
-
-
-
-### Run tests locally
-```shell
-flake8
-python -m unittest discover ./iblrig/test
-```
-
-### Build the documentation
-```shell
-# make sure pre-requisites are installed
-pip install -r requirements-dev.txt   
-# create the static directory
-rm -rf ./docs/build
-mkdir -p ./docs/build/html/_static  
-# unit tests generate task diagrams
-python -m unittest discover ./iblrig/test  
-# generate class diagrams
-pyreverse -o png -m y --ignore iblrig.test -A --output-directory ./docs/build/html/_static ./iblrig_tasks 
-# build and serve the docs locally
-sphinx-autobuild ./docs/source ./docs/build/html/  
-```
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -1,15 +1,11 @@
-# Configuration file for the Sphinx documentation builder.
-#
-# For the full list of built-in configuration values, see the documentation:
-# https://www.sphinx-doc.org/en/master/usage/configuration.html
-
-# -- Project information -----------------------------------------------------
-# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+from importlib.metadata import version
+from datetime import date
 
 project = 'iblrig'
-copyright = '2018, International Brain Laboratory'
+copyright = f'2018 – {date.today().year} International Brain Laboratory'
 author = 'International Brain Laboratory'
-release = '8.x'
+release = version('iblrig')
+version = '.'.join(release.split('.')[:3])
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

diff --git a/docs/source/developer_guide.rst b/docs/source/developer_guide.rst
@@ -1,3 +1,64 @@
+===============
+Developer Guide
+===============
+
+Versioning Scheme
+=================
+
+IBLRIG v8 uses `Semantic Versioning 2.0.0 <https://semver.org/spec/v2.0.0.html>`_.
+Its version string (currently "|version|") is a combination of three fields, separated by dots:
+
+.. centered:: ``MAJOR`` . ``MINOR`` . ``PATCH``
+
+* The ``MAJOR`` field is only incremented for breaking changes, i.e., changes that are not backward compatible with previous changes.
+  Releases of IBLRIG v8, for instance, are generally incompatible with IBLRIG v7.
+* The ``MINOR`` field will be incremented upon adding new, backwards compatible features.
+* The ``PATCH`` field will be incremented with each new, backwards compatible bugfix release that does not implement a new feature.
+
+On the developer side, these 3 fields are manually controlled by adding the respective version string to a commit as a `git tag <https://git-scm.com/book/en/v2/Git-Basics-Tagging>`_, for instance:
+
+.. code-block:: console
+
+   git tag 8.8.4
+   git push origin --tags
+
+The version string displayed by IBLRIG *may* include additional fields, such as in "|version|.post3+dirty".
+Here,
+
+* ``.post3`` indicates the third unversioned commit after the latest versioned release, and
+* ``+dirty`` indicates the presence of uncommited changes in your local repository of IBLRIG.
+
+Both of these fields are inferred by `setuptools_scm <https://pypi.org/project/setuptools-scm/>`_ and do not require manual interaction from the developer.
+
+
+Running Tests Locally
+=====================
+
+.. code-block:: console
+
+   flake8
+   python -m unittest discover ./iblrig/test
+
+
+Building the documentation
+==========================
+
+.. code-block:: console
+
+   # make sure pre-requisites are installed
+   pip install --upgrade -e .[DEV]
+   # create the static directory
+   rm -rf ./docs/build
+   mkdir -p ./docs/build/html/_static
+   # unit tests generate task diagrams
+   python -m unittest discover ./iblrig/test
+   # generate class diagrams
+   pyreverse -o png -m y --ignore iblrig.test -A --output-directory ./docs/build/html/_static ./iblrig_tasks
+   # build and serve the docs locally
+   sphinx-autobuild ./docs/source ./docs/build/html/
+
+
+
 Guide to Creating Your Own Task
 ===============================
 

diff --git a/docs/source/installation.rst b/docs/source/installation.rst
@@ -3,9 +3,9 @@ Installation guide
 
 .. prerequisites::
 
-    *   Windows OS
-    *   git installation.
-    *   Recommended: Notepad++ or a decent text file editor
+   *  Windows OS
+   *  git installation.
+   *  Recommended: Notepad++ or a decent text file editor
 
 
 Install Python 3.10
@@ -15,78 +15,86 @@ Open an Administrator: Windows Powershell prompt and run the following:
 
 .. code-block:: powershell
 
-    Set-ExecutionPolicy RemoteSigned
+   Set-ExecutionPolicy RemoteSigned
 
 Then exit the admin window and open a new Windows Powershell prompt (no admin mode) and run the following:
 
 .. code-block:: powershell
 
-    New-Item -ItemType Directory -Force -Path C:\Temp
-    Invoke-WebRequest -Uri https://www.python.org/ftp/python/3.10.11/python-3.10.11-amd64.exe -OutFile C:\Temp\python-3.10.11-amd64.exe
-    Start-Process -NoNewWindow -Wait -FilePath C:\Temp\python-3.10.11-amd64.exe -ArgumentList "/passive", "InstallAllUsers=0", "Include_launcher=0", "Include_test=0"
+   New-Item -ItemType Directory -Force -Path C:\Temp
+   Invoke-WebRequest -Uri https://www.python.org/ftp/python/3.10.11/python-3.10.11-amd64.exe -OutFile C:\Temp\python-3.10.11-amd64.exe
+   Start-Process -NoNewWindow -Wait -FilePath C:\Temp\python-3.10.11-amd64.exe -ArgumentList "/passive", "InstallAllUsers=0", "Include_launcher=0", "Include_test=0"
 
 
 .. exercise:: You can check that everything worked by running the following command:
 
-    .. code-block:: powershell
+   .. code-block:: powershell
 
-        C:\Users\IBLuser\AppData\Local\Programs\Python\Python310\.\python.exe --version
+      C:\Users\IBLuser\AppData\Local\Programs\Python\Python310\.\python.exe --version
 
-    Should return `Python 3.10.11`
+   Should return `Python 3.10.11`
 
 .. warning:: Make sure you exit the Administrator Powershell prompt before going to the next steps.
 
 
 Install iblrigv8
 ----------------
 
-1.  From the Powershell command line, clone iblrig to the ‘iblrigv8’ directory, and switch to iblrigv8 branch
+1. From the Powershell command line, clone iblrig to the ‘iblrigv8’ directory, and switch to iblrigv8 branch
 
-    .. code-block:: powershell
+   .. code-block:: powershell
 
-        cd C:\
-        git clone https://github.com/int-brain-lab/iblrig.git iblrigv8
-        cd iblrigv8
-        git checkout iblrigv8
+      cd C:\
+      git clone https://github.com/int-brain-lab/iblrig.git iblrigv8
+      cd iblrigv8
+      git checkout iblrigv8
 
 
-2.  Install a new virtual environment and update pip (modify the <Username> value if needed)
+2. Install a new virtual environment and update pip (modify the <Username> value if needed)
 
-    .. code-block:: powershell
+   .. code-block:: powershell
 
-        C:\Users\IBLuser\AppData\Local\Programs\Python\Python310\.\python.exe -m venv C:\iblrigv8\venv
-        C:\iblrigv8\venv\scripts\python.exe -m pip install --upgrade pip wheel
+      C:\Users\IBLuser\AppData\Local\Programs\Python\Python310\.\python.exe -m venv C:\iblrigv8\venv
+      C:\iblrigv8\venv\scripts\python.exe -m pip install --upgrade pip wheel
 
 
-3.  Install iblrig in editable mode
+3. Install iblrig in editable mode
 
-    .. code-block:: powershell
+   .. code-block:: powershell
 
-        C:\iblrigv8\venv\scripts\Activate.ps1
-        cd C:\iblrigv8
-        pip install -e .
-        pip install -r requirements.txt
+      C:\iblrigv8\venv\scripts\Activate.ps1
+      cd C:\iblrigv8
+      pip install -e .
 
 
-4.  Install additional tasks and extractors for personal projects (optional)
+4. Install additional tasks and extractors for personal projects (optional)
 
-    .. code-block:: powershell
+   .. code-block:: powershell
 
-        cd C:\
-        git clone https://github.com/int-brain-lab/project_extraction.git
-        cd project_extraction
-        pip install -e .
+      cd C:\
+      git clone https://github.com/int-brain-lab/project_extraction.git
+      cd project_extraction
+      pip install -e .
 
 
-5.  Install Bonsai in portable mode
+5. Install Bonsai in portable mode
 
-    .. code-block:: powershell
+   .. code-block:: powershell
 
-        cd C:\iblrigv8\Bonsai
-        powershell.exe .\install.ps1
-        cd ..
+      cd C:\iblrigv8\Bonsai
+      powershell.exe .\install.ps1
+      cd ..
 
 
+Update iblrigv8
+---------------
+
+   .. code-block:: powershell
+
+      C:\iblrigv8\venv\scripts\Activate.ps1
+      cd C:\iblrigv8
+      pip install --upgrade -e .
+
 
 Configuration instructions
 --------------------------
@@ -99,16 +107,16 @@ Copy template settings files.
 
 .. code-block::
 
-    cd C:\iblrigv8\settings
-    cp hardware_settings_template.yaml hardware_settings.yaml
-    cp iblrig_settings_template.yaml iblrig_settings.yaml
-    explorer C:\iblrigv8\settings
+   cd C:\iblrigv8\settings
+   cp hardware_settings_template.yaml hardware_settings.yaml
+   cp iblrig_settings_template.yaml iblrig_settings.yaml
+   explorer C:\iblrigv8\settings
 
 
 Update the 2 settings files, these values can likely be found in the `C:\iblrig_params\.iblrig_params.json` file if working with a existing rig
 
-*   iblrig_settings.yaml
-*   hardware_settings.yaml
+*  iblrig_settings.yaml
+*  hardware_settings.yaml
 
 
 Setup ONE
@@ -122,26 +130,26 @@ See instructions for that here: https://int-brain-lab.github.io/iblenv/notebooks
 
 .. exercise:: Make sure you can connect to Alyx !
 
-    Open a Python shell in the environment and connect to Alyx (you may have to setup ONE)
+   Open a Python shell in the environment and connect to Alyx (you may have to setup ONE)
 
-    .. code-block::
+   .. code-block::
 
-        C:\iblrigv8\venv\scripts\Activate.ps1
-        ipython
+      C:\iblrigv8\venv\scripts\Activate.ps1
+      ipython
 
-    Then at the Ipython prompt
+   Then at the Ipython prompt
 
-    .. code-block:: python
+   .. code-block:: python
 
-        from one.api import ONE
-        one = ONE(username='your_username', password='your_password', base_url='https://alyx.internationalbrainlab.org')
+      from one.api import ONE
+      one = ONE(username='your_username', password='your_password', base_url='https://alyx.internationalbrainlab.org')
 
 
 .. exercise:: You can check that everything went fine by running the test suite:
 
-    .. code-block:: powershell
+   .. code-block:: powershell
 
-        cd C:\iblrigv8
-        python -m unittest discover
+      cd C:\iblrigv8
+      python -m unittest discover
 
-    The tests should pass to completion after around 40 seconds
+   The tests should pass to completion after around 40 seconds
-Original file line number
+Diff line change
@@ Expand Up / @@ -5,4 +5,4 @@ exclude = @@
         .git,
         scratch_*.py,
         Bonsai,
-        venv*
+        venv*