Skip to content

Commit

Permalink
Merge branch 'main' into documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
bclenet committed Nov 30, 2023
2 parents d212e1d + c03e9d1 commit 6466b29
Show file tree
Hide file tree
Showing 25 changed files with 849 additions and 162 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -139,3 +139,6 @@ dmypy.json
# Pyre type checker
.pyre/
*pyscript*

# For mac users
*.DS_Store
7 changes: 7 additions & 0 deletions CONTENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
### Contents overview

- :snake: :package: `narps_open/` contains the Python package with all the pipelines logic.
- :brain: `data/` contains data that is used by the pipelines, as well as the (intermediate or final) results data. Instructions to download data are available in [INSTALL.md](/INSTALL.md#data-download-instructions).
- :blue_book: `docs/` contains the documentation for the project. Start browsing it with the entry point [docs/README.md](/docs/README.md)
- :orange_book: `examples/` contains notebooks examples to launch of the reproduced pipelines.
- :microscope: `tests/` contains the tests of the narps_open package.
22 changes: 15 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,19 @@
# How to contribute to NARPS Open Pipelines ?

General guidelines can be found [here](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) in the GitHub documentation.
For the reproductions, we are especially looking for contributors with the following profiles:
- 👩‍🎤 SPM, FSL, AFNI or nistats has no secrets for you? You know this fMRI analysis software by heart 💓. Please help us by reproducing the corresponding NARPS pipelines. 👣 after step 1, follow the fMRI expert trail.
- 🧑‍🎤 You are a nipype guru? 👣 after step 1, follow the nipype expert trail.

## Reproduce a pipeline :keyboard:
:thinking: Not sure which one to start with ? You can have a look on [this table](https://github.com/Inria-Empenn/narps_open_pipelines/wiki/pipeline_status) giving the work progress status for each pipeline. This will help choosing the one that best suits you!
# Step 1: Choose a pipeline to reproduce :keyboard:
:thinking: Not sure which pipeline to start with ? 🚦The [pipeline dashboard](https://github.com/Inria-Empenn/narps_open_pipelines/wiki/pipeline_status) provides the progress status for each pipeline. You can pick any pipeline that is in red (not started).

Need more information ? You can have a look to the pipeline description [here](https://docs.google.com/spreadsheets/d/1FU_F6kdxOD4PRQDIHXGHS4zTi_jEVaUqY_Zwg0z6S64/edit?usp=sharing). Also feel free to use the `narps_open.utils.description` module of the project, as described [in the documentation](/docs/description.md).
Need more information to make a decision? The `narps_open.utils.description` module of the project, as described [in the documentation](/docs/description.md) provides easy access to all the info we have on each pipeline.

When you are ready, [start an issue](https://github.com/Inria-Empenn/narps_open_pipelines/issues/new/choose) and choose **Pipeline reproduction**!

### If you have experience with NiPype
# Step 2: Reproduction

## 🧑‍🎤 NiPype trail

We created templates with modifications to make and holes to fill to create a pipeline. You can find them in [`narps_open/pipelines/templates`](/narps_open/pipelines/templates).

Expand All @@ -21,9 +25,9 @@ Feel free to have a look to the following pipelines, these are examples :
| 2T6S | SPM | Yes | [/narps_open/pipelines/team_2T6S.py](/narps_open/pipelines/team_2T6S.py) |
| X19V | FSL | Yes | [/narps_open/pipelines/team_X19V.py](/narps_open/pipelines/team_2T6S.py) |

### If you have experience with the original software package but not with NiPype
## 👩‍🎤 fMRI software trail

A fantastic tool named [Giraffe](https://giraffe.tools/porcupine/TimVanMourik/GiraffePlayground/master) is available. It allows you to create a graph of your pipeline using NiPype functions but without coding! Just save your NiPype script in a .py file and send it as a new issue, we will convert this script to a script which works with our specific parameters.
...

## Find or propose an issue :clipboard:
Issues are very important for this project. If you want to contribute, you can either **comment an existing issue** or **proposing a new issue**.
Expand Down Expand Up @@ -64,3 +68,7 @@ Once your PR is ready, you may add a reviewer to your PR, as described [here](ht
Please turn your Draft Pull Request into a "regular" Pull Request, by clicking **Ready for review** in the Pull Request page.

**:wave: Thank you in advance for contributing to the project!**

## Additional resources

- git and Gitub: general guidelines can be found [here](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) in the GitHub documentation.
44 changes: 12 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,45 +15,24 @@
<img src="https://img.shields.io/github/commit-activity/m/Inria-Empenn/narps_open_pipelines" /></a>
</p>

## Table of contents
**The goal of the NARPS Open Pipelines project is to create a codebase reproducing the 70 pipelines of the NARPS study (Botvinik-Nezer et al., 2020) and share this as an open resource for the community**.

- [Project presentation](#project-presentation)
- [Getting Started](#getting-started)
- [Contents overview](#contents-overview)
- [Installation](#installation)
- [Contributing](#contributing)
- [References](#references)
- [Funding](#funding)
We base our reproductions on the [original descriptions provided by the teams](https://github.com/poldrack/narps/blob/1.0.1/ImageAnalyses/metadata_files/analysis_pipelines_for_analysis.xlsx) and test the quality of the reproductions by comparing our results with the original results published on NeuroVault.

## Project presentation
:vertical_traffic_light: See [the pipeline dashboard](https://github.com/Inria-Empenn/narps_open_pipelines/wiki/pipeline_status) to view our current progress at a glance.

Neuroimaging workflows are highly flexible, leaving researchers with multiple possible options to analyze a dataset [(Carp, 2012)](https://www.frontiersin.org/articles/10.3389/fnins.2012.00149/full).
However, different analytical choices can cause variation in the results [(Botvinik-Nezer et al., 2020)](https://www.nature.com/articles/s41586-020-2314-9), leading to what was called a "vibration of effects" [(Ioannidis, 2008)](https://pubmed.ncbi.nlm.nih.gov/18633328/) also known as analytical variability.
## Contributing

**The goal of the NARPS Open Pipelines project is to create a codebase reproducing the 70 pipelines of the NARPS project (Botvinik-Nezer et al., 2020) and share this as an open resource for the community**.
NARPS open pipelines uses [nipype](https://nipype.readthedocs.io/en/latest/index.html) as a workflow manager and provides a series of templates and examples to help reproduce the different teams’ analyses.

To perform the reproduction, we are lucky to be able to use the [descriptions provided by the teams](https://github.com/poldrack/narps/blob/1.0.1/ImageAnalyses/metadata_files/analysis_pipelines_for_analysis.xlsx).
We also created a [shared spreadsheet](https://docs.google.com/spreadsheets/d/1FU_F6kdxOD4PRQDIHXGHS4zTi_jEVaUqY_Zwg0z6S64/edit?usp=sharing) that can be used to add comments on pipelines (e.g.: identify the ones that are not reproducible with NiPype).

:vertical_traffic_light: Lastly, please find [here in the project's wiki](https://github.com/Inria-Empenn/narps_open_pipelines/wiki/pipeline_status) a dashboard to see pipelines work progresses at first glance.

## Getting Started

### Contents overview

- :snake: :package: `narps_open/` contains the Python package with all the pipelines logic.
- :brain: `data/` contains data that is used by the pipelines, as well as the (intermediate or final) results data. Instructions to download data are available in [INSTALL.md](/INSTALL.md#data-download-instructions).
- :blue_book: `docs/` contains the documentation for the project. Start browsing it [here](/docs/README.md) !
- :orange_book: `examples/` contains notebooks examples to launch of the reproduced pipelines.
- :microscope: `tests/` contains the tests of the narps_open package.
There are many ways you can contribute 🤗 :wave: Any help is welcome ! Follow the guidelines in [CONTRIBUTING.md](/CONTRIBUTING.md) if you wish to get involved !

### Installation

To get the pipelines running, please follow the installation steps in [INSTALL.md](/INSTALL.md)

### Contributing

:wave: Any help is welcome ! Follow the guidelines in [CONTRIBUTING.md](/CONTRIBUTING.md) if you wish to get involved !
## Getting started
If you are interested in using the codebase to run the pipelines, see the [user documentation (work-in-progress)].

## References

Expand All @@ -64,7 +43,7 @@ To get the pipelines running, please follow the installation steps in [INSTALL.m

## Funding

This project is supported by Région Bretagne (Boost MIND).
This project is supported by Région Bretagne (Boost MIND) and by Inria (Exploratory action GRASP).

## Credits

Expand All @@ -73,5 +52,6 @@ This project is developed in the Empenn team by Boris Clenet, Elodie Germani, Je
In addition, this project was presented and received contributions during the following events:
- OHBM Brainhack 2022 (June 2022): Elodie Germani, Arshitha Basavaraj, Trang Cao, Rémi Gau, Anna Menacher, Camille Maumet.
- e-ReproNim FENS NENS Cluster Brainhack (June 2023) : Liz Bushby, Boris Clénet, Michael Dayan, Aimee Westbrook.
- OHBM Brainhack 2023 (July 2023): Arshitha Basavaraj, Boris Clénet, Rémi Gau, Élodie Germani, Yaroslav Halchenko, Camille Maumet, Paul Taylor.
- ORIGAMI lab hackathon (Sept 2023):
- [OHBM Brainhack 2023](https://ohbm.github.io/hackathon2023/) (July 2023): Arshitha Basavaraj, Boris Clénet, Rémi Gau, Élodie Germani, Yaroslav Halchenko, Camille Maumet, Paul Taylor.
- [ORIGAMI lab](https://neurodatascience.github.io/) hackathon (September 2023):
- [Brainhack Marseille 2023](https://brainhack-marseille.github.io/) (December 2023):
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
## Contribute to the code
* :goggles: [description](/docs/description.md) - Conveniently access descriptions of the pipelines, as written by the teams involved in NARPS.
* :writing_hand: [pipelines](/docs/pipelines.md) - How to write pipelines.
* :compass: [core](/docs/core.md) - Helpful functions for writing pipelines.
* :microscope: [testing](/docs/testing.md) - How to test the code.

## For maintainers
Expand Down
117 changes: 117 additions & 0 deletions docs/core.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Core functions you can use to write pipelines

Here are a few functions that could be useful for creating a pipeline with Nipype. These functions are meant to stay as unitary as possible.

These are intended to be inserted in a nipype.Workflow inside a [nipype.Function](https://nipype.readthedocs.io/en/latest/api/generated/nipype.interfaces.utility.wrappers.html#function) interface, or for some of them (see associated docstring) as part of a [nipype.Workflow.connect](https://nipype.readthedocs.io/en/latest/api/generated/nipype.pipeline.engine.workflows.html#nipype.pipeline.engine.workflows.Workflow.connect) method.

In the following example, we use the `list_intersection` function of `narps_open.core.common`, in both of the mentioned cases.

```python
from nipype import Node, Function, Workflow
from narps_open.core.common import list_intersection

# First case : a Function Node
intersection_node = Node(Function(
function = list_intersection,
input_names = ['list_1', 'list_2'],
output_names = ['output']
), name = 'intersection_node')
intersection_node.inputs.list_1 = ['001', '002', '003', '004']
intersection_node.inputs.list_2 = ['002', '004', '005']
print(intersection_node.run().outputs.output) # ['002', '004']

# Second case : inside a connect node
# We assume that there is a node_0 returning ['001', '002', '003', '004'] as `output` value
test_workflow = Workflow(
base_dir = '/path/to/base/dir',
name = 'test_workflow'
)
test_workflow.connect([
# node_1 will receive the evaluation of :
# list_intersection(['001', '002', '003', '004'], ['002', '004', '005'])
# as in_value
(node_0, node_1, [(('output', list_intersection, ['002', '004', '005']), 'in_value')])
])
test_workflow.run()
```

> [!TIP]
> Use a [nipype.MapNode](https://nipype.readthedocs.io/en/latest/api/generated/nipype.pipeline.engine.nodes.html#nipype.pipeline.engine.nodes.MapNode) to run these functions on lists instead of unitary contents. E.g.: the `remove_file` function of `narps_open.core.common` only removes one file at a time, but feel free to pass a list of files using a `nipype.MapNode`.
```python
from nipype import MapNode, Function
from narps_open.core.common import remove_file

# Create the MapNode so that the `remove_file` function handles lists of files
remove_files_node = MapNode(Function(
function = remove_file,
input_names = ['_', 'file_name'],
output_names = []
), name = 'remove_files_node', iterfield = ['file_name'])

# ... A couple of lines later, in the Worlflow definition
test_workflow = Workflow(base_dir = '/home/bclenet/dev/tests/nipype_merge/', name = 'test_workflow')
test_workflow.connect([
# ...
# Here we assume the select_node's output `out_files` is a list of files
(select_node, remove_files_node, [('out_files', 'file_name')])
# ...
])
```

## narps_open.core.common

This module contains a set of functions that nearly every pipeline could use.

* `remove_file` remove a file when it is not needed anymore (to save disk space)

```python
from narps_open.core.common import remove_file

# Remove the /path/to/the/image.nii.gz file
remove_file('/path/to/the/image.nii.gz')
```

* `elements_in_string` : return the first input parameter if it contains one element of second parameter (None otherwise).

```python
from narps_open.core.common import elements_in_string

# Here we test if the file 'sub-001_file.nii.gz' belongs to a group of subjects.
elements_in_string('sub-001_file.nii.gz', ['005', '006', '007']) # Returns None
elements_in_string('sub-001_file.nii.gz', ['001', '002', '003']) # Returns 'sub-001_file.nii.gz'
```

> [!TIP]
> This can be generalised to a group of files, using a `nipype.MapNode`!
* `clean_list` : remove elements of the first input parameter (list) if it is equal to the second parameter.

```python
from narps_open.core.common import clean_list

# Here we remove subject 002 from a group of subjects.
clean_list(['002', '005', '006', '007'], '002')
```

* `list_intersection` : return the intersection of two lists.

```python
from narps_open.core.common import list_intersection

# Here we keep only subjects that are in the equalRange group and selected for the analysis.
equal_range_group = ['002', '004', '006', '008']
selected_for_analysis = ['002', '006', '010']
list_intersection(equal_range_group, selected_for_analysis) # Returns ['002', '006']
```

## narps_open.core.image

This module contains a set of functions dedicated to computations on images.

* `get_voxel_dimensions` : returns the voxel dimensions of an image

```python
# Get dimensions of voxels along x, y, and z in mm (returns e.g.: [1.0, 1.0, 1.0]).
get_voxel_dimensions('/path/to/the/image.nii.gz')
```
33 changes: 25 additions & 8 deletions docs/status.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,18 @@
# Access the work progress status pipelines

The class `PipelineStatusReport` of module `narps_open.utils.status` allows to create a report containing the following information for each pipeline:
* a work progress status : `idle`, `progress`, or `done`;
* the software it uses (collected from the `categorized_for_analysis.analysis_SW` of the [team description](/docs/description.md)) ;
* whether it uses data from fMRIprep or not ;
* a list of issues related to it (the opened issues of the project that have the team ID inside their title or description) ;
* a work progress status : `idle`, `progress`, or `done`.
* a list of pull requests related to it (the opened pull requests of the project that have the team ID inside their title or description) ;
* whether it was excluded from the original NARPS analysis ;
* a reproducibility rating :
* default score is 4;
* -1 if the team did not use fmriprep data;
* -1 if the team used several pieces of software (e.g.: FSL and AFNI);
* -1 if the team used custom or marginal software (i.e.: something else than SPM, FSL, AFNI or nistats);
* -1 if the team did not provided his source code.

This allows contributors to best select the pipeline they want/need to contribute to. For this purpose, the GitHub Actions workflow [`.github/workflows/pipeline_status.yml`](/.github/workflows/pipeline_status.yml) allows to dynamically generate the report and to store it in the [project's wiki](https://github.com/Inria-Empenn/narps_open_pipelines/wiki).

Expand Down Expand Up @@ -55,22 +63,31 @@ python narps_open/utils/status --json
# "softwares": "FSL",
# "fmriprep": "No",
# "issues": {},
# "status": "idle"
# "excluded": "No",
# "reproducibility": 3,
# "reproducibility_comment": "",
# "issues": {},
# "pulls": {},
# "status": "2-idle"
# },
# "0C7Q": {
# "softwares": "FSL, AFNI",
# "fmriprep": "Yes",
# "issues": {},
# "excluded": "No",
# "reproducibility": 3,
# "reproducibility_comment": "",
# "issues": {},
# "pulls": {},
# "status": "idle"
# },
# ...

python narps_open/utils/status --md
# | team_id | status | softwares used | fmriprep used ? | related issues |
# | --- |:---:| --- | --- | --- |
# | 08MQ | :red_circle: | FSL | No | |
# | 0C7Q | :red_circle: | FSL, AFNI | Yes | |
# | 0ED6 | :red_circle: | SPM | No | |
# | 0H5E | :red_circle: | SPM | No | |
# ...
# | team_id | status | main software | fmriprep used ? | related issues | related pull requests | excluded from NARPS analysis | reproducibility |
# | --- |:---:| --- | --- | --- | --- | --- | --- |
# | Q6O0 | :green_circle: | SPM | Yes | | | No | :star::star::star::black_small_square:<br /> |
# | UK24 | :orange_circle: | SPM | No | [2](url_issue_2), | | No | :star::star::black_small_square::black_small_square:<br /> |
# ...
```
Empty file added narps_open/core/__init__.py
Empty file.
65 changes: 65 additions & 0 deletions narps_open/core/common.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
#!/usr/bin/python
# coding: utf-8

""" Common functions to write pipelines """

def remove_file(_, file_name: str) -> None:
"""
Fully remove files generated by a Node, once they aren't needed anymore.
This function is meant to be used in a Nipype Function Node.
Parameters:
- _: input only used for triggering the Node
- file_name: str, a single absolute filename of the file to remove
"""
# This import must stay inside the function, as required by Nipype
from os import remove

try:
remove(file_name)
except OSError as error:
print(error)

def elements_in_string(input_str: str, elements: list) -> str: #| None:
"""
Return input_str if it contains one element of the elements list.
Return None otherwise.
This function is meant to be used in a Nipype Function Node.
Parameters:
- input_str: str
- elements: list of str, elements to be searched in input_str
"""
if any(e in input_str for e in elements):
return input_str
return None

def clean_list(input_list: list, element = None) -> list:
"""
Remove elements of input_list that are equal to element and return the resultant list.
This function is meant to be used in a Nipype Function Node. It can be used inside a
nipype.Workflow.connect call as well.
Parameters:
- input_list: list
- element: any
Returns:
- input_list with elements equal to element removed
"""
return [f for f in input_list if f != element]

def list_intersection(list_1: list, list_2: list) -> list:
"""
Returns the intersection of two lists.
This function is meant to be used in a Nipype Function Node. It can be used inside a
nipype.Workflow.connect call as well.
Parameters:
- list_1: list
- list_2: list
Returns:
- list, the intersection of list_1 and list_2
"""
return [e for e in list_1 if e in list_2]
Loading

0 comments on commit 6466b29

Please sign in to comment.