Skip to content

Commit

Permalink
Merge branch 'Inria-Empenn:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
bclenet authored Nov 20, 2023
2 parents 91dc744 + 83a58ed commit c03e9d1
Show file tree
Hide file tree
Showing 25 changed files with 846 additions and 158 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -139,3 +139,6 @@ dmypy.json
# Pyre type checker
.pyre/
*pyscript*

# For mac users
*.DS_Store
7 changes: 7 additions & 0 deletions CONTENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
### Contents overview

- :snake: :package: `narps_open/` contains the Python package with all the pipelines logic.
- :brain: `data/` contains data that is used by the pipelines, as well as the (intermediate or final) results data. Instructions to download data are available in [INSTALL.md](/INSTALL.md#data-download-instructions).
- :blue_book: `docs/` contains the documentation for the project. Start browsing it with the entry point [docs/README.md](/docs/README.md)
- :orange_book: `examples/` contains notebooks examples to launch of the reproduced pipelines.
- :microscope: `tests/` contains the tests of the narps_open package.
22 changes: 15 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,19 @@
# How to contribute to NARPS Open Pipelines ?

General guidelines can be found [here](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) in the GitHub documentation.
For the reproductions, we are especially looking for contributors with the following profiles:
- 👩‍🎤 SPM, FSL, AFNI or nistats has no secrets for you? You know this fMRI analysis software by heart 💓. Please help us by reproducing the corresponding NARPS pipelines. 👣 after step 1, follow the fMRI expert trail.
- 🧑‍🎤 You are a nipype guru? 👣 after step 1, follow the nipype expert trail.

## Reproduce a pipeline :keyboard:
:thinking: Not sure which one to start with ? You can have a look on [this table](https://github.com/Inria-Empenn/narps_open_pipelines/wiki/pipeline_status) giving the work progress status for each pipeline. This will help choosing the one that best suits you!
# Step 1: Choose a pipeline to reproduce :keyboard:
:thinking: Not sure which pipeline to start with ? 🚦The [pipeline dashboard](https://github.com/Inria-Empenn/narps_open_pipelines/wiki/pipeline_status) provides the progress status for each pipeline. You can pick any pipeline that is in red (not started).

Need more information ? You can have a look to the pipeline description [here](https://docs.google.com/spreadsheets/d/1FU_F6kdxOD4PRQDIHXGHS4zTi_jEVaUqY_Zwg0z6S64/edit?usp=sharing). Also feel free to use the `narps_open.utils.description` module of the project, as described [in the documentation](/docs/description.md).
Need more information to make a decision? The `narps_open.utils.description` module of the project, as described [in the documentation](/docs/description.md) provides easy access to all the info we have on each pipeline.

When you are ready, [start an issue](https://github.com/Inria-Empenn/narps_open_pipelines/issues/new/choose) and choose **Pipeline reproduction**!

### If you have experience with NiPype
# Step 2: Reproduction

## 🧑‍🎤 NiPype trail

We created templates with modifications to make and holes to fill to create a pipeline. You can find them in [`narps_open/pipelines/templates`](/narps_open/pipelines/templates).

Expand All @@ -21,9 +25,9 @@ Feel free to have a look to the following pipelines, these are examples :
| 2T6S | SPM | Yes | [/narps_open/pipelines/team_2T6S.py](/narps_open/pipelines/team_2T6S.py) |
| X19V | FSL | Yes | [/narps_open/pipelines/team_X19V.py](/narps_open/pipelines/team_2T6S.py) |

### If you have experience with the original software package but not with NiPype
## 👩‍🎤 fMRI software trail

A fantastic tool named [Giraffe](https://giraffe.tools/porcupine/TimVanMourik/GiraffePlayground/master) is available. It allows you to create a graph of your pipeline using NiPype functions but without coding! Just save your NiPype script in a .py file and send it as a new issue, we will convert this script to a script which works with our specific parameters.
...

## Find or propose an issue :clipboard:
Issues are very important for this project. If you want to contribute, you can either **comment an existing issue** or **proposing a new issue**.
Expand Down Expand Up @@ -64,3 +68,7 @@ Once your PR is ready, you may add a reviewer to your PR, as described [here](ht
Please turn your Draft Pull Request into a "regular" Pull Request, by clicking **Ready for review** in the Pull Request page.

**:wave: Thank you in advance for contributing to the project!**

## Additional resources

- git and Gitub: general guidelines can be found [here](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) in the GitHub documentation.
37 changes: 9 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,45 +15,26 @@
<img src="https://img.shields.io/github/commit-activity/m/Inria-Empenn/narps_open_pipelines" /></a>
</p>

## Table of contents

- [Project presentation](#project-presentation)
- [Getting Started](#getting-started)
- [Contents overview](#contents-overview)
- [Installation](#installation)
- [Contributing](#contributing)
- [References](#references)
- [Funding](#funding)

## Project presentation

Neuroimaging workflows are highly flexible, leaving researchers with multiple possible options to analyze a dataset [(Carp, 2012)](https://www.frontiersin.org/articles/10.3389/fnins.2012.00149/full).
However, different analytical choices can cause variation in the results [(Botvinik-Nezer et al., 2020)](https://www.nature.com/articles/s41586-020-2314-9), leading to what was called a "vibration of effects" [(Ioannidis, 2008)](https://pubmed.ncbi.nlm.nih.gov/18633328/) also known as analytical variability.
**The goal of the NARPS Open Pipelines project is to create a codebase reproducing the 70 pipelines of the NARPS study (Botvinik-Nezer et al., 2020) and share this as an open resource for the community**.

**The goal of the NARPS Open Pipelines project is to create a codebase reproducing the 70 pipelines of the NARPS project (Botvinik-Nezer et al., 2020) and share this as an open resource for the community**.
We base our reproductions on the [original descriptions provided by the teams](https://github.com/poldrack/narps/blob/1.0.1/ImageAnalyses/metadata_files/analysis_pipelines_for_analysis.xlsx) and test the quality of the reproductions by comparing our results with the original results published on NeuroVault.

To perform the reproduction, we are lucky to be able to use the [descriptions provided by the teams](https://github.com/poldrack/narps/blob/1.0.1/ImageAnalyses/metadata_files/analysis_pipelines_for_analysis.xlsx).
We also created a [shared spreadsheet](https://docs.google.com/spreadsheets/d/1FU_F6kdxOD4PRQDIHXGHS4zTi_jEVaUqY_Zwg0z6S64/edit?usp=sharing) that can be used to add comments on pipelines (e.g.: identify the ones that are not reproducible with NiPype).
:vertical_traffic_light: See [the pipeline dashboard](https://github.com/Inria-Empenn/narps_open_pipelines/wiki/pipeline_status) to view our current progress at a glance.

:vertical_traffic_light: Lastly, please find [here in the project's wiki](https://github.com/Inria-Empenn/narps_open_pipelines/wiki/pipeline_status) a dashboard to see pipelines work progresses at first glance.
## Contributing

## Getting Started
NARPS open pipelines uses [nipype](https://nipype.readthedocs.io/en/latest/index.html) as a workflow manager and provides a series of templates and examples to help reproduce the different teams’ analysis.

### Contents overview

- :snake: :package: `narps_open/` contains the Python package with all the pipelines logic.
- :brain: `data/` contains data that is used by the pipelines, as well as the (intermediate or final) results data. Instructions to download data are available in [INSTALL.md](/INSTALL.md#data-download-instructions).
- :blue_book: `docs/` contains the documentation for the project. Start browsing it with the entry point [docs/README.md](/docs/README.md)
- :orange_book: `examples/` contains notebooks examples to launch of the reproduced pipelines.
- :microscope: `tests/` contains the tests of the narps_open package.
There are many ways you can contribute 🤗 :wave: Any help is welcome ! Follow the guidelines in [CONTRIBUTING.md](/CONTRIBUTING.md) if you wish to get involved !

### Installation

To get the pipelines running, please follow the installation steps in [INSTALL.md](/INSTALL.md)

### Contributing

:wave: Any help is welcome ! Follow the guidelines in [CONTRIBUTING.md](/CONTRIBUTING.md) if you wish to get involved !
## Getting started
If you are interested in using the codebase to run the pipelines, see the [user documentation (work-in-progress)].

## References

Expand All @@ -64,7 +45,7 @@ To get the pipelines running, please follow the installation steps in [INSTALL.m

## Funding

This project is supported by Région Bretagne (Boost MIND).
This project is supported by Région Bretagne (Boost MIND) and by Inria (Exploratory action GRASP).

## Credits

Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ Here are the available topics :
* :microscope: [testing](/docs/testing.md) details the testing features of the project, i.e.: how is the code tested ?
* :package: [ci-cd](/docs/ci-cd.md) contains the information on how continuous integration and delivery (knowned as CI/CD) is set up.
* :writing_hand: [pipeline](/docs/pipelines.md) tells you all you need to know in order to write pipelines
* :compass: [core](/docs/core.md) a list of helpful functions when writing pipelines
* :vertical_traffic_light: [status](/docs/status.md) contains the information on how to get the work progress status for a pipeline.
117 changes: 117 additions & 0 deletions docs/core.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Core functions you can use to write pipelines

Here are a few functions that could be useful for creating a pipeline with Nipype. These functions are meant to stay as unitary as possible.

These are intended to be inserted in a nipype.Workflow inside a [nipype.Function](https://nipype.readthedocs.io/en/latest/api/generated/nipype.interfaces.utility.wrappers.html#function) interface, or for some of them (see associated docstring) as part of a [nipype.Workflow.connect](https://nipype.readthedocs.io/en/latest/api/generated/nipype.pipeline.engine.workflows.html#nipype.pipeline.engine.workflows.Workflow.connect) method.

In the following example, we use the `list_intersection` function of `narps_open.core.common`, in both of the mentioned cases.

```python
from nipype import Node, Function, Workflow
from narps_open.core.common import list_intersection

# First case : a Function Node
intersection_node = Node(Function(
function = list_intersection,
input_names = ['list_1', 'list_2'],
output_names = ['output']
), name = 'intersection_node')
intersection_node.inputs.list_1 = ['001', '002', '003', '004']
intersection_node.inputs.list_2 = ['002', '004', '005']
print(intersection_node.run().outputs.output) # ['002', '004']

# Second case : inside a connect node
# We assume that there is a node_0 returning ['001', '002', '003', '004'] as `output` value
test_workflow = Workflow(
base_dir = '/path/to/base/dir',
name = 'test_workflow'
)
test_workflow.connect([
# node_1 will receive the evaluation of :
# list_intersection(['001', '002', '003', '004'], ['002', '004', '005'])
# as in_value
(node_0, node_1, [(('output', list_intersection, ['002', '004', '005']), 'in_value')])
])
test_workflow.run()
```

> [!TIP]
> Use a [nipype.MapNode](https://nipype.readthedocs.io/en/latest/api/generated/nipype.pipeline.engine.nodes.html#nipype.pipeline.engine.nodes.MapNode) to run these functions on lists instead of unitary contents. E.g.: the `remove_file` function of `narps_open.core.common` only removes one file at a time, but feel free to pass a list of files using a `nipype.MapNode`.
```python
from nipype import MapNode, Function
from narps_open.core.common import remove_file

# Create the MapNode so that the `remove_file` function handles lists of files
remove_files_node = MapNode(Function(
function = remove_file,
input_names = ['_', 'file_name'],
output_names = []
), name = 'remove_files_node', iterfield = ['file_name'])

# ... A couple of lines later, in the Worlflow definition
test_workflow = Workflow(base_dir = '/home/bclenet/dev/tests/nipype_merge/', name = 'test_workflow')
test_workflow.connect([
# ...
# Here we assume the select_node's output `out_files` is a list of files
(select_node, remove_files_node, [('out_files', 'file_name')])
# ...
])
```

## narps_open.core.common

This module contains a set of functions that nearly every pipeline could use.

* `remove_file` remove a file when it is not needed anymore (to save disk space)

```python
from narps_open.core.common import remove_file

# Remove the /path/to/the/image.nii.gz file
remove_file('/path/to/the/image.nii.gz')
```

* `elements_in_string` : return the first input parameter if it contains one element of second parameter (None otherwise).

```python
from narps_open.core.common import elements_in_string

# Here we test if the file 'sub-001_file.nii.gz' belongs to a group of subjects.
elements_in_string('sub-001_file.nii.gz', ['005', '006', '007']) # Returns None
elements_in_string('sub-001_file.nii.gz', ['001', '002', '003']) # Returns 'sub-001_file.nii.gz'
```

> [!TIP]
> This can be generalised to a group of files, using a `nipype.MapNode`!
* `clean_list` : remove elements of the first input parameter (list) if it is equal to the second parameter.

```python
from narps_open.core.common import clean_list

# Here we remove subject 002 from a group of subjects.
clean_list(['002', '005', '006', '007'], '002')
```

* `list_intersection` : return the intersection of two lists.

```python
from narps_open.core.common import list_intersection

# Here we keep only subjects that are in the equalRange group and selected for the analysis.
equal_range_group = ['002', '004', '006', '008']
selected_for_analysis = ['002', '006', '010']
list_intersection(equal_range_group, selected_for_analysis) # Returns ['002', '006']
```

## narps_open.core.image

This module contains a set of functions dedicated to computations on images.

* `get_voxel_dimensions` : returns the voxel dimensions of an image

```python
# Get dimensions of voxels along x, y, and z in mm (returns e.g.: [1.0, 1.0, 1.0]).
get_voxel_dimensions('/path/to/the/image.nii.gz')
```
33 changes: 25 additions & 8 deletions docs/status.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,18 @@
# Access the work progress status pipelines

The class `PipelineStatusReport` of module `narps_open.utils.status` allows to create a report containing the following information for each pipeline:
* a work progress status : `idle`, `progress`, or `done`;
* the software it uses (collected from the `categorized_for_analysis.analysis_SW` of the [team description](/docs/description.md)) ;
* whether it uses data from fMRIprep or not ;
* a list of issues related to it (the opened issues of the project that have the team ID inside their title or description) ;
* a work progress status : `idle`, `progress`, or `done`.
* a list of pull requests related to it (the opened pull requests of the project that have the team ID inside their title or description) ;
* whether it was excluded from the original NARPS analysis ;
* a reproducibility rating :
* default score is 4;
* -1 if the team did not use fmriprep data;
* -1 if the team used several pieces of software (e.g.: FSL and AFNI);
* -1 if the team used custom or marginal software (i.e.: something else than SPM, FSL, AFNI or nistats);
* -1 if the team did not provided his source code.

This allows contributors to best select the pipeline they want/need to contribute to. For this purpose, the GitHub Actions workflow [`.github/workflows/pipeline_status.yml`](/.github/workflows/pipeline_status.yml) allows to dynamically generate the report and to store it in the [project's wiki](https://github.com/Inria-Empenn/narps_open_pipelines/wiki).

Expand Down Expand Up @@ -55,22 +63,31 @@ python narps_open/utils/status --json
# "softwares": "FSL",
# "fmriprep": "No",
# "issues": {},
# "status": "idle"
# "excluded": "No",
# "reproducibility": 3,
# "reproducibility_comment": "",
# "issues": {},
# "pulls": {},
# "status": "2-idle"
# },
# "0C7Q": {
# "softwares": "FSL, AFNI",
# "fmriprep": "Yes",
# "issues": {},
# "excluded": "No",
# "reproducibility": 3,
# "reproducibility_comment": "",
# "issues": {},
# "pulls": {},
# "status": "idle"
# },
# ...

python narps_open/utils/status --md
# | team_id | status | softwares used | fmriprep used ? | related issues |
# | --- |:---:| --- | --- | --- |
# | 08MQ | :red_circle: | FSL | No | |
# | 0C7Q | :red_circle: | FSL, AFNI | Yes | |
# | 0ED6 | :red_circle: | SPM | No | |
# | 0H5E | :red_circle: | SPM | No | |
# ...
# | team_id | status | main software | fmriprep used ? | related issues | related pull requests | excluded from NARPS analysis | reproducibility |
# | --- |:---:| --- | --- | --- | --- | --- | --- |
# | Q6O0 | :green_circle: | SPM | Yes | | | No | :star::star::star::black_small_square:<br /> |
# | UK24 | :orange_circle: | SPM | No | [2](url_issue_2), | | No | :star::star::black_small_square::black_small_square:<br /> |
# ...
```
Empty file added narps_open/core/__init__.py
Empty file.
65 changes: 65 additions & 0 deletions narps_open/core/common.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
#!/usr/bin/python
# coding: utf-8

""" Common functions to write pipelines """

def remove_file(_, file_name: str) -> None:
"""
Fully remove files generated by a Node, once they aren't needed anymore.
This function is meant to be used in a Nipype Function Node.
Parameters:
- _: input only used for triggering the Node
- file_name: str, a single absolute filename of the file to remove
"""
# This import must stay inside the function, as required by Nipype
from os import remove

try:
remove(file_name)
except OSError as error:
print(error)

def elements_in_string(input_str: str, elements: list) -> str: #| None:
"""
Return input_str if it contains one element of the elements list.
Return None otherwise.
This function is meant to be used in a Nipype Function Node.
Parameters:
- input_str: str
- elements: list of str, elements to be searched in input_str
"""
if any(e in input_str for e in elements):
return input_str
return None

def clean_list(input_list: list, element = None) -> list:
"""
Remove elements of input_list that are equal to element and return the resultant list.
This function is meant to be used in a Nipype Function Node. It can be used inside a
nipype.Workflow.connect call as well.
Parameters:
- input_list: list
- element: any
Returns:
- input_list with elements equal to element removed
"""
return [f for f in input_list if f != element]

def list_intersection(list_1: list, list_2: list) -> list:
"""
Returns the intersection of two lists.
This function is meant to be used in a Nipype Function Node. It can be used inside a
nipype.Workflow.connect call as well.
Parameters:
- list_1: list
- list_2: list
Returns:
- list, the intersection of list_1 and list_2
"""
return [e for e in list_1 if e in list_2]
25 changes: 25 additions & 0 deletions narps_open/core/image.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/python
# coding: utf-8

""" Image functions to write pipelines """

def get_voxel_dimensions(image: str) -> list:
"""
Return the voxel dimensions of a image in millimeters.
Arguments:
image: str, string that represent an absolute path to a Nifti image.
Returns:
list, size of the voxels in the image in millimeters.
"""
# This import must stay inside the function, as required by Nipype
from nibabel import load

voxel_dimensions = load(image).header.get_zooms()

return [
float(voxel_dimensions[0]),
float(voxel_dimensions[1]),
float(voxel_dimensions[2])
]
Loading

0 comments on commit c03e9d1

Please sign in to comment.