Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: Add benchmark page #3312

Merged
merged 6 commits into from
Jun 18, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,14 @@ build:
jobs:
post_checkout:
- git fetch --unshallow
# Use -W builds for PRs
- |
if (echo $READTHEDOCS_VERSION_NAME | grep -E '^[0-9]+$' >/dev/null); then
export SPHINXOPTS="-W"
fi
- env
pre_build:
- env
effigies marked this conversation as resolved.
Show resolved Hide resolved

python:
install:
Expand Down
86 changes: 86 additions & 0 deletions docs/benchmarks.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
.. include:: links.rst

----------------------
Performance benchmarks
----------------------

Datasets and commands
---------------------

Datasets
~~~~~~~~
+------------+----------------------------------------------------------------------------------------+
| Dataset | Description |
+============+========================================================================================+
| A | 6 T1w, 3 T2w, 2 PhaseDiff, 4 SE-BOLD (195 volumes / series; total 784), 4 sbref |
+------------+----------------------------------------------------------------------------------------+
| B | 2 T1w, 6 PEPolar fieldmaps, 8 SE-BOLD (4274 volumes total) |
+------------+----------------------------------------------------------------------------------------+

fMRIPrep versions and modes
~~~~~~~~~~~~~~~~~~~~~~~~~~~

All commands took the form ``fmriprep sourcedata/raw . participant $OPTIONS``.
The specific options for each version or mode are presented in the following table.

+------------+----------------------------------------------------------------------------------------+
| Version / | Options |
| Mode | |
+============+========================================================================================+
| 23.1.4 | ``--cifti-output --output-spaces MNI152NLin2009cAsym`` |
+------------+----------------------------------------------------------------------------------------+
| 23.2.0a2 / | ``--level minimal --cifti-output --output-spaces MNI152NLin2009cAsym`` |
| fit | |
+------------+----------------------------------------------------------------------------------------+
| 23.2.0a2 / | ``--level full --cifti-output --output-spaces MNI152NLin2009cAsym`` |
| fit + | |
| transform | |
+------------+----------------------------------------------------------------------------------------+

Machine details
~~~~~~~~~~~~~~~

Tests were run on a desktop machine running no other significant tests.

* Processor: Intel i9-10900 CPU @ 2.80GHz, 20-core
* Memory: 64GiB
* Storage: PC801 NVMe SK hynix 2TB
* OS: Ubuntu 22.04
* Environment: Docker images published to https://hub.docker.com/r/nipreps/fmriprep/

Benchmarks
----------

Dataset A
~~~~~~~~~

+------------+---------+---------+---------+---------+---------+
| Version / | Runtime | Scratch | Scratch | Output | Output |
| Mode | | Size | Files | Size | files |
+============+=========+=========+=========+=========+=========+
| 23.1.4 | 2h24m | 54.8GB | 36.8K | 2.30GB | 176 |
+------------+---------+---------+---------+---------+---------+
| 23.2.0a2 | 1h35m | 2.91GB | 5.89K | 602MB | 128 |
| / fit | | | | | |
+------------+---------+---------+---------+---------+---------+
| 23.2.0a2 | 1h47m | 19.8GB | 10.0K | 6.37GB | 206 |
| / fit + | | | | | |
| transform | | | | | |
+------------+---------+---------+---------+---------+---------+

Dataset B
~~~~~~~~~

+------------+---------+---------+---------+---------+---------+
| Version / | Runtime | Scratch | Scratch | Output | Output |
| Mode | | Size | Files | Size | files |
+============+=========+=========+=========+=========+=========+
| 23.1.4 | 4h25m | 121GB | 157K | 5.10GB | 286 |
+------------+---------+---------+---------+---------+---------+
| 23.2.0a2 | 1h29m | 1.88GB | 12.0K | 543MB | 206 |
| / fit | | | | | |
+------------+---------+---------+---------+---------+---------+
| 23.2.0a2 | 2h7m | 56.5GB | 19.8K | 14.7GB | 348 |
| / fit + | | | | | |
| transform | | | | | |
+------------+---------+---------+---------+---------+---------+
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Contents
usage
workflows
outputs
benchmarks
spaces
faq
api
Expand Down
38 changes: 25 additions & 13 deletions docs/workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,13 @@ single reference template (see `Longitudinal processing`_).

from niworkflows.utils.spaces import Reference, SpatialReferences
from smriprep.workflows.anatomical import init_anat_preproc_wf
spaces=SpatialReferences([
('MNI152Lin', {}),
('fsaverage', {'density': '10k'}),
('T1w', {}),
('fsnative', {})
])
spaces.checkpoint()
wf = init_anat_preproc_wf(
bids_root='.',
freesurfer=True,
Expand All @@ -71,12 +78,7 @@ single reference template (see `Longitudinal processing`_).
output_dir='.',
skull_strip_mode='force',
skull_strip_template=Reference('MNI152NLin2009cAsym'),
spaces=SpatialReferences([
('MNI152Lin', {}),
('fsaverage', {'density': '10k'}),
('T1w', {}),
('fsnative', {})
]),
spaces=spaces,
skull_strip_fixed_seed=False,
t1w=['sub-01/anat/sub-01_T1w.nii.gz'],
t2w=[],
Expand Down Expand Up @@ -267,9 +269,9 @@ packages, including FreeSurfer and the `Connectome Workbench`_.
:simple_form: yes

from smriprep.workflows.surfaces import init_surface_recon_wf
wf = init_surface_recon_wf(omp_nthreads=1,
hires=True,
precomputed={})
wf = init_surface_recon_wf(
omp_nthreads=1, hires=True, precomputed={}, fs_no_resume=False,
)

See also *sMRIPrep*'s
:py:func:`~smriprep.workflows.surfaces.init_surface_recon_wf`
Expand Down Expand Up @@ -401,6 +403,7 @@ Slice time correction

from fmriprep.workflows.bold import init_bold_stc_wf
wf = init_bold_stc_wf(
mem_gb={'filesize': 1},
metadata={'RepetitionTime': 2.0,
'SliceTiming': [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]},
)
Expand Down Expand Up @@ -479,8 +482,9 @@ EPI to T1w registration
wf = init_bbreg_wf(
omp_nthreads=1,
use_bbr=True,
bold2t1w_dof=9,
bold2t1w_init='register')
bold2anat_dof=9,
bold2anat_init='t2w',
)

The alignment between the reference :abbr:`EPI (echo-planar imaging)` image
of each run and the reconstructed subject using the gray/white matter boundary
Expand Down Expand Up @@ -510,7 +514,16 @@ Resampling BOLD runs onto standard spaces
:simple_form: yes

from fmriprep.workflows.bold.apply import init_bold_volumetric_resample_wf
wf = init_bold_volumetric_resample_wf(metadata={}, fieldmap_id='fmap')
wf = init_bold_volumetric_resample_wf(
metadata={
'RepetitionTime': 2.0,
'PhaseEncodingDirection': 'j-',
'TotalReadoutTime': 0.03
},
mem_gb={'resampled': 1},
jacobian=True,
fieldmap_id='fmap',
)

This sub-workflow concatenates the transforms calculated upstream (see
`Head-motion estimation`_, `Susceptibility Distortion Correction (SDC)`_ --if
Expand Down Expand Up @@ -564,7 +577,6 @@ HCP Grayordinates

from fmriprep.workflows.bold.resampling import init_bold_fsLR_resampling_wf
wf = init_bold_fsLR_resampling_wf(
estimate_goodvoxels=True,
grayord_density='92k',
omp_nthreads=1,
mem_gb=1,
Expand Down
2 changes: 2 additions & 0 deletions fmriprep/workflows/bold/apply.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ def init_bold_volumetric_resample_wf(
'PhaseEncodingDirection': 'j-',
'TotalReadoutTime': 0.03
},
mem_gb={'resampled': 1},
jacobian=True,
fieldmap_id='my_fieldmap',
)

Expand Down
2 changes: 1 addition & 1 deletion fmriprep/workflows/bold/registration.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ def init_bold_reg_wf(
omp_nthreads=1,
use_bbr=True,
bold2anat_dof=9,
bold2anat_init='auto')
bold2anat_init='t2w')

Parameters
----------
Expand Down
1 change: 1 addition & 0 deletions fmriprep/workflows/bold/stc.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def init_bold_stc_wf(

from fmriprep.workflows.bold import init_bold_stc_wf
wf = init_bold_stc_wf(
mem_gb={'filesize': 1},
metadata={"RepetitionTime": 2.0,
"SliceTiming": [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]},
)
Expand Down