-
Notifications
You must be signed in to change notification settings - Fork 5
General Structure
This project was written for processing functional MRI (EPI) data for the EMU project. Specifically, the project currently supports task and resting state EPI analyses, de/refacing of T1w files, and automated hippocampal subfield segmentation. Each pipeline is organized within a workflow which manage the various resources, and these workflows are written to be run on the HPC.
Generally, this project is organized around two main packages, workflow and resources. The workflow modules manage the resource package, and the user is intended to control the workflow from the set of scripts found in cli. The resource package contains sub-packages, one for each type of software employed. Currently, there are sub-packages for AFNI, ASHS, FreeSurfer, fMRIprep and an extra package for reports.
Likewise, a unique workflow module exists for each type of analysis. For example, the control_afni module controls all AFNI resources for subject task and RS pipelines as well as group-level analyses.
Package documentation can be found here (TBD), assuming sphinx is cooperative.
Users can control the workflow package, and therefore the various resources, via command line interface (cli) scripts. A set of scripts are supplied for the various types of analyses currently supported. These scripts utilize the output of checks.py (logs/completed_preprocessing.tsv) to determine which subjects need which files; this dependency was made so files could be moved from the HPC to local storage. The script checks.py can be used both locally and remotely, and should be run to update the referenced log before every cli script execution. That is, run checks.py before each new batch! Finally, note that checks.py requires internet connectivity.
Batches of jobs can be scheduled via scripts in cron, which will control cli. These are currently outdated but left as examples.
Files and scripts needed for quality checks are found in qc. Quality checks were used in pipeline validation for non-default options, and also supply an example of how a non-default deconvolution could be employed.
Some resources require docker/singularity images (such as resources.ashs.hippseg). The dockerfile used to create the image, and any scripts that serve as a resource/entrypoint in the image are kept in dockerfiles so this project is self-contained.
The numpy docstring format utilizing a flake8 linter is employed in all scripts, and sphinx, via napoleon, is used to generate documentation files in docs. Some modules have extra formatting that may not entirely comply to numpy format, this is for compatibility with sphinx-readthedocs.io generation.
Every cli script also contains a __doc__ docstring which contains usage examples, and many scripts and modules contain default arguments. Help is accessible via python foo.py for cli scripts, or help(module_name).