This repository includes two scripts written in Python using RADICAL Cybertools (specifically Ensemble Toolkit), to benchmark the performance of simplified drug workflows used to fight against the COVID-19 pandemic. benchmark_mixed.py
is a heterogeneous workflow with both CPU and GPU tasks, while benchmark_gpu.py
contains GPU tasks only for comparison purposes. CPU tasks use NAMD to perform simulation of large biomolecular systems. GPU tasks use GROMACS to do free energy calculation involving ensemble MD simulations.
(python3)
. "/sw/summit/python/3.7/anaconda3/5.3.0/etc/profile.d/conda.sh"
conda create -n benchmark python=3.7 -y
conda activate benchmark
pip install radical.entk radical.pilot radical.saga radical.utils --upgrade
export RMQ_HOSTNAME=129.114.17.233
export RADICAL_PILOT_DBURL="mongodb://rct:[email protected]:27017/rct_test"
Note that installation of the binaries used by either CPU tasks or GPU tasks are not needed, since we are using pre-installed binaries on Summit in shared locations from other efforts.
res_desc
includes:- 'resource': HPC facility name, e.g., 'ornl.summit'
- 'queue':
- 'batch' for up to 2 hours time to completion at
ornl.summit
- 'killable' for up to 24 hours time to completion at
ornl.summit
- 'batch' for up to 2 hours time to completion at
- 'cpus': On Summit each node has 42 cores and each core has 4 hardware threads, so we consider 168 parallel threads
- 'gpus': a number of GPU cores equal to N simulations (1 MD/ML run: 1 GPU)
The main script requires to be placed at a writable space before running. Output files are stored in sub-directories of the current directory. Locate this code repository at $MEMBERWORK/{{PROJECTID}}/ and run the script there. $HOME directory does not work when running the script since it becomes a read-only filesystem when a job is running.
$ python benchmark_mixed.py
$ python benchmark_gpu.py
Use average time to completion (TTX) of workflows running on Summit, unit: minute
Single Task TTX: e.g., 1 CPU task and/or 1 GPU task on 1 node
Multiple Task TTX: e.g., 41 CPU tasks and 6 GPU tasks on 1 node; 100 CPU tasks and 50 GPU tasks on 10 nodes