Home

Running on Palmetto

1. Checkout sailfish

git clone [email protected]:clemson-cal/sailfish.git

2. Install dependency Python modules

pip3 install --user cffi # only needed for running in CPU mode
pip3 install --user cupy-cuda111 # only needed for running in GPU mode

3. Get a compute node

qsub -I -l select=1:ncpus=56:mem=200gb:ngpus=2:gpu_model=a100,walltime=24:00:00

4. Load the cuda module

module load cuda/11.1.1-gcc/9.5.0

5. Go to the sailfish directory and run something

cd sailfish
./scripts/main.py -g kitp-code-comparison

Running on workstations

The primary difference when running on a workstation is installing the correct version of cupy for your machine.

NVIDIA GPUs

On a workstation running CUDA version 11.7 with an NVIDIA GPU, one would run

pip3 install --user cupy-cuda117 # only needed for running in GPU mode

To check which CUDA version is installed, you can run

nvidia-smi

AMD GPUs

At the moment, cupy only seems to support versions 4.3 and 5.0 of ROCm. You can check which version is installed using

apt show rocm-libs # on a system with the apt package manager

yum info rocm-libs # on a system with the yum package manager

Then, install the version of cupy corresponding to your ROCm version, e.g.

pip3 install --user cupy-rocm-5-0