GGP Quick Start Guide

Installation

The preferred build is using CMake and GGP requires C++14 support, which implies a requirement of gcc >= 5.

Note that GGP requires a recent enough version of cmake. If your system does not have a recent enough version installed or available as module you can download precompiled binaries or sources from the CMake Download page and install them in your home directory.

Here we just describe a quick start, assuming you clone the latest development version from github.

1. Clone the repository git clone https://github.com/cpviolator/generic-GPU-project
2. Create a build directory mkdir build; cd build
3. Run cmake cmake ../generic-GPU-project
4. Select options ccmake . This will show all options and you can modify them. A short description of the options is included as well.
5. Run make using a parallel build make -j 16. In general the number of processes to use in parallel for building should match or slightly exceed the number of cores on your system,

Make sure you used the correct architecture for your GPUs in step 3. Default architecture is sm_70 but you may want to specify different architectures such as -DGGP_GPU_ARCH=sm_60 for a Pascal GPU or -DGGP_GPU_ARCH=sm_80 for A100. More details can be found at GGP-Build-With-CMake.

Internal Tests

TODO

Using the Library

Include the header file include/ggp.h in your application, link against lib/libggp.a.

Kernel Autotuning

GGP uses runtime autotuning to maximize performance of each kernel on a given GPU. This brings better performance portability across both GPU architectures and different lattice volumes, parameters, etc. This tuning process takes some time and will generally slow things down the first time a given kernel is called during a run. To avoid this one-time overhead in subsequent runs (using the same action, solver, lattice volume, etc.), the optimal parameters are cached to disk. For this to work, the GGP_RESOURCE_PATH environment variable must be set, pointing to a writeable directory. Note that since the tuned parameters are hardware-specific, this "resource directory" should not be shared between jobs running on different systems (e.g., two clusters with different GPUs installed). Attempting to use parameters tuned for one card on a different card may lead to unexpected errors. In addition GGP will refuse to run with an outdated tunecache to avoid as using an outdated (i.e. tuned for an older version of GGP) may result in undefined behavior. The tunecache.tsv file is dumped at the end of run in the location specified by the GGP_RESOURCE_PATH environment. If this is not specified then autotuning will be cached only within the scope of the run, but lost when the job ends.

Debugging

GGP has two specific debugging modes: HOST_DEBUG and DEVICE_DEBUG.

• HOST_DEBUG compiles all host code using the -g flag and ensures that all CUDA error reporting is done synchronously (e.g., the GPU and CPU are synchronized prior to fetching the error state). For most debugging, HOST_DEBUG is all that should be needed since most bugs tend to be in CPU code. There is a noticeable performance impact enabling HOST_DEBUG, at the 20-50% level, with the penalty being greater at smaller local volumes.
• DEVICE_DEBUG compiles all GPU kernels using the -G flag. This provides for accurate line reporting in cuda-gdb and cuda-memch. There is a huge performance penalty impact from enabling this, at the 100x level.