The ParaStation Communication Library (pscom for short) is an open-source low-level communication library, especially designed for the employment in HPC systems.
This guide describes the installation of pscom.
For installations of ParaStation MPI, the default install directory is typically /opt/parastation/.
- CMake, at least version 3.10
- C compiler with C99 support
- IB Vebs library for plugin OpenIB
- UCP library for plugin UCP
- Portals4 library for plugin Portals (BXI support)
- PSM2 library for plugin PSM
- Extoll and Velo libraries for plugins Extoll and Velo
- DAPL library for plugin DAPL
- CUDA libraries for CUDA awareness of
pscom
poptlibrary, a command line option parsing library used by some of the tools in thebinfolder- cmocka Library for unit tests
- Python packages
cmake-format,cmakelintandPyYamlfor CMake style checking. Install them via Python's package management system, e.gpip3 install cmake-format cmakelint PyYaml. clang-formatfor applying the coding style (cf. Contributing)
If CMake does not detect a library on your system that is required for a plugin or feature you want to use, check if the install directory of that library is contained in your system's and compiler's library and header environment paths, e.g., LIBRARY_PATH, LD_LIBRARY_PATH, PATH, and C_PATH.
Alternatively, for some libraries it is possible to specify the local install directory in your system by passing the following options to CMake:
| Library | CMake option | Default value |
|---|---|---|
| CUDA | -DCUDA_HOME |
/usr/local/cuda |
| popt | -DPOPT_HOME |
/usr |
| Extoll | -DEXTOLL_HOME |
/opt/extoll/x86_64 |
| PSM2 | -DPSM2_HOME |
/usr |
| UCP | -DUCP_HOME |
/usr |
| Portals4 | -DPORTALS4_HOME |
/usr |
To install pscom in a custom location, pass -DCMAKE_INSTALL_PREFIX=pscom/install/dir/ to CMake.
Default install location is /opt/parastation.
This location will be used, if CMAKE_INSTALL_PREFIX is not explicitly set.
Currently, pscom offers 5 different build types
Build Type & CMAKE_BUILD_TYPE value |
Purpose |
|---|---|
| None | No debug info or optimizations |
| Debug | Debug info, few optimizations without interference with debugging |
| Release | No debug info (esp. no assertions), aggressive optimization |
| RelWithDebInfo (default) | Debug info (esp. assertions), aggressive optimization |
| MinSizeRel | Debug info, code size optimized |
Set the value of -DCMAKE_BUILD_TYPE=<selected build type> to select a build type different than the default.
CMake auto-detects which headers and libraries are present on your system and disables all plugins for which the requirements cannot be met.
A respective summary of enabled/ disabled plugins is printed at the end of the CMake run.
If you want to disable a plugin explicitly (even if requirements are met by your system), you can pass the option -D<plugin name>_ENABLED=0 to CMake, for example -DPSM2_ENABLED=0 to disable the PSM plugin.
For runtime configuration options of plugins including customization of their priorities, see here.
If the CUDA library is found on your system, pscom is automatically compiled with CUDA awareness features enabled.
If you want to disable CUDA awareness, you can pass -DCUDA_ENABLED=0 to CMake.
To enable the unit tests, pass -DUTEST_ENABLED=1 to CMake. Unit tests require the cmocka library.
Code coverage analysis is only available if unit tests are enabled. The feature is disabled by default. To enable code coverage analysis, pass -DCOVERAGE_ENABLED=1 to CMake.
With the help of the performance analysis infrastructure of pscom (perf code module) it is possible to calculate and print the CPU cycles and time spent on specific parts of the code. To enable this feature, pass -DPERF_ENABLED=1 to CMake.
To use this feature, add instructions perf_add("my_identifier"); to the code. For each perf_add instruction, pscom prints the CPU cycles/ time difference to the previous perf_add statement along with the provided identifier. To reset the time measurements, i.e., start again from 0.0, add a perf_add line with an identifier that has the prefix "reset_". All measurement results are printed per process in the atexit handler of pscom.
It is highly recommended to run CMake from a separate folder.
In the top level pscom directory, execute
$ mkdir build
$ cd buildExecute the following from within this newly created folder to compile pscom for the default installation path /opt/parastation:
$ cmake ..
$ makeCMake will report any missing dependencies. Check the output of CMake carefully.
Add -DCMAKE_INSTALL_PREFIX=custom/install/path as additional config parameter to the CMake line to change the installation path.
Finally, to install pscom in your system (superuser rights might be required depending on the install location), run
$ make installIf you have installed pscom to a custom location, you need to add this custom path to your system's environment variables so that other sofware - for example ParaStation MPI - can find and work with pscom.
$ export LIBRARY_PATH=pscom/install/dir/lib[64]:${LIBRARY_PATH}
$ export LD_LIBRARY_PATH=pscom/install/dir/lib[64]:${LD_LIBRARY_PATH}
$ export CPATH=pscom/install/dir/include:${CPATH}
$ export PATH=pscom/install/dir/bin:${PATH}To ensure that all commits conform to the coding style, the pre-commit hook should be activated. Therefore, you have to link this hook from the top-level source directory:
$ ln -s ../../scripts/hooks/pre-commit .git/hooks/pre-commitThis automatically runs clang-format, cmake-format and cmakelint on all changed files. Currently, we rely on clang-format in version 16.0.6, cmake-format in version 0.16.13 and cmakelint in version 1.4.2 for checking the coding style.
cmake-format configuration file is YAML file that needs PyYaml (our version 6.0.1) for parsing.
Additionally, we use "Include What You Use" to avoid upward dependencies. This can be enabled by passing -DIWYU_ENABLED=ON to CMake. Currently, we rely on include-what-you-use in version 0.22.