diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5754621e..970fc90c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -342,6 +342,13 @@ jobs: echo '' >> $GITHUB_STEP_SUMMARY echo '- for details, see the `coverage-report` artifact' >> $GITHUB_STEP_SUMMARY echo '- to compare to the report from the `main` branch, see ' >> $GITHUB_STEP_SUMMARY + ### test relocatability + - name: test relocatability + run: | + mv iguana relocated + source relocated/bin/this_iguana.sh --verbose # do not use --githubCI option, since we want this environment to be for only this step + relocated/bin/iguana-example-basic test_data.hipo ${{ env.num_events }} + mv relocated iguana ### set iguana environment, since the next steps will check the iguana installation - name: set iguana environment run: source iguana/bin/this_iguana.sh --verbose --githubCI diff --git a/README.md b/README.md index f3335226..d93e8936 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ Iguana is not a framework for _reading_ data, rather it is a set of algorithms t ### For Users 1. [Setup Guide](doc/setup.md) 1. [Examples](examples/README.md) +1. [Configuring the Algorithms](doc/configuration.md) 1. [Troubleshooting](doc/troubleshooting.md) 1. [API documentation](https://jeffersonlab.github.io/iguana/doxygen) diff --git a/doc/configuration.md b/doc/configuration.md new file mode 100644 index 00000000..9592ff69 --- /dev/null +++ b/doc/configuration.md @@ -0,0 +1,58 @@ +# Configuring the Algorithms + +Most of the algorithms are configurable using a YAML configuration file. If you are using Iguana for your analysis, you likely want to customize the algorithm configurations. + +The default configuration is installed in the `etc/` subdirectory of the Iguana installation. If you have set the Iguana environment variables using, _e.g._ `source this_iguana.sh`, or if you are using the version of Iguana installed on `ifarm`, you will have the environment variable `$IGUANA_CONFIG_PATH` set to include this `etc/` directory. + +There are a few ways to configure the algorithms; see the sections below for the options + +> [!IMPORTANT] +> While algorithm developers are encouraged _not_ to make breaking changes to their algorithms or configuration, in some cases certain changes cannot be prevented. Thus if you have your own algorithm configurations, you may want to keep up-to-date on any changes of the algorithm. We will try to announce all breaking changes in the Iguana release notes. + +> [!NOTE] +> If the Iguana installation is relocated, the environment variable `$IGUANA_CONFIG_PATH` _must_ be used at runtime. + +## Option 1: Copy the default directory, and modify + +First, copy the default configuration directory to your work area, or to your analysis code source tree; we'll call the copied directory `my_iguana_config`, as an example. If `$IGUANA_CONFIG_PATH` is the default configuration directory (_i.e._ you have not set or modified this variable yourself), you may run: +```bash +cp -r $IGUANA_CONFIG_PATH my_iguana_config +``` + +You may then freely modify any configuration file within `my_iguana_config/`. To use this directory in your algorithms, you may do any one of the following: + +1. Since `$IGUANA_CONFIG_PATH` allows multiple paths, delimited by colons (`:`), prepend `my_iguana_config/` to `$IGUANA_CONFIG_PATH`; the safest way is to use an absolute path: +```bash +export IGUANA_CONFIG_PATH=`pwd`/my_iguana_config:$IGUANA_CONFIG_PATH # bash or zsh +setenv IGUANA_CONFIG_PATH `pwd`/my_iguana_config:$IGUANA_CONFIG_PATH # tcsh or csh +``` +The algorithms will then search `my_iguana_config` for the configuration before searching the default paths. You may add multiple paths, if needed. +Paths which appear first in `$IGUANA_CONFIG_PATH` will be prioritized when the algorithm searches for configuration parameters; this behavior is similar to that of `$PATH` or `$LD_LIBRARY_PATH`. + +2. Use `Algorithm::SetConfigDirectory` instead of prepending `$IGUANA_CONFIG_PATH`. + +## Option 2: Write your own YAML file + +Make a new YAML file, containing just the options you want to override; use `Algorithm::SetConfigFile` to use this file for each algorithm. +See existing algorithms' configuration, and just copy what you need into your own YAML file; be mindful of the indentation, and that the top-level set of YAML nodes are the algorithm names. For example: + +Algorithm default configuration files: +```yaml +### default configuration file 1 +physics::AlgorithmA + cuts: [-1, 1] +``` +```yaml +### default configuration file 2 +physics::AlgorithmB + value: 3 +``` +Custom YAML file, widening `AlgorithmA`'s `cuts` and increasing `AlgorithmB`'s `value`: +```yaml +### custom YAML file +physics::AlgorithmA + cuts: [-2, 2] + +physics::AlgorithmB + value: 5 +``` diff --git a/doc/setup.md b/doc/setup.md index 434509df..74d1120d 100644 --- a/doc/setup.md +++ b/doc/setup.md @@ -162,4 +162,5 @@ The following environment variables are set or modified; not all of them are nee | `LD_LIBRARY_PATH` (Linux) or `DYLD_LIBRARY_PATH` (macOS) | adds paths to dependency and Iguana libraries | | `PYTHONPATH` | adds paths to dependency and Iguana Python packages, if Python bindings are installed | | `ROOT_INCLUDE_PATH` | adds paths to dependency and Iguana header files, for usage in ROOT | +| `IGUANA_CONFIG_PATH` | path to iguana algorithm configuration files (`.yaml`); users may override this with their own path; multiple paths may be specified, delimited by colons (`:`), where paths listed first will override paths listed later (similar behavior as `$PATH`); this variable is _only_ necessary if the Iguana installation has been relocated | | `IGUANA` | the path to the Iguana installation prefix, equivalent to `pkg-config iguana --variable prefix`; this is only for consumers that do not use `pkg-config` or the other standard environment variables, however usage of this variable is _discouraged_ since the installation layout may vary | diff --git a/examples/iguana-example-basic.cc b/examples/iguana-example-basic.cc index ea3039ea..c70484f9 100644 --- a/examples/iguana-example-basic.cc +++ b/examples/iguana-example-basic.cc @@ -57,7 +57,7 @@ int main(int argc, char** argv) seq.SetOption("clas12::EventBuilderFilter", "log", "debug"); seq.SetOption("clas12::MomentumCorrection", "log", "debug"); - // set algorithm options + // set algorithm options (overrides configuration files) seq.SetOption>("clas12::EventBuilderFilter", "pids", {11, 211, -211}); // start the algorithms diff --git a/meson.build b/meson.build index fc03f287..dae9a4d0 100644 --- a/meson.build +++ b/meson.build @@ -171,7 +171,7 @@ project_test_env.set( # set preprocessor macros add_project_arguments( - '-DIGUANA_ETC="' + get_option('prefix') / project_etc + '"', + '-DIGUANA_ETCDIR="' + get_option('prefix') / project_etc + '"', language: ['cpp'], ) @@ -218,6 +218,7 @@ if get_option('z_install_envfile') 'python': get_option('bind_python') ? 'true' : 'false', 'libdir': get_option('libdir'), 'root': ROOT_dep.found() ? 'true' : 'false', + 'etcdir': project_etc, }, ) foreach shell : ['csh', 'tcsh'] diff --git a/meson/this_iguana.sh.in b/meson/this_iguana.sh.in index d006e205..d9339652 100644 --- a/meson/this_iguana.sh.in +++ b/meson/this_iguana.sh.in @@ -82,6 +82,9 @@ if @root@; then unset var fi +# set config file path; append, rather than prepend, in case the user already has their own +export IGUANA_CONFIG_PATH=${IGUANA_CONFIG_PATH:+${IGUANA_CONFIG_PATH}:}$IGUANA/@etcdir@ + # print environment variables if $arg_verbose; then print_var() { echo "$@" | sed 's/:/\n /g'; } @@ -107,6 +110,9 @@ ROOT_INCLUDE_PATH -- $(if @root@; then echo 'ADDED iguana and dependency include PYTHONPATH -- $(if @python@; then echo 'ADDED iguana python bindings:'; else echo 'has NOT been changed'; fi) $(print_var ${PYTHONPATH-}) +IGUANA_CONFIG_PATH -- ADDED iguana algorithms' default configuration directory + $(print_var ${IGUANA_CONFIG_PATH-}) + ---------------------------- """ unset -f print_var @@ -117,6 +123,7 @@ if $arg_githubCI; then echo PKG_CONFIG_PATH=$PKG_CONFIG_PATH >> $GITHUB_ENV echo PATH=$PATH >> $GITHUB_ENV echo @ld_path@=$@ld_path@ >> $GITHUB_ENV + echo IGUANA_CONFIG_PATH=$IGUANA_CONFIG_PATH >> $GITHUB_ENV if @python@; then echo PYTHONPATH=$PYTHONPATH >> $GITHUB_ENV fi diff --git a/src/iguana/services/ConfigFileReader.cc b/src/iguana/services/ConfigFileReader.cc index 51035a9d..a97fd0b9 100644 --- a/src/iguana/services/ConfigFileReader.cc +++ b/src/iguana/services/ConfigFileReader.cc @@ -1,18 +1,39 @@ #include "ConfigFileReader.h" #include +#include +#include namespace iguana { ConfigFileReader::ConfigFileReader(std::string_view name) : Object(name) { - // add config files from installation prefix + // the algorithms need to know where the config files are; + // first, add config files from installation prefix; since this + // is added first, it's the lowest priority in the configuration + // search path AddDirectory(GetConfigInstallationPrefix()); + // next, add `IGUANA_CONFIG_PATH` paths, which provides: + // - user override of the configuration path(s) + // - a fallback, if `GetConfigInstallationPrefix` is wrong, which happens + // if the Iguana installation is relocated + auto user_paths_env_var = std::getenv("IGUANA_CONFIG_PATH"); + if(user_paths_env_var != nullptr) { + std::istringstream user_paths_stream(user_paths_env_var); + std::string user_path_token; + decltype(m_directories) user_paths; + while(getline(user_paths_stream, user_path_token, ':')) // tokenize `IGUANA_CONFIG_PATH` + user_paths.push_front(user_path_token); + for(auto const& user_path : user_paths) // add the paths in the correct order + AddDirectory(user_path); + // after these default paths have been added, the user + // may still override by calling `AddDirectory` etc. themselves + } } std::string ConfigFileReader::GetConfigInstallationPrefix() { - return IGUANA_ETC; + return IGUANA_ETCDIR; } void ConfigFileReader::AddDirectory(std::string const& dir) diff --git a/src/iguana/services/ConfigFileReader.h b/src/iguana/services/ConfigFileReader.h index 4992a44e..6df04ad5 100644 --- a/src/iguana/services/ConfigFileReader.h +++ b/src/iguana/services/ConfigFileReader.h @@ -15,6 +15,9 @@ namespace iguana { ConfigFileReader(std::string_view name = "config"); /// Get the config files' _fixed_ installation prefix + /// @warning if the Iguana installation is _relocated_, this directory will **not** be correct, + /// since it is compiled in the shared library; as a fallback, you may use the environment variable + /// `$IGUANA_CONFIG_PATH`. /// @return the absolute path to the installed configuration file directory static std::string GetConfigInstallationPrefix();