Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use RCDB to get the beam energy #256

Merged
merged 22 commits into from
Oct 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/install-dependency-packages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ GENERAL_PACKAGE_LIST_LINUX=(
openssl
gsl
davix
### RCDB dependencies
mariadb
)
IGUANA_PACKAGE_LIST_LINUX=(
fmt
Expand All @@ -56,6 +58,8 @@ GENERAL_PACKAGE_LIST_MACOS=(
libxext
openssl
gsl
### RCDB dependencies
mariadb
)
IGUANA_PACKAGE_LIST_MACOS=(
fmt
Expand Down Expand Up @@ -139,6 +143,8 @@ case $runner in
### link homebrew's gcc, for gfortran
brew unlink gcc
brew link gcc
### kluge ssl linker issue (see, e.g., https://github.com/brianmario/mysql2/issues/795)
echo "LIBRARY_PATH=${LIBRARY_PATH:+${LIBRARY_PATH}:}$(pkg-config libssl --variable libdir)" | tee -a $GITHUB_ENV
;;

*)
Expand Down
18 changes: 17 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -351,14 +351,28 @@ jobs:
- name: set ROOT environment
if: ${{ matrix.id != 'noROOT' }}
run: iguana_src/.github/source-ROOT.sh root
###### RCDB
- name: get RCDB
uses: actions/checkout@v4
with:
repository: JeffersonLab/rcdb
ref: main
path: rcdb_src
- name: set RCDB CI vars
id: rcdb
working-directory: rcdb_src
run: |
echo prefix=$(pwd) >> $GITHUB_OUTPUT
echo ver=$(git rev-parse HEAD) >> $GITHUB_OUTPUT
###### summarize dependencies
- name: summarize dependencies
if: ${{ matrix.id == 'cpp' }}
run: |
echo '### Dependencies' >> $GITHUB_STEP_SUMMARY
echo '| Dependency | Version |' >> $GITHUB_STEP_SUMMARY
echo '| --- | --- |' >> $GITHUB_STEP_SUMMARY
echo "| \`hipo\` | ${{ env.hipo_ref }} |" >> $GITHUB_STEP_SUMMARY
echo "| \`hipo\` | \`${{ env.hipo_ref }}\` |" >> $GITHUB_STEP_SUMMARY
echo "| \`rcdb\` | \`${{ steps.rcdb.outputs.ver }}\` |" >> $GITHUB_STEP_SUMMARY
echo "| \`root\` | $(root --version 2>&1 | head -n1) |" >> $GITHUB_STEP_SUMMARY
echo "| \`ruby\` | $(ruby --version) |" >> $GITHUB_STEP_SUMMARY
cat pkg_summary.md >> $GITHUB_STEP_SUMMARY
Expand All @@ -368,6 +382,8 @@ jobs:
meson setup iguana_build iguana_src \
--prefix=$(pwd)/iguana \
--pkg-config-path=$(pwd)/hipo/lib/pkgconfig \
-Drcdb:home=${{ steps.rcdb.outputs.prefix }} \
-Dz_require_rcdb=true \
-Dwerror=true \
-Dinstall_examples=true \
-Dtest_data_file=$(pwd)/test_data.hipo \
Expand Down
2 changes: 1 addition & 1 deletion doc/gen/meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,5 @@ doc_tgt = custom_target(
build_by_default: true,
command: [ prog_doxygen, doxyfile ],
install: true,
install_dir: join_paths(get_option('datadir'), 'doc', meson.project_name()),
install_dir: get_option('datadir') / 'doc' / meson.project_name(),
)
2 changes: 1 addition & 1 deletion doc/ifarm.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
```bash
meson/resolve-dependencies.py --help # prints the usage guide
```
Use the `--env` option and set the resulting environment variables.
Use the resulting arguments when setting up or configuring your build directory.

## Build and Install

Expand Down
23 changes: 18 additions & 5 deletions doc/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,15 @@ cmake --install build-hipo
- After installation, depending on ROOT's installation prefix you may also need to set your environment so
ROOT may be found; this is typically done by `source /path/to/root/bin/thisroot.sh`

### 🟩 Optional: `RCDB`: Run Condition Database
<https://github.com/JeffersonLab/rcdb>
- RCDB is optional, but needed for algorithms that use, _e.g._, the beam energy
- You do not need to compile RCDB, just clone the repository
- You may need to use the latest version on the main branch, rather than the most recent tag
- Iguana uses `mysql` for RCDB; you may need to install `mariadb` or `mysql` client on your system
- [`mariadb` is an open source fork of `mysql`](https://mariadb.org/)
- depending on your OS's packages, you may need the "dev" version, _e.g._, `libmariadb-dev`

<a name="building"></a>
## 🟠 Building and Installing

Expand All @@ -87,10 +96,9 @@ Use [`meson/resolve-dependencies.py`](../meson/resolve-dependencies.py) to help
/path/to/iguana-source/meson/resolve-dependencies.py --help # prints the usage guide
```
Tell it where your dependencies are installed and it will tell you the build options
that you need for Step 2; you can also choose to write those build options to an INI (native) file.
that you need for Step 2.

Alternatively, you may use environment variables; see the [note on dependency
resolution](dependency_resolution.md) for more general guidance.
See the [note on dependency resolution](dependency_resolution.md) for more general guidance.


### 🟩 Step 2: Generate a build directory
Expand Down Expand Up @@ -118,9 +126,11 @@ All build options, their current values, and their descriptions may be found by
meson configure # outputs in a pager (`less`); you may scroll, or press 'q' to quit
meson configure --no-pager # do not use a pager
```
**but that's a _lot_ of text!** The _most important_ build options are near the bottom, under **"Project options"**.
**but that's a _lot_ of text!** The _most important_ build options are under the **"Project options"**
sections: the first such section is for `iguana`, and the rest are for subprojects (_e.g._, `rcdb`).

To see _just_ the project options, run the following (which requires [`jq`](https://jqlang.github.io/jq/)):
```
```bash
/path/to/iguana-source/meson/dump-build-options.sh .
```

Expand All @@ -130,6 +140,9 @@ meson configure -Dinstall_examples=true
```
You can add as many `-D<option>=<value>` arguments as you need.

> [!NOTE]
> To set a subproject's option, you must prefix the subproject name. For example, for `rcdb`, use `-Drcdb:<option>=<value>`.

### 🟩 Step 4: Compile and Install
Now compile and install Iguana:
```bash
Expand Down
5 changes: 3 additions & 2 deletions examples/iguana_ex_fortran_01_action_functions.f
Original file line number Diff line number Diff line change
Expand Up @@ -190,9 +190,10 @@ program iguana_ex_fortran_01_action_functions
c 'key_vz_filter' must be passed to the action function;
call iguana_clas12_zvertexfilter_prepareevent(
& algo_vz_filter, runnum(1), key_vz_filter)
c similarly for the inclusive kinematics algorithm
c similarly for the inclusive kinematics algorithm; use '-1'
c for the beam energy, so RCDB is used to get the energy
call iguana_physics_inclusivekinematics_prepareevent(
& algo_inc_kin, runnum(1), key_inc_kin)
& algo_inc_kin, runnum(1), -1, key_inc_kin)

c call iguana filters
c - the `logical` variable `accept` must be initialized to
Expand Down
29 changes: 23 additions & 6 deletions meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ project(
'libdir': 'lib',
'licensedir': 'share/licenses/iguana',
'pkgconfig.relocatable': 'true',
'force_fallback_for': ['rcdb'],
},
version: run_command(
meson.project_source_root() / 'meson' / 'detect-version.sh',
Expand All @@ -27,6 +28,9 @@ use_chameleon = get_option('bind_fortran')
pkg = import('pkgconfig')
fs = import('fs')

# build subprojects
rcdb_subproj = subproject('rcdb', required: get_option('z_require_rcdb'))

# resolve dependencies
prog_minver = find_program('meson' / 'minimum-version.sh')
# NOTE: those that are typically installed by package managers should use `meson/minimum-version.sh`
Expand Down Expand Up @@ -56,13 +60,23 @@ prog_ruby = find_program(
version: '>=' + run_command(prog_minver, 'ruby', check: true).stdout().strip(),
required: use_chameleon,
)
rcdb_dep = dependency(
'rcdb',
fallback: ['rcdb', 'rcdb_dep'],
required: get_option('z_require_rcdb'),
)
thread_dep = dependency(
'threads',
required: true, # FIXME: actually, it's only needed for multi-threading tests/examples
)

# list of dependencies
# FIXME: for users which use LD_LIBRARY_PATH, we should try to keep this list
# ordered such that the ones users are *least likely* to try to build
# themselves are listed last (see FIXME in meson/this_iguana.sh.in)
# NOTE: omit ROOT (handled differently, and most ROOT users already have it in their environment)
dep_list = []
foreach dep : [ hipo_dep, fmt_dep, yamlcpp_dep, ROOT_dep ]
foreach dep : [ hipo_dep, fmt_dep, yamlcpp_dep, rcdb_dep ]
if dep.found()
dep_list += dep
endif
Expand All @@ -75,9 +89,6 @@ dep_pkgconfig_dirs = []
foreach dep : dep_list

# get library and include dirs
if dep.name() == 'ROOT'
continue # skip ROOT (handled differently, and most ROOT users already have it in their environment)
endif
if dep.type_name() == 'pkgconfig'
libdirs = [ dep.get_variable(pkgconfig: 'libdir') ]
incdirs = [ dep.get_variable(pkgconfig: 'includedir') ]
Expand All @@ -88,7 +99,13 @@ foreach dep : dep_list
endforeach
incdirs = ROOT_dep.get_variable(cmake: 'PACKAGE_INCLUDE_DIRS').split(';')
else
error('Cannot determine how dependency "' + dep.name() + '" was found')
name = dep.get_variable(internal: 'name', default_value: dep.name())
if name == 'rcdb'
incdirs = [ dep.get_variable(internal: 'includedir') ]
else
warning(f'Unknown dependency "@name@"')
continue
endif
endif

# append to `dep_*_dirs` arrays, uniquely
Expand Down Expand Up @@ -148,7 +165,7 @@ endif
# general project vars
project_inc = include_directories('src')
project_libs = []
project_deps = declare_dependency(dependencies: [ fmt_dep, yamlcpp_dep, hipo_dep ]) # do NOT include ROOT here
project_deps = declare_dependency(dependencies: dep_list) # do NOT include ROOT here
project_etc = get_option('sysconfdir') / meson.project_name()
project_test_env = environment()
project_pkg_vars = [
Expand Down
1 change: 1 addition & 0 deletions meson.options
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ option('test_multithreading', type: 'integer', value: 4, min: 0, descriptio
# expert options: the defaults should be reasonable for a local installation; different values may be preferred for installation in common areas
option('z_install_envfile', type: 'boolean', value: true, description: 'Install a sourceable environment variable file')
option('z_require_root', type: 'boolean', value: false, description: 'Fail if ROOT is not found')
option('z_require_rcdb', type: 'boolean', value: false, description: 'Fail if RCDB is not found')
1 change: 1 addition & 0 deletions meson/release/install-cvmfs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ msg "meson setup"
meson setup $buildDir $sourceDir \
--native-file=$nativeFile \
--prefix=$installDir \
-Drcdb:home=$RCDB_HOME \
-Dtest_data_file=$testFile
msg "meson install"
meson install -C $buildDir
Expand Down
103 changes: 40 additions & 63 deletions meson/resolve-dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,21 +22,20 @@ class Formatter(argparse.ArgumentDefaultsHelpFormatter, argparse.RawDescriptionH
parser_deps.add_argument('--fmt', default=SYSTEM_ASSUMPTION, type=str, help='path to `fmt` installation')
parser_deps.add_argument('--yaml', default=SYSTEM_ASSUMPTION, type=str, help='path to `yaml-cpp` installation')
parser_deps.add_argument('--root', default=SYSTEM_ASSUMPTION, type=str, help='path to `ROOT` installation')
parser_deps.add_argument('--rcdb', default='$RCDB_HOME', type=str, help='path to `RCDB` installation')
parser_output = parser.add_argument_group('output control')
parser_output.add_argument('--cli', default=False, action=argparse.BooleanOptionalAction, help='only print the `meson` CLI options, and nothing else')
parser_output.add_argument('--env', default=False, action=argparse.BooleanOptionalAction, help='generate environment variable `export` commands instead')
parser_output.add_argument('--ini', default=NOT_USED, type=str, help='if set, generate an INI file (meson native file) with this name; you may then use it with `meson setup --native-file=_____`')
parser_output.add_argument('--verbose', default=False, action=argparse.BooleanOptionalAction, help='verbose output')
args = parser.parse_args()

# verbosity
verbose = not args.cli
def print_verbose(message):
if(verbose):
if(args.verbose):
print(message)

# functions to set dependency paths
pkg_config_path = set()
cmake_prefix_path = set()
extra_args = []
def use_system(dep):
print_verbose(f'{dep}: {SYSTEM_ASSUMPTION}')
def use_pkg_config(dep, pc_file, arg):
Expand All @@ -63,71 +62,49 @@ def use_cmake(dep, arg):
cmake_prefix_path.add(path)
else:
use_system(dep)
def use_env_var(dep, build_var_name, user_val, env_var_name):
if(user_val == f'${env_var_name}'):
if env_var_name in os.environ:
print_verbose(f'{dep}: using environment variable "{env_var_name}" for build variable "{build_var_name}"')
extra_args.append([build_var_name, os.environ[env_var_name]])
else:
print(f'{dep}: you did not specify where {dep} is found, and the fallback environment variable "${env_var_name}" is not set; {dep} will be ignored', file=sys.stderr)
else:
print_verbose(f'{dep}: using user value "{user_val}" for build variable "{build_var_name}"')
extra_args.append([build_var_name, user_val])

# resolve dependencies #########################
use_pkg_config('hipo', 'hipo4.pc', args.hipo)
use_pkg_config('fmt', 'fmt.pc', args.fmt)
use_pkg_config('yaml', 'yaml-cpp.pc', args.yaml)
use_cmake('ROOT', args.root)
use_env_var('rcdb', 'rcdb:home', args.rcdb, 'RCDB_HOME')
################################################


# generate a native file
if(args.ini!=NOT_USED):
def ini_string_arr(arr):
contents = ','.join(map(lambda s: f'\'{s}\'', arr))
return f'[{contents}]'
ini_config = ConfigParser(allow_no_value=True)
ini_config.add_section('built-in options')
if(len(cmake_prefix_path) > 0):
ini_config.set('built-in options', 'cmake_prefix_path', ini_string_arr(cmake_prefix_path))
if(len(pkg_config_path) > 0):
ini_config.set('built-in options', 'pkg_config_path', ini_string_arr(pkg_config_path))
with open(args.ini, 'w') as fp:
ini_config.write(fp)

# generate CLI options
if(verbose or args.cli):
if(len(pkg_config_path)==0 and len(cmake_prefix_path)==0):
print_verbose(textwrap.dedent(f'''
==========================================================================================
All of your dependencies are assumed to be in the system default locations.
- If they are not, please run:
{sys.argv[0]} --help
- Otherwise, you do not need to set or modify any build options for dependency resolution.
==========================================================================================
'''))
exit(0)
else:
if(args.env):
print_verbose(textwrap.dedent('''
==================================================
| Here are the environment variables you need: |
==================================================
'''))
else:
print_verbose(textwrap.dedent('''
===============================================
| Here are the build options that you need: |
===============================================
'''))
cli_opts = []
if(len(pkg_config_path) > 0):
if(args.env):
cli_opts.append(f'export PKG_CONFIG_PATH={":".join(pkg_config_path)}' + '${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}')
else:
cli_opts.append(f'--pkg-config-path={",".join(pkg_config_path)}')
if(len(cmake_prefix_path) > 0):
if(args.env):
cli_opts.append(f'export CMAKE_PREFIX_PATH={":".join(cmake_prefix_path)}' + '${CMAKE_PREFIX_PATH:+:${CMAKE_PREFIX_PATH}}')
else:
cli_opts.append(f'--cmake-prefix-path={",".join(cmake_prefix_path)}')
if(args.ini==NOT_USED):
if(args.env):
for cli_opt in cli_opts:
print(cli_opt)
else:
print(f'{" ".join(cli_opts)}')
else:
print(f'--native-file={args.ini}')
print_verbose('\n')
if(len(pkg_config_path)==0 and len(cmake_prefix_path)==0 and len(extra_args)==0):
print_verbose(textwrap.dedent(f'''
==========================================================================================
All of your dependencies are assumed to be in the system default locations.
- If they are not, please run:
{sys.argv[0]} --help
- Otherwise, you do not need to set or modify any build options for dependency resolution.
==========================================================================================
'''))
else:
print_verbose(textwrap.dedent('''
===============================================
| Here are the build options that you need: |
===============================================
'''))
cli_opts = []
if(len(pkg_config_path) > 0):
cli_opts.append(f'--pkg-config-path={",".join(pkg_config_path)}')
if(len(cmake_prefix_path) > 0):
cli_opts.append(f'--cmake-prefix-path={",".join(cmake_prefix_path)}')
if(len(extra_args) > 0):
for extra_arg in extra_args:
cli_opts.append(f'-D{extra_arg[0]}={extra_arg[1]}')
print(f'{" ".join(cli_opts)}')
print_verbose('\n')
4 changes: 0 additions & 4 deletions src/iguana/algorithms/meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,6 @@ foreach algo : algo_dict
],
)
algo_bind_c_sources += tgt[0]
else
warning('Algorithm "' + algo_name + '" has no bindings')
endif

else
Expand All @@ -154,8 +152,6 @@ foreach algo : algo_dict
vdor_headers += algo_dir / src_file
endforeach
endif
else
warning('Algorithm "' + algo_name + '" has no validator')
endif

endforeach
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ actions:
inputs:
- name: runnum
type: int
- name: beam_energy
type: double
outputs:
- type: concurrent_key_t
cast: int
Expand Down
Loading
Loading