-
Notifications
You must be signed in to change notification settings - Fork 14
Running your own client
The client requires the requests
and DaemonLite
python modules to run. Install them via pip
or
whatever installation method you prefer.
There are two main pieces of information you need to run your own client.
- Server URL: For example https://www.moosebuild.org
- Build key: This is generated for you when you first sign in to Civet and will be displayed on the "recipes" page. This should not be shared.
The basic client is client/client.py
. It is intended to run via command line or cron. All
parameters are put on the command line. You will also need to set and export the BUILD_ROOT
environment variable to where you want all testing done.
Other useful environment variables include:
-
MOOSE_JOBS
: Will causemake
andrun_tests
to use-j $MOOSE_JOBS
-
MAKE_JOBS
: Will causemake
to use-j $MAKE_JOBS
. OverridesMOOSE_JOBS
-
MAX_MAKE_LOAD
: Will causemake
to use-j $MAKE_JOBS -l $MAX_MAKE_LOAD
-
TEST_JOBS
: Will causerun_tests
to use-j $TEST_JOBS
. OverridesMOOSE_JOBS
-
MAX_TEST_LOAD
: Will causerun_tests
to use-j $TEST_JOBS -l $MAX_TEST_LOAD
An example command line might be (executed in the client
directory):
./client.py --url https://www.moosebuild.org --build-key <Your assigned key> --configs linux-gnu --name <username>_client --insecure
If your recipes need a certain environment to be run in, for example, a certain set of modules loaded, or PATH
to be set, it is usually better to set them in the terminal in which the client is run. Every step of a recipe is run in its own process and inherits the environment from the client. So if you need modules loaded, you could do a module purge
followed by a module load ...
in every step of your recipe or just set it once in the client terminal before launching the client.
The INL client is in client/inl_client.py
and is what is used internally at INL. It runs as a daemon and has the ability
to load different modules based on the build config. It can also poll more than one Civet server.
The inl_client.py
is configured with the client/settings.py
file. The default file explains the required variables. Note that this is a python file so you can use regular python expressions to build the variables. The required variables are:
-
SERVERS
: A list of servers to poll. Each entry in the list is a tuple('<server URL>', '<build_key>', ssl_cert)
wheressl_cert
can beFalse
to indicate that SSL verification is not done. Example:SERVERS = [ ('https://www.moosebuild.org', '1234', False), ]
-
CONFIG_MODULES
: A dictionary of build configs to poll for. The keys are thebuild_configs
to listen for, which should correspond to what is specified in your recipes. The dictionary values are lists of modules to load for the configuration. Example:CONFIG_MODULES = {'linux-gnu': ['moose-dev-gcc'],}
-
ENVIRONMENT
: A dictionary of environment variables. These will be inserted into the running client environment. Example:ENVIRONMENT = { "MAKE_JOBS": "16", }
-
NUM_CLIENTS
: Number of clients to start.
Note that the inl_client.py
hard codes the BUILD_ROOT
to $HOME/civet/client_root_<client number>
.
After configuring client/settings.py
you could start the client with:
./inl_client.py --client 0 --daemon start
To quit the client:
./inl_client.py --client 0 --daemon stop
Internally we use client/control.py
to start up the inl_client.py
on each build machine. The main feature of this is the ability the "gracefully" shutdown a client. This stops the client after it has finished whatever job it is currently running.