Running your own client

The client requires the requests and DaemonLite python modules to run. Install them via pip or whatever installation method you prefer.

There are two main pieces of information you need to run your own client.

1. Server URL: For example https://civet.inl.gov
2. Build key: This is generated for you when you first sign in to Civet and will be displayed on the "recipes" page. This should not be shared.

Basic client

The basic client is client/client.py. It is intended to run via command line or cron. All parameters are put on the command line. You will also need to set and export the BUILD_ROOT environment variable to where you want all testing done. Other useful environment variables include:

• MOOSE_JOBS : Will cause make and run_tests to use -j $MOOSE_JOBS
• MAKE_JOBS : Will cause make to use -j $MAKE_JOBS. Overrides MOOSE_JOBS
• MAX_MAKE_LOAD : Will cause make to use -j $MAKE_JOBS -l $MAX_MAKE_LOAD
• TEST_JOBS : Will cause run_tests to use -j $TEST_JOBS. Overrides MOOSE_JOBS
• MAX_TEST_LOAD : Will cause run_tests to use -j $TEST_JOBS -l $MAX_TEST_LOAD

An example command line might be (executed in the client directory):

./client.py --url https://civet.inl.gov --build-key <Your assigned key> --configs linux-gnu --name <username>_client --insecure

If your recipes need a certain environment to be run in, for example, a certain set of modules loaded, or PATH to be set, it is usually better to set them in the terminal in which the client is run. Every step of a recipe is run in its own process and inherits the environment from the client. So if you need modules loaded, you could do a module purge followed by a module load ... in every step of your recipe or just set it once in the client terminal before launching the client.

INL client

The INL client is in client/inl_client.py and is what is used internally at INL. It runs as a daemon and has the ability to load different modules based on the build config. It can also poll more than one Civet server.

Settings

The inl_client.py is configured with the client/settings.py file. The default file explains the required variables. Note that this is a python file so you can use regular python expressions to build the variables. The required variables are:

• SERVERS: A list of servers to poll. Each entry in the list is a tuple ('<server URL>', '<build_key>', ssl_cert) where ssl_cert can be False to indicate that SSL verification is not done. Example:

  SERVERS = [ ('https://civet.inl.gov', '1234', False), ]

• CONFIG_MODULES: A dictionary of build configs to poll for. The keys are the build_configs to listen for, which should correspond to what is specified in your recipes. The dictionary values are lists of modules to load for the configuration. Example:

  CONFIG_MODULES = {'linux-gnu': ['moose-dev-gcc'],}

• ENVIRONMENT: A dictionary of environment variables. These will be inserted into the running client environment. Example:

  ENVIRONMENT = { "MAKE_JOBS": "16", }

• NUM_CLIENTS: Number of clients to start.

Note that by default the inl_client.py sets BUILD_ROOT to $CIVET_HOME/build_<client_number>. If CIVET_HOME is not set, the default is $HOME/civet. The client does not create BUILD_ROOT (although it could be created in the recipes) so it is a good idea to create it before running the client.

After configuring client/settings.py you could start the client with:

./inl_client.py --client 0 --daemon start

To quit the client:

./inl_client.py --client 0 --daemon stop

Controlling the client

Internally we use a script similar to client/control.sh to start up the inl_client.py on each build machine. The main feature of this is the ability the "gracefully" shutdown a client. This stops the client after it has finished whatever job it is currently running.