From bd0bff1e8b4dadb294d21bb7e8a5383395ff968f Mon Sep 17 00:00:00 2001 From: Ray Osborn Date: Mon, 23 Oct 2023 17:20:16 -0500 Subject: [PATCH] Add an experiment section --- docs/source/experiment.rst | 47 ++++++++++++++++++++++++++++++++++++ docs/source/index.rst | 1 + docs/source/installation.rst | 43 +++++++++++++++++++++++++++------ 3 files changed, 83 insertions(+), 8 deletions(-) create mode 100644 docs/source/experiment.rst diff --git a/docs/source/experiment.rst b/docs/source/experiment.rst new file mode 100644 index 0000000..783dd35 --- /dev/null +++ b/docs/source/experiment.rst @@ -0,0 +1,47 @@ +Experiment Setup +================ +Data that is processed by the NXRefine package are stored as HDF5 files +stored according to the `NeXus format `_, +which is an international standard for the storage of x-ray and neutron +scattering data. NXRefine uses a hierarchical folder structure to +store both the experimental data, ingested from the raw data generated +on an x-ray beamline, and processed data generated by the data reduction +workflow. Scripts and plugins to the NeXpy GUI are used to facilitate +the creation of both the folder hierarchy and the NeXus files +themselves. + +In the next section, we will describe the workflow used to both ingest +the raw data and transform it into reciprocal space coordinates and +pair-distribution-functions. In this section, we will describe the +framework, within which these processes function. + +Experiment Files +---------------- +NXRefine assumes that measurements on a set of samples can be grouped +together as a single experiment, typically scheduled within the same +time period. For example, measurements at synchrotron x-ray facilities +are often grouped by the proposal number used to acquire beam time, and +are therefore labelled by a proposal number and/or a run cycle. For +example, on beamline 6-ID-D at the Advanced Photon Source, measurements +resulting from Proposal No. GUP-75969 may be scheduled in a specific +run cycle, say 23-1, and stored in ``/data/GUP-75969-23-1``. + +Here is the structure of an experiment directory:: + + experiment + └── tasks + ├── nxdatabase.db + ├── settings.ini + └── calibrations + ├── CeO2_87keV_600mm.tiff + └── configurations + ├── exp_87keV_600mm.nxs + └── sample1 + └── label1 + ├── sample1_100K.nxs + ├── sample1_200K.nxs + └── sample1_300K.nxs + └── label2 + └── sample1_300K.nxs + ├── sample2 + ├── sample 3 diff --git a/docs/source/index.rst b/docs/source/index.rst index f912d20..e8e7ed7 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -35,6 +35,7 @@ temperature, are complete. introduction installation + experiment Indices and tables ================== diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 96df612..cb336fa 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -80,12 +80,6 @@ within the server directory. variable if it is defined. This allows system administrators to set up a unique lock file directory for all their users. -It is suggested that users add the following to their .bashrc file:: - - export NX_LOCKDIRECTORY=/path/to/parent/nxserver/locks - export NX_LOCK=10 - nxserver -d /path/to/parent/nxserver - Server Directory ^^^^^^^^^^^^^^^^ Here is the structure of the ``nxserver`` directory:: @@ -122,10 +116,43 @@ Here is the structure of the ``nxserver`` directory:: be described later), so that they can be customized if necessary. These settings are defined later. +* **nxsetup.sh** + + A shell script that could be used to initialize paths to the server + directory or environment variables used by NeXpy. This could be run + within a user's ``~/.bashrc`` file, or by other shell scripts used to + launch NXRefine workflow jobs (see below). Here is an example of what + this file could contain.:: + + export NX_LOCKDIRECTORY=/path/to/parent/nxserver/locks + export NX_LOCK=10 + nxserver -d /path/to/parent/nxserver + + Other commands, *e.g.*, to initialize a particular conda environment, + could be also be added to this file. + * **nxcommand.sh** - A shell script that may be used if jobs need to be wrapped before - submission to the job queue, *e.g.*, using ``qsub``. + A shell script that is used if jobs need to be wrapped before + submission to the job queue, *e.g.*, using ``qsub``. Here is an + example, in which ``nxsetup.sh`` is run in order to initialize + NXRefine.:: + + echo `date` "USER ${USER} JOB_ID ${JOB_ID}" + source /path/to/parent/nxserver/nxsetup.sh + + +* **task_list** + + A directory that contains files that implement a file-based FIFO + queuing system for server jobs. + +* **locks** + + A directory that contains files that implement the + `nexusformat `_ file-locking system. + Locked files can be viewed, and removed if they are stale, using the + ``Show File Locks`` dialog in the NeXpy ``File`` menu. .. note:: The log files can be viewed using the ``Manage Server`` dialog and the settings file can be modified using the ``Edit