Skip to content

Commit

Permalink
Add an experiment section
Browse files Browse the repository at this point in the history
  • Loading branch information
rayosborn committed Oct 23, 2023
1 parent d1d7c60 commit bd0bff1
Show file tree
Hide file tree
Showing 3 changed files with 83 additions and 8 deletions.
47 changes: 47 additions & 0 deletions docs/source/experiment.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
Experiment Setup
================
Data that is processed by the NXRefine package are stored as HDF5 files
stored according to the `NeXus format <http://www.nexusformat.org/>`_,
which is an international standard for the storage of x-ray and neutron
scattering data. NXRefine uses a hierarchical folder structure to
store both the experimental data, ingested from the raw data generated
on an x-ray beamline, and processed data generated by the data reduction
workflow. Scripts and plugins to the NeXpy GUI are used to facilitate
the creation of both the folder hierarchy and the NeXus files
themselves.

In the next section, we will describe the workflow used to both ingest
the raw data and transform it into reciprocal space coordinates and
pair-distribution-functions. In this section, we will describe the
framework, within which these processes function.

Experiment Files
----------------
NXRefine assumes that measurements on a set of samples can be grouped
together as a single experiment, typically scheduled within the same
time period. For example, measurements at synchrotron x-ray facilities
are often grouped by the proposal number used to acquire beam time, and
are therefore labelled by a proposal number and/or a run cycle. For
example, on beamline 6-ID-D at the Advanced Photon Source, measurements
resulting from Proposal No. GUP-75969 may be scheduled in a specific
run cycle, say 23-1, and stored in ``/data/GUP-75969-23-1``.

Here is the structure of an experiment directory::

experiment
└── tasks
├── nxdatabase.db
├── settings.ini
└── calibrations
├── CeO2_87keV_600mm.tiff
└── configurations
├── exp_87keV_600mm.nxs
└── sample1
└── label1
├── sample1_100K.nxs
├── sample1_200K.nxs
└── sample1_300K.nxs
└── label2
└── sample1_300K.nxs
├── sample2
├── sample 3
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ temperature, are complete.

introduction
installation
experiment

Indices and tables
==================
Expand Down
43 changes: 35 additions & 8 deletions docs/source/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -80,12 +80,6 @@ within the server directory.
variable if it is defined. This allows system administrators
to set up a unique lock file directory for all their users.

It is suggested that users add the following to their .bashrc file::

export NX_LOCKDIRECTORY=/path/to/parent/nxserver/locks
export NX_LOCK=10
nxserver -d /path/to/parent/nxserver

Server Directory
^^^^^^^^^^^^^^^^
Here is the structure of the ``nxserver`` directory::
Expand Down Expand Up @@ -122,10 +116,43 @@ Here is the structure of the ``nxserver`` directory::
be described later), so that they can be customized if necessary.
These settings are defined later.

* **nxsetup.sh**

A shell script that could be used to initialize paths to the server
directory or environment variables used by NeXpy. This could be run
within a user's ``~/.bashrc`` file, or by other shell scripts used to
launch NXRefine workflow jobs (see below). Here is an example of what
this file could contain.::

export NX_LOCKDIRECTORY=/path/to/parent/nxserver/locks
export NX_LOCK=10
nxserver -d /path/to/parent/nxserver

Other commands, *e.g.*, to initialize a particular conda environment,
could be also be added to this file.

* **nxcommand.sh**

A shell script that may be used if jobs need to be wrapped before
submission to the job queue, *e.g.*, using ``qsub``.
A shell script that is used if jobs need to be wrapped before
submission to the job queue, *e.g.*, using ``qsub``. Here is an
example, in which ``nxsetup.sh`` is run in order to initialize
NXRefine.::

echo `date` "USER ${USER} JOB_ID ${JOB_ID}"
source /path/to/parent/nxserver/nxsetup.sh
<NXSERVER>

* **task_list**

A directory that contains files that implement a file-based FIFO
queuing system for server jobs.

* **locks**

A directory that contains files that implement the
`nexusformat <https://nexpy.github.io/nexpy/>`_ file-locking system.
Locked files can be viewed, and removed if they are stale, using the
``Show File Locks`` dialog in the NeXpy ``File`` menu.

.. note:: The log files can be viewed using the ``Manage Server`` dialog
and the settings file can be modified using the ``Edit
Expand Down

0 comments on commit bd0bff1

Please sign in to comment.