From f582f23957ff47fad2003136a1342c9137dcbbdf Mon Sep 17 00:00:00 2001 From: Pete Jemian Date: Mon, 5 Nov 2018 17:46:52 -0600 Subject: [PATCH 1/6] #634 NXdata --- base_classes/NXdata.nxdl.xml | 36 ++++++++++-------------------------- 1 file changed, 10 insertions(+), 26 deletions(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 4d869ef94d..753934a9c7 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -266,11 +266,14 @@ - Errors (uncertainties) associated with axis ``VARIABLE``. + "Errors" (actually *uncertainties*) associated with axis ``VARIABLE``. Client is responsible for defining the dimensions of the data. The name of this field may be changed to fit the circumstances - but is matched with the *VARIABLE* - field with ``_errors`` appended. + but is matched with the *VARIABLE* field with ``_errors`` appended. + + This pattern of using ``VARIABLE_errors`` can be used + throughout a NeXus data file to associate uncertainties + with a dataset named ``VARIABLE``. @@ -321,29 +324,10 @@ Do not use the ``axes`` attribute with the ``axis`` attribute. - - - - Specify the name (or names) of the uncertainties (errors) - of the dependent axes as plottable data. - NOTE: The ``uncertainties`` attribute uses the same syntax - as the ``axes`` attribute, a string or an array of strings - for multiple uncertainties. - - Examples:: - - @uncertainties="data_errors" - - @uncertainties="Idev" - - @uncertainties=["dQw", "dQl"] - - - + data label From a30df1f18040784ce3234b2a76a94d5efdceb9dc Mon Sep 17 00:00:00 2001 From: Pete Jemian Date: Mon, 5 Nov 2018 17:47:57 -0600 Subject: [PATCH 2/6] #634 --- base_classes/NXdata.nxdl.xml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 753934a9c7..d74e757ed3 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -273,7 +273,9 @@ This pattern of using ``VARIABLE_errors`` can be used throughout a NeXus data file to associate uncertainties - with a dataset named ``VARIABLE``. + with a dataset named ``VARIABLE``. This pattern also + applies to other relationships such as ``VARIABLE_resolutions`` + to connect additional data with a certain dataset. From b64bb10d80d1b4ee691b2b0ea75bd9574add6c93 Mon Sep 17 00:00:00 2001 From: koennecke Date: Thu, 23 Jan 2020 11:21:53 +0100 Subject: [PATCH 3/6] Updated data rules to contain the NIAC decision on errors --- manual/source/datarules.rst | 56 +++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/manual/source/datarules.rst b/manual/source/datarules.rst index 4a2e54c3c4..f87ce643ac 100644 --- a/manual/source/datarules.rst +++ b/manual/source/datarules.rst @@ -119,6 +119,62 @@ value is stored with a name built by appending _set to the normal NeXus field na The temperature field will hold the readback from the cryostat/furnace/whatever. The field temperature_set will hold the target value for the temperature as set by the experiment control software. + +.. _Design-Variants: + + +Variants +######### + +Sometimes it is necessary to store alternate values of a NeXus dataset +in a NeXus file. A common example may be the beam center of which a +rough value is available at data acquisition. But later on, a better beam +center is calculated as part of the data reduction. In order to store +this without losing the historical information, the original field can be given a variant attribute that points to +a new dataset containing the obsolete value. If even better values +become available, further datasets can be inserted into the chain of variant attributes +pointing to the preceeding value for the dataset. A reader can thus +keep the best value in the pre-defined dataset, and also be able to +follow the variant chain and locate older variants. + +A little example is in order to illustrate the scheme: + +.. code-block:: text + + beam_center_x + @variant=beam_center_x_refined + beam_center_x_refined + @variant=beam_center_x_initial_guess + beam_center_x_initial_guess + +NeXus borrowed this scheme from CIF. In this way all the different +variants of a dataset can be preserved. The expectation is that +variants will be rarely used and NXprocess groups with the results of +data reduction will be written instead. + + +Uncertainties or Errors +######################## + +It is desirable to store experimental errors together with the data. NeXus supports this through +a convention: uncertainties or experimental errors on data are stored in a separate field which has a name +consisting of the original name of the data with _errors appended to it. These uncertainties fields have the +sampe shape as the original data field. + +An example, from NXdetector: + +.. code-block:: text + data + data_errors + beam_center_x + beam_center_x_errors + +Where data errors would contain the erros on data, and beam_center_x_errors the error on +the beam center for x. + + +-- _Design-Uncertainties: + .. _Design-ArrayStorageOrder: NeXus Array Storage Order From 6e6d4cf58c9abc3ec88eea9144142d94bec3b225 Mon Sep 17 00:00:00 2001 From: Pete Jemian Date: Thu, 23 Jan 2020 12:53:51 -0600 Subject: [PATCH 4/6] DOC #721 describe reserved field suffixes --- manual/source/datarules.rst | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/manual/source/datarules.rst b/manual/source/datarules.rst index f87ce643ac..67a9e04f05 100644 --- a/manual/source/datarules.rst +++ b/manual/source/datarules.rst @@ -80,6 +80,7 @@ described by the following :index:`rules `: will be flagged as a warning during data file validation. +.. _use-underscore: .. rubric:: Use of underscore in descriptive names @@ -105,6 +106,7 @@ adequately documented. Terms with other names are permitted but might not be recognized by standard software. Rather than persist in using names not specified in the standard, please suggest additions to the :ref:`NIAC`. +.. _target_value: The data stored in NeXus fields must be readback values. This means values as read from the detector, other hardware etc. There are occasions where it is sensible to store the target value the variable was supposed to have. In this case the target @@ -119,6 +121,25 @@ value is stored with a name built by appending _set to the normal NeXus field na The temperature field will hold the readback from the cryostat/furnace/whatever. The field temperature_set will hold the target value for the temperature as set by the experiment control software. +.. _reserved_suffixes: + +.. rubric:: Reserved field name suffixes + +When naming a field (or dataset), NeXus has reserved certain suffixes to the names +so that a specific meaning may be attached. Consider a field named ``DATASET``, +the following table lists the suffixes reserved by NeXus. + +================= ========================= ================================= +suffix reference meaning +================= ========================= ================================= +`_axes` :ref:`NXdata` (!NXDL not yet changed from @axes!) String array naming data fields for each axis of ``DATASET`` +`_end` :ref:`NXtransformations` end points of the motions that start with ``DATASET`` +`_errors` :ref:`NXdata` uncertainties (a.k.a., errors) +`_increment_set` :ref:`NXtransformations` intended average range through which the corresponding axis moves during the exposure of a frame +`_indices` :ref:`NXdata` Integer array that defines the indices of the signal field which need to be used in the ``DATASET`` in order to reference the corresponding axis value +`_set` :ref:`target_value` Target value of ``DATASET`` +================= ========================= ================================= + .. _Design-Variants: From ba5a141cf497e44f72a9203ba674b3a84bc03721 Mon Sep 17 00:00:00 2001 From: Ben Watts Date: Thu, 23 Jan 2020 20:40:12 +0100 Subject: [PATCH 5/6] remove "_axes" from table Moving "axes" from an attribute to a fieldname suffix potentially follows from the NIAC decision about uncertainties, but it hasn't been explicitly discussed by the NIAC yet and so shouldn't be stated in the documentation at this time. --- manual/source/datarules.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/manual/source/datarules.rst b/manual/source/datarules.rst index 67a9e04f05..a7f91111b4 100644 --- a/manual/source/datarules.rst +++ b/manual/source/datarules.rst @@ -132,7 +132,6 @@ the following table lists the suffixes reserved by NeXus. ================= ========================= ================================= suffix reference meaning ================= ========================= ================================= -`_axes` :ref:`NXdata` (!NXDL not yet changed from @axes!) String array naming data fields for each axis of ``DATASET`` `_end` :ref:`NXtransformations` end points of the motions that start with ``DATASET`` `_errors` :ref:`NXdata` uncertainties (a.k.a., errors) `_increment_set` :ref:`NXtransformations` intended average range through which the corresponding axis moves during the exposure of a frame @@ -140,6 +139,10 @@ suffix reference meaning `_set` :ref:`target_value` Target value of ``DATASET`` ================= ========================= ================================= +.. Note that the following line might be added to the above table pending discussion: + + `_axes` :ref:`NXdata` String array naming data fields for each axis of ``DATASET`` + .. _Design-Variants: From b58410cf35a516bd2fd5e8c69f5d7b63fe169efb Mon Sep 17 00:00:00 2001 From: Pete Jemian Date: Thu, 23 Jan 2020 15:33:39 -0600 Subject: [PATCH 6/6] TST #634 GitHub API creds changed --- utils/dev_create_release_notes.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils/dev_create_release_notes.py b/utils/dev_create_release_notes.py index 8c1ed43d4f..a868d7e600 100755 --- a/utils/dev_create_release_notes.py +++ b/utils/dev_create_release_notes.py @@ -5,4 +5,4 @@ ''' from create_release_notes import main -main("v3.3", head="master", milestone="NXDL 2018.5") +main("NXDL 2018.5", head="master", milestone="NXDL 2020.1")