adds support for configuring hardware objects with YAML files #1062
Conversation
…uredObject now cleaned
The method self.name() is now a property self.name. Also, use the f-string for generatng log messages, for a more modern look and feel of the code.
Behave in the same way as when loading XML configured HWO. The 'hardware_objects' dictionary is for example used by mxcubeweb adapters layer. If the HWO is not listed there, the adaptor for it will not load.
Adds an optional argument 'yaml_export_directory' to HardwareRepository.init_hardware_repository() function. When argument is provided, the HWO YAML config files will be written to the specified directory. The aim of this feature is assist in migrating beamline XML configuration to YAML style.
When loading HWOBJ with XML config file, populate its _config attribute _before_ calling init() method. This way, the self.config attribute can be used inside init() method for both YAML and XML configured hardware objects. Before this change, self.config was only available for YAML configured objects. For XML configured objects, the self.config was 'None'.
Only set 'msg0' variable when loading the 'beamline_config.yml' file. Otherwise the call to HWOBJ.init() will be skipped on line 225, if it have any sub-HWOBJs configured.
Update Session HWOBJ to access its configuration using 'self.config' attribute and get_property() method. Drops unused 'template' and 'suffix' attributes. Converts 'synchrotron_name', 'beamline_name' and 'endstation_name' to be proxy attributes for fields in 'self.config', this way there no need copy values. Replace self["foo"] calls, they are not supported when YAML config file is used.
Update and exapand documentation of YAML configuration files. - remove outdated information - update names of changed identifiers - add more details and some examples Co-authored-by: fabcor <[email protected]>
When warning that get_object_by_role() is deprecated, suggest using an attribute instead. Using get_property() to fetch objects does not work.
Adds a documentation section 'YAML configuration migration', with details on how to migrate to YAML configration. Co-authored-by: fabcor <[email protected]>
Make it possible to use get_object_by_role() on HWOBJs loaded with YAML config files, at least in some situations. This hack will make it easier to migrate to using YAML config files, as not all code invoking get_object_by_role() needs to be updated stright away.
Fix and issue where MXCuBE would not start if some of the specified YAML files could not be found. With this change, the behavior is the same as with missing XML files. An error message for the HWOBJ is added to the summary table, and the loading process continues.
When loading hardware objects from YAML configuration file, check if that file have been loaded earlier. Raise an error if we detect that same file is being reloaded. The aim is to enforce strict tree-like structure of hardware objects. This to avoid confusion of the situations when same hardware object is attached at different points.
Add a 'acquisition_limit_values' proxy attribute to the Beamline HWOBJ. This way the limits can be accessed with: HWR.beamline.acquisition_limit_values This is how MXCuBE-web reads the limits. Let's support this style for a while, for backward compability reason.
Update AbstractDetector class to access it's 'beam' setting with get_property(). Replaces self["beam"] expression, as it does not work when YAML config file is used.
Change code implementing Beam and Slits HWOBJs to work both with YAML and XML configure files. Add 'name' named attribute to __init__(), as it's required when loading from YAML. Renamed '_aperture', '_definer' and '_slits' attributes to 'aperture', 'definer' and 'slits', as this is the new style for attaching child HWOBJs. Added a backward compability hack, so that child HWOBJs attributes get populated when loading from XML configure file.
Drop proxy attributes 'kappa' and 'kappa_phi' from GenericDiffractometer class. These interfere with attaching of sub-HWOBJs for diffractometer HWOBJ. Add named parameter 'name' to DiffractometerMockup.__init__(). This parameter is required when loading HWOBJ from YAML configuration file.
Make it possible to create SampleChangerMockup and AbstractSampleChanger derived HWOBJs with ClassName(name=foo) expressions. This is the expression used when loading from a YAML configuration file.
The 'camera' attribute is now automagically set from the 'objects' settings in YAML/XML configure file.
The print_log() method moved from HardwareObjectNode class to ConfiguredObject, update the test accordingly.
The HardwareObjectNode.objects_names() method have been renamed to HardwareObjectNode._objects_names(), update the test accordingly.
* the HardwareObjectYaml does not take 'name' named attribute anymore * method 'name()' has changed to attribute 'name'
The 'mock_procedure' HWOBJ is attached directly to the beamline object.
Use the new YAML configuration file format for the files used by the tests.
The 'name' attribute have been replaced by 'id'.
Update fixture code of MAXIV MachInfo HWOBJ tests. Replace calls to set_property() with direct creation of _config attribute. Using set_property() does not work anymore.
Make it possible to create XRFMockup and CatsMaintMockup derived HWOBJs with ClassName(name=foo) expressions. This is the expression used when loading from a YAML configuration file.
Adds a section on removed set_property() method to 'YAML configuration migration' guide.
Don't use [] expression to read config properties. Renamed attributes self.cameraAxes and self.gonioAxes, to self._camera_axes and self._gonio_axes, as they now collide with the 'auto created' properties on the ConfiguredObject superclass. Use calls to self.get_property() when populating these attributes.
When 'export yaml configuration' option is invoked, write configurations to '*.yaml' files. This is the cannonical file extension for YAML files.
Adds section '<tangoname> tag no longer supported' to 'XML to YAML config migration document'.
This way, it's possible to link to generated API documentation for CommandContainer and ChannelObject classes. Without this change, sphix refuses to create a link for CommandContainer and ChannelObject classes, from other section of the documentation.
elmjag
changed the title
|
@elmjag, nice coincidence, I just sent you an email regarding this. As I mentioned in the mail it looks very well overall I had some question about the |
| @@ -482,15 +482,15 @@ def takeSnapshot(self, *args, **kwargs): | |||
| # img.save(*args) | |||
| except Exception: | |||
| logging.getLogger("HWR").exception( | |||
| "%s: could not save snapshot", self.name() | |||
| "%s: could not save snapshot", self.id | |||
There was a problem hiding this comment.
I was wondering where the id attribute gets assigned and I noticed that .name() is sometimes replaced by .name and sometimes with id
|
At the same time as Elmir made this PR I also sent en email regarding the same topic. After an exchange with Elmir we decided that it was better to extract my comments from the mail and share them here: Objects and Configuration: I was wondering if we should avoid assignment at initialization of the roles in the objects section to a "role" attribute of the object. I think that this "implicit" assignment may lead to confusion and cause issues with existing code. We could perhaps add an attribute that contains the objects just like we have In the context of validation both the Commands and channels: I'm looking through the last changes and its looking good as far as I can see. I noticed few nitty gritty things noted below.
I'm not very familiar with EPICS but what you suggest looks reasonable and it will work nicely with Tango and Exporter. These are just a few details, its looking really good overall ! |
docs/source/dev/commands_channels.md
Outdated
|
|
||
| ```yaml | ||
| class: <hardware-object-class> | ||
| tango: |
|
I do not understand what you mean when you say that As I understand it, the roles are an integral part of the class structure, and defined in the code. That makes them reliable and makes it possible for linters to check for uses of missing roles. Using Pydantic instead would make a lot of sense if you wanted the roles to be variable and locally configurable, or generally if you wanted it to be easier to change them around. But personally, that is exactly what I would not want. What is the advantage, as you see it, of defining roles in Pydantic instead of simply hard-coding them in? If it is the ability to have additional roles in site-specific code, could we not instead add those attributes in site-specific subclasses?. |
I would be alright with that as well, its less implicit and the result would be very similar. I don't see an issue with this If these attributes are defined before hand and an error is raised if the attribute does not exist. Both would lead to something fairly well defined. Adding "public" attributes to an HardwareObject automatically as a part of configuration otherwise IMO has to be a very special case. Isolating these these attributes in a object specifically for that purpose would then be a better alternative. We risk to overwrite attributes the author meant to use for other things if its not done without warning or error. But I'm as I mentioned in the beginning fine with this if the attribute (and preferably its type) is defined on the HardwareObject before the configuration step is done. |
elmjag
force-pushed
the
xml_yaml_conversion
branch
from
04f24cd to
4219954
Compare
Adds a section the describes how to configure hardware object's commands and channels using YAML configuration files. Documents the format for Tango, exporter and EPICS protocols.
Adds support for 'tango', 'exporter' and 'epics' sections to YAML configuration files. These section are used to configure Command and Channel objects for hardware objects.
When using YAML config file for ISPyBClient hardware object,
it feels logical to use YAML boolean type for 'loginTranslate'
property. For example a following config file:
class: ISPyBClient.ISPyBClient
configuration:
authServerType: ispyb
loginTranslate: false
This change fixes a bug where 'self.loginTranslate' would be
assigned True value for the config file above.
The 'zoom' attribute is now automagically set from the 'objects' settings in YAML/XML configure file.
elmjag
force-pushed
the
xml_yaml_conversion
branch
from
4219954 to
02a5b97
Compare
Adds support for new YAML format for hardware objects configuration files.
With this changes it's possible configure all of the beamline hardware objects using YAML files. See https://mxcubecore--1062.org.readthedocs.build/en/1062/dev/configuration_files.html#yaml-configured-objects for details on the new YAML config format.
This is a breaking change, see https://mxcubecore--1062.org.readthedocs.build/en/1062/dev/yaml_conf_migration.html for details on known non-backward compatible changes.
A level of backward compatibility with XML hardware configuration files is retained, so it should be possible to mix XML and YAML configuration files. See the above document for details.
Putting this as Draft for now. Let's discuss first discuss these changes at the upcoming developers meeting.