Skip to content

The idea and concept behind WMDS

Jump to bottom
Jörg Klausen edited this page Jul 26, 2023 · 22 revisions

An important aspect of the World Meteorological Organization (WMO) Integrated Global Observing System (WIGOS) implementation is ensuring maximum usefulness of WIGOS observations. Observations without metadata are of very limited use: it is only when accompanied by adequate metadata (data describing the data) that the full potential of the observations can be utilized. The availability of WIGOS metadata is essential for the effective planning and management of WIGOS observing systems. These metadata are also crucial for the Rolling Review of Requirements process and similar activities at national level.

Two complementary types of metadata are required: discovery metadata and station/observation metadata. Discovery metadata facilitate data discovery, access and retrieval. These are called WMO Information System (WIS) metadata and are specified and handled as part of WIS. Station/observational metadata enable data values to be interpreted in context. These constitute WMO Integrated Global Observing System (WIGOS) metadata and are the subject of the WIGOS standard (WMDS) and its formal representation (WMDR). They describe metadata required for the effective utilization of observations from all WIGOS component observing systems by all users.

The evolution of the WIGOS Metadata Standard (WMO-No. 1192) is under responsibility of the Standing Committee on Information Management and Technology (SC-IMT) of the Infrastructure Commission.

More information: https://community.wmo.int/activity-areas/wigos

WIGOS Metadata Standard (WMDS)

WIGOS metadata should describe the observing facility, the observed variable, the conditions under which it was observed, how it was measured, and how the data have been processed, in order to provide users with confidence that the data are appropriate for their application. The WIGOS Metadata Standard (WMO-No. 1192) is a semantic standard that specifies the metadata elements for observations that can be recorded and exchanged. It applies to all WIGOS component observing systems.

WIGOS Metadata Representation (WMDR)

WMDR 1.0

The Standard is expressed formally as a UML model, expressed as the WMDR XML Schema Definition, and implemented in OSCAR/Surface, which is the WMO official authoritative repository of metadata on surface-based meteorological, climatological, hydrological and other related environmental observations that are required for international exchange.

Requirements

The WMDR must be compliant with ISO 19156 (Geographic information — Observations and measurements, O&M). An observation is an act that results in the estimation of the value of a feature property, and involves application of a specified procedure, such as a sensor, instrument, algorithm or process chain. The procedure may be applied in-situ, remotely, or ex-situ with respect to the sampling location. Use of a common model for observation metadata allows data to be combined unambiguously, across discipline boundaries. Observation details are also important for data discovery and for data quality estimation. An observation is defined in terms of the set of properties that support these applications.

Feature property, examples:

  • Temperature of the atmosphere at a specified distance from the ground
  • Mole fraction of sodium in the upper ocean
  • Cloud type
  • Mass concentration of sulphate in the PM1 to PM2.5 size range of ambient aerosol in the free troposhere

O&M defines a core set of properties for an observation:

  • feature of interest, e.g., the atmosphere
  • observed property, e.g., Mass concentration of sulphate in the particle phase [in the atmosphere]. This can/should be further specified by documenting the particle size range, the relative humidity, the temperature, perhaps the pressure.
  • procedure – the instrument, algorithm or process used (which may be described using SensorML)
  • phenomenon time – the real-world time associated with the result
  • result
  • result time – the time when the result was generated
  • valid time – the period during which the result may be used

The WMDR should be aligned with INSPIRE, an EU directive providing a framework for a spatial data infrastructure.

Evolution of the WIGOS metadata information model - towards WMDR 2.0

To evolve the WMDR UML model (documentation), a coherent definition and common understanding of terminology is needed. The objective of the WMDS remain, namely to describe observations comprehensively, to 'enable adequate use of observations'. WMDR 1.0 was developed with this in mind and has proven useful since its inception around 2015. Modeling WMDR in compliance with ISO19156 (O&M) has also proved useful. However, some shortcomings have become apparent that motivated a review of the basics, to re-assess the metadata elements relevant to meet the original objective, to compare the existing WMDR against them, and expand, or perhaps in part better define terms in use.

Elements to describe observations

The observed variable is defined in the WIGOS metadata standard as category "Observed variables - measurand" (1-01). In the current WIGOS metadata representation (WMDR 1.0), this is realized in separate code tables, one for each of the major domains. The code tables 1-01-0x are set up with a variable ID (here called "notation"), a path representing the hierarchy (syntax: \[domain]\[sub-domain_1]\ ... \[sub-domain_n]\[variable name]), the variable name itself and a description. This approach has obvious limitations, the perhaps most important one being the fact that the same quantity (e.g., temperature) needs to be listed multiple times depending on the domain and/or subdomain concerned. Furthermore, additional aspects/factors import for a comprehensive documentation of an observation were folded into the name of the quantity.

To describe an observation, a more useful and future-proof approach is to more explicitly consider the most relevant aspects/factors that determine what defines the measurand. Clearly, this delineation is always somewhat subjective. The guiding principle must be 'fit-for-purpose', and judgement of experts from many disciplines is necessary to gauge this. The following is proposed:

Observation = domain x subdomain(s) x species/quantity x geometry x vertical distance from reference surface x (particle) size x pressure x temperature x humidity x sample treatment x method (of observation) x wave length(s)/frequency x instrument x operator.

A metadata model such as WMDR re-groups and organizes all these aspects in various feature types, thereby also creating dependencies and hierarchy. This is also the approach of O&M.

  • observation = observed variable x geometry x vertical distance from reference surface x procedure
  • observed variable = domain x subdomain(s) x species/quantity
  • procedure = sampling x processing x reporting
  • sampling = size x pressure x temperature x humidity
  • processing = sample treatment x method x method details x instrument x operator
  • method details = wave length(s)/frequency x free text
  • operator = an individual involved in making the observation

Observed variables: Examples

  • domain = {atmosphere, terrestrial, marine, space, ...}
  • subdomain(s) involves concepts like matrix and further grouping of variables, "matrix" refers to concepts like "gas phase", "particle phase", "total atmospheric deposition" etc.
  • species/quantity = biogeochemical species or physical quantity, e.g., carbon dioxide, active layer thickness, chlorophyll, temperature

Mapping between WMDR and WIGOS code lists

For WMDR 1.0(RC9), the mapping is a follows

WMDR1.0(RC9) WMO Codes Registry
observedProperty http://codes.wmo.int/wmdr/ObservedVariableAtmosphere
http://codes.wmo.int/wmdr/ObservedVariableEarth
http://codes.wmo.int/wmdr/ObservedVariableOcean
http://codes.wmo.int/wmdr/ObservedVariableOuterSpace
http://codes.wmo.int/wmdr/ObservedVariableTerrestrial

FT2022-2 introduces new WMDR code lists 'Domain' and 'Matrix'. With this, the following improvement is possible without a change of the model, but it requires a change to the specification of the business rules and parsers wishing to exploit this. This change is currently (June 2022) work in progress, the current draft is available here. Essentially, it makes explicit use of featureOfInterest, observedProperty and OM_Observation/property, whereby featureOfInterest and OM_Observation/property are viewed as 'constraints' of observedProperty:

  • Subject of interest (ObservedVariable) = observedProperty of featureOfInterest [with wmdrConstraints]
    • observedProperty = [wmdrProperty of] wmdrObservedVariable
    • featureOfInterest = [wmdrMatrix (in/of) the] wmdrDomain
    • wmdrConstraints = OM_Observation/property:namedValue

NOTE 1: The prefix wmdr indicates that a WMDR code lists exists (or will be needed)

NOTE 2: wmdrProperty = {'concentration', 'intensity', 'bending angle', 'colour', ...} is a new code list to be established, whose use is conditional on wmdrObservedVariable in the case where the value of wmdrObservedVariable leaves room for interpretation.

NOTE 3: wmdrObservedVariable is a consolidated combination of the existing codelists wmdrObservedVariableX, where X = {'Atmosphere', 'Ocean', ...}

NOTE 4: Use of wmdrMatrix is conditional on the context as determined by wmdrObservedVariable and featureOfInterest

NOTE 5: wmdrConstraints = {'particleSizeMax', 'particleSizeMin', 'relativeHumidityMax', 'relativeHumidityMin', 'temperatureMax', 'temperatureMin', 'wavelengthRangeLower', 'wavelengthRangeUpper', ...} is a new code list to be established, whose use is conditional on wmdrObservedVariable in the case where the value of wmdrObservedVariable leaves room for interpretation.

Useful links:

TT-WIGOSMD Official WMO Page

Home

Meetings

Changing Codetables

Clone this wiki locally