Data export in hierarchical data formats (HDF5, netCDF) fails when field names contain special characters

[amotl]

Hi again,

when trying to export data in hierarchical data formats, where its fieldnames contain special characters like /, the process croaks.

HDF5

ERROR: ValueError: the ``/`` character is not allowed in object names: 'VAR[m/s]'

ERROR: ValueError: the ``/`` character is not allowed in object names: 'VAR[m/s]'

------------------------------------------------------------
Entry point:
Filename:    /opt/kotori/lib/python3.5/site-packages/kotori/io/export/tabular.py
Line number: 60
Function:    render
Code:        df.to_hdf(t.name, group_name, format='table', data_columns=True, index=False)
------------------------------------------------------------
Source of exception:
Filename:    /opt/kotori/lib/python3.5/site-packages/tables/path.py
Line number: 165
Function:    check_name_validity
Code:        "in object names: %r" % name)

Traceback (most recent call last):
  File "/opt/kotori/lib/python3.5/site-packages/kotori/io/export/tabular.py", line 60, in render
    df.to_hdf(t.name, group_name, format='table', data_columns=True, index=False)
  File "/opt/kotori/lib/python3.5/site-packages/pandas/core/generic.py", line 2530, in to_hdf
    pytables.to_hdf(path_or_buf, key, self, **kwargs)
  File "/opt/kotori/lib/python3.5/site-packages/pandas/io/pytables.py", line 278, in to_hdf
    f(store)
  File "/opt/kotori/lib/python3.5/site-packages/pandas/io/pytables.py", line 271, in <lambda>
    f = lambda store: store.put(key, value, **kwargs)
  File "/opt/kotori/lib/python3.5/site-packages/pandas/io/pytables.py", line 959, in put
    self._write_to_group(key, value, append=append, **kwargs)
  File "/opt/kotori/lib/python3.5/site-packages/pandas/io/pytables.py", line 1525, in _write_to_group
    s.write(obj=value, append=append, complib=complib, **kwargs)
  File "/opt/kotori/lib/python3.5/site-packages/pandas/io/pytables.py", line 4214, in write
    self._handle.create_table(self.group, **options)
  File "/opt/kotori/lib/python3.5/site-packages/tables/file.py", line 1066, in create_table
    track_times=track_times)
  File "/opt/kotori/lib/python3.5/site-packages/tables/table.py", line 781, in __init__
    self.description = Description(description, ptparams=parentnode._v_file.params)
  File "/opt/kotori/lib/python3.5/site-packages/tables/description.py", line 541, in __init__
    check_name_validity(k)
  File "/opt/kotori/lib/python3.5/site-packages/tables/path.py", line 165, in check_name_validity
    "in object names: %r" % name)
ValueError: the ``/`` character is not allowed in object names: 'VAR[m/s]'

The issue can be reproduced like:

http "https://mah.panodata.net/api/mah/testdrive/area-42/node-1/data.hdf5?from=2021-09-21T09:20:00%2B02:00&to=2021-09-21T10:10:00%2B02:00"

netCDF

ERROR: KeyError: 'VAR[m/s]'

ERROR: KeyError: 'VAR[m/s]'

------------------------------------------------------------
Entry point:
Filename:    /opt/kotori/lib/python3.5/site-packages/kotori/io/export/tabular.py
Line number: 76
Function:    render
Code:        df.to_xarray().to_netcdf(path=t.name, format='NETCDF4', engine='netcdf4', group=group_name)
------------------------------------------------------------
Source of exception:
Filename:    /opt/kotori/lib/python3.5/site-packages/xarray/backends/netCDF4_.py
Line number: 66
Function:    get_array
Code:        variable = ds.variables[self.variable_name]

Traceback (most recent call last):
  File "/opt/kotori/lib/python3.5/site-packages/kotori/io/export/tabular.py", line 76, in render
    df.to_xarray().to_netcdf(path=t.name, format='NETCDF4', engine='netcdf4', group=group_name)
  File "/opt/kotori/lib/python3.5/site-packages/xarray/core/dataset.py", line 1540, in to_netcdf
    invalid_netcdf=invalid_netcdf,
  File "/opt/kotori/lib/python3.5/site-packages/xarray/backends/api.py", line 1074, in to_netcdf
    dataset, store, writer, encoding=encoding, unlimited_dims=unlimited_dims
  File "/opt/kotori/lib/python3.5/site-packages/xarray/backends/api.py", line 1120, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "/opt/kotori/lib/python3.5/site-packages/xarray/backends/common.py", line 303, in store
    variables, check_encoding_set, writer, unlimited_dims=unlimited_dims
  File "/opt/kotori/lib/python3.5/site-packages/xarray/backends/common.py", line 341, in set_variables
    name, v, check, unlimited_dims=unlimited_dims
  File "/opt/kotori/lib/python3.5/site-packages/xarray/backends/netCDF4_.py", line 514, in prepare_variable
    target = NetCDF4ArrayWrapper(name, self)
  File "/opt/kotori/lib/python3.5/site-packages/xarray/backends/netCDF4_.py", line 39, in __init__
    array = self.get_array()
  File "/opt/kotori/lib/python3.5/site-packages/xarray/backends/netCDF4_.py", line 66, in get_array
    variable = ds.variables[self.variable_name]
KeyError: 'VAR[m/s]'

The issue can be reproduced like:

http "https://mah.panodata.net/api/mah/testdrive/area-42/node-1/data.nc?from=2021-09-21T09:20:00%2B02:00&to=2021-09-21T10:10:00%2B02:00"

With kind regards,
Andreas.

/cc @mhaberler