Skip to content

Commit

Permalink
add link to CF conventions on packed data in doc/user-guide/io.rst (#…
Browse files Browse the repository at this point in the history
…9045)

* add link to CF conventions on packed data in doc/user-guide/io.rst

* add whats-new.rst entry

* add more details to type determination

* fix whats-new.rst

* remove a comma

---------

Co-authored-by: Deepak Cherian <[email protected]>
Co-authored-by: Justus Magin <[email protected]>
  • Loading branch information
3 people authored and andersy005 committed Jun 14, 2024
1 parent 84f7d4c commit d7604f1
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 2 deletions.
8 changes: 6 additions & 2 deletions doc/user-guide/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -416,7 +416,8 @@ read the data.
Scaling and type conversions
............................

These encoding options work on any version of the netCDF file format:
These encoding options (based on `CF Conventions on packed data`_) work on any
version of the netCDF file format:

- ``dtype``: Any valid NumPy dtype or string convertible to a dtype, e.g., ``'int16'``
or ``'float32'``. This controls the type of the data written on disk.
Expand All @@ -427,14 +428,17 @@ These encoding options work on any version of the netCDF file format:
output file, unless explicitly disabled with an encoding ``{'_FillValue': None}``.
- ``scale_factor`` and ``add_offset``: Used to convert from encoded data on disk to
to the decoded data in memory, according to the formula
``decoded = scale_factor * encoded + add_offset``.
``decoded = scale_factor * encoded + add_offset``. Please note that ``scale_factor``
and ``add_offset`` must be of same type and determine the type of the decoded data.

These parameters can be fruitfully combined to compress discretized data on disk. For
example, to save the variable ``foo`` with a precision of 0.1 in 16-bit integers while
converting ``NaN`` to ``-9999``, we would use
``encoding={'foo': {'dtype': 'int16', 'scale_factor': 0.1, '_FillValue': -9999}}``.
Compression and decompression with such discretization is extremely fast.

.. _CF Conventions on packed data: https://cfconventions.org/cf-conventions/cf-conventions.html#packed-data

.. _io.string-encoding:

String encoding
Expand Down
2 changes: 2 additions & 0 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ Bug fixes

Documentation
~~~~~~~~~~~~~
- Add link to CF Conventions on packed data and sentence on type determination in doc/user-guide/io.rst (:issue:`9041`, :pull:`9045`).
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_.


Internal Changes
Expand Down

0 comments on commit d7604f1

Please sign in to comment.