Tests for the IO module (#492)

<!-- Please ensure the PR fulfills the following requirements! -->
<!-- If this is your first PR, make sure to add your details to the
AUTHORS.rst! -->
### Pull Request Checklist:
- [x] This PR addresses an already opened issue (for bug fixes /
features)
    - This PR fixes #xyz
- [x] (If applicable) Documentation has been added / updated (for bug
fixes / features).
- [x] (If applicable) Tests have been added.
- [x] This PR does not seem to break the templates.
- [x] CHANGELOG.rst has been updated (with summary of main changes).
- [x] Link to issue (:issue:`number`) and pull request (:pull:`number`)
has been added.

### What kind of change does this PR introduce?

* Mainly adds tests to the IO module, along with bugfixes:
  * Adds a missing library (`openpyxl`) to the requirements.
  * Improves the code in `estimate_chunks` to support n dimensions.
* Fixes a bug in `subset_maxsize` where the last year would be dropped.
* Fixes a bug in `clean_incomplete` where the `zmetadata` file would not
be removed.
  * Added more flexibility to `clean_incomplete`.
  * Fixed a bug in `_coerce_attrs` where encoding would get dropped.
* Fixed bugs in `save_to_zarr` with `mode=='a'`: encoding of existing
variables is now always dropped & the code now crashes if trying to both
append dimensions with new data and add a new variable to an existing
zarr.
* Fixed bugs in `save_to_zarr` where the code would crash when skipping
a variable if `bitround` was not None.
* Fixed a bug in `make_toc` where the French name for the sheet would
get dropped if given a Dataset instead of a DataArray.
* Better default argument for `column` in `save_to_table`. It stays
unchanged if given a Dataset with multiple variables, but is now `None`
otherwise.
* Fixed a bug in `save_to_table` where `engine_kwargs` would not be used
if saving an Excel with a single sheet.

- An option was added to `publish_release_notes` to only print the notes
relevant to the latest version, rather than the whole Changelog.

### Does this PR introduce a breaking change?

-To remove some clutter, `publish_release_notes` and `show_versions`
have been moved from `utils.py` to `testing.py`.

### Other information:

CHANGELOG.rst

@@ -2,6 +2,35 @@
 Changelog
 =========
 
+v0.11.0 (unreleased)
+--------------------
+Contributors to this version: Gabriel Rondeau-Genesse (:user:`RondeauG`).
+
+New features and enhancements
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+* N/A
+
+Breaking changes
+^^^^^^^^^^^^^^^^
+* ``xs.utils.publish_release_notes`` and ``xs.utils.show_versions`` have been moved to ``xs.testing``. (:pull:`492`).
+
+Bug fixes
+^^^^^^^^^
+* Added a missing library (``openpyxl``) to the requirements. (:pull:`492`).
+* Fixed a bug in ``xs.io.subset_maxsize`` where the function would drop the last year. (:pull:`492`).
+* Fixed a bug in ``xs.io.clean_incomplete`` where the `.zmetadata` file was not removed. (:pull:`492`).
+* Fixed a bug in the saving of datasets where encoding was sometimes not applied, resulting for example in rechunking not being respected. (:pull:`492`).
+* Fixed multiple bugs in ``xs.io.save_to_zarr`` with `mode='a'`. (:pull:`492`).
+* Fixed a few minor bugs in ``xs.io.save_to_table``. (:pull:`492`).
+
+Internal changes
+^^^^^^^^^^^^^^^^
+* Added a new parameter `latest` to ``xs.testing.publish_release_notes`` to only print the latest release notes. (:pull:`492`).
+* The estimation method in ``xs.io.estimate_chunks`` has been improved. (:pull:`492`).
+* A new parameter `incomplete` has been added to ``xs.io.clean_incomplete`` to remove incomplete variables. (:pull:`492`).
+* Continued work on adding tests. (:pull:`492`).
+
+
 v0.10.1 (2024-11-04)
 --------------------
 Contributors to this version: Gabriel Rondeau-Genesse (:user:`RondeauG`), Pascal Bourgault (:user:`aulemahal`), Éric Dupuis (:user:`coxipi`).

docs/notebooks/4_ensembles.ipynb

@@ -40,10 +40,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "from xclim.testing import open_dataset\n",
+    "import pooch\n",
+    "import xarray as xr\n",
+    "from xclim.testing.utils import nimbus\n",
     "\n",
     "import xscen as xs\n",
     "\n",
+    "downloader = pooch.HTTPDownloader(headers={\"User-Agent\": f\"xscen-{xs.__version__}\"})\n",
+    "\n",
     "datasets = {\n",
     "    \"ACCESS\": \"EnsembleStats/BCCAQv2+ANUSPLIN300_ACCESS1-0_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc\",\n",
     "    \"BNU-ESM\": \"EnsembleStats/BCCAQv2+ANUSPLIN300_BNU-ESM_historical+rcp45_r1i1p1_1950-2100_tg_mean_YS.nc\",\n",
@@ -53,7 +57,8 @@
     "}\n",
     "\n",
     "for d in datasets:\n",
-    "    ds = open_dataset(datasets[d]).isel(lon=slice(0, 4), lat=slice(0, 4))\n",
+    "    file = nimbus().fetch(datasets[d], downloader=downloader)\n",
+    "    ds = xr.open_dataset(file).isel(lon=slice(0, 4), lat=slice(0, 4))\n",
     "    ds = xs.climatological_op(\n",
     "        ds,\n",
     "        op=\"mean\",\n",

environment-dev.yml

@@ -10,8 +10,8 @@ dependencies:
   - cftime
   - cf_xarray >=0.7.6
   - clisops >=0.10
-  - dask
-  - flox
+  - dask >=2024.8.1,<2024.11  # FIXME: https://github.com/Ouranosinc/xclim/issues/1992
+  - flox !=0.9.14  # FIXME: 0.9.14 is a broken version. This pin could be removed eventually.
   - fsspec
   - geopandas
   - h5netcdf
@@ -21,6 +21,7 @@ dependencies:
   - netCDF4
   - numcodecs
   - numpy >=1.24
+  - openpyxl
   - pandas >=2.2
   - parse
   - pyyaml
@@ -31,7 +32,7 @@ dependencies:
   - toolz
   - xarray >=2023.11.0, !=2024.6.0
   - xclim >=0.53.2, <0.54
-  - xesmf >=0.7
+  - xesmf >=0.7, <0.8.8  # FIXME: 0.8.8 currently creates segfaults on ReadTheDocs.
   - zarr >=2.13
   # Opt
   - nc-time-axis >=1.3.1

environment.yml

@@ -10,8 +10,8 @@ dependencies:
   - cftime
   - cf_xarray >=0.7.6
   - clisops >=0.10
-  - dask
-  - flox
+  - dask >=2024.8.1,<2024.11  # FIXME: https://github.com/Ouranosinc/xclim/issues/1992
+  - flox !=0.9.14  # FIXME: 0.9.14 is a broken version. This pin could be removed eventually.
   - fsspec
   - geopandas
   - h5netcdf
@@ -21,6 +21,7 @@ dependencies:
   - netCDF4
   - numcodecs
   - numpy >=1.24
+  - openpyxl
   - pandas >=2.2
   - parse
   - pyyaml
@@ -31,7 +32,7 @@ dependencies:
   - toolz
   - xarray >=2023.11.0, !=2024.6.0
   - xclim >=0.53.2, <0.54
-  - xesmf >=0.7
+  - xesmf >=0.7, <0.8.8  # FIXME: 0.8.8 currently creates segfaults on ReadTheDocs.
   - zarr >=2.13
   # To install from source
   - setuptools >=65.0.0

pyproject.toml

@@ -43,8 +43,8 @@ dependencies = [
   "cftime",
   "cf_xarray >=0.7.6",
   "clisops >=0.10",
-  "dask",
-  "flox",
+  "dask >=2024.8.1,<2024.11", # FIXME: https://github.com/Ouranosinc/xclim/issues/1992
+  "flox !=0.9.14", # FIXME: 0.9.14 is a broken version. This pin could be removed eventually.
   "fsspec",
   "geopandas",
   "h5netcdf",
@@ -54,6 +54,7 @@ dependencies = [
   "netCDF4",
   "numcodecs",
   "numpy >=1.24",
+  "openpyxl",
   "pandas >=2.2",
   "parse",
   # Used when opening catalogs.
@@ -111,7 +112,7 @@ docs = [
   "sphinxcontrib-napoleon"
 ]
 extra = [
-  "xesmf>=0.7"
+  "xesmf>=0.7, <0.8.8" # FIXME: 0.8.8 currently creates segfaults on ReadTheDocs.
 ]
 all = ["xscen[dev]", "xscen[docs]", "xscen[extra]"]
 

src/xscen/data/fr/LC_MESSAGES/xscen.po

@@ -19,8 +19,8 @@ msgstr "Description"
 msgid "Units"
 msgstr "Unités"
 
-msgid "Content"
-msgstr "Contenu"
-
 msgid "Global attributes"
 msgstr "Attributs globaux"
+
+msgid "Content"
+msgstr "Contenu"