Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix hawkins sutton smoke tests, linting #1769

Merged
merged 6 commits into from
Jun 7, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/notebooks/ensembles.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -281,14 +281,14 @@
"source": [
"### Change significance and model agreement\n",
"\n",
"When communicating climate change through plots of projected change, it is often useful to add information on the statistical significance of the values. A common way to represent this information without overloading the figures is through hatching patterns superimposed on the primary data. Two aspects are usually shown : \n",
"When communicating climate change through plots of projected change, it is often useful to add information on the statistical significance of the values. A common way to represent this information without overloading the figures is through hatching patterns superimposed on the primary data. Two aspects are usually shown: \n",
"\n",
"- change significance : whether most of the ensemble members project a statistically significant climate change signal, in comparison to their internal variability.\n",
"- model agreement : whether the different ensemble members agree on the sign of the change.\n",
"- change significance: whether most of the ensemble members project a statistically significant climate change signal, in comparison to their internal variability.\n",
"- model agreement: whether the different ensemble members agree on the sign of the change.\n",
"\n",
"We can then divide the plotted points into categories each with its own hatching pattern, usually leaving the robust data (models agree and enough show a significant change) without hatching. \n",
"\n",
"Xclim provides some tools to help in generating these hatching masks. First is [xc.ensembles.robustness_fractions](../apidoc/xclim.ensembles.rst#xclim.ensembles._robustness.robustness_fractions) that can characterize the change significance and sign agreement across ensemble members. To demonstrate its usage, we'll first generate some fake annual mean temperature data. Here, `ref` is the data on the reference period and `fut` is a future projection. There are 5 different members in the ensemble. We tweaked the generation so that all models agree on significant change in the \"south\" while agreement and signifiance of change decreases as we go north and east."
"Xclim provides some tools to help in generating these hatching masks. First is [xc.ensembles.robustness_fractions](../apidoc/xclim.ensembles.rst#xclim.ensembles._robustness.robustness_fractions) that can characterize the change significance and sign agreement across ensemble members. To demonstrate its usage, we'll first generate some fake annual mean temperature data. Here, `ref` is the data on the reference period and `fut` is a future projection. There are five (5) different members in the ensemble. We tweaked the generation so that all models agree on significant change in the \"South\" while agreement and significance of change decreases as we go North and East."
]
},
{
Expand Down
8 changes: 4 additions & 4 deletions docs/notebooks/sdba-advanced.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@
"\n",
"Some Adjustment methods require that the adjusted data (`sim`) be of the same length (same number of points) than the training data (`ref` and `hist`). These requirements often ensure conservation of statistical properties and a better representation of the climate change signal over the long adjusted timeseries.\n",
"\n",
"In opposition to a conventional \"rolling window\", here it is the _years_ that are the base units of the window, not the elements themselves. `xclim` implements `xc.core.calendar.stack_periods` and `xc.core.calendar.unstack_periods` to manipulate data in that goal. The \"stack\" function cuts the data in overlapping windows of a certain length and stacks them along a new `\"period\"` dimension, alike to xarray's `da.rolling(time=win).construct('period')`, but with yearly steps. The stride (or step) between each window can also be controlled. This argument is an indicator of how many years overlap between each window. With a value of 1, a window will have `window - 1` years overlapping with the previous one. The default (`None`) is to have `stride = window` will result in no overlap at all. The default units in which `window` and `stride` are given is a year (\"YS\"), but can be changed with argument `freq`.\n",
"In opposition to a conventional \"rolling window\", here it is the _years_ that are the base units of the window, not the elements themselves. `xclim` implements `xc.core.calendar.stack_periods` and `xc.core.calendar.unstack_periods` to manipulate data in that goal. The \"stack\" function cuts the data in overlapping windows of a certain length and stacks them along a new `\"period\"` dimension, alike to xarray's `da.rolling(time=win).construct('period')`, but with yearly steps. The stride (or step) between each window can also be controlled. This argument is an indicator of how many years overlap between each window. With a value of `1`, a window will have `window - 1` years overlapping with the previous one. The default (`None`) is to have `stride = window` will result in no overlap at all. The default units in which `window` and `stride` are given is a year (\"YS\"), but can be changed with argument `freq`.\n",
"\n",
"By chunking the result along this `'period'` dimension, it is expected to be more computationally efficient (when using `dask`) than looping over the windows with a for-loop (or a `GroupyBy`)\n",
"\n",
Expand All @@ -355,13 +355,13 @@
"1. The constructed array has the same \"time\" axis for all windows. This is a problem if the actual _year_ is of importance for the adjustment, but this is not the case for any of xclim's current adjustment methods.\n",
"2. The input timeseries must be in a calendar with uniform year lengths. For daily data, this means only the \"360_day\", \"noleap\" and \"all_leap\" calendars are supported.\n",
"\n",
"The \"unstack\" function does the opposite : it concatenates the windows together to recreate the original timeseries. It only works for the no-overlap case where `stride = window` and for the non-ambiguous one where `stride` divides `window` into an odd number (N) of parts. In that latter situation, the middle parts of each period are kept when reconstructing the timeseries, in addition to the first (last) parts of the first (last) period needed to get a full timeseries.\n",
"The \"unstack\" function does the opposite: it concatenates the windows together to recreate the original timeseries. It only works for the no-overlap case where `stride = window` and for the non-ambiguous one where `stride` divides `window` into an odd number (N) of parts. In that latter situation, the middle parts of each period are kept when reconstructing the timeseries, in addition to the first (last) parts of the first (last) period needed to get a full timeseries.\n",
"\n",
"Quantile Delta Mapping requires that the adjustment period should be of a length similar to the training one. As our `ref` and `hist` cover 15 years but `sim` covers 31 years, we will transform `sim` by stacking windows of 15 years. With a stride of 5 years, this means the first window goes from 2000 to 2014 (inclusive). Then 2005-2019, 2010-2024 and 2015-2029. The last year will be dropped as it can't be included in any complete window.\n",
"Quantile Delta Mapping requires that the adjustment period should be of a length similar to the training one. As our `ref` and `hist` cover 15 years but `sim` covers 31 years, we will transform `sim` by stacking windows of 15 years. With a stride of five (5) years, this means the first window goes from 2000 to 2014 (inclusive). Then 2005-2019, 2010-2024 and 2015-2029. The last year will be dropped as it can't be included in any complete window.\n",
"\n",
"<div class=\"alert alert-warning\">\n",
"\n",
"In the following example, `QDM` is configurated with `group=\"time.dayofyear\"` which will perform the adjustment for each day of year (doy) separately. When using `stack_periods` the extracted windows are all concatenated along the new `period` axis and they all share the same time coordinate. As such, for the doy information to make sense, we must use a calendar with uniform year lengths. Otherwise, the doys would shift one day at each leap year.\n",
"In the following example, `QDM` is configured with `group=\"time.dayofyear\"` which will perform the adjustment for each day of year (doy) separately. When using `stack_periods` the extracted windows are all concatenated along the new `period` axis, and they all share the same time coordinate. As such, for the `doy` information to make sense, we must use a calendar with uniform year lengths. Otherwise, the `doy` values would shift one day at each leap year.\n",
"\n",
"</div>"
]
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ values = [
]

[tool.codespell]
skip = 'xclim/data/*.json,docs/_build,docs/notebooks/xclim_training/*.ipynb,docs/references.bib,__pycache__,*.nc,*.png,*.gz,*.whl'
skip = 'xclim/data/*.json,docs/_build,docs/notebooks/xclim_training/*.ipynb,docs/references.bib,__pycache__,*.gz,*.nc,*.png,*.svg,*.whl'
ignore-words-list = "absolue,astroid,bloc,bui,callendar,degreee,environnement,hanel,inferrable,lond,nam,nd,ressources,sie,vas"

[tool.coverage.run]
Expand Down
6 changes: 5 additions & 1 deletion tests/test_partitioning.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@
import xarray as xr

from xclim.ensembles import fractional_uncertainty, hawkins_sutton, lafferty_sriver
from xclim.ensembles._filters import _concat_hist, _model_in_all_scens, _single_member
from xclim.ensembles._filters import ( # noqa: F401
_concat_hist,
_model_in_all_scens,
_single_member,
)


def test_hawkins_sutton_smoke(open_dataset):
Expand Down
37 changes: 20 additions & 17 deletions xclim/ensembles/_filters.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,20 @@
import xarray as xr


def _concat_hist(da, **hist):
"""Concatenate historical scenario with future scenarios along time.
def _concat_hist(da: xr.DataArray, **hist) -> xr.DataArray:
r"""Concatenate historical scenario with future scenarios along the time dimension.

Parameters
----------
da : xr.DataArray
Input data where the historical scenario is stored alongside other, future, scenarios.
hist: {str: str}
Mapping of the scenario dimension name to the historical scenario coordinate, e.g. `scenario="historical"`.
Input data where the historical scenario is stored alongside other, future, scenarios.
\*\*hist : dict
Mapping of the scenario dimension name to the historical scenario coordinate, e.g. `scenario="historical"`.

Returns
-------
xr.DataArray
Data with the historical scenario is stacked in time before each one of the other scenarios.
Data with the historical scenario is stacked in time before each one of the other scenarios.

Notes
-----
Expand Down Expand Up @@ -51,6 +51,7 @@ def _concat_hist(da, **hist):

# Select historical scenario and drop it from the data
h = da.sel(**hist).dropna("time", how="all")
h = h.drop_vars(dim)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relevant changes here.

ens = da.drop_sel(**hist)

index = ens[dim]
Expand All @@ -59,15 +60,17 @@ def _concat_hist(da, **hist):
return xr.concat([h, bare], dim="time").assign_coords({dim: index})


def _model_in_all_scens(da, dimensions=None):
def _model_in_all_scens(
da: xr.DataArray, dimensions: dict | None = None
) -> xr.DataArray:
"""Return data with only simulations that have at least one member in each scenario.

Parameters
----------
da: xr.DataArray
Input data with dimensions for time, member, model and scenario.
dimensions: dict
Mapping from original dimension names to standard dimension names: scenario, model, member.
da : xr.DataArray
Input data with dimensions for time, member, model and scenario.
dimensions : dict, optional
Mapping from original dimension names to standard dimension names: scenario, model, member.

Returns
-------
Expand Down Expand Up @@ -100,20 +103,20 @@ def _model_in_all_scens(da, dimensions=None):
return da.sel(model=ok).rename(dimensions)


def _single_member(da, dimensions=None):
def _single_member(da: xr.DataArray, dimensions: dict | None = None) -> xr.DataArray:
"""Return data for a single member per model.

Parameters
----------
da : xr.DataArray
Input data with dimensions for time, member, model and scenario.
dimensions: dict
Mapping from original dimension names to standard dimension names: scenario, model, member.
Input data with dimensions for time, member, model and scenario.
dimensions : dict
Mapping from original dimension names to standard dimension names: scenario, model, member.

Returns
-------
xr.DataArray
Data with only one member per model.
Data with only one member per model.

Notes
-----
Expand Down Expand Up @@ -147,6 +150,6 @@ def _single_member(da, dimensions=None):
return out.rename(dimensions)


def reverse_dict(d):
def reverse_dict(d: dict) -> dict:
"""Reverse dictionary."""
return {v: k for (k, v) in d.items()}
Loading