Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop Python3.8, support Pandas v2.2, reduce Warnings #1565

Merged
merged 50 commits into from
Jan 29, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
de79e09
pre-commit update
Zeitsperre Dec 18, 2023
28d7c0e
drop all Python3.8 in top-level
Zeitsperre Dec 18, 2023
839585f
pyupgrade fixes
Zeitsperre Dec 18, 2023
08ae98d
include tests in ruff checks
Zeitsperre Dec 18, 2023
fc84c89
limit to Python3.11
Zeitsperre Dec 18, 2023
8853ec0
drop Python3.8
Zeitsperre Dec 18, 2023
1ee9d24
fix import, fix docs
Zeitsperre Dec 18, 2023
edde1e2
small signature adjustments, docstring fixes
Zeitsperre Dec 18, 2023
f911164
safer Python calls
Zeitsperre Dec 18, 2023
6fe7566
update read-the-docs package name
Zeitsperre Jan 3, 2024
dc21fb0
Merge branch 'master' into drop-python38
Zeitsperre Jan 3, 2024
288346f
Merge branch 'master' into drop-python38
Zeitsperre Jan 10, 2024
084dc96
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 10, 2024
a4b4c35
Merge branch 'master' into drop-python38
Zeitsperre Jan 22, 2024
8c7dcfb
fix toml error
Zeitsperre Jan 22, 2024
08b2289
small adjustments to frequencies
Zeitsperre Jan 22, 2024
b01c6ce
Add fixme notice about silently failing tests
Zeitsperre Jan 22, 2024
91a7030
remove deprecated winter_storm indice and indicator
Zeitsperre Jan 22, 2024
ffc6ad8
raise version pins on numpy, scipy, and pandas; remove legacy version…
Zeitsperre Jan 22, 2024
ee43930
update CHANGES.rst
Zeitsperre Jan 22, 2024
6e7c832
Merge branch 'master' into drop-python38
Zeitsperre Jan 22, 2024
c37c487
Apply suggestions from code review
Zeitsperre Jan 23, 2024
a27aa62
Merge branch 'master' into drop-python38
Zeitsperre Jan 23, 2024
109bd5a
merge topd22 commit 1
aulemahal Jan 22, 2024
b6718fd
Merge branch 'master' into drop-python38
aulemahal Jan 23, 2024
da806bb
reduce test warnings, fix some docstrings, add pyarrow to dependencie…
Zeitsperre Jan 23, 2024
25bcf0b
docstring fixes and warnings reduction
Zeitsperre Jan 23, 2024
276dd4d
silence numba.vectorize decorator warnings
Zeitsperre Jan 23, 2024
d4dcff0
investigate weekofyear FutureWarning
Zeitsperre Jan 23, 2024
c7eab8b
Remove deprecated code - remove warnings - avoid using YE with cftime
aulemahal Jan 23, 2024
d1ce1dd
Merge branch 'drop-python38' of github.com:Ouranosinc/xclim into drop…
aulemahal Jan 23, 2024
389e26d
remove debugging leftover
aulemahal Jan 23, 2024
e8e85e2
silence pint-related warnings, add note on how to eventually resolve
Zeitsperre Jan 23, 2024
dd68994
remove noqa
Zeitsperre Jan 23, 2024
bad864f
ignore division by zero errors in to_additive_space
Zeitsperre Jan 23, 2024
2c8ecec
handle pint string formatting, point to ongoing issue upstream
Zeitsperre Jan 23, 2024
5946c61
filter DeprecationWarnings stemming from pint value string formatting
Zeitsperre Jan 23, 2024
57ce43f
fix rl for xr 2024.01
aulemahal Jan 23, 2024
bad8f84
Remove mention of older scipy in tests
aulemahal Jan 23, 2024
ab481ca
Remove debugging print
aulemahal Jan 23, 2024
a7a3d1e
Merge branch 'master' into drop-python38
Zeitsperre Jan 24, 2024
21c966d
Merge branch 'master' into drop-python38
Zeitsperre Jan 25, 2024
d0d9422
Apply suggestions from code review
Zeitsperre Jan 25, 2024
cb97a9c
Merge branch 'master' into drop-python38
Zeitsperre Jan 25, 2024
5f59e31
remove Python3.8-backported library
Zeitsperre Jan 25, 2024
e54626c
Merge branch 'master' into drop-python38
Zeitsperre Jan 26, 2024
69f1e6b
update CHANGES.rst
Zeitsperre Jan 26, 2024
d679f7b
update pyupgrade style
Zeitsperre Jan 26, 2024
ca84199
Update CHANGES.rst
Zeitsperre Jan 29, 2024
8bd6efc
Merge branch 'master' into drop-python38
Zeitsperre Jan 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ jobs:
strategy:
matrix:
include:
- tox-env: py39-coverage-sbck-eofs
- tox-env: py39-coverage-sbck
python-version: "3.9"
markers: -m 'not slow'
- tox-env: py310-coverage # No markers -- includes slow tests
Expand Down
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ repos:
rev: v3.15.0
hooks:
- id: pyupgrade
args: ['--py38-plus']
args: ['--py39-plus']
exclude: 'xclim/core/indicator.py'
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
Expand Down Expand Up @@ -59,7 +59,7 @@ repos:
rev: 1.7.1
hooks:
- id: nbqa-pyupgrade
args: [ '--py38-plus' ]
args: [ '--py39-plus' ]
- id: nbqa-black
additional_dependencies: [ 'black==24.1.0' ]
- id: nbqa-isort
Expand Down
24 changes: 16 additions & 8 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,19 @@ New features and enhancements

Breaking changes
^^^^^^^^^^^^^^^^
* With pandas 2.2 and xarray 2023.11.0, many frequency strings where changed : Y becomes YE, M -> ME, Q -> QE. A and AS are removed. T, L, U, N become min, ms, us and ns.
* `bump2version` has been replaced with `bump-my-version` to bump the version number using configurations set in the `pyproject.toml` file. (:issue:`1557`, :pull:`1569`).
* `xclim` base Python version has been raised to `python>=3.9`. Python3.9+ coding conventions are now supported. (:issue:`1268`, :pull:`1565`).
* `xclim` base dependencies have been raised to `pandas>=2.2.0` and `xarray>=2023.11.0` to reflect changes to time frequency codes introduced in `pandas==2.2.0`. (:issue:`1534`, :pull:`1565`; see also: `pydata/xarray GH/8394 <https://github.com/pydata/xarray/issues/8394>`_ and ). Many default frequency string outputs have been modified (:
* 'Y' (year) -> 'YE' (year end). (see: `pandas PR/55792 <https://github.com/pandas-dev/pandas/pull/55792>`_).
* 'M' (month) -> 'ME' (month end). (see: `pandas PR/52064 <https://github.com/pandas-dev/pandas/pull/52064>`_).
* 'Q' (quarter) -> 'QE' (quarter end). (see: `pandas PR/55553 <https://github.com/pandas-dev/pandas/pull/55553>`_)
* 'A' and 'AS' have been removed (use 'Y' and 'YS' instead). (see: `pandas PR/55252 <https://github.com/pandas-dev/pandas/pull/55252>`_).
Zeitsperre marked this conversation as resolved.
Show resolved Hide resolved
* 'T' (minute), 'L' (millisecond), 'U' (microsecond), and 'N' (nanosecond) -> 'min', 'ms', 'us', and 'ns'. (see: `pandas PR/54061 <https://github.com/pandas-dev/pandas/pull/54061>`_).
* `bump2version` has been replaced with `bump-my-version` to bump the version number using configurations set in the ``pyproject.toml`` file. (:issue:`1557`, :pull:`1569`).
* `xclim`'s units registry and units formatting are now extended from `cf-xarray`. The exponent sign "^" is now never added in the ``units`` attribute. For example, square meters are given as "m2" instead of "m^2" by xclim, both are still accepted as input. (:issue:`1010`, :pull:`1590`).
* `yamale` is now listed as a core dependency (was previously listed in the `dev` installation recipe). (:issue:`1595`, :pull:`1596`).
* Due to a licensing limitation, the calculation of empirical orthogonal function based on `eofs` (``xclim.sdba.properties.first_eof``) has been removed from `xclim`. (:issue:`1620`, :pull:`1621`).
* `black` formatting style has been updated to the 2024 stable conventions. `isort` has been added to the `dev` installation recipe. (:pull:`1626`).
* The indice and indicator for ``winter_storm`` has been removed (deprecated since `xclim` v0.46.0 in favour of ``snd_storm_days``). (:pull:`1565`).
* `xclim` base dependencies have risen to `pandas>=2.2.0` and `xarray>=2023.11.0` to reflect changes to time frequency codes introduced in `pandas==2.2.0`. (:issue:`1534`, :pull:`1565`; see also: `pydata/xarray GH/8394 <https://github.com/pydata/xarray/issues/8394>`_).
* `xclim` has dropped support for `scipy` version below v1.9.0 and `numpy` versions below v1.20.0. (:pull:`1565`).

Bug fixes
Expand All @@ -41,19 +48,20 @@ Bug fixes

Internal changes
^^^^^^^^^^^^^^^^
* The `flake8` configuration has been migrated from `setup.cfg` to `.flake8`; `setup.cfg` has been removed. (:pull:`1569`)
* The `bump-version.yml` workflow has been adjusted to bump the `patch` version when the last version is determined to have been a `release` version; otherwise, the `build` version is bumped. (:issue:`1557`, :pull:`1569`).
* The `flake8` configuration has been migrated from ``setup.cfg`` to ``.flake8``; ``setup.cfg`` has been removed. (:pull:`1569`)
* The ``bump-version.yml`` workflow has been adjusted to bump the `patch` version when the last version is determined to have been a `release` version; otherwise, the `build` version is bumped. (:issue:`1557`, :pull:`1569`).
* The GitHub Workflows now use the `step-security/harden-runner` action to monitor source code, actions, and dependency safety. All workflows now employ more constrained permissions rule sets to prevent security issues. (:pull:`1577`, :pull:`1578`, :pull:`1597`).
* Updated the CONTRIBUTING.rst directions to showcase the new versioning system. (:issue:`1557`, :pull:`1573`).
* The `codespell` library is now a development dependency for the `dev` installation recipe with configurations found within `pyproject.toml`. This is also now a linting step and integrated as a `pre-commit` hook. For more information, see the `codespell documentation <https://github.com/codespell-project/codespell>`_ (:pull:`1576`).
* Updated the ``CONTRIBUTING.rst`` directions to showcase the new versioning system. (:issue:`1557`, :pull:`1573`).
* The `codespell` library is now a development dependency for the `dev` installation recipe with configurations found within ``pyproject.toml``. This is also now a linting step and integrated as a `pre-commit` hook. For more information, see the `codespell documentation <https://github.com/codespell-project/codespell>`_ (:pull:`1576`).
* Climate indicators search page now prioritizes the "official" indicators (atmos, land, seaIce and generic), virtual submodules can be added to search through checkbox option. (:issue:`1559`, :pull:`1593`).
* The OpenSSF StepSecurity bot has contributed some changes to the workflows and pre-commit. (:issue:`1181`, :pull:`1606`):
* Dependabot has been configured to monitor the `xclim` repository for dependency updates. The ``actions-version-updater.yml`` workflow has been deprecated.
* GitHub Actions are now pinned to their commit hashes to prevent unexpected changes in the future.
* A new GitHub Workflow (``workflow-warning.yml``) has been added to warn maintainers when a forked repository has been used to open a Pull Request that modifies GitHub Workflows.
* `pylint` has been configured to provide some overhead checks of the `xclim` codebase as well as run as part of `xclim`'s `pre-commit` hooks.
* Some small adjustments to code organization to address `pylint` errors.
* `dev` formatting tools (`black`, `blackdoc`, `isort`) are now pinned to their `pre-commit` hook version equivalents in both `pyproject.toml` and `tox.ini`. (:pull:`1626`).
* `dev` formatting tools (`black`, `blackdoc`, `isort`) are now pinned to their `pre-commit` hook version equivalents in both ``pyproject.toml`` and ``tox.ini``. (:pull:`1626`).
* `black`, `isort`, and `pyupgrade` code formatters no longer target Python3.8 coding style conventions. (:pull:`1565`).

v0.47.0 (2023-12-01)
--------------------
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ Ready to contribute? Here's how to set up `xclim` for local development.

#. Create a development environment. We recommend using ``conda``::

$ conda create -n xclim python=3.9 --file=environment.yml
$ conda create -n xclim python=3.10 --file=environment.yml
$ python -m pip install -e ".[dev]"

#. Create a branch for local development::
Expand Down
11 changes: 1 addition & 10 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -105,17 +105,8 @@ Afterwards, `SBCK` can be installed from PyPI using `pip`:

.. code-block:: shell

$ python -m pip install SBCK
$ python -m pip install pybind11 sbck

Another experimental function :py:indicator:`xclim.sdba.property.first_eof` makes use of the `eofs`_ library, which is available on both PyPI and conda-forge:

.. code-block:: shell

$ python -m pip install eofs
# or alternatively,
$ conda install -c conda-forge eofs

.. _eofs: https://ajdawson.github.io/eofs/
.. _SBCK: https://github.com/yrobink/SBCK
.. _Eigen3: https://eigen.tuxfamily.org/index.php

Expand Down
3 changes: 1 addition & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ dependencies:
- cftime >=1.4.1
- Click >=8.1
- dask >=2.6.0
- importlib-resources # For Python3.8
- jsonpickle
- lmoments3
- numba
Expand All @@ -23,7 +22,7 @@ dependencies:
- pyarrow # Strongly encouraged for Pandas v2.2.0+
- pyyaml
- scikit-learn >=0.21.3
- scipy >=1.10.0
- scipy >=1.9.0
- statsmodels
- xarray >=2023.11.0
- yamale
Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ dependencies = [
[project.optional-dependencies]
dev = [
# Dev tools and testing
"black >=23.12.0",
"blackdoc",
"bump-my-version",
"black ==24.1.0",
"blackdoc ==0.3.9",
"bump-my-version >=0.17.1",
"codespell",
"coverage[toml]",
"flake8",
Expand Down
1 change: 0 additions & 1 deletion tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ env_list =
docs
notebooks_doctests
offline-prefetch
; opt-slow
py39-upstream-doctest
py310
py311
Expand Down
Loading