Skip to content

Commit

Permalink
Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e…
Browse files Browse the repository at this point in the history
…xtension_arrays
  • Loading branch information
ilan-gold committed Apr 16, 2024
2 parents 4791799 + e9dc53f commit c649362
Show file tree
Hide file tree
Showing 31 changed files with 904 additions and 235 deletions.
11 changes: 5 additions & 6 deletions .github/workflows/ci-additional.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,13 @@ jobs:
runs-on: "ubuntu-latest"
needs: detect-ci-trigger
if: needs.detect-ci-trigger.outputs.triggered == 'false'

defaults:
run:
shell: bash -l {0}

env:
CONDA_ENV_FILE: ci/requirements/environment.yml
PYTHON_VERSION: "3.11"

steps:
- uses: actions/checkout@v4
with:
Expand Down Expand Up @@ -128,7 +127,7 @@ jobs:
python -m mypy --install-types --non-interactive --cobertura-xml-report mypy_report xarray/
- name: Upload mypy coverage to Codecov
uses: codecov/codecov-action@v4.1.0
uses: codecov/codecov-action@v4.2.0
with:
file: mypy_report/cobertura.xml
flags: mypy
Expand Down Expand Up @@ -182,7 +181,7 @@ jobs:
python -m mypy --install-types --non-interactive --cobertura-xml-report mypy_report xarray/
- name: Upload mypy coverage to Codecov
uses: codecov/codecov-action@v4.1.0
uses: codecov/codecov-action@v4.2.0
with:
file: mypy_report/cobertura.xml
flags: mypy39
Expand Down Expand Up @@ -243,7 +242,7 @@ jobs:
python -m pyright xarray/
- name: Upload pyright coverage to Codecov
uses: codecov/codecov-action@v4.1.0
uses: codecov/codecov-action@v4.2.0
with:
file: pyright_report/cobertura.xml
flags: pyright
Expand Down Expand Up @@ -302,7 +301,7 @@ jobs:
python -m pyright xarray/
- name: Upload pyright coverage to Codecov
uses: codecov/codecov-action@v4.1.0
uses: codecov/codecov-action@v4.2.0
with:
file: pyright_report/cobertura.xml
flags: pyright39
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,14 @@ jobs:
run: |
python -c "import xarray"
- name: Restore cached hypothesis directory
uses: actions/cache@v4
with:
path: .hypothesis/
key: cache-hypothesis
enableCrossOsArchive: true
save-always: true

- name: Run tests
run: python -m pytest -n 4
--timeout 180
Expand All @@ -143,7 +151,7 @@ jobs:
path: pytest.xml

- name: Upload code coverage to Codecov
uses: codecov/codecov-action@v4.1.0
uses: codecov/codecov-action@v4.2.0
with:
file: ./coverage.xml
flags: unittests
Expand Down
116 changes: 116 additions & 0 deletions .github/workflows/hypothesis.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
name: Slow Hypothesis CI
on:
push:
branches:
- "main"
pull_request:
branches:
- "main"
types: [opened, reopened, synchronize, labeled]
schedule:
- cron: "0 0 * * *" # Daily “At 00:00” UTC
workflow_dispatch: # allows you to trigger manually

jobs:
detect-ci-trigger:
name: detect ci trigger
runs-on: ubuntu-latest
if: |
github.repository == 'pydata/xarray'
&& (github.event_name == 'push' || github.event_name == 'pull_request')
outputs:
triggered: ${{ steps.detect-trigger.outputs.trigger-found }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 2
- uses: xarray-contrib/ci-trigger@v1
id: detect-trigger
with:
keyword: "[skip-ci]"

hypothesis:
name: Slow Hypothesis Tests
runs-on: "ubuntu-latest"
needs: detect-ci-trigger
if: |
always()
&& (
(github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')
|| needs.detect-ci-trigger.outputs.triggered == 'true'
|| contains( github.event.pull_request.labels.*.name, 'run-slow-hypothesis')
)
defaults:
run:
shell: bash -l {0}

env:
CONDA_ENV_FILE: ci/requirements/environment.yml
PYTHON_VERSION: "3.12"

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for all branches and tags.

- name: set environment variables
run: |
echo "TODAY=$(date +'%Y-%m-%d')" >> $GITHUB_ENV
- name: Setup micromamba
uses: mamba-org/setup-micromamba@v1
with:
environment-file: ci/requirements/environment.yml
environment-name: xarray-tests
create-args: >-
python=${{env.PYTHON_VERSION}}
pytest-reportlog
cache-environment: true
cache-environment-key: "${{runner.os}}-${{runner.arch}}-py${{env.PYTHON_VERSION}}-${{env.TODAY}}-${{hashFiles(env.CONDA_ENV_FILE)}}"

- name: Install xarray
run: |
python -m pip install --no-deps -e .
- name: Version info
run: |
conda info -a
conda list
python xarray/util/print_versions.py
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
- name: Restore cached hypothesis directory
id: restore-hypothesis-cache
uses: actions/cache/restore@v4
with:
path: .hypothesis/
key: cache-hypothesis-${{ runner.os }}-${{ github.run_id }}
restore-keys: |
cache-hypothesis-
- name: Run slow Hypothesis tests
if: success()
id: status
run: |
python -m pytest --hypothesis-show-statistics --run-slow-hypothesis properties/*.py \
--report-log output-${{ matrix.python-version }}-log.jsonl
# explicitly save the cache so it gets updated, also do this even if it fails.
- name: Save cached hypothesis directory
id: save-hypothesis-cache
if: always() && steps.status.outcome != 'skipped'
uses: actions/cache/save@v4
with:
path: .hypothesis/
key: cache-hypothesis-${{ runner.os }}-${{ github.run_id }}

- name: Generate and publish the report
if: |
failure()
&& steps.status.outcome == 'failure'
&& github.event_name == 'schedule'
&& github.repository_owner == 'pydata'
uses: xarray-contrib/issue-from-pytest-log@v1
with:
log-path: output-${{ matrix.python-version }}-log.jsonl
issue-title: "Nightly Hypothesis tests failed"
issue-label: "topic-hypothesis"
2 changes: 1 addition & 1 deletion .github/workflows/upstream-dev-ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ jobs:
run: |
python -m mypy --install-types --non-interactive --cobertura-xml-report mypy_report
- name: Upload mypy coverage to Codecov
uses: codecov/codecov-action@v4.1.0
uses: codecov/codecov-action@v4.2.0
with:
file: mypy_report/cobertura.xml
flags: mypy
Expand Down
8 changes: 4 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,24 +13,24 @@ repos:
- id: mixed-line-ending
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: 'v0.2.0'
rev: 'v0.3.4'
hooks:
- id: ruff
args: ["--fix", "--show-fixes"]
# https://github.com/python/black#version-control-integration
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 24.1.1
rev: 24.3.0
hooks:
- id: black-jupyter
- repo: https://github.com/keewis/blackdoc
rev: v0.3.9
hooks:
- id: blackdoc
exclude: "generate_aggregations.py"
additional_dependencies: ["black==24.1.1"]
additional_dependencies: ["black==24.3.0"]
- id: blackdoc-autoupdate-black
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.8.0
rev: v1.9.0
hooks:
- id: mypy
# Copied from setup.cfg
Expand Down
48 changes: 37 additions & 11 deletions doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,44 @@ What's New
np.random.seed(123456)
.. _whats-new.2024.03.0:
.. _whats-new.2024.04.0:

v2024.03.0 (unreleased)
v2024.04.0 (unreleased)
-----------------------

New Features
~~~~~~~~~~~~


Breaking changes
~~~~~~~~~~~~~~~~


Bug fixes
~~~~~~~~~


Internal Changes
~~~~~~~~~~~~~~~~


.. _whats-new.2024.03.0:

v2024.03.0 (Mar 29, 2024)
-------------------------

This release brings performance improvements for grouped and resampled quantile calculations, CF decoding improvements,
minor optimizations to distributed Zarr writes, and compatibility fixes for Numpy 2.0 and Pandas 3.0.

Thanks to the 18 contributors to this release:
Anderson Banihirwe, Christoph Hasse, Deepak Cherian, Etienne Schalk, Justus Magin, Kai Mühlbauer, Kevin Schwarzwald, Mark Harfouche, Martin, Matt Savoie, Maximilian Roos, Ray Bell, Roberto Chang, Spencer Clark, Tom Nicholas, crusaderky, owenlittlejohns, saschahofmann

New Features
~~~~~~~~~~~~
- Partial writes to existing chunks with ``region`` or ``append_dim`` will now raise an error
(unless ``safe_chunks=False``); previously an error would only be raised on
new variables. (:pull:`8459`, :issue:`8371`, :issue:`8882`)
By `Maximilian Roos <https://github.com/max-sixty>`_.
- Grouped and resampling quantile calculations now use the vectorized algorithm in ``flox>=0.9.4`` if present.
By `Deepak Cherian <https://github.com/dcherian>`_.
- Do not broadcast in arithmetic operations when global option ``arithmetic_broadcast=False``
Expand All @@ -37,16 +67,14 @@ New Features
- Allow creating :py:class:`xr.Coordinates` objects with no indexes (:pull:`8711`)
By `Benoit Bovy <https://github.com/benbovy>`_ and `Tom Nicholas
<https://github.com/TomNicholas>`_.
- Enable plotting of ``datetime.dates``. (:issue:`8866`, :pull:`8873`)
By `Sascha Hofmann <https://github.com/saschahofmann>`_.

Breaking changes
~~~~~~~~~~~~~~~~

- Don't allow overwriting index variables with ``to_zarr`` region writes. (:issue:`8589`, :pull:`8876`).
By `Deepak Cherian <https://github.com/dcherian>`_.

Deprecations
~~~~~~~~~~~~


Bug fixes
~~~~~~~~~
Expand All @@ -73,11 +101,9 @@ Bug fixes
- Warn and return bytes undecoded in case of UnicodeDecodeError in h5netcdf-backend
(:issue:`5563`, :pull:`8874`).
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_.


Documentation
~~~~~~~~~~~~~

- Fix bug incorrectly disallowing creation of a dataset with a multidimensional coordinate variable with the same name as one of its dims.
(:issue:`8884`, :pull:`8886`)
By `Tom Nicholas <https://github.com/TomNicholas>`_.

Internal Changes
~~~~~~~~~~~~~~~~
Expand Down
21 changes: 21 additions & 0 deletions properties/conftest.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,24 @@
import pytest


def pytest_addoption(parser):
parser.addoption(
"--run-slow-hypothesis",
action="store_true",
default=False,
help="run slow hypothesis tests",
)


def pytest_collection_modifyitems(config, items):
if config.getoption("--run-slow-hypothesis"):
return
skip_slow_hyp = pytest.mark.skip(reason="need --run-slow-hypothesis option to run")
for item in items:
if "slow_hypothesis" in item.keywords:
item.add_marker(skip_slow_hyp)


try:
from hypothesis import settings
except ImportError:
Expand Down
Loading

0 comments on commit c649362

Please sign in to comment.