Skip to content

Commit

Permalink
CI: Test both Dask's multithreaded and local cluster in CI
Browse files Browse the repository at this point in the history
  • Loading branch information
remi-braun committed Dec 12, 2024
1 parent fb7f636 commit 749ea2d
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
- OPTIM: Don't download files stored on cloud when applying `ci.assert_files_equal` on them
- OPTIM: Offer the ability to give the archived file list directly to `path.get_archived_file_list` and `files.read_archived_file`, as this operation is expensive when done with large archives stored on the cloud (and thus better done only once).
Propagated into `path.get_archived_path`, `path.get_archived_rio_path`, `vectors.read`, `xml.read_archive`, files.read_archived_xml` and `files.read_archived_html`
- CI: Test both Dask's multithreaded and local cluster in CI
- DEPR: Deprecate `xml_regex` and `file_regex` arguments from above-mentioned functions in favor of an harmonized `regex` argument.
- DEPS: Reduce `dask`-related dependencies to the libraries related to this project (abandon `dask[complete]` as we don't need most of the embedded libraries).

Expand Down
6 changes: 5 additions & 1 deletion CI/SCRIPTS/script_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,11 @@ def dask_env_wrapper(*_args, **_kwargs):
"""S3 environment wrapper"""
try:
with dask.get_or_create_dask_client():
print("Using DASK")
print("Using DASK multithreaded.")
function(*_args, **_kwargs)

with dask.get_or_create_dask_client():
print("Using DASK with local cluster")
function(*_args, **_kwargs)
except ImportError:
pass
Expand Down

0 comments on commit 749ea2d

Please sign in to comment.