Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pre-commit.ci] pre-commit autoupdate #30

Merged
merged 3 commits into from
Dec 4, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10"]
python-version: ["3.8", "3.9", "3.10", "3.11"]
runs-on: [ubuntu-latest]

steps:
Expand Down
28 changes: 14 additions & 14 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
repos:
- repo: https://github.com/psf/black
rev: 22.3.0
rev: 23.11.0
hooks:
- id: black-jupyter

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
rev: v4.5.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
Expand All @@ -19,7 +19,7 @@ repos:
- id: trailing-whitespace

- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.9.0
rev: v1.10.0
hooks:
- id: python-check-blanket-noqa
- id: python-check-blanket-type-ignore
Expand All @@ -30,14 +30,14 @@ repos:
- id: rst-inline-touching-normal

- repo: https://github.com/pre-commit/mirrors-prettier
rev: "v2.7.1"
rev: "v3.1.0"
hooks:
- id: prettier
types_or: [yaml, markdown, html, css, scss, javascript, json]
args: [--prose-wrap=always]

- repo: https://github.com/asottile/blacken-docs
rev: v1.12.1
rev: 1.16.0
hooks:
- id: blacken-docs
additional_dependencies: [black==22.3.0]
Expand All @@ -49,26 +49,26 @@ repos:
args: ["-a", "from __future__ import annotations"] # Python 3.7+

- repo: https://github.com/asottile/pyupgrade
rev: v2.34.0
rev: v3.15.0
hooks:
- id: pyupgrade
args: ["--py37-plus"]

- repo: https://github.com/asottile/setup-cfg-fmt
rev: v1.20.1
rev: v2.5.0
hooks:
- id: setup-cfg-fmt

- repo: https://github.com/hadialqattan/pycln
rev: v1.3.3
rev: v2.4.0
hooks:
- id: pycln
additional_dependencies: [click<8.1]
args: [--all]
stages: [manual]

- repo: https://github.com/asottile/yesqa
rev: v1.3.0
rev: v1.5.0
hooks:
- id: yesqa
exclude: docs/conf.py
Expand All @@ -77,26 +77,26 @@ repos:
- flake8-print

- repo: https://github.com/pycqa/flake8
rev: 4.0.1
rev: 6.1.0
hooks:
- id: flake8
exclude: docs/conf.py
additional_dependencies: *flake8_dependencies

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.4.1
rev: v1.7.1
hooks:
- id: mypy
files: src
args: []

- repo: https://github.com/codespell-project/codespell
rev: v2.1.0
rev: v2.2.6
hooks:
- id: codespell

- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.8.0.4
rev: v0.9.0.6
hooks:
- id: shellcheck

Expand All @@ -109,7 +109,7 @@ repos:
exclude: .pre-commit-config.yaml

- repo: https://github.com/mgedmin/check-manifest
rev: "0.48"
rev: "0.49"
hooks:
- id: check-manifest
stages: [manual]
10 changes: 3 additions & 7 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ author_email = [email protected]
maintainer = CoffeaTeam admins
maintainer_email = [email protected]
license = BSD-3-Clause
license_file = LICENSE
license_files = LICENSE
platforms =
Any
classifiers =
Expand All @@ -21,10 +21,6 @@ classifiers =
Programming Language :: Python
Programming Language :: Python :: 3
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Topic :: Scientific/Engineering
project_urls =
Documentation = https://fsspec_xrootd.readthedocs.io/
Expand All @@ -37,7 +33,7 @@ packages = find:
install_requires =
fsspec
typing-extensions>=3.7;python_version<'3.8'
python_requires = >=3.7
python_requires = >=3.8
include_package_data = True
package_dir =
=src
Expand All @@ -61,7 +57,7 @@ test =
pytest>=6

[flake8]
extend-ignore = E203, E501, E722, B950
extend-ignore = E203, E501, E722, B950, B905
select = C,E,F,W,T,B,B9,I
per-file-ignores =
tests/*: T
Expand Down
28 changes: 15 additions & 13 deletions src/fsspec_xrootd/xrootd.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from __future__ import annotations

Check warning on line 1 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Missing module docstring

import asyncio
import io
Expand All @@ -9,24 +9,27 @@
from functools import partial
from typing import Any, Callable, Iterable

from fsspec.asyn import ( # type: ignore[import]
from fsspec.asyn import ( # type: ignore[import-not-found]
AsyncFileSystem,
_run_coros_in_chunks,
sync_wrapper,
)
from fsspec.spec import AbstractBufferedFile # type: ignore[import]
from XRootD import client # type: ignore[import]
from XRootD.client.flags import ( # type: ignore[import]
from fsspec.spec import AbstractBufferedFile # type: ignore[import-not-found]
from XRootD import client # type: ignore[import-not-found]

Check failure on line 18 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Unable to import 'XRootD'
from XRootD.client.flags import ( # type: ignore[import-not-found]

Check failure on line 19 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Unable to import 'XRootD.client.flags'
DirListFlags,
MkDirFlags,
OpenFlags,
QueryCode,
StatInfoFlags,
)
from XRootD.client.responses import HostList, XRootDStatus # type: ignore[import]
from XRootD.client.responses import ( # type: ignore[import-not-found]

Check failure on line 26 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Unable to import 'XRootD.client.responses'
HostList,
XRootDStatus,
)


class ErrorCodes(IntEnum):

Check warning on line 32 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Missing class docstring
INVALID_PATH = 400


Expand All @@ -34,7 +37,7 @@
future: asyncio.Future[tuple[XRootDStatus, Any]],
status: XRootDStatus,
content: Any,
servers: HostList,

Check warning on line 40 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Unused argument 'servers'
) -> None:
"""Sets result of _async_wrap() future.

Expand All @@ -53,7 +56,7 @@
return
try:
future.get_loop().call_soon_threadsafe(future.set_result, (status, content))
except Exception as exc:

Check warning on line 59 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Catching too general exception Exception
future.get_loop().call_soon_threadsafe(future.set_exception, exc)


Expand Down Expand Up @@ -141,8 +144,7 @@
return deets


class XRootDFileSystem(AsyncFileSystem): # type: ignore[misc]

Check warning on line 147 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Missing class docstring

Check warning on line 147 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Method '_cp_file' is abstract in class 'AsyncFileSystem' but is not overridden in child class 'XRootDFileSystem'

Check warning on line 147 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Method '_get_file' is abstract in class 'AsyncFileSystem' but is not overridden in child class 'XRootDFileSystem'

Check warning on line 147 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Method '_pipe_file' is abstract in class 'AsyncFileSystem' but is not overridden in child class 'XRootDFileSystem'

Check warning on line 147 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Method '_put_file' is abstract in class 'AsyncFileSystem' but is not overridden in child class 'XRootDFileSystem'

Check warning on line 147 in src/fsspec_xrootd/xrootd.py

View workflow job for this annotation

GitHub Actions / Format

Method 'cp_file' is abstract in class 'AbstractFileSystem' but is not overridden in child class 'XRootDFileSystem'

protocol = "root"
root_marker = "/"
default_timeout = 60
Expand Down Expand Up @@ -195,12 +197,12 @@

@classmethod
def _strip_protocol(cls, path: str | list[str]) -> Any:
if type(path) == str:
if isinstance(path, str):
if path.startswith(cls.protocol):
return client.URL(path).path.rstrip("/") or cls.root_marker
# assume already stripped
return path.rstrip("/") or cls.root_marker
elif type(path) == list:
elif isinstance(path, list):
return [cls._strip_protocol(item) for item in path]
else:
raise ValueError("Strip protocol not given string or list")
Expand Down Expand Up @@ -474,7 +476,7 @@
coros, batch_size=batch_size, nofiles=True
)
result_bufs = []
for (status, buffers) in results:
for status, buffers in results:
if not status.ok:
raise OSError(
f"File did not vector_read properly: {status.message}"
Expand All @@ -498,7 +500,6 @@
batch_size: int | None = None,
**kwargs: Any,
) -> list[bytes]:

# TODO: on_error
if max_gap is not None:
# use utils.merge_offset_ranges
Expand Down Expand Up @@ -546,7 +547,6 @@
cache_options: dict[Any, Any] | None = None,
**kwargs: Any,
) -> XRootDFile:

return XRootDFile(
self,
path,
Expand Down Expand Up @@ -599,8 +599,10 @@
**kwargs,
)
if compression is not None:
from fsspec.compression import compr # type: ignore[import]
from fsspec.core import get_compression # type: ignore[import]
from fsspec.compression import compr # type: ignore[import-not-found]
from fsspec.core import ( # type: ignore[import-not-found]
get_compression,
)

compression = get_compression(path, compression)
compress = compr[compression]
Expand Down
Loading