Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add automatic downscaling when encountering OOM error #51

Merged
merged 13 commits into from
Mar 7, 2024
Merged
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed

- Added automatic downscaling of the `rows_per_bucket` parameter for ways grouping operation [#50](https://github.com/kraina-ai/quackosm/issues/50)

## [0.4.4] - 2024-02-14

### Fixed
Expand Down
8 changes: 5 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,9 @@ preview = true
[tool.ruff]
line-length = 100
target-version = "py39"
extend-exclude = ["old"]

[tool.ruff.lint]
select = [
"E",
"W", # pycodestyle
Expand All @@ -138,12 +141,11 @@ select = [
# "ANN", # flake8-annotations
]
ignore = ["D212"]
extend-exclude = ["old"]

[tool.ruff.pydocstyle]
[tool.ruff.lint.pydocstyle]
convention = "google"

[tool.ruff.pycodestyle]
[tool.ruff.lint.pycodestyle]
max-doc-length = 100

[tool.mypy]
Expand Down
16 changes: 9 additions & 7 deletions quackosm/_osm_tags_filters.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
def merge_osm_tags_filter(
osm_tags_filter: Union[
OsmTagsFilter, GroupedOsmTagsFilter, Iterable[OsmTagsFilter], Iterable[GroupedOsmTagsFilter]
]
],
) -> OsmTagsFilter:
"""
Merge OSM tags filter into `OsmTagsFilter` type.
Expand All @@ -52,12 +52,14 @@
elif is_expected_type(osm_tags_filter, GroupedOsmTagsFilter):
return _merge_grouped_osm_tags_filter(cast(GroupedOsmTagsFilter, osm_tags_filter))
elif is_expected_type(osm_tags_filter, Iterable):
return _merge_multiple_osm_tags_filters([
merge_osm_tags_filter(
cast(Union[OsmTagsFilter, GroupedOsmTagsFilter], sub_osm_tags_filter)
)
for sub_osm_tags_filter in osm_tags_filter
])
return _merge_multiple_osm_tags_filters(

Check warning on line 55 in quackosm/_osm_tags_filters.py

View check run for this annotation

Codecov / codecov/patch

quackosm/_osm_tags_filters.py#L55

Added line #L55 was not covered by tests
[
merge_osm_tags_filter(
cast(Union[OsmTagsFilter, GroupedOsmTagsFilter], sub_osm_tags_filter)
)
for sub_osm_tags_filter in osm_tags_filter
]
)

raise AttributeError(
"Provided tags don't match required type definitions"
Expand Down
9 changes: 9 additions & 0 deletions quackosm/_rich_progress.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@
TOTAL_STEPS = 33


def log_message(message: str) -> None:
try: # pragma: no cover
from rich import print as rprint

rprint(message)
except ImportError:
print(message)

Check warning on line 17 in quackosm/_rich_progress.py

View check run for this annotation

Codecov / codecov/patch

quackosm/_rich_progress.py#L16-L17

Added lines #L16 - L17 were not covered by tests


class TaskProgressSpinner:
def __init__(self, step_name: str, step_number: str):
self.step_name = step_name
Expand Down
Loading
Loading