feat: automatically discard insignificant osm extracts and allow user…

…s downloading extracts by name (#133)

* refactor: simplify code for osm extracts index download and add parent data

* chore: remove unused type

* fix: change doctests after changes to extracts

* fix: change osm extracts tests

* fix: change path rename to replace

* chore: change bbbike and geobfarik extracts ids

* chore: send tree example

* fix: change doctests fixtures

* feat: add function for getting osm extract by name

* fix: change geofabrik US parent clause

* feat: add display extracts tree functionality

* feat: change dict-based tree building to pure python printing

* feat: refactor extracts tree and add basic new example notebook

* feat: expand API and change get_extract_by_query logic

* feat: update examples

* chore: modify docstrings

* chore: change docstring fixtures scope

* chore: add option to use full names in the osm extracts tree

* chore: added doctests for new functions

* ci: change docs generation workflows

* feat: add osm extracts displaying to CLI

* chore: change default use_full_names value to True

* feat: add option to use rich pager for long outputs

* ci: debug osm extracts test

* ci: add osm extracts index cache download

* feat: change default osm extract source from geofabrik to any

* ci: debug osm extracts test

* chore: apply refurb suggestions

* ci: debug osm extracts test

* chore: mock rich get console for tests

* chore: remove unused print statement

* ci: revert debug changes

* fix(pre-commit.ci): auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* ci: change pdm-export step

* chore: add metadata markers to the lock file

* ci: change pre-commit config

* ci: change pre-commit config

* chore: remove comment

* chore: ignore geohash license

* chore: change function name

* chore: clear code

* feat: change osm extracts selection algorithm

* chore: change doctests outputs

* feat: add osm extracts querying to cli

* chore: write tests for osm extracts cli api

* chore: add function deprecation

* chore: apply refurb suggestion

* test: debug cli test

* test: debug cli test

* test: debug cli test

* test: debug cli test

* chore: restore cli tests

* chore: refactor import for faster CLI operation

* chore: change examples notebooks

* chore: clean docstrings in new function

* feat: add option to match names with spaces instead of floors

* docs: add new section to README

* fix: add whitespace to readme

* feat: add new basic usage notebook

* chore: change plot color in the cli notebook

* docs: clean osm tags filter notebook

* feat: add geometry_coverage_iou_threshold parameter for choosing OSM extracts

* refactor: simplify osm extracts selection algorithm

* feat: add flowchart for osm extracts search algorithm

* docs: add new osm extracts visualizations

* docs: finish osm extracts examples notebook

* fix: change readme example

* fix: change readme header

* chore: test new readme cli help input

* chore: change readme cli help console output

* chore: remove todo comment

* chore: add no cover pragmas

* feat: expose function display_available_extracts

* chore: add changelog entries

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

.github/workflows/_tests.yml

@@ -18,6 +18,7 @@ jobs:
     env:
       OS: ${{ matrix.os }}
       PYTHON: ${{ matrix.python-version }}
+      COLUMNS: 120
     steps:
       - uses: actions/checkout@v3
       - name: Setup Python ${{ matrix.python-version }}

.github/workflows/ci-dev.yml

@@ -67,6 +67,8 @@ jobs:
             ${{ runner.os }}-pip-dev-
       - name: Install pdm
         run: pip install pdm
+      - name: Regenerate lock with environment markers
+        run: pdm lock --update-reuse --strategy inherit_metadata
       - name: Generate requirements.txt
         run: pdm export --no-default -G docs -G visualization -G cli-dev -f requirements -o requirements.txt
       - name: Install dependencies

.github/workflows/ci-prod.yml

@@ -58,6 +58,8 @@ jobs:
             ${{ runner.os }}-pip-prod-
       - name: Install pdm
         run: pip install pdm
+      - name: Regenerate lock with environment markers
+        run: pdm lock --update-reuse --strategy inherit_metadata
       - name: Generate requirements.txt
         run: pdm export --no-default -G docs -G visualization -G cli-dev -f requirements -o requirements.txt
       - name: Install dependencies

.github/workflows/generate-dev-docs.yml

@@ -27,6 +27,8 @@ jobs:
             ${{ runner.os }}-pip-dev-
       - name: Install pdm
         run: pip install pdm
+      - name: Regenerate lock with environment markers
+        run: pdm lock --update-reuse --strategy inherit_metadata
       - name: Generate requirements.txt
         run: pdm export --no-default -G docs -G visualization -G cli-dev -f requirements -o requirements.txt
       - name: Install dependencies

.github/workflows/manual_tests.yml

@@ -13,6 +13,7 @@ jobs:
     env:
       OS: ${{ matrix.os }}
       PYTHON: ${{ matrix.python-version }}
+      COLUMNS: 120
     steps:
       - uses: actions/checkout@v3
       - name: Setup Python ${{ matrix.python-version }}

.github/workflows/update_usage_plots.yml

@@ -29,6 +29,8 @@ jobs:
             ${{ runner.os }}-pip-dev-
       - name: Install pdm
         run: pip install pdm
+      - name: Regenerate lock with environment markers
+        run: pdm lock --update-reuse --strategy inherit_metadata
       - name: Generate requirements.txt
         run: pdm export --no-default -G docs -G visualization -G cli-dev -f requirements -o requirements.txt
       - name: Install dependencies

.pre-commit-config.yaml

@@ -11,7 +11,7 @@ repos:
     rev: 'v0.5.0'
     hooks:
       - id: ruff
-        args: [--fix, --exit-non-zero-on-fix]
+        args: [--fix, --exit-non-zero-on-fix, --show-fixes]
   - repo: https://github.com/PyCQA/docformatter
     rev: v1.7.5
     hooks:
@@ -28,7 +28,7 @@ repos:
     hooks:
       - id: pdm-lock-check
       - id: pdm-export
-        args: ["-o", "requirements.txt", "--without-hashes", "-d", "-G", "all"]
+        args: ["-o", "requirements.txt", "--without-hashes", "-dG:all"]
         files: ^pdm.lock$
         stages: [manual]
   - repo: https://github.com/kynan/nbstripout

CHANGELOG.md

@@ -7,8 +7,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Functions `convert_osm_extract_to_parquet` and `convert_osm_extract_to_geodataframe` with option to search and download OSM extracts by text query [#119](https://github.com/kraina-ai/quackosm/issues/119)
+- Function for downloading an OSM extract PBF file using a text query (`quackosm.osm_extracts.download_extract_by_query`)
+- Function for displaying available OSM extracts from multiple sources (`quackosm.osm_extracts.display_available_extracts` and `--show-extracts` / `--show-osm-extracts` in cli) in the form of a tree
+- New parameter `geometry_coverage_iou_threshold` (and `--iou-threshold` in cli) to enable configuration of the Intersection over Union metric value sensitivity for covering the geometry with OSM extracts
+- Two new notebook examples for documentation purposes - basic usage and OSM extracts deep dive
+- Improved tests configuration by downloading precalculated extracts indexes from a dedicated repository
+
+### Changed
+
+- Refactored searching OSM extracts for a given geometry filter to utilize Intersection over Union metric [#110](https://github.com/kraina-ai/quackosm/issues/110) [#115](https://github.com/kraina-ai/quackosm/issues/115)
+- Moved multiple modules imports inside certain functions to speed up CLI responsiveness
+- Replaced default `Geofabrik` OSM extract download source with `any` to include all available resources
+- Refactored OSM extracts sources cache files to calculate area in kilometers squared and added `parent` and `file_name` fields
+
+### Deprecated
+
+- Function `find_smallest_containing_extract` from `quackosm.osm_extracts` have been deprecated in favor of `find_smallest_containing_extracts`
+
 ## [0.8.3] - 2024-07-25
 
+### Added
+
+- New function `quackosm.geocode_to_geometry` for quick geocoding of the text query to a geometry
+
 ### Changed
 
 - Replaced `OSMnx` dependency with `GeoPy` for geometry geocoding [#135](https://github.com/kraina-ai/quackosm/issues/135)