-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add tests for Xarray io functions #30
Conversation
Fixed merge conflict between remote branches local vs cryocloud on updated main.py search
Thanks for starting this @Jack-Hayes ! If adding a new run-time dependency (odc.stac) you'll want to add it here (with version constraints) Line 33 in f366a45
And then also lower in that file (with "*") so that pixi recognizes it: Line 211 in f366a45
I moved the 'contributing' guide from the readme to this page in the docs recently https://coincident.readthedocs.io/en/latest/user_guide/contribute.html |
src/coincident/search/main.py
Outdated
|
||
# Used to access formatters | ||
from pystac_client.item_search import ItemSearch as _ItemSearch | ||
from shapely.validation import make_valid |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice. I didn't know about this function. Currently, the code just checks for ccw ordering
coincident/src/coincident/search/main.py
Lines 80 to 82 in f366a45
if ( | |
not shapely_geometry.exterior.is_ccw | |
): # Apparently NASA CMR enforces polygon CCW order |
Out of curiosity, were did you encounter an invalid geometry where this was needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A lot of flight extent polygons from the NOAA Costal LiDAR catalog are snake-like and non-contiguous which resulted 'invalid' polygons even after a simplify()
with a tolerance of 0.01 (and I believe some of the really bad flight extent polygons were still invalid after simplify(0.01).make_valid()
, so I buffered those by a small amount before searching for PCD sites).
There were also issues with simplify(0.01)
making holes in some of the NEON flight extent polygons which were created by merging all of the tile extents in the flight, and make_valid()
fixed some of these (see one of our Slack threads on this!). I ended up using a minor buffer
on these as well for initial PCD site searches rather than simplify
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gotcha thanks! Could leave this for a separate PR, but if including, you could add one of these polygons to test (can add as a geojson in a 'tests/data' subfolder).
src/coincident/search/main.py
Outdated
max(y for x, y in coordinates), | ||
) | ||
items = stac.to_pystac_items(gf) | ||
ds = odc.stac.load( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will be slow for large areas because by default chunks=None
. chunks='auto'
chunks={}
requires dask, but should to be much faster.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, great point 😊 I'm assuming we'll eventually add support for dask so I might as well incorporate it here?
Related: #22
Related: #23
This PR implements returning an xarray.core.dataset.Dataset raster for cop30 and worldcover datasets rather than the respective item metadata. There are pretty big issues with efficiency in terms of loading in these rasters for large AOIs. I haven't implemented any GDAL enhancements (OSGeo/gdal#11317) or addressed odc-stac for accepting the geodataframe/stac-geoparquet directly (I'm still using to_pystac_items here).