Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add the Deltares World Flood dataset #58

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

asinghvi17
Copy link

@asinghvi17 asinghvi17 commented Mar 28, 2023

Description at https://planetarycomputer.microsoft.com/dataset/deltares-floods and https://nbviewer.org/github/microsoft/AIforEarthDataSets/blob/main/data/deltares-floods.ipynb

Basically a computation of coastal flooding with certain parameters (return time, year) on a specified DEM.
This is ~1gb in size.

Needs docs and tests (if possible)

@codecov
Copy link

codecov bot commented Mar 28, 2023

Codecov Report

Attention: Patch coverage is 0% with 25 lines in your changes missing coverage. Please review.

Project coverage is 69.89%. Comparing base (14423e0) to head (648b12d).
Report is 13 commits behind head on master.

Files with missing lines Patch % Lines
src/deltares/worldflood.jl 0.00% 25 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master      #58      +/-   ##
==========================================
- Coverage   72.25%   69.89%   -2.37%     
==========================================
  Files          19       20       +1     
  Lines         739      764      +25     
==========================================
  Hits          534      534              
- Misses        205      230      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@rafaqz
Copy link
Member

rafaqz commented Mar 28, 2023

If the file is too big you can add tests to run locally but keep them out of CI.

Also note keywords are pretty standardized here, so go for res instead of resolution. We also pass the whole string 10km because the units can be mixed and it gets confusing. The string makes it much clearer what is being specified in a script.

I would also just use year.

Thinking about this, we really need a trait interface for dimensional keywords so we can automatically make series out of them. It's a bit hacky and specific how its currently done, Rasters.jl just special cases month and date keywords. We would need to add year for this to work, but again it wont scale in the long term.

See:
https://github.com/rafaqz/Rasters.jl/blob/main/src/sources/rasterdatasources.jl#L95-L112

@asinghvi17
Copy link
Author

In this case year is actually a parameter, not a time - it indicates the year of reference for sea level.

@asinghvi17
Copy link
Author

asinghvi17 commented Mar 30, 2023

julia> Raster(Deltares{WorldFlood}, nothing; lazy = true)
[ Info: No `name` or `key` keyword provided, using first valid layer with name `:inun`
432000×216000×1 Raster{Union{Missing, Float32},3} inun with dimensions: 
  X Mapped{Float64} Float64[-180.00000003333332, -179.9991667, , 179.99833329985603, 179.99916663318936] ForwardOrdered Regular Points crs: EPSG mappedcrs: EPSG,
  Y Mapped{Float64} Float64[-90.00000003326134, -89.99916669992801, , 89.9983333, 89.99916663333333] ForwardOrdered Regular Points crs: EPSG mappedcrs: EPSG,
  Ti Sampled{Dates.DateTime} Dates.DateTime[Dates.DateTime("2010-01-01T00:00:00")] ForwardOrdered Irregular Points
extent: Extent(X = (-180.00000003333332, 179.99916663318936), Y = (-90.00000003326134, 89.99916663333333), Ti = (Dates.DateTime("2010-01-01T00:00:00"), Dates.DateTime("2010-01-01T00:00:00")))
missingval: missing
crs: EPSG:4326
mappedcrs: EPSG:4326

from file:
/Users/anshul/downloads/Deltares/WorldFlood/GFM_global_NASADEM90m_2050slr_rp0100_masked.nc

Not sure what to put as a layer - might end up being the year.

@rafaqz
Copy link
Member

rafaqz commented Mar 30, 2023

Yeah, we a squishing a wide range of datset shapes into a regular structure, and it doesnt always work so cleanly.

What are the options to be layers?

You can just have a single layer name :inun that will be the no argument default.

@rafaqz
Copy link
Member

rafaqz commented Apr 16, 2024

Just remembered this PR. SRTM already doesn't have a layer as there is only one. This could be the same?

We may need to add/fix dispatch in Rasters.jl to handle that.

@rafaqz
Copy link
Member

rafaqz commented Oct 9, 2024

Bump!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants