Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Advanced merging and publishing of partitioned job results #56

Open
soxofaan opened this issue Aug 3, 2022 · 2 comments
Open

Advanced merging and publishing of partitioned job results #56

soxofaan opened this issue Aug 3, 2022 · 2 comments

Comments

@soxofaan
Copy link
Member

soxofaan commented Aug 3, 2022

from #52

  • metadata of final result: properly merge metadata of subjobs (e.g. STAC collection)
  • "publication" of results: e.g. VRT generation of (partial) result, or upload to a STAC catalog
@m-mohr
Copy link
Member

m-mohr commented Feb 24, 2023

Do I understand correctly that it is planned if a job is split and executed at EODC and VITO, the results stay in the data centers and the aggregator combines them just by providing a Collection pointing to the sub results? (If yes, great. If no, why?)

@soxofaan
Copy link
Member Author

Indeed, merging at the metadata level and keeping the actual data as-is would at least be the initial and obvious approach.

I remember people discussing in the forum (I think) about a feature to also stitch sub-results to a single geotiff or netcdf, but that's a lot less trivial to achieve, and no concrete plans there so far.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants