Processes/Collections: Return supported back-ends as a list?

[m-mohr]

Would it make sense to return a list of back-ends in the JSON metadata that support a certain collection / process?

For example:

{
  "id": "mask",
  "backends": ["eodc", "vito"], // list id that is exposed in capabilities?
  ...
}

This would allow us to add a "compliance" matrix to the documentation, which a user can get a quick overview from what is supported where.

For example (just listed random names with randomly placed yes/no icons):

This should be relatively easy to implement both server- and client-side, I think.

Thoughts? @jdries @soxofaan

[soxofaan]

yes, it would be straightforward to add that metadata.
It's just a matter of standardizing the field name and format.

FYI, for collections provided by multiple back-ends there is already a summaries entry at key "provider:backend", e.g. see this test:

openeo-aggregator/tests/test_backend.py

Line 146 in 8e50569

"summaries": {"provider:backend": ["b1", "b2"]},

(because there is no pure id-based overlap yet between VITO and EODC, this is not yet visible in production)

[m-mohr]

I don't think we need to standardize this yet, this can be a platform-specific API extension and could be implemented right away.

[soxofaan]

I don't meant to standardize on level of openEO API (yet), just agree between aggregator and client/docs/website on a scheme that's reasonably future proof

[m-mohr]

I've updated the example above to "backends": ["eodc", "vito"], // list id that is exposed in capabilities? - It could be any name and I don't have a strong preference, but I think what I would list in the array are the back-end IDs from the capabilities. Unfortunately, the ids for the back-ends are not very wisely chosen and it woudl result in: "backends": ["0.0.1", "vitoremotesensingopeneoapi-1.0.0"] and I would have hoped to get something like "vito" or "eodc" returned. Other than that, I think we could continue with the implementation. It's new and we can just implement it client-side against whatever is returned by the aggregator.

[soxofaan]

FYI: the aggregator uses "internal" ids for the back-ends like this:

openeo-aggregator/src/openeo_aggregator/config.py

Lines 44 to 49 in 8e50569

	DEFAULT_CONFIG = AggregatorConfig(
	aggregator_backends={
	"vito": "https://openeo.vito.be/openeo/1.0/",
	# "creo": "https://openeo.creo.vito.be/openeo/1.0/",
	"eodc": "https://openeo.eodc.eu/v1.0/",
	# "eodcdev": "https://openeo-dev.eodc.eu/v1.0/",

which is probably better (and conflict-free by design) than the self-chosen back-end ids

[m-mohr]

URL as id is also a good idea, maybe the connect URL instead of the versioned one?

[m-mohr]

Some additional ideas I had this morning: https://github.com/openEOPlatform/architecture-docs/issues/85#issuecomment-945554415

Especially: In STAC world "provider:backend" is a bit strange, better use something project-specific, e.g. "openeo:backend". This is a bit different from what I said before, sorry.

[m-mohr]

Would it also make sense to add the back-end flag to batch jobs and other back-end-specific resources?
We were wondering about it today how we can ensure a somewhat good UX for e.g. the batch job overview once a back-end is not available? Then the number of jobs could go down, which could be surprising to a user...

[soxofaan]

Especially: In STAC world "provider:backend" is a bit strange, better use something project-specific, e.g. "openeo:backend". This is a bit different from what I said before, sorry.

FYI: this ticket #10 and PR #11 just introduce top-level key "backends" for collections and processes, while the "provider:backend" reference is (at the moment) a "summaries" property for load_collection filtering .
Do you mean to also use "openeo:backends" (plural) here?

[m-mohr]

I'm thinking about that right now, especially about the "extension" behavior in the API. From STAC I'm used to extension:field_name, so could continue this here, which would lead to something like federation:backends which I'd use throughout the whole API then. But I'm not 100% set on using a prefix and I'd also be okay to just use backends throughout the whole API, too. We don't necessarily need to follow STAC here, not even in summaries.

[soxofaan]

I would also favor namespaced scheme like extension:field_name , that feels more future proof,
but I'd let you choose as you have more experience in the STAC ecosystem

[m-mohr]

Good to know, then I'll proceed with that. In the past we had some voices that aimed to remove the : from keys, but they were quiet recently so I'll proceed with that. This is not really a STAC topic though (except that we re-use the field also in Collection summaries).

[soxofaan]

this is apparently already fixed/covered under #76/#77/#95