Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Generalize schema configuration and dumping #952

Merged
merged 1 commit into from
Jan 7, 2025

Conversation

mferrera
Copy link
Collaborator

@mferrera mferrera commented Jan 3, 2025

Toward #895 + schema versioning

This creates a pattern for defining schema versions, filenames, and paths. These can then be used throughout the code without being hardcoded strings.

The dev and prod URLs are also now present in the code. In the main branch the $id URLs now point to the dev enviroment, as this environment tracks main anyway. The idea is that these URLs switching to the production ones will be a step during the promotion to the staging environment. Currently this is done "secretly" with a shell script in the docker start-up instance, but as the number of schemas grows, I think it makes more sense to have this be explicit and transparent. What you see is what you get.

Follow-up PRs:

  • Implement SchemBase on InplaceVolumesResult
  • Adjust update-schema to also check and dump that schema
  • Adjust update-schema to take a --release flag
    • This flag will change the $id to use the production URL
    • It's used when placing the schema into the staging branch (will be tested)
  • Remove the behind-the-scenes Docker script modifying this field in the schemas

Then, additional schemas can be easily dumped. They will all go into the schemas/ directory, and the contents of this directory will mirror what's on the Radix instance.

Effectively that will accomplish schema versioning as well.

@mferrera mferrera self-assigned this Jan 3, 2025
@mferrera mferrera marked this pull request as ready for review January 3, 2025 13:17
@mferrera mferrera requested review from tnatt and slangeveld January 3, 2025 13:22
@mferrera mferrera force-pushed the dump-product-schema branch from db3b4fa to f6720fd Compare January 3, 2025 14:01
@mferrera mferrera force-pushed the dump-product-schema branch 2 times, most recently from f4d578d to 09b96d6 Compare January 7, 2025 12:41
@mferrera mferrera requested a review from tnatt January 7, 2025 12:46
Copy link
Collaborator

@tnatt tnatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One step closer 🚀

src/fmu/dataio/_metadata.py Show resolved Hide resolved
src/fmu/dataio/_model/schema.py Outdated Show resolved Hide resolved

@classmethod
def __init_subclass__(cls, **kwargs: dict[str, Any]) -> None:
super().__init_subclass__(**kwargs)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always learn something new 🙂 never seen this __init_subclass__ before

Copy link
Collaborator Author

@mferrera mferrera Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It didn't feel quite right to use use a Pydantic model as an abstract interface with methods (and Pydantic also didn't seem to like using an ABC as a mixin), so Pydantic-light it was

This creates a pattern for defining schema versions, filenames, and
paths. These can then be used throughout the code without being
hardcoded strings.

The dev and prod URLs are also now present in the code. In the main
branch the `$id` URLs now point to the dev enviroment, as this
environment tracks main anyway. The idea is that these URLs switching to
the production ones will be a step during the promotion to the staging
environment. Currently this is done "secretly" with a shell script but
as the number of schemas grows, I think it makes more sense to have this
be explicit.
@mferrera mferrera force-pushed the dump-product-schema branch from 09b96d6 to 5e00283 Compare January 7, 2025 13:48
@mferrera mferrera merged commit e604248 into equinor:main Jan 7, 2025
16 checks passed
@mferrera mferrera deleted the dump-product-schema branch January 7, 2025 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants