MAINT: Refactor 'schema'/'internal' classes #962
+188
−223
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,18 +1,32 @@ | ||
| """ | ||
| This module contains models used to output the metadata that sit beside the exported | ||
| data. | ||
|
|
||
| It contains internal data structures that are designed to depend on external modules, | ||
| but not the other way around. This design ensures modularity and flexibility, allowing | ||
| external modules to be potentially separated into their own repositories without | ||
| dependencies on the internals. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING, Final, List, Literal, Optional, Union | ||
|
|
||
| from pydantic import ( | ||
| AnyHttpUrl, | ||
| BaseModel, | ||
| Field, | ||
| ) | ||
|
|
||
| from ._logging import null_logger | ||
| from ._model import data, fields | ||
| from ._model.enums import FMUClass | ||
| from ._model.global_configuration import GlobalConfiguration | ||
| from ._model.product import Product | ||
| from ._model.root import CaseMetadata, FmuResultsSchema, ObjectMetadata | ||
| from .exceptions import InvalidMetadataError | ||
| from .providers._filedata import FileDataProvider | ||
| from .providers.objectdata._base import UnsetData | ||
| from .providers.objectdata._provider import objectdata_provider_factory | ||
|
|
||
| if TYPE_CHECKING: | ||
|
|
@@ -24,6 +38,38 @@ | |
| logger: Final = null_logger(__name__) | ||
|
|
||
|
|
||
| class JsonSchemaMetadata(BaseModel): | ||
| """Mixin to inject the $schema field into exported metadata.""" | ||
|
|
||
| schema_: AnyHttpUrl = Field( | ||
| default_factory=lambda: AnyHttpUrl(FmuResultsSchema.url()), | ||
| alias="$schema", | ||
| frozen=True, | ||
| ) | ||
|
|
||
|
|
||
| class ObjectMetadataExport(JsonSchemaMetadata, ObjectMetadata, populate_by_name=True): | ||
| """Wraps the schema ObjectMetadata, adjusting some values to optional for pragmatic | ||
| purposes when exporting metadata.""" | ||
|
|
||
| # These type ignores are for making the field optional | ||
| fmu: Optional[fields.FMU] # type: ignore | ||
| access: Optional[fields.SsdlAccess] # type: ignore | ||
| masterdata: Optional[fields.Masterdata] # type: ignore | ||
| # !! Keep UnsetData first in this union | ||
| data: Union[UnsetData, data.AnyData] # type: ignore | ||
| preprocessed: Optional[bool] = Field(alias="_preprocessed", default=None) | ||
|
|
||
|
Comment on lines
+51
to
+61
There was a problem hiding this comment. We are no longer creating invalid metadata since version 2.6.0 #793. ➡️ hence we can soon drop the optional fields here: 💥 🙂 There was a problem hiding this comment. I didn't fully realize this... maybe we can have a chat about it |
||
|
|
||
| class CaseMetadataExport(JsonSchemaMetadata, CaseMetadata, populate_by_name=True): | ||
| """Adds the optional description field for backward compatibility.""" | ||
|
|
||
| class_: Literal[FMUClass.case] = Field( | ||
| default=FMUClass.case, alias="class", title="metadata_class" | ||
| ) | ||
| description: Optional[List[str]] = Field(default=None) | ||
|
|
||
|
|
||
| def _get_meta_filedata( | ||
| dataio: ExportData, | ||
| obj: types.Inferrable, | ||
|
|
@@ -72,7 +118,7 @@ def generate_export_metadata( | |
| dataio: ExportData, | ||
| fmudata: FmuProvider | None = None, | ||
| product: Product | None = None, | ||
| ) -> ObjectMetadataExport: | ||
| """ | ||
| Main function to generate the full metadata | ||
|
|
||
|
|
@@ -102,7 +148,7 @@ def generate_export_metadata( | |
|
|
||
| objdata = objectdata_provider_factory(obj, dataio, product) | ||
|
|
||
| return ObjectMetadataExport( # type: ignore[call-arg] | ||
| class_=objdata.classname, | ||
| fmu=_get_meta_fmu(fmudata) if fmudata else None, | ||
| masterdata=( | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thought for a separate PR, could this be removed from here and used in the
_model.rootinstead? 🙂Right now we have a mismatch between the
$schemadefined in the main schema and the "exported". That would take us one step closer to removing the need for a separate model for the exported metadata, I look forward to the day we can align them fully!There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it can't actually, this is a bit confusing:
It's confusing because our schema is getting validated by a 'worldwide' schema (validating that it's a valid schema), while our metadata is validated by our schema
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
annoying 😩 .. and confusing yes