diff --git a/core/esmf-aspect-meta-model-python/README.md b/core/esmf-aspect-meta-model-python/README.md index 73ea992..913c6d7 100644 --- a/core/esmf-aspect-meta-model-python/README.md +++ b/core/esmf-aspect-meta-model-python/README.md @@ -1,84 +1,141 @@ The Aspect Model Loader as part of the Python SDK provided by the [*Eclipse Semantic Modeling Framework*]( https://projects.eclipse.org/projects/dt.esmf). + +* [An Aspect of the Meta Model](#an-aspect-of-the-meta-model) + * [Set Up SAMM Aspect Meta Model](#set-up-samm-aspect-meta-model) + * [Install poetry](#install-poetry) + * [Install project dependencies](#install-project-dependencies) + * [Download SAMM files](#download-samm-files) + * [Download SAMM release](#download-samm-release) + * [Download SAMM branch](#download-samm-branch) + * [Download SAMM CLI](#download-samm-cli) + * [Download test models](#download-test-models) + * [Aspect Meta Model Loader usage](#aspect-meta-model-loader-usage) + * [Samm Units](#samm-units) + * [SAMM CLI wrapper](#samm-cli-wrapper) +* [Scripts](#scripts) +* [Automation Tasks](#automation-tasks) + * [tox](#tox) + * [GitHub actions](#github-actions) + * [Check New Pull Request](#check-new-pull-request) + * [Build release](#build-release) + + # An Aspect of the Meta Model The `esmf-aspect-model-loader` package provides the Python implementation for the SAMM Aspect Meta Model, or SAMM. Each Meta Model element and each Characteristic class is represented by an interface with a corresponding implementation. -## Usage +## Set Up SAMM Aspect Meta Model -An Aspect of the Meta Model can be created as follows using the provided `AspectInstantiator`. +Before getting started to use the `esmf-aspect-model-loader` library you need to apply some set up actions: -``` -aspect_loader = AspectLoader() -model_elements = aspect_loader.load_aspect_model("absolute/path/to/turtle.ttl"); -aspect = model_elements[0] -``` +Required +- [Install poetry](#install-poetry) +- [Install project dependencies](#install-project-dependencies) +- [Download SAMM files](#download-samm-files) +Optional +- [Download SAMM CLI](#download-samm-cli) (need to use SAMM CLI functions) +- [Download SAMM test models](#download-test-models) (for running integration tests) -or +### Install poetry -``` -aspect_loader = AspectLoader() -aspect_urn = "urn:samm:org.eclipse.esmf.samm:aspect.model:0.0.1#AspectName" -aspect = aspect_loader.load_aspect_model("absolute/path/to/turtle.ttl", aspect_urn); +`Poetry` used as a dependency management for the `esmf-aspect-model-loader`. Follow the next [instruction](https://python-poetry.org/docs/#installation) + to install it. + +To check the poetry version run: +```console +poetry --version ``` -## Automatic Deployment +### Install project dependencies -A [GitHub action called 'Release'](https://github.com/eclipse-esmf/esmf-sdk-py-aspect-model-loader/actions/workflows/tagged_release.yml) -has been set up for the `esmf-aspect-model-loader`. This action checks the code quality by running tests, the [static type checker MyPy](https://github.com/python/mypy) and -the [code formatter 'Black'](https://github.com/psf/black). +Poetry provides convenient functionality for working with dependencies in the project. +To automatically download and install all the necessary libraries, just run one command +```console +poetry install +``` -## Set Up SAMM Aspect Meta Model for development +### Download SAMM files -In order to download the SAMM sources, it is required to run `poetry install` once in the `esmf-aspect-model-loader` -module. There are two possibilities to download the SAMM files and extract the Turtle sources for the Meta Model. +There are two possibilities to download the SAMM files and extract the Turtle sources for the Meta Model: +SAMM release or SAMM branch -### Possibility 1 (downloading a release) +#### Download SAMM release -The `download_samm_release` script may be executed with +This script downloads a release JAR-file from GitHub, extracts them for further usage in the Aspect Model Loader: -``` +To run script, execute the next command. +```console poetry run download-samm-release -``` +``` +The version of the SAMM release is specified in the python script. + +Link to all Releases: [SAMM Releases](https://github.com/eclipse-esmf/esmf-semantic-aspect-meta-model/releases) -It downloads a release JAR-file from GitHub and extracts the SAMM Files. -The version is specified in the python script. +#### Download SAMM branch -Link to all Releases: https://github.com/eclipse-esmf/esmf-semantic-aspect-meta-model/releases +The script uses the GitHub API and downloads the files from the `main` GitHub branch. -### Possibility 2 (downloading from the repository) +If the script is run in a pipeline, it uses a GitHub token to authorize. If the script is run locally, +the API is called without a token. This may cause problems because unauthorized API calls are limited. -It may happen that there is no .jar file that is up to date with the changes of the SAMM. -This script is an alternative to the `download_samm_release.py` and extracts the files from the repository -directly instead of using the newest release. +Run the next command to download and start working with the Aspect Model Loader. +```console +poetry run download-samm-branch +``` +Link to all branches: [SAMM Releases](https://github.com/eclipse-esmf/esmf-semantic-aspect-meta-model/branches) -The script uses the GitHub API and downloads the files from the `main` branch. If the script is run in a -pipeline, it uses a GitHub token to authorize. If the script is run locally, the API is called without a token. -This may cause problems because unauthorized API calls are limited. +### Download SAMM CLI -This script can be executed with +The SAMM CLI is a command line tool for the validation of Aspect Models and the generation of artifacts, +such as documentation or code, from Aspect Models. +Run the next command to download the SAMM client: +```console +poetry run download-samm-cli ``` -poetry run download-samm-branch + +How to work with SAMM CLI from the Aspect Model Loader you can read in the chapter [SMM ClI wrapper](#samm-cli-wrapper). + +More detailed information about SAMM CLI functionality can be found in the +[SAMM CLI documentation](https://eclipse-esmf.github.io/esmf-developer-guide/tooling-guide/samm-cli.html). + +### Download test models + +This script is downloading esmf-test-aspect-models from the Maven Central. Version of the package is specified +in the script. + +Test models reused in integration tests suite + +Command to run script: +```console +poetry run download-test-models ``` -to download and start working with the Aspect Model Loader. -## Semantic Aspect Meta Model +## Aspect Meta Model Loader usage -To be able to use SAMM meta model classes you need to download the corresponding files. -Details are described in [Set up SAMM Aspect Meta Model for development](#set-up-samm-aspect-meta-model-for-development). +An Aspect of the Meta Model can be loaded as follows: +```python +from esmf_aspect_meta_model_python import AspectLoader -This module contains classes for working with Aspect data. +loader = AspectLoader() +model_elements = loader.load_aspect_model("absolute/path/to/turtle.ttl") +aspect = model_elements[0] -SAMM meta model contains: -- SammUnitsGraph: provide a functionality for working with units([units.ttl](./esmf_aspect_meta_model_python/samm_aspect_meta_model/samm/unit/2.1.0/units.ttl)) data +# or you can provide an Aspect URN -### SammUnitsGraph +loader = AspectLoader() +aspect_urn = "urn:samm:org.eclipse.esmf.samm:aspect.model:0.0.1#AspectName" +model_elements = loader.load_aspect_model("absolute/path/to/turtle.ttl", aspect_urn) +aspect = model_elements[0] +``` -The class contains functions for accessing units of measurement. +## Samm Units + +SAMMUnitsGraph is a class contains functions for accessing units of measurement. ```python from esmf_aspect_meta_model_python.samm_meta_model import SammUnitsGraph @@ -92,3 +149,78 @@ units_graph.print_description(unit_data) # ... # symbol: V ``` + +## SAMM CLI wrapper + +The SAMM CLI is a command line tool provided number of functions for working with Aspect Models. + +ore detailed information about SAMM CLI functionality can be found in the +[SAMM CLI documentation](https://eclipse-esmf.github.io/esmf-developer-guide/tooling-guide/samm-cli.html). + +Python aspect Model Loader provide a wrapper class to be able to call SAMM CLI functions from the python code. +For instance, validation a model can be done with the next code snippet + +```python +from esmf_aspect_meta_model_python.samm_cli_functions import SammCli + +samm_cli = SammCli() +model_path = "Paht_to_the_model//Model.ttl" +samm_cli.validate(model_path) +# Input model is valid +``` + +List of SAMMCLI functions: +- validate +- to_openapi +- to_schema +- to_json +- to_html +- to_png +- to_svg + +# Scripts + +The Aspect Model Loader provide scripts for downloading some additional code and data. +Provided scripts: + - download-samm-release + - download-samm-branch + - download-samm-cli + - download-test-models + +All scripts run like a poetry command. The poetry is available from the folder where [pyproject.toml](pyproject.toml) +is located. + +# Automation Tasks +## tox + +`tox` is used for the tests automation purpose. There are two environments with different purposes and tests can +be running with the tox: +- pep8: static code checks (PEP8 style) with MyPy and Black +- py310: unit and integration tests + +```console +# run all checks use the next command +poetry run tox + +# run only pep8 checks +poetry run tox -e pep8 + +# run tests +poetry run tox -e py310 +``` + +## GitHub actions + +There are two actions on the GitHub repo: +- [Check New Pull Request](../../.github/workflows/push_request_check.yml) +- [Build release](../../.github/workflows/tagged_release.yml) + +### Check New Pull Request +This action run after creation or updating a pull request and run all automation tests with tox command. + +### Build release +Prepare and publish a new release for the `esmf-aspect-model-loader` to the PyPi: +[esmf-aspect-model-loader](https://pypi.org/project/esmf-aspect-model-loader/.) + +A list of the available releases on the GitHub: +[Releases](https://github.com/eclipse-esmf/esmf-sdk-py-aspect-model-loader/releases). diff --git a/core/esmf-aspect-meta-model-python/esmf_aspect_meta_model_python/loader/aspect_loader.py b/core/esmf-aspect-meta-model-python/esmf_aspect_meta_model_python/loader/aspect_loader.py index 7d65392..77909fb 100644 --- a/core/esmf-aspect-meta-model-python/esmf_aspect_meta_model_python/loader/aspect_loader.py +++ b/core/esmf-aspect-meta-model-python/esmf_aspect_meta_model_python/loader/aspect_loader.py @@ -52,17 +52,18 @@ def convert_file_path(file_path: Union[str, Path]) -> str: return file_path - def load_aspect_model(self, file_path: Union[Path, str]): + def load_aspect_model(self, file_path: Union[Path, str], aspect_urn: str = ""): """Load aspect model to RDF GRAPH. Create an aspect object with all the including properties and operations with the turtle file :param file_path: path to the turtle file. Can be either a string or a Path object + :param aspect_urn: urn of the Aspect node :return: instance of the aspect """ file_path = self.convert_file_path(file_path) _ = self._graph.parse(file_path) - loaded_aspect_model = self._graph.to_python() + loaded_aspect_model = self._graph.to_python(aspect_urn) return loaded_aspect_model diff --git a/core/esmf-aspect-meta-model-python/pyproject.toml b/core/esmf-aspect-meta-model-python/pyproject.toml index bbac1fe..6096390 100644 --- a/core/esmf-aspect-meta-model-python/pyproject.toml +++ b/core/esmf-aspect-meta-model-python/pyproject.toml @@ -50,6 +50,7 @@ cache_dir = ".pytest_cache" download-samm-release = "esmf_aspect_meta_model_python.samm_aspect_meta_model.download_samm_release:main" download-samm-branch = "esmf_aspect_meta_model_python.samm_aspect_meta_model.download_samm_branch:main" download-samm-cli = "scripts.download_samm_cli:download_samm_cli" +download-test-models = "scripts.download_test_models:download_test_models" [build-system] requires = ["poetry>=0.12"] diff --git a/core/esmf-aspect-meta-model-python/scripts/constants.py b/core/esmf-aspect-meta-model-python/scripts/constants.py new file mode 100644 index 0000000..7177915 --- /dev/null +++ b/core/esmf-aspect-meta-model-python/scripts/constants.py @@ -0,0 +1,33 @@ +# Copyright (c) 2024 Robert Bosch Manufacturing Solutions GmbH +# +# See the AUTHORS file(s) distributed with this work for additional +# information regarding authorship. +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at https://mozilla.org/MPL/2.0/. +# +# SPDX-License-Identifier: MPL-2.0 + +from os.path import join +from string import Template + + +SAMM_VERSION = "2.7.0" + + +class SAMMCliConstants: + BASE_PATH = Template("https://github.com/eclipse-esmf/esmf-sdk/releases/download/v$version_number/$file_name") + LINUX_FILE_NAME = Template("samm-cli-$version_number-linux-x86_64.tar.gz") + VERSION = SAMM_VERSION + WIN_FILE_NAME = Template("samm-cli-$version_number-windows-x86_64.zip") + + +class TestModelConstants: + FOLDER_TO_EXTRACT = "valid" + MAVEN_URL = Template( + "https://repo1.maven.org/maven2/org/eclipse/esmf/esmf-test-aspect-models/$version_number/" + "esmf-test-aspect-models-$version_number.jar" + ) + TEST_MODELS_PATH = join("tests", "integration", "java_models", "resources") + VERSION = SAMM_VERSION diff --git a/core/esmf-aspect-meta-model-python/scripts/download_samm_cli.py b/core/esmf-aspect-meta-model-python/scripts/download_samm_cli.py index f22a1b2..033e4e5 100644 --- a/core/esmf-aspect-meta-model-python/scripts/download_samm_cli.py +++ b/core/esmf-aspect-meta-model-python/scripts/download_samm_cli.py @@ -12,24 +12,19 @@ import sys import zipfile -from string import Template - -BASE_PATH = Template("https://github.com/eclipse-esmf/esmf-sdk/releases/download/v$version_number/$file_name") -LINUX_FILE_NAME = Template("samm-cli-$version_number-linux-x86_64.tar.gz") -SAMM_CLI_VERSION = "2.6.1" -WIN_FILE_NAME = Template("samm-cli-$version_number-windows-x86_64.zip") +from scripts.constants import SAMMCliConstants as Const def get_samm_cli_file_name(): """Get a SAMM CLI file name for the current platform.""" if platform.system() == "Windows": - file_name = WIN_FILE_NAME.substitute(version_number=SAMM_CLI_VERSION) + file_name = Const.WIN_FILE_NAME.substitute(version_number=Const.VERSION) elif platform.system() == "Linux": - file_name = LINUX_FILE_NAME.substitute(version_number=SAMM_CLI_VERSION) + file_name = Const.LINUX_FILE_NAME.substitute(version_number=Const.VERSION) else: raise NotImplementedError( - f"Please download a SAMM CLI manually for your operation system from '{BASE_PATH}'" + f"Please download a SAMM CLI manually for your operation system from '{Const.BASE_PATH}'" ) return file_name @@ -67,7 +62,7 @@ def download_samm_cli(): print(error) else: print(f"Start downloading SAMM CLI {samm_cli_file_name}") - url = BASE_PATH.substitute(version_number=SAMM_CLI_VERSION, file_name=samm_cli_file_name) + url = Const.BASE_PATH.substitute(version_number=Const.VERSION, file_name=samm_cli_file_name) dir_path = Path(__file__).resolve().parents[1] archive_file = os.path.join(dir_path, samm_cli_file_name) diff --git a/core/esmf-aspect-meta-model-python/scripts/download_test_models.py b/core/esmf-aspect-meta-model-python/scripts/download_test_models.py index 883495f..ef355e2 100644 --- a/core/esmf-aspect-meta-model-python/scripts/download_test_models.py +++ b/core/esmf-aspect-meta-model-python/scripts/download_test_models.py @@ -16,29 +16,23 @@ import requests - -FOLDER_TO_EXTRACT = "valid" -TEST_MODELS_PATH = join("tests", "integration", "java_models", "resources") -VERSION = "2.7.0" +from scripts.constants import TestModelConstants as Const def get_model_files_path(version: str) -> str: """Get a path for storing test models.""" base_path = Path(__file__).parents[1].absolute() - models_path = join(base_path, TEST_MODELS_PATH, f"esmf-test-aspect-models-{version}.jar") + models_path = join(base_path, Const.TEST_MODELS_PATH, f"esmf-test-aspect-models-{version}.jar") return models_path -def download_test_models(version: str = VERSION): +def download_test_models(version: str = Const.VERSION): """Downloads and extract the esmf-test-aspect-models.""" model_files_path = get_model_files_path(version) print(f"Start downloading esmf-test-aspect-models version {version}") - url = ( - f"https://repo1.maven.org/maven2/org/eclipse/esmf/esmf-test-aspect-models/{version}/" - f"esmf-test-aspect-models-{version}.jar" - ) + url = Const.MAVEN_URL.substitute(version_number=Const.VERSION) response = requests.get(url, allow_redirects=True) resource_folder = Path(model_files_path).parent.absolute() @@ -53,7 +47,7 @@ def download_test_models(version: str = VERSION): extracted_file_path = Path(model_files_path).parents[0].absolute() archive = ZipFile(model_files_path) for file_name in archive.namelist(): - if file_name.startswith(FOLDER_TO_EXTRACT): + if file_name.startswith(Const.FOLDER_TO_EXTRACT): archive.extract(file_name, extracted_file_path) archive.close() diff --git a/core/esmf-aspect-meta-model-python/tests/integration/java_models/test_loading_aspects.py b/core/esmf-aspect-meta-model-python/tests/integration/java_models/test_loading_aspects.py index fbdc0fb..048b6b3 100644 --- a/core/esmf-aspect-meta-model-python/tests/integration/java_models/test_loading_aspects.py +++ b/core/esmf-aspect-meta-model-python/tests/integration/java_models/test_loading_aspects.py @@ -7,8 +7,7 @@ from pathlib import Path from esmf_aspect_meta_model_python.loader.aspect_loader import AspectLoader - -SAMM_VERSION = "2.0.0" +from scripts.constants import SAMM_VERSION def get_test_files():