Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend information from leapp saved to leappdb #847

Merged
merged 4 commits into from
May 13, 2024
Merged

Conversation

dkubek
Copy link
Member

@dkubek dkubek commented Nov 21, 2023

Extend leappdb for leapp-inspector

This commit combines multiple additions to the leapp database.

First addition is tracking of entity metadata. The Metadata model stores the metadata of entities such as Actor or Workflow. This data is stored in a new table metadata of the leapp.db file.

  1. metadata of discovered actors.

    For an actor, the metadata stored contain:

    class_name - the name of the actor class
    name - the name given to the actor
    description - the actor's description
    phase - phase of execution of the actor
    tags - names of any tags associated with an actor
    consumes - list of all messages the actor consumes
    produces - list of all messages the actor produces
    path - the path to the actor source file

  2. workflow metadata.

    For a workflow, the metadata stored contain:

    name - name of the workflow
    short_name - short name of the workflow
    tag - workflow tag
    description - workflow description
    phases - all phases associated with the workflow

Next addition is tracking of dialog question. Previously leapp was not able to detect the actual question asked from the user as it could be generated dynamically when actor is called and depend on the configuration of the user's system.

Last addition includes storing the actor exit status. Exit status is now saved as an audit event actor-exit-status. Exit status 0 represents successful execution or StopActorExecution/StopActorExecutionError, while 1 indicates an unexpected and unhandled exception.

These changes collectively improve the metadata handling capabilities of, ensuring accurate storage and retrieval of essential information for various entities.

Copy link

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

  • review please @oamg/developers to notify leapp developers of the review request
  • /packit copr-build to submit a public copr build using packit

To launch regression testing public members of oamg organization can leave the following comment:

  • /rerun to schedule basic regression tests using this pr build and leapp-repository*master* as artifacts
  • /rerun 42 to schedule basic regression tests using this pr build and leapp-repository*PR42* as artifacts
  • /rerun-sst to schedule sst tests using this pr build and leapp-repository*master* as artifacts
  • /rerun-sst 42 to schedule sst tests using this pr build and leapp-repository*PR42* as artifacts

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra.

@pirat89 pirat89 marked this pull request as draft November 21, 2023 14:40
@dkubek dkubek added the wip label Nov 23, 2023
@dkubek dkubek changed the title [Draft] Extend information from leapp saved to leappdb Extend information from leapp saved to leappdb Mar 1, 2024
@dkubek dkubek removed the wip label Mar 1, 2024
@dkubek dkubek marked this pull request as ready for review March 1, 2024 11:02
@pirat89
Copy link
Member

pirat89 commented May 10, 2024

/packit copr-build

dkubek added 3 commits May 10, 2024 15:15
This commit combines multiple additions to the leapp database.

First addition is tracking of entity metadata. The `Metadata` model
stores the metadata of entities such as `Actor` or `Workflow`. This data
is stored in a new table `metadata` of the `leapp.db` file.

  1. metadata of *discovered* actors.

     For an actor, the metadata stored contain:

	`class_name` 	- the name of the actor class
	`name`		- the name given to the actor
	`description`	- the actor's description
	`phase` 	- phase of execution of the actor
	`tags`		- names of any tags associated with an actor
	`consumes` 	- list of all messages the actor consumes
	`produces`	- list of all messages the actor produces
	`path`		- the path to the actor source file

  2. workflow metadata.

     For a workflow, the metadata stored contain:

       `name`		- name of the workflow
       `short_name` 	- short name of the workflow
       `tag`		- workflow tag
       `description`	- workflow description
       `phases` 	- all phases associated with the workflow

Next addition is tracking of dialog question. Previously leapp was not
able to detect the actual question asked from the user as it could be
generated dynamically when actor is called and depend on the
configuration of the user's system.

Last addition includes storing the actor exit status. Exit status is now
saved as an audit event `actor-exit-status`. Exit status 0 represents
successful execution or `StopActorExecution`/`StopActorExecutionError`,
while 1 indicates an unexpected and unhandled exception.

These changes collectively improve the metadata handling capabilities
of, ensuring accurate storage and retrieval of essential information for
various entities.
@pirat89
Copy link
Member

pirat89 commented May 10, 2024

rebased to fix the test execution

@pirat89 pirat89 added this to the 8.10/9.5 milestone May 10, 2024
@pirat89
Copy link
Member

pirat89 commented May 10, 2024

@dkubek seems good to me 💯 I will do some testing yet on rhel8 to be sure and I think we can squash&merge it.

@pirat89
Copy link
Member

pirat89 commented May 10, 2024

/packit copr-build

Copy link
Member

@pirat89 pirat89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems to be working like a charm. Manually tested:

  • (pre)upgrade 7 -> 8 & 8 -> 9
  • update of existing leapp.db to the new format (7, 8)
    • run older leapp, install 847 PR build, run it again, check leapp.db
  • run downgraded leapp (released one) after new leapp.db has been created already

In all cases it worked without problems.

I've been thinking whether the StopActorExecutionError should be handled as success or fail in the leapp.db as this one is honestly trickey. People who works with leapp.db directly, could be confused by this, but I understand reasoning for this logic. I think it's ok to go this way and in case we realize later we would like to change the behaviour, it should not be problematic to do it any time. Especially before the next upstream release. So merging to enable works on other features that are waiting for this PR.

@pirat89 pirat89 merged commit 91b309a into oamg:master May 13, 2024
20 checks passed
@pirat89 pirat89 added the changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant label May 13, 2024
pirat89 added a commit to pirat89/leapp that referenced this pull request Aug 16, 2024
## Packaging
- Start building for EL 9 in the upstream repository on COPR (oamg#855)

## Framework
### Enhancements
- Minor update in the summary overview to highlight what is present in the pre-upgrade report (oamg#858)
- Store metadata about actors, workflows, and dialogs inside leapp audit db (oamg#847, oamg#867)

## Leapp (tool)
### Enhancements
- Implement singleton leapp execution to prevent multiple running leapp instances on the system in the same time (oamg#851)

## stdlib
### Fixes
- Close properly all file descriptors when executing shell commands via `run` (oamg#880)

## Modifications
- Code is now Python3.12 compatible (oamg#855)
@pirat89 pirat89 mentioned this pull request Aug 16, 2024
pirat89 added a commit that referenced this pull request Aug 16, 2024
## Packaging
- Start building for EL 9 in the upstream repository on COPR (#855)

## Framework
### Enhancements
- Minor update in the summary overview to highlight what is present in the pre-upgrade report (#858)
- Store metadata about actors, workflows, and dialogs inside leapp audit db (#847, #867)

## Leapp (tool)
### Enhancements
- Implement singleton leapp execution to prevent multiple running leapp instances on the system in the same time (#851)

## stdlib
### Fixes
- Close properly all file descriptors when executing shell commands via `run` (#880)

## Modifications
- Code is now Python3.12 compatible (#855)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog-checked The merger/reviewer checked the changelog draft document and updated it when relevant enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants