Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DAME and DataTransformation #41

Closed
8 of 9 tasks
dgarijo opened this issue Apr 22, 2020 · 4 comments
Closed
8 of 9 tasks

DAME and DataTransformation #41

dgarijo opened this issue Apr 22, 2020 · 4 comments
Labels
enhancement New feature or request question Further information is requested
Milestone
End-Of-June Release

Comments

@dgarijo
Copy link
Contributor

dgarijo commented Apr 22, 2020 •
edited by mosoriob
Loading

@sirspock, this will add new dependencies in the model catalog and will need to be discussed in detail. This new functionality implies the execution of data transformation and postprocessing steps for a model.

Assuming this information is available in the knowledge base, DAME will be able to invoke the transformation pipeline (dockerized) to produce inputs to be used by the model. Initially this will be done in 2 steps: the user activates the transformation giving the original input; and then executes the model with DAME.

  • Test the pull request Minh
  • Add Data Transformation resource to the Model Catalog issue
  • Fix v1.5.0 tests issue
    • After, release v1.5.0 server
    • Update the model catalog client python
  • Test: mic and dame using model catalog client 1.5.0
    • Release the model catalog client python
  • Search dataset-id using the DataCatalog python client
  • Overwrite the parameters
@dgarijo dgarijo added enhancement New feature or request question Further information is requested labels Apr 22, 2020
@dgarijo dgarijo added this to the End-Of-May Release milestone Apr 22, 2020
@mosoriob
Copy link
Collaborator

We need to talk with the other team. They haven't share any plan of the architecture with me.
Do you have any information?

Without a plan, this is going to fail and DAME cannot fix it.

@dgarijo
Copy link
Contributor Author

dgarijo commented Apr 29, 2020

I do not have additional information, and I think you are right. We should engage with them soon or this feature will not be ready.
What I know is that they will provide a container with the transformations.

@dgarijo
Copy link
Contributor Author

dgarijo commented Apr 29, 2020

Conversation started. They have: https://github.com/mintproject/MINT-Transformation for declaring their containers. Please let me know if you don't have access.

@mosoriob mosoriob changed the title DAME should support data pre-processing and pos-processing when appropriate DAME and DataTransformation Jun 23, 2020
@mosoriob
Copy link
Collaborator

mosoriob commented Jul 9, 2020

The first version of this issue has been addressed.

@mosoriob mosoriob closed this as completed Jul 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Milestone
End-Of-June Release
Development

No branches or pull requests
2 participants
@dgarijo @mosoriob