You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Nov 29, 2023. It is now read-only.
To know how changes are affecting model performance, it could be nice to be able to run a model with commands from PRs and record how it compares to the baseline/other models
@jacobbieker I would be interested in submitting a PR!
Do we have a workflow in the works? e.g. from above blog would be using chatops gh action to communicate using pr, some gh action to submit (Argo or CML) ML workflow (we have to select which models to run against by modifying config.yaml or some kind of matrix) in addition to logging to Neptune and display the metrics and plots on the pr for ease of comparison.
Some questions like do we trigger this workflow by making some changes to default parameters or simply run with default parameters for selected models for every trigger?
Hi! We currently don't have a workflow for this, and there isn't one in the works. I think the ideal way would be to trigger it based off when a model or datamodule config changes. We've used CML before for Satip.
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Detailed Description
To know how changes are affecting model performance, it could be nice to be able to run a model with commands from PRs and record how it compares to the baseline/other models
GitHub has some actions that can be accomplished to do that: https://github.blog/2020-06-17-using-github-actions-for-mlops-data-science/
Context
We need to compare our models to other PV predictions, and so this could be a nice way to record some model performance, in addition to Neptune.ai
Possible Implementation
https://github.blog/2020-06-17-using-github-actions-for-mlops-data-science/
https://github.com/machine-learning-apps/actions-ml-cicd
The text was updated successfully, but these errors were encountered: