Use Prefect to deploy Jobs

[alejandroarmas]

Jobs are called Flows inside of Prefect. Now, we can automate and monitor managed workflows. Building on top of this, Deployments also enable remote flow run execution. We trigger ad-hoc parametrized flow runs, to train our ML models, via the UI or an API call. You could also schedule automatic flow runs that run anywhere you can run a Prefect agent, including on remote infrastructure

This is done using:

1. Prefect orchestration engine, either Prefect Cloud or a local Prefect server.
2. An agent and work pool.

Example code found in this repository.

More background found here