Support for Poetry

[wildfauve]

I have just started looking at DBX to fill the Databricks CI/CD gap. I'm also reasonably new to Databricks.

I have managed to create a Spark python job built with Poetry. I can manually copy this to the cluster and it runs happily.

My question is this. Would python repo packaged with Poetry be supported by DBX?

[renardeinside]

hi @wildfauve ,

yes, repo packaged by poetry is supported by dbx. You'll only need to slightly change the build logic.
By default, dbx builds the package using pip wheel command. You can disable this behaviour by doing the following:

poetry build -f wheel # first, build the wheel using poetry
dbx deploy --no-rebuild # disable automatic pip-based rebuild

Hope this helps.