Scheduling multiple tasks on same job cluster #403
-
|
I have a doubt and need some help. Is it possible to schedule multiple tasks on the same job cluster in dbx and how do I pass parameters separately for each task via "--parameters" option in "dbx launch" command. This is a wonderful tool for my needs just need to figure out a couple of details before taking databricks in production. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
hi @HiteshDream , This question is about two separate things - one is about multiple jobs sharing same job cluster, another is about parameter passing. As for shared job cluster, please take a look at this documentation section - it's called cluster reuse feature. You can find an example multitask job with cluster reuse here. As for passing launch parameters at the |
Beta Was this translation helpful? Give feedback.
hi @HiteshDream ,
This question is about two separate things - one is about multiple jobs sharing same job cluster, another is about parameter passing.
As for shared job cluster, please take a look at this documentation section - it's called cluster reuse feature.
You can find an example multitask job with cluster reuse here.
As for passing launch parameters at the
dbx launchtime, this functionality is currently not supported (although it will be supported in the upcoming 0.7.X release which we hopefully will deliver on 22nd of August).