ENH Add additional options to override SLURM arguments for JIDSlurmOperator

[gadorlhiac]

Description

This PR gives DAG operators more control to override certain passed SLURM arguments. It also separates the substitution process into a separate method.

Checklist

• Provide a max_nodes option to the JIDSlurmOperator to control across how many nodes a job will be split on.
• Separate out the SLURM argument substitutions into a separate method.
• Add max_nodes=1 to relevant Tasks, i.e., CrystFEL software which should only be run on one node.
• Update documentation
• Handle case where ntasks is not passed. Default to 1 core (properly). Previously would just substitute the value 1, with no flag.

PR Type:

• New feature/Enhancement

Address issues:

• NA

Testing

Screenshots

[valmar]

This looks OK. Just one question: currently it looks like the _sub_overridable_argument explicitely will work only for max_cores not for generic options. Is this by design? Do we anticipate overriding anything else?