Update port strategy to random assignment

[gwarmstrong]

This introduces a random port assignment strategy to reduce port collisions when running jobs in non-exclusive mode.

[Kipok]

Looks great, thanks! Did you run gpu tests (rm/qwen) with this locally?

[Kipok]

Let's remove exclusive=True if all works well

[gwarmstrong]

Looks great, thanks! Did you run gpu tests (rm/qwen) with this locally?

Yeah those tests look good locally

Let's remove exclusive=True if all works well

I'll test it on the cluster tomorrow with the exclusive flag dropped.