Fine-grained spill metrics

[gerashegalov]

This PR fixes #8880. it removes task metrics

• gpuSpillBlockTime
• gpuReadSpillTime

and adds the following metrics instead:

• gpuSpillToHostTime
• gpuSpillToDiskTime
• gpuReadSpillFromHostTime
• gpuReadSpillFromDiskTime

Signed-off-by: Gera Shegalov [email protected]

[gerashegalov]

build

[revans2]

Could you also look at updating the tuning docs for these changes. I think that doc changed to be in an internal NVIDIA repo so @mattahrens might need to help you get it done right.

Also @jlowe with me initially trying to call out project rapids4spark, so I want to check with him that he is okay with the names of the metrics here.

[jlowe]

Updates look good, happy to see fewer metrics being added. Still wondering about host alloc time. Do we have a GPU alloc time metric or plans to add one?

[revans2]

Updates look good, happy to see fewer metrics being added. Still wondering about host alloc time. Do we have a GPU alloc time metric or plans to add one?

I think there may have been some miscommunication here. Or at least I didn't do a good job of this. In #8880 I asked for "metric for the amount of time a task was blocked on host memory allocation" This was intended to be analogous to gpuRetryBlockTime. But it was before we decided to use the same state machine for both. I will handle that part. I need to split up the metric so we can get GPU and CPU numbers for it. But it should be doable.

[gerashegalov]

build

[gerashegalov]

build