Revert "Allow benchmark runner config: openxla for inference runs. …

[frgossen]

…(#5939)"

This reverts commit dcdd66e.

Introduced openxla to unfold for both training and inference. This is not intended and leads to duplicate experiment runs.

[frgossen]

Will introduce tests in #6093

[ysiraichi]

This is not intended and leads to duplicate experiment runs.

I remember @JackCaoG mentioning we were trying to get rid of openxla_eval (comment). Is it not the case?

[JackCaoG]

@ysiraichi it is still the plan.. but I don't think we will be able to do that for 2.2 releases. We might as well just keep testing it until we deprecate it.

[frgossen]

I agree, replacing openxla_eval with openxla is generally a good idea. This change is unfortunately not doing that exactly.

[ysiraichi]

Correct. #5939 only adds openxla as a possible backend for eval benchmark runs. i.e. reverting that PR would make it not possible to run models in eval mode with openxla dynamo backend. I guess the real question is: do we want to disallow that?

[frgossen]

Reverting this brings us back to the previous state where eval goes together with openxla_eval and train together with openxla? Is there any difference between openxla_eval+eval and openxla+eval?

[ysiraichi]

Yes. openxla_eval doesn't use AOTAutograd. i.e. it doesn't get a nn.Module-traced dynamo graph. This makes the set models that run/fail also different.

[frgossen]

Thanks for explaining that. So we actually want to have these different combinations. Eventually, we want to get rid of the openxla_eval backend. Is that correct?

[ysiraichi]

As far as I understand, that's the idea. Maybe @JackCaoG can confirm this.

[frgossen]

Thanks. In that case, I will just close this PR and update the test cases instead.