[GPU] Improve reference kernel coverage.

[dyoussif]

Addresses https://jira.devtools.intel.com/browse/MFDNN-12444.
Broaden ocl:ref coverage and add some smoke tests for key compute bound primitives.

[dyoussif]

make test
disable device_cpu
disable run_scans
disable benchdnn_all
enable benchdnn_nightly
enable benchdnn_matmul
enable benchdnn_conv
enable benchdnn_ip
enable benchdnn_deconv
enable arch_xe-lp
enable arch_xe2-lpg
enable arch_xe2-hpg-bmg

[mgouicem]

My 2 cents on targeting specific implementations in our ci coverage: it would increase ci testing scope with no clear value since ref impls should be tested by HW configuration that do not dispatch optimized ones.

For that reason, I would keep --global-impl flag out of the input files, and have it used for development purpose only.

[mgouicem]

these do not seem to cover decompression as fp-mathmode is not set here. Am I missing something?

[dyoussif]

thanks for pointing this out. Added.

[kealan-barbieri]

should the default value just be dropped if it doesnt restrict any value?

[dyoussif]

done.

[dyoussif]

My 2 cents on targeting specific implementations in our ci coverage: it would increase ci testing scope with no clear value since ref impls should be tested by HW configuration that do not dispatch optimized ones.

For that reason, I would keep --global-impl flag out of the input files, and have it used for development purpose only.

This PR was opened in response to an issue where oneDNN was run on HW that was not supported yet and the reference implementation returned unimplemented, ultimately causing a primitive creation failure. The idea here was to introduce some limited testing to have better coverage of ref kernels.

[dyoussif]

make test
disable device_cpu
disable run_scans
disable benchdnn_all
enable benchdnn_nightly
enable benchdnn_matmul
enable benchdnn_conv
enable benchdnn_ip
enable benchdnn_deconv
enable arch_xe-lp
enable arch_xe2-lpg
enable arch_xe2-hpg-bmg

[echeresh]

@dzarukin Please take a look at this usage. Is it okay to have --global-impl in batch files? If not, please advise on how to handle this better.

[dzarukin]

I would suggest to copy-exact the content of the smoke file with a proper comment (if that's the target coverage set) and use --impl in the batch file instead.

[dyoussif]

I created separate files for ref smoke tests.

[dyoussif]

make test
disable device_cpu
disable run_scans
disable benchdnn_all
enable benchdnn_nightly
enable benchdnn_matmul
enable benchdnn_conv
enable benchdnn_ip
enable benchdnn_deconv
enable arch_xe-lp
enable arch_xe2-lpg
enable arch_xe2-hpg-bmg

[dyoussif]

make test
disable device_cpu
disable run_scans
disable benchdnn_all
enable benchdnn_nightly
enable arch_xe-lp
enable arch_xe2-lpg
enable arch_xe2-hpg-bmg

[mgouicem]

This PR was opened in response to an issue where oneDNN was run on HW that was not supported yet and the reference implementation returned unimplemented, ultimately causing a primitive creation failure. The idea here was to introduce some limited testing to have better coverage of ref kernels.

Thanks for the clarification. I wonder if it might be less ressource consuming to add a configuration that would fallback to ref impl in our test matrix vs running reference impl test on all configurations.
Do you know if we can set some environment Level-Zero / NEO environment variable to force our impl to fallback to reference?

[dyoussif]

This PR was opened in response to an issue where oneDNN was run on HW that was not supported yet and the reference implementation returned unimplemented, ultimately causing a primitive creation failure. The idea here was to introduce some limited testing to have better coverage of ref kernels.

Thanks for the clarification. I wonder if it might be less ressource consuming to add a configuration that would fallback to ref impl in our test matrix vs running reference impl test on all configurations. Do you know if we can set some environment Level-Zero / NEO environment variable to force our impl to fallback to reference?

It doesn't look like there is some environment variable to do this on the driver side. The smoke tests here are quite small; based off my testing should be about 5-10 mins per primitive to run.

[echeresh]

This PR was opened in response to an issue where oneDNN was run on HW that was not supported yet and the reference implementation returned unimplemented, ultimately causing a primitive creation failure. The idea here was to introduce some limited testing to have better coverage of ref kernels.

Thanks for the clarification. I wonder if it might be less ressource consuming to add a configuration that would fallback to ref impl in our test matrix vs running reference impl test on all configurations. Do you know if we can set some environment Level-Zero / NEO environment variable to force our impl to fallback to reference?

It doesn't look like there is some environment variable to do this on the driver side. The smoke tests here are quite small; based off my testing should be about 5-10 mins per primitive to run.

"smoke tests" sounds like something that could be covered with randomized test cases as well: #2434 + @rjoursler

We may have 100-200 small/medium test cases (that we can call "smoke" or "random") per primitive which can be used to test reference implementations. Once the test generator uniformly supports various features we should have pretty good coverage with very limited amount of testing.