[BUG] compiler bridge build concurrency for non-JDK8 PR checks

[gerashegalov]

Describe the bug
The same issue as #10105 just showed up in the PR check following the merge of #9925. The reason in the check is that the compiler bridge artifact has java class file version in its file name (among other things).

So when non-JDK8 jobs kick in they end up behaving like #10105 .

[INFO] Using incremental compilation using Mixed compile order
[INFO] Compiler bridge file: /home/runner/.m2/repository/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.6.1-bin_2.12.15__61.0-1.6.1_20220117T214008.jar
[INFO] Compiler bridge file is not installed yet
[INFO] Using incremental compilation using Mixed compile order
[INFO] Compiler bridge file: /home/runner/.m2/repository/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.6.1-bin_2.12.15__61.0-1.6.1_20220117T214008.jar
[INFO] Compiler bridge file is not installed yet

It does not lead to a complete failure because retry logic kicks in but it defeats the optimization purpose #9925

[mattahrens]

@gerashegalov -- would removing -T1C fix this issue as well (similar to #10105)?

[NvTimLiu]

@gerashegalov -- would removing -T1C fix this issue as well (similar to #10105)?

Same concern, if removing -T1C can fix the issue, we're good to remove it from pre-merge/nightly CI.

[gerashegalov]

@NvTimLiu @mattahrens yes this is the simplest workaround

for this issue #10115 remove --threads 1C here

spark-rapids/.github/workflows/mvn-verify-check.yml

Line 32 in f28f258

--threads 1C

for #10105

spark-rapids/jenkins/spark-nightly-build.sh

Lines 106 to 109 in f28f258

	if [[ "${SKIP_TESTS}" == "true" ]]; then
	# if skip test, we could try speed up build with multiple-threads
	MVN="${MVN} -T1C"
	fi

[NvTimLiu]

Can we close the issue ass #10153 got merged?

[gerashegalov]

#10153 is a quick bandaid fix for this PR

I would like to keep it open to do a real fix, eventually

The PR check workflows were actually designed to deal with the problem, so either:

• a nuanced fix is to disable threads only for JDK9+
• or better fix make sure that JDK11, 17-based compiler bridge artifacts already are guaranteed to exist via cache-dependencies step.

[NvTimLiu]

Thanks @gerashegalov, hand over it issue to you for tracking.