Skip to content

Commit

Permalink
Change Databricks 14.3 shim name to spark350db143 (#11728)
Browse files Browse the repository at this point in the history
Append "143" to the DB14.3 shim name because both Databricks 14.3.x and 15.4.x are based on Spark version 3.5.0

Signed-off-by: timl <[email protected]>
  • Loading branch information
NvTimLiu authored Nov 18, 2024
1 parent a8010cc commit 9b06ae3
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion jenkins/databricks/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,12 @@ SCALA_VERSION=`mvn help:evaluate -q -pl dist -Dexpression=scala.binary.version -
VERSION_NUM=${BASE_SPARK_VERSION_TO_INSTALL_DATABRICKS_JARS//.}
SPARK_VERSION_STR=spark$VERSION_NUM
SPARK_PLUGIN_JAR_VERSION=`mvn help:evaluate -q -pl dist -Dexpression=project.version -DforceStdout`
DB_SHIM_NAME=${SPARK_VERSION_STR}db
# Append 143 into the db shim version because Databricks 14.3.x and 15.4.x are both based on spark version 3.5.0
if [[ "$DB_RUNTIME" == "14.3"* ]]; then
DB_SHIM_NAME="${SPARK_VERSION_STR}db143"
else
DB_SHIM_NAME="${SPARK_VERSION_STR}db"
fi
DBJARFPATH=./aggregator/target/${DB_SHIM_NAME}/rapids-4-spark-aggregator_$SCALA_VERSION-$SPARK_PLUGIN_JAR_VERSION-${DB_SHIM_NAME}.jar
echo "Databricks jar is: $DBJARFPATH"
MVN="mvn -Dmaven.wagon.http.retryHandler.count=3 -DretryFailedDeploymentCount=3"
Expand Down

0 comments on commit 9b06ae3

Please sign in to comment.