From c40a8449273b0bf42ca9edbdbfce6d237023de6d Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Mon, 4 Dec 2023 13:16:55 +0530 Subject: [PATCH 01/10] update docs for 23.12 release Signed-off-by: Suraj Aralihalli --- docs/archive.md | 80 +++++++++++++++++++++++++++++++++++++++++++++ docs/dev/testing.md | 4 +-- docs/download.md | 43 ++++++++++++++---------- 3 files changed, 107 insertions(+), 20 deletions(-) diff --git a/docs/archive.md b/docs/archive.md index dae04b46bb0..83108f7e200 100644 --- a/docs/archive.md +++ b/docs/archive.md @@ -5,6 +5,86 @@ nav_order: 15 --- Below are archived releases for RAPIDS Accelerator for Apache Spark. +## Release v23.10.0 +### Hardware Requirements: + +The plugin is tested on the following architectures: + + GPU Models: NVIDIA P100, V100, T4, A10/A100, L4 and H100 GPUs + +### Software Requirements: + + OS: Ubuntu 20.04, Ubuntu 22.04, CentOS 7, or Rocky Linux 8 + + NVIDIA Driver*: R470+ + + Runtime: + Scala 2.12 + Python, Java Virtual Machine (JVM) compatible with your spark-version. + + * Check the Spark documentation for Python and Java version compatibility with your specific + Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1. + Please be aware that we do not currently support Spark builds with Scala 2.13. + + Supported Spark versions: + Apache Spark 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4 + Apache Spark 3.3.0, 3.3.1, 3.3.2, 3.3.3 + Apache Spark 3.4.0, 3.4.1 + Apache Spark 3.5.0 + + Supported Databricks runtime versions for Azure and AWS: + Databricks 10.4 ML LTS (GPU, Scala 2.12, Spark 3.2.1) + Databricks 11.3 ML LTS (GPU, Scala 2.12, Spark 3.3.0) + Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2) + + Supported Dataproc versions: + GCP Dataproc 2.0 + GCP Dataproc 2.1 + +*Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet +for your hardware's minimum driver version. + +*For Cloudera and EMR support, please refer to the +[Distributions](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#which-distributions-are-supported) section of the FAQ. + +#### RAPIDS Accelerator's Support Policy for Apache Spark +The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html) + +### Download v23.10.0 +* Download the [RAPIDS + Accelerator for Apache Spark 23.10.0 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.10.0/rapids-4-spark_2.12-23.10.0.jar) + +This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with +CUDA 11.8 through CUDA 12.0. + +### Verify signature +* Download the [RAPIDS Accelerator for Apache Spark 23.10.0 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.10.0/rapids-4-spark_2.12-23.10.0.jar) + and [RAPIDS Accelerator for Apache Spark 23.10.0 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.10.0/rapids-4-spark_2.12-23.10.0.jar.asc) +* Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com). +* Import the public key: `gpg --import PUB_KEY` +* Verify the signature: `gpg --verify rapids-4-spark_2.12-23.10.0.jar.asc rapids-4-spark_2.12-23.10.0.jar` + +The output of signature verify: + + gpg: Good signature from "NVIDIA Spark (For the signature of spark-rapids release jars) " + +### Release Notes +New functionality and performance improvements for this release include: +* Introduced support for Spark 3.5.0. +* Improved memory management for better control in YARN and K8s on CSP. +* Strengthened Parquet and ORC tests for enhanced stability and support. +* Reduce GPU out-of-memory (OOM) occurrences. +* Enhanced driver log with actionable insights. +* Qualification and Profiling tool: + * Enhanced user experience with the availability of the 'ascli' tool for qualification and + profiling across all platforms. + * The qualification tool now accommodates CPU-fallback transitions and broadens the speedup factor coverage. + * Extended diagnostic support for user tools to cover EMR, Databricks AWS, and Databricks Azure. + * Introduced support for cluster configuration recommendations in the profiling tool for supported platforms. + +For a detailed list of changes, please refer to the +[CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md). + ## Release v23.08.2 ### Hardware Requirements: diff --git a/docs/dev/testing.md b/docs/dev/testing.md index 9d92ae4aacf..318d3d0584e 100644 --- a/docs/dev/testing.md +++ b/docs/dev/testing.md @@ -5,5 +5,5 @@ nav_order: 2 parent: Developer Overview --- An overview of testing can be found within the repository at: -* [Unit tests](https://github.com/NVIDIA/spark-rapids/tree/branch-23.10/tests#readme) -* [Integration testing](https://github.com/NVIDIA/spark-rapids/tree/branch-23.10/integration_tests#readme) +* [Unit tests](https://github.com/NVIDIA/spark-rapids/tree/branch-23.12/tests#readme) +* [Integration testing](https://github.com/NVIDIA/spark-rapids/tree/branch-23.12/integration_tests#readme) diff --git a/docs/download.md b/docs/download.md index 1c7e26fc090..0c1e11e2439 100644 --- a/docs/download.md +++ b/docs/download.md @@ -18,12 +18,12 @@ cuDF jar, that is either preinstalled in the Spark classpath on all nodes or sub that uses the RAPIDS Accelerator For Apache Spark. See the [getting-started guide](https://docs.nvidia.com/spark-rapids/user-guide/latest/getting-started/overview.html) for more details. -## Release v23.10.0 +## Release v23.12.0 ### Hardware Requirements: The plugin is tested on the following architectures: - GPU Models: NVIDIA P100, V100, T4, A10/A100, L4 and H100 GPUs + GPU Models: NVIDIA V100, T4, A10/A100, L4 and H100 GPUs ### Software Requirements: @@ -32,7 +32,7 @@ The plugin is tested on the following architectures: NVIDIA Driver*: R470+ Runtime: - Scala 2.12 + Scala 2.12, 2.13 Python, Java Virtual Machine (JVM) compatible with your spark-version. * Check the Spark documentation for Python and Java version compatibility with your specific @@ -49,10 +49,16 @@ The plugin is tested on the following architectures: Databricks 10.4 ML LTS (GPU, Scala 2.12, Spark 3.2.1) Databricks 11.3 ML LTS (GPU, Scala 2.12, Spark 3.3.0) Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2) + Databricks 13.3 ML LTS (GPU, Scala 2.12, Spark 3.4.1) Supported Dataproc versions: GCP Dataproc 2.0 GCP Dataproc 2.1 + + Supported Dataproc Serverless versions: + Spark runtime 1.1 LTS + Spark runtime 2.0 + Spark runtime 2.1 *Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet for your hardware's minimum driver version. @@ -63,19 +69,22 @@ for your hardware's minimum driver version. #### RAPIDS Accelerator's Support Policy for Apache Spark The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html) -### Download v23.10.0 +### Download v23.12.0 +* Download the [RAPIDS + Accelerator for Apache Spark 23.12.0 jar - Scala 2.12](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar) * Download the [RAPIDS - Accelerator for Apache Spark 23.10.0 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.10.0/rapids-4-spark_2.12-23.10.0.jar) + Accelerator for Apache Spark 23.12.0 jar - Scala 2.13](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/23.12.0/rapids-4-spark_2.13-23.12.0.jar) This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with CUDA 11.8 through CUDA 12.0. ### Verify signature -* Download the [RAPIDS Accelerator for Apache Spark 23.10.0 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.10.0/rapids-4-spark_2.12-23.10.0.jar) - and [RAPIDS Accelerator for Apache Spark 23.10.0 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.10.0/rapids-4-spark_2.12-23.10.0.jar.asc) +* Download the [RAPIDS Accelerator for Apache Spark 23.12.0 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar) + and [RAPIDS Accelerator for Apache Spark 23.12.0 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar.asc) * Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com). * Import the public key: `gpg --import PUB_KEY` -* Verify the signature: `gpg --verify rapids-4-spark_2.12-23.10.0.jar.asc rapids-4-spark_2.12-23.10.0.jar` +* Verify the signature: `gpg --verify rapids-4-spark_2.12-23.12.0.jar.asc rapids-4-spark_2.12-23.12.0.jar` +* You can use similar steps to verify RAPIDS Accelerator Scala 2.13 jar. The output of signature verify: @@ -83,17 +92,15 @@ The output of signature verify: ### Release Notes New functionality and performance improvements for this release include: -* Introduced support for Spark 3.5.0. -* Improved memory management for better control in YARN and K8s on CSP. -* Strengthened Parquet and ORC tests for enhanced stability and support. -* Reduce GPU out-of-memory (OOM) occurrences. -* Enhanced driver log with actionable insights. + +* Introduced support for chunked reading of ORC files. +* Memory usage limitations have been enforced for various operations. +* Enhanced performance for join and aggregation operations. +* Scala 2.13 is now supported, and RAPIDS Accelerator also built and tested with Scala 2.13. +* Kernel optimizations have been implemented to improve Parquet read performance. +* Databricks 13.3 ML LTS is now supported, expanding compatibility and functionality. * Qualification and Profiling tool: - * Enhanced user experience with the availability of the 'ascli' tool for qualification and - profiling across all platforms. - * The qualification tool now accommodates CPU-fallback transitions and broadens the speedup factor coverage. - * Extended diagnostic support for user tools to cover EMR, Databricks AWS, and Databricks Azure. - * Introduced support for cluster configuration recommendations in the profiling tool for supported platforms. + * For a detailed list of changes, please refer to the [CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md). From 967f9e49b4a47815dacd49dfe50c228c4f45940d Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Mon, 4 Dec 2023 13:17:37 +0530 Subject: [PATCH 02/10] update docs for 23.12 release Signed-off-by: Suraj Aralihalli --- CONTRIBUTING.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1cc52e5472a..f832cd2facd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -130,15 +130,15 @@ mvn -pl dist -PnoSnapshots package -DskipTests Verify that shim-specific classes are hidden from a conventional classloader. ```bash -$ javap -cp dist/target/rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl +$ javap -cp dist/target/rapids-4-spark_2.12-23.12.0-SNAPSHOT-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl Error: class not found: com.nvidia.spark.rapids.shims.SparkShimImpl ``` However, its bytecode can be loaded if prefixed with `spark3XY` not contained in the package name ```bash -$ javap -cp dist/target/rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar spark320.com.nvidia.spark.rapids.shims.SparkShimImpl | head -2 -Warning: File dist/target/rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar(/spark320/com/nvidia/spark/rapids/shims/SparkShimImpl.class) does not contain class spark320.com.nvidia.spark.rapids.shims.SparkShimImpl +$ javap -cp dist/target/rapids-4-spark_2.12-23.12.0-SNAPSHOT-cuda11.jar spark320.com.nvidia.spark.rapids.shims.SparkShimImpl | head -2 +Warning: File dist/target/rapids-4-spark_2.12-23.12.0-SNAPSHOT-cuda11.jar(/spark320/com/nvidia/spark/rapids/shims/SparkShimImpl.class) does not contain class spark320.com.nvidia.spark.rapids.shims.SparkShimImpl Compiled from "SparkShims.scala" public final class com.nvidia.spark.rapids.shims.SparkShimImpl { ``` @@ -181,7 +181,7 @@ mvn package -pl dist -am -Dbuildver=340 -DallowConventionalDistJar=true Verify `com.nvidia.spark.rapids.shims.SparkShimImpl` is conventionally loadable: ```bash -$ javap -cp dist/target/rapids-4-spark_2.12-23.10.0-SNAPSHOT-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl | head -2 +$ javap -cp dist/target/rapids-4-spark_2.12-23.12.0-SNAPSHOT-cuda11.jar com.nvidia.spark.rapids.shims.SparkShimImpl | head -2 Compiled from "SparkShims.scala" public final class com.nvidia.spark.rapids.shims.SparkShimImpl { ``` From 7fec072b4f27ab8dc4a231caaa462ddae4f5f85b Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Mon, 4 Dec 2023 14:18:00 +0530 Subject: [PATCH 03/10] drop pascal support - docs Signed-off-by: Suraj Aralihalli --- docs/download.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/download.md b/docs/download.md index 0c1e11e2439..c3095a58c5a 100644 --- a/docs/download.md +++ b/docs/download.md @@ -92,13 +92,13 @@ The output of signature verify: ### Release Notes New functionality and performance improvements for this release include: - * Introduced support for chunked reading of ORC files. * Memory usage limitations have been enforced for various operations. * Enhanced performance for join and aggregation operations. * Scala 2.13 is now supported, and RAPIDS Accelerator also built and tested with Scala 2.13. * Kernel optimizations have been implemented to improve Parquet read performance. * Databricks 13.3 ML LTS is now supported, expanding compatibility and functionality. +* Discontinues support for Pascal-based Nvidia GPUs. * Qualification and Profiling tool: * From bbe9aea360c370465f594618184068943098cc48 Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Tue, 5 Dec 2023 09:26:24 +0530 Subject: [PATCH 04/10] update download docs Signed-off-by: Suraj Aralihalli --- docs/download.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/docs/download.md b/docs/download.md index c3095a58c5a..c482c35b0ef 100644 --- a/docs/download.md +++ b/docs/download.md @@ -32,7 +32,7 @@ The plugin is tested on the following architectures: NVIDIA Driver*: R470+ Runtime: - Scala 2.12, 2.13 + Scala 2.12 Python, Java Virtual Machine (JVM) compatible with your spark-version. * Check the Spark documentation for Python and Java version compatibility with your specific @@ -57,8 +57,6 @@ The plugin is tested on the following architectures: Supported Dataproc Serverless versions: Spark runtime 1.1 LTS - Spark runtime 2.0 - Spark runtime 2.1 *Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet for your hardware's minimum driver version. @@ -95,12 +93,12 @@ New functionality and performance improvements for this release include: * Introduced support for chunked reading of ORC files. * Memory usage limitations have been enforced for various operations. * Enhanced performance for join and aggregation operations. -* Scala 2.13 is now supported, and RAPIDS Accelerator also built and tested with Scala 2.13. * Kernel optimizations have been implemented to improve Parquet read performance. -* Databricks 13.3 ML LTS is now supported, expanding compatibility and functionality. * Discontinues support for Pascal-based Nvidia GPUs. * Qualification and Profiling tool: - * + * Profiling Tool now processes Spark Driver log for GPU runs, enhancing feature analysis. + * Auto-tuner recommendations include AQE settings for optimized performance. + * New configurations in Profiler for enabling off-default features: udfCompiler, incompatibleDateFormats, hasExtendedYearValues. For a detailed list of changes, please refer to the [CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md). From 5745432625d1672fa7e4760a1e815d816708bd6b Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Tue, 5 Dec 2023 09:27:31 +0530 Subject: [PATCH 05/10] update download docs Signed-off-by: Suraj Aralihalli --- docs/download.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/download.md b/docs/download.md index c482c35b0ef..3e1f3bb1289 100644 --- a/docs/download.md +++ b/docs/download.md @@ -49,7 +49,6 @@ The plugin is tested on the following architectures: Databricks 10.4 ML LTS (GPU, Scala 2.12, Spark 3.2.1) Databricks 11.3 ML LTS (GPU, Scala 2.12, Spark 3.3.0) Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2) - Databricks 13.3 ML LTS (GPU, Scala 2.12, Spark 3.4.1) Supported Dataproc versions: GCP Dataproc 2.0 From 6cef0c57dc14f8dd9b8f4a72ea946fd6b254ede2 Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Tue, 5 Dec 2023 09:38:58 +0530 Subject: [PATCH 06/10] add scala 2.13 support Signed-off-by: Suraj Aralihalli --- docs/download.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/download.md b/docs/download.md index 3e1f3bb1289..89e4a1090f1 100644 --- a/docs/download.md +++ b/docs/download.md @@ -32,12 +32,11 @@ The plugin is tested on the following architectures: NVIDIA Driver*: R470+ Runtime: - Scala 2.12 + Scala 2.12, 2.13 Python, Java Virtual Machine (JVM) compatible with your spark-version. * Check the Spark documentation for Python and Java version compatibility with your specific - Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1. - Please be aware that we do not currently support Spark builds with Scala 2.13. + Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1. Supported Spark versions: Apache Spark 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4 @@ -93,6 +92,7 @@ New functionality and performance improvements for this release include: * Memory usage limitations have been enforced for various operations. * Enhanced performance for join and aggregation operations. * Kernel optimizations have been implemented to improve Parquet read performance. +* Scala 2.13 is now supported, and RAPIDS Accelerator also built and tested with Scala 2.13. * Discontinues support for Pascal-based Nvidia GPUs. * Qualification and Profiling tool: * Profiling Tool now processes Spark Driver log for GPU runs, enhancing feature analysis. From 35cc91f7e0c1781b06eb4e3f4b2f3f7ceb14d8d3 Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Wed, 6 Dec 2023 16:04:05 +0530 Subject: [PATCH 07/10] address feedback for release notes Signed-off-by: Suraj Aralihalli --- docs/download.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/download.md b/docs/download.md index 89e4a1090f1..baa2a15e241 100644 --- a/docs/download.md +++ b/docs/download.md @@ -89,11 +89,12 @@ The output of signature verify: ### Release Notes New functionality and performance improvements for this release include: * Introduced support for chunked reading of ORC files. -* Memory usage limitations have been enforced for various operations. +* Enhanced partial support for additional time zones and added stack function support. * Enhanced performance for join and aggregation operations. * Kernel optimizations have been implemented to improve Parquet read performance. -* Scala 2.13 is now supported, and RAPIDS Accelerator also built and tested with Scala 2.13. -* Discontinues support for Pascal-based Nvidia GPUs. +* RAPIDS Accelerator also built and tested with Scala 2.13. +* Last version to support Pascal-based Nvidia GPUs; discontinued in the next release. +* Introducing support for AST expressions. * Qualification and Profiling tool: * Profiling Tool now processes Spark Driver log for GPU runs, enhancing feature analysis. * Auto-tuner recommendations include AQE settings for optimized performance. From a00ecb61f88c8f78ab8942f03fc3a30a825a4fc7 Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Wed, 6 Dec 2023 16:07:03 +0530 Subject: [PATCH 08/10] address feedback for release notes Signed-off-by: Suraj Aralihalli --- docs/download.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/download.md b/docs/download.md index baa2a15e241..ef7b50b189c 100644 --- a/docs/download.md +++ b/docs/download.md @@ -89,7 +89,7 @@ The output of signature verify: ### Release Notes New functionality and performance improvements for this release include: * Introduced support for chunked reading of ORC files. -* Enhanced partial support for additional time zones and added stack function support. +* Enhanced support for additional time zones and added stack function support. * Enhanced performance for join and aggregation operations. * Kernel optimizations have been implemented to improve Parquet read performance. * RAPIDS Accelerator also built and tested with Scala 2.13. From c3b2790d927f2c23203bfb7f99a23b5cb2522ea6 Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Thu, 7 Dec 2023 20:59:58 +0530 Subject: [PATCH 09/10] update verify signature Signed-off-by: Suraj Aralihalli --- docs/download.md | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/docs/download.md b/docs/download.md index ef7b50b189c..5206b44727f 100644 --- a/docs/download.md +++ b/docs/download.md @@ -62,25 +62,28 @@ for your hardware's minimum driver version. *For Cloudera and EMR support, please refer to the [Distributions](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#which-distributions-are-supported) section of the FAQ. -#### RAPIDS Accelerator's Support Policy for Apache Spark +### RAPIDS Accelerator's Support Policy for Apache Spark The RAPIDS Accelerator maintains support for Apache Spark versions available for download from [Apache Spark](https://spark.apache.org/downloads.html) -### Download v23.12.0 -* Download the [RAPIDS - Accelerator for Apache Spark 23.12.0 jar - Scala 2.12](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar) -* Download the [RAPIDS - Accelerator for Apache Spark 23.12.0 jar - Scala 2.13](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/23.12.0/rapids-4-spark_2.13-23.12.0.jar) +### Download RAPIDS Accelerator for Apache Spark v23.12.0 +- **Scala 2.12:** + - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.12 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar) + - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.12 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar.asc) + +- **Scala 2.13:** + - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.13 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/23.12.0/rapids-4-spark_2.13-23.12.0.jar) + - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.13 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/23.12.0/rapids-4-spark_2.13-23.12.0.jar.asc) This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with CUDA 11.8 through CUDA 12.0. ### Verify signature -* Download the [RAPIDS Accelerator for Apache Spark 23.12.0 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar) - and [RAPIDS Accelerator for Apache Spark 23.12.0 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar.asc) * Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com). * Import the public key: `gpg --import PUB_KEY` -* Verify the signature: `gpg --verify rapids-4-spark_2.12-23.12.0.jar.asc rapids-4-spark_2.12-23.12.0.jar` -* You can use similar steps to verify RAPIDS Accelerator Scala 2.13 jar. +* Verify the signature for Scala 2.12 jar: + `gpg --verify rapids-4-spark_2.12-23.12.0.jar.asc rapids-4-spark_2.12-23.12.0.jar` +* Verify the signature for Scala 2.13 jar: + `gpg --verify rapids-4-spark_2.13-23.12.0.jar.asc rapids-4-spark_2.13-23.12.0.jar` The output of signature verify: @@ -94,7 +97,6 @@ New functionality and performance improvements for this release include: * Kernel optimizations have been implemented to improve Parquet read performance. * RAPIDS Accelerator also built and tested with Scala 2.13. * Last version to support Pascal-based Nvidia GPUs; discontinued in the next release. -* Introducing support for AST expressions. * Qualification and Profiling tool: * Profiling Tool now processes Spark Driver log for GPU runs, enhancing feature analysis. * Auto-tuner recommendations include AQE settings for optimized performance. From 0f616e0d8a0d45b68de77fecef61ea075bff9ed3 Mon Sep 17 00:00:00 2001 From: Suraj Aralihalli Date: Thu, 7 Dec 2023 21:41:05 +0530 Subject: [PATCH 10/10] jars -> jar Signed-off-by: Suraj Aralihalli --- docs/download.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/download.md b/docs/download.md index 5206b44727f..e68af9c65ae 100644 --- a/docs/download.md +++ b/docs/download.md @@ -68,11 +68,11 @@ The RAPIDS Accelerator maintains support for Apache Spark versions available for ### Download RAPIDS Accelerator for Apache Spark v23.12.0 - **Scala 2.12:** - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.12 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar) - - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.12 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar.asc) + - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.12 jar.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.12.0/rapids-4-spark_2.12-23.12.0.jar.asc) - **Scala 2.13:** - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.13 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/23.12.0/rapids-4-spark_2.13-23.12.0.jar) - - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.13 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/23.12.0/rapids-4-spark_2.13-23.12.0.jar.asc) + - [RAPIDS Accelerator for Apache Spark 23.12.0 - Scala 2.13 jar.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.13/23.12.0/rapids-4-spark_2.13-23.12.0.jar.asc) This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with CUDA 11.8 through CUDA 12.0.