diff --git a/docs/archive.md b/docs/archive.md index 9a5816804d7..14c9e9a209f 100644 --- a/docs/archive.md +++ b/docs/archive.md @@ -5,6 +5,84 @@ nav_order: 15 --- Below are archived releases for RAPIDS Accelerator for Apache Spark. +## Release v23.08.1 +### Hardware Requirements: + +The plugin is tested on the following architectures: + + GPU Models: NVIDIA P100, V100, T4, A10/A100, L4 and H100 GPUs + +### Software Requirements: + + OS: Ubuntu 20.04, Ubuntu 22.04, CentOS 7, or Rocky Linux 8 + + NVIDIA Driver*: R470+ + + Runtime: + Scala 2.12 + Python, Java Virtual Machine (JVM) compatible with your spark-version. + + * Check the Spark documentation for Python and Java version compatibility with your specific + Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1. + Please be aware that we do not currently support Spark builds with Scala 2.13. + + Supported Spark versions: + Apache Spark 3.1.1, 3.1.2, 3.1.3 + Apache Spark 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4 + Apache Spark 3.3.0, 3.3.1, 3.3.2 + Apache Spark 3.4.0, 3.4.1 + Apache Spark 3.5.0 + + Supported Databricks runtime versions for Azure and AWS: + Databricks 10.4 ML LTS (GPU, Scala 2.12, Spark 3.2.1) + Databricks 11.3 ML LTS (GPU, Scala 2.12, Spark 3.3.0) + Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2) + + Supported Dataproc versions: + GCP Dataproc 2.0 + GCP Dataproc 2.1 + +*Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet +for your hardware's minimum driver version. + +*For Cloudera and EMR support, please refer to the +[Distributions](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#which-distributions-are-supported) section of the FAQ. + +### Download v23.08.1 +* Download the [RAPIDS + Accelerator for Apache Spark 23.08.1 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.1/rapids-4-spark_2.12-23.08.1.jar) + +This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with +CUDA 11.8 through CUDA 12.0. + +Note that v23.08.0 is deprecated. + +### Verify signature +* Download the [RAPIDS Accelerator for Apache Spark 23.08.1 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.1/rapids-4-spark_2.12-23.08.1.jar) + and [RAPIDS Accelerator for Apache Spark 23.08.1 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.1/rapids-4-spark_2.12-23.08.1.jar.asc) +* Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com). +* Import the public key: `gpg --import PUB_KEY` +* Verify the signature: `gpg --verify rapids-4-spark_2.12-23.08.1.jar.asc rapids-4-spark_2.12-23.08.1.jar` + +The output if signature verify: + + gpg: Good signature from "NVIDIA Spark (For the signature of spark-rapids release jars) " + +### Release Notes +New functionality and performance improvements for this release include: +* Compatibility with Databricks AWS & Azure 12.2 ML LTS. +* Enhanced stability and support for ORC and Parquet. +* Reduction of out-of-memory (OOM) occurrences. +* Corner case evaluation for data formats, operators and expressions +* Qualification and Profiling tool: + * Profiling tool now supports Azure Databricks and AWS Databricks. + * Qualification tool can provide advice on unaccelerated operations. + * Improve user experience through CLI design. + * Qualification tool provides configuration and migration recommendations for Dataproc and EMR. + +For a detailed list of changes, please refer to the +[CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md). + ## Release v23.06.0 Hardware Requirements: diff --git a/docs/download.md b/docs/download.md index 20474069d28..df73124f12d 100644 --- a/docs/download.md +++ b/docs/download.md @@ -18,7 +18,7 @@ cuDF jar, that is either preinstalled in the Spark classpath on all nodes or sub that uses the RAPIDS Accelerator For Apache Spark. See the [getting-started guide](https://nvidia.github.io/spark-rapids/Getting-Started/) for more details. -## Release v23.08.1 +## Release v23.08.2 ### Hardware Requirements: The plugin is tested on the following architectures: @@ -29,49 +29,53 @@ The plugin is tested on the following architectures: OS: Ubuntu 20.04, Ubuntu 22.04, CentOS 7, or Rocky Linux 8 - NVIDIA Driver*: 470+ + NVIDIA Driver*: R470+ - Python 3.6+, Scala 2.12, Java 8, Java 17 + Runtime: + Scala 2.12 + Python, Java Virtual Machine (JVM) compatible with your spark-version. + + * Check the Spark documentation for Python and Java version compatibility with your specific + Spark version. For instance, visit `https://spark.apache.org/docs/3.4.1` for Spark 3.4.1. + Please be aware that we do not currently support Spark builds with Scala 2.13. Supported Spark versions: Apache Spark 3.1.1, 3.1.2, 3.1.3 Apache Spark 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.2.4 Apache Spark 3.3.0, 3.3.1, 3.3.2 Apache Spark 3.4.0, 3.4.1 + Apache Spark 3.5.0 - Supported Databricks runtime versions: - Azure/AWS: - Databricks 10.4 ML LTS (GPU, Scala 2.12, Spark 3.2.1) - Databricks 11.3 ML LTS (GPU, Scala 2.12, Spark 3.3.0) - Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2) + Supported Databricks runtime versions for Azure and AWS: + Databricks 10.4 ML LTS (GPU, Scala 2.12, Spark 3.2.1) + Databricks 11.3 ML LTS (GPU, Scala 2.12, Spark 3.3.0) + Databricks 12.2 ML LTS (GPU, Scala 2.12, Spark 3.3.2) Supported Dataproc versions: GCP Dataproc 2.0 GCP Dataproc 2.1 -*Some hardware may have a minimum driver version greater than v450.80.02+. Check the GPU spec sheet +*Some hardware may have a minimum driver version greater than R470. Check the GPU spec sheet for your hardware's minimum driver version. *For Cloudera and EMR support, please refer to the -[Distributions](./FAQ.md#which-distributions-are-supported) section of the FAQ. +[Distributions](https://docs.nvidia.com/spark-rapids/user-guide/latest/faq.html#which-distributions-are-supported) section of the FAQ. -### Download v23.08.1 +### Download v23.08.2 * Download the [RAPIDS - Accelerator for Apache Spark 23.08.1 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.1/rapids-4-spark_2.12-23.08.1.jar) + Accelerator for Apache Spark 23.08.2 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.2/rapids-4-spark_2.12-23.08.2.jar) -This package is built against CUDA 11.8, all CUDA 11.x and 12.x versions are supported through [CUDA forward -compatibility](https://docs.nvidia.com/deploy/cuda-compatibility/index.html). It is tested -on V100, T4, A10, A100, L4 and H100 GPUs with CUDA 11.8-12.0. For those using other types of GPUs -which do not have CUDA forward compatibility (for example, GeForce), CUDA 11.8 or later is required. +This package is built against CUDA 11.8. It is tested on V100, T4, A10, A100, L4 and H100 GPUs with +CUDA 11.8 through CUDA 12.0. Note that v23.08.0 is deprecated. ### Verify signature -* Download the [RAPIDS Accelerator for Apache Spark 23.08.1 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.1/rapids-4-spark_2.12-23.08.1.jar) - and [RAPIDS Accelerator for Apache Spark 23.08.1 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.1/rapids-4-spark_2.12-23.08.1.jar.asc) +* Download the [RAPIDS Accelerator for Apache Spark 23.08.2 jar](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.2/rapids-4-spark_2.12-23.08.2.jar) + and [RAPIDS Accelerator for Apache Spark 23.08.2 jars.asc](https://repo1.maven.org/maven2/com/nvidia/rapids-4-spark_2.12/23.08.2/rapids-4-spark_2.12-23.08.2.jar.asc) * Download the [PUB_KEY](https://keys.openpgp.org/search?q=sw-spark@nvidia.com). * Import the public key: `gpg --import PUB_KEY` -* Verify the signature: `gpg --verify rapids-4-spark_2.12-23.08.1.jar.asc rapids-4-spark_2.12-23.08.1.jar` +* Verify the signature: `gpg --verify rapids-4-spark_2.12-23.08.2.jar.asc rapids-4-spark_2.12-23.08.2.jar` The output if signature verify: @@ -88,7 +92,8 @@ New functionality and performance improvements for this release include: * Qualification tool can provide advice on unaccelerated operations. * Improve user experience through CLI design. * Qualification tool provides configuration and migration recommendations for Dataproc and EMR. - +* Fixes Databricks build issues from the previous 23.08 release. + For a detailed list of changes, please refer to the [CHANGELOG](https://github.com/NVIDIA/spark-rapids/blob/main/CHANGELOG.md).