-
Notifications
You must be signed in to change notification settings - Fork 197
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
This reverts commit b760453. The various issues around symbol visibility caused by raft currently supporting a mix of compiled library and header-only usage make this wheel unsafe to use in various environments at the moment. We will revisit this work once all of the compiled components of raft are fully split out into cuvs. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Corey J. Nolet (https://github.com/cjnolet) - Ray Douglass (https://github.com/raydouglass) URL: #2305
- Loading branch information
Showing
37 changed files
with
237 additions
and
521 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -67,45 +67,41 @@ jobs: | |
node_type: "gpu-v100-latest-1" | ||
run_script: "ci/build_docs.sh" | ||
sha: ${{ inputs.sha }} | ||
wheel-build-cpp: | ||
wheel-build-pylibraft: | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
with: | ||
matrix_filter: group_by([.ARCH, (.CUDA_VER|split(".")|map(tonumber)|.[0])]) | map(max_by(.PY_VER|split(".")|map(tonumber))) | ||
build_type: ${{ inputs.build_type || 'branch' }} | ||
branch: ${{ inputs.branch }} | ||
sha: ${{ inputs.sha }} | ||
date: ${{ inputs.date }} | ||
script: ci/build_wheel_cpp.sh | ||
wheel-build-python: | ||
needs: wheel-build-cpp | ||
script: ci/build_wheel_pylibraft.sh | ||
wheel-publish-pylibraft: | ||
needs: wheel-build-pylibraft | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/wheels-build[email protected] | ||
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish[email protected] | ||
with: | ||
build_type: ${{ inputs.build_type || 'branch' }} | ||
branch: ${{ inputs.branch }} | ||
sha: ${{ inputs.sha }} | ||
date: ${{ inputs.date }} | ||
script: ci/build_wheel_python.sh | ||
wheel-publish-cpp: | ||
needs: wheel-build-cpp | ||
package-name: pylibraft | ||
wheel-build-raft-dask: | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish[email protected] | ||
uses: rapidsai/shared-workflows/.github/workflows/wheels-build[email protected] | ||
with: | ||
build_type: ${{ inputs.build_type || 'branch' }} | ||
branch: ${{ inputs.branch }} | ||
sha: ${{ inputs.sha }} | ||
date: ${{ inputs.date }} | ||
package-name: raft | ||
package-type: cpp | ||
wheel-publish-python: | ||
needs: wheel-build-python | ||
script: ci/build_wheel_raft_dask.sh | ||
wheel-publish-raft-dask: | ||
needs: wheel-build-raft-dask | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
with: | ||
build_type: ${{ inputs.build_type || 'branch' }} | ||
branch: ${{ inputs.branch }} | ||
sha: ${{ inputs.sha }} | ||
date: ${{ inputs.date }} | ||
package-name: raft | ||
package-type: python | ||
package-name: raft_dask |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,9 +19,9 @@ jobs: | |
- conda-python-build | ||
- conda-python-tests | ||
- docs-build | ||
- wheel-build-cpp | ||
- wheel-build-python | ||
- wheel-build-pylibraft | ||
- wheel-tests-pylibraft | ||
- wheel-build-raft-dask | ||
- wheel-tests-raft-dask | ||
- devcontainer | ||
secrets: inherit | ||
|
@@ -74,30 +74,29 @@ jobs: | |
arch: "amd64" | ||
container_image: "rapidsai/ci-conda:latest" | ||
run_script: "ci/build_docs.sh" | ||
wheel-build-cpp: | ||
wheel-build-pylibraft: | ||
needs: checks | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
with: | ||
matrix_filter: group_by([.ARCH, (.CUDA_VER|split(".")|map(tonumber)|.[0])]) | map(max_by(.PY_VER|split(".")|map(tonumber))) | ||
build_type: pull-request | ||
script: ci/build_wheel_cpp.sh | ||
wheel-build-python: | ||
needs: wheel-build-cpp | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
with: | ||
build_type: pull-request | ||
script: ci/build_wheel_python.sh | ||
script: ci/build_wheel_pylibraft.sh | ||
wheel-tests-pylibraft: | ||
needs: wheel-build-python | ||
needs: wheel-build-pylibraft | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
with: | ||
build_type: pull-request | ||
script: ci/test_wheel_pylibraft.sh | ||
wheel-build-raft-dask: | ||
needs: wheel-tests-pylibraft | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
with: | ||
build_type: pull-request | ||
script: "ci/build_wheel_raft_dask.sh" | ||
wheel-tests-raft-dask: | ||
needs: wheel-build-python | ||
needs: wheel-build-raft-dask | ||
secrets: inherit | ||
uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
with: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
#!/bin/bash | ||
# Copyright (c) 2023-2024, NVIDIA CORPORATION. | ||
|
||
set -euo pipefail | ||
|
||
package_name=$1 | ||
package_dir=$2 | ||
underscore_package_name=$(echo "${package_name}" | tr "-" "_") | ||
|
||
# Clear out system ucx files to ensure that we're getting ucx from the wheel. | ||
rm -rf /usr/lib64/ucx | ||
rm -rf /usr/lib64/libuc* | ||
|
||
source rapids-configure-sccache | ||
source rapids-date-string | ||
|
||
version=$(rapids-generate-version) | ||
git_commit=$(git rev-parse HEAD) | ||
|
||
RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})" | ||
|
||
# This is the version of the suffix with a preceding hyphen. It's used | ||
# everywhere except in the final wheel name. | ||
PACKAGE_CUDA_SUFFIX="-${RAPIDS_PY_CUDA_SUFFIX}" | ||
|
||
# Patch project metadata files to include the CUDA version suffix and version override. | ||
pyproject_file="${package_dir}/pyproject.toml" | ||
version_file="${package_dir}/${underscore_package_name}/_version.py" | ||
|
||
sed -i "s/name = \"${package_name}\"/name = \"${package_name}${PACKAGE_CUDA_SUFFIX}\"/g" ${pyproject_file} | ||
echo "${version}" > VERSION | ||
sed -i "/^__git_commit__ / s/= .*/= \"${git_commit}\"/g" ${version_file} | ||
|
||
# For nightlies we want to ensure that we're pulling in alphas as well. The | ||
# easiest way to do so is to augment the spec with a constraint containing a | ||
# min alpha version that doesn't affect the version bounds but does allow usage | ||
# of alpha versions for that dependency without --pre | ||
alpha_spec='' | ||
if ! rapids-is-release-build; then | ||
alpha_spec=',>=0.0.0a0' | ||
fi | ||
|
||
if [[ ${package_name} == "raft-dask" ]]; then | ||
sed -r -i "s/pylibraft==(.*)\"/pylibraft${PACKAGE_CUDA_SUFFIX}==\1${alpha_spec}\"/g" ${pyproject_file} | ||
sed -r -i "s/libucx(.*)\"/libucx${PACKAGE_CUDA_SUFFIX}\1${alpha_spec}\"/g" ${pyproject_file} | ||
sed -r -i "s/ucx-py==(.*)\"/ucx-py${PACKAGE_CUDA_SUFFIX}==\1${alpha_spec}\"/g" ${pyproject_file} | ||
sed -r -i "s/rapids-dask-dependency==(.*)\"/rapids-dask-dependency==\1${alpha_spec}\"/g" ${pyproject_file} | ||
sed -r -i "s/dask-cuda==(.*)\"/dask-cuda==\1${alpha_spec}\"/g" ${pyproject_file} | ||
sed -r -i "s/distributed-ucxx==(.*)\"/distributed-ucxx${PACKAGE_CUDA_SUFFIX}==\1${alpha_spec}\"/g" ${pyproject_file} | ||
else | ||
sed -r -i "s/rmm(.*)\"/rmm${PACKAGE_CUDA_SUFFIX}\1${alpha_spec}\"/g" ${pyproject_file} | ||
fi | ||
|
||
if [[ $PACKAGE_CUDA_SUFFIX == "-cu12" ]]; then | ||
sed -i "s/cuda-python[<=>\.,0-9a]*/cuda-python>=12.0,<13.0a0/g" ${pyproject_file} | ||
sed -i "s/cupy-cuda11x/cupy-cuda12x/g" ${pyproject_file} | ||
fi | ||
|
||
cd "${package_dir}" | ||
|
||
# Hardcode the output dir | ||
python -m pip wheel . -w dist -vvv --no-deps --disable-pip-version-check | ||
|
||
mkdir -p final_dist | ||
python -m auditwheel repair -w final_dist --exclude "libucp.so.0" dist/* | ||
|
||
RAPIDS_PY_WHEEL_NAME="${underscore_package_name}_${RAPIDS_PY_CUDA_SUFFIX}" rapids-upload-wheels-to-s3 final_dist |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
#!/bin/bash | ||
# Copyright (c) 2023-2024, NVIDIA CORPORATION. | ||
|
||
set -euo pipefail | ||
|
||
# Set up skbuild options. Enable sccache in skbuild config options | ||
export SKBUILD_CMAKE_ARGS="-DDETECT_CONDA_ENV=OFF;-DFIND_RAFT_CPP=OFF" | ||
|
||
ci/build_wheel.sh pylibraft python/pylibraft |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
#!/bin/bash | ||
# Copyright (c) 2023-2024, NVIDIA CORPORATION. | ||
|
||
set -euo pipefail | ||
|
||
# Set up skbuild options. Enable sccache in skbuild config options | ||
export SKBUILD_CMAKE_ARGS="-DDETECT_CONDA_ENV=OFF;-DFIND_RAFT_CPP=OFF" | ||
|
||
ci/build_wheel.sh raft-dask python/raft-dask |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.