Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UCXX support #1983

Merged
merged 100 commits into from
May 7, 2024
Merged

Add UCXX support #1983

merged 100 commits into from
May 7, 2024

Conversation

pentschev
Copy link
Member

@pentschev pentschev commented Nov 9, 2023

Add support for UCXX. It is our intention to soon switch from UCX-Py to UCXX and archive the former.

This PR adds support for UCXX on the C++ backend but retains the original UCX implementation for now (based on the UCP layer), moving to UCXX will simplify RAFT code a bit given the UCXX implementation requires fewer lines of boilerplate code.

On the Python front raft-dask tests are added for both UCX-Py (which there weren't any) and UCXX. The UCX-Py tests continue to use the UCX (UCP layer) implementation, whereas the UCXX tests use the UCXX C++ implementation.

When the switch is complete we can remove all previous UCX/UCX-Py code from the RAFT codebase. If for some reason using the UCX (UCP layer) is preferred on the C++ backend instead of the UCXX C++ implementation this is possible, but UCX-Py code will be archived and dropped in favor of the UCXX Python backend.

@cjnolet cjnolet added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Nov 15, 2023
@github-actions github-actions bot added the ci label Dec 12, 2023
@pentschev pentschev changed the base branch from branch-23.12 to branch-24.02 December 12, 2023 19:07
Copy link

copy-pr-bot bot commented Dec 12, 2023

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

1 similar comment
Copy link

copy-pr-bot bot commented Dec 12, 2023

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@pentschev pentschev closed this Dec 12, 2023
@pentschev pentschev reopened this Dec 12, 2023
ci/test_wheel_raft_dask.sh Outdated Show resolved Hide resolved
ci/test_wheel_raft_dask.sh Outdated Show resolved Hide resolved
ci/test_wheel_raft_dask.sh Outdated Show resolved Hide resolved
Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One change on the ucx dependency in conda, otherwise I'm generally fine with getting this PR merged.

conda/recipes/raft-dask/meta.yaml Outdated Show resolved Hide resolved
ci/test_wheel_raft_dask.sh Show resolved Hide resolved
@vyasr vyasr mentioned this pull request May 6, 2024
@vyasr
Copy link
Contributor

vyasr commented May 6, 2024

/merge

@rapids-bot rapids-bot bot merged commit 19842a2 into rapidsai:branch-24.06 May 7, 2024
69 checks passed
@pentschev pentschev deleted the ucxx-support branch May 8, 2024 12:31
abc99lr pushed a commit to abc99lr/raft that referenced this pull request May 10, 2024
Add support for [UCXX](https://github.com/rapidsai/ucxx). It is our intention to soon switch from UCX-Py to UCXX and archive the former.

This PR adds support for UCXX on the C++ backend but retains the original UCX implementation for now (based on the UCP layer), moving to UCXX will simplify RAFT code a bit given the UCXX implementation requires fewer lines of boilerplate code.

On the Python front raft-dask tests are added for both UCX-Py (which there weren't any) and UCXX. The UCX-Py tests continue to use the UCX (UCP layer) implementation, whereas the UCXX tests use the UCXX C++ implementation.

When the switch is complete we can remove all previous UCX/UCX-Py code from the RAFT codebase. If for some reason using the UCX (UCP layer) is preferred on the C++ backend instead of the UCXX C++ implementation this is possible, but UCX-Py code will be archived and dropped in favor of the UCXX Python backend.

Authors:
  - Peter Andreas Entschev (https://github.com/pentschev)
  - Akira Naruse (https://github.com/anaruse)
  - Vyas Ramasubramani (https://github.com/vyasr)
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)
  - Divye Gala (https://github.com/divyegala)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: rapidsai#1983
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci CMake cpp improvement Improvement / enhancement to an existing function non-breaking Non-breaking change python
Projects
Development

Successfully merging this pull request may close these issues.

8 participants