[RELEASE] cugraph v24.06 #4455
Merged
[RELEASE] cugraph v24.06 #4455
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Forward-merge branch-24.04 into branch-24.06 [skip ci]
Forward-merge branch-24.04 into branch-24.06 [skip ci]
Forward-merge branch-24.04 to branch-24.06
Forward-merge branch-24.04 to branch-24.06
Closes #4190 cc: @jnke2016 This PR adds a function to `comms.py` which returns a mapping of workers to ranks. This is then sorted in `part_utils.py` before being used to submit jobs to `dask`. This should fix a bug in MG `ego_graph` where induced subgraphs were being returned in seemingly random orders (while the results are correct). Authors: - Ralph Liu (https://github.com/nv-rliu) Approvers: - Joseph Nke (https://github.com/jnke2016) - Don Acosta (https://github.com/acostadon) - Rick Ratzel (https://github.com/rlratzel) - Ray Douglass (https://github.com/raydouglass) URL: #4262
Forward-merge branch-24.04 into branch-24.06 [skip ci]
Forward-merge branch-24.04 into branch-24.06
) Always persists the dask dataframe in the cuGraph-DGL graph storage object. This resolves a bug where the dataframe was unpersisted, causing `'TypeError("Could not construct DataFrame from <class \'tuple\'>")'` to be raised when trying to access it after saving it as a dataset. Authors: - Alex Barghi (https://github.com/alexbarghi-nv) Approvers: - Vibhu Jawa (https://github.com/VibhuJawa) - Rick Ratzel (https://github.com/rlratzel) URL: #4296
`https://pypi.ngc.nvidia.com` was retired a while ago, and RAPIDS wheels are now available from `https://pypi.nvidia.com` (see https://docs.rapids.ai/install). This proposes removing one lingering reference to `pypi.ngc.nvidia.com` left over in this project's docs. ### How I tested this ```shell docker run \ --rm \ python:3.10 \ pip install --dry-run cugraph-cu12 --extra-index-url=https://pypi.nvidia.com ``` saw the expected output ```text Collecting cugraph-cu12 Downloading https://pypi.nvidia.com/cugraph-cu12/cugraph_cu12-24.2.0-cp310-cp310-manylinux_2_28_aarch64.whl (1318.6 MB) ``` This is the only use of that `ngc.nvidia.com` URL in this project. ```shell git grep -E 'ngc\.' ``` Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Don Acosta (https://github.com/acostadon) URL: #4282
…gle counting & K-hop neighbor tests to include edge masked graph view inputs (#4261) This PR pulls updates from #4253 Better be reviewed after #4253 is merged. * Update triangle count to internally use edge masking (this is cheaper than creating a new graph in both memory footprint and compute) to * include only the edges with endpoints reachable within two hops (if the list of vertices to compute triangle counts is provided) * Exclude self-loops * Exclude vertices that do not belong to 2-Core * Note that we are still creating a new graph when we are creating a degree ordered directed graph (as this graph is now significantly smaller than the original graph and edge masking has non-negligible overhead and computing triangle counts on the final degree ordered directed graph is computationally intensive). * Update SG/MG Triangle counting & K-hop neighbors C++ tests to include edge masked graph view inputs Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Joseph Nke (https://github.com/jnke2016) - Naim (https://github.com/naimnv) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4261
This PR fixes the algorithms table in `notebooks/cugraph_benchmarks/synth_release_single_gpu.ipynb` to render properly on GH.
Closes #4165 This PR removes this statement from `cugraph/structure/graph_classes.py` > Undirected Graphs are not currently supported.
This PR updates the nx-cugraph README.md with the latest upstream URLs by running the `update_readme.py` script on a current `objects.inv` file downloaded from NetworkX. The latest `objects.inv` file has updated URLs, which results in links in the README being updated too. This should resolve style check errors recently seen in [other PRs](#4320) due to the upstream change in `objects.inv` in NetworkX. Since CI runs pre-commit on all files, this caused the nx-cugraph checks to run on PRs that didn't change nx-cugraph files.
Forward-merge branch-24.04 into branch-24.06
Tests GNN examples, which were previously excluded from CI. Also updates the GNN packages to support testing with CUDA 12 in pip (not conda). Also fixes a bug in `graph_sage_sg.py` and updates `graph_sage_mg.py` to quit gracefully if dask hasn't been started. Authors: - Alex Barghi (https://github.com/alexbarghi-nv) Approvers: - Jake Awe (https://github.com/AyodeAwe) - Rick Ratzel (https://github.com/rlratzel) URL: #4317
Authors: - Don Acosta (https://github.com/acostadon) Approvers: - Brad Rees (https://github.com/BradReesWork) URL: #4272
There is an unnecessary cuda check made in the cuGraph-PyG `CuGraphStore` object causing a deprecation warning. This PR removes the check. Authors: - Alex Barghi (https://github.com/alexbarghi-nv) Approvers: - Rick Ratzel (https://github.com/rlratzel) URL: #4239
#3062 added pre-commit hooks for formatting C and C++ code. However, libcugraph_etl and the C API unit tests are currently excluded from that check. The intention was to get back around to them and include them in the testing. However, that slipped through the cracks. This PR adds the pre-commit hooks for those files as well. Authors: - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Bradley Dice (https://github.com/bdice) - Seunghwa Kang (https://github.com/seunghwak) - Mark Harris (https://github.com/harrism) URL: #4332
Previously the cugraph CMakeLists.txt genernated non-relocatable targets due to embedding absolute paths to RAFT and the CUDAToolkit. While refactoring those issues out I also made a general cleanup pass over the rest of the CMake code. Authors: - Robert Maynard (https://github.com/robertmaynard) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) - Bradley Dice (https://github.com/bdice) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4309
* Adds the ability to run `pylibcugraph` without UCX/dask within PyTorch DDP. * Adds the new distributed sampler which uses the new nccl+ddp path to perform bulk sampling. Closes #4200 Closes #4201 Closes #4246 Closes #3851 Authors: - Alex Barghi (https://github.com/alexbarghi-nv) Approvers: - Seunghwa Kang (https://github.com/seunghwak) - Rick Ratzel (https://github.com/rlratzel) - Chuck Hastings (https://github.com/ChuckHastings) - Jake Awe (https://github.com/AyodeAwe) - Joseph Nke (https://github.com/jnke2016) URL: #4278
Removes the need for us to install GTest in our cugraph CI containers. Authors: - Robert Maynard (https://github.com/robertmaynard) Approvers: - Bradley Dice (https://github.com/bdice) - Chuck Hastings (https://github.com/ChuckHastings) - Jake Awe (https://github.com/AyodeAwe) URL: #4344
#4319) Use appropriate edge src and dst properties based on is_multi_gpu flag Authors: - Naim (https://github.com/naimnv) Approvers: - Seunghwa Kang (https://github.com/seunghwak) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4319
Fix a bug potentially accessing thrust::optional object's value when the optional object does not contain a value (thrust::nullopt). This may manifest only when cugraph is built with a debug option (no deadcode elimination optimization). Authors: - Seunghwa Kang (https://github.com/seunghwak) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) - Joseph Nke (https://github.com/jnke2016) - Naim (https://github.com/naimnv) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4320
This PR fixes the [build error](https://github.com/rapidsai/cugraph/actions/runs/8713790822/job/23902856859?pr=4352#step:7:6141) in the pip devcontainers. Authors: - Paul Taylor (https://github.com/trxcllnt) Approvers: - Naim (https://github.com/naimnv) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4353
This PR gets rid of the use of the deprecated `cuco::sentinel` inline namespace. It's required to bump the cuco version in rapidsai/rapids-cmake#569 Authors: - Yunsong Wang (https://github.com/PointKernel) Approvers: - Seunghwa Kang (https://github.com/seunghwak) URL: #4274
Authors: - Naim (https://github.com/naimnv) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Jake Awe (https://github.com/AyodeAwe) URL: #4357
…4267) * Add `cugraph::test::graph_to_host_csc()` * Add `cugraph::test::print_edges()` to support debugging * Fix a bug in transform_e when an edge masked input graph view object is passed. * SG & MG PageRank * SG & MG HITS * SG & MG eigenvector centrality * SG & MG katz centrality * SG & MG betweenness centrality * SG & MG edge-betweenness centrality * SG & MG BFS * SG & MG SSSP * SG & MG extract BFS paths * SG OD shortest distances * SG & MG graph coarsening Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Naim (https://github.com/naimnv) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4267
…seed vertices in renumbering (#4329) Upon cugraph-pyg requests, this PR updates GNM sampling post processing functions to optionally consider seed vertices in renumbering. Previously, seed vertices with 0 neighbors (to sample from) are ignored in renumbering. Once this PR is merged and if seed vertices is provided, seed vertices (regardless of whether they appear in the sampled edge list or not) will have a smaller vertex ID than vertices that first appear as seed vertices' neighbors or later hops. Marked as non-breaking assuming that this functions are only internally used. Authors: - Seunghwa Kang (https://github.com/seunghwak) - Alex Barghi (https://github.com/alexbarghi-nv) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - Alex Barghi (https://github.com/alexbarghi-nv) URL: #4329
Threshold for comparing cugraph & reference HITS results (floating point numbers) was too tight leading to false alarms in H100. This PR increases the threshold to avoid the false alarm. Authors: - Seunghwa Kang (https://github.com/seunghwak) Approvers: - Naim (https://github.com/naimnv) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4352
Fixes a broken link https://github.com/rapidsai/cugraph-ops/blob/branch-23.04/README.md -> https://github.com/rapidsai/cugraph/blob/branch-24.04/readme_pages/cugraph_ops.md Authors: - Ray Bell (https://github.com/raybellwaves) - Alex Barghi (https://github.com/alexbarghi-nv) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Don Acosta (https://github.com/acostadon) URL: #4279
Restructure to pass stream instead of handle and synchronize appropriately. Closes #4236 Authors: - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Seunghwa Kang (https://github.com/seunghwak) URL: #4275
This PR restructures the current sampling primitive implementation and adds biased sampling support. Closes #4288 Authors: - Seunghwa Kang (https://github.com/seunghwak) - Chuck Hastings (https://github.com/ChuckHastings) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) URL: #4430
Closes #4440 This PR updates `enable_batch` to use the updated implementation for `replicate_edgelist`. Authors: - Ralph Liu (https://github.com/nv-rliu) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Joseph Nke (https://github.com/jnke2016) URL: #4441
Update size_ field of kv_cuco_store_t with correct values. Authors: - Naim (https://github.com/naimnv) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - Seunghwa Kang (https://github.com/seunghwak) URL: #4444
This PR 1. Performs edge triangle count in chunk 2. Enables k - 1 core optimization 3. Add C++ tests for edge triangle count 4. Move edge triangle count to the stable API 5. Implement MG edge triangle count and add tests 6. Update 'mg_graph_to_sg_graph' to support 'edge_ids' along with tests closes #4370 closes #4371 Authors: - Joseph Nke (https://github.com/jnke2016) - Rick Ratzel (https://github.com/rlratzel) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - Seunghwa Kang (https://github.com/seunghwak) URL: #4382
**[WIP]** I'm using this PR to debug/add support for `DASK_DATAFRAME__QUERY_PLANNING=True`. **NOTES**: - Depends on dask/dask-expr#1041 [Merged] - Depends on dask/dask-expr#1044 Authors: - Richard (Rick) Zamora (https://github.com/rjzamora) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Ray Douglass (https://github.com/raydouglass) URL: #4325
To fix the [CI nightly issue](https://github.com/rapidsai/cugraph/actions/runs/9188624604/job/25298484338#step:8:837) in cugraph-dgl wheel test for CUDA 11. Authors: - Tingyu Wang (https://github.com/tingyu66) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Brad Rees (https://github.com/BradReesWork) - Jake Awe (https://github.com/AyodeAwe) URL: #4447
This notebook will be used to demontstrate how to use nx-cugraph and show the speed-up. Authors: - Don Acosta (https://github.com/acostadon) Approvers: - Brad Rees (https://github.com/BradReesWork) URL: #4366
Similar to rapidsai/cudf#15552, we are testing [building RAPIDS with CCCL's main branch](NVIDIA/cccl#1667) to get ahead of any breaking changes. Authors: - Paul Taylor (https://github.com/trxcllnt) - Ralph Liu (https://github.com/nv-rliu) - Seunghwa Kang (https://github.com/seunghwak) - Ray Bell (https://github.com/raybellwaves) Approvers: - Chuck Hastings (https://github.com/ChuckHastings) - Seunghwa Kang (https://github.com/seunghwak) - Jake Awe (https://github.com/AyodeAwe) URL: #4404
added content to document c++ algorithms and fixed links that were pointing to the previously removed content resolves #4431 Resolves #4116 Authors: - Don Acosta (https://github.com/acostadon) Approvers: - Rick Ratzel (https://github.com/rlratzel) - Brad Rees (https://github.com/BradReesWork) - Chuck Hastings (https://github.com/ChuckHastings) URL: #4435
Adding a tutorial to get started with cugraph. There are more to follow but this is identified as an important one. closes #4385 Authors: - Don Acosta (https://github.com/acostadon) Approvers: - Brad Rees (https://github.com/BradReesWork) URL: #4396
Adds benchmark for `nx_cugraph.ego_graph` _Note: this code is not part of any installed package and does not affect the release. It is being added so it can be included in the `branch-24.06` sources for reference._ Authors: - Rick Ratzel (https://github.com/rlratzel) Approvers: - Alex Barghi (https://github.com/alexbarghi-nv) - Erik Welch (https://github.com/eriknw) URL: #4451
Distributed sampling in cuGraph-PyG. Also renames the existing API to clarify that it is dask based. Adds a dependency on `tensordict` for `cuGraph-PyG` which supports the new `TensorDictFeatureStore`. Also no longer installs `torch-cluster` and `torch-spline-conv` in CI for testing since that results in an `ImportError` and neither of those packages are needed. Requires PyG 2.5. Should be merged after #4335 Merge after #4355 Closes #4248 Closes #4249 Closes #3383 Closes #3942 Closes #3836 Closes #4202 Closes #4051 Closes #4326 Closes #4252 Partially addresses #3805 Authors: - Alex Barghi (https://github.com/alexbarghi-nv) - Seunghwa Kang (https://github.com/seunghwak) - Tingyu Wang (https://github.com/tingyu66) - Ralph Liu (https://github.com/nv-rliu) Approvers: - Tingyu Wang (https://github.com/tingyu66) - Brad Rees (https://github.com/BradReesWork) - Jake Awe (https://github.com/AyodeAwe) URL: #4384
Reimplements the WG feature store for PyG using the `FeatureStore` interface. Merge after #4384 Closes rapidsai/wholegraph#47 Closes #4399 Authors: - Alex Barghi (https://github.com/alexbarghi-nv) - Seunghwa Kang (https://github.com/seunghwak) - Tingyu Wang (https://github.com/tingyu66) - Ralph Liu (https://github.com/nv-rliu) Approvers: - Tingyu Wang (https://github.com/tingyu66) - Vibhu Jawa (https://github.com/VibhuJawa) - Brad Rees (https://github.com/BradReesWork) - Ray Douglass (https://github.com/raydouglass) URL: #4432
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
❄️ Code freeze for
branch-24.06and v24.06 releaseWhat does this mean?
Only critical/hotfix level issues should be merged into
branch-24.06until release (merging of this PR).What is the purpose of this PR?
branch-24.06intomainfor the release