Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forward-merge branch-23.10 to branch-23.12 #3904

Merged
merged 8 commits into from
Oct 14, 2023
Merged

Conversation

GPUtester
Copy link
Contributor

Forward-merge triggered by push to branch-23.10 that creates a PR to keep branch-23.12 up-to-date. If this PR is unable to be immediately merged due to conflicts, it will remain open for the team to manually merge.

Created based on code from @dongxuy04 

Adds support for `WholeGraph` `WholeMemory` in the cuGraph `FeatureStore` class.  This enables both DGL and PyG to take advantage of distributed feature store functionality.

Adds `pylibwholegraph` as a testing dependency so the feature store can be tested.  Adds appropriate SG and MG tests.

Authors:
  - Alex Barghi (https://github.com/alexbarghi-nv)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)
  - Brad Rees (https://github.com/BradReesWork)
  - Vibhu Jawa (https://github.com/VibhuJawa)

URL: #3874
@GPUtester GPUtester requested review from a team as code owners September 30, 2023 23:45
@GPUtester
Copy link
Contributor Author

FAILURE - Unable to forward-merge due to conflicts, manual merge is necessary. Do not use the Resolve conflicts option in this PR, follow these instructions https://docs.rapids.ai/maintainers/forward-merger/
IMPORTANT: When merging this PR, do not use the auto-merger (i.e. the /merge comment). Instead, an admin must manually merge by changing the merging strategy to Create a Merge Commit. Otherwise, history will be lost and the branches become incompatible.

tingyu66 and others added 5 commits October 3, 2023 17:45
…MFG creation (#3887)

Allow cugraph-dgl dataloader to consume sampled outputs from BulkSampler in CSC format.

Authors:
  - Tingyu Wang (https://github.com/tingyu66)
  - Seunghwa Kang (https://github.com/seunghwak)
  - Alex Barghi (https://github.com/alexbarghi-nv)

Approvers:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - Vibhu Jawa (https://github.com/VibhuJawa)

URL: #3887
This handles isolated nodes in `louvain_communities` similar to what is done in #3886. This is expected to be a temporary fix until pylibcugraph can handle isolated nodes.

As a bonus, I added `isolates` algorithm 🎉

CC @naimnv @rlratzel

Authors:
  - Erik Welch (https://github.com/eriknw)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #3897
Integrates the new CSR bulk sampler output, allowing reading of batches without having to call CSC conversion or count the numbers of vertices and edges in each batch.  Should result in major performance improvements, especially for small batches.

Authors:
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - Seunghwa Kang (https://github.com/seunghwak)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Ray Douglass (https://github.com/raydouglass)
  - Tingyu Wang (https://github.com/tingyu66)

URL: #3873
This PR increases the minimum timeout when waiting for the workers to complete their tasks.

Authors:
  - Joseph Nke (https://github.com/jnke2016)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Vibhu Jawa (https://github.com/VibhuJawa)
  - Rick Ratzel (https://github.com/rlratzel)
  - Jake Awe (https://github.com/AyodeAwe)

URL: #3907
… `cugraph.Graph` (#3895)

This PR attempts to fix #3790 

Please note that I  have not being able to cause failure locally so it is really hard for me to know if it actually fixes anything or not  . 

MRE being used to test locally: https://gist.github.com/VibhuJawa/4b1ec24022b6e2dd7879cd2e8d3fab67


CC: @jnke2016 , @rlratzel ,

CC:  @rjzamora , Please let me know what i can do better here.

Authors:
  - Vibhu Jawa (https://github.com/VibhuJawa)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - Joseph Nke (https://github.com/jnke2016)

URL: #3895
Running certain notebooks requires downloading datasets via https://github.com/rapidsai/cugraph/blob/branch-23.10/notebooks/cugraph_benchmarks/dataPrep.sh which uses `wget`, so this PR ensures the `test_notebooks` environment created in `rapidsai/docker` will have `wget` installed.

Authors:
   - Ray Douglass (https://github.com/raydouglass)

Approvers:
   - Jake Awe (https://github.com/AyodeAwe)
@codecov-commenter
Copy link

codecov-commenter commented Oct 7, 2023

Codecov Report

Attention: 62 lines in your changes are missing coverage. Please review.

Comparison is base (9dd01d8) 68.36% compared to head (40a5f8e) 68.82%.
Report is 16 commits behind head on branch-23.12.

Additional details and impacted files
@@               Coverage Diff                @@
##           branch-23.12    #3904      +/-   ##
================================================
+ Coverage         68.36%   68.82%   +0.46%     
================================================
  Files               196      198       +2     
  Lines             13344    13609     +265     
================================================
+ Hits               9122     9367     +245     
- Misses             4222     4242      +20     
Files Coverage Δ
...on/cugraph-dgl/cugraph_dgl/dataloading/__init__.py 100.00% <ø> (ø)
...ure/graph_implementation/simpleDistributedGraph.py 77.85% <100.00%> (+0.25%) ⬆️
...ython/nx-cugraph/nx_cugraph/algorithms/__init__.py 100.00% <100.00%> (ø)
python/nx-cugraph/nx_cugraph/classes/digraph.py 86.95% <ø> (+2.95%) ⬆️
python/nx-cugraph/nx_cugraph/convert.py 89.14% <ø> (+2.59%) ⬆️
.../cugraph_dgl/dataloading/utils/sampling_helpers.py 96.96% <98.03%> (+0.28%) ⬆️
python/cugraph-dgl/cugraph_dgl/nn/conv/base.py 87.68% <50.00%> (-0.56%) ⬇️
python/cugraph/cugraph/dask/common/part_utils.py 40.58% <94.73%> (+3.87%) ⬆️
...hon/cugraph-dgl/cugraph_dgl/dataloading/dataset.py 91.11% <77.77%> (-1.92%) ⬇️
...thon/cugraph-pyg/cugraph_pyg/data/cugraph_store.py 81.13% <75.00%> (ø)
... and 8 more

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@BradReesWork BradReesWork added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 9, 2023
@GPUtester GPUtester requested a review from a team as a code owner October 11, 2023 14:28
@rlratzel
Copy link
Contributor

/ok to test

@raydouglass raydouglass merged commit 98cf958 into branch-23.12 Oct 14, 2023
221 of 239 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants