Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] New Graph Interface and Loaders for Distributed Sampling in DGL #4486

Merged
merged 51 commits into from
Jul 31, 2024

Conversation

alexbarghi-nv
Copy link
Member

@alexbarghi-nv alexbarghi-nv commented Jun 13, 2024

  • New Graph class that duck-types DGLGraph
  • New WholeGraph feature store that implements DGL's FeatureStorage
  • New Sampler interface
  • New Loader interface
  • New Sampler for distributed uniform neighbor sampling without dask
  • New Loader for distributed uniform neighbor sampling without dask

This PR also adds pydantic and torchdata as test dependencies of cuGraph-DGL, because they are unlisted runtime dependencies of the latest version of DGL (which we will be testing with in the new cugraph-gnn repository).

This PR also changes the test file structure to match the rest of cuGraph, by using the _mg suffix to denote multi-GPU tests and moving them to the main directory under "tests".

New examples will be added in a future PR. This PR is just for basic functionality and tests.

Closes #3805
Closes rapidsai/cugraph-gnn#4
Closes #4400
Closes #4247
Closes #3303

@alexbarghi-nv alexbarghi-nv self-assigned this Jun 13, 2024
@alexbarghi-nv alexbarghi-nv added feature request New feature or request non-breaking Non-breaking change and removed python ci conda labels Jun 13, 2024
@alexbarghi-nv alexbarghi-nv added this to the 24.08 milestone Jun 13, 2024
Copy link
Member

@VibhuJawa VibhuJawa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good to me, Have added some nits. I think we should add a function to go from dgl.DGLGraph to cugraph_dgl.Graph , having that will help a lot in testing and usability for folks.

@alexbarghi-nv
Copy link
Member Author

Mostly looks good to me, Have added some nits. I think we should add a function to go from dgl.DGLGraph to cugraph_dgl.Graph , having that will help a lot in testing and usability for folks.

I added cugraph_dgl_graph_from_heterograph which converts a DGLGraph to a cugraph_dgl.Graph along with appropriate tests.

Copy link
Member

@VibhuJawa VibhuJawa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for all the changes @alexbarghi-nv , Looks good to go from my side

Comment on lines +35 to +36
# Have to use import_optional even though these are required
# dependencies in order to build properly.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting, and unfortunate. I'm glad you added this comment.

@alexbarghi-nv
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit b38e010 into rapidsai:branch-24.08 Jul 31, 2024
130 of 131 checks passed
@alexbarghi-nv alexbarghi-nv deleted the dgl-dist-sampling branch July 31, 2024 16:11
alexbarghi-nv added a commit to rapidsai/cugraph-gnn that referenced this pull request Aug 14, 2024
seberg added a commit to seberg/cugraph that referenced this pull request Aug 24, 2024
See rapidsaigh-4486 for adding torchdata/pydantic and rapidsaigh-4464 for introducing
the torch pin due to NCCL difficulties.

It is entirely possible that this still fails due to the NCCL issue!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conda feature request New feature or request non-breaking Non-breaking change python
Projects
None yet
5 participants