Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge branch-23.10 into branch-23.12 #3898

Merged
merged 7 commits into from
Sep 28, 2023
Merged

Merge branch-23.10 into branch-23.12 #3898

merged 7 commits into from
Sep 28, 2023

Conversation

rlratzel
Copy link
Contributor

Resolves merge conflicts in #3890

trxcllnt and others added 7 commits September 27, 2023 19:08
@trxcllnt
Add cuGraph devcontainers (rapidsai#3838)
8b87915 
This PR adds some [devcontainers](https://containers.dev/) to help simplify building the cuGraph C++ and Python libraries.

It also adds an optional job to the `pr.yaml` to [build the cuGraph libs in each devcontainer](https://github.com/trxcllnt/cugraph/blob/fea/devcontainers/.github/workflows/pr.yaml#L113-L119), so the build caches are populated for devs by CI.

A devcontainer can be launched by clicking the "Reopen in Container" button that VSCode shows when opening the repo (or by using the "Rebuild and Reopen in Container" command from the command palette):
![image](https://user-images.githubusercontent.com/178183/221771999-97ab29d5-e718-4e5f-b32f-2cdd51bba25c.png)

Clicking this button will cause VSCode to prompt the user to select one of these devcontainer variants:
![image](https://github.com/rapidsai/rmm/assets/178183/68d4b264-4fc2-4008-92b6-cb4bdd19b29f)

On startup, the devcontainer creates or updates the conda/pip environment using `cugraph/dependencies.yaml`. The envs/package caches are cached on the host via volume mounts, which are described in more detail in [`.devcontainer/README.md`](https://github.com/trxcllnt/cugraph/blob/fea/devcontainers/.devcontainer/README.md).

The container includes convenience functions to clean, configure, and build the various cuGraph components:

```shell
$ clean-cugraph-cpp # only cleans the C++ build dir
$ clean-cugraph-python # only cleans the Python build dir
$ clean-cugraph # cleans both C++ and Python build dirs

$ configure-cugraph-cpp # only configures cugraph C++ lib

$ build-cugraph-cpp # only builds cugraph C++ lib
$ build-cugraph-python # only builds cugraph Python lib
$ build-cugraph # builds both C++ and Python libs
```

* The C++ build script is a small wrapper around `cmake -S ~/cugraph/cpp -B ~/cugraph/cpp/build` and `cmake --build ~/cugraph/cpp/build`
* The Python build script is a small wrapper around `pip install --editable ~/cugraph/cpp`

Unlike `build.sh`, these convenience scripts *don't* install the libraries after building them. Instead, they automatically inject the correct arguments to build the C++ libraries from source and use their build dirs as package roots:

```shell
$ cmake -S ~/cugraph/cpp -B ~/cugraph/cpp/build
$ CMAKE_ARGS="-Dcugraph_ROOT=~/cugraph/cpp/build" \ # <-- this argument is automatic
  pip install -e ~/cugraph/cpp
```

Authors:
  - Paul Taylor (https://github.com/trxcllnt)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Jake Awe (https://github.com/AyodeAwe)

URL: rapidsai#3838
@alexbarghi-nv
Integrate C++ Renumbering and Compression (rapidsai#3841)
84207c3 
- [x] C API
- [x] PLC
- [x] Python API
- [x] Bulk Sampling API
- [x] Documentation for Python SG
- [x] Documentation for Python MG
- [x] Documentation for Bulk Sampler
- [x] Resolve the C++ empty batch issue with new check
- [x] Add FutureWarnings for all temporary flags
- [x] Remove all print statements and pytest tags
- [x] Verify cuGraph-PyG and cuGraph-DGL tests

Authors:
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Tingyu Wang (https://github.com/tingyu66)
  - Seunghwa Kang (https://github.com/seunghwak)
  - Joseph Nke (https://github.com/jnke2016)
  - Brad Rees (https://github.com/BradReesWork)

URL: rapidsai#3841
@eriknw
Add entry point to tell NetworkX about nx-cugraph without importing i…
f0d6333 
…t. (rapidsai#3848)

This allows NetworkX docstrings to be updated (among other things).

This will have a companion PR in NetworkX. We still need to determine (and agree) on the dict returned by this entry point, and NetworkX doesn't need to use everything I have here. We should probably add a string for `"description"` that gives a very short description of the backend, and maybe `"url"` or `"homepage"` or whatever so online docs can have links.

Here's how to use the entry point (Python >= 3.10) after installing it:
```python
In [1]: from importlib.metadata import entry_points

In [2]: items = entry_points(group="networkx.plugin_info")

In [3]: [plugin] = items

In [4]: plugin.load()()
Out[4]:
{'backend_name': 'cugraph',
 'project': 'nx-cugraph',
 'package': 'nx_cugraph',
 'functions': {'betweenness_centrality',
  'edge_betweenness_centrality',
  'louvain_communities'},
 'extra_docstrings': {'betweenness_centrality': '`weight` parameter is not yet supported.',
  'edge_betweenness_centrality': '`weight` parameter is not yet supported.',
  'louvain_communities': '`threshold` and `seed` parameters are currently ignored.'},
 'extra_parameters': {'louvain_communities': {'max_level': 'Upper limit of the number of macro-iterations.'}}}
```

CC @rlratzel @betochimas

Authors:
  - Erik Welch (https://github.com/eriknw)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - Ray Douglass (https://github.com/raydouglass)

URL: rapidsai#3848
@jnke2016
Temporarily disable the deletion of the dask dataframe (rapidsai#3814)
f57119b 
temporarily disable the deletion of the dask dataframe

Authors:
  - Joseph Nke (https://github.com/jnke2016)
  - Naim (https://github.com/naimnv)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Rick Ratzel (https://github.com/rlratzel)

URL: rapidsai#3814
@rlratzel
Updates the source build docs to include libcugraphops as a build pre…
91fbcca 
…requisite (rapidsai#3893)

closes rapidsai#3722 

Updates the source build docs to include `libcugraphops` as a build prerequisite.

Authors:
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)

URL: rapidsai#3893
@galipremsagar
Pin dask and distributed for 23.10 release (rapidsai#3896)
b24121f 
This PR pins `dask` and `distributed` to `2023.9.2` for `23.10` release.

xref: rapidsai/cudf#14225

Authors:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)
  - Peter Andreas Entschev (https://github.com/pentschev)
  - Brad Rees (https://github.com/BradReesWork)

URL: rapidsai#3896
@rlratzel
Merge branch-23.10 into branch-23.12
8dc0493
@rlratzel rlratzel requested review from a team as code owners September 28, 2023 21:39
@raydouglass raydouglass added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Sep 28, 2023
@raydouglass raydouglass merged commit 0e7df55 into rapidsai:branch-23.12 Sep 28, 2023
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@raydouglass raydouglass raydouglass approved these changes

Assignees
No one assigned
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@rlratzel @raydouglass @trxcllnt @alexbarghi-nv @eriknw @jnke2016 @galipremsagar