Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Python 3.12, update Kafka dependencies to 2.5.x #16745

Merged
merged 13 commits into from
Sep 6, 2024
Merged

Add support for Python 3.12, update Kafka dependencies to 2.5.x #16745

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d4ba292
Add support for Python 3.12.
jameslamb Sep 4, 2024
8b29502
Merge branch 'branch-24.10' into python-3.12
jameslamb Sep 4, 2024
05ae241
Merge branch 'branch-24.10' of github.com:rapidsai/cudf into python-3.12
jameslamb Sep 5, 2024
8c206d7
upgrade python-confluent-kafka and librdkafka to 2.5.x
jameslamb Sep 5, 2024
b69a380
resolve deprecation warning about ast.Num
jameslamb Sep 5, 2024
0cf85e1
Merge branch 'python-3.12' of github.com:jameslamb/cudf into python-3.12
jameslamb Sep 5, 2024
cd70846
ignore warnings about datetime.datetime.utcnow()
jameslamb Sep 5, 2024
afafa83
skip doctests on to_dict() methods
jameslamb Sep 5, 2024
54d96ce
remove extra trailing comma
jameslamb Sep 5, 2024
3af92ef
typo
jameslamb Sep 5, 2024
d87b19c
cudf gets its warnings-filtering from pytest.ini, not pyproject.toml
jameslamb Sep 5, 2024
e727fef
Merge branch 'branch-24.10' of github.com:rapidsai/cudf into python-3.12
jameslamb Sep 5, 2024
d70a9a3
Merge branch 'branch-24.10' into python-3.12
jameslamb Sep 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ concurrency:
jobs:
cpp-build:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -37,7 +37,7 @@ jobs:
python-build:
needs: [cpp-build]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -46,7 +46,7 @@ jobs:
upload-conda:
needs: [cpp-build, python-build]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-upload-packages.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-upload-packages.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -57,7 +57,7 @@ jobs:
if: github.ref_type == 'branch'
needs: python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.12
with:
arch: "amd64"
branch: ${{ inputs.branch }}
Expand All @@ -69,7 +69,7 @@ jobs:
sha: ${{ inputs.sha }}
wheel-build-libcudf:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
# build for every combination of arch and CUDA version, but only for the latest Python
matrix_filter: group_by([.ARCH, (.CUDA_VER|split(".")|map(tonumber)|.[0])]) | map(max_by(.PY_VER|split(".")|map(tonumber)))
Expand All @@ -81,7 +81,7 @@ jobs:
wheel-publish-libcudf:
needs: wheel-build-libcudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -92,7 +92,7 @@ jobs:
wheel-build-pylibcudf:
needs: [wheel-publish-libcudf]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -102,7 +102,7 @@ jobs:
wheel-publish-pylibcudf:
needs: wheel-build-pylibcudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -113,7 +113,7 @@ jobs:
wheel-build-cudf:
needs: wheel-publish-pylibcudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -123,7 +123,7 @@ jobs:
wheel-publish-cudf:
needs: wheel-build-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -134,7 +134,7 @@ jobs:
wheel-build-dask-cudf:
needs: wheel-publish-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]))
Expand All @@ -146,7 +146,7 @@ jobs:
wheel-publish-dask-cudf:
needs: wheel-build-dask-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand All @@ -157,7 +157,7 @@ jobs:
wheel-build-cudf-polars:
needs: wheel-publish-pylibcudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]))
Expand All @@ -169,7 +169,7 @@ jobs:
wheel-publish-cudf-polars:
needs: wheel-build-cudf-polars
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-publish.yaml@python-3.12
with:
build_type: ${{ inputs.build_type || 'branch' }}
branch: ${{ inputs.branch }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pandas-tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
pandas-tests:
# run the Pandas unit tests
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.12
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]))
Expand Down
48 changes: 24 additions & 24 deletions .github/workflows/pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
- pandas-tests
- pandas-tests-diff
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/pr-builder.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/pr-builder.yaml@python-3.12
if: always()
with:
needs: ${{ toJSON(needs) }}
Expand Down Expand Up @@ -104,39 +104,39 @@ jobs:
- '!notebooks/**'
checks:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/checks.yaml@python-3.12
with:
enable_check_generated_files: false
conda-cpp-build:
needs: checks
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-build.yaml@python-3.12
with:
build_type: pull-request
conda-cpp-checks:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-post-build-checks.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-post-build-checks.yaml@python-3.12
with:
build_type: pull-request
enable_check_symbols: true
conda-cpp-tests:
needs: [conda-cpp-build, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-cpp-tests.yaml@python-3.12
if: needs.changed-files.outputs.test_cpp == 'true'
with:
build_type: pull-request
conda-python-build:
needs: conda-cpp-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-python-build.yaml@python-3.12
with:
build_type: pull-request
conda-python-cudf-tests:
needs: [conda-python-build, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@python-3.12
if: needs.changed-files.outputs.test_python == 'true'
with:
build_type: pull-request
Expand All @@ -145,15 +145,15 @@ jobs:
# Tests for dask_cudf, custreamz, cudf_kafka are separated for CI parallelism
needs: [conda-python-build, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/conda-python-tests.yaml@python-3.12
if: needs.changed-files.outputs.test_python == 'true'
with:
build_type: pull-request
script: "ci/test_python_other.sh"
conda-java-tests:
needs: [conda-cpp-build, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.12
if: needs.changed-files.outputs.test_java == 'true'
with:
build_type: pull-request
Expand All @@ -164,7 +164,7 @@ jobs:
static-configure:
needs: checks
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.12
with:
build_type: pull-request
# Use the wheel container so we can skip conda solves and since our
Expand All @@ -174,7 +174,7 @@ jobs:
conda-notebook-tests:
needs: [conda-python-build, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.12
if: needs.changed-files.outputs.test_notebooks == 'true'
with:
build_type: pull-request
Expand All @@ -185,7 +185,7 @@ jobs:
docs-build:
needs: conda-python-build
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.12
with:
build_type: pull-request
node_type: "gpu-v100-latest-1"
Expand All @@ -195,7 +195,7 @@ jobs:
wheel-build-libcudf:
needs: checks
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
# build for every combination of arch and CUDA version, but only for the latest Python
matrix_filter: group_by([.ARCH, (.CUDA_VER|split(".")|map(tonumber)|.[0])]) | map(max_by(.PY_VER|split(".")|map(tonumber)))
Expand All @@ -204,29 +204,29 @@ jobs:
wheel-build-pylibcudf:
needs: [checks, wheel-build-libcudf]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
build_type: pull-request
script: "ci/build_wheel_pylibcudf.sh"
wheel-build-cudf:
needs: wheel-build-pylibcudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
build_type: pull-request
script: "ci/build_wheel_cudf.sh"
wheel-tests-cudf:
needs: [wheel-build-cudf, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.12
if: needs.changed-files.outputs.test_python == 'true'
with:
build_type: pull-request
script: ci/test_wheel_cudf.sh
wheel-build-cudf-polars:
needs: wheel-build-pylibcudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]))
Expand All @@ -235,7 +235,7 @@ jobs:
wheel-tests-cudf-polars:
needs: [wheel-build-cudf-polars, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.12
if: needs.changed-files.outputs.test_python == 'true'
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
Expand All @@ -247,7 +247,7 @@ jobs:
wheel-build-dask-cudf:
needs: wheel-build-cudf
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-build.yaml@python-3.12
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))]))
Expand All @@ -256,7 +256,7 @@ jobs:
wheel-tests-dask-cudf:
needs: [wheel-build-dask-cudf, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.12
if: needs.changed-files.outputs.test_python == 'true'
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
Expand All @@ -265,7 +265,7 @@ jobs:
script: ci/test_wheel_dask_cudf.sh
devcontainer:
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/build-in-devcontainer.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/build-in-devcontainer.yaml@python-3.12
with:
arch: '["amd64"]'
cuda: '["12.5"]'
Expand All @@ -276,7 +276,7 @@ jobs:
unit-tests-cudf-pandas:
needs: [wheel-build-cudf, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.12
if: needs.changed-files.outputs.test_python == 'true'
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
Expand All @@ -287,7 +287,7 @@ jobs:
# run the Pandas unit tests using PR branch
needs: [wheel-build-cudf, changed-files]
secrets: inherit
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/wheels-test.yaml@python-3.12
if: needs.changed-files.outputs.test_python == 'true'
with:
# This selects "ARCH=amd64 + the latest supported Python + CUDA".
Expand All @@ -299,7 +299,7 @@ jobs:
pandas-tests-diff:
# diff the results of running the Pandas unit tests and publish a job summary
needs: pandas-tests
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@python-3.12
with:
node_type: cpu4
build_type: pull-request
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/pr_issue_status_automation.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ on:

jobs:
get-project-id:
uses: rapidsai/shared-workflows/.github/workflows/project-get-item-id.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/project-get-item-id.yaml@python-3.12
if: github.event.pull_request.state == 'open'
secrets: inherit
permissions:
Expand All @@ -34,7 +34,7 @@ jobs:

update-status:
# This job sets the PR and its linked issues to "In Progress" status
uses: rapidsai/shared-workflows/.github/workflows/project-get-set-single-select-field.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/project-get-set-single-select-field.yaml@python-3.12
if: ${{ github.event.pull_request.state == 'open' && needs.get-project-id.outputs.ITEM_PROJECT_ID != '' }}
needs: get-project-id
with:
Expand All @@ -50,7 +50,7 @@ jobs:

update-sprint:
# This job sets the PR and its linked issues to the current "Weekly Sprint"
uses: rapidsai/shared-workflows/.github/workflows/project-get-set-iteration-field.yaml@branch-24.10
uses: rapidsai/shared-workflows/.github/workflows/project-get-set-iteration-field.yaml@python-3.12
if: ${{ github.event.pull_request.state == 'open' && needs.get-project-id.outputs.ITEM_PROJECT_ID != '' }}
needs: get-project-id
with:
Expand Down
Loading
Loading