feat(docker): integrate `cuda`/`no-cuda` jobs into single job #5363

youtalk · 2024-10-23T13:59:40Z

Description

Resolved #5082

As a result of the activity autowarefoundation/autoware.universe#8695, the only packages dependent on CUDA are the sensing/perception components of autoware.universe. Therefore, the container images with the -cuda suffix are actually only needed for universe-sensing-perception-devel and universe-sensing-perception.

Additionally, since there is currently only one arm64 self-hosted runner, the no-cuda/cuda jobs in the docker-build-and-push-arm64 workflow experience double the waiting time.

This PR consolidates the no-cuda/cuda jobs and changes the process so that only the CUDA dependent packages in the sensing/perception component are built in an additional stage.

Tests performed

https://github.com/autowarefoundation/autoware/actions/runs/11718481450
https://github.com/autowarefoundation/autoware/actions/runs/11718483615

Effects on system behavior

Not applicable.

Interface changes

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.
The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

The PR follows the pull request guidelines.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

Signed-off-by: Yutaka Kondo <[email protected]>

This reverts commit 0c7459d.

Signed-off-by: Yutaka Kondo <[email protected]>

youtalk · 2024-10-28T06:52:17Z

@wenrir

Additionally, have you considered using matrix instead? In order to combine the cude and non-cuda workflows? they seem to do basically the same thing.

One of the purposes of this PR is rather to stop the matrix, because in this organization, there is currently only one arm64 self-hosted runner available, and using the matrix would require double the execution time.
https://github.com/autowarefoundation/autoware/pull/5363/files#diff-de0ada9957e6211b26d1e4d4d49c0525a9f36224a5b8c743b5ff7339130f2aabL17-L34

wenrir · 2024-10-28T07:31:37Z

@youtalk, thank you for the quick action and response, I understand (and agree) with the feedback.

(I'm unable to resolve my own comments, sorry)

oguzkaganozt

@youtalk I think in overal terms the PR solves the complicated nature of docker-build workflows and better utilizes the cache by seperating CUDA and non-CUDA builds.

But in the other hand Dockerfile would become rather complicated to track and maintain, since it is not preferrable to have multiple Dockerfiles I suggest putting some of the recurring scripts to a directory.

For example:

docker/
├── Dockerfile
├── scripts/
│   ├── install-rosdep-dependencies.sh
│   ├── build-ros-packages.sh
│   └── cleanup.sh
└── etc/
    └── ros_entrypoint.sh

I think this kind of improvement can be done in a seperate PR, and I can draft a PR for that after this PR.

And also with this PR we are not doing health-check builds with CUDA anymore as i can understand or am i missing something ? And if so would it miss some CUDA related build fails ?

Signed-off-by: Yutaka Kondo <[email protected]>

youtalk · 2024-11-07T07:37:07Z

@oguzkaganozt I've fix a minor cache problem c009784. I hope the CI jobs will be succeeded.

https://github.com/autowarefoundation/autoware/actions/runs/11718481450
https://github.com/autowarefoundation/autoware/actions/runs/11718483615

@mitsudome-r @xmfcx docker-build (cuda) and docker-build (no-cuda) will no longer be executed, so please remove them from the required CI checks if you approved this PR. And after this PR is merged, please add a required check for docker-build.

xmfcx · 2024-11-07T07:41:35Z

Removed ✔️

oguzkaganozt

LGTM

* chore(docker): remove `/autoware/log` after `colcon build` (autowarefoundation#5329) * chore(.github): always run `Show disk space` (autowarefoundation#5354) always show disk space Signed-off-by: Yutaka Kondo <[email protected]> * build(autoware.repos): remove ament_cmake fork repository (autowarefoundation#5360) Signed-off-by: Chengyong Lin <[email protected]> * fix: remove `ndt_omp` (autowarefoundation#5390) Removed ndt_omp Signed-off-by: Shintaro Sakoda <[email protected]> * fix: change the organization of `awsim_sensor_kit_launch` from RobotecAI to tier4 (autowarefoundation#5403) Fixed organization of `awsim_sensor_kit_launch` from RobotecAI to tier4 Signed-off-by: Shintaro Sakoda <[email protected]> * chore(.devcontainer): rename `.devcontainer` directories (autowarefoundation#5407) rename .devcontainer dirs Signed-off-by: Yutaka Kondo <[email protected]> * feat(docker): integrate `cuda`/`no-cuda` jobs into single job (autowarefoundation#5363) --------- Signed-off-by: Yutaka Kondo <[email protected]> Signed-off-by: Chengyong Lin <[email protected]> Signed-off-by: Shintaro Sakoda <[email protected]> Co-authored-by: chgyg <[email protected]> Co-authored-by: SakodaShintaro <[email protected]> Co-authored-by: SakodaShintaro <[email protected]>

youtalk · 2024-11-08T02:44:05Z

@mitsudome-r @xmfcx docker-build (cuda) and docker-build (no-cuda) will no longer be executed, so please remove them from the required CI checks if you approved this PR. And after this PR is merged, please add a required check for docker-build.

@mitsudome-r @xmfcx Please add a required check for docker-build.

xmfcx · 2024-11-08T08:14:24Z

@youtalk -san, docker-build is a composite action, not a workflow.

Do you mean to add docker-build-and-push as a required workflow?

But that workflow only runs on:

on:
  push:
    branches:
      - main
    tags:
  workflow_dispatch:

not every PR.

What should I do then?

xmfcx · 2024-11-08T13:09:10Z

Sorry, I've just realized it was part of the health-check.
I've added it to the required checks, tested in:

feat(.github): split docker-build-and-push step to no-cuda and cuda steps #5414

youtalk · 2024-11-09T01:45:27Z

@xmfcx Thank you!

* chore(docker): remove `/autoware/log` after `colcon build` (autowarefoundation#5329) * chore(.github): always run `Show disk space` (autowarefoundation#5354) always show disk space Signed-off-by: Yutaka Kondo <[email protected]> * build(autoware.repos): remove ament_cmake fork repository (autowarefoundation#5360) Signed-off-by: Chengyong Lin <[email protected]> * fix: remove `ndt_omp` (autowarefoundation#5390) Removed ndt_omp Signed-off-by: Shintaro Sakoda <[email protected]> * fix: change the organization of `awsim_sensor_kit_launch` from RobotecAI to tier4 (autowarefoundation#5403) Fixed organization of `awsim_sensor_kit_launch` from RobotecAI to tier4 Signed-off-by: Shintaro Sakoda <[email protected]> * chore(.devcontainer): rename `.devcontainer` directories (autowarefoundation#5407) rename .devcontainer dirs Signed-off-by: Yutaka Kondo <[email protected]> * feat(docker): integrate `cuda`/`no-cuda` jobs into single job (autowarefoundation#5363) * chore(.github): rename `bake-target` to `target-image` and add descriptions to args (autowarefoundation#5413) * rename target-image and add descriptions Signed-off-by: Yutaka Kondo <[email protected]> * fix feedback Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * feat(docker): install CUDA development/runtime libraries only 1 time (autowarefoundation#5419) * add base-cuda Signed-off-by: Yutaka Kondo <[email protected]> * update svg Signed-off-by: Yutaka Kondo <[email protected]> * not cache cuda packages Signed-off-by: Yutaka Kondo <[email protected]> * push base-cuda image Signed-off-by: Yutaka Kondo <[email protected]> * Update docker-build-and-push.yaml * Update docker-build-and-push-arm64.yaml * separate jobs Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> * refactor(docker): replace the multiple `rosdep` commands with `resolve_rosdep_keys.sh`. (autowarefoundation#5424) * run resolve_rosdep_keys.sh Signed-off-by: Yutaka Kondo <[email protected]> * style(pre-commit): autofix * update .dockerignore Signed-off-by: Yutaka Kondo <[email protected]> * chmod +x Signed-off-by: Yutaka Kondo <[email protected]> * fix location Signed-off-by: Yutaka Kondo <[email protected]> * remove rosdep update Signed-off-by: Yutaka Kondo <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * refactor(docker): replace the multiple `colcon` commands with `build_and_clean.sh`. (autowarefoundation#5425) * run resolve_rosdep_keys.sh Signed-off-by: Yutaka Kondo <[email protected]> * style(pre-commit): autofix * update .dockerignore Signed-off-by: Yutaka Kondo <[email protected]> * chmod +x Signed-off-by: Yutaka Kondo <[email protected]> * fix location Signed-off-by: Yutaka Kondo <[email protected]> * remove rosdep update Signed-off-by: Yutaka Kondo <[email protected]> * run build_and_clean.sh Signed-off-by: Yutaka Kondo <[email protected]> * style(pre-commit): autofix --------- Signed-off-by: Yutaka Kondo <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * refactor(docker): replace the multiple `rm -rf` commands with `cleanup_system.sh`. (autowarefoundation#5426) * fix(.github): fix `target-image` (autowarefoundation#5428) fix target-image Signed-off-by: Yutaka Kondo <[email protected]> * chore: remove tvm_utility artifacts from Ansible tasks (autowarefoundation#5427) Signed-off-by: Esteve Fernandez <[email protected]> --------- Signed-off-by: Yutaka Kondo <[email protected]> Signed-off-by: Chengyong Lin <[email protected]> Signed-off-by: Shintaro Sakoda <[email protected]> Signed-off-by: Esteve Fernandez <[email protected]> Co-authored-by: chgyg <[email protected]> Co-authored-by: SakodaShintaro <[email protected]> Co-authored-by: SakodaShintaro <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Esteve Fernandez <[email protected]>

youtalk added 18 commits October 23, 2024 22:13

add cuda-devel stage

58cd0f3

Signed-off-by: Yutaka Kondo <[email protected]>

remove setup_args

3e8a539

Signed-off-by: Yutaka Kondo <[email protected]>

update comments

3038e56

Signed-off-by: Yutaka Kondo <[email protected]>

update dockerfile

c4037be

Signed-off-by: Yutaka Kondo <[email protected]>

update workflow

298362f

Signed-off-by: Yutaka Kondo <[email protected]>

fix

a47546f

Signed-off-by: Yutaka Kondo <[email protected]>

fix

3e98cc8

Signed-off-by: Yutaka Kondo <[email protected]>

use buildjet

ded34cd

Signed-off-by: Yutaka Kondo <[email protected]>

update docker-build-and-push

bc9862a

Signed-off-by: Yutaka Kondo <[email protected]>

rename cuda stages

6adc59a

Signed-off-by: Yutaka Kondo <[email protected]>

update cuda packages

295589e

Signed-off-by: Yutaka Kondo <[email protected]>

add cuda workflow

da6f153

Signed-off-by: Yutaka Kondo <[email protected]>

divide jobs

48b0c5f

Signed-off-by: Yutaka Kondo <[email protected]>

fix

84acbfb

Signed-off-by: Yutaka Kondo <[email protected]>

not divide jobs

701e968

Signed-off-by: Yutaka Kondo <[email protected]>

remove redundant

22dec98

Signed-off-by: Yutaka Kondo <[email protected]>

integrate

059e869

Signed-off-by: Yutaka Kondo <[email protected]>

Revert "integrate"

b22a9c7

This reverts commit 0c7459d.

youtalk added type:containers Docker containers, containerization of components, or container orchestration. component:openadkit Issues or Features related to Open AD Kit tag:run-health-check Run health-check labels Oct 23, 2024

youtalk self-assigned this Oct 23, 2024

fix rebase

37adf25

Signed-off-by: Yutaka Kondo <[email protected]>

youtalk force-pushed the upstream-refine-cuda-workflow branch from d63ab93 to 37adf25 Compare October 23, 2024 21:44

fix rebase

e38ae2d

Signed-off-by: Yutaka Kondo <[email protected]>

youtalk marked this pull request as ready for review October 23, 2024 22:10

youtalk requested a review from oguzkaganozt as a code owner October 23, 2024 22:10

youtalk requested review from xmfcx, kaspermeck-arm and mitsudome-r October 23, 2024 22:10

fix precommit

4610f84

Signed-off-by: Yutaka Kondo <[email protected]>

oguzkaganozt reviewed Oct 31, 2024

View reviewed changes

youtalk added 4 commits November 7, 2024 16:14

change cache-from/to

c009784

Signed-off-by: Yutaka Kondo <[email protected]>

update cuda_utils

add210d

Signed-off-by: Yutaka Kondo <[email protected]>

update svg

cda2bd1

Signed-off-by: Yutaka Kondo <[email protected]>

Merge branch 'main' into upstream-refine-cuda-workflow

db5cb42

xmfcx closed this Nov 7, 2024

xmfcx reopened this Nov 7, 2024

oguzkaganozt self-requested a review November 7, 2024 15:12

oguzkaganozt approved these changes Nov 7, 2024

View reviewed changes

youtalk merged commit 9dca304 into main Nov 7, 2024
29 of 30 checks passed

youtalk deleted the upstream-refine-cuda-workflow branch November 7, 2024 22:39

tier4-autoware-public-bot bot mentioned this pull request Nov 8, 2024

chore: sync upstream tier4/autoware#5

Open

youtalk mentioned this pull request Nov 8, 2024

chore(.github): rename bake-target to target-image and add descriptions to args #5413

Merged

4 tasks

youtalk mentioned this pull request Nov 11, 2024

feat(docker): install CUDA development/runtime libraries only 1 time #5419

Merged

4 tasks

youtalk mentioned this pull request Dec 18, 2024

ci(build-and-test*): use autoware:core-devel image and reconfigure workflows autowarefoundation/autoware.core#129

Closed

youtalk mentioned this pull request Jan 14, 2025

docs(docker): update Multi-stage Dockerfile structure section #5651

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(docker): integrate `cuda`/`no-cuda` jobs into single job #5363

feat(docker): integrate `cuda`/`no-cuda` jobs into single job #5363

youtalk commented Oct 23, 2024 •

edited

Loading

youtalk commented Oct 28, 2024

wenrir commented Oct 28, 2024

oguzkaganozt left a comment

youtalk commented Nov 7, 2024

xmfcx commented Nov 7, 2024

oguzkaganozt left a comment

youtalk commented Nov 8, 2024

xmfcx commented Nov 8, 2024 •

edited

Loading

xmfcx commented Nov 8, 2024

youtalk commented Nov 9, 2024

feat(docker): integrate cuda/no-cuda jobs into single job #5363

feat(docker): integrate cuda/no-cuda jobs into single job #5363

Conversation

youtalk commented Oct 23, 2024 • edited Loading

Description

Tests performed

Effects on system behavior

Interface changes

Pre-review checklist for the PR author

In-review checklist for the PR reviewers

Post-review checklist for the PR author

youtalk commented Oct 28, 2024

wenrir commented Oct 28, 2024

oguzkaganozt left a comment

Choose a reason for hiding this comment

youtalk commented Nov 7, 2024

xmfcx commented Nov 7, 2024

oguzkaganozt left a comment

Choose a reason for hiding this comment

youtalk commented Nov 8, 2024

xmfcx commented Nov 8, 2024 • edited Loading

xmfcx commented Nov 8, 2024

youtalk commented Nov 9, 2024

feat(docker): integrate `cuda`/`no-cuda` jobs into single job #5363

feat(docker): integrate `cuda`/`no-cuda` jobs into single job #5363

youtalk commented Oct 23, 2024 •

edited

Loading

xmfcx commented Nov 8, 2024 •

edited

Loading