Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(docker): revert "cache apt archives to the Docker build cache" #4793

Merged
merged 2 commits into from
Jun 3, 2024
Merged

feat(docker): revert "cache apt archives to the Docker build cache" #4793

merged 2 commits into from
Jun 3, 2024

Conversation

youtalk
Copy link
Member

@youtalk youtalk commented May 29, 2024

Reverts #4778

Even though sharing=locked is set, it seems that could not open lock file sometimes occurs, so I revert it until the issue is resolved.

https://github.com/autowarefoundation/autoware/actions/runs/9284779370/job/25547896727#step:7:1852

#47 108.7 fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to lock apt for exclusive operation: Failed to lock directory /var/lib/apt/lists/: E:Could not open lock file /var/lib/apt/lists/lock - open (2: No such file or directory)"}

@youtalk youtalk changed the title Revert "feat(docker): cache apt archives to the Docker build cache" feat(docker): revert "cache apt archives to the Docker build cache" May 29, 2024
@youtalk youtalk marked this pull request as ready for review May 29, 2024 12:08
@youtalk youtalk requested review from mitsudome-r and xmfcx as code owners May 29, 2024 12:08
@youtalk youtalk requested a review from oguzkaganozt May 29, 2024 12:08
@youtalk youtalk self-assigned this May 29, 2024
@youtalk youtalk enabled auto-merge (squash) May 29, 2024 12:08
@youtalk
Copy link
Member Author

youtalk commented May 30, 2024

I'm really not sure but the heavier cuda job was succeded but the ligher no-cuda was failed.
I'm trying to rerun https://github.com/autowarefoundation/autoware/actions/runs/9284779370/job/25581422108

@youtalk youtalk disabled auto-merge May 30, 2024 03:44
@youtalk youtalk closed this May 30, 2024
@youtalk youtalk deleted the revert-4778-upstream-cache-apt-archives branch May 30, 2024 03:45
@youtalk
Copy link
Member Author

youtalk commented May 30, 2024

The second try https://github.com/autowarefoundation/autoware/actions/runs/9284779370/job/25581422108 was succeeded.
I'll decide whether to revert after observing the changes in the build success or failure a little longer.

@youtalk youtalk restored the revert-4778-upstream-cache-apt-archives branch June 3, 2024 06:52
@youtalk
Copy link
Member Author

youtalk commented Jun 3, 2024

It is clear that the docker-build-and-push-main has started to fail after it.#4778, so I will revert.

image

@youtalk youtalk reopened this Jun 3, 2024
@youtalk youtalk enabled auto-merge (squash) June 3, 2024 06:58
@youtalk
Copy link
Member Author

youtalk commented Jun 3, 2024

@oguzkaganozt @mitsudome-r Please review this.

@xmfcx xmfcx disabled auto-merge June 3, 2024 14:08
@xmfcx xmfcx merged commit d192a60 into main Jun 3, 2024
13 checks passed
@xmfcx xmfcx deleted the revert-4778-upstream-cache-apt-archives branch June 3, 2024 14:08
@tier4-autoware-public-bot tier4-autoware-public-bot bot mentioned this pull request Jun 4, 2024
Open
youtalk added a commit to youtalk/autoware that referenced this pull request Jun 4, 2024
@youtalk
feat(docker): revert "cache apt archives to the Docker build cache" (a…
0248f77 
…utowarefoundation#4793)

Revert "feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778)"
Signed-off-by: Yutaka Kondo <[email protected]>
youtalk added a commit to youtalk/autoware that referenced this pull request Jun 4, 2024
@youtalk
feat(docker): revert "cache apt archives to the Docker build cache" (a…
770a823 
…utowarefoundation#4793) (#26)

Revert "feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778)"
Signed-off-by: Yutaka Kondo <[email protected]>
youtalk added a commit to youtalk/autoware that referenced this pull request Jun 11, 2024
@youtalk @mitsudome-r
@badai-nguyen @Naophis @xmfcx @awf-autoware-bot @pre-commit-ci @amadeuszsz @oguzkaganozt
feat: upstream to origin (#42)
98a1ab4 
* chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719)

* refactor(ci): reuse `load-env.yaml` to load env (autowarefoundation#4734)

* reuse load-env.yaml to load env

* update base_image

Signed-off-by: Yutaka Kondo <[email protected]>

* fix base_image

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* refactor(docker): introduce `src-imported` stage (autowarefoundation#4712)

* introduce src-imported stage

Signed-off-by: Yutaka Kondo <[email protected]>

* empty

Signed-off-by: Yutaka Kondo <[email protected]>

* generate install package lists

* append || true

Signed-off-by: Yutaka Kondo <[email protected]>

* cat packages

Signed-off-by: Yutaka Kondo <[email protected]>

* chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719)

* wip

Signed-off-by: Yutaka Kondo <[email protected]>

* wip

Signed-off-by: Yutaka Kondo <[email protected]>

* Update Dockerfile

* Update Dockerfile

* install pacmod

* rename

* Revert "install pacmod"

This reverts commit ebff10f.

* install pacmod3

* register pamod3 repos

* ignore hadolint

* revert

* run set-dev-env.sh

* pip install

* copy

* rosdep update

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* feat(docker): install minimum packages in `base` stage (autowarefoundation#4751)

* install ssh only and copy only once

Signed-off-by: Yutaka Kondo <[email protected]>

* install gosu

Signed-off-by: Yutaka Kondo <[email protected]>

* fix dir

Signed-off-by: Yutaka Kondo <[email protected]>

* revert ansible

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* chore: add discussion templates for general category (autowarefoundation#4764)

Signed-off-by: Ryohsuke Mitsudome <[email protected]>

* feat(ci,docker): change `vcs import` to `COPY src` (autowarefoundation#4738)

* introduce src-imported stage

Signed-off-by: Yutaka Kondo <[email protected]>

* empty

Signed-off-by: Yutaka Kondo <[email protected]>

* generate install package lists

* append || true

Signed-off-by: Yutaka Kondo <[email protected]>

* cat packages

Signed-off-by: Yutaka Kondo <[email protected]>

* chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719)

* wip

Signed-off-by: Yutaka Kondo <[email protected]>

* wip

Signed-off-by: Yutaka Kondo <[email protected]>

* Update Dockerfile

* Update Dockerfile

* install pacmod

* rename

* Revert "install pacmod"

This reverts commit ebff10f.

* install pacmod3

* register pamod3 repos

* ignore hadolint

* revert

* run set-dev-env.sh

* pip install

* copy

* rosdep update

* vcs import outside dockerfile

Signed-off-by: Yutaka Kondo <[email protected]>

* refine order and remove /autoware/src at last

* fix hadolint

* remove tar files

Signed-off-by: Yutaka Kondo <[email protected]>

* shell: bash

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* feat(docker,ansible): download artifacts at runtime (autowarefoundation#4771)

* feat(ci): enable Docker build cache to 5 times faster container build (autowarefoundation#4730)

* introduce src-imported stage

Signed-off-by: Yutaka Kondo <[email protected]>

* empty

Signed-off-by: Yutaka Kondo <[email protected]>

* generate install package lists

* append || true

Signed-off-by: Yutaka Kondo <[email protected]>

* cat packages

Signed-off-by: Yutaka Kondo <[email protected]>

* chore(docker): separate `rosdep install` and `colcon build` RUN commands (autowarefoundation#4719)

* wip

Signed-off-by: Yutaka Kondo <[email protected]>

* wip

Signed-off-by: Yutaka Kondo <[email protected]>

* Update Dockerfile

* Update Dockerfile

* install pacmod

* rename

* Revert "install pacmod"

This reverts commit ebff10f.

* install pacmod3

* register pamod3 repos

* ignore hadolint

* revert

* run set-dev-env.sh

* pip install

* copy

* rosdep update

* vcs import outside dockerfile

Signed-off-by: Yutaka Kondo <[email protected]>

* enable docker build cache

Signed-off-by: Yutaka Kondo <[email protected]>

* empty

Signed-off-by: Yutaka Kondo <[email protected]>

* add no-cuda/cuda suffix

Signed-off-by: Yutaka Kondo <[email protected]>

* remove --download-artifacts

Signed-off-by: Yutaka Kondo <[email protected]>

* remove --download-artifacts

Signed-off-by: Yutaka Kondo <[email protected]>

* add --data-path option

Signed-off-by: Yutaka Kondo <[email protected]>

* wip

Signed-off-by: Yutaka Kondo <[email protected]>

* run download_artifacts

Signed-off-by: Yutaka Kondo <[email protected]>

* remove todo

Signed-off-by: Yutaka Kondo <[email protected]>

* revert run

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* feat(docker): cache `ccache` to the Docker build cache (autowarefoundation#4772)

* fix: add download url of yolox with semantic segmentation header (autowarefoundation#4012)

Signed-off-by: badai-nguyen <[email protected]>

* chore: add error handling for held packages during rmw installation (autowarefoundation#4781)

* chore: add error handling for held packages during apt installation

Signed-off-by: Naophis <[email protected]>

* feat: Add check for held ROS packages during installation

Signed-off-by: Naophis <[email protected]>

---------

Signed-off-by: Naophis <[email protected]>

* refactor(docker,ci): rename `autoware-openadk` to `autoware` and `adkit` to `openadkit` (autowarefoundation#4785)

* rename adkit to openadkit

Signed-off-by: Yutaka Kondo <[email protected]>

* rename autoware-openadkit to autoware

Signed-off-by: Yutaka Kondo <[email protected]>

* fix hcl path

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* feat(ci): disable `Upload Artifacts` on `docker-build-and-push` action (autowarefoundation#4789)

* remove upload artifact

Signed-off-by: Yutaka Kondo <[email protected]>

* remove build and push

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778)

* mount apt cache

Signed-off-by: Yutaka Kondo <[email protected]>

* reorder

Signed-off-by: Yutaka Kondo <[email protected]>

* remove upload artifact

Signed-off-by: Yutaka Kondo <[email protected]>

* remove build and push

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* chore: update code owners (autowarefoundation#4797)

Signed-off-by: M. Fatih Cırıt <[email protected]>

* ci(pre-commit-ansible): autoupdate (autowarefoundation#4676)

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <[email protected]>

* ci: pre-commit autoupdate (autowarefoundation#4670)

[pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/igorshubovych/markdownlint-cli: v0.39.0 → v0.41.0](igorshubovych/markdownlint-cli@v0.39.0...v0.41.0)
- [github.com/AleksaC/hadolint-py: v2.12.1-beta → v2.12.1b3](AleksaC/hadolint-py@v2.12.1-beta...v2.12.1b3)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* ci(pre-commit): autoupdate (autowarefoundation#4662)

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <[email protected]>

* ci(pre-commit-optional): autoupdate (autowarefoundation#4766)

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <[email protected]>

* fix(ci): run `vsc import` on `build-main` workflows (autowarefoundation#4799)

Signed-off-by: Yutaka Kondo <[email protected]>

* feat(docker): cache `ccache` to the Docker build cache (autowarefoundation#4800)

cache ccache again

Signed-off-by: Yutaka Kondo <[email protected]>

* chore: hold check of plotjuggler-ros package and display warning if held (autowarefoundation#4805)

* chore: Hold check of plotjuggler-ros package and display warning if held

Signed-off-by: Naophis <[email protected]>

* chore: fix condition for checking held ROS package in dev_tools role

Signed-off-by: Naophis <[email protected]>

* style(pre-commit): autofix

* chore: Update dev_tools role to fix condition for checking held ROS package

Signed-off-by: Naophis <[email protected]>

---------

Signed-off-by: Naophis <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* feat(docker): revert "cache apt archives to the Docker build cache" (autowarefoundation#4793)

Revert "feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778)"
Signed-off-by: Yutaka Kondo <[email protected]>

* chore(ci): add a workflow for DCO check to avoid DCO bot system failure (autowarefoundation#4825)

Signed-off-by: Ryohsuke Mitsudome <[email protected]>

* feat(ci): add CPU architecture suffixes to the Docker build cache (autowarefoundation#4826)

* add arch prefix

Signed-off-by: Yutaka Kondo <[email protected]>

* rename

Signed-off-by: Yutaka Kondo <[email protected]>

* use lib_dir

Signed-off-by: Yutaka Kondo <[email protected]>

* change suffix

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>

* feat(simulator.repos): use autoware_msgs_support branch for scenario_simulator (autowarefoundation#4823)

Signed-off-by: Ryohsuke Mitsudome <[email protected]>
Co-authored-by: Yutaka Kondo <[email protected]>

* feat(ansible): add lidar_transfusion artifacts (autowarefoundation#4835)

* refactor(docker): fix colcon `--cmake-args` (autowarefoundation#4828)

fix colcon cmake-args

Signed-off-by: Yutaka Kondo <[email protected]>

* refactor(ci): use platform arg to replace raw string (autowarefoundation#4844)

use platform arg to replace raw string

Signed-off-by: Yutaka Kondo <[email protected]>

* refactor(ci): install `vcstool` and run `vcs import` on `docker-build-and-push.yaml` (autowarefoundation#4845)

* refactor docker-build-and-push

Signed-off-by: Yutaka Kondo <[email protected]>

* fix shell

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>
Co-authored-by: oguzkaganozt <[email protected]>

* fix platform

Signed-off-by: Yutaka Kondo <[email protected]>

---------

Signed-off-by: Yutaka Kondo <[email protected]>
Signed-off-by: Ryohsuke Mitsudome <[email protected]>
Signed-off-by: badai-nguyen <[email protected]>
Signed-off-by: Naophis <[email protected]>
Signed-off-by: M. Fatih Cırıt <[email protected]>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ryohsuke Mitsudome <[email protected]>
Co-authored-by: badai nguyen <[email protected]>
Co-authored-by: Naophis <[email protected]>
Co-authored-by: M. Fatih Cırıt <[email protected]>
Co-authored-by: awf-autoware-bot[bot] <94889083+awf-autoware-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Amadeusz Szymko <[email protected]>
Co-authored-by: oguzkaganozt <[email protected]>
@youtalk youtalk mentioned this pull request Jul 11, 2024
Merged
4 tasks
pravinkmr26 pushed a commit to pravinkmr26/autoware that referenced this pull request Jul 15, 2024
@youtalk @pravinkmr26
feat(docker): revert "cache apt archives to the Docker build cache" (a…
fb78122 
…utowarefoundation#4793)

Revert "feat(docker): cache apt archives to the Docker build cache (autowarefoundation#4778)"
Signed-off-by: Yutaka Kondo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xmfcx xmfcx xmfcx approved these changes

@mitsudome-r mitsudome-r Awaiting requested review from mitsudome-r

@oguzkaganozt oguzkaganozt Awaiting requested review from oguzkaganozt oguzkaganozt is a code owner

Assignees

@youtalk youtalk

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@youtalk @xmfcx