Skip to content

Commit

Permalink
CI: Split the Build Jobs for Arm64 and x86_64
Browse files Browse the repository at this point in the history
This PR creates the new CI Build Jobs `arm64-01` and `x86_64-01`. The new jobs will split and offload the Arm64 and x86_64 Build Targets from `other`. This will reduce our usage of GitHub Runners, to comply with the ASF Policy for GitHub Actions. (Recently we see more PRs for Arm64 and x86_64)

Before the Split: Simple PRs (One Arch and/or One Board) for Arm64 and x86_64 require almost 1 hour for CI Build
- `other` (57 mins): AVR, SPARC, x86, PinePhone, QEMU Arm64, QEMU x86_64

After the Split: Simple PRs for Arm64 and x86_64 will complete under 30 mins
- `other` (24 mins): AVR, SPARC, x86
- `arm64-01` (29 mins): PinePhone, QEMU Arm64
- `x86_64-01` (9 mins): QEMU x86_64

To skip more unnecessary builds: Our Build Rules `arch.yml` shall ignore the label "Area: Documentation", so that a Simple PR + Docs is still a Simple PR. Previously we experienced longer CI Build Times, just because we added docs to our Simple PR. (Now our PR shall be built exactly like a Simple PR)

The updated CI code is explained here: apache#13775
  • Loading branch information
lupyuen committed Oct 15, 2024
1 parent 39690a9 commit aa7a394
Show file tree
Hide file tree
Showing 5 changed files with 32 additions and 28 deletions.
28 changes: 17 additions & 11 deletions .github/workflows/arch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,16 @@ jobs:
exit
fi
# Ignore the Label "Area: Documentation", because it won't affect the Build Targets
query='.labels | map(select(.name != "Area: Documentation")) | '
select_name='.[].name'
select_length='length'
# Get the Labels for the PR: "Arch: risc-v \n Board: risc-v \n Size: XS"
# If GitHub CLI Fails: Build all targets
labels=$(gh pr view $pr --repo $GITHUB_REPOSITORY --json labels --jq '.labels[] | .name' || echo "")
numlabels=$(gh pr view $pr --repo $GITHUB_REPOSITORY --json labels --jq '.[] | length' || echo "")
labels=$(gh pr view $pr --repo $GITHUB_REPOSITORY --json labels --jq "$query$select_name" || echo "")
numlabels=$(gh pr view $pr --repo $GITHUB_REPOSITORY --json labels --jq "$query$select_length" || echo "")
echo "labels=$labels"
echo "numlabels=$numlabels" | tee -a $GITHUB_OUTPUT
# Identify the Size, Arch and Board Labels
Expand Down Expand Up @@ -166,37 +172,37 @@ jobs:
# For "Arch / Board: arm": Build arm-01, arm-02, ...
if [[ "$arch_contains_arm" == "1" || "$board_contains_arm" == "1" ]]; then
if [[ "$board" != *"arm"* ]]; then
if [[ "$board" != *"arm-"* ]]; then
skip_build=1
fi
# For "Arch / Board: arm64": Build other
# For "Arch / Board: arm64": Build arm64-01
elif [[ "$arch_contains_arm64" == "1" || "$board_contains_arm64" == "1" ]]; then
if [[ "$board" != *"other"* ]]; then
if [[ "$board" != *"arm64-"* ]]; then
skip_build=1
fi
# For "Arch / Board: risc-v": Build risc-v-01, risc-v-02
# For "Arch / Board: risc-v": Build risc-v-01, risc-v-02, ...
elif [[ "$arch_contains_riscv" == "1" || "$board_contains_riscv" == "1" ]]; then
if [[ "$board" != *"risc-v"* ]]; then
if [[ "$board" != *"risc-v-"* ]]; then
skip_build=1
fi
# For "Arch / Board: simulator": Build sim-01, sim-02
elif [[ "$arch_contains_sim" == "1" || "$board_contains_sim" == "1" ]]; then
if [[ "$board" != *"sim"* ]]; then
if [[ "$board" != *"sim-"* ]]; then
skip_build=1
fi

# For "Arch / Board: x86_64": Build other
# For "Arch / Board: x86_64": Build x86_64-01
elif [[ "$arch_contains_x86_64" == "1" || "$board_contains_x86_64" == "1" ]]; then
if [[ "$board" != *"other"* ]]; then
if [[ "$board" != *"x86_64-"* ]]; then
skip_build=1
fi

# For "Arch / Board: xtensa": Build xtensa-01, xtensa-02
elif [[ "$arch_contains_xtensa" == "1" || "$board_contains_xtensa" == "1" ]]; then
if [[ "$board" != *"xtensa"* ]]; then
if [[ "$board" != *"xtensa-"* ]]; then
skip_build=1
fi

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ jobs:
os: Linux
boards: |
[
"arm-01", "other", "risc-v-01", "sim-01", "xtensa-01",
"arm-01", "risc-v-01", "sim-01", "xtensa-01", "arm64-01", "x86_64-01", "other",
"arm-02", "risc-v-02", "sim-02", "xtensa-02",
"arm-03", "risc-v-03",
"arm-04", "risc-v-04",
Expand Down
11 changes: 11 additions & 0 deletions tools/ci/testlist/arm64-01.dat
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
/arm64

# arm64 Boards build by CMake
CMake,qemu-armv8a:citest
CMake,qemu-armv8a:citest_smp
CMake,qemu-armv8a:nsh
CMake,qemu-armv8a:nsh_fiq
CMake,qemu-armv8a:nsh_gicv2
CMake,qemu-armv8a:nsh_smp
CMake,qemu-armv8a:nsh_smp_tickless
CMake,qemu-armv8a:sotest
16 changes: 0 additions & 16 deletions tools/ci/testlist/other.dat
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,6 @@

/x86

# x86_64-elf-gcc from homebrew doesn't seem to
# provide __udivdi3 etc for -m32
/x86_64

# Sparc-gaisler-elf toolchain doesn't provide macOS binaries
/sparc
-xx3823:nsh

/arm64

# arm64 Boards build by CMake
CMake,qemu-armv8a:citest
CMake,qemu-armv8a:citest_smp
CMake,qemu-armv8a:nsh
CMake,qemu-armv8a:nsh_fiq
CMake,qemu-armv8a:nsh_gicv2
CMake,qemu-armv8a:nsh_smp
CMake,qemu-armv8a:nsh_smp_tickless
CMake,qemu-armv8a:sotest
3 changes: 3 additions & 0 deletions tools/ci/testlist/x86_64-01.dat
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# x86_64-elf-gcc from homebrew doesn't seem to
# provide __udivdi3 etc for -m32
/x86_64

0 comments on commit aa7a394

Please sign in to comment.