Enable rust-vmm coverage test in CI

1. Enabled the aarch64 coverage test in `test_coverage` script. 2. Seperated the coverage json sample file to two sample files for different machine achitectures. 3. Updated Buildkite pipeline container version. 4. Updated README for the description of coverage files and container version. Signed-off-by: Henry Wang <[email protected]>
rust-vmm · Feb 21, 2020 · cd7096e · cd7096e
1 parent c309d06
commit cd7096e
Show file tree

Hide file tree

Showing 5 changed files with 48 additions and 23 deletions.
diff --git a/.buildkite/pipeline.yml b/.buildkite/pipeline.yml
@@ -11,7 +11,7 @@ steps:
       os: linux
     plugins:
       - docker#v3.0.1:
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "build-gnu-arm"
@@ -24,7 +24,7 @@ steps:
       os: linux
     plugins:
       - docker#v3.0.1:
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "build-musl-x86"
@@ -37,7 +37,7 @@ steps:
       os: linux
     plugins:
       - docker#v3.0.1:
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "build-musl-arm"
@@ -50,7 +50,7 @@ steps:
       os: linux
     plugins:
       - docker#v3.0.1:
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "style"
@@ -62,7 +62,7 @@ steps:
       os: linux
     plugins:
       - docker#v3.0.1:
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "unittests-gnu-x86"
@@ -76,7 +76,7 @@ steps:
     plugins:
       - docker#v3.0.1:
           privileged: true
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "unittests-gnu-arm"
@@ -90,7 +90,7 @@ steps:
     plugins:
       - docker#v3.0.1:
           privileged: true
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "unittests-musl-x86"
@@ -104,7 +104,7 @@ steps:
     plugins:
       - docker#v3.0.1:
           privileged: true
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "unittests-musl-arm"
@@ -118,7 +118,7 @@ steps:
     plugins:
       - docker#v3.0.1:
           privileged: true
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "clippy-x86"
@@ -131,7 +131,7 @@ steps:
       os: linux
     plugins:
       - docker#v3.0.1:
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "check-warnings-x86"
@@ -145,7 +145,7 @@ steps:
     plugins:
       - docker#v3.0.1:
           privileged: true
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "check-warnings-arm"
@@ -159,7 +159,7 @@ steps:
     plugins:
       - docker#v3.0.1:
           privileged: true
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
           always-pull: true
 
   - label: "coverage-x86"
@@ -173,5 +173,19 @@ steps:
     plugins:
       - docker#v3.0.1:
           privileged: true
-          image: "rustvmm/dev:v4"
+          image: "rustvmm/dev:v5"
+          always-pull: true
+
+  - label: "coverage-arm"
+    commands:
+      - pytest rust-vmm-ci/integration_tests/test_coverage.py
+    retry:
+      automatic: false
+    agents:
+      platform: arm.metal
+      os: linux
+    plugins:
+      - docker#v3.0.1:
+          privileged: true
+          image: "rustvmm/dev:v5"
           always-pull: true
diff --git a/README.md b/README.md
@@ -23,9 +23,13 @@ git submodule add https://github.com/rust-vmm/rust-vmm-ci.git
 git commit -s -m "Added rust-vmm-ci as submodule"
 ```
 
-2. Create the coverage test configuration file named coverage_config.json in
-the root of the repository. One example of the configuration file can be found
-in [coverage_config.json.sample](coverage_config.json.sample).
+2. Create the coverage test configuration file named coverage_config_ARCH.json in
+the root of the repository, where ARCH is the architecture of the machine.
+There are two coverage test configuration files, one per each platform.
+The example of the configuration file for x86_64 architecture can be found
+in [coverage_config_x86_64.json.sample](coverage_config_x86_64.json.sample),
+and the example of the configuration file for aarch64 architecture can be found
+in [coverage_config_aarch64.json.sample](coverage_config_aarch64.json.sample).
 
 The json must have the following fields:
 - *coverage_score*: The coverage of the repository.
@@ -73,7 +77,7 @@ steps:
       platform: x86_64.metal
     plugins:
       - docker#v3.0.1:
-          image: "rustvmm/dev:v3"
+          image: "rustvmm/dev:v5"
           always-pull: true
 ```
 
@@ -105,7 +109,7 @@ steps:
       platform: x86_64.metal
     plugins:
       - docker#v3.0.1:
-          image: "rustvmm/dev:v3"
+          image: "rustvmm/dev:v5"
 always-pull: true
 ```
 
@@ -188,9 +192,9 @@ docker run --device=/dev/kvm \
            -it \
            --security-opt seccomp=unconfined \
            --volume $(pwd)/kvm-ioctls:/kvm-ioctls \
-           rustvmm/dev:v3
+           rustvmm/dev:v5
 cd kvm-ioctls/
-pytest --profile=devel tests/test_coverage.py
+pytest --profile=devel rust-vmm-ci/integration_tests/test_coverage.py
 ```
 
 If the PR coverage is higher than the upstream coverage, the coverage file

diff --git a/coverage_config.json.sample → coverage_config_aarch64.json.sample b/coverage_config.json.sample → coverage_config_aarch64.json.sample
diff --git a/coverage_config_x86_64.json.sample b/coverage_config_x86_64.json.sample
@@ -0,0 +1,5 @@
+{
+  "coverage_score": 90,
+  "exclude_path": "my_autogenerated_file.rs,second_file.rs",
+  "crate_features": "my_dummy_feature,dummy_feature2"
+}
diff --git a/integration_tests/test_coverage.py b/integration_tests/test_coverage.py
@@ -2,14 +2,16 @@
 # SPDX-License-Identifier: Apache-2.0
 """Test the coverage and update the threshold when coverage is increased."""
 
-import json, os, re, shutil, subprocess
+import json, os, re, shutil, subprocess, platform
 import pytest
 
 from utils import get_repo_root_path
 
 REPO_ROOT_PATH = get_repo_root_path()
-COVERAGE_CONFIG_PATH = os.path.join(REPO_ROOT_PATH, "coverage_config.json")
-
+if platform.machine() == "x86_64":
+    COVERAGE_CONFIG_PATH = os.path.join(REPO_ROOT_PATH, "coverage_config_x86_64.json")
+elif platform.machine() == "aarch64":
+    COVERAGE_CONFIG_PATH = os.path.join(REPO_ROOT_PATH, "coverage_config_aarch64.json")
 
 def _read_test_config():
     """