[ci] Fix flexible-ipam e2e config #6324

gran-vmv · 2024-05-13T08:30:13Z

Use k8s version 1.30.1-1.1.
Supress clean up error when initialize e2e.
Enhance redeploy k8s determination logic.
Unmask kubelet service after installation.

tnqn

Could you describe what's being fixed here?
I saw 4 unrelated changes in the commit but it's unclear to me what bugs are behind them.

tnqn · 2024-05-15T13:45:40Z

ci/jenkins/test.sh

-    clean_ns "monitoring"
-    clean_ns "antrea-ipam-test"
-    clean_ns "antrea-test"
+    clean_ns "monitoring" || true


clean_ns already ignores some delete errors, why it needs to futher ignore errors here?

In flexible-ipam testbed, the k8s may not be deployed at this moment, thus we need to ignore the error.

tnqn · 2024-05-15T13:47:52Z

ci/jenkins/test.sh

+    if [[ -n $POD_CIDRS ]]; then
+        for POD_CIDR in "${POD_CIDRS[@]}"; do
+            if [[ $POD_CIDR =~ .*:.* ]]
+            then
+                (( HAS_IPV6++ ))
+            else
+                (( HAS_IPV4++ ))
+            fi
+        done
+        if [[ ${IP_MODE} == "ipv4" ]]; then
+            (( HAS_IPV4-- ))
+        elif [[ ${IP_MODE} == "ipv6" ]]; then
+            (( HAS_IPV6-- ))
        else
-            (( HAS_IPV4++ ))
+            (( HAS_IPV4-- ))
+            (( HAS_IPV6-- ))
+        fi
+        if [ ${HAS_IPV4} -eq ${INITIAL_VALUE} ] && [ ${HAS_IPV6} -eq ${INITIAL_VALUE} ]; then
+          return 0


The change is fine but I don't see what's wrong with the previous code too.

We added a check if [[ -n $POD_CIDRS ]]; then for this code, to skip this check when kubectl cannot get the POD_CIDRS.
Previously the redeployment will be skipped here if k8s is not deployed and POD_CIDRS is empty.

tnqn · 2024-05-15T13:48:33Z

ci/jenkins/test.sh

@@ -896,7 +898,7 @@ function redeploy_k8s_if_ip_mode_changes() {
        else
            NODE_IP_STRING="${IPV4S[i]},${IPV6S[i]}"
        fi
-        ssh -o StrictHostKeyChecking=no -i "${WORKDIR}/id_rsa" -n ubuntu@${IPV4S[i]} "echo \"KUBELET_EXTRA_ARGS=--node-ip=${NODE_IP_STRING}\" | sudo tee /etc/default/kubelet; sudo systemctl restart kubelet"
+        ssh -o StrictHostKeyChecking=no -i "${WORKDIR}/id_rsa" -n ubuntu@${IPV4S[i]} "echo \"KUBELET_EXTRA_ARGS=--node-ip=${NODE_IP_STRING}\" | sudo tee /etc/default/kubelet; sudo systemctl unmask kubelet; sudo systemctl restart kubelet"


I don't find any code masking the service, why it needs to be unmasked?

After we installed kubelet 1.28.9, the service is in masked state, we need to unmask it first.

Is this service always masked in newer K8s?

The service may be masked for the K8s package in the new repo.
We changed the apt repo in testbed VM since old K8s repo is unavailable:
https://apt.kubernetes.io/
->
https://pkgs.k8s.io/core:/stable:/v1.30/deb/

Can we have the K8s apt repository update in this script? If test.sh will also re-install K8s in IPAM testbed, I feel it's unnecessary to require users to update the https://pkgs.k8s.io/core:/stable:/v1.30/deb to a newer one manually on testbeds in the future.

ci/jenkins/test.sh

luolanzone · 2024-05-22T08:25:20Z

ci/jenkins/test.sh

@@ -896,7 +898,7 @@ function redeploy_k8s_if_ip_mode_changes() {
        else
            NODE_IP_STRING="${IPV4S[i]},${IPV6S[i]}"
        fi
-        ssh -o StrictHostKeyChecking=no -i "${WORKDIR}/id_rsa" -n ubuntu@${IPV4S[i]} "echo \"KUBELET_EXTRA_ARGS=--node-ip=${NODE_IP_STRING}\" | sudo tee /etc/default/kubelet; sudo systemctl restart kubelet"
+        ssh -o StrictHostKeyChecking=no -i "${WORKDIR}/id_rsa" -n ubuntu@${IPV4S[i]} "echo \"KUBELET_EXTRA_ARGS=--node-ip=${NODE_IP_STRING}\" | sudo tee /etc/default/kubelet; sudo systemctl unmask kubelet; sudo systemctl restart kubelet"


Is this service always masked in newer K8s?

ci/jenkins/test.sh

gran-vmv · 2024-05-30T03:05:48Z

/test-all
/test-flexible-ipam-e2e

luolanzone · 2024-06-17T09:24:44Z

@gran-vmv the new triggered test is still failed, but I think it should be caused by another PR #6010, which should be fixed by this one #6452, you may rerun after the PR is merged.

luolanzone · 2024-06-17T14:56:31Z

@gran-vmv is this fix related with following failures?

--- FAIL: TestAntreaIPAM/testAntreaIPAMPodConnectivityDifferentNodes (691.76s)
--- FAIL: TestAntreaIPAM/testAntreaIPAMOVSRestartSameNode (56.44s)
--- FAIL: TestConnectivity/testPingLargeMTU (21.18s)

I saw these failures in both v1.15.2 and 2.0.1 patch releases e2e jobs, not sure they are flaky tests or caused by testbed upgrades.

Use k8s version 1.30.1-1.1. Supress clean up error when initialize e2e. Enhance redeploy k8s determination logic. Signed-off-by: gran <[email protected]>

gran-vmv · 2024-06-18T02:45:27Z

/test-flexible-ipam-e2e

github-actions · 2024-09-17T00:04:31Z

This PR is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days

gran-vmv force-pushed the ipam-e2e-k8s branch from 46d35f8 to c016896 Compare May 13, 2024 08:41

gran-vmv requested review from tnqn, luolanzone and wenyingd May 14, 2024 02:43

gran-vmv self-assigned this May 14, 2024

gran-vmv added area/test/e2e Issues or PRs related to Antrea specific end-to-end testing. area/test/infra Issues or PRs related to test infrastructure (Jenkins configuration, Ansible playbook, Kind wrappers labels May 14, 2024

tnqn reviewed May 15, 2024

View reviewed changes

gran-vmv requested a review from tnqn May 16, 2024 05:40

XinShuYang reviewed May 17, 2024

View reviewed changes

ci/jenkins/test.sh Outdated Show resolved Hide resolved

gran-vmv force-pushed the ipam-e2e-k8s branch 9 times, most recently from bf8726c to c277a71 Compare May 22, 2024 08:02

luolanzone reviewed May 22, 2024

View reviewed changes

gran-vmv force-pushed the ipam-e2e-k8s branch 2 times, most recently from e40a45b to f531d23 Compare May 23, 2024 05:42

gran-vmv requested review from luolanzone and XinShuYang May 23, 2024 05:45

gran-vmv added the action/backport Indicates a PR that requires backports. label May 23, 2024

gran-vmv force-pushed the ipam-e2e-k8s branch 3 times, most recently from f89026e to 904ad2e Compare May 30, 2024 03:05

gran-vmv force-pushed the ipam-e2e-k8s branch from 904ad2e to cf4f1ea Compare June 6, 2024 03:42

gran-vmv force-pushed the ipam-e2e-k8s branch 2 times, most recently from 594eb1c to cf81121 Compare June 6, 2024 03:45

[ci] Fix flexible-ipam e2e config

423a94b

Use k8s version 1.30.1-1.1. Supress clean up error when initialize e2e. Enhance redeploy k8s determination logic. Signed-off-by: gran <[email protected]>

gran-vmv force-pushed the ipam-e2e-k8s branch from cf81121 to 423a94b Compare June 18, 2024 02:41

github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] Fix flexible-ipam e2e config #6324

[ci] Fix flexible-ipam e2e config #6324

gran-vmv commented May 13, 2024 •

edited

Loading

tnqn left a comment

tnqn May 15, 2024

gran-vmv May 16, 2024

tnqn May 15, 2024

gran-vmv May 16, 2024

tnqn May 15, 2024

gran-vmv May 16, 2024

luolanzone May 22, 2024

gran-vmv May 23, 2024

luolanzone Jun 3, 2024

luolanzone May 22, 2024

gran-vmv commented May 30, 2024

luolanzone commented Jun 17, 2024 •

edited

Loading

luolanzone commented Jun 17, 2024

gran-vmv commented Jun 18, 2024

github-actions bot commented Sep 17, 2024

[ci] Fix flexible-ipam e2e config #6324

Are you sure you want to change the base?

[ci] Fix flexible-ipam e2e config #6324

Conversation

gran-vmv commented May 13, 2024 • edited Loading

tnqn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gran-vmv commented May 30, 2024

luolanzone commented Jun 17, 2024 • edited Loading

luolanzone commented Jun 17, 2024

gran-vmv commented Jun 18, 2024

github-actions bot commented Sep 17, 2024

gran-vmv commented May 13, 2024 •

edited

Loading

luolanzone commented Jun 17, 2024 •

edited

Loading