Description

Hi Everyone,

In our setup, we have VLAN interfaces that are only available/configured on the Kubernetes Master Nodes.

The following NetworkAttachmentDefinition has been created:

---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: vlan201-external-subnet
  namespace: kube-system
spec:
  config: '{
      "cniVersion": "0.3.1",
      "type": "macvlan",
      "master": "bond0.201",
      "mode": "bridge",
      "ipam": {
        "type": "kube-ovn",
        "server_socket": "/run/openvswitch/kube-ovn-daemon.sock",
        "provider": "vlan201-external-subnet.kube-system"
      }
    }'

The VpcNatGateway is deployed with the following selector, which forces it to run on kube-ovn masters:

---
kind: VpcNatGateway
apiVersion: kubeovn.io/v1
metadata:
  name: vlan201-nat-gw
spec:
  vpc: vlan201-vpc
  subnet: vlan201-internal-subnet
  lanIp: 100.105.0.100
  selector:
    - "kubernetes.io/os: linux"
    - "kube-ovn/role: master"
  externalSubnets:
    - vlan201-external-subnet

The issue comes when a LoadBalancer Service gets created and wants to request an IP in the vlan201-external-subnet Subnet. If the lb-svc-* Pod gets scheduled on a non-kube-ovn master node, it fails to get an IP. Manually editing the Deployment and specifying a nodeSelector fixes this issue:

      nodeSelector:
        kube-ovn/role: master
        kubernetes.io/os: linux

Is there already a way to specify this nodeSelector? Looking at the source code, it does not seem to be the case:

• kube-ovn/pkg/controller/service_lb.go

  Lines 104 to 139 in 06ac37d

  	dp = &v1.Deployment{
  	ObjectMeta: metav1.ObjectMeta{
  	Name: name,
  	},
  	Spec: v1.DeploymentSpec{
  	Replicas: ptr.To(int32(1)),
  	Selector: &metav1.LabelSelector{
  	MatchLabels: labels,
  	},
  	Template: corev1.PodTemplateSpec{
  	ObjectMeta: metav1.ObjectMeta{
  	Labels: labels,
  	Annotations: podAnnotations,
  	},
  	Spec: corev1.PodSpec{
  	Containers: []corev1.Container{
  	{
  	Name: "lb-svc",
  	Image: vpcNatImage,
  	Command: []string{"sleep", "infinity"},
  	ImagePullPolicy: corev1.PullIfNotPresent,
  	SecurityContext: &corev1.SecurityContext{
  	Privileged: ptr.To(true),
  	AllowPrivilegeEscalation: ptr.To(true),
  	},
  	Resources: resources,
  	},
  	},
  	TerminationGracePeriodSeconds: ptr.To(int64(0)),
  	},
  	},
  	Strategy: v1.DeploymentStrategy{
  	Type: v1.RecreateDeploymentStrategyType,
  	},
  	},
  	}

Thanks in advance for your support!

Who will benefit from this feature?

No response

Anything else?

No response