[Question] How the crc vm on Windows host connecting external network?

[elon-s]

Hi. I'm using crc on Windows and it works well. I'm just curious about how the crc vm connecting to external network. I can see all the outgoing traffic are sent out by tap0 interface in vm, but how tap0 interface interacting with Windows host? I didn't find any network adapter created on Windows. Can anybody give a detailed explanation?

[cfergeau]

There's a gvforwarder process running in the VM which handles the network traffic from tap0 and sends it to the host over vsock/hvsock. On the host, this network data is handled by the crc daemon which is started as a service. There is no network adapter in Windows, the network traffic is sent/received by this daemon and a userland network stack.
See https://github.com/containers/gvisor-tap-vsock

[elon-s]

@cfergeau Thanks for your answer, but I still have few questions unclear.

1. After your explanation, I think the route on Windows is:
   Pod eth0<->veth<->br0<->tun0<->tap0<-vsock->crc daemon on host<->userland network stack<->host adapter<->external
   Is my understanding correct?
2. For linux, I found there is a crc interface(192.168.130.1) on the host, and all the vm traffic forward to this interface. Is this forwarding using unix socket as described in the gvisor-tap-vsock document? In the qemu options, I found this:
   -chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-1-crc/fs0-fs.sock -device {"driver":"vhost-user-fs-pci","id":"fs0","chardev":"chr-vu-fs0","tag":"dir0","bus":"pci.1","addr":"0x0"} -netdev {"type":"tap","fd":"27","vhost":true,"vhostfd":"28","id":"hostnet0"} -device {"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"**52:fd:fc:07:21:82**","bus":"pci.2","addr":"0x0"}
   52:fd:fc:07:21:82 is the vm interface ens2p0's mac addr. so the route on Linux maybe like:
   Pod eth0<->veth<->br0<->tun0<->ens2p0<-unix socket->crc interface on host<->crc daemon<->userland network stack<->host adapter<->external
   Is that right?
3. Also the gvforwarder process, I did't find it in the VM, but this one
   /usr/bin/conmon --api-version 1 ... -n gvisor-tap-vsock .... Is this the gvforwarder process?

[cfergeau]

1. I'm not familiar with networking from inside a container, so I cannot confirm the beginning (Pod eth0<->veth<->br0<->), but the overall flow looks correct ("vsock" is used as AF_VSOCK in the linux guest, and as hyperv sockets in the host https://stackoverflow.com/questions/60696166/using-the-hyper-v-sockets-between-windows-host-and-linux-guest )

2. The 192.168.130.1 interface is something managed by libvirt, in this scenario, gvisor-tap-vsock is not used. On linux, you need crc config set network-mode user to use gvisor-tap-vsock. On linux when using user-mode networking, the network traffic flow will be similar to 1. When there is a 192.168.130.1 interface, the VM has a paravirtualized virtio-net device and the VM has a "real" IP on the host, so network traffic goes directly from VM/hypervisor to 192.168.130.1 then to the host adapter.

3. It's indeed running in a container, I forgot about this, the container name is gvisor-tap-vsock, the container is https://github.com/containers/gvisor-tap-vsock/blob/main/images/Dockerfile

[elon-s]

One more question, what if I want to send data initiatively from external network to pod in crc cluster?
For Linux, it seems there are 2 SNAT iptable rules applied when pod connecting external network, is it possible to add some DNAT rules to forward data sending to host ip to destination as pod ip?
Fow Windows, is it possible to send data to the crc daemon from other host and then crc daemon forward data into cluster?