Potential best practice for dealing with NAT in the BGP with enterprise VPN use case

[rannyh]

In the discussion around the PR that defines the use case (https://github.com/cncf/cnf-wg/pull/60), the issue of dealing with NAT by a BGP application was raised.

The challenge is for the application to advertise its externally accessible IP address to BGP peers, instead of the local, private address. An approach taken by other protocols (e.g., SIP the Session Initiation Protocol) is to replace the self address in the protocol payload with a publicly routable IP address that was assigned to the application by an external load balancer.

So, for example, in the BGP startup flow shown here:
https://www.eventhelix.com/RealtimeMantra/Networking/bgp_startup.pdf
the application would place its externally assigned IP address in the "BGP Identifier" header, instead of its local IP address. Finding the external address may be done in one of several ways:
1/ STUN (Session Traversal Utilities for NAT - https://en.wikipedia.org/wiki/STUN): The application would query a STUN server before establishing BGP connections with peers.
2/ K8S API - The application will query the K8S orchestrator about the assigned IP address for the service it is running (similar to "kubectl describe services xyzzy"
3/ Any other ideas?

The first method requires an external entity (STUN server). The second assumes the application has admin privileges to the cluster its running on.

[electrocucaracha]

STUN can be an interesting case, I'm not sure how it behaves given that some cases there is a double-overlay on the Kubernetes Network.

[taylor]

See BGP EVPN Route-Reflector blog post https://pantheon.tech/lightyio-bgp-evpn-rr/

[taylor]

https://cilium.io/blog/2021/05/20/cilium-110

Cilium 1.10: WireGuard, BGP Support, Egress IP Gateway, New Cilium CLI, XDP Load Balancer, Alibaba Cloud Integration and more

[jeffsaelens]

Do we expand this discussion beyond just NAT, and look at BGP in this space at a high level?

• Control plane considerations? K8s vs the network?
• SmartNIC, CNI, NSM style approaches?
• What does a routed interface look like in a POD?
• How do we get the interface, the service and the connectivity that we need to enable these use case?
• Stick to a purely control plane discussion for the interim

CNI based approaches? Peering with the underlay and directly advertising POD space? How do you handle the scale and convergence issues?

• Best practices around how to manage aggregates, what to advertise and where?
• Routed services with specific interfaces?
• Set route targets and other BGP metadata via the CNI/K8s API?

Offload based approaches?