How To: setup a ptp wireguard tunnel with vopono at either end?

[goodboy]

Absolutely amazing project btw; finally something to help with
automating the magic of the linux networking stack with wireguard
💥

I'm starting this issue as both source of discussion around
learning more about how vopono has been designed as well
hopefully for content that might drive more advanced usage and
features. I will be incrementally updating it as I learn more,
start trying to grok the code base, and hopefully get some (good)
feedback 😄.

---

The "Ask"

I realize maybe this isn't (yet) a core use case, but for many
purposes being able to share certain (sets) of unsecured protocols
over a wg tunnel would be super handy in building and deploying
many classes of highly distributed sw applications 😎

I've tried with little success to get a basic PTP setup that is
normally quite easy to configure with std wg / wg-quick as per
this lovely post:
https://www.procustodibus.com/blog/2020/11/wireguard-point-to-point-config/

and the wg-quick guide:
https://www.wireguard.com/quickstart/

---

Premises

• i want to deploy (possibly at the thread spawning level) vopono
  net ns + wireguard automation around processes that will
  communicate with each other over wg tunnels using a (potential
  ecosystem of) protocol(s).
• this will obviously require custom wireguard configs,
  (presumably) different then the default firewall rules, possibly
  additional config options to vopono itself?
• this should all work just as well on a LAN as it does in a real
  world PTP architecture (as per the post above)

---

Technical questions

To start this is going to be a few questions that hopefully can be
explained as just a lack of my own knowledge; i'm pretty rusty with
the advanced networking features in linux, though I used to know
some of this stuff..

• why for the veth tunnel is the default route always via-ed to
  10.200.1.1?

  • to me this addr should always be set to the tunnel interface
    addr but maybe i don't grok how that bridge is supposed to
    work (i still need to refresh on veth) .
  • some resource links for other interested users on veth
    tunnels:
    • https://developers.redhat.com/blog/2018/10/22/introduction-to-linux-interfaces-for-virtual-networking#veth
  • why not use macvlans for the namespace briding instead?
    • https://developers.redhat.com/blog/2018/10/22/introduction-to-linux-interfaces-for-virtual-networking#macvlan

    Before MACVLAN, if you wanted to connect to physical
    network from a VM or namespace, you would have needed to
    create TAP/VETH devices and attach one side to a bridge and
    attach a physical interface to the bridge on the host at the
    same time, as shown below.

• inside the net ns the same private addr subnet space is always
  used, and the same incremental addresses allocated?

  • this doesn't bode well for multi-peer setups (addr collisions)
    and I'm wondering would be it be sane to think about a way in
    which vopono could detect a (custom) multi-peer config and
    then automatically be allowed (or configured) to partition the
    subnet space in some way?

---

Basic deployment how to?

• i've tried within a LAN deploying a PTP style setup with
  2 (metal) hosts and have had zero luck being able to http GET
  a little webserver running in one of the namespaces.

  • The post above shows doing this with python -m http.server to
    verify the PTP endpoints are working.
  • I can't for the life of me figure out why it's not when using
    vopono, even when using -o 80 to forward the http port over
    the tunnel.. I've caved to just finally believing that my
    firewall knowledge is just so terrible I must be missing
    something obvious..

• sure would seem handy to have a guide on how to setup pinging
  from one namespace to another much like the wg quickstart:
  https://www.wireguard.com/quickstart/

---

More docs and details on custom-config and advanced usage

The vopono -v exec debug logging is amazing for figuring out what
automation is actually going on behind the scenes :thumbs_up: but
it seems there's very little docs on custom config setups in
general and definitely very little explanation for the default firewall
settings, veth bridging around the net ns, configuration of the
wireguard interfaces.

I'd for sure be willing to at least do some in depth writeups on
what all these commands mean, are doing, and possibly if more
features get added for "advanced custom config" setups, extend the
user guide (with more sections) to make grokin advanced usage and
config easier.

I think this is really important for both advanced users as well
any potential hacker/code-contribs.

[jamesmcm]

Hey,

Thanks, there's a lot of good questions here.

PtP Wireguard

So for the PtP Wireguard use-case you'd want to be able to run something like:

# On client end
$ vopono exec --provider ptp --protocol wireguard --custom ptpclient.conf firefox
# On server end
$ vopono exec --provider ptp --protocol wireguard --custom ptpclient.conf webserver

And have them able to communicate only inside the network namespace e.g. in this case the client could see the webserver, but only inside the netns.

So you just need point-to-point connection and don't need those outgoing connections to also be over a VPN, etc.?

I think this should be doable, one thing I want to add before the next release is an empty / None provider and protocol to just create a blank namespace with basic connectivity for testing (with --keep-alive). It'd also help with testing stuff like the Cloudflare Warp issues - #235

I'll try this out when I get some time (it'd be really helpful for me as my ISP disallows port forwarding) but I do want to fix the Cloudflare Warp issues first.

LAN access

I can't for the life of me figure out why it's not when using
vopono, even when using -o 80 to forward the http port over
the tunnel.. I've caved to just finally believing that my
firewall knowledge is just so terrible I must be missing
something obvious..

What were you using as the vopono command? As any of the VPN configs will restrict traffic to the VPN tunnel which doesn't make sense for a LAN (i.e. you'll connect to Mullvad, and won't be able to connect back from there to your LAN (assuming no public access, etc.)). Note you can use -f to let the host proxy to the port, but that shouldn't be necessary here anyway.

Also note that simulating a LAN with network namespaces is not the same, as one of the main parts here is that we're dealing with traffic sent over the chosen adapter (i.e. your ethernet adapter - to the LAN / Internet) - specifically with the masquerade rule. That said, it should be doable to get working with different firewall rules (see the veth diagram on the Red Hat article you linked for example), but it is a very different problem from the normal LAN since you aren't using the same network interface, etc.

Thread-level access

I actually wanted to provide a library to do this - see issue #49 . Separating vopono into the part creating the network namespace and basic connectivity, and the VPN provider config part. But the main issue I hit was dealing with RTNETLINK for setting Nftables rules. Also OpenVPN provides no library until OpenVPN3 (which also breaks other parts of the config we currently use 😹 - see issue #47 ) - but you can see the messy prototype code in libvopono - https://github.com/jamesmcm/libvopono/blob/master/src/lib.rs

Another issue was working out how the API should be, you can't really do anything async since you need to guarantee that the right threads are in the right namespaces, which is a bit of a pain (it's more like C at this point, and a lot of network stuff is async). FWIW in that code the idea is you spawn a network namespace, and can then run closures in it by forking the process.

The permissions are also a bit more complicated and it might really force switching to a daemon (running as root) and client (local user) model like other apps (e.g. Cloudflare Warp) - but this is a huge change and really affects how easy it is to just download and run for most users. There is a difficult trade-off in not worsening the user experience for most users i.e. just running a browser in Mullvad, etc. and also spending time on these features vs. things which more of a direct effect for more users like port forwarding in ProtonVPN #226 or even a GUI.

One thing to look at for the NfTables stuff is the Mullvad Rust code for their desktop client - https://github.com/mullvad/mullvadvpn-app/blob/97c9771bde9a01b327cc040eee60b35f169cc84e/talpid-routing/src/unix/linux.rs#L267-L283

But for an existing network namespace, you should just be able to do - https://github.com/jamesmcm/libvopono/blob/master/src/lib.rs#L95-L113 to move a thread / process to it.

Technical Questions

why for the veth tunnel is the default route always via-ed to
10.200.1.1?

It's arbitrary (but needs to be a local address). Also for the host to netns addresses they need to be unique, so this was just an easy way to mostly ensure that (up to 255 addresses) and also easily pair and convert from one to the other.

why not use macvlans for the namespace briding instead?

I'd never heard of it, I don't think it existed when I first pieced together the bash scripts. But this is a really good idea to try out, it'd simplify some of the steps at least (at a cost of maybe dropping support for earlier Linux versions, unless it were an option like iptables vs. nftables). Here's an example I found btw - https://unix.stackexchange.com/questions/742684/how-to-use-network-namespaces-for-vpn-split-tunneling

There's a lot of things to fix like this, like IPv6 endpoint support too (it breaks the masquerade address translation 😿 ) - #181 , and fixing the use of network interface altnames #230

this doesn't bode well for multi-peer setups (addr collisions)
and I'm wondering would be it be sane to think about a way in
which vopono could detect a (custom) multi-peer config

These IP addresses are for the network namespaces though. What do you mean by a multi-peer config in this case? Like a VLAN over Wireguard?

As mentioned above, doing this between local network namespaces will be difficult due to needing to allow the network namespaces to communicate to one another (instead of just out over the host's network interface). If you really want a local "network" of network namespaces, I think it'd be easier to manage with Docker Compose.

Documentation

So I originally collected some bash scripts together and got it to work with that and some Red Hat articles. Unfortunately I can't find them now, but this was very close - https://bbs.archlinux.org/viewtopic.php?id=215085

But I agree the documentation situation isn't great in general. I'll see if I can at least add the commands and order of steps for a normal run with Wireguard and OpenVPN for example and put them in the CONTRIBUTING.md

[goodboy]

@jamesmcm Yo!

No, thank you so much for the in depth response and clarification.

Sorry i've ghosted; I WILL be coming back to this as soon as I can, just been hit recently by boring life stuff the past couple weeks.

A couple projects i work on are extremely interested in getting the PtP stuff going and obviously I (/we) would like to use that motivation to help make vopono better 😎

I'm hoping to have a proper per-section breakdown discussion-response going within the next couple weeks 👍🏼