-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[vnet][1] setup TUN and IPv6 on MacOS #40893
Conversation
The PR changelog entry failed validation: Changelog entry not found in the PR body. Please add a "no-changelog" label to the PR, or changelog lines starting with |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is currently no DNS support, no app forwarding, no Teleport login of any kind, that will all come in following PRs. At this point the VNet process just logs that it has handled the connection and closes it immediately.
To make it more clear, at the moment all you can do is try to reach the IPv6 address directly, for example:
$ tsh vnet -d
2024-04-26T17:08:07+02:00 INFO Spawning child process as root to create and setup TUN device vnet/setup_darwin.go:44
2024-04-26T17:08:12+02:00 INFO Created TUN device. device:utun4 vnet/setup.go:83
2024-04-26T17:08:12+02:00 INFO [VNET] Running Teleport VNet. ipv6:fd75:7ee9:a8d5:: trace_id:c9080ff4b2b4cd396cdb04c01d2c8a3f span_id:0d365c617eaf8431 vnet/vnet.go:226
2024-04-26T17:08:12+02:00 DEBU Forwarding IP packets between OS and VNet. trace_id:c9080ff4b2b4cd396cdb04c01d2c8a3f span_id:0d365c617eaf8431 vnet/vnet.go:338
So I grab fd75:7ee9:a8d5::
and then:
$ curl -g -6 'http://[fd75:7ee9:a8d5::]:80'
Maybe there's an easier way to do this, but running that is what worked for me to trigger the following output from tsh vnet -d
:
2024-04-26T17:08:47+02:00 DEBU [VNET] Handling TCP connection. request:{80 fd75:7ee9:a8d5:: 63653 fd75:7ee9:a8d5::1} trace_id:c9080ff4b2b4cd396cdb04c01d2c8a3f span_id:0d365c617eaf8431 vnet/vnet.go:244
2024-04-26T17:08:47+02:00 DEBU [VNET] No handler for address. request:{80 fd75:7ee9:a8d5:: 63653 fd75:7ee9:a8d5::1} addr:fd75:7ee9:a8d5:: trace_id:c9080ff4b2b4cd396cdb04c01d2c8a3f span_id:0d365c617eaf8431 vnet/vnet.go:249
2024-04-26T17:08:47+02:00 DEBU [VNET] Finished handling TCP connection. request:{80 fd75:7ee9:a8d5:: 63653 fd75:7ee9:a8d5::1} trace_id:c9080ff4b2b4cd396cdb04c01d2c8a3f span_id:0d365c617eaf8431 vnet/vnet.go:251
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be even more clear, even this no longer works, since I removed promiscuous mode in the previous PR this no longer handles packets destined for an IP address that hasn't been assigned yet, and outside of tests, no IPs are assigned yet. Things start actually working in #41031 where we assign an IP for DNS, and querying DNS for an app name assigns an IP to that app
0582cfc
to
0e28254
Compare
62dad65
to
40ec0c1
Compare
90fc2d7
to
6dd426d
Compare
|
||
// VnetAdminSetupSubCommand is the sub-command tsh vnet uses to perform | ||
// a setup as a privileged user. | ||
VnetAdminSetupSubCommand = "vnet-admin-setup" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
VnetAdminSetupSubCommand = "vnet-admin-setup" | |
VNetAdminSetupSubCommand = "vnet-admin-setup" |
To be consistent with nklaassen/vnet0
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ravicious and I have decided to capitalize as Vnet in code - otherwise JS gRPC codegen starts creating things called vNetXxxxx
. Logs and errors still use VNet
.
} | ||
|
||
func configureOS(ctx context.Context, cfg *osConfig) error { | ||
if cfg.tunIPv6 != "" && cfg.tunName != "" { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider inverting the condition here so that the return nil
is indented and the body that does most of the work is not in a conditional.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to be adding more conditions later on - see https://github.com/gravitational/teleport/pull/41032/files#diff-7627839bca33fec5fb081fb7297edbdbaf62435ee4e8d8105c76ccbb7edf8150
07c8f8a
to
08a8d97
Compare
4805a51
to
8ef62be
Compare
08a8d97
to
c65c36f
Compare
c65c36f
to
bd00c44
Compare
Co-authored-by: Isaiah Becker-Mayer <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved because it's all logically correct AFAICT, left some suggestions for how to keep all the various concurrency mechanisms more readily understandable for future you / others. Your call whether to take them or leave them.
lib/vnet/vnet.go
Outdated
g.Go(func() error { | ||
// When the context is canceled for any reason, the caller may have canceled it or one of the other | ||
// concurrent tasks, destroy everything and quit. | ||
<-ctx.Done() | ||
close(m.destroyed) | ||
m.linkEndpoint.Close() | ||
err := trace.Wrap(m.tun.Close(), "closing TUN device") | ||
allErrors <- err | ||
return err | ||
}) | ||
return trace.Wrap(g.Wait()) | ||
} | ||
|
||
// Destroy closes the link endpoint, waits for all goroutines to terminate, and destroys the networking stack. | ||
func (m *Manager) Destroy() error { | ||
close(m.destroyed) | ||
m.linkEndpoint.Close() | ||
// Deliberately ignoring the error from g.Wait() to return an aggregate of all errors. | ||
_ = g.Wait() | ||
m.wg.Wait() | ||
m.stack.Destroy() | ||
return nil | ||
|
||
close(allErrors) | ||
return trace.NewAggregateFromChannel(allErrors, context.Background()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider adding something like these comments / rearrangement to try and keep the order/reasons for all of the various concurrency mechanisms easy to recall going forward (note that I may have somethings wrong here, this is just what I inferred based on studying the code):
g.Go(func() error {
// When the context is canceled for any reason (the caller may have canceled it or one of the other
// concurrent tasks), ensure the `forwardBetweenTunAndNetstack` goroutine is stopped. (The `statsHandler`)
// goroutine will stop on its own when the context is canceled.)
<-ctx.Done()
// Close the link endpoint and TUN device to cause the `forwardBetweenTunAndNetstack` function to
// return.
m.linkEndpoint.Close()
err := trace.Wrap(m.tun.Close(), "closing TUN device")
allErrors <- err
return err
})
// Deliberately ignoring the error from g.Wait() to return an aggregate of all errors.
_ = g.Wait()
// Close the destroyed channel to signal that the VNet is no longer running, and that any in
// flight connections in the `stack` should be closed.
close(m.destroyed)
// Wait for all in flight connections to be closed.
m.wg.Wait()
// Now we can destroy the `stack` to free all resources.
m.stack.Destroy()
close(allErrors)
return trace.NewAggregateFromChannel(allErrors, context.Background())
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for the review! I added a bunch of comments in ebef36a
This is the second in a series of PRs implementing Teleport VNet RFD. parent child
This commit adds a hidden (for now)
tsh vnet
command which starts up Teleport VNet. It is only supported on MacOS.When run as root,
tsh vnet
is able to create a TUN virtual network device in its own process. When run as a regular user, it exec's a child process as root using an AppleScript hack to get administrator privileges. The root process then passes the file descriptor for the TUN device to the non-root process over a unix-domain socket. This applescript hack will be superseded by a daemonized process once implemented, but it will probably stick around for development versions.When creating the TUN device, the root process installs in the host an IPv6 route for all link-local addresses in the range of the VNet network, identified by a 64-bit prefix consisting of the 8-bit ULA prefix, a 40-bit randomly generated global ID, and a 16-bit subnet ID.
After getting the TUN device to the calling process,
tsh vnet
initiates bidirectional forwarding of traffic between the host OS and the VNet network.There is currently no DNS support, no app forwarding, no Teleport login of any kind, that will all come in following PRs.
Because this is all MacOS-specific code that must either run as root or get root through an interactive UI element, and would need to mess with the host networking stack, none of it is unit tested. The virtual networking component is being tested in #40889 and the rest of the functionality to be implemented will be tested similarly.