Add support for building for DO Debian, DO Ubuntu, Dockerfile debian-slim #8

gobengo · 2018-02-22T04:24:21Z

You can now provision remote linux hosts over SSH using ./build/bin/remote <user>@<ip>

exit_node_ip=165.227.241.194
./build/remote root@$exit_node_ip
# will provision the exitnode on that host, then reboot it and wait for it to come back

I tested this on Digital Ocean: Ubuntu 16.04, 17.10

--

There is also now a Dockerfile in here, so the following should start to be useful.

Note: It doesn't work all the way now. Opening this PR to illustrate blockers.

docker build -t exitnode .
docker run exitnode

...

+ make install
rm -f /usr/local/bin/babeld
mkdir -p /usr/local/bin
cp -f babeld /usr/local/bin
mkdir -p /usr/local/man/man8
cp -f babeld.man /usr/local/man/man8/babeld.8
+ REQUIRED_MODULES='nf_conntrack_netlink nf_conntrack nfnetlink l2tp_netlink l2tp_core l2tp_eth'
+ for module in '$REQUIRED_MODULES'
+ grep -q nf_conntrack_netlink /etc/modules
+ echo '\nnf_conntrack_netlink'
+ modprobe nf_conntrack_netlink
modprobe: ERROR: ../libkmod/libkmod.c:557 kmod_search_moddep() could not open moddep file '/lib/modules/4.9.49-moby/modules.dep.bin'
The command '/bin/sh -c ./build/docker/docker-build' returned a non-zero code: 1

…sion on remote host, plus Dockerfile

gobengo · 2018-02-22T04:59:37Z

Full output of docker build .
https://travis-ci.org/gobengo/exitnode/builds/344650136

gobengo · 2018-02-22T05:14:02Z

@jhpoelen

Well the good news is that travis-ci actually seems like it's running the tunneldigger broker just fine? https://travis-ci.org/gobengo/exitnode/builds/344651270 . It runs until I cancel it.

The bad news is:
When I run locally on my mac:

ben@bengo-air ~/dev/sudomesh/exitnode travis ⚡ docker run --privileged -it -e IP=1.2.3.4 exitnode
Starting in fungible mode
[INFO/tunneldigger.broker] Initializing the tunneldigger broker.
[INFO/tunneldigger.broker] Registered script '/opt/tunneldigger/broker/scripts/up_hook.sh' for hook 'session.up'.
[INFO/tunneldigger.broker] Registered script '/opt/tunneldigger/broker/scripts/down_hook.sh' for hook 'session.down'.
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/opt/tunneldigger/broker/main.py", line 78, in <module>
    tunnel_manager.initialize()
  File "broker/broker.py", line 145, in initialize
    self.netlink = l2tp.NetlinkInterface()
  File "broker/l2tp.py", line 68, in __init__
    self.family_id = controller.get_family_id(L2TP_GENL_NAME)
  File "broker/genetlink.py", line 82, in get_family_id
    return m.attrs[CTRL_ATTR_FAMILY_ID].u16()
KeyError: 1

Update: This would have happened if the OS doesn't support L2TP.
I think, since "Docker for Mac" actually involves running a Linux VM, that in my case that VM just doesn't have the right kernel modules. So when this container is sharing same kernel and running on that VM, it can't L2TP. "Docker for Mac" keeps that Linux VM pretty opaque.

jhpoelen · 2018-02-22T15:37:27Z

README.md

 ```

-Expected output should be something like:
+This should SSH to the exit node, configure it, reboot, and wait for the reboot to complete.


suggest to include expected output

jhpoelen · 2018-02-22T15:38:43Z

README.md

+# run
+docker run --privileged -e IP=$exit_node_ip exitnode
+# run bash for debugging
+docker run --privileged -it -e IP=$exit_node_ip exitnode bash


what is the use case for running an exit node in docker? Does this allow for connecting physical home node to a docker container? If so that would be awesome! Suggest to document how to do this, if you haven't already.

what is the use case for running an exit node in docker?

I am willing/eager to host an exitnode on my existing cluster, but don't want to pay more $/month for a dedicated VM if I don't have to. My existing cluster dynamically schedules containers across it, as long as I can build the workload into a docker container.

Even if I didn't have an existing cluster, if I wanted to run other apps on the same server, I'd want to isolate them in separate linux containers so they can use different system deps, and docker/Dockerfiles are one of the easier ways to build/distribute/run lxc containers.

Does this allow for connecting physical home node to a docker container? If so that would be awesome! Suggest to document how to do this, if you haven't already.

That's the idea. Will document further when I actually get it working. Next steps

- Determine why docker run ... won't work on my mac, but does on Travis-CI

- See if the container works on my k8s cluster, since it works on Travis-CI

Thanks for taking the time to elaborate! The docker case sounds really cool - and I'd like to suggest to add it as part of a separate pull request after you get it working and with examples of how to use it. Perhaps @bennlich and @paidforby have some ideas about this.

Sounds useful and practical to me. I was hoping to run nginx and host some webpages on my exit node eventually, but I would want to give access to other contributors. I could create separate UNIX users or run a CMS (like wordpress), still I would worry about potential conflicts or unintended side effects. Wrapping the exitnode up in a docker container would put my mind at ease.

very much agree with @gobengo and @paidforby about the benefits of docker. I was making an argument for making sure it works / examples before introduce it. Won't block it, but am weary of introducing functionality that can't be tested / used.

A dockerized exit node sounds super awesome 👍

@bennlich @jhpoelen FWIW I did diagnose and resolve the above issues, and got it running in Docker and in my Kubernetes cluster (container requires --privileged flag, I think because L2TP requires privileged syscalls?, but that's fine). I have not tested the exitnode yet from a home node.

So I have three things to test, and will do so in this order:

exitnode in digitalocean as created by ./build/remote

exitnode in docker in digitalocean - just to prove docker does{,n't} create problems

exitnode in docker in my kubernetes cluster (on AWS FWIW) - I may never test this, or at least if it doesn't work I'm not in a rush to debug this. No one else needs this setup. Though if the mesh need more exitnodes this is the cheapest place for me to run one (.... or a dozen) because it can be packed into my exiting EC2 Instances.

jhpoelen · 2018-02-22T15:39:36Z

build/bin/install-dependencies

+echo "release_name=$release_name"
+
+
+# Upgrade kernel on OS that require it


is this upgrading the kernel or installing extra modules?

jhpoelen · 2018-02-22T15:40:54Z

build/bin/install-dependencies

+echo "release_name=$release_name"
+if [ "$release_name" == '"Ubuntu"' ]; then
+  echo "OS is Ubuntu. Need to upgrade kernel"
+  # This would NOT work on Debian 8.10, (@TODO test others).


nice! By "would NOT work" you mean something like would not be executed correct?

jhpoelen · 2018-02-22T15:43:08Z

build/remote

+#!/usr/bin/env bash
+
+# This script should provision a remote linux server as an exit node
+# usage: PUBLIC_IP=165.227.241.194 ./build/remote [email protected]


why are you using PUBLIC_IP if you are not using it?

It's used below on line 27. e.g. you can provide it if the PUBLIC_IP is different from the ip/hostname you are SSHing to.

But it doesn't really need to be in this usage note. That's from before I made it use your SSH host as IP if no env var is provided.

jhpoelen · 2018-02-22T15:45:07Z

create_exitnode.sh

-cp /opt/exitnode/l2tp_broker.cfg $TUNNELDIGGER_HOME/broker/l2tp_broker.cfg
+echo "EXITNODE_DIR=$EXITNODE_DIR"
+if [[ ! $EXITNODE_DIR ]]; then
+  echo "cloning jhpoelan/exitnode to /opt/exitnode"


jhpoelen/exitnode --> sudomesh/exitnode

jhpoelen · 2018-02-22T15:47:35Z

create_exitnode.sh

@@ -142,4 +118,4 @@ service sudomesh-gateway start
 service tunneldigger start
 service babeld start

-reboot now
+# reboot now


suggest to remove commented command

…come back

…ction

…rward is not already 1 (e.g. if docker or other container runtime sets it up for you)

bennlich · 2018-03-12T17:01:33Z

This PR was superceded by #11

gobengo added 11 commits February 21, 2018 17:21

Break up create_exitnode.sh and add ./build/bin/remote that can provi…

ba617fe

…sion on remote host, plus Dockerfile

Work with ubuntu

9e6ef23

Works on Debian again

e20c666

only install kernel updates for Ubuntu

eabe3ff

better README

2d087bd

build/bin/lint and lint build/remote

47bed2e

lint bash in /build

3010695

lint

1e96fc1

move build

dcfd2b9

Merge branch 'master' of github.com:gobengo/exitnode into dockerfile

2b2e5d3

explain files

356ac59

gobengo force-pushed the dockerfile branch from 72ff94f to 356ac59 Compare February 22, 2018 04:42

gobengo added 2 commits February 21, 2018 20:48

travis?

b449e51

dont be so strict in create_exitnode.sh

0b7d96d

fix typo in do_start if /etc/init.d/sudomesh-gateway

2e766ec

jhpoelen reviewed Feb 22, 2018

View reviewed changes

gobengo added 6 commits February 22, 2018 08:08

Add expected output to README for ./build/remote

459fb2f

hide some unneded output from ./build/remote when waiting for ssh to …

19368e2

…come back

Clarify comment about liux-image-extra package in install-dependencies

667ede7

remove extra output from ./create_extinode.sh

52cce8c

Remove commented reboot command from ./create_exitnode.sh

156cce1

Clone exitnode from sudomesh, not jhpoelen

17be021

gobengo added 4 commits February 22, 2018 08:21

Remove usage comment from ./build/remote since there is now usage fun…

49bd8ef

…ction

Only edit /etc/sysctl.conf and /proc/sys/net/ipv4/ip_forward if ip_fo…

4a0d509

…rward is not already 1 (e.g. if docker or other container runtime sets it up for you)

Merge branch 'travis' into dockerfile

d4b4f66

use wait_for_ssh function in build/remote

3532e90

gobengo mentioned this pull request Mar 11, 2018

add more potential exitnode ips to home nodes sudomesh/bugs#17

Closed

gobengo closed this Mar 12, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for building for DO Debian, DO Ubuntu, Dockerfile debian-slim #8

Add support for building for DO Debian, DO Ubuntu, Dockerfile debian-slim #8

gobengo commented Feb 22, 2018

gobengo commented Feb 22, 2018 •

edited

Loading

gobengo commented Feb 22, 2018 •

edited

Loading

jhpoelen Feb 22, 2018

gobengo Feb 22, 2018

jhpoelen Feb 22, 2018

gobengo Feb 22, 2018 •

edited

Loading

jhpoelen Feb 22, 2018

paidforby Feb 22, 2018

jhpoelen Feb 22, 2018

bennlich Feb 23, 2018

gobengo Mar 12, 2018 •

edited

Loading

jhpoelen Feb 22, 2018

jhpoelen Feb 22, 2018

jhpoelen Feb 22, 2018

gobengo Feb 22, 2018

jhpoelen Feb 22, 2018

jhpoelen Feb 22, 2018

bennlich commented Mar 12, 2018

		echo "release_name=$release_name"


		# Upgrade kernel on OS that require it

Add support for building for DO Debian, DO Ubuntu, Dockerfile debian-slim #8

Add support for building for DO Debian, DO Ubuntu, Dockerfile debian-slim #8

Conversation

gobengo commented Feb 22, 2018

gobengo commented Feb 22, 2018 • edited Loading

gobengo commented Feb 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gobengo Feb 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gobengo Mar 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bennlich commented Mar 12, 2018

gobengo commented Feb 22, 2018 •

edited

Loading

gobengo commented Feb 22, 2018 •

edited

Loading

gobengo Feb 22, 2018 •

edited

Loading

gobengo Mar 12, 2018 •

edited

Loading