ssh tunnels - for beginners

Table of Contents

• What you will learn
• ssh - client and server
• Understanding the command structure
• The three (3) involved hosts
  • The start of the tunnel
  • The end of the tunnel
  • The final destination
• Skipping the shell
• Jump-hosts
• Exercises
  • Preparations
  • Questions and answers

What you will learn

This document aims to teach how to understand ssh-tunnels reasonably well intuitively, including the command structure.

Disclaimer: I am not an expert. Welcome to provide feedback! (Post an Issue, for example.) Any typos? Any technical terms used incorrectly?

The idea is that if you understand stuff, you will be able to remember it. So you will be able to really understand - and construct - various ssh-tunnel commands, perhaps without searching the web. Examples:

# A common and quite simple variant
ssh -L 9000:some-server:5000  myuser@ssh-server

# A variant that might be confusing. (What does "localhost" refer to?)
ssh -L localhost:9000:localhost:5000  myuser@ssh-server

# The most complex variant in this document
ssh -nNT -R 0.0.0.0:9000:some-server:5000 -J myuser@ssh-jumphost myuser@ssh-server

Prerequisites: Basic knowledge of ssh - how to log in etc.

There are a few exercises with questions/answers too. There is also a docker-compose file setting up some hosts to use for experimentation. (Prerequisites: Have docker, docker-compose installed. Know some basic usage of them. Some basic understanding of what a http-server is and what the program curl does.)

Keywords: ssh, tunnel, port-forwarding, local port-forwarding, remote port-forwarding, tutorial, explainer

ssh - client and server

We have a local host, say "ssh-client", on which you can use ssh. And we have a remote host, say "ssh-server", to which you can log in using say "myuser" and "password". With regular ssh-usage, you would log in from "ssh-client" to "ssh-server" (and get a shell there) using:

ssh myuser@ssh-server

       ______________
       |            |
       |            | ssh-client
       |____________|
             |
             |
             |  ssh connection
             V
       ______o_______
       |            |
       |            | ssh-server
       |____________|

Here, however, we will describe some simple variants of ssh-tunnelling. We can use the ssh-session involving the ssh-client and the ssh-server to tunnel traffic securely (in an encrypted way).

Here is a depiction:

                 port/socket
                 opened by ssh
               /
              |
              |
       _______o________
       |      |       |  ssh client
       | tunnel start |  (or server)
       |_____| |______|
             | |
             | |
             | | ssh tunnel
             | |
       ______| |______
       |     | |     |  ssh server
       | tunnel end  |  (or client)
       |______|______|
              |
              |
              V
       _______o_______
       | final       |
       | destination |
       |_____________|

Understanding the command structure

An ssh tunnel can be created with the -L (Local) or -R (Remote) flag.

• The flag indicates where the tunnel should start - Locally (at the ssh-client) or Remotely (at the ssh-server).

• The end of the tunnel will (obviously) be on the opposite side of the ssh-connection (with -L at the ssh-server, with -R at the ssh-client.)

• Aside from the start of the tunnel and the end of the tunnel, there is a third party involved too: The final destination. (Oftentimes, this is the same host as the end of the tunnel.)

So, we have three parties involved:

1. The start of the tunnel.

2. The end of the tunnel.

3. The final destination.

(We could also say that we have more parties involved later - the parties from which the connections are finally made. That is, when the ssh-tunnel has been created and is actually being used. That is, when connections are made to the start of the tunnel, the traffic is routed through the tunnel and reaching the final destination. It is perhaps common that these connections are made from the same host has the start of the tunnel, but this is not necessarily so.)

Let’s just look at an example. We want to:

• let the ssh-client open socket localhost:9000 for listening

• let ssh route the traffic through a tunnel to the ssh-server

• and let the ssh-server route the traffic further to the final destination - some-server:5000:

ssh -L localhost:9000:some-server:5000 myuser@ssh-server

Another example:

• let the ssh-server open socket localhost:9000 for listening

• let ssh route the traffic through a tunnel to the ssh-client

• and let the ssh-client route the traffic further to the final destination - some-server:5000:

ssh -R localhost:9000:some-server:5000 myuser@ssh-server

We can break down the command as follows:

ssh -L localhost:9000:some-server:5000  myuser@ssh-server
    |________________|________________|
        |                  |
   specifies               |
   the start of            |
   the tunnel           specifies
                        the final
                        destination

This is pretty much the gist of this document. If you are in a hurry, you can stop reading now.

The three (3) involved hosts

We will continue with some details regarding:

• The start of the tunnel

• The end of the tunnel

• The final destination

The start of the tunnel

The start of the tunnel is constituted by a socket opened by ssh for listening.

• ssh -L: "Local" - it is the ssh-client that opens the socket.

• ssh -R: "Remote" - it is the ssh-server that opens the socket.

So, for example:

• ssh -L localhost:9000:…​ - the local ssh-client opens a socket, port 9000 on its localhost.

• ssh -R localhost:9000:…​ - the remote ssh-server opens a socket, port 9000 on its localhost. (Yes, note the "localhost" - in this context it is interpreted by the party that is instructed to create the start of the tunnel, which here is the remote ssh-server. When typing the command one might be misled to think that anything saying "localhost" would refer to the host where the command is invoked - the ssh-client - but that is not the case.)

We can also note that this whole thing is sometimes referred to as "port-forwarding":

• ssh -L: "Local" - Local port forwarding.

• ssh -R: "Remote" - Remote port forwarding.

In all examples so far, we have specified "localhost" as the bind address for the socket (the start of the tunnel). "localhost" is an alias for 127.0.0.1, the loop-back interface. Doing so, we allow connections only from the same host. That is, we allow connections only from ssh-client itself (if using -L) or ssh-server itself (if using -R).

But we could also tell ssh to open a socket on all interfaces, not just the loop-back interface, by using 0.0.0.0 (an empty bind address) or *:

ssh -L 0.0.0.0:9000:some-server:5000  myuser@ssh-server
ssh -R 0.0.0.0:9000:some-server:5000  myuser@ssh-server

Whether this is allowed depends on ssh-configuration (an option named "GatewayPorts"). If it works, it allows connections from other hosts (than the start of the tunnel) to use the ssh-tunnel.

Note: If "localhost" is enough given the use-case at hand, it should probably be used. (It might be considered more secure, since it does not allow inbound connections from other hosts).

It is common to see the bind address specification left out:

ssh -L 9000:some-server:5000  myuser@ssh-server

What this means (localhost:9000 or 0.0.0.0:9000) might depend on configuration (an option named "GatewayPorts"), but it is not uncommon for this to mean that "localhost" is implicitly used. (Some people prefer to spell it out in order to be more explicit.)

The end of the tunnel

The end of the tunnel is not really explicitly specified on the command line. It is implicitly determined as the being at opposite side from the start of the tunnel (obviously):

• ssh -L: "Local" - it is the ssh-client that opens the socket, so the "end" of the tunnel is at the ssh-server.

• ssh -R: "Remote" - it is the ssh-server that opens the socket, so the "end" of the tunnel is at the ssh-client.

The final destination

From the end of the tunnel, the traffic is then forwarded to the final destination. In the example above it is some-server:5000. So the final destination must (obviously) be reachable from the end of the tunnel.

Note also that what is specified on the command line as "the final destination" is interpreted by the end of the tunnel, not at the start of the tunnel. This is significant, for example in the quite typical case where we specify localhost as the final destination.

Consider for example a -L-tunnel, where we want the final destination to be the same host as the end of the tunnel, that is the ssh-server. So, we want the final destination to be something like ssh-server:5000. We can specify that as localhost:5000:

ssh -L localhost:9000:localhost:5000  myuser@ssh-server

Note that the two localhost here refer to two different hosts. We have specified that the tunnel should start at localhost:9000. This "localhost" is the loopback interface at the start of the tunnel. (For a -L tunnel it is the ssh-client). And then we have specified that the final destination should be localhost:5000. This is interpreted by the end of the tunnel, so "localhost" is the loopback interface at the end of the tunnel. (For a -L tunnel it is the ssh-server).

When typing the command, one could easily be misled to think that anything saying "localhost" refers to the host where you are sitting - the ssh-client. But as we see here, this is not necessarily the case.

Skipping the shell

From https://blog.trackets.com/2014/05/17/ssh-tunnel-local-and-remote-port-forwarding-explained-with-examples.html: You might have noticed that every time we create a tunnel you also SSH into the server and get a shell. This isn’t usually necessary, as you’re just trying to create a tunnel. To avoid this we can run SSH with the -nNT flags, such as the following, which will cause SSH to not allocate a tty and only do the port forwarding.

ssh -nNT -L localhost:9000:some-server:5000 myuser@ssh-server

Jump-hosts

In many corporate environments, administrators may require that when you ssh from your machine to various other machines, you must pass through some jumphost. For example like this:

ssh -J myuser@ssh-jumphost myuser@ssh-server

This creates a pretty much regular ssh-session between the ssh-client and ssh-server. And ssh-tunnels can be created as per usual, for example:

ssh -L localhost:9000:some-server:5000 -J myuser@ssh-jumphost myuser@ssh-server

This does not affect where the tunnel starts or ends - it is the ssh-client and ssh-server that constitute the start and end of the tunnel.

TODO - will the tunnel traffic sort of "pass through" the jumphost? Can this be elaborated on?

Exercises

Preparations

We will use docker and docker-compose to set up some hosts to experiment with.

• ssh-client - the host on which we will create various ssh-tunnels

  • also runs a http server process (port 5000) that can act as final destination

  • in some cases, we will try to "use" the ssh-tunnel from here

• ssh-server - the ssh server that will take part in tunnel creation

  • also runs a http server process (port 5000) that can act as final destination

  • in some cases, we will try to "use" the ssh-tunnel from here

• some-server - runs a http server (port 5000) that can act as final destination

• ssh-jumphost - a host that can be used as an ssh-jumphost

• test-client - a host from which we can use ssh tunnels

  • in some cases, we will try to "use" the ssh-tunnel from here

Start the whole thing using:

docker-compose up -d

I might be convenient to open 4 terminals/shells:

1. The main work shell: docker-compose exec ssh-client bash (used for creating tunnels)

2. docker-compose exec ssh-client bash (this shell can be used for testing tunnels)

3. docker-compose exec test-client bash (used for testing tunnels)

4. docker-compose exec ssh-server bash (used for testing tunnels)

In your (main work) shell, "enter" the ssh-client.

docker-compose exec ssh-client bash

#our environment with the docker-containers is limited,
# ssh needs the -4 flag. (Without it, there will be warning
# messages emitted when creating tunnels, saying stuff like
# "bind [::1]:9000: Address not available")
alias ssh='ssh -4'

Make a few simple sanity tests - these should all work:

ssh myuser@ssh-server
# password is "password"
# exit the shell to get back to ssh-client

ssh -J myuser@ssh-jumphost myuser@ssh-server
# exit the shell to get back to ssh-client

# Check that the http server processes are running, by connecting to them with curl:
curl ssh-client:5000
curl ssh-server:5000
curl some-server:5000

# Notice that the http servers respond with a message
# indicating their host names. This will facilitate
# our testing.

Questions and answers

Ok, let’s stay on ssh-client and create some tunnels. (Answers below.)

1. Use ssh to open port 9000, and route traffic through a tunnel to ssh-server, with final destination to some-server on port 5000.

   • Test from ssh-client using curl localhost:9000, the response should indicate that some-server port 5000 has been reached.

   • Test from test-client using curl ssh-client:9000. Should this work?

2. Create the same tunnel, except that it can also be used from test-client.

   • Test from ssh-client using curl localhost:9000, the response should indicate that some-server port 5000 has been reached.

   • Test from test-client using curl ssh-client:9000, the response should indicate that ssh-server port 5000 has been reached.

3. Use ssh to open port 9000 on ssh-client’s localhost, and route traffic through a tunnel to ssh-server, with final destination to ssh-server itself on port 5000.

   • Test from ssh-client using curl localhost:9000, the response should indicate that ssh-server port 5000 has been reached.

4. Create the same tunnel as in 1 but using ssh-jumphost as jump host.

   • Test like in 3.

5. Create a tunnel that can be used to connect from test-client to some-server:5000 as final destination. The tunnel shall start at ssh-server port 9000, and shall pass through the jumphost, and end at ssh-client.

   • Test from test-client using curl ssh-server:9000, response should indicate that some-server port 5000 has been reached.

Answers (the -nNT flags are optional):

1. ssh -nNT -L localhost:9000:some-server:5000 myuser@ssh-server

   • Testing from test-client should not work. The socket on ssh-client (localhost:9000) is created on loop-back interface (localhost), so it can only be reached from ssh-client itself.

2. ssh -nNT -L 0.0.0.0:9000:some-server:5000 myuser@ssh-server

3. ssh -nNT -L localhost:9000:localhost:5000 myuser@ssh-server

4. ssh -nNT -L localhost:9000:localhost:5000 -J myuser@ssh-jumphost myuser@ssh-server.

5. ssh -nNT -R 0.0.0.0:9000:some-server:5000 -J myuser@ssh-jumphost myuser@ssh-server