-
Notifications
You must be signed in to change notification settings - Fork 78
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
tunnel-server: add basic deployment guide (#405)
* tunnel-server: add basic deployment guide
- Loading branch information
Roy Razon
authored
Feb 11, 2024
1 parent
8a9d527
commit 66c0be3
Showing
5 changed files
with
236 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
/tls.crt | ||
/tls.key | ||
/ssh_host_key | ||
/cookie_secret | ||
/config.env |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,107 @@ | ||
# Deploying your own instance of the Tunnel Server | ||
|
||
This directory contains an example deployment of the Tunnel Server on Kubernetes. | ||
|
||
Note that this is an advanced task which requires some networking and Kubernetes know-how. | ||
|
||
## Why | ||
|
||
Deploying a private instance of the Tunnel Server allows for fine-grained control of: | ||
|
||
- The URLs created for preview environments: e.g, use a custom domain. | ||
- Geolocation of the server: reduced distance to environments can result better network performance. | ||
- Security and privacy: deploy everything in your VPC, no traffic to 3rd parties. | ||
|
||
## Requirements | ||
|
||
- A Kubernetes cluster | ||
- A TLS certificate for your domain | ||
- `kubectl` and `kustomize` | ||
|
||
## Overview | ||
|
||
The Tunnel Server natively listens on three ports: | ||
- A SSH port which accepts tunneling SSH connections from environments | ||
- A HTTP port which accepts requests from clients (browsers, etc) | ||
- A TLS port which directs incoming connections to either the SSH server or the HTTP server according to the [SNI server name](https://en.wikipedia.org/wiki/Server_Name_Indication). | ||
|
||
The `SSH_HOSTNAMES` env var determines which hostnames are directed to the SSH server and the rest are directed to the HTTP server. If the env var is not specified, the hostname of the `BASE_URL` is used. | ||
|
||
In this deployment scheme, the TLS port is exposed using a [`LoadBalancer-type K8S Service`](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer). The base hostname `yourdomain.example` is used for tunneling SSH connections and any hostname on the subdomain `*.yourdomain.example` will be used for client requests. | ||
|
||
## Instructions | ||
|
||
### 1. Setup the domain and the TLS certificate | ||
|
||
Make sure the certificate is for both the base domain and the wildcard subdomain, e.g, `yourdomain.example` and `*.yourdomain.example`. | ||
|
||
Put the cert and key (in PEM format) in the files `tls.crt` and `tls.key` | ||
|
||
Copy `config.env.example` to `config.env` and set your domain in the `BASE_URL` variable. | ||
|
||
### 2. Generate a cookie secret | ||
|
||
The cookie secret is a simple text-based secret (like a password) in a file. | ||
|
||
```bash | ||
LC_ALL=C tr -dc A-Za-z0-9 </dev/urandom | head -c 40 > cookie_secret | ||
``` | ||
|
||
### 3. Generate a SSH host key | ||
|
||
```bash | ||
ssh-keygen -t ed25519 -N "" -f ssh_host_key | ||
``` | ||
|
||
### 4. Generate and deploy the configuration | ||
|
||
Review the generated configuration: | ||
|
||
```bash | ||
kustomize build . | ||
``` | ||
|
||
To deploy to the K8S cluster: | ||
|
||
```bash | ||
kustomize build . | kubectl apply -f - | ||
``` | ||
|
||
Make sure the two deployments `tunnel-server` and `tunnel-server-stunnel` exist and that their pods are running. | ||
|
||
### 5. Test the SSH endpoint | ||
|
||
This requires OpenSSH and a recent-enough OpenSSL CLI with support for the `-quiet` option. | ||
|
||
To test the SSH endpoint (replace `$MY_DOMAIN` with your domain): | ||
|
||
```bash | ||
MY_DOMAIN=yourdomain.example | ||
|
||
# NOTE: for EKS, replace .ip with .hostname below | ||
EXTERNAL_IP=$(kubectl get service tunnel-server -o jsonpath='{.status.loadBalancer.ingress[0].ip}') | ||
|
||
ssh -nT -o "ProxyCommand openssl s_client -quiet -verify_quiet -servername $MY_DOMAIN -connect %h:%p" -p 443 foo@$EXTERNAL_IP hello | ||
``` | ||
|
||
### 6. Create DNS records for the `tunnel-server` Service external IP or hostname | ||
|
||
Create two DNS records: `*.yourdomain.example` and `yourdomain.example`, both pointing to the external IP of the `tunnel-server` service. | ||
|
||
The address is not guaranteed to be static. According to your Kubernetes provider, there could be multiple ways to define a DNS entry for it. Here are some guides: | ||
|
||
- Amazon AWS: [EKS](https://docs.aws.amazon.com/eks/latest/userguide/network-load-balancing.html) | ||
- Google Cloud: [GKE](https://cloud.google.com/kubernetes-engine/docs/concepts/service-load-balancer) | ||
- Azure: [AKS](https://learn.microsoft.com/en-us/azure/aks/load-balancer-standard) | ||
|
||
### 7. Use your Tunnel Server instance with the Preevy CLI | ||
|
||
The `up` and `urls` commands accept a `-t` flag which can be used to set the Tunnel Server URL. Specify `ssh+tls://yourdomain.example` to use your instance. | ||
|
||
### 8. Cleanup: Delete the deployed objects | ||
|
||
To delete the deployed Tunnel Server and associated Kubernetes objects, run: | ||
|
||
```bash | ||
kustomize build . | kubectl delete -f - | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
BASE_URL=https://local.livecycle.run |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
apiVersion: kustomize.config.k8s.io/v1beta1 | ||
kind: Kustomization | ||
resources: | ||
- tunnel-server.yaml | ||
secretGenerator: | ||
- files: | ||
- tls.crt | ||
- tls.key | ||
name: tunnel-server-tls | ||
type: kubernetes.io/tls | ||
- files: | ||
- ssh_host_key | ||
name: tunnel-server-ssh | ||
type: Opaque | ||
- files: [cookie_secret] | ||
name: tunnel-server-cookies | ||
type: Opaque | ||
configMapGenerator: | ||
- name: tunnel-server-config | ||
envs: [config.env] | ||
images: | ||
- name: ghcr.io/livecycle/preevy/tunnel-server | ||
newTag: main-8a9d527 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: tunnel-server | ||
spec: | ||
replicas: 1 | ||
selector: | ||
matchLabels: | ||
app: tunnel-server | ||
template: | ||
metadata: | ||
labels: | ||
app: tunnel-server | ||
spec: | ||
volumes: | ||
- name: ssh | ||
secret: | ||
defaultMode: 400 | ||
secretName: tunnel-server-ssh | ||
- name: tls-cert | ||
secret: | ||
defaultMode: 400 | ||
secretName: tunnel-server-tls | ||
containers: | ||
- env: | ||
- name: COOKIE_SECRETS | ||
valueFrom: | ||
secretKeyRef: | ||
name: tunnel-server-cookies | ||
key: cookie_secret | ||
- name: SSH_HOST_KEY_PATH | ||
value: /etc/livecycle-ssh/ssh_host_key | ||
- name: BASE_URL | ||
valueFrom: | ||
configMapKeyRef: | ||
name: tunnel-server-config | ||
key: BASE_URL | ||
- name: DEBUG | ||
value: "1" | ||
- name: NODE_ENV | ||
value: production | ||
image: ghcr.io/livecycle/preevy/tunnel-server:main | ||
imagePullPolicy: IfNotPresent | ||
name: tunnel-server | ||
ports: | ||
- containerPort: 8888 | ||
name: metrics | ||
protocol: TCP | ||
- containerPort: 2222 | ||
name: ssh | ||
protocol: TCP | ||
- containerPort: 3000 | ||
name: http | ||
protocol: TCP | ||
- containerPort: 8443 | ||
name: tls | ||
protocol: TCP | ||
resources: | ||
limits: | ||
cpu: "1" | ||
ephemeral-storage: 1Gi | ||
memory: 2Gi | ||
requests: | ||
cpu: "1" | ||
ephemeral-storage: 1Gi | ||
memory: 2Gi | ||
securityContext: | ||
capabilities: | ||
drop: | ||
- NET_RAW | ||
volumeMounts: | ||
- mountPath: /etc/livecycle-ssh | ||
name: ssh | ||
readOnly: true | ||
- mountPath: /app/tls/cert.pem | ||
name: tls-cert | ||
readOnly: true | ||
subPath: tls.crt | ||
- mountPath: /app/tls/key.pem | ||
name: tls-cert | ||
readOnly: true | ||
subPath: tls.key | ||
--- | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: tunnel-server | ||
spec: | ||
type: LoadBalancer | ||
ports: | ||
- name: tls | ||
port: 443 | ||
protocol: TCP | ||
targetPort: 8443 | ||
- name: metrics | ||
port: 8888 | ||
protocol: TCP | ||
targetPort: 8888 | ||
selector: | ||
app: tunnel-server |