Deployment

Introduction

If you are serving up to 40-50 students simultaneously logged in, I think that deploying FROG to a single server should be sufficient. We still need a better understanding of performance limits, and the code could probably be made a lot more efficient.

To serve 250+ students over https, this is how we currently do it.

Architecture

We use Digital Ocean droplets, as well as two CHILI servers for Nginx load-balancing and SSL reverse proxying, because Let's Encrypt requires domain names, which we don't get from Digital Ocean. In the future, we want to investigate how to do the setup automatically, and possibly use Kubernetes.

Our infrastructure

• icchilisrv4: Will load-balance and SSL terminate the Meteor servers
• icchilisrv3: Will load-balance and SSL terminate the ShareDB servers
• icchilisrv1: SSL terminates file upload servers (if needed)
• frog-head: DO 1GB, runs Meteor, and Mongo for Meteor. Used by teacher
• frog1-3: DO 512MB, additional Meteor servers
• sharedb-head: DO 1GB, runs ShareDB, Mongo for ShareDB, Redis for ShareDB
• sharedb1-3: DO 512MB, additional ShareDB servers
• minio1-2: Optional file upload servers

Steps

Creating droplets

• create all the DO clusters listed above, selecting any SSH keys needed, and Monitoring. Use most recent Ubuntu (16.04) image, location: Frankfurt
• a good idea is to add the servers to /etc/hosts, so make it easier to do ssh sharedb-head etc

Install packages, generate keys

• ssh into all of the new servers in different iTerm panes. Note that if you have previously used frog1 etc for different servers, you might have to remove these from ~/.ssh/known_hosts
• turn on send input to all servers
• run apt get and apt upgrade
• apt install autossh
• ssh-keygen -t rsa -b 4096 (no password etc)
• cat ~/.ssh/id_rsa.pub
• now you need to copy all the SSH keys from frog* and sharedb1-3 to sharedb-head, and from frog1-3 to frog-head - they should be added to ~/.ssh/authorized_keys, this will allow SSH login and autossh for port forwarding
• install mongodb on sharedb-head and frog-head, and redis-server on sharedb-head
• change port for mongodb on sharedb-head to 27300 (edit /etc/mongodb.conf, uncomment and set port, then service mongodb restart

Set up autossh links

• send input to all servers, then tmux
• find the ip of frog-head and sharedb-head, I'm going to use frog-head and sharedb-head below, but it must be replaced by the actual IPs
• for frog1-3: /usr/lib/autossh/autossh -M 0 -N -q -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -p 22 root@sharedb -L 27017:127.0.0.1:27017 -L 6379:127.0.0.1:6379