Skip to content

Deploying Estuary on a cloud virtual machine

Jump to bottom Edit New page
Carl Testa edited this page Dec 24, 2021 · 2 revisions

This wiki page shows the basic steps to build Estuary on a cloud virtual machine. For this wiki tutorial, I will be using Digital Ocean (https://www.digitalocean.com/) and Ubuntu 18.04.

Why building Estuary on a cloud virtual machine?

Estuary is always available at estuary.mcmaster.ca, but there could be some circumstances where having Estuary deployed on a cloud server could be useful. For example:

  1. If you are making last-minute tweaks to your forked version of Estuary and you're unable to wait for your pull request to be integrated into the Estuary's main repository. This has happened to me, especially when teaching workshops where people were developing their own languages.
  2. If you are developing an installation with Estuary and want to have full control over the aesthetics of the interface and its resources.

Why use a paid cloud service?

Digital Ocean is a cloud service provider that allows you to use a virtual computer with a Linux-based OS. Although using a private service might not go with the Free/Libre Open Source "spirit" of Estuary, I found it useful when you're unsure about setting up a free home server. Cloud servers also allow you to increase/decrease the virtual computer memory, thus supporting more people -than a home server- to get connected. I usually pay around between $10-15 per month (also, Digital Ocean charges you by the hour so you can use it for a day or less without being charged for all the month).

Setting up the virtual computer

  1. Make an account in Digital Ocean and log in.
  2. Once you're logged in go to the left menu and click Projects -> New Project.
  3. You'll be taken to a "Create project" screen where you have to fill the name, description, and the purpose of your project.
  1. After clicking "Create project", you'll be taken to the "Move sources" screen, just click "skip for now".
  2. Now you'll be asked to build a "droplet", that is a Linux-based virtual computer. Click "Get started with a Droplet".
  3. Here you have to select the virtual computer's characteristics. Select Ubuntu for the OS and the $10 "standard" plan (Selecting a lower cost plan will allow you to increase the memory anytime and decrease it back to the one provided by the $10 plan).
  1. If you scroll down you'll see more customization options but for now, just go to the bottom of the page and click "Create droplet". Here is a link that describes the droplet customization in detail: https://www.digitalocean.com/docs/droplets/how-to/create/

Setting up the root user

  1. Click on the three dots to the right of your newly created droplet.
  2. Click "Access console" from the dropdown menu.
  1. Once in the console, type "root" to log in, then copy the default password sent to your email by Digital Ocean and paste it onto the terminal. Customize your password when you're be prompted to change the password.

Setting up a sudo user

Estuary requires you to install Nix (https://nixos.org/nix/). You'll need to create a sudo user to install Nix as installing it as "root" is not supported by the Nix's default script. Do the following:

  1. In the console type adduser and a username right after. Like this:
  1. Set up your new user’s password and follow the prompts to set the user’s information.
  2. Type usermod -aG sudo and the new username to add the user to the sudo group (this will give you administrator access to install Nix and Estuary).

Installing Nix

Still in the droplet's console do the following:

  1. Switch from root to your user by typing su - and your username.
  1. Install Nix by typing curl -L https://nixos.org/nix/install | sh
  2. Reboot the virtual computer by typing sudo reboot to update the path to Nix.
  3. Once it is rebooted switch back to your user and type which nix to confirm that Nix is in the virtual computer's path. If the console returns a path to Nix, you're good to go!

Building Estuary

  1. Clone your forked version of Estuary into your virtual machine.

  1. Install make by typing sudo apt install make.

  2. There are different ways of building Estuary (https://github.com/dktr0/estuary/blob/master/BUILDING.md), but I usually do it the following way (type them once at a time):
    cd estuary
    make fullBuild
    make selfCertificates
    make runServer

hello

Add a custom sidebar
Clone this wiki locally