Skip to content

Commit

Permalink
Update documentation regarding containerlab migration.
Browse files Browse the repository at this point in the history
  • Loading branch information
Gerrit91 committed Nov 15, 2021
1 parent f596728 commit 4a6b590
Show file tree
Hide file tree
Showing 3 changed files with 44 additions and 66 deletions.
108 changes: 43 additions & 65 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,70 +1,61 @@
# mini-lab

The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development.
The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a metal-stack partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development.

![overview components](docs/overview.png)

This project can also be used as a template for writing your own metal-stack deployments.
This project can also be used as a template for writing your own metal-stack deployments.

<!-- TOC depthFrom:2 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 -->
<!-- TOC depthfrom:2 depthto:6 withlinks:true updateonsave:false orderedlist:false -->

- [mini-lab](#mini-lab)
- [Requirements](#requirements)
- [Known Limitations](#known-limitations)
- [Try it out](#try-it-out)
- [Requirements](#requirements)
- [Known Limitations](#known-limitations)
- [Try it out](#try-it-out)
- [Reinstall machine](#reinstall-machine)
- [Remove machine](#remove-machine)
- [Development of metal-api, metal-hammer and metal-core](#development-of-metal-api-metal-hammer-and-metal-core)
- [Free machine](#free-machine)
- [Flavors](#flavors)

<!-- /TOC -->

## Requirements

- Linux machine with hardware virtualization support
- [Vagrant](https://www.vagrantup.com/) == 2.2.14 with vagrant-libvirt plugin >= 0.3.0 (for running the switch and machine VMs)
- kvm as hypervisor for the VMs
- Ubuntu 20.04:

sudo apt update
sudo apt install -y qemu qemu-kvm libvirt-daemon bridge-utils virtinst libvirt-dev git curl build-essential wget

- kvm as hypervisor for the VMs (you can check through the `kvm-ok` command)
- [docker](https://www.docker.com/) >= 18.09 (for using kind and our deployment base image)
- [docker-compose](https://docs.docker.com/compose/) >= 1.25.4 (for ease of use and for parallelizing control plane and partition deployment)
- [kind](https://github.com/kubernetes-sigs/kind/releases) == v0.9.0 (for hosting the metal control plane on a kubernetes cluster v1.19.1)
- [ovmf](https://wiki.ubuntu.com/UEFI/OVMF) to have a uefi firmware for virtual machines
- the lab creates a virtual network 192.168.121.0/24 on your host machine, this hopefully does not overlap with other networks you have
- [containerlab](https://containerlab.srlinux.dev/install/)
- the lab creates a docker network on your host machine (`172.17.0.1`), this hopefully does not overlap with other networks you have
- (recommended) haveged to have enough random entropy (only needed if the PXE process does not work)

Here is some code that should help you setting up most of the requirements:

```bash
# Install kvm
sudo apt install -y git curl qemu qemu-kvm havaged

# Install Docker
curl -fsSL https://get.docker.com | sh
# if you want to be on the safe side, follow the original installation
# instructions at https://docs.docker.com/engine/install/ubuntu/

# Install vagrant and other stuff
wget https://releases.hashicorp.com/vagrant/2.2.14/vagrant_2.2.14_x86_64.deb
sudo apt-get install ./vagrant_2.2.14_x86_64.deb qemu-kvm virt-manager ovmf net-tools libvirt-dev haveged

# Ensure that your user is member of the group "libvirt" and "docker"
# Ensure that your user is member of the group "docker"
# you need to login again in order to make this change take effect
sudo usermod -G libvirt,docker -a ${USER}
sudo usermod -G docker -a ${USER}

# Install libvirt plugin for vagrant
vagrant plugin install vagrant-libvirt
# Install containerlab
bash -c "$(curl -sL https://get-clab.srlinux.dev)"

# Install kind (kubernetes in docker), for more details see https://kind.sigs.k8s.io/docs/user/quick-start/#installation
curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
sudo curl -Lo /usr/local/bin/kind "https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64"
sudo chmod +x /usr/local/bin/kind

# Install docker-compose, for more details see https://docs.docker.com/compose/install/
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo curl -Lo /usr/local/bin/docker-compose "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)"
sudo chmod +x /usr/local/bin/docker-compose
```

The following ports are getting used statically:
The following ports are used statically on your host machine:

| Port | Bind Address | Description |
|:----:|:------------ |:---------------------------------- |
Expand All @@ -82,15 +73,18 @@ The following ports are getting used statically:

## Try it out

Start the mini-lab with a kind cluster, a metal-api instance as well as some vagrant VMs with two leaf switches and two machine skeletons.

```bash
git clone https://github.com/metal-stack/mini-lab.git
cd mini-lab
```

Start the mini-lab with a kind cluster, a metal-api instance as well as two containers wrapping leaf switches and another container that hosts two user-allocatable machines:

```bash
make
```

Two machines in status `PXE booting` are visible with `metalctl machine ls`
After the deployment and waiting for a short amoung of time, two machines in status `PXE booting` become visible through `metalctl machine ls`:

```bash
docker-compose run metalctl machine ls
Expand All @@ -100,7 +94,7 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   PXE Booting 3s
2294c949-88f6-5390-8154-fa53d93a3313 PXE Booting 5s
```

Wait until the machines reach the waiting state
Wait until the machines reach the waiting state:

```bash
docker-compose run metalctl machine ls
Expand All @@ -110,42 +104,42 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   Waiting 8s
2294c949-88f6-5390-8154-fa53d93a3313   Waiting 8s v1-small-x86 vagrant
```

Create a machine/firewall with
Create a firewall and a machine with:

```bash
make firewall
make machine
```

__Alternatively__ you may want to issue the `metalctl` commands by your own:
__Alternatively__, you may want to issue the `metalctl` commands on your own:

```bash
docker-compose run metalctl network allocate \
--partition vagrant \
--partition mini-lab \
--project 00000000-0000-0000-0000-000000000000 \
--name vagrant
--name user-private-network

# Lookup the network ID and create a machine
# lookup the network ID and create a machine
docker-compose run metalctl machine create \
--description test \
--name machine \
--hostname machine \
--project 00000000-0000-0000-0000-000000000000 \
--partition vagrant \
--partition mini-lab \
--image ubuntu-20.04 \
--size v1-small-x86 \
--networks <network-ID>

# Create a firewall that is also connected to the virtual internet-vagrant-lab network
# create a firewall that is also connected to the virtual internet-mini-lab network
docker-compose run metalctl machine create \
--description fw \
--name fw \
--hostname fw \
--project 00000000-0000-0000-0000-000000000000 \
--partition vagrant \
--partition mini-lab \
--image firewall-ubuntu-2.0 \
--size v1-small-x86 \
--networks internet-vagrant-lab,$(privatenet)
--networks internet-mini-lab,$(privatenet)
```

See the installation process in action
Expand All @@ -170,12 +164,10 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258   Phoned Home 2s 21s machin
Login with user name metal and the console password from

```bash
docker-compose run metalctl machine describe e0ab02d2-27cd-5a5e-8efc-080ba80cf258 | grep password

consolepassword: ...
docker-compose run metalctl machine consolepassword e0ab02d2-27cd-5a5e-8efc-080ba80cf258
```

If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the vagrant boxes of the leaf switches:
If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the leaf switches:

```bash
# Add the route to the network internet-vagrant-lab 100.255.254.0/24 via leaf01 and leaf02, whose IPs are dynamically allocated. Make sure there's no old route before execution.
Expand All @@ -185,7 +177,7 @@ make route
ssh [email protected]
```

To remove the kind cluster and the vagrant boxes, run
To remove the kind cluster and VMs, run:

```bash
make cleanup
Expand All @@ -201,9 +193,9 @@ docker-compose run metalctl machine reinstall \
e0ab02d2-27cd-5a5e-8efc-080ba80cf258
```

### Remove machine
### Free machine

Remove a machine with
Free a machine with

```bash
docker-compose run metalctl machine rm e0ab02d2-27cd-5a5e-8efc-080ba80cf258
Expand All @@ -222,17 +214,3 @@ In order to start specific flavor, you can define the flavor as follows:
export MINI_LAB_FLAVOR=cluster-api
make
```

## Development of metal-api, metal-hammer and metal-core

To simplify developing changes for the `metal-api`, `metal-hammer` and `metal-core`, it is possible to use development artifacts from within the mini-lab.
See the [dev instructions](DEV_INSTRUCTIONS.md) for more details.

FROM ubuntu:20.04

RUN apt update -y && \
DEBIAN_FRONTEND=noninteractive apt install -y virt-manager ovmf net-tools haveged \
qemu qemu-kvm bridge-utils virtinst libvirt-dev libvirt-daemon-system build-essential libvirt-clients

CMD ["/usr/sbin/init"]

2 changes: 1 addition & 1 deletion docs/overview.drawio

Large diffs are not rendered by default.

Binary file modified docs/overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 4a6b590

Please sign in to comment.