-
Notifications
You must be signed in to change notification settings - Fork 12
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update documentation regarding containerlab migration.
- Loading branch information
Showing
3 changed files
with
44 additions
and
66 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,70 +1,61 @@ | ||
# mini-lab | ||
|
||
The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development. | ||
The mini-lab is a small, virtual setup to locally run the metal-stack. It deploys the metal control plane and a metal-stack partition with two simulated leaf switches. The lab can be used for trying out metal-stack, demonstration purposes or development. | ||
|
||
![overview components](docs/overview.png) | ||
|
||
This project can also be used as a template for writing your own metal-stack deployments. | ||
ℹ This project can also be used as a template for writing your own metal-stack deployments. | ||
|
||
<!-- TOC depthFrom:2 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 --> | ||
<!-- TOC depthfrom:2 depthto:6 withlinks:true updateonsave:false orderedlist:false --> | ||
|
||
- [mini-lab](#mini-lab) | ||
- [Requirements](#requirements) | ||
- [Known Limitations](#known-limitations) | ||
- [Try it out](#try-it-out) | ||
- [Requirements](#requirements) | ||
- [Known Limitations](#known-limitations) | ||
- [Try it out](#try-it-out) | ||
- [Reinstall machine](#reinstall-machine) | ||
- [Remove machine](#remove-machine) | ||
- [Development of metal-api, metal-hammer and metal-core](#development-of-metal-api-metal-hammer-and-metal-core) | ||
- [Free machine](#free-machine) | ||
- [Flavors](#flavors) | ||
|
||
<!-- /TOC --> | ||
|
||
## Requirements | ||
|
||
- Linux machine with hardware virtualization support | ||
- [Vagrant](https://www.vagrantup.com/) == 2.2.14 with vagrant-libvirt plugin >= 0.3.0 (for running the switch and machine VMs) | ||
- kvm as hypervisor for the VMs | ||
- Ubuntu 20.04: | ||
|
||
sudo apt update | ||
sudo apt install -y qemu qemu-kvm libvirt-daemon bridge-utils virtinst libvirt-dev git curl build-essential wget | ||
|
||
- kvm as hypervisor for the VMs (you can check through the `kvm-ok` command) | ||
- [docker](https://www.docker.com/) >= 18.09 (for using kind and our deployment base image) | ||
- [docker-compose](https://docs.docker.com/compose/) >= 1.25.4 (for ease of use and for parallelizing control plane and partition deployment) | ||
- [kind](https://github.com/kubernetes-sigs/kind/releases) == v0.9.0 (for hosting the metal control plane on a kubernetes cluster v1.19.1) | ||
- [ovmf](https://wiki.ubuntu.com/UEFI/OVMF) to have a uefi firmware for virtual machines | ||
- the lab creates a virtual network 192.168.121.0/24 on your host machine, this hopefully does not overlap with other networks you have | ||
- [containerlab](https://containerlab.srlinux.dev/install/) | ||
- the lab creates a docker network on your host machine (`172.17.0.1`), this hopefully does not overlap with other networks you have | ||
- (recommended) haveged to have enough random entropy (only needed if the PXE process does not work) | ||
|
||
Here is some code that should help you setting up most of the requirements: | ||
|
||
```bash | ||
# Install kvm | ||
sudo apt install -y git curl qemu qemu-kvm havaged | ||
|
||
# Install Docker | ||
curl -fsSL https://get.docker.com | sh | ||
# if you want to be on the safe side, follow the original installation | ||
# instructions at https://docs.docker.com/engine/install/ubuntu/ | ||
|
||
# Install vagrant and other stuff | ||
wget https://releases.hashicorp.com/vagrant/2.2.14/vagrant_2.2.14_x86_64.deb | ||
sudo apt-get install ./vagrant_2.2.14_x86_64.deb qemu-kvm virt-manager ovmf net-tools libvirt-dev haveged | ||
|
||
# Ensure that your user is member of the group "libvirt" and "docker" | ||
# Ensure that your user is member of the group "docker" | ||
# you need to login again in order to make this change take effect | ||
sudo usermod -G libvirt,docker -a ${USER} | ||
sudo usermod -G docker -a ${USER} | ||
|
||
# Install libvirt plugin for vagrant | ||
vagrant plugin install vagrant-libvirt | ||
# Install containerlab | ||
bash -c "$(curl -sL https://get-clab.srlinux.dev)" | ||
|
||
# Install kind (kubernetes in docker), for more details see https://kind.sigs.k8s.io/docs/user/quick-start/#installation | ||
curl -Lo ./kind https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64 | ||
chmod +x ./kind | ||
sudo mv ./kind /usr/local/bin/kind | ||
sudo curl -Lo /usr/local/bin/kind "https://kind.sigs.k8s.io/dl/latest/kind-linux-amd64" | ||
sudo chmod +x /usr/local/bin/kind | ||
|
||
# Install docker-compose, for more details see https://docs.docker.com/compose/install/ | ||
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose | ||
sudo curl -Lo /usr/local/bin/docker-compose "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" | ||
sudo chmod +x /usr/local/bin/docker-compose | ||
``` | ||
|
||
The following ports are getting used statically: | ||
The following ports are used statically on your host machine: | ||
|
||
| Port | Bind Address | Description | | ||
|:----:|:------------ |:---------------------------------- | | ||
|
@@ -82,15 +73,18 @@ The following ports are getting used statically: | |
|
||
## Try it out | ||
|
||
Start the mini-lab with a kind cluster, a metal-api instance as well as some vagrant VMs with two leaf switches and two machine skeletons. | ||
|
||
```bash | ||
git clone https://github.com/metal-stack/mini-lab.git | ||
cd mini-lab | ||
``` | ||
|
||
Start the mini-lab with a kind cluster, a metal-api instance as well as two containers wrapping leaf switches and another container that hosts two user-allocatable machines: | ||
|
||
```bash | ||
make | ||
``` | ||
|
||
Two machines in status `PXE booting` are visible with `metalctl machine ls` | ||
After the deployment and waiting for a short amoung of time, two machines in status `PXE booting` become visible through `metalctl machine ls`: | ||
|
||
```bash | ||
docker-compose run metalctl machine ls | ||
|
@@ -100,7 +94,7 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258 PXE Booting 3s | |
2294c949-88f6-5390-8154-fa53d93a3313 PXE Booting 5s | ||
``` | ||
|
||
Wait until the machines reach the waiting state | ||
Wait until the machines reach the waiting state: | ||
|
||
```bash | ||
docker-compose run metalctl machine ls | ||
|
@@ -110,42 +104,42 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258 Waiting 8s | |
2294c949-88f6-5390-8154-fa53d93a3313 Waiting 8s v1-small-x86 vagrant | ||
``` | ||
|
||
Create a machine/firewall with | ||
Create a firewall and a machine with: | ||
|
||
```bash | ||
make firewall | ||
make machine | ||
``` | ||
|
||
__Alternatively__ you may want to issue the `metalctl` commands by your own: | ||
__Alternatively__, you may want to issue the `metalctl` commands on your own: | ||
|
||
```bash | ||
docker-compose run metalctl network allocate \ | ||
--partition vagrant \ | ||
--partition mini-lab \ | ||
--project 00000000-0000-0000-0000-000000000000 \ | ||
--name vagrant | ||
--name user-private-network | ||
|
||
# Lookup the network ID and create a machine | ||
# lookup the network ID and create a machine | ||
docker-compose run metalctl machine create \ | ||
--description test \ | ||
--name machine \ | ||
--hostname machine \ | ||
--project 00000000-0000-0000-0000-000000000000 \ | ||
--partition vagrant \ | ||
--partition mini-lab \ | ||
--image ubuntu-20.04 \ | ||
--size v1-small-x86 \ | ||
--networks <network-ID> | ||
|
||
# Create a firewall that is also connected to the virtual internet-vagrant-lab network | ||
# create a firewall that is also connected to the virtual internet-mini-lab network | ||
docker-compose run metalctl machine create \ | ||
--description fw \ | ||
--name fw \ | ||
--hostname fw \ | ||
--project 00000000-0000-0000-0000-000000000000 \ | ||
--partition vagrant \ | ||
--partition mini-lab \ | ||
--image firewall-ubuntu-2.0 \ | ||
--size v1-small-x86 \ | ||
--networks internet-vagrant-lab,$(privatenet) | ||
--networks internet-mini-lab,$(privatenet) | ||
``` | ||
|
||
See the installation process in action | ||
|
@@ -170,12 +164,10 @@ e0ab02d2-27cd-5a5e-8efc-080ba80cf258 Phoned Home 2s 21s machin | |
Login with user name metal and the console password from | ||
|
||
```bash | ||
docker-compose run metalctl machine describe e0ab02d2-27cd-5a5e-8efc-080ba80cf258 | grep password | ||
|
||
consolepassword: ... | ||
docker-compose run metalctl machine consolepassword e0ab02d2-27cd-5a5e-8efc-080ba80cf258 | ||
``` | ||
|
||
If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the vagrant boxes of the leaf switches: | ||
If you want to access the firewall with SSH or have internet connectivity from the firewall and machine, you'll need to have a static route configured that points to the leaf switches: | ||
|
||
```bash | ||
# Add the route to the network internet-vagrant-lab 100.255.254.0/24 via leaf01 and leaf02, whose IPs are dynamically allocated. Make sure there's no old route before execution. | ||
|
@@ -185,7 +177,7 @@ make route | |
ssh [email protected] | ||
``` | ||
|
||
To remove the kind cluster and the vagrant boxes, run | ||
To remove the kind cluster, the switches and machines, run: | ||
|
||
```bash | ||
make cleanup | ||
|
@@ -201,9 +193,9 @@ docker-compose run metalctl machine reinstall \ | |
e0ab02d2-27cd-5a5e-8efc-080ba80cf258 | ||
``` | ||
|
||
### Remove machine | ||
### Free machine | ||
|
||
Remove a machine with | ||
Free a machine with | ||
|
||
```bash | ||
docker-compose run metalctl machine rm e0ab02d2-27cd-5a5e-8efc-080ba80cf258 | ||
|
@@ -222,17 +214,3 @@ In order to start specific flavor, you can define the flavor as follows: | |
export MINI_LAB_FLAVOR=cluster-api | ||
make | ||
``` | ||
|
||
## Development of metal-api, metal-hammer and metal-core | ||
|
||
To simplify developing changes for the `metal-api`, `metal-hammer` and `metal-core`, it is possible to use development artifacts from within the mini-lab. | ||
See the [dev instructions](DEV_INSTRUCTIONS.md) for more details. | ||
|
||
FROM ubuntu:20.04 | ||
|
||
RUN apt update -y && \ | ||
DEBIAN_FRONTEND=noninteractive apt install -y virt-manager ovmf net-tools haveged \ | ||
qemu qemu-kvm bridge-utils virtinst libvirt-dev libvirt-daemon-system build-essential libvirt-clients | ||
|
||
CMD ["/usr/sbin/init"] | ||
|
Large diffs are not rendered by default.
Oops, something went wrong.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.