Skip to content

Commit

Permalink
Merge pull request #422 from bottlerocket-os/main
Browse files Browse the repository at this point in the history
Move to prod: Brupop documentation
  • Loading branch information
stockholmux authored Feb 14, 2024
2 parents 07dcf13 + f1e01b0 commit 9adf06f
Show file tree
Hide file tree
Showing 32 changed files with 1,341 additions and 9 deletions.
136 changes: 134 additions & 2 deletions assets/scss/_styles_project.scss
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
@import '_home_svg.scss';

.btn-lg, .btn-group-lg > .btn {
border-radius: 6px;
}
Expand Down Expand Up @@ -437,7 +436,7 @@ nav.foldable-nav .with-child, nav.foldable-nav .without-child {
stroke: none;
}


path.exit-flag {
fill: $dark-blue;
}
Expand Down Expand Up @@ -508,6 +507,139 @@ nav.foldable-nav .with-child.depad {
padding-left: 0;
}

.start-align-labels.brupop-diagram {
.outer-label {
&.active-volume-label,
&.outer-label {
text-anchor: start;
}
}
}

.brupop-diagram,
.brupop-state-machine {
.line-arrow-connector {
stroke-miterlimit: 10;
stroke: $dark-blue;
.connector {

fill: none;
pointer-events : stroke;
&.dotted {
stroke-dasharray: 1 2;
}
}
.arrow-head {
fill: $dark-blue;
}

}

.label {
font-size: 12px;
font-family: $td-fonts-serif;
font-weight: 300;

&.active-volume-label {
fill: $white;
text-anchor: middle;
}
&.outer-label {
fill: $dark-blue;
text-anchor: middle;
}

}
}
.brupop-state-machine {
.state {
fill: $tan;
rx: 9;
ry: 9;
pointer-events: all;
}
}

.brupop-diagram {
.node,
.agent,
.api-server,
.controller,
.unused-container,
.unused-volume,
.active-volume,
.line-arrow-connector .arrow-head,
.ellipses,
.future-volume,
.label-backer,
.wait {
pointer-events: all;
}
.node {
fill: $tan;
rx: 3;
}
.agent,
.api-server,
.controller,
.unused-container,
.unused-volume,
.active-volume,
.future-volume {
rx: 2;
}
.agent {
fill: $light-teal;
}

.label-backer,
.unused-container,
.unused-volume {
fill: $white;
}


.active-volume {
fill: $dark-blue;
}
.future-volume {
fill: url(#stripes);
}

.wait {
stroke-width: 2px;
stroke: $light-blue;
fill: none;
}

.api-server {
fill: $dark-orange;
}
.controller {
fill: $light-blue;
}
.ellipses {
fill: $tan;
stroke: none;
rx: 5;
ry: 5;
}

#stripes {
width: 7;
height: 7;
rect {
fill: $dark-blue;
}
line {
stroke: #ffffff;
opacity: 0.1;
stroke-width: 7px;
}
}

}

/* old docs notice */
.pageinfo.olddocs {
margin-left: 0;
Expand Down
10 changes: 9 additions & 1 deletion assets/scss/_variables_project.scss
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@

$google_font_name: "IBM Plex Sans";
$google_font_family: "IBM+Plex+Sans+Condensed:ital,wght@0,300;0,600;1,300;1,600&family=IBM+Plex+Sans:ital,wght@0,100;0,300;0,600;1,100;1,300;1,600";
$google_font_family: "IBM+Plex+Sans:ital,wght@0,100;0,300;0,600;1,100;1,300;1,600";

// this is a work around for the baked in css2 vs css call to google fonts. I don't like having two calls, but there isn't a clean way to work around this
$google_font_family_secondary: "IBM+Plex+Sans+Condensed:ital,wght@0,300;0,600;1,300;1,600";
$web-font-path_secondary: "https://fonts.googleapis.com/css?family=#{$google_font_family_secondary}&display=swap";
@import url($web-font-path_secondary);



$heading_font_stack: "'IBM Plex Sans Condensed', sans-serif";

Expand Down
1 change: 1 addition & 0 deletions assets/scss/rtl/_main.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
// override. This is for RTL support, we don't need it.
4 changes: 4 additions & 0 deletions content/en/brupop/1.3.x/_index.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
+++
type="docs"
title="1.3.x (Current)"
+++
91 changes: 91 additions & 0 deletions content/en/brupop/1.3.x/concepts/index.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
+++
title = "Concepts"
type = "docs"
description = "Introduction to the components and concepts used in Brupop"
weight = 1
+++

You can update Bottlerocket in a couple of ways:

* **node replacement** where new instances with a new version of the OS replace nodes with older versions of the OS,
* **in-place updates** where the node downloads and reboots into a new version of the OS while maintaining the same instance/machine.

There is no single preferred nor advised method to update a node; both methods have pros and cons depending on your situation.

You can trigger an {{< cross-project-current-link project="os" url="/en/os/x.x.x/update/methods/in-place/#apiclient-commands">}}in-place update manually with the API{{< /cross-project-current-link >}} or you can use the Bottlerocket Update Operator (Brupop).
**Brupop is a Kubernetes operator for managing in-place updates of Bottlerocket on Kubernetes.**

If you use Bottlerocket on ECS or intend to replace nodes in Kubernetes, Brupop is not for you.
Even if you do plan to do in-place updates Brupop is not required as you can manage in-place updates in other ways.
However, Brupop offers a declarative, automated way to manage in-place Bottlerocket updates.

## Controlled updates

Brupop uses the [Kubernetes controller pattern](https://kubernetes.io/docs/concepts/architecture/controller/) in an effort to safely update all the nodes whilst minimizing disruptions to workloads.
To achieve this, Brupop does the following:

* Controls the rate and flow of updates across the entire cluster,
* First prevents new workloads from being scheduled to the node then drains existing workloads prior to updates,
* Contains and prevents the propagation of update problems when the controller detects update failures.

{{< brupop/components-diagram >}}

Brupop collects the state of each node with an agent.
The Brupop Agent runs in a container on each node as a [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/).
This agent sends the state to an API Server.
API Server instances run in the cluster itself and communicates with the Kubernetes API to record the state as a custom resource.

{{< brupop/agent-api-server-control-plane >}}

{{< alert title="Bottlerocket API Server vs Brupop API Server?" color="success" >}}
Don’t confuse Bottlerocket’s {{< cross-project-current-link project="os" url="/en/os/x.x.x/concepts/api-driven/">}}API Server{{< /cross-project-current-link >}} with Brupop’s API Server, these are two distinct things, just with the same name.
In this part of the documentation, unless otherwise noted, assume that “API Server” refers to the Brupop API Server.
{{< /alert >}}

The Controller also runs in a container on the cluster where it regularly evaluates the information about the state of each node and the cluster as a whole; based on this information it supplies instructions to the individual agents about update actions.

{{< brupop/agent-controller-diagram >}}

## States

At any given point nodes are in one of five Brupop states: **idle**, **staged & performed update**, **rebooted into update**, **monitoring update** or **error reset**.
A node is never in more than one state.
The state of each node is represented as a [Kubernetes Custom Resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) called a `BottlerocketShadow` resource or `brs`.

{{< brupop/state-machine >}}

### Idle

A node in the **idle** state does not have a pending update in-process.
Most of the time your nodes will remain in this state.

### Staged & Performed Update

{{< brupop/staged-and-performed >}}

Bottlerocket uses multiple partitions to manage in-place updates.
The OS runs from one partition and, when a new update is available, the update is downloaded and installed into the other.
The Brupop controller periodically requests the agent to check for and download the most recent version of Bottlerocket.
Once downloaded, Bottlerocket modifies the bootloader configuration to boot from the partition with the update and the agent changes the state to **Staged & Performed Update** with the Brupop API server.

### Reboot into Update

{{< brupop/reboot-into-update >}}

To minimize disruptions to the workloads running in the cluster, the controller signals to Kubernetes to prevent new workloads from being scheduled on to the node as well as shut down existing workloads (drain).
Once drained, the agent triggers a reboot into the new OS and changes the state to **Rebooted Into Update** with the Brupop API server.

### Monitoring Update

{{< brupop/monitoring >}}

Once the node reboots the update is technically complete, however the time whilst all your workloads startup is critical.
Bottlerocket’s versioning and variant scheme is built to mitigate incompatibilities between OS versions, there is always a chance that an unforeseen incompatibility exists with some component of your architecture.
Brupop’s state machine has a reserved state for monitoring these incompatibilities (**Monitoring Updates**), however as of this version, this state is a noop.
You can suggest a direction for this state on the [Brupop GitHub Repo](https://github.com/bottlerocket-os/bottlerocket-update-operator/issues/new?assignees=&labels=&projects=&template=issue.md&title=Suggestion%20for%20monitoring%20state).

Consequently, the Agent immediately transitions through **Monitoring Updates** back to **Idle** with the API server.

### Error Reset

In the situation that any of the above states fail, the state becomes **Error Reset** before transitioning back to **Idle**.
35 changes: 35 additions & 0 deletions content/en/brupop/1.3.x/operate/index.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
+++
type = "docs"
title = "Operate & Observe"
weight = 10
description = "Understanding the day-to-day use of Brupop"
+++

After installation on your cluster Brupop runs in the background and generally requires no intervention.
Your nodes will check for updates and apply them according to your configuration and the Bottlerocket update waves.

However, you can observe the status of the updates by [adhoc query](#adhoc-query) or setup [on-going monitoring](#on-going-monitoring).

## Adhoc Query

If you want to see the update status of your nodes, use `kubectl` to get the custom resource `brs` :

```shell
kubectl get brs --namespace brupop-bottlerocket-aws
```

`kubectl` returns the [state](../concepts/#states), current version, target state, and target version.
For example:

```shell
AME STATE VERSION TARGET STATE TARGET VERSION
brs-node-1 Idle 1.17.0 Idle
brs-node-2 Idle 1.17.0 StagedUpdate 1.18.0
```

## On-going monitoring

To facilitate on-going monitoring the Brupop API server and controller provides you with metrics endpoints (`/metrics`) compatible with [Prometheus](https://prometheus.io/).
The metrics endpoints expose two metrics: one that describes the current version of each node (`brupop_hosts_version`) and another for the [state](../concepts/#states) of each node (`brupop_hosts_state`).

For a sample configuration of using Prometheus with Brupop see the {{< github-link-at-version url="https://github.com/bottlerocket-os/bottlerocket-update-operator/blob/vx.x.x/deploy/examples/prometheus-resources.yaml" project="brupop" >}}configuration on the Brupop GitHub Repo{{</ github-link-at-version >}}.
14 changes: 14 additions & 0 deletions content/en/brupop/1.3.x/setup/_index.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
+++
type = "docs"
title = "Setup"
weight = 5
description = "Steps to use and configure Brupop on your Bottlerocket nodes"
+++

Setting up Brupop for the first time has three major steps:

- Installing the prerequisite, `cert-manager` on your cluster,
- Installing Brupop itself,
- Labeling the nodes you want to update with Brupop.

Many clusters require nothing more than the three above steps, but familiarize yourself with the additional configuration options before installing as you may need to tweak the configuration for your particular needs.
50 changes: 50 additions & 0 deletions content/en/brupop/1.3.x/setup/cert-manager/index.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
+++
title = "Prerequisite: cert-manager"
type = "docs"
description = "Prepare your cluster for Brupop"
weight = 1
+++

Brupop uses [cert-manager](https://cert-manager.io/) to manage self-signed certificates.
You can install it with `kubectl` or [helm](https://helm.sh/).

{{% alert title="Note" color="success" %}}
This guide uses the most recent release of `cert-manager`, {{< brupop/cert-manager-version >}}, but there is no particular hard dependency on this version.
{{% /alert %}}

## Installing `cert-manager` using `kubectl`

Use `kubectl` to install cert-manager:

```shell
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v{{< brupop/cert-manager-version >}}/cert-manager.yaml
```

## Installing `cert-manager` using `helm`

First, add the `cert-manager` helm chart:

```shell
helm repo add jetstack https://charts.jetstack.io
```

Then update your local chart:

```shell
helm repo update
```

Finally, install `cert-manager` including its CRDs:

```shell
helm install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v{{< brupop/cert-manager-version >}} \
--set installCRDs=true
```

## Next step

After installing `cert-manager`, go ahead and [install Brupop itself](../install/).
Loading

0 comments on commit 9adf06f

Please sign in to comment.