Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: document node drain configmap PCP-3323 #3575

Merged
merged 6 commits into from
Aug 13, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 119 additions & 6 deletions docs/docs-content/clusters/cluster-management/os-patching.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,17 @@ cluster nodes.

The following options are available for OS patching:

- Patch OS on boot
- Enable scheduled patching
- Patch OS on-demand
- [Patch OS on boot](#patch-os-on-boot)
- [Enable scheduled patching](#enable-scheduled-patching)
- [Patch OS on-demand](#perform-on-demand-patching)

You can use a combination of these options to patch the operating system to the latest version. For example, you can
patch OS on boot, set up scheduled patching every month, and also perform on-demand patches at any given point in time.

Palette offers the option to provide custom node drain configuration, which gives you fine-grained control of drain
behaviour during updates. Refer to the [Configure OS Patching Drain Policy](#configure-os-patching-drain-policy) for
further information.

## Patch OS on Boot

During the cluster creation, while configuring the cluster, you can select **Patch OS on boot**. In this case, the
Expand Down Expand Up @@ -154,15 +158,15 @@ updates.
- For EKS clusters, you can trigger OS updates from Palette. This would request AWS to update cluster node groups to
the latest patch version.

### Prerequisite
### Prerequisites

- An active cluster in Palette.

### Instructions
### Enablement

1. Log in to [Palette](https://console.spectrocloud.com).

2. From the left **Main Menu**, click **Cluster**. Select the cluster you want to patch to view cluster details.
2. From the left **Main Menu**, click **Clusters**. Select the cluster you want to patch to view cluster details.

3. In the upper-right corner, click on **Settings** and select **On-Demand Update**. This will trigger an OS update
immediately if a newer version is available.
Expand All @@ -180,3 +184,112 @@ updates.
| ----------------------- | --------------------------------------- |
| Last Applied Patch Time | The date and time of the last OS patch. |
| Patched Version | The latest patched version. |

## Configure OS Patching Drain Policy
addetz marked this conversation as resolved.
Show resolved Hide resolved
addetz marked this conversation as resolved.
Show resolved Hide resolved

The node drain operation refers to the process of evicting all of your pods from a node before you perform node
maintenance. Typical node maintenance activities are hardware upgrades, kernel updates and OS patching. Palette provides
the ability to configure the drain policy during on-boot, scheduled, and on-demand OS patching. Users can specify a node
drain policy using a ConfigMap, which can be applied to deployed clusters using a cluster profile or through `kubectl`.

### Limitations

- This operation is not available for existing Kubernetes clusters that have been previously imported into Palette.
- This operation is not available for managed Kubernetes Services such as EKS and AKS.
- For EKS clusters, you can trigger OS updates from Palette. This would request AWS to update cluster node groups to
the latest patch version.

### Prerequisites

- An active cluster in Palette.

### Enablement

1. Log in to [Palette](https://console.spectrocloud.com).

2. From the left **Main Menu**, click **Profiles**. Then, click **Add Cluster Profile**. The cluster profile creation
wizard appears.

3. Fill in a name in the **Name** field. Optionally, add a description in the **Description** field. Select the
**Add-on** box for the **Type**. Click **Next**.

4. Click **Add Manifest**. The manifest editor appears.

5. Add a name in the **Layer Name** field. Then, click **New manifest**. Assign a name to the internal manifest and
click on the blue button. An empty editor displays on the right side of the screen.

6. Paste the snippet below into the empty editor. This snippet includes example values within a ConfigMap to demonstrate
how to control the drain policy of the cluster. The ConfigMap provides the following drain configuration parameters.

| **Parameter** | **Description** | **Value Type** | **Default Value** |
| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | ----------------- |
| `data.profiling` | Flag to indicate whether the profiling server should capture profiling data. If enabled, the port `8082` should be exposed to the profiling server. | `enable` / `disable` | `disable` |
| `data.feature.workloads` | Flag to indicate whether to push workload metadata to the Palette server. | `enable` /`disable` | `enable` |
| `data.drain.timeout` | The length of time, specified in nanoseconds, to wait for the drain operation to complete. There is no configured timeout if set to zero. | `duration` | `unset` |
| `data.drain.gracePeriod` | The period of time, specified in seconds, to wait for each pod to terminate gracefully. The value provided by the pods will be used if set to a negative value. | `int` | `-1` |
| `data.drain.deleteLocalData` | Flag to indicate whether to continue if one or more affected pods do not have specified volumes, resulting in the deletion of their local data. | `boolean` | `false` |
| `data.drain.ignoreDaemonSets` | Flag to indicate whether to ignore DaemonSet managed pods. | `boolean` | `false` |
| `data.drain.force` | Flag to indicate whether to continue if one or more affected pods do not have a declared controller. | `boolean` | `false` |
| `data.drain.disableEviction` | Flag to indicate whether to force drain to use delete operations. | `boolean` | `false` |
| `data.drain.skipWaitForDeleteTimeout` | The length of time, in seconds, to wait for pod deletion to complete. If the time elapses, the drain will continue. | `int` | `0` |

```yaml
addetz marked this conversation as resolved.
Show resolved Hide resolved
apiVersion: v1
kind: ConfigMap
metadata:
name: palette-agent-config
namespace: "cluster-{{ .spectro.system.cluster.uid }}"
data:
profiling: disable
feature.workloads: enable
drain: |
{
"timeout": 600000000,
"gracePeriod": 600,
"deleteLocalData": true,
"ignoreDaemonSets": true,
"force": true,
"disableEviction": false,
"skipWaitForDeleteTimeout": 600
}
```

:::info

You can also specify the drain policy ConfigMap directly on the cluster by using `kubectl`. Download the
[Kubeconfig](./kubeconfig.md) file to connect to the cluster and issue the
addetz marked this conversation as resolved.
Show resolved Hide resolved
`kubectl create configmap palette-agent-config --namespace <cluster-id>` command. Then, execute the command
addetz marked this conversation as resolved.
Show resolved Hide resolved
`kubectl edit configmap palette-agent-config --namespace <cluster-id>` command and paste the configuration provided.

:::

7. Click **Confirm & Create** to save the manifest. The manifest editor closes.

8. Click **Next**. The summary of your Add-on cluster profile appears. Then, click **Finish Configuration** to complete
the cluster profile creation flow. The list of cluster profiles displays.

9. From the left **Main Menu**, click **Clusters**. Select the cluster you want to patch to view cluster details.

10. Select the **Profile** tab. Then, click `+` next to **Addon Layers**, then select the Add-on profile you created
previously. Click **Confirm**, then click **Save**. Wait for Palette to apply this manifest to your cluster.

11. In the upper-right corner, click on **Settings** and select **On-Demand Update**. This will trigger an OS update
immediately if a newer version is available.

### Validate

1. Log in to [Palette](https://console.spectrocloud.com).

2. Click on the cluster you patched to view cluster details.

3. Click on the **Nodes** tab of the cluster details page. You can find the current OS version and when the most recent
patch was applied.

| **Field** | **Description** |
| ----------------------- | --------------------------------------- |
| Last Applied Patch Time | The date and time of the last OS patch. |
| Patched Version | The latest patched version. |

4. Select the **Workloads** tab. The list of namespaces appears. Then, select the **Pods** tab. The list of deployed
pods and their details appears. The **Age** column of the pods shows that they were migrated, not restarted, during
cluster patching.