docs: PAC-828 Document addition of namespace labels for pods needing …

…elevated privileges (#1706)

* PAC-828 Document addition of namespace labels for pods needing elevated privileges

* fix formatting

* clarify cluster profile

* Update docs/docs-content/troubleshooting/cluster-deployment.md

Co-authored-by: Rita Watson <[email protected]>

* Update docs/docs-content/troubleshooting/cluster-deployment.md

Co-authored-by: Rita Watson <[email protected]>

* Style guide edits

* Add clarification for multiple namespaces

* Update docs/docs-content/troubleshooting/cluster-deployment.md

Co-authored-by: Rita Watson <[email protected]>

* Update docs/docs-content/troubleshooting/cluster-deployment.md

Co-authored-by: Rita Watson <[email protected]>

* Update docs/docs-content/troubleshooting/cluster-deployment.md

Co-authored-by: Rita Watson <[email protected]>

* Incorporate feedback

* Incorporate feedback

* vale comments

* change step numbers

* Update docs/docs-content/troubleshooting/cluster-deployment.md

Co-authored-by: Rita Watson <[email protected]>

* Update docs/docs-content/troubleshooting/cluster-deployment.md

Co-authored-by: Rita Watson <[email protected]>

* Update docs/docs-content/troubleshooting/cluster-deployment.md

Co-authored-by: Rita Watson <[email protected]>

* Add second example

* vale comments

* second person voice

* Explicitly call out how to provide versions

---------

Co-authored-by: Lenny Chen <[email protected]>
Co-authored-by: Rita Watson <[email protected]>
Co-authored-by: Lenny Chen <[email protected]>

docs/docs-content/troubleshooting/cluster-deployment.md

@@ -17,7 +17,8 @@ tags: ["troubleshooting", "cluster-deployment"]
 The following steps will help you troubleshoot errors in the event issues arise while deploying a cluster.  
 
 
-## Scenario - Instances Continuously Delete Every 30 Minutes
+## Instances Continuously Delete Every 30 Minutes
+
 
 An instance is launched and terminated every 30 minutes prior to completion of its deployment, and the **Events Tab** lists errors with the following message:
 
@@ -94,6 +95,66 @@ Common reasons for why a service may fail are:
 
 6. Check stdout for errors. You can also open a support ticket. Visit our [support page](http://support.spectrocloud.io/).
 
+## Deployment Violates Pod Security
+Cluster deployment fails with the following message. 
+
+```
+Error creating: pods <name of pod> is forbidden: violates PodSecurity "baseline:v<k8s version>": non-default capabilities …
+```
+
+This can happen when the cluster profile uses Kubernetes 1.25 or later and also includes packs that create pods that require elevated privileges. 
+
+### Debug Steps
+
+To address this issue, you can change the Pod Security Standards of the namespace where the pod is being created. 
+
+1. Log in to [Palette](https://console.spectrocloud.com).
+
+2. Navigate to the left **Main Menu** and click on **Profiles**. 
+
+3. Select the profile you are using to deploy the cluster. Palette displays the profile stack and details.
+4. Click on the pack layer in the profile stack that contains the pack configuration.
+
+5. In the pack's YAML file, add a subfield in the `pack` section called `namespaceLabels` if it does not already exist.
+
+6. In the `namespaceLabels` section, add a line with the name of your namespace as the key and add `pod-security.kubernetes.io/enforce=privileged,pod-security.kubernetes.io/enforce-version=v<k8s_version>` as its value. Replace `<k8s_version>` with the version of Kubernetes on your cluster and only include the major and minor version following the lowercase letter `v`. For example, `v1.25` and `v1.28`.
+7. If a key matching your namespace already exists, add the labels to the value corresponding to that key. 
+
+:::caution
+
+We recommend only applying the labels to namespaces where pods fail to be created. 
+If your pack creates multiple namespaces, and you are unsure which ones contain pods that need the elevated privileges, you can access the cluster with the kubectl CLI and use the `kubectl get pods` command. 
+This command lists pods and their namespaces so you can identify the pods that are failing at creation.
+
+For guidance in using the CLI, review [Access Cluster with CLI](./clusters/cluster-management/palette-webctl/#access-cluster-with-cli). To learn more about kubectl pod commands, refer to the [Kubernetes](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#get) documentation.
+
+:::
+
+### Examples
+
+The following example shows a pack that creates a namespace called `"monitoring"`. In this example, the `monitoring` namespace does not have any pre-existing labels. 
+You need to add the `namespaceLabels` line as well as the corresponding key-value pair under it to apply the labels to the `monitoring` namespace. 
+
+```yaml
+pack:
+  namespace: "monitoring"
+
+  namespaceLabels:
+    "monitoring": "pod-security.kubernetes.io/enforce=privileged,pod-security.kubernetes.io/enforce-version=v1.28"
+
+```
+
+This second example is similar to the first one. However, in this example, the `monitoring` key already exists under `namespaceLabels`, with its original value being `"org=spectro,team=dev"`. Therefore, you add the labels to the existing value:
+
+```yaml
+pack:
+  namespace: "monitoring"
+
+  namespaceLabels:
+    "monitoring": "org=spectro,team=dev,pod-security.kubernetes.io/enforce=privileged,pod-security.kubernetes.io/enforce-version=v1.28"
+```
+
+
 
 ## Gateway Installer Registration Failures