Merge pull request #1511 from VedRatan/user-debug-docs

docs: Added guide for the error which occurs during installation of Kubearmor

deployments/controller/updaterscript.yaml

@@ -0,0 +1,71 @@
+  apiVersion: apps/v1
+  kind: DaemonSet
+  metadata:
+    labels:
+      kubearmor-app: updater
+    name: updater
+    namespace: kubearmor
+  spec:
+    revisionHistoryLimit: 10
+    selector:
+      matchLabels:
+        kubearmor-app: updater
+    template:
+      metadata:
+        labels:
+          kubearmor-app: updater
+      spec:
+        containers:
+        - args:
+          - |
+            grep "bpf" /rootfs/sys/kernel/security/lsm >/dev/null
+            [[ $? -eq 0 ]] && echo "sysfs already has BPF enabled" && sleep infinity
+            grep "GRUB_CMDLINE_LINUX.*bpf" /rootfs/etc/default/grub >/dev/null
+            [[ $? -eq 0 ]] && echo "grub already has BPF enabled" && sleep infinity
+            cat <<EOF >/rootfs/updater.sh
+            #!/bin/bash
+            lsmlist=\$(cat /sys/kernel/security/lsm)
+            echo "current lsmlist=\$lsmlist"
+            sed -i "s/^GRUB_CMDLINE_LINUX=.*$/GRUB_CMDLINE_LINUX=\"lsm=\$lsmlist,bpf\"/g" /etc/default/grub
+            command -v grub2-mkconfig >/dev/null 2>&1 && grub2-mkconfig -o /boot/grub2.cfg
+            command -v grub-mkconfig >/dev/null 2>&1 && grub-mkconfig -o /boot/grub.cfg
+            command -v aa-status >/dev/null 2>&1 || yum install apparmor-utils -y
+            command -v update-grub >/dev/null 2>&1 && update-grub
+            command -v update-grub2 >/dev/null 2>&1 && update-grub2
+            reboot
+            EOF
+            cat /rootfs/updater.sh
+            chmod +x /rootfs/updater.sh
+            chroot /rootfs/ /bin/bash /updater.sh
+          image: debian
+          command:
+            - "bash"
+            - "-c"
+          imagePullPolicy: Always
+          name: updater
+          resources: {}
+          securityContext:
+            privileged: true
+          terminationMessagePath: /dev/termination-log
+          terminationMessagePolicy: File
+          volumeMounts:
+          - mountPath: /rootfs
+            mountPropagation: HostToContainer
+            name: rootfs
+            readOnly: false
+        dnsPolicy: ClusterFirstWithHostNet
+        hostNetwork: true
+        hostPID: true
+        nodeSelector:
+          kubernetes.io/os: linux
+        restartPolicy: Always
+        schedulerName: default-scheduler
+        securityContext: {}
+        terminationGracePeriodSeconds: 30
+        tolerations:
+        - operator: Exists
+        volumes:
+        - hostPath:
+            path: /
+            type: DirectoryOrCreate
+          name: rootfs

getting-started/FAQ.md

@@ -289,12 +289,110 @@ EOF
 docker exec -it kind-control-plane bash -c "apt update && apt install apparmor-utils -y && systemctl restart containerd"
 ```
 
+The above command will install the AppArmor utilities in the kind-control-plane, we can also use this command to install these in minikube as well as in all the other docker based Kubernetes environments.
+
 After this, exit out of the node shell and follow the [getting-started guide](https://github.com/kubearmor/KubeArmor/blob/main/getting-started/deployment_guide.md).
 
+It might be possible that apart from the dockerized kubenetes environment AppArmor might not be available on the master node itself in the Kubernetes cluster. To check for the same you can run the below command to check for the AppArmor support in kernel config:
+
+```
+cat /boot/config-$(uname -r) | grep -e "APPARMOR"
+```
+
+Following flags need to exist and set to `y`
+```ini
+CONFIG_SECURITY_APPARMOR=y
+```
+
+Run the command to install apparmor:
+
+```
+apt update && apt install apparmor-utils -y
+```
+
+You need to restart your CRI in-order to make APPARMOR available as a kernel config security.
+
+If not then we need to install AppArmor utils on the master node itself.
+
 If the `kubearmor-relay` pod goes into CrashLoopBackOff, apply the following patch:
 ```sh
 kubectl patch deploy -n $(kubectl get deploy -l kubearmor-app=kubearmor-relay -A -o custom-columns=:'{.metadata.namespace}',:'{.metadata.name}') --type=json -p='[{"op": "add", "path": "/spec/template/metadata/annotations/container.apparmor.security.beta.kubernetes.io~1kubearmor-relay-server", "value": "unconfined"}]'
 ```
 
 </details>
 
+<details>
+<summary><h4>Debug KubeArmor installation issue</h4></summary>
+In certain scenarios, the expected behavior of KubeArmor might not be observed. One way to investigate this is by using the KubeArmor Command Line Interface (CLI) utility, commonly referred to as [karmor cli](https://github.com/kubearmor/kubearmor-client). 
+
+To check the status and configuration of KubeArmor, you can use the following command:
+
+```
+karmor probe
+```
+
+```
+pc:~$ karmor probe
+
+Found KubeArmor running in Kubernetes
+
+Daemonset :
+
+kubearmor Desired: 1 Ready: 1 Available: 1 Deployments :
+
+kubearmor-controller        Desired: 1   Ready: 1   Available: 1 
+kubearmor-operator          Desired: 1   Ready: 1   Available: 1 
+kubearmor-relay             Desired: 1   Ready: 1   Available: 1
+
+Containers :
+
+kubearmor -apparmor-containerd-98c2c-z772n     Running: 1    Image Version: kubearmor/kubearmor:stable 
+kubearmor-controller -6b5d689967-4wxnh         Running: 2    Image Version: gcr.io/kubebuilder/kube-rbac-proxy:v0.12. 
+kubearmor -operator -6fb47dd855-6tk5r          Running: 1    Image Version: kubearmor/kubearmor-operator: latest
+kubearmor -relay-6966976dbb-hq96h              Running: 1    Image Version: kubearmor/kubearmor-relay-server
+
+Node 1 :
+
+OS Image:                    Debian GNU/Linux 11 (bullseye)
+
+Kernel Version:              6.2.0-36-generic
+
+Kubelet Version:             v1.27.3
+
+Container Runtime:           containerd://1.7.1
+
+Active LSM:
+
+Host Security:               false
+
+Container Security:          false
+
+Container Default Posture:   audit(File)   audit(Capabilities)    audit (Network) 
+Host Default Posture:        audit(File)   audit(Capabilities)   audit (Network) 
+Host Visibility:             none
+
+Armored Up pods :
+
+------------------------------------------------------------
+
+| NAMESPACE | DEFAULT POSTURE | VISIBILITY | NAME | POLICY |
+```
+
+When executing this command, check the output for the value of **ActiveLSM** field, if it is not assigned any value, it means that no active LSM is available for KubeArmor to enforce policies. Under normal circumstances, this value should be assigned a specific Linux Security Module (LSM) that KubeArmor uses to enforce security policies. Additionally, ensure that the **Container Security** field is set to true.
+
+However, there are situations where ActiveLSM might not be assigned any value. This situation indicates that Kubearmor is unable to identify the appropriate LSM in a environment, which is commonly used in Kubernetes setups.
+
+To address this issue, KubeArmor provides a solution involving the use of BPF-LSM. BPF (Berkeley Packet Filter) is a technology that allows efficient packet filtering in the Linux kernel. Enabling support for BPF LSM ensures that KubeArmor can apply and enforce policies as expected in Dockerized environments associated with Kubernetes. Please note that BPFLSM is only available on kernel versions above 5.8 or on RHEL distros > 8.5.
+
+So we need to enable [bpf-lsm](FAQ.md#checking-and-enabling-support-for-bpf-lsm) for Kubearmor to apply and enforce policies as expected.
+
+You can also enable AppArmor if you want to use it as a security module to enforce KubeArmor policies, please refer [here](FAQ.md#using-kubearmor-with-kind-clusters). There is a chance that neither AppArmor nor BPF-LSM is enabled on some nodes. 
+
+**We can apply the following manifest which automatically detects and installs BPFLSM/AppArmor whichever is needed in kubernetes worker nodes.**
+
+```
+kubectl apply -f https://raw.githubusercontent.com/kubearmor/KubeArmor/main/deployments/controller/updaterscript.yaml
+```
+
+**Warning:** After running the above script the nodes will restart.
+</details>

getting-started/deployment_guide.md

@@ -74,6 +74,8 @@ sh: 1: apt: Permission denied
 command terminated with exit code 126
 ```
 
+If you don't see Permission denied please refer [here](FAQ.md#debug-kubearmor-installation-issue-in-dockerized-kubernetes-environment) to debug this issue
+
 </details>
 
 <details>
@@ -178,6 +180,9 @@ cat: /run/secrets/kubernetes.io/serviceaccount/token: Permission denied
 }
 ```
 
+If you don't see Permission denied please refer [here](FAQ.md#debug-kubearmor-installation) to debug this issue.
+
+
 </details>
 
 <details>
@@ -284,4 +289,6 @@ kubectl exec -it $POD -- bash -c "chroot"
 ```
 Any binary other than `bash` and `nginx` would be permission denied.
 
+If you don't see Permission denied please refer [here](FAQ.md#debug-kubearmor-installation) to debug this issue
+
 </details>