-
Notifications
You must be signed in to change notification settings - Fork 37
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
* DOC-806: adding revised Datadog integration doc * Optimised images with calibre/image-actions * fix: modify images and line numbers * fix: move to webp * fix: move to webp * ci: auto-formatting prettier issues * Optimised images with calibre/image-actions * Optimised images with calibre/image-actions * Optimised images with calibre/image-actions * docs: first edits * Apply suggestions from code review * ci: auto-formatting prettier issues * docs: style guide revision * Optimised images with calibre/image-actions * Optimised images with calibre/image-actions * docs: fix podsecurity issue * docs: fix merge conflicts * Optimised images with calibre/image-actions * docs: rename file * Optimised images with calibre/image-actions * Optimised images with calibre/image-actions * docs: apply suggestions from code review --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Karl Cardenas <[email protected]> Co-authored-by: Will <[email protected]> Co-authored-by: wcrum <[email protected]> Co-authored-by: vault-token-factory-spectrocloud[bot] <133815545+vault-token-factory-spectrocloud[bot]@users.noreply.github.com> Co-authored-by: caroldelwing <[email protected]> Co-authored-by: karl-cardenas-coding <[email protected]> (cherry picked from commit f913d35) Co-authored-by: dmitry-spectro <[email protected]>
- Loading branch information
1 parent
05822af
commit 7123ada
Showing
6 changed files
with
171 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,171 @@ | ||
| --- | ||
| sidebar_label: "Monitor Clusters with Datadog" | ||
| title: "Monitor Host Clusters with Datadog" | ||
| description: "Learn how to use Datadog to monitor Palette host clusters." | ||
| icon: "" | ||
| hide_table_of_contents: false | ||
| sidebar_position: 50 | ||
| hiddenFromNav: false | ||
| tags: ["audit-logs", "datadog", "monitoring"] | ||
| --- | ||
|
|
||
| You can use [Datadog](https://www.datadoghq.com/), a monitoring and analytics tool, to visualize the audit logs of your | ||
| host clusters. | ||
|
|
||
| This guide will help you through the process of configuring your clusters deployed with Palette to export audit logs to | ||
| Datadog. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - A [Palette account](https://console.spectrocloud.com) with tenant admin access. | ||
|
|
||
| - A Datadog account. Visit the [Datadog official website](https://app.datadoghq.eu/signup) to create a Datadog account. | ||
|
|
||
| - An active [host cluster](../glossary-all.md#host-cluster). Refer to the | ||
| [Deploy a Cluster](../clusters/public-cloud/deploy-k8s-cluster.md) tutorial for instructions on how to deploy a | ||
| cluster. | ||
|
|
||
| - [Kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) installed and configured to access your host cluster. Refer | ||
| to the [Access Cluster with CLI](../clusters/cluster-management/palette-webctl.md) page for guidance on how to access | ||
| your cluster with the kubectl CLI. | ||
|
|
||
| ## Configure Host Cluster to Export Logs | ||
|
|
||
| 1. Log in to Datadog and select **Kubernetes** from the left **Main Menu**. | ||
|
|
||
| 2. Select **Helm Chart**. Datadog provides you with an API key. Copy and save the key. | ||
|
|
||
|  | ||
|
|
||
| 3. Log in to [Palette](https://console.spectrocloud.com/). | ||
|
|
||
| 4. From the left **Main Menu**, select **Tenant Settings**. | ||
|
|
||
| 5. Click **Registries** and select **Add New Helm Registry** under **Helm Registries** to add the Datadog Helm | ||
| repository. | ||
|
|
||
| 6. Assign the registry a name, for example, `Datadog`, and include `https://helm.datadoghq.com` as the endpoint. Click | ||
| **Confirm**. The synchronization may take a few minutes to complete. | ||
|
|
||
|  | ||
|
|
||
| 7. Once the synchronization is finished, click on **Profiles** from the left **Main Menu** and select **Add Cluster | ||
| Profile** to create a Datadog add-on cluster profile. | ||
|
|
||
| 8. Give the profile a name, select the type **Add-on**, and click **Next**. | ||
|
|
||
| 9. Click on **Add Helm chart**, choose **Public packs**, and select the **Datadog** registry. | ||
|
|
||
| 10. Next, select the latest version of the `datadog` chart. | ||
|
|
||
|  | ||
|
|
||
| 11. Click **Values** under **Pack Details** to configure the Datadog Helm chart. Modify the parameters according to the | ||
| following table. | ||
|
|
||
| | **Parameter** | **Modification** | | ||
| | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | **pack.namespace** | Provide a namespace for the Datadog installation, for example, `datadog`. | | ||
| | **datadog.apiKey** | Insert your Datadog API key. | | ||
| | **datadog.site** | Insert your regional Datadog website, for example, `us5.datadoghq.com`. Refer to the [Getting Started with Datadog Sites](https://docs.datadoghq.com/getting_started/site/) page for a complete list of websites. | | ||
| | **datadog.logs.enabled** | Set it to `true` to enable the Datadog Agent log collection. | | ||
| | **datadog.logs.containerCollectAll** | Set it to `true` to allow log collection for all containers. | | ||
| | **datadog.processAgent.processCollection** | Set it to `true` to enable process collection. | | ||
| | **datadog.networkMonitoring.enabled** | Set it to `true` to enable network performance monitoring. | | ||
| | **datadog.serviceMonitoring.enabled** | Set it to `true` to enable universal service monitoring. | | ||
| | **datadog.securityAgent.compliance.enabled** | Set it to `true` to enable the security agent. | | ||
|
|
||
| 12. Next, paste the following snippet under the `clusterAgent.volumes` section to enable the volume and volume mount for | ||
| collecting audit logs. Replace `/var/log/apiserver` with the `audit-log` path configured in the Kubernetes layer of | ||
| your host cluster. You can find this path in the Kubernetes layer of your host cluster profile in Palette. | ||
|
|
||
| ```yaml {3,13} | ||
| volumes: | ||
| - hostPath: | ||
| path: /var/log/apiserver | ||
| name: auditdir | ||
| - name: dd-agent-config | ||
| configMap: | ||
| name: dd-agent-config | ||
| items: | ||
| - key: kubernetes-audit-log | ||
| path: conf.yaml | ||
| volumeMounts: | ||
| - name: auditdir | ||
| mountPath: /var/log/apiserver | ||
| - name: dd-agent-config | ||
| mountPath: /conf.d/kubernetes_audit.d | ||
| ``` | ||
| 13. Additionally, if your host cluster uses Kubernetes version 1.25 or later, add the following lines under the | ||
| `pack.namespace` section to accommodate the elevated privileges required by the Datadog pods. Replace | ||
| `<k8s_version>` with your cluster's Kubernetes version and only include the major and minor version following the | ||
| lowercase letter **v**. For example, `v1.30`. Refer to the | ||
| [Deployment Violates Pod Security](../troubleshooting/cluster-deployment.md#deployment-violates-pod-security) | ||
| troubleshooting guide for more information. | ||
|
|
||
| ```yaml {3-4} | ||
| pack: | ||
| namespace: "datadog" | ||
| namespaceLabels: | ||
| "datadog": "pod-security.kubernetes.io/enforce=privileged,pod-security.kubernetes.io/enforce-version=v<k8s_version>" | ||
| ``` | ||
|
|
||
| 14. Once you finish making the necessary changes, click **Confirm & Create**, then select **Next**. | ||
|
|
||
| 15. Review the cluster profile and click **Finish Configuration**. | ||
|
|
||
| 16. In Palette, attach the Datadog add-on cluster profile to your host cluster. Refer to the | ||
| [Attach an Add-on Profile](../clusters/imported-clusters/attach-add-on-profile.md) guide for instructions. | ||
|
|
||
| 17. Open a terminal window and use the `cat` command to create a ConfigMap resource for the host cluster to collect | ||
| data. | ||
|
|
||
| ```yaml | ||
| cat << EOF > datadog-configmap.yaml | ||
| kind: ConfigMap | ||
| apiVersion: v1 | ||
| metadata: | ||
| name: dd-agent-config | ||
| namespace: datadog | ||
| data: | ||
| kubernetes-audit-log: |- | ||
| logs: | ||
| - type: file | ||
| path: /var/log/apiserver/audit.log | ||
| source: kubernetes.audit | ||
| service: audit | ||
| EOF | ||
| ``` | ||
|
|
||
| 18. Issue the following command to apply the ConfigMap to your host cluster. | ||
|
|
||
| ```bash | ||
| kubectl apply --namespace datadog --filename datadog-configmap.yaml | ||
| ``` | ||
|
|
||
| ```text hideClipboard | ||
| configmap/dd-agent-config created | ||
| ``` | ||
|
|
||
| 19. After a few minutes, the Datadog agent will start reporting. Log in to Datadog and select **Kubernetes** from the | ||
| left **Main Menu** to verify that the audit logs are being transferred successfully to Datadog. | ||
|
|
||
|  | ||
|
|
||
| ## Validate | ||
|
|
||
| 1. Log in to Datadog. | ||
|
|
||
| 2. Click on **Logs** in the left **Main Menu**, select **Explorer**, and verify that your host cluster audit logs are | ||
| visible. | ||
|
|
||
|  | ||
|
|
||
| ## Resources | ||
|
|
||
| - [Cluster Monitoring](../clusters/cluster-management/monitoring/monitoring.md) | ||
|
|
||
| - [Audit Logs](./audit-logs.md) | ||
|
|
||
| - [Monitoring Kubernetes with Datadog](https://www.datadoghq.com/blog/monitoring-kubernetes-with-datadog/) |
Binary file added
BIN
+41.7 KB
static/assets/docs/images/audit-logs_monitor-with-datadog_datadog-agent.webp
Binary file not shown.
Binary file added
BIN
+361 KB
static/assets/docs/images/audit-logs_monitor-with-datadog_datadog-logs.webp
Binary file not shown.
Binary file added
BIN
+59.4 KB
static/assets/docs/images/audit-logs_monitor-with-datadog_datadog-pack.webp
Binary file not shown.
Binary file added
BIN
+142 KB
static/assets/docs/images/audit-logs_monitor-with-datadog_datadog-ui.webp
Binary file not shown.
Binary file added
BIN
+84.2 KB
static/assets/docs/images/audit-logs_monitor-with-datadog_registry.webp
Binary file not shown.