diff --git a/manifests/modules/observability/logging/pods/.workshop/terraform/main.tf b/manifests/modules/observability/logging/pods/.workshop/terraform/main.tf index c2634ccd1..91e2e3e68 100644 --- a/manifests/modules/observability/logging/pods/.workshop/terraform/main.tf +++ b/manifests/modules/observability/logging/pods/.workshop/terraform/main.tf @@ -5,12 +5,4 @@ resource "random_string" "fluentbit_log_group" { locals { cw_log_group_name = "/${var.addon_context.eks_cluster_id}/worker-fluentbit-logs-${random_string.fluentbit_log_group.result}" -} - -module "aws_for_fluentbit" { - source = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.32.1//modules/kubernetes-addons/aws-for-fluentbit" - - cw_log_group_name = local.cw_log_group_name - - addon_context = var.addon_context } \ No newline at end of file diff --git a/website/docs/observability/logging/pod-logging/cloudwatch-addon-setup.md b/website/docs/observability/logging/pod-logging/cloudwatch-addon-setup.md new file mode 100644 index 000000000..808c0f435 --- /dev/null +++ b/website/docs/observability/logging/pod-logging/cloudwatch-addon-setup.md @@ -0,0 +1,82 @@ +--- +title: "Install Amazon CloudWatch Observability EKS add-on" +sidebar_position: 30 +--- + +For Kubernetes cluster components that run in pods, these write to files inside the `/var/log` directory, bypassing the default logging mechanism. We can implement pod-level logging by installing the Amazon CloudWatch EKS add-on + + +**Install the Amazon CloudWatch Observability EKS add-on** + +First, we need to check if OpenID Connect (OIDC) provider is present for the cluster or not. +Run the command and check if you get a valid value. + +```bash +$ oidc_id=$(aws eks describe-cluster --name eks-workshop --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5) +$ aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4 +D09AE1314AF7E745C940B3B6958C404E +``` + +If not run the following command to create an OpenID Connect (OIDC) provider, if the cluster doesn't have one already + +```bash +$ eksctl utils associate-iam-oidc-provider --cluster eks-workshop --approve +``` + +Enter the following command to create the IAM role with the CloudWatchAgentServerPolicy policy attached, and configure the agent service account to assume that role using OIDC. + +```bash +$ eksctl create iamserviceaccount \ +> --name cloudwatch-agent \ +> --namespace amazon-cloudwatch --cluster eks-workshop \ +> --role-name eksworkshop-service-account-role \ +> --attach-policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \ +> --role-only \ +> --approve +2024-09-30 04:04:42 [ℹ] 1 iamserviceaccount (amazon-cloudwatch/cloudwatch-agent) was included (based on the include/exclude rules) +2024-09-30 04:04:42 [!] serviceaccounts in Kubernetes will not be created or modified, since the option --role-only is used +2024-09-30 04:04:42 [ℹ] 1 task: { create IAM role for serviceaccount "amazon-cloudwatch/cloudwatch-agent" } +2024-09-30 04:04:42 [ℹ] building iamserviceaccount stack "eksctl-eks-workshop-addon-iamserviceaccount-amazon-cloudwatch-cloudwatch-agent" +2024-09-30 04:04:43 [ℹ] deploying stack "eksctl-eks-workshop-addon-iamserviceaccount-amazon-cloudwatch-cloudwatch-agent" +2024-09-30 04:04:43 [ℹ] waiting for CloudFormation stack "eksctl-eks-workshop-addon-iamserviceaccount-amazon-cloudwatch-cloudwatch-agent" +2024-09-30 04:05:13 [ℹ] waiting for CloudFormation stack "eksctl-eks-workshop-addon-iamserviceaccount-amazon-cloudwatch-cloudwatch-agent" +``` + +Install the add-on by entering the following command. Replace **111122223333** with your account ID +You check the arn by going to IAM Roles and search for **eksworkshop-service-account-role** + + + +```bash +$ aws eks create-addon --addon-name amazon-cloudwatch-observability --cluster-name eks-workshop --service-account-role-arn arn:aws:iam::111122223333:role/eksworkshop-service-account-role +{ + "addon": { + "addonName": "amazon-cloudwatch-observability", + "clusterName": "eks-workshop", + "status": "CREATING", + "addonVersion": "v2.1.1-eksbuild.1", + "health": { + "issues": [] + }, + "addonArn": "arn:aws:eks:us-west-2:697541213674:addon/eks-workshop/amazon-cloudwatch-observability/2cc92090-9e76-5b02-7e9c-b71ed079085c", + "createdAt": "2024-09-30T04:12:30.454000+00:00", + "modifiedAt": "2024-09-30T04:12:30.469000+00:00", + "serviceAccountRoleArn": "arn:aws:iam::697541213674:role/eksworkshop-service-account-role", + "tags": {} + } +} +``` +Check if CloudWatch Observability EKS add-on is installed +```bash +$ aws eks list-addons --cluster-name eks-workshop +{ + "addons": [ + "amazon-cloudwatch-observability", + "coredns", + "kube-proxy", + "vpc-cni" + ] +} +``` + + diff --git a/website/docs/observability/logging/pod-logging/fluent-bit-cloudwatch.md b/website/docs/observability/logging/pod-logging/eks-cloudwatch.md similarity index 100% rename from website/docs/observability/logging/pod-logging/fluent-bit-cloudwatch.md rename to website/docs/observability/logging/pod-logging/eks-cloudwatch.md diff --git a/website/docs/observability/logging/pod-logging/fluentbit-setup.md b/website/docs/observability/logging/pod-logging/fluentbit-setup.md deleted file mode 100644 index 973191e72..000000000 --- a/website/docs/observability/logging/pod-logging/fluentbit-setup.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -title: "Using Fluent Bit" -sidebar_position: 30 ---- - -For Kubernetes cluster components that run in pods, these write to files inside the `/var/log` directory, bypassing the default logging mechanism. We can implement pod-level logging by deploying a node-level logging agent as a DaemonSet on each node, such as Fluent Bit. - -[Fluent Bit](https://fluentbit.io/) is a lightweight log processor and forwarder that allows you to collect data and logs from different sources, enrich them with filters and send them to multiple destinations like CloudWatch, Kinesis Data Firehose, Kinesis Data Streams and Amazon OpenSearch Service. - -AWS provides a Fluent Bit image with plugins for both CloudWatch Logs and Kinesis Data Firehose. The [AWS for Fluent Bit](https://github.com/aws/aws-for-fluent-bit) image is available on the [Amazon ECR Public Gallery](https://gallery.ecr.aws/aws-observability/aws-for-fluent-bit). - -In the following section, you will see how to validate Fluent Bit agent is running as a daemonSet to send the containers / pods logs to CloudWatch Logs. - -First, we can validate the resources created for Fluent Bit by entering the following command. Each node should have one pod: - -```bash -$ kubectl get all -n aws-for-fluent-bit -NAME READY STATUS RESTARTS AGE -pod/aws-for-fluent-bit-vfsbe 1/1 Running 0 99m -pod/aws-for-fluent-bit-kmvnk 1/1 Running 0 99m -pod/aws-for-fluent-bit-rxhs7 1/1 Running 0 100m - -NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE -daemonset.apps/aws-for-fluent-bit 2 2 2 2 2 104m -``` - -The ConfigMap for aws-for-fluent-bit is configured to stream the contents of files in the directory `/var/log/containers/*.log` from each node to the CloudWatch log group `/eks-workshop/worker-fluentbit-logs`: - -```bash -$ kubectl describe configmaps -n aws-for-fluent-bit -Name: aws-for-fluent-bit -Namespace: aws-for-fluent-bit -Labels: app.kubernetes.io/instance=aws-for-fluent-bit - app.kubernetes.io/managed-by=Helm - app.kubernetes.io/name=aws-for-fluent-bit - app.kubernetes.io/version=2.21.5 - helm.sh/chart=aws-for-fluent-bit-0.1.18 -Annotations: meta.helm.sh/release-name: aws-for-fluent-bit - meta.helm.sh/release-namespace: aws-for-fluent-bit - -Data -==== -fluent-bit.conf: ----- -[SERVICE] - Parsers_File /fluent-bit/parsers/parsers.conf - -[INPUT] - Name tail - Tag kube.* - Path /var/log/containers/*.log - DB /var/log/flb_kube.db - Parser docker - Docker_Mode On - Mem_Buf_Limit 5MB - Skip_Long_Lines On - Refresh_Interval 10 - -[FILTER] - Name kubernetes - Match kube.* - Kube_URL https://kubernetes.default.svc.cluster.local:443 - Merge_Log On - Merge_Log_Key data - Keep_Log On - K8S-Logging.Parser On - K8S-Logging.Exclude On -[OUTPUT] - Name cloudwatch - Match * - region us-east-1 - log_group_name /eks-workshop/worker-fluentbit-logs - log_stream_prefix fluentbit- - auto_create_group true - -........... - -``` diff --git a/website/docs/observability/logging/pod-logging/index.md b/website/docs/observability/logging/pod-logging/index.md index 185a72f28..8ab64a1db 100644 --- a/website/docs/observability/logging/pod-logging/index.md +++ b/website/docs/observability/logging/pod-logging/index.md @@ -14,10 +14,6 @@ Prepare your environment for this section: $ prepare-environment observability/logging/pods ``` -This will make the following changes to your lab environment: - -- Install AWS for Fluent Bit in the Amazon EKS cluster - You can view the Terraform that applies these changes [here](https://github.com/VAR::MANIFESTS_OWNER/VAR::MANIFESTS_REPOSITORY/tree/VAR::MANIFESTS_REF/manifests/modules/observability/logging/pods/.workshop/terraform). :::