Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 979 Removing Fluent bit section and adding cloudwatch add on #1121

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,4 @@ resource "random_string" "fluentbit_log_group" {

locals {
cw_log_group_name = "/${var.addon_context.eks_cluster_id}/worker-fluentbit-logs-${random_string.fluentbit_log_group.result}"
}

module "aws_for_fluentbit" {
source = "github.com/aws-ia/terraform-aws-eks-blueprints?ref=v4.32.1//modules/kubernetes-addons/aws-for-fluentbit"

cw_log_group_name = local.cw_log_group_name

addon_context = var.addon_context
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
title: "Install Amazon CloudWatch Observability EKS add-on"
sidebar_position: 30
---

For Kubernetes cluster components that run in pods, these write to files inside the `/var/log` directory, bypassing the default logging mechanism. We can implement pod-level logging by installing the Amazon CloudWatch EKS add-on


**Install the Amazon CloudWatch Observability EKS add-on**

First, we need to check if OpenID Connect (OIDC) provider is present for the cluster or not.
Run the command and check if you get a valid value.

```bash
$ oidc_id=$(aws eks describe-cluster --name eks-workshop --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5)
$ aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4
D09AE1314AF7E745C940B3B6958C404E
```

If not run the following command to create an OpenID Connect (OIDC) provider, if the cluster doesn't have one already

```bash
$ eksctl utils associate-iam-oidc-provider --cluster eks-workshop --approve
```

Enter the following command to create the IAM role with the CloudWatchAgentServerPolicy policy attached, and configure the agent service account to assume that role using OIDC.

```bash
$ eksctl create iamserviceaccount \
> --name cloudwatch-agent \
> --namespace amazon-cloudwatch --cluster eks-workshop \
> --role-name eksworkshop-service-account-role \
> --attach-policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \
> --role-only \
> --approve
2024-09-30 04:04:42 [ℹ] 1 iamserviceaccount (amazon-cloudwatch/cloudwatch-agent) was included (based on the include/exclude rules)
2024-09-30 04:04:42 [!] serviceaccounts in Kubernetes will not be created or modified, since the option --role-only is used
2024-09-30 04:04:42 [ℹ] 1 task: { create IAM role for serviceaccount "amazon-cloudwatch/cloudwatch-agent" }
2024-09-30 04:04:42 [ℹ] building iamserviceaccount stack "eksctl-eks-workshop-addon-iamserviceaccount-amazon-cloudwatch-cloudwatch-agent"
2024-09-30 04:04:43 [ℹ] deploying stack "eksctl-eks-workshop-addon-iamserviceaccount-amazon-cloudwatch-cloudwatch-agent"
2024-09-30 04:04:43 [ℹ] waiting for CloudFormation stack "eksctl-eks-workshop-addon-iamserviceaccount-amazon-cloudwatch-cloudwatch-agent"
2024-09-30 04:05:13 [ℹ] waiting for CloudFormation stack "eksctl-eks-workshop-addon-iamserviceaccount-amazon-cloudwatch-cloudwatch-agent"
```

Install the add-on by entering the following command. Replace **111122223333** with your account ID
You check the arn by going to IAM Roles and search for **eksworkshop-service-account-role**

<ConsoleButton url="https://console.aws.amazon.com/iam/home?#roles" service="console" label="Open IAM console"/>

```bash
$ aws eks create-addon --addon-name amazon-cloudwatch-observability --cluster-name eks-workshop --service-account-role-arn arn:aws:iam::111122223333:role/eksworkshop-service-account-role
{
"addon": {
"addonName": "amazon-cloudwatch-observability",
"clusterName": "eks-workshop",
"status": "CREATING",
"addonVersion": "v2.1.1-eksbuild.1",
"health": {
"issues": []
},
"addonArn": "arn:aws:eks:us-west-2:697541213674:addon/eks-workshop/amazon-cloudwatch-observability/2cc92090-9e76-5b02-7e9c-b71ed079085c",
"createdAt": "2024-09-30T04:12:30.454000+00:00",
"modifiedAt": "2024-09-30T04:12:30.469000+00:00",
"serviceAccountRoleArn": "arn:aws:iam::697541213674:role/eksworkshop-service-account-role",
"tags": {}
}
}
```
Check if CloudWatch Observability EKS add-on is installed
```bash
$ aws eks list-addons --cluster-name eks-workshop
{
"addons": [
"amazon-cloudwatch-observability",
"coredns",
"kube-proxy",
"vpc-cni"
]
}
```


78 changes: 0 additions & 78 deletions website/docs/observability/logging/pod-logging/fluentbit-setup.md

This file was deleted.

4 changes: 0 additions & 4 deletions website/docs/observability/logging/pod-logging/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@ Prepare your environment for this section:
$ prepare-environment observability/logging/pods
```

This will make the following changes to your lab environment:

- Install AWS for Fluent Bit in the Amazon EKS cluster

You can view the Terraform that applies these changes [here](https://github.com/VAR::MANIFESTS_OWNER/VAR::MANIFESTS_REPOSITORY/tree/VAR::MANIFESTS_REF/manifests/modules/observability/logging/pods/.workshop/terraform).
:::

Expand Down