K8s Node Upgrade 1.30.2-20240904

[ben851]

Summary | Résumé

Upgrading dev and staging to latest K8s worker node version

Related Issues | Cartes liées

Chore

Test instructions | Instructions pour tester la modification

TF Apply works
Smoke test staging

Release Instructions | Instructions pour le déploiement

None.

Reviewer checklist | Liste de vérification du réviseur

• This PR does not break existing functionality.
• This PR does not violate GCNotify's privacy policies.
• This PR does not raise new security concerns. Refer to our GC Notify Risk Register document on our Google drive.
• This PR does not significantly alter performance.
• Additional required documentation resulting of these changes is covered (such as the README, setup instructions, a related ADR or the technical documentation).

⚠ If boxes cannot be checked off before merging the PR, they should be moved to the "Release Instructions" section with appropriate steps required to verify before release. For example, changes to celery code may require tests on staging to verify that performance has not been affected.

[github-actions]

Staging: eks

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

⚠️   Warning: resources will be destroyed by this change!

Plan: 1 to add, 2 to change, 1 to destroy

Show summary

CHANGE	NAME
update	aws_cloudwatch_metric_alarm.service-callback-too-many-failures-critical[0]
	aws_eks_node_group.notification-canada-ca-eks-node-group-k8s
recreate	aws_secretsmanager_secret_version.eks_karpenter_ami_id

Show plan

Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # aws_cloudwatch_metric_alarm.service-callback-too-many-failures-critical[0] will be updated in-place
  ~ resource "aws_cloudwatch_metric_alarm" "service-callback-too-many-failures-critical" {
      ~ alarm_actions             = [
          - "arn:aws:sns:ca-central-1:239043911459:alert-warning",
          + "arn:aws:sns:ca-central-1:239043911459:alert-critical",
        ]
      ~ alarm_description         = "Service reached the max number of callback retries 25 times in 5 minutes" -> "Service reached the max number of callback retries 100 times in 10 minutes"
        id                        = "service-callback-too-many-failures-warning"
      ~ period                    = 300 -> 600
        tags                      = {}
      ~ threshold                 = 25 -> 100
        # (14 unchanged attributes hidden)
    }

  # aws_eks_node_group.notification-canada-ca-eks-node-group-k8s will be updated in-place
  ~ resource "aws_eks_node_group" "notification-canada-ca-eks-node-group-k8s" {
        id                   = "notification-canada-ca-staging-eks-cluster:notification-canada-ca-staging-eks-primary-node-group-k8s"
      ~ release_version      = "1.30.2-20240828" -> "1.30.2-20240904"
        tags                 = {
            "CostCenter"             = "notification-canada-ca-staging"
            "Name"                   = "notification-canada-ca"
            "karpenter.sh/discovery" = "notification-canada-ca-staging-eks-cluster"
        }
        # (15 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

  # aws_secretsmanager_secret_version.eks_karpenter_ami_id must be replaced
-/+ resource "aws_secretsmanager_secret_version" "eks_karpenter_ami_id" {
      ~ arn            = "arn:aws:secretsmanager:ca-central-1:239043911459:secret:EKS_KARPENTER_AMI_ID-3QJNfC" -> (known after apply)
      ~ id             = "arn:aws:secretsmanager:ca-central-1:239043911459:secret:EKS_KARPENTER_AMI_ID-3QJNfC|terraform-20240903144517121800000002" -> (known after apply)
      ~ secret_string  = (sensitive value) # forces replacement
      ~ version_id     = "terraform-20240903144517121800000002" -> (known after apply)
      ~ version_stages = [
          - "AWSCURRENT",
        ] -> (known after apply)
        # (1 unchanged attribute hidden)
    }

Plan: 1 to add, 2 to change, 1 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"

Show Conftest results

WARN - plan.json - main - Cloudwatch log metric pattern is invalid: ["aws_cloudwatch_log_metric_filter.celery-error[0]"]
WARN - plan.json - main - Cloudwatch log metric pattern is invalid: ["aws_cloudwatch_log_metric_filter.scanfiles-timeout[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_acm_certificate.client_vpn"]
WARN - plan.json - main - Missing Common Tags: ["aws_acm_certificate.notification-canada-ca"]
WARN - plan.json - main - Missing Common Tags: ["aws_acm_certificate.notification-canada-ca-alt[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_acmpca_certificate_authority.client_vpn"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb.notification-canada-ca"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb_listener.internal_alb_tls"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb_listener.notification-canada-ca"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb_target_group.internal_nginx_http"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb_target_group.notification-canada-ca-admin"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb_target_group.notification-canada-ca-api"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb_target_group.notification-canada-ca-document"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb_target_group.notification-canada-ca-document-api"]
WARN - plan.json - main - Missing Common Tags: ["aws_alb_target_group.notification-canada-ca-documentation"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.notification-canada-ca-eks-application-logs[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.notification-canada-ca-eks-cluster-logs[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.notification-canada-ca-eks-prometheus-logs[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.admin-evicted-pods[0]"]
WARN - plan.json - main - Missing Common...