Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a budget alert for when we use a lot of cloudwatch data scanning #1704

Merged
merged 2 commits into from
Dec 19, 2024

Conversation

ben851
Copy link
Contributor

@ben851 ben851 commented Dec 19, 2024

Summary | Résumé

We had a large charge in AWS Cloudwatch for querying data. This budget will alert us when we're going crazy in the future.

Related Issues | Cartes liées

Chore

Before merging this PR

Read code suggestions left by the
cds-ai-codereviewer bot. Address
valid suggestions and shortly write down reasons to not address others. To help
with the classification of the comments, please use these reactions on each of the
comments made by the AI review:

Classification Reaction Emoticon
Useful +1 👍
Noisy eyes 👀
Hallucination confused 😕
Wrong but teachable rocket 🚀
Wrong and incorrect -1 👎

The classifications will be extracted and summarized into an analysis of how helpful
or not the AI code review really is.

Test instructions | Instructions pour tester la modification

TF Apply
Verify budget shows data in target environment

Release Instructions | Instructions pour le déploiement

None.

Reviewer checklist | Liste de vérification du réviseur

  • This PR does not break existing functionality.
  • This PR does not violate GCNotify's privacy policies.
  • This PR does not raise new security concerns. Refer to our GC Notify Risk Register document on our Google drive.
  • This PR does not significantly alter performance.
  • Additional required documentation resulting of these changes is covered (such as the README, setup instructions, a related ADR or the technical documentation).

⚠ If boxes cannot be checked off before merging the PR, they should be moved to the "Release Instructions" section with appropriate steps required to verify before release. For example, changes to celery code may require tests on staging to verify that performance has not been affected.

[review]
@ben851 ben851 requested a review from a team December 19, 2024 18:52
@ben851 ben851 requested a review from jimleroyer as a code owner December 19, 2024 18:52
@@ -23,6 +23,38 @@ resource "aws_budgets_budget" "notify_global" {
}
}

resource "aws_budgets_budget" "cloudwatch_data_scanned" {
name = "coudwatch-data-scanned-budget"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a typo in the budget name. It should be cloudwatch-data-scanned-budget instead of coudwatch-data-scanned-budget.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link

staging: common

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

⚠️   Warning: resources will be destroyed by this change!

Plan: 5 to add, 0 to change, 4 to destroy
Show summary
CHANGE NAME
add aws_budgets_budget.cloudwatch_data_scanned
recreate module.notify_slack_critical.module.lambda.null_resource.archive[0]
module.notify_slack_general.module.lambda.null_resource.archive[0]
module.notify_slack_ok.module.lambda.null_resource.archive[0]
module.notify_slack_warning.module.lambda.null_resource.archive[0]
Show plan
Resource actions are indicated with the following symbols:
  + create
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # aws_budgets_budget.cloudwatch_data_scanned will be created
  + resource "aws_budgets_budget" "cloudwatch_data_scanned" {
      + account_id        = (known after apply)
      + arn               = (known after apply)
      + budget_type       = "USAGE"
      + id                = (known after apply)
      + limit_amount      = "10000"
      + limit_unit        = "GB"
      + name              = "cloudwatch-data-scanned-budget"
      + name_prefix       = (known after apply)
      + tags_all          = (known after apply)
      + time_period_end   = "2087-06-15_00:00"
      + time_period_start = (known after apply)
      + time_unit         = "DAILY"

      + cost_filter {
          + name   = "UsageType"
          + values = [
              + "CAN1-DataScanned-Bytes",
            ]
        }

      + cost_types (known after apply)

      + notification {
          + comparison_operator        = "GREATER_THAN"
          + notification_type          = "ACTUAL"
          + subscriber_email_addresses = []
          + subscriber_sns_topic_arns  = [
              + "arn:aws:sns:ca-central-1:239043911459:alert-general",
            ]
          + threshold                  = 100
          + threshold_type             = "PERCENTAGE"
        }
      + notification {
          + comparison_operator        = "GREATER_THAN"
          + notification_type          = "ACTUAL"
          + subscriber_email_addresses = []
          + subscriber_sns_topic_arns  = [
              + "arn:aws:sns:ca-central-1:239043911459:alert-general",
            ]
          + threshold                  = 80
          + threshold_type             = "PERCENTAGE"
        }
    }

  # module.notify_slack_critical.module.lambda.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "434405101312382330" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "timestamp" = "1734450609736004200" -> "1734634570101046000"
            # (1 unchanged element hidden)
        }
    }

  # module.notify_slack_general.module.lambda.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "5658221801416866527" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "timestamp" = "1734450602570921000" -> "1734634563989798000"
            # (1 unchanged element hidden)
        }
    }

  # module.notify_slack_ok.module.lambda.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "4965436936612757236" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "timestamp" = "1734450609738267800" -> "1734634570133648000"
            # (1 unchanged element hidden)
        }
    }

  # module.notify_slack_warning.module.lambda.null_resource.archive[0] must be replaced
-/+ resource "null_resource" "archive" {
      ~ id       = "780507589162386774" -> (known after apply)
      ~ triggers = { # forces replacement
          ~ "timestamp" = "1734450609783591000" -> "1734634570092058000"
            # (1 unchanged element hidden)
        }
    }

Plan: 5 to add, 0 to change, 4 to destroy.

Warning: Argument is deprecated

  with aws_s3_bucket.csv_bucket,
  on s3.tf line 5, in resource "aws_s3_bucket" "csv_bucket":
   5: resource "aws_s3_bucket" "csv_bucket" {

Use the aws_s3_bucket_server_side_encryption_configuration resource instead

(and 65 more similar warnings elsewhere)

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.ad_hoc"]
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.build_tables"]
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.primary"]
WARN - plan.json - main - Missing Common Tags: ["aws_athena_workgroup.support"]
WARN - plan.json - main - Missing Common Tags: ["aws_budgets_budget.cloudwatch_data_scanned"]
WARN - plan.json - main - Missing Common Tags: ["aws_budgets_budget.notify_global"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_event_rule.aws_health[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.route53_resolver_query_log[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.sns_deliveries[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.sns_deliveries_failures[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.sns_deliveries_failures_us_west_2[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_log_group.sns_deliveries_us_west_2[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-bulk-not-being-processed-critical[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-bulk-not-being-processed-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-inflights-not-being-processed-critical[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-inflights-not-being-processed-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-not-being-processed-critical[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.bulk-not-being-processed-warning[0]"]
WARN - plan.json - main - Missing Common Tags: ["aws_cloudwatch_metric_alarm.contact-3-500-error-15-minutes-critical[0]"]
WARN - plan.json - main - Missing Common...

Copy link
Contributor

@P0NDER0SA P0NDER0SA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

Copy link
Member

@jimleroyer jimleroyer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merci chef.

@ben851 ben851 merged commit 79bd4ff into main Dec 19, 2024
29 checks passed
@ben851 ben851 deleted the cloudwatch-data-budget branch December 19, 2024 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants