Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add template category columns to notifications dataset #1478

Merged
merged 4 commits into from
Jul 31, 2024

Conversation

andrewleith
Copy link
Member

Summary | Résumé

This PR adds the following fields to the Notifications dataset to expose the template categories properties of each notifications template:

  • tc_id
  • tc_name_en
  • tc_name_fr
  • tc_email_process_type
  • tc_sms_process_type
  • tc_sms_sending_vehicle

Related Issues | Cartes liées

Test instructions | Instructions pour tester la modification

TODO: Fill in test instructions for the reviewer.

Release Instructions | Instructions pour le déploiement

None.

Reviewer checklist | Liste de vérification du réviseur

  • This PR does not break existing functionality.
  • This PR does not violate GCNotify's privacy policies.
  • This PR does not raise new security concerns. Refer to our GC Notify Risk Register document on our Google drive.
  • This PR does not significantly alter performance.
  • Additional required documentation resulting of these changes is covered (such as the README, setup instructions, a related ADR or the technical documentation).

⚠ If boxes cannot be checked off before merging the PR, they should be moved to the "Release Instructions" section with appropriate steps required to verify before release. For example, changes to celery code may require tests on staging to verify that performance has not been affected.

@andrewleith andrewleith requested a review from jimleroyer as a code owner July 30, 2024 16:08
@andrewleith andrewleith requested a review from sastels July 31, 2024 12:33
Copy link
Contributor

@sastels sastels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!!

Copy link

Staging: quicksight

✅   Terraform Init: success
✅   Terraform Validate: success
✅   Terraform Format: success
✅   Terraform Plan: success
✅   Conftest: success

Plan: 0 to add, 2 to change, 0 to destroy
Show summary
CHANGE NAME
update aws_quicksight_data_set.notifications
aws_s3_object.manifest_file
Show plan
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_quicksight_data_set.notifications will be updated in-place
  ~ resource "aws_quicksight_data_set" "notifications" {
        id             = "239043911459,notifications"
        name           = "Notifications"
        tags           = {}
        # (6 unchanged attributes hidden)

      - physical_table_map {
          - physical_table_map_id = "notifications" -> null

          - custom_sql {
              - data_source_arn = "arn:aws:quicksight:ca-central-1:239043911459:datasource/NotificationCanadaCastaging" -> null
              - name            = "notifications" -> null
              - sql_query       = <<-EOT
                    with notification_data as (
                                select 
                                  id as notification_id,
                                  created_at as notification_created_at,
                                  queue_name as notification_queue_name,
                                  sent_at as notification_sent_at,
                                  notification_status,
                                  notification_type,
                                  updated_at notification_updated_at,
                                  job_id,
                                  api_key_id,
                                  key_type as api_key_type,
                                  service_id,
                                  template_id,
                                  reference as notification_reference
                                from notifications
                              union
                                select 
                                  id as notification_id,
                                  created_at as notification_created_at,
                                  queue_name as notification_queue_name,
                                  sent_at as notification_sent_at,
                                  notification_status,
                                  notification_type,
                                  updated_at notification_updated_at,
                                  job_id,
                                  api_key_id,
                                  key_type as api_key_type,
                                  service_id,
                                  template_id,
                                  reference as notification_reference
                                from notification_history
                            ),
                            service_data as (
                              select
                                s.id as service_id,
                                s.active as service_active,
                                count_as_live as service_count_as_live,
                                s.go_live_at as service_go_live_at,
                                s.name as service_name,
                                s.message_limit as service_message_limit,
                                s.rate_limit as service_rate_limit,
                                s.sms_daily_limit as service_sms_daily_limit,
                                o.name as organisation_name,
                                s.organisation_id
                              from services s join organisation o on s.organisation_id = o.id
                            ),
                            template_data as (
                              select
                                id as template_id,
                                created_at as template_created_at,
                                name as template_name,
                                updated_at as template_updated_at,
                                version as template_version
                              from templates
                            )
                            select
                              notification_id, notification_created_at, notification_sent_at, notification_status,
                              notification_queue_name, notification_type, notification_updated_at, 
                              notification_reference, job_id, api_key_id, api_key_type,
                              s.*, t.*
                            from notification_data n 
                              join service_data s on n.service_id = s.service_id
                              join template_data t on n.template_id = t.template_id
                EOT -> null

              - columns {
                  - name = "notification_id" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "notification_created_at" -> null
                  - type = "DATETIME" -> null
                }
              - columns {
                  - name = "notification_sent_at" -> null
                  - type = "DATETIME" -> null
                }
              - columns {
                  - name = "notification_updated_at" -> null
                  - type = "DATETIME" -> null
                }
              - columns {
                  - name = "notification_queue_name" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "notification_status" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "notification_type" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "notification_reference" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "job_id" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "api_key_id" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "api_key_type" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "service_id" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "service_active" -> null
                  - type = "BOOLEAN" -> null
                }
              - columns {
                  - name = "service_count_as_live" -> null
                  - type = "BOOLEAN" -> null
                }
              - columns {
                  - name = "service_go_live_at" -> null
                  - type = "DATETIME" -> null
                }
              - columns {
                  - name = "service_name" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "service_message_limit" -> null
                  - type = "INTEGER" -> null
                }
              - columns {
                  - name = "service_rate_limit" -> null
                  - type = "INTEGER" -> null
                }
              - columns {
                  - name = "service_sms_daily_limit" -> null
                  - type = "INTEGER" -> null
                }
              - columns {
                  - name = "organisation_name" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "organisation_id" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "template_id" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "template_created_at" -> null
                  - type = "DATETIME" -> null
                }
              - columns {
                  - name = "template_name" -> null
                  - type = "STRING" -> null
                }
              - columns {
                  - name = "template_updated_at" -> null
                  - type = "DATETIME" -> null
                }
              - columns {
                  - name = "template_version" -> null
                  - type = "INTEGER" -> null
                }
            }
        }
      + physical_table_map {
          + physical_table_map_id = "notifications"

          + custom_sql {
              + data_source_arn = "arn:aws:quicksight:ca-central-1:239043911459:datasource/NotificationCanadaCastaging"
              + name            = "notifications"
              + sql_query       = <<-EOT
                    with notification_data as (
                                select 
                                  id as notification_id,
                                  created_at as notification_created_at,
                                  queue_name as notification_queue_name,
                                  sent_at as notification_sent_at,
                                  notification_status,
                                  notification_type,
                                  updated_at notification_updated_at,
                                  job_id,
                                  api_key_id,
                                  key_type as api_key_type,
                                  service_id,
                                  template_id,
                                  reference as notification_reference
                                from notifications
                              union
                                select 
                                  id as notification_id,
                                  created_at as notification_created_at,
                                  queue_name as notification_queue_name,
                                  sent_at as notification_sent_at,
                                  notification_status,
                                  notification_type,
                                  updated_at notification_updated_at,
                                  job_id,
                                  api_key_id,
                                  key_type as api_key_type,
                                  service_id,
                                  template_id,
                                  reference as notification_reference
                                from notification_history
                            ),
                            service_data as (
                              select
                                s.id as service_id,
                                s.active as service_active,
                                count_as_live as service_count_as_live,
                                s.go_live_at as service_go_live_at,
                                s.name as service_name,
                                s.message_limit as service_message_limit,
                                s.rate_limit as service_rate_limit,
                                s.sms_daily_limit as service_sms_daily_limit,
                                o.name as organisation_name,
                                s.organisation_id
                              from services s join organisation o on s.organisation_id = o.id
                            ),
                            template_data as (
                              select
                                t.id as template_id,
                                t.created_at as template_created_at,
                                t.name as template_name,
                                t.updated_at as template_updated_at,
                                t.version as template_version,
                                tc.id as tc_id, 
                                tc.name_en as tc_name_en, 
                                tc.name_fr as tc_name_fr, 
                                tc.email_process_type as tc_email_process_type, 
                                tc.sms_process_type as tc_sms_process_type, 
                                tc.sms_sending_vehicle as tc_sms_sending_vehicle
                              from templates t
                              join template_categories tc on tc.id = t.template_category_id
                            )
                            select
                              notification_id, notification_created_at, notification_sent_at, notification_status,
                              notification_queue_name, notification_type, notification_updated_at, 
                              notification_reference, job_id, api_key_id, api_key_type,
                              s.*, t.*
                            from notification_data n 
                              join service_data s on n.service_id = s.service_id
                              join template_data t on n.template_id = t.template_id
                EOT

              + columns {
                  + name = "notification_id"
                  + type = "STRING"
                }
              + columns {
                  + name = "notification_created_at"
                  + type = "DATETIME"
                }
              + columns {
                  + name = "notification_sent_at"
                  + type = "DATETIME"
                }
              + columns {
                  + name = "notification_updated_at"
                  + type = "DATETIME"
                }
              + columns {
                  + name = "notification_queue_name"
                  + type = "STRING"
                }
              + columns {
                  + name = "notification_status"
                  + type = "STRING"
                }
              + columns {
                  + name = "notification_type"
                  + type = "STRING"
                }
              + columns {
                  + name = "notification_reference"
                  + type = "STRING"
                }
              + columns {
                  + name = "job_id"
                  + type = "STRING"
                }
              + columns {
                  + name = "api_key_id"
                  + type = "STRING"
                }
              + columns {
                  + name = "api_key_type"
                  + type = "STRING"
                }
              + columns {
                  + name = "service_id"
                  + type = "STRING"
                }
              + columns {
                  + name = "service_active"
                  + type = "BOOLEAN"
                }
              + columns {
                  + name = "service_count_as_live"
                  + type = "BOOLEAN"
                }
              + columns {
                  + name = "service_go_live_at"
                  + type = "DATETIME"
                }
              + columns {
                  + name = "service_name"
                  + type = "STRING"
                }
              + columns {
                  + name = "service_message_limit"
                  + type = "INTEGER"
                }
              + columns {
                  + name = "service_rate_limit"
                  + type = "INTEGER"
                }
              + columns {
                  + name = "service_sms_daily_limit"
                  + type = "INTEGER"
                }
              + columns {
                  + name = "organisation_name"
                  + type = "STRING"
                }
              + columns {
                  + name = "organisation_id"
                  + type = "STRING"
                }
              + columns {
                  + name = "template_id"
                  + type = "STRING"
                }
              + columns {
                  + name = "template_created_at"
                  + type = "DATETIME"
                }
              + columns {
                  + name = "template_name"
                  + type = "STRING"
                }
              + columns {
                  + name = "template_updated_at"
                  + type = "DATETIME"
                }
              + columns {
                  + name = "template_version"
                  + type = "INTEGER"
                }
              + columns {
                  + name = "tc_id"
                  + type = "STRING"
                }
              + columns {
                  + name = "tc_name_en"
                  + type = "STRING"
                }
              + columns {
                  + name = "tc_name_fr"
                  + type = "STRING"
                }
              + columns {
                  + name = "tc_email_process_type"
                  + type = "STRING"
                }
              + columns {
                  + name = "tc_sms_process_type"
                  + type = "STRING"
                }
              + columns {
                  + name = "tc_sms_sending_vehicle"
                  + type = "STRING"
                }
            }
        }

        # (4 unchanged blocks hidden)
    }

  # aws_s3_object.manifest_file will be updated in-place
  ~ resource "aws_s3_object" "manifest_file" {
      ~ etag                   = "4f558e8d8cdbbf914a95755cbda61968" -> "221f592f333f2fc284626cfdb8c4bc80"
        id                     = "quicksight/s3-manifest-sms-usage.json"
        tags                   = {}
      + version_id             = (known after apply)
        # (11 unchanged attributes hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Saved the plan to: plan.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "plan.tfplan"
Show Conftest results
WARN - plan.json - main - Missing Common Tags: ["aws_cloudformation_stack.sms-usage-notifications"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight-rds"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight-s3-usage"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight_vpc_connection_ec2"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_policy.quicksight_vpc_connection_iam"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_role.quicksight"]
WARN - plan.json - main - Missing Common Tags: ["aws_iam_role.vpc_connection_role"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.jobs"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.login_events"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.notifications"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.organisation"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.services"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.sms_usage"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.templates"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_set.users"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_source.rds"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_data_source.s3_sms_usage"]
WARN - plan.json - main - Missing Common Tags: ["aws_quicksight_vpc_connection.rds"]
WARN - plan.json - main - Missing Common Tags: ["aws_s3_object.manifest_file"]

38 tests, 19 passed, 19 warnings, 0 failures, 0 exceptions

@andrewleith andrewleith merged commit fec1e1d into main Jul 31, 2024
25 checks passed
@andrewleith andrewleith deleted the feat/update-quicksight-notifications branch July 31, 2024 12:43
@whabanks whabanks mentioned this pull request Aug 6, 2024
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants