Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enabling xray, disable annual limit #3213

Merged
merged 1 commit into from
Dec 17, 2024
Merged

enabling xray, disable annual limit #3213

merged 1 commit into from
Dec 17, 2024

Conversation

ben851
Copy link
Collaborator

@ben851 ben851 commented Dec 17, 2024

What happens when your PR merges?

Xray didn't have an effect on staging sending speeds. Will see if annual limits messes it up.

What are you changing?

  • Releasing a new version of Notify
  • Changing kubernetes configuration

@ben851 ben851 requested a review from jimleroyer as a code owner December 17, 2024 20:24
Copy link

xray-daemon	xray     	15      	2024-12-17 20:13:04.615970079 +0000 UTC	deployed	aws-xray-4.0.8	3.3.12     

ingress	nginx    	16      	2024-12-17 20:13:06.967267926 +0000 UTC	deployed	nginx-ingress-1.1.2	3.4.2      

Comparing release=notify-documentation, chart=charts/notify-documentation
Comparing release=notify-api, chart=charts/notify-api
notification-canada-ca, notify-api, Deployment (apps) has changed:
  # Source: notify-api/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: notify-api
    labels:
      app: notify-api
  spec:
    priorityClassName: high-priority
    strategy:
      
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    selector:
      matchLabels:
        app: notify-api
    template:
      metadata:
        labels:
          app: notify-api
      spec:
        serviceAccountName: notify-api
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        terminationGracePeriodSeconds: 60
        securityContext:
          {}
        initContainers:
          - name: init-postgres
            image: alpine
            command:
              [
                "sh",
                "-c",
                "until nslookup $POSTGRES_HOST; do echo waiting for postgres; sleep 2; done;",
              ]
            env:
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: POSTGRES_HOST   
          - name: migrate-db
            image: "public.ecr.aws/cds-snc/notify-api:latest"
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true   
            env:
              # Includes common ENV Variables
              - name: ADMIN_BASE_DOMAIN
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_DEBUG_ROUTE
                value: "true"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_SES_REGION
                value: "us-east-1"
              - name: AWS_SES_SMTP
                value: "email-smtp.us-east-1.amazonaws.com"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18005555555"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: AWS_XRAY_TRACING_ENABLED
-               value: "false"
+               value: "true"
              - name: BASE_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: CRM_ORG_LIST_URL
                value: "https://raw.githubusercontent.com/cds-snc/gc-organisations-qa/main/data/all.json"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENTATION_DOMAIN
                value: "documentation.staging.notification.cdssandbox.xyz"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "false"
              - name: FF_SALESFORCE_CONTACT
                value: "false"
              - name: FLASK_APP
                value: "application.py"
              - name: FRESH_DESK_API_URL
                value: "https://cds-snc.freshdesk.com"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_APP_NAME
                value: "notification-api-staging"
              - name: NEW_RELIC_CONFIG_FILE
                value: "/app/newrelic.ini"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "false"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "1"
              - name: SALESFORCE_DOMAIN
                value: "test"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: SQL_ALCHEMY_POOL_SIZE
                value: "256"
              - name: ZENDESK_API_URL
                value: "https://api.getbase.com"
              - name: ZENDESK_SELL_API_URL
                value: "https://cds-snc.zendesk.com"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: ADMIN_CLIENT_SECRET
              - name: AWS_ROUTE53_ZONE
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: AWS_ROUTE53_ZONE
              - name: AWS_SES_ACCESS_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: AWS_SES_ACCESS_KEY
              - name: AWS_SES_SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: AWS_SES_SECRET_KEY
              - name: CRM_GITHUB_PERSONAL_ACCESS_TOKEN
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: CRM_GITHUB_PERSONAL_ACCESS_TOKEN
              - name: CYPRESS_AUTH_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: CYPRESS_AUTH_CLIENT_SECRET
              - name: CYPRESS_USER_PW_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: CYPRESS_USER_PW_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: DANGEROUS_SALT
              - name: DEBUG_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: DEBUG_KEY
              - name: FRESH_DESK_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: FRESH_DESK_API_KEY
              - name: FRESH_DESK_PRODUCT_ID
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: FRESH_DESK_PRODUCT_ID
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: REDIS_URL
              - name: SALESFORCE_ENGAGEMENT_PRODUCT_ID
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_ENGAGEMENT_PRODUCT_ID
              - name: SALESFORCE_ENGAGEMENT_RECORD_TYPE
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_ENGAGEMENT_RECORD_TYPE
              - name: SALESFORCE_ENGAGEMENT_STANDARD_PRICEBOOK_ID
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_ENGAGEMENT_STANDARD_PRICEBOOK_ID
              - name: SALESFORCE_GENERIC_ACCOUNT_ID
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_GENERIC_ACCOUNT_ID
              - name: SALESFORCE_PASSWORD
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_PASSWORD
              - name: SALESFORCE_SECURITY_TOKEN
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_SECURITY_TOKEN
              - name: SALESFORCE_USERNAME
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_USERNAME
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SQLALCHEMY_DATABASE_URI
              - name: SRE_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SRE_CLIENT_SECRET
              - name: ZENDESK_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: ZENDESK_API_KEY
              - name: ZENDESK_SELL_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: ZENDESK_SELL_API_KEY
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
            command:
              [
                "sh",
                "-c",
                "flask db upgrade",
              ]
        containers:
          - name: notify-api
            securityContext:
              {}
            image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            env:
              # Includes common ENV Variables
              - name: ADMIN_BASE_DOMAIN
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_DEBUG_ROUTE
                value: "true"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_SES_REGION
                value: "us-east-1"
              - name: AWS_SES_SMTP
                value: "email-smtp.us-east-1.amazonaws.com"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18005555555"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: AWS_XRAY_TRACING_ENABLED
-               value: "false"
+               value: "true"
              - name: BASE_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: CRM_ORG_LIST_URL
                value: "https://raw.githubusercontent.com/cds-snc/gc-organisations-qa/main/data/all.json"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENTATION_DOMAIN
                value: "documentation.staging.notification.cdssandbox.xyz"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "false"
              - name: FF_SALESFORCE_CONTACT
                value: "false"
              - name: FLASK_APP
                value: "application.py"
              - name: FRESH_DESK_API_URL
                value: "https://cds-snc.freshdesk.com"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_APP_NAME
                value: "notification-api-staging"
              - name: NEW_RELIC_CONFIG_FILE
                value: "/app/newrelic.ini"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "false"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "1"
              - name: SALESFORCE_DOMAIN
                value: "test"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: SQL_ALCHEMY_POOL_SIZE
                value: "256"
              - name: ZENDESK_API_URL
                value: "https://api.getbase.com"
              - name: ZENDESK_SELL_API_URL
                value: "https://cds-snc.zendesk.com"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: ADMIN_CLIENT_SECRET
              - name: AWS_ROUTE53_ZONE
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: AWS_ROUTE53_ZONE
              - name: AWS_SES_ACCESS_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: AWS_SES_ACCESS_KEY
              - name: AWS_SES_SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: AWS_SES_SECRET_KEY
              - name: CRM_GITHUB_PERSONAL_ACCESS_TOKEN
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: CRM_GITHUB_PERSONAL_ACCESS_TOKEN
              - name: CYPRESS_AUTH_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: CYPRESS_AUTH_CLIENT_SECRET
              - name: CYPRESS_USER_PW_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: CYPRESS_USER_PW_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: DANGEROUS_SALT
              - name: DEBUG_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: DEBUG_KEY
              - name: FRESH_DESK_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: FRESH_DESK_API_KEY
              - name: FRESH_DESK_PRODUCT_ID
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: FRESH_DESK_PRODUCT_ID
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: REDIS_URL
              - name: SALESFORCE_ENGAGEMENT_PRODUCT_ID
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_ENGAGEMENT_PRODUCT_ID
              - name: SALESFORCE_ENGAGEMENT_RECORD_TYPE
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_ENGAGEMENT_RECORD_TYPE
              - name: SALESFORCE_ENGAGEMENT_STANDARD_PRICEBOOK_ID
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_ENGAGEMENT_STANDARD_PRICEBOOK_ID
              - name: SALESFORCE_GENERIC_ACCOUNT_ID
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_GENERIC_ACCOUNT_ID
              - name: SALESFORCE_PASSWORD
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_PASSWORD
              - name: SALESFORCE_SECURITY_TOKEN
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_SECURITY_TOKEN
              - name: SALESFORCE_USERNAME
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SALESFORCE_USERNAME
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SQLALCHEMY_DATABASE_URI
              - name: SRE_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: SRE_CLIENT_SECRET
              - name: ZENDESK_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: ZENDESK_API_KEY
              - name: ZENDESK_SELL_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-api
                    key: ZENDESK_SELL_API_KEY
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
                
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true          
               
            ports:
              - name: http
                containerPort: 6011
                protocol: TCP
            readinessProbe:
              httpGet:
                path: /_status?simple=true
                port: 6011
              initialDelaySeconds: 10
              periodSeconds: 3
              timeoutSeconds: 1
              successThreshold: 3
              failureThreshold: 10
            livenessProbe:
              httpGet:
                path: "/_status?simple=true"
                port: 6011
              initialDelaySeconds: 30
              periodSeconds: 3
              timeoutSeconds: 1
              successThreshold: 1
              failureThreshold: 3
            resources:
              limits:
                cpu: 1200m
                memory: 900Mi
              requests:
                cpu: 250m
                memory: 700Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-api
        
        nodeSelector:
          eks.amazonaws.com/capacityType: ON_DEMAND

Comparing release=notify-admin, chart=charts/notify-admin
notification-canada-ca, notify-admin, Deployment (apps) has changed:
  # Source: notify-admin/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: notify-admin
    labels:
      app: notify-admin
  spec:
    priorityClassName: high-priority
    strategy:
      
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    selector:
      matchLabels:
        app: notify-admin
    template:
      metadata:
        labels:
          app: notify-admin
      spec:
        serviceAccountName: notify-admin
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        terminationGracePeriodSeconds: 60
        securityContext:
          {}
        initContainers:
          - name: init-postgres
            image: alpine
            command:
              [
                "sh",
                "-c",
                "until nc -z -w 2 $API_HOST_NAME 443; do echo waiting for api; sleep 2; done"
              ]
            env:
              - name: API_HOST_NAME
                value: api.staging.notification.cdssandbox.xyz            
        containers:
          - name: notify-admin
            securityContext:
              {}
            image: "public.ecr.aws/cds-snc/notify-admin:latest"
            imagePullPolicy: Always
            env:
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "staging.notification.cdssandbox.xyz"
              - name: ALLOW_DEBUG_ROUTE
                value: "true"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: AWS_XRAY_TRACING_ENABLED
-               value: "false"
+               value: "true"
              - name: BASE_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: BULK_SEND_AWS_BUCKET
                value: "notification-canada-ca-staging-bulk-send"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CONTACT_EMAIL
                value: "[email protected]"
              - name: CRM_ORG_LIST_URL
                value: "https://raw.githubusercontent.com/cds-snc/gc-organisations/main/data/all.json"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENTATION_DOMAIN
                value: "https://documentation.staging.notification.cdssandbox.xyz"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_SALESFORCE_CONTACT
                value: "false"
              - name: FLASK_APP
                value: "application.py"
              - name: GC_ARTICLES_API
                value: "articles.alpha.canada.ca/notification-gc-notify"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: IP_GEOLOCATE_SERVICE
                value: "http://ipv4.notification-canada-ca.svc.cluster.local:8080"
              - name: NEW_RELIC_APP_NAME
                value: "notification-admin-staging"
              - name: NEW_RELIC_CONFIG_FILE
                value: "/app/newrelic.ini"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: SHOW_STYLE_GUIDE
                value: "true"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: ADMIN_CLIENT_SECRET
              - name: CRM_GITHUB_PERSONAL_ACCESS_TOKEN
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: CRM_GITHUB_PERSONAL_ACCESS_TOKEN
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: DANGEROUS_SALT
              - name: DEBUG_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: DEBUG_KEY
              - name: GC_ARTICLES_API_AUTH_PASSWORD
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: GC_ARTICLES_API_AUTH_PASSWORD
              - name: GC_ARTICLES_API_AUTH_USERNAME
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: GC_ARTICLES_API_AUTH_USERNAME
              - name: MIXPANEL_PROJECT_TOKEN
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: MIXPANEL_PROJECT_TOKEN
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: NEW_RELIC_LICENSE_KEY
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: SECRET_KEY
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: SQLALCHEMY_DATABASE_URI
              - name: WAF_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-admin
                    key: WAF_SECRET
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
                
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true          
               
            ports:
              - name: http
                containerPort: 6012
                protocol: TCP
            readinessProbe:
              httpGet:
                path: /_status?simple=true
                port: 6012
              initialDelaySeconds: 10
              periodSeconds: 3
              timeoutSeconds: 1
              successThreshold: 3
              failureThreshold: 10
            livenessProbe:
              httpGet:
                path: "/_status?simple=true"
                port: 6012
              initialDelaySeconds: 30
              periodSeconds: 3
              timeoutSeconds: 1
              successThreshold: 1
              failureThreshold: 3
            resources:
              limits:
                cpu: 1200m
                memory: 900Mi
              requests:
                cpu: 250m
                memory: 700Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-admin
        
        nodeSelector:
          eks.amazonaws.com/capacityType: ON_DEMAND

Comparing release=notify-document-download, chart=charts/notify-document-download
notification-canada-ca, notify-document-download, Deployment (apps) has changed:
  # Source: notify-document-download/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: notify-document-download
    labels:
      app: notify-document-download
  spec:
    selector:
      matchLabels:
        app.kubernetes.io/name: notify-document-download
        app.kubernetes.io/instance: notify-document-download
    template:
      metadata:
        labels:
          helm.sh/chart: notify-document-download-0.1.0
          app.kubernetes.io/name: notify-document-download
          app.kubernetes.io/instance: notify-document-download
          app.kubernetes.io/version: "1.16.0"
          app.kubernetes.io/managed-by: Helm
          app: notify-document-download
      spec:
        serviceAccountName: notify-document-download
        securityContext:
          {}
        containers:
          - name: notify-document-download
            securityContext:
              {}
            image: "public.ecr.aws/cds-snc/notify-document-download-api:latest"
            imagePullPolicy: Always
            env:
              # NON-SECRET CONFIGURATIONS
              - name: ALLOW_DEBUG_ROUTE
                value: "true"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BACKEND_HOSTNAME
                value: "api.document.staging.notification.cdssandbox.xyz"
              - name: DEBUG
                value: "false"
              - name: DOCUMENTS_BUCKET
                value: "notification-canada-ca-staging-document-download"
              - name: EXTRA_MIME_TYPES
                value: "b7b2104c-011f-436a-a25f-3fd66b6591e4:application/octet-stream,b7b2104c-011f-436a-a25f-3fd66b6591e4:application/xml,b7b2104c-011f-436a-a25f-3fd66b6591e4:text/xml,b7b2104c-011f-436a-a25f-3fd66b6591e4:application/json,dea2d718-b7fb-4003-868a-1832fd025d7a:image/svg+xml"
              - name: HTTP_SCHEME
                value: "https"
              - name: NEW_RELIC_APP_NAME
                value: "notification-dd-api-staging"
              - name: NEW_RELIC_CONFIG_FILE
                value: "/app/newrelic.ini"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: SCAN_FILES_DOCUMENTS_BUCKET
                value: "notification-canada-ca-staging-document-download-scan-files"

            # SECRET CONFIGURATIONS
              - name: AUTH_TOKENS
                valueFrom:
                  secretKeyRef:
                    name: document-download
                    key: AUTH_TOKENS
              - name: DEBUG_KEY
                valueFrom:
                  secretKeyRef:
                    name: document-download
                    key: DEBUG_KEY
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: document-download
                    key: NEW_RELIC_LICENSE_KEY
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: document-download
                    key: SECRET_KEY
                
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true          
              
            ports:
              - name: http
                containerPort: 7000
                protocol: TCP
            livenessProbe:
              httpGet:
                path: /_status
                port: 7000
              initialDelaySeconds: 30
            readinessProbe:
              httpGet:
                path: /_status
                port: 7000
              initialDelaySeconds: 30
            resources:
              limits:
                cpu: 400m
                memory: 800Mi
              requests:
                cpu: 100m
                memory: 400Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-document-download
        
        nodeSelector:
          eks.amazonaws.com/capacityType: ON_DEMAND

Comparing release=notify-celery, chart=charts/notify-celery
notification-canada-ca, notify-celery-beat, Deployment (apps) has changed:
  # Source: notify-celery/templates/deployment-beat.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      app: notify-celery-beat
    name: notify-celery-beat
    namespace: notification-canada-ca
  spec:
    replicas: 3
    revisionHistoryLimit: 5
    selector:
      matchLabels:
        app: notify-celery-beat
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: notify-celery-beat
      spec:
        nodeSelector: 
          eks.amazonaws.com/capacityType: ON_DEMAND
        serviceAccountName: notify-celery
        priorityClassName: 
        initContainers:
          - name: init-postgres
            image: alpine
            command:
              [
                "sh",
                "-c",
                "until nslookup $(POSTGRES_HOST); do echo waiting for postgres; sleep 2; done;",
              ]
            env:  
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
        containers:
          - image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            name: notify-celery-beat
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true  
            env:
              # Includes specific ENV Variables
              - name: NEW_RELIC_APP_NAME
                value: notification-celery-beat-dev
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18449521252"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CELERY_CONCURRENCY
                value: "4"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "True"
              - name: FIDO2_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_EMAIL_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: env
                value: "staging"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: ADMIN_CLIENT_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: DANGEROUS_SALT
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_URI
            command: ["/bin/sh"]
            args: ["-c", "sh /app/scripts/run_celery_beat.sh"]
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-celery
           
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 30
  status: {}
notification-canada-ca, notify-celery-email-send-primary, Deployment (apps) has changed:
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      app: notify-celery-email-send-primary
    name:  notify-celery-email-send-primary
    namespace: notification-canada-ca
  spec:
    replicas: 3
    revisionHistoryLimit: 5
    selector:
      matchLabels:
        app: notify-celery-email-send-primary
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: notify-celery-email-send-primary
      spec:
        nodeSelector: 
          eks.amazonaws.com/capacityType: ON_DEMAND
        serviceAccountName: notify-celery
        priorityClassName: high-priority
        initContainers:
        - name: wait-cwagent-ready
          env:
            - name: STATSD_HOST
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          image: "public.ecr.aws/cds-snc/notify-api:latest"
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh"]
          args: ["-c", "sh /app/scripts/cwcheck.sh"]
        containers:
          - name: notify-celery-email-send-primary
            image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true  
            env:
              # Includes specific ENV Variables
              - name: NEW_RELIC_APP_NAME
                value: notification-celery-email-send-primary-dev
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18449521252"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CELERY_CONCURRENCY
                value: "4"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "True"
              - name: FIDO2_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_EMAIL_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: env
                value: "staging"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: ADMIN_CLIENT_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: DANGEROUS_SALT
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_URI
            lifecycle:
              preStop:
                exec:
                  command:
                  - /bin/bash
                  - -c
                  - /app/scripts/run_celery_exit.sh
            command: ["/bin/sh"]
            args: ["-c", "sh /app/scripts/run_celery_send_email.sh"]
            resources:
              requests:
                cpu: 100m
                memory: 500Mi
              limits:
                cpu: 550m
                memory: 1024Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-celery
           
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 60
  status: {}
notification-canada-ca, notify-celery-email-send-scalable, Deployment (apps) has changed:
  # Source: notify-celery/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      app: notify-celery-email-send-scalable
    name:  notify-celery-email-send-scalable
    namespace: notification-canada-ca
  spec:
    replicas: 3
    revisionHistoryLimit: 5
    selector:
      matchLabels:
        app: notify-celery-email-send-scalable
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: notify-celery-email-send-scalable
      spec:
        nodeSelector: 
          karpenter.sh/capacity-type: spot
        serviceAccountName: notify-celery
        priorityClassName: celery
        initContainers:
        - name: wait-cwagent-ready
          env:
            - name: STATSD_HOST
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          image: "public.ecr.aws/cds-snc/notify-api:latest"
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh"]
          args: ["-c", "sh /app/scripts/cwcheck.sh"]
        containers:
          - name: notify-celery-email-send-scalable
            image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true  
            env:
              # Includes specific ENV Variables
              - name: NEW_RELIC_APP_NAME
                value: notification-celery-email-send-scalable-dev
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18449521252"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CELERY_CONCURRENCY
                value: "4"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "True"
              - name: FIDO2_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_EMAIL_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: env
                value: "staging"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: ADMIN_CLIENT_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: DANGEROUS_SALT
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_URI
            lifecycle:
              preStop:
                exec:
                  command:
                  - /bin/bash
                  - -c
                  - /app/scripts/run_celery_exit.sh
            command: ["/bin/sh"]
            args: ["-c", "sh /app/scripts/run_celery_send_email.sh"]
            resources:
              requests:
                cpu: 100m
                memory: 500Mi
              limits:
                cpu: 550m
                memory: 1024Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-celery
           
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 60
  status: {}
notification-canada-ca, notify-celery-primary, Deployment (apps) has changed:
  # Source: notify-celery/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      app: notify-celery-primary
    name:  notify-celery-primary
    namespace: notification-canada-ca
  spec:
    replicas: 3
    revisionHistoryLimit: 5
    selector:
      matchLabels:
        app: notify-celery-primary
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: notify-celery-primary
      spec:
        nodeSelector: 
          eks.amazonaws.com/capacityType: ON_DEMAND
        serviceAccountName: notify-celery
        priorityClassName: high-priority
        initContainers:
        - name: wait-cwagent-ready
          env:
            - name: STATSD_HOST
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          image: "public.ecr.aws/cds-snc/notify-api:latest"
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh"]
          args: ["-c", "sh /app/scripts/cwcheck.sh"]
        containers:
          - name: notify-celery-primary
            image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true  
            env:
              # Includes specific ENV Variables
              - name: NEW_RELIC_APP_NAME
                value: notification-celery-main-primary-dev
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18449521252"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CELERY_CONCURRENCY
                value: "4"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "True"
              - name: FIDO2_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_EMAIL_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: env
                value: "staging"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: ADMIN_CLIENT_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: DANGEROUS_SALT
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_URI
            lifecycle:
              preStop:
                exec:
                  command:
                  - /bin/bash
                  - -c
                  - /app/scripts/run_celery_exit.sh
            command: ["/bin/sh"]
            args: ["-c", "sh /app/scripts/run_celery_core_tasks.sh"]
            resources:
              requests:
                cpu: 100m
                memory: 500Mi
              limits:
                cpu: 550m
                memory: 1024Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-celery
           
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 60
  status: {}
notification-canada-ca, notify-celery-scalable, Deployment (apps) has changed:
  # Source: notify-celery/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      app: notify-celery-scalable
    name:  notify-celery-scalable
    namespace: notification-canada-ca
  spec:
    replicas: 3
    revisionHistoryLimit: 5
    selector:
      matchLabels:
        app: notify-celery-scalable
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: notify-celery-scalable
      spec:
        nodeSelector: 
          karpenter.sh/capacity-type: spot
        serviceAccountName: notify-celery
        priorityClassName: celery
        initContainers:
        - name: wait-cwagent-ready
          env:
            - name: STATSD_HOST
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          image: "public.ecr.aws/cds-snc/notify-api:latest"
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh"]
          args: ["-c", "sh /app/scripts/cwcheck.sh"]
        containers:
          - name: notify-celery-scalable
            image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true  
            env:
              # Includes specific ENV Variables
              - name: NEW_RELIC_APP_NAME
                value: notification-celery-main-scalable-dev
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18449521252"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CELERY_CONCURRENCY
                value: "4"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "True"
              - name: FIDO2_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_EMAIL_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: env
                value: "staging"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: ADMIN_CLIENT_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: DANGEROUS_SALT
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_URI
            lifecycle:
              preStop:
                exec:
                  command:
                  - /bin/bash
                  - -c
                  - /app/scripts/run_celery_exit.sh
            command: ["/bin/sh"]
            args: ["-c", "sh /app/scripts/run_celery_core_tasks.sh"]
            resources:
              requests:
                cpu: 100m
                memory: 500Mi
              limits:
                cpu: 550m
                memory: 1024Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-celery
           
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 60
  status: {}
notification-canada-ca, notify-celery-sms, Deployment (apps) has changed:
  # Source: notify-celery/templates/deployment-beat.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      app: notify-celery-sms
    name: notify-celery-sms
    namespace: notification-canada-ca
  spec:
    replicas: 3
    revisionHistoryLimit: 5
    selector:
      matchLabels:
        app: notify-celery-sms
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: notify-celery-sms
      spec:
        nodeSelector: 
          eks.amazonaws.com/capacityType: ON_DEMAND
        serviceAccountName: notify-celery
        priorityClassName: 
        initContainers:
          - name: init-postgres
            image: alpine
            command:
              [
                "sh",
                "-c",
                "until nslookup $(POSTGRES_HOST); do echo waiting for postgres; sleep 2; done;",
              ]
            env:  
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
        containers:
          - image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            name: notify-celery-sms
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true  
            env:
              # Includes specific ENV Variables
              - name: NEW_RELIC_APP_NAME
                value: notification-celery-sms-dev
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18449521252"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CELERY_CONCURRENCY
                value: "4"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "True"
              - name: FIDO2_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_EMAIL_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: env
                value: "staging"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: ADMIN_CLIENT_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: DANGEROUS_SALT
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_URI
            command: ["/bin/sh"]
            args: ["-c", "sh /app/scripts/run_celery_beat.sh"]
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-celery
           
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 60
  status: {}
notification-canada-ca, notify-celery-sms-send-primary, Deployment (apps) has changed:
  # Source: notify-celery/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      app: notify-celery-sms-send-primary
    name:  notify-celery-sms-send-primary
    namespace: notification-canada-ca
  spec:
    replicas: 3
    revisionHistoryLimit: 5
    selector:
      matchLabels:
        app: notify-celery-sms-send-primary
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: notify-celery-sms-send-primary
      spec:
        nodeSelector: 
          eks.amazonaws.com/capacityType: ON_DEMAND
        serviceAccountName: notify-celery
        priorityClassName: high-priority
        initContainers:
        - name: wait-cwagent-ready
          env:
            - name: STATSD_HOST
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          image: "public.ecr.aws/cds-snc/notify-api:latest"
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh"]
          args: ["-c", "sh /app/scripts/cwcheck.sh"]
        containers:
          - name: notify-celery-sms-send-primary
            image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true  
            env:
              # Includes specific ENV Variables
              - name: NEW_RELIC_APP_NAME
                value: notification-celery-sms-send-primary-dev
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18449521252"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CELERY_CONCURRENCY
                value: "4"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "True"
              - name: FIDO2_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_EMAIL_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: env
                value: "staging"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: ADMIN_CLIENT_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: DANGEROUS_SALT
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_URI
            lifecycle:
              preStop:
                exec:
                  command:
                  - /bin/bash
                  - -c
                  - /app/scripts/run_celery_exit.sh
            command: ["/bin/sh"]
            args: ["-c", "sh /app/scripts/run_celery_send_sms.sh"]
            resources:
              requests:
                cpu: 100m
                memory: 500Mi
              limits:
                cpu: 550m
                memory: 1024Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-celery
           
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 60
  status: {}
notification-canada-ca, notify-celery-sms-send-scalable, Deployment (apps) has changed:
  # Source: notify-celery/templates/deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      app: notify-celery-sms-send-scalable
    name:  notify-celery-sms-send-scalable
    namespace: notification-canada-ca
  spec:
    replicas: 3
    revisionHistoryLimit: 5
    selector:
      matchLabels:
        app: notify-celery-sms-send-scalable
    strategy:
      rollingUpdate:
        maxSurge: 25%
        maxUnavailable: 25%
      type: RollingUpdate
    template:
      metadata:
        labels:
          app: notify-celery-sms-send-scalable
      spec:
        nodeSelector: 
          karpenter.sh/capacity-type: spot
        serviceAccountName: notify-celery
        priorityClassName: celery
        initContainers:
        - name: wait-cwagent-ready
          env:
            - name: STATSD_HOST
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          image: "public.ecr.aws/cds-snc/notify-api:latest"
          imagePullPolicy: IfNotPresent
          command: ["/bin/sh"]
          args: ["-c", "sh /app/scripts/cwcheck.sh"]
        containers:
          - name: notify-celery-sms-send-scalable
            image: "public.ecr.aws/cds-snc/notify-api:latest"
            imagePullPolicy: Always
            volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true  
            env:
              # Includes specific ENV Variables
              - name: NEW_RELIC_APP_NAME
                value: notification-celery-sms-send-scalable-dev
              - name: STATSD_HOST
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              # Includes common ENV Variables
              - name: ADMIN_BASE_URL
                value: "https://staging.notification.cdssandbox.xyz"
              - name: ALLOW_HTML_SERVICE_IDS
                value: "4de8b784-03a8-4ba8-a440-3bfea1b04fe6,ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: API_HOST_NAME
                value: "https://api.staging.notification.cdssandbox.xyz"
              - name: ASSET_DOMAIN
                value: "https://assets.staging.notification.cdssandbox.xyz"
              - name: ASSET_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-asset-upload"
              - name: AWS_PINPOINT_REGION
                value: "us-west-2"
              - name: AWS_REGION
                value: "ca-central-1"
              - name: AWS_US_TOLL_FREE_NUMBER
                value: "+18449521252"
              - name: AWS_XRAY_CONTEXT_MISSING
                value: "LOG_WARNING"
              - name: AWS_XRAY_DAEMON_ADDRESS
                value: "xray-daemon-aws-xray.xray.svc.cluster.local:2000"
              - name: AWS_XRAY_SDK_ENABLED
-               value: "false"
+               value: "true"
              - name: BATCH_INSERTION_CHUNK_SIZE
                value: "10"
              - name: BULK_SEND_TEST_SERVICE_ID
                value: "ea608120-148a-4eba-a64c-4d9a8010e7b0"
              - name: CELERY_CONCURRENCY
                value: "4"
              - name: CSV_UPLOAD_BUCKET_NAME
                value: "notification-canada-ca-staging-csv-upload"
              - name: DOCUMENT_DOWNLOAD_API_HOST
                value: "http://notify-document-download.notification-canada-ca.svc.cluster.local:7000"
              - name: FF_ANNUAL_LIMIT
-               value: "true"
+               value: "false"
              - name: FF_CLOUDWATCH_METRICS_ENABLED
                value: "True"
              - name: FIDO2_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: HC_EN_SERVICE_ID
                value: "c2fe9fac-2f28-40ca-b152-08ee41cd6843"
              - name: HC_FR_SERVICE_ID
                value: "changeme"
              - name: NEW_RELIC_DISTRIBUTED_TRACING_ENABLED
                value: "true"
              - name: NEW_RELIC_MONITOR_MODE
                value: "true"
              - name: NOTIFICATION_QUEUE_PREFIX
                value: "eks-notification-canada-ca"
              - name: NOTIFY_EMAIL_DOMAIN
                value: "staging.notification.cdssandbox.xyz"
              - name: NOTIFY_ENVIRONMENT
                value: "staging"
              - name: REDIS_ENABLED
                value: "true"
              - name: SENTRY_URL
                value: "https://[email protected]/1522933"
              - name: env
                value: "staging"
              # Includes secret ENV Variables
              - name: ADMIN_CLIENT_SECRET
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: ADMIN_CLIENT_SECRET
              - name: DANGEROUS_SALT
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: DANGEROUS_SALT
              - name: NEW_RELIC_LICENSE_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: NEW_RELIC_LICENSE_KEY
              - name: POSTGRES_HOST
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: POSTGRES_HOST
              - name: REDIS_PUBLISH_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_PUBLISH_URL
              - name: REDIS_URL
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: REDIS_URL
              - name: SECRET_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SECRET_KEY
              - name: SENDGRID_API_KEY
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SENDGRID_API_KEY
              - name: SQLALCHEMY_DATABASE_READER_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_READER_URI
              - name: SQLALCHEMY_DATABASE_URI
                valueFrom:
                  secretKeyRef:
                    name: notify-celery
                    key: SQLALCHEMY_DATABASE_URI
            lifecycle:
              preStop:
                exec:
                  command:
                  - /bin/bash
                  - -c
                  - /app/scripts/run_celery_exit.sh
            command: ["/bin/sh"]
            args: ["-c", "sh /app/scripts/run_celery_send_sms.sh"]
            resources:
              requests:
                cpu: 100m
                memory: 500Mi
              limits:
                cpu: 550m
                memory: 1024Mi
            
        volumes:
          - name: secrets-store-inline
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: notify-celery
           
        dnsPolicy: ClusterFirst
        restartPolicy: Always
        schedulerName: default-scheduler
        securityContext: {}
        terminationGracePeriodSeconds: 60
  status: {}

Comparing release=k8s-event-logger, chart=/tmp/helmfile2904080160/amazon-cloudwatch/staging/k8s-event-logger/k8s-event-logger/1.1.8/k8s-event-logger
Comparing release=karpenter-crd, chart=/tmp/helmfile2904080160/karpenter/staging/karpenter-crd/karpenter-crd/0.36.1/karpenter-crd
Comparing release=karpenter, chart=/tmp/helmfile2904080160/karpenter/staging/karpenter/karpenter/0.36.1/karpenter
Comparing release=karpenter-nodepool, chart=charts/karpenter-nodepool
Comparing release=priority-classes, chart=deliveryhero/priority-class
Comparing release=secrets-store-csi-driver, chart=secrets-store-csi-driver/secrets-store-csi-driver
Comparing release=aws-secrets-provider, chart=aws-secrets-manager/secrets-store-csi-driver-provider-aws
Comparing release=kube-state-metrics, chart=prometheus-community/kube-state-metrics
Comparing release=blazer, chart=stakater/application
Comparing release=ingress, chart=charts/nginx-ingress
Comparing release=xray-daemon, chart=okgolove/aws-xray

@ben851 ben851 merged commit 16d8b40 into main Dec 17, 2024
3 checks passed
@ben851 ben851 deleted the xray-annual branch December 17, 2024 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants