Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Helm Migration for Staging #3186

Merged
merged 18 commits into from
Dec 12, 2024
Merged

Helm Migration for Staging #3186

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
1323387
turning on targetgroup bindings and turning on staging for helmfile
P0NDER0SA Dec 2, 2024
87d6b71
adding dummy services back to celery so that our metrics can read them
P0NDER0SA Dec 3, 2024
a6e713d
Merge branch 'celery-services' into helm-migration-staging-tgb-enabled
P0NDER0SA Dec 3, 2024
bcf3e7d
just makign this consistent for rollout
P0NDER0SA Dec 3, 2024
3a75fa3
Merge remote-tracking branch 'origin/main' into helm-migration-stagin…
ben851 Dec 5, 2024
704924e
Merge branch 'main' into helm-migration-staging-tgb-enabled
P0NDER0SA Dec 10, 2024
3794548
Merge branch 'helm-migration-staging-tgb-enabled' of github.com:cds-s…
P0NDER0SA Dec 10, 2024
18f289a
fixing celery hpa
P0NDER0SA Dec 10, 2024
075b794
removing kustomize apply ins staging workflow
P0NDER0SA Dec 10, 2024
c9fb2c7
turning off kustomize deployments for our helm migrated components on…
P0NDER0SA Dec 10, 2024
59f32e8
Merge branch 'main' into helm-migration-staging-tgb-enabled
P0NDER0SA Dec 11, 2024
ac720d4
Merge branch 'main' into helm-migration-staging-tgb-enabled
P0NDER0SA Dec 12, 2024
8414414
Merge branch 'main' into helm-migration-staging-tgb-enabled
P0NDER0SA Dec 12, 2024
f08f30f
fixing xray url
ben851 Dec 12, 2024
9cbdebd
cleaning up env vars thing
ben851 Dec 12, 2024
c22dbbd
removing syntax check since helmfile auto does it
ben851 Dec 12, 2024
beca1fa
switch to helmfile sync
ben851 Dec 12, 2024
15dcb8b
xray url fix again
ben851 Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/helmfile_production_apply.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ jobs:
id: helmfile_apply
run: |
pushd helmfile
helmfile --environment production apply
helmfile --environment production sync
popd

- name: Notify Slack channel if this job failed
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/helmfile_staging_apply.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ jobs:
id: helmfile_apply
run: |
pushd helmfile
helmfile --environment staging apply
helmfile --environment staging sync
popd

- name: Notify Slack channel if this job failed
Expand Down
30 changes: 0 additions & 30 deletions .github/workflows/syntax_check.yaml
View file
Open in desktop

This file was deleted.

35 changes: 4 additions & 31 deletions .github/workflows/test_env_vars.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,13 @@ on:
pull_request:
branches:
- main
paths:
- "VERSION"
push:
branches:
- main
paths:
- "VERSION"

defaults:
run:
Expand All @@ -32,35 +36,13 @@ jobs:
aws-secret-access-key: ${{ secrets.STAGING_AWS_SECRET_ACCESS_KEY }}
aws-region: ca-central-1

- name: Get staging env var keys
run: |
make decrypt-staging
ENV_KEYS_STAGING="$(make env-keys-staging)"
echo "::add-mask::$ENV_KEYS_STAGING"
echo "ENV_KEYS_STAGING=$ENV_KEYS_STAGING" >> $GITHUB_ENV

- name: Configure AWS production credentials
uses: aws-actions/configure-aws-credentials@5fd3084fc36e372ff1fff382a39b10d03659f355 # v2.2.0
with:
aws-access-key-id: ${{ secrets.PRODUCTION_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.PRODUCTION_AWS_SECRET_ACCESS_KEY }}
aws-region: ca-central-1

- name: Get production env var keys
run: |
make decrypt-production
ENV_KEYS_PRODUCTION="$(make env-keys-production)"
echo "::add-mask::$ENV_KEYS_PRODUCTION"
echo "ENV_KEYS_PRODUCTION=$ENV_KEYS_PRODUCTION" >> $GITHUB_ENV

- name: Test example and staging env vars
uses: ./.github/actions/diff-env-vars
with:
env1-name: EXAMPLE
env1-vars: ${{ env.ENV_KEYS_EXAMPLE }}
env2-name: STAGING
env2-vars: ${{ env.ENV_KEYS_STAGING }}

- name: Test example and production env vars
uses: ./.github/actions/diff-env-vars
if: always()
Expand All @@ -69,12 +51,3 @@ jobs:
env1-vars: ${{ env.ENV_KEYS_EXAMPLE }}
env2-name: PRODUCTION
env2-vars: ${{ env.ENV_KEYS_PRODUCTION }}

- name: Test staging and production env vars
uses: ./.github/actions/diff-env-vars
if: always()
with:
env1-name: STAGING
env1-vars: ${{ env.ENV_KEYS_STAGING }}
env2-name: PRODUCTION
env2-vars: ${{ env.ENV_KEYS_PRODUCTION }}
100 changes: 50 additions & 50 deletions env/dev/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,23 @@ resources:
- fluentbit.yaml
- cwagent.yaml
- cwagent-configmap.yaml
- notification-service-account.yaml
- api-target-group.yaml
- admin-target-group.yaml
- document-download-api-target-group.yaml
- documentation-target-group.yaml
# - notification-service-account.yaml
# - api-target-group.yaml
# - admin-target-group.yaml
# - document-download-api-target-group.yaml
# - documentation-target-group.yaml
- ../../base/prometheus-cloudwatch
- ../../base/notify-admin
- ../../base/notify-api
- ../../base/notify-celery-other
- ../../base/notify-celery-main-primary
- ../../base/notify-celery-sms-send-primary
- ../../base/notify-celery-email-send-primary
- ../../base/notify-celery-main-scalable
- ../../base/notify-celery-sms-send-scalable
- ../../base/notify-celery-email-send-scalable
- ../../base/notify-document-download
- ../../base/notify-documentation
# - ../../base/notify-admin
# - ../../base/notify-api
# - ../../base/notify-celery-other
# - ../../base/notify-celery-main-primary
# - ../../base/notify-celery-sms-send-primary
# - ../../base/notify-celery-email-send-primary
# - ../../base/notify-celery-main-scalable
# - ../../base/notify-celery-sms-send-scalable
# - ../../base/notify-celery-email-send-scalable
# - ../../base/notify-document-download
# - ../../base/notify-documentation
- ../../base/notify-system

images:
Expand All @@ -34,45 +34,45 @@ images:
- name: documentation
newName: public.ecr.aws/cds-snc/notify-documentation:latest

patches:
- path: performance/admin-hpa-patch.yaml
- path: performance/api-hpa-patch.yaml
- path: performance/celery-email-send-primary-deployment-patch.yaml
- path: performance/celery-email-send-scalable-deployment-patch.yaml
- path: performance/celery-email-send-scalable-hpa-patch.yaml
- path: performance/celery-primary-deployment-patch.yaml
- path: performance/celery-scalable-deployment-patch.yaml
- path: performance/celery-scalable-hpa-patch.yaml
- path: performance/celery-sms-send-primary-deployment-patch.yaml
- path: performance/celery-sms-send-scalable-deployment-patch.yaml
- path: performance/celery-sms-send-scalable-hpa-patch.yaml
# patches:
# - path: performance/admin-hpa-patch.yaml
# - path: performance/api-hpa-patch.yaml
# - path: performance/celery-email-send-primary-deployment-patch.yaml
# - path: performance/celery-email-send-scalable-deployment-patch.yaml
# - path: performance/celery-email-send-scalable-hpa-patch.yaml
# - path: performance/celery-primary-deployment-patch.yaml
# - path: performance/celery-scalable-deployment-patch.yaml
# - path: performance/celery-scalable-hpa-patch.yaml
# - path: performance/celery-sms-send-primary-deployment-patch.yaml
# - path: performance/celery-sms-send-scalable-deployment-patch.yaml
# - path: performance/celery-sms-send-scalable-hpa-patch.yaml

- path: services/admin-service-patch.yaml
- path: services/api-service-patch.yaml
- path: services/document-download-api-service-patch.yaml
- path: services/documentation-service-patch.yaml
# - path: services/admin-service-patch.yaml
# - path: services/api-service-patch.yaml
# - path: services/document-download-api-service-patch.yaml
# - path: services/documentation-service-patch.yaml

- path: cwagent/cwagent-deployment-patch.yaml

- path: nodeselectors/admin-node-selector-patch.yaml
- path: nodeselectors/celery-api-node-selector-patch.yaml
- path: nodeselectors/celery-beat-node-selector-patch.yaml
- path: nodeselectors/celery-email-send-node-selector-patch.yaml
- path: nodeselectors/celery-email-send-scalable-node-selector-patch.yaml
- path: nodeselectors/celery-primary-node-selector-patch.yaml
- path: nodeselectors/celery-scalable-node-selector-patch.yaml
- path: nodeselectors/celery-sms-node-selector-patch.yaml
- path: nodeselectors/celery-sms-send-node-selector-patch.yaml
- path: nodeselectors/celery-sms-send-scalable-node-selector-patch.yaml
- path: nodeselectors/document-download-api-node-selector-patch.yaml
- path: nodeselectors/documentation-node-selector-patch.yaml
# - path: nodeselectors/admin-node-selector-patch.yaml
# - path: nodeselectors/celery-api-node-selector-patch.yaml
# - path: nodeselectors/celery-beat-node-selector-patch.yaml
# - path: nodeselectors/celery-email-send-node-selector-patch.yaml
# - path: nodeselectors/celery-email-send-scalable-node-selector-patch.yaml
# - path: nodeselectors/celery-primary-node-selector-patch.yaml
# - path: nodeselectors/celery-scalable-node-selector-patch.yaml
# - path: nodeselectors/celery-sms-node-selector-patch.yaml
# - path: nodeselectors/celery-sms-send-node-selector-patch.yaml
# - path: nodeselectors/celery-sms-send-scalable-node-selector-patch.yaml
# - path: nodeselectors/document-download-api-node-selector-patch.yaml
# - path: nodeselectors/documentation-node-selector-patch.yaml

- path: celery-init-delete/celery-email-send-primary-init-delete-patch.yaml
- path: celery-init-delete/celery-email-send-scalable-init-delete-patch.yaml
- path: celery-init-delete/celery-primary-init-delete-patch.yaml
- path: celery-init-delete/celery-scalable-init-delete-patch.yaml
- path: celery-init-delete/celery-sms-send-primary-init-delete-patch.yaml
- path: celery-init-delete/celery-sms-send-scalable-init-delete-patch.yaml
# - path: celery-init-delete/celery-email-send-primary-init-delete-patch.yaml
# - path: celery-init-delete/celery-email-send-scalable-init-delete-patch.yaml
# - path: celery-init-delete/celery-primary-init-delete-patch.yaml
# - path: celery-init-delete/celery-scalable-init-delete-patch.yaml
# - path: celery-init-delete/celery-sms-send-primary-init-delete-patch.yaml
# - path: celery-init-delete/celery-sms-send-scalable-init-delete-patch.yaml

configMapGenerator:
- envs:
Expand Down
98 changes: 49 additions & 49 deletions env/sandbox/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,24 @@ resources:
- fluentbit.yaml
- cwagent.yaml
- cwagent-configmap.yaml
- notification-service-account.yaml
- api-target-group.yaml
- admin-target-group.yaml
- document-download-api-target-group.yaml
- documentation-target-group.yaml
# - notification-service-account.yaml
# - api-target-group.yaml
# - admin-target-group.yaml
# - document-download-api-target-group.yaml
# - documentation-target-group.yaml
- aws-auth-configmap.yaml
- ../../base/prometheus-cloudwatch
- ../../base/notify-admin
- ../../base/notify-api
- ../../base/notify-celery-other
- ../../base/notify-celery-main-primary
- ../../base/notify-celery-sms-send-primary
- ../../base/notify-celery-email-send-primary
- ../../base/notify-celery-main-scalable
- ../../base/notify-celery-sms-send-scalable
- ../../base/notify-celery-email-send-scalable
- ../../base/notify-document-download
- ../../base/notify-documentation
# - ../../base/notify-admin
# - ../../base/notify-api
# - ../../base/notify-celery-other
# - ../../base/notify-celery-main-primary
# - ../../base/notify-celery-sms-send-primary
# - ../../base/notify-celery-email-send-primary
# - ../../base/notify-celery-main-scalable
# - ../../base/notify-celery-sms-send-scalable
# - ../../base/notify-celery-email-send-scalable
# - ../../base/notify-document-download
# - ../../base/notify-documentation
- ../../base/notify-system

images:
Expand All @@ -36,44 +36,44 @@ images:
newName: public.ecr.aws/cds-snc/notify-documentation:latest

patches:
- path: performance/admin-hpa-patch.yaml
- path: performance/api-hpa-patch.yaml
- path: performance/celery-email-send-primary-deployment-patch.yaml
- path: performance/celery-email-send-scalable-deployment-patch.yaml
- path: performance/celery-email-send-scalable-hpa-patch.yaml
- path: performance/celery-primary-deployment-patch.yaml
- path: performance/celery-scalable-deployment-patch.yaml
- path: performance/celery-scalable-hpa-patch.yaml
- path: performance/celery-sms-send-primary-deployment-patch.yaml
- path: performance/celery-sms-send-scalable-deployment-patch.yaml
- path: performance/celery-sms-send-scalable-hpa-patch.yaml
# - path: performance/admin-hpa-patch.yaml
# - path: performance/api-hpa-patch.yaml
# - path: performance/celery-email-send-primary-deployment-patch.yaml
# - path: performance/celery-email-send-scalable-deployment-patch.yaml
# - path: performance/celery-email-send-scalable-hpa-patch.yaml
# - path: performance/celery-primary-deployment-patch.yaml
# - path: performance/celery-scalable-deployment-patch.yaml
# - path: performance/celery-scalable-hpa-patch.yaml
# - path: performance/celery-sms-send-primary-deployment-patch.yaml
# - path: performance/celery-sms-send-scalable-deployment-patch.yaml
# - path: performance/celery-sms-send-scalable-hpa-patch.yaml

- path: services/admin-service-patch.yaml
- path: services/api-service-patch.yaml
- path: services/document-download-api-service-patch.yaml
- path: services/documentation-service-patch.yaml
# - path: services/admin-service-patch.yaml
# - path: services/api-service-patch.yaml
# - path: services/document-download-api-service-patch.yaml
# - path: services/documentation-service-patch.yaml

- path: cwagent/cwagent-deployment-patch.yaml

- path: nodeselectors/admin-node-selector-patch.yaml
- path: nodeselectors/celery-api-node-selector-patch.yaml
- path: nodeselectors/celery-beat-node-selector-patch.yaml
- path: nodeselectors/celery-email-send-node-selector-patch.yaml
- path: nodeselectors/celery-email-send-scalable-node-selector-patch.yaml
- path: nodeselectors/celery-primary-node-selector-patch.yaml
- path: nodeselectors/celery-scalable-node-selector-patch.yaml
- path: nodeselectors/celery-sms-node-selector-patch.yaml
- path: nodeselectors/celery-sms-send-node-selector-patch.yaml
- path: nodeselectors/celery-sms-send-scalable-node-selector-patch.yaml
- path: nodeselectors/document-download-api-node-selector-patch.yaml
- path: nodeselectors/documentation-node-selector-patch.yaml
# - path: nodeselectors/admin-node-selector-patch.yaml
# - path: nodeselectors/celery-api-node-selector-patch.yaml
# - path: nodeselectors/celery-beat-node-selector-patch.yaml
# - path: nodeselectors/celery-email-send-node-selector-patch.yaml
# - path: nodeselectors/celery-email-send-scalable-node-selector-patch.yaml
# - path: nodeselectors/celery-primary-node-selector-patch.yaml
# - path: nodeselectors/celery-scalable-node-selector-patch.yaml
# - path: nodeselectors/celery-sms-node-selector-patch.yaml
# - path: nodeselectors/celery-sms-send-node-selector-patch.yaml
# - path: nodeselectors/celery-sms-send-scalable-node-selector-patch.yaml
# - path: nodeselectors/document-download-api-node-selector-patch.yaml
# - path: nodeselectors/documentation-node-selector-patch.yaml

- path: celery-init-delete/celery-email-send-primary-init-delete-patch.yaml
- path: celery-init-delete/celery-email-send-scalable-init-delete-patch.yaml
- path: celery-init-delete/celery-primary-init-delete-patch.yaml
- path: celery-init-delete/celery-scalable-init-delete-patch.yaml
- path: celery-init-delete/celery-sms-send-primary-init-delete-patch.yaml
- path: celery-init-delete/celery-sms-send-scalable-init-delete-patch.yaml
# - path: celery-init-delete/celery-email-send-primary-init-delete-patch.yaml
# - path: celery-init-delete/celery-email-send-scalable-init-delete-patch.yaml
# - path: celery-init-delete/celery-primary-init-delete-patch.yaml
# - path: celery-init-delete/celery-scalable-init-delete-patch.yaml
# - path: celery-init-delete/celery-sms-send-primary-init-delete-patch.yaml
# - path: celery-init-delete/celery-sms-send-scalable-init-delete-patch.yaml

configMapGenerator:
- envs:
Expand Down
Loading
Loading