Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CAPT-1703 Migrate production environment to AKS #2887

Merged
merged 28 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
02261ba
Production deployment workflow
vacabor Jun 20, 2024
85e9d48
Trailing newline
vacabor Jun 20, 2024
a75e7d9
Use a datatype that exists
vacabor Jun 20, 2024
58d9255
Merge branch 'master' into CAPT-1703-prod-aks-migration
vacabor Jun 20, 2024
f38d7c5
Merge branch 'master' into CAPT-1703-prod-aks-migration
vacabor Jun 21, 2024
788c5e5
Wait for test deployment before deploying to production
vacabor Jun 24, 2024
90e40d5
Merge branch 'master' into CAPT-1703-prod-aks-migration
vacabor Jun 24, 2024
5c7c334
Add HA and PG backups for production
vacabor Jun 24, 2024
9201c0a
Merge branch 'CAPT-1703-prod-aks-migration' of github.com:DFE-Digital…
vacabor Jun 24, 2024
48c7070
Refactor running migrations into deployment action
vacabor Jun 25, 2024
21aaab5
WIP: Run the action on this branch for testing
vacabor Jun 25, 2024
a4dfafa
Provide PR_NUMBER when running migrations
vacabor Jun 25, 2024
97ad209
Perform azure/login ahead of migrations
vacabor Jun 25, 2024
3d551b3
WIP: Run the test/prod deploys on this branch for testing
vacabor Jun 25, 2024
9401464
Turn off HA initially to allow the SKU to change
vacabor Jun 26, 2024
6e71ed1
Merge branch 'master' of github.com:DFE-Digital/claim-additional-paym…
vacabor Jun 26, 2024
da47883
Revert "Turn off HA initially to allow the SKU to change"
vacabor Jun 26, 2024
e974aa6
Require the build job for prod deploy per PR feedback
vacabor Jun 27, 2024
7312f50
Revert "WIP: Run the action on this branch for testing"
vacabor Jun 27, 2024
a4ee510
Revert "WIP: Run the test/prod deploys on this branch for testing"
vacabor Jun 27, 2024
24cd1fb
WIP: Use the new production canonical_hostname for testing
vacabor Jun 28, 2024
e2363be
Merge branch 'master' into CAPT-1703-prod-aks-migration
vacabor Jul 3, 2024
9c16a5c
Clean up duplicate workflow steps with a matrix
vacabor Jul 5, 2024
593a520
Merge branch 'master' into CAPT-1703-prod-aks-migration
vacabor Jul 5, 2024
e9d72c6
Merge branch 'master' into CAPT-1703-prod-aks-migration
vacabor Jul 10, 2024
67fdaba
Merge branch 'master' of github.com:DFE-Digital/claim-additional-paym…
vacabor Jul 10, 2024
e514f83
Switch back to targetting master
vacabor Jul 10, 2024
5bdcc89
Merge branch 'CAPT-1703-prod-aks-migration' of github.com:DFE-Digital…
vacabor Jul 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/workflows/build_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,59 @@ jobs:
SLACK_TITLE: Failure deploying release to test
SLACK_MESSAGE: Failure deploying release to test - Docker tag ${{ needs.build.outputs.docker-image-tag }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}

deploy_production:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same code as above. Can we use a matrix instead? With max-parallel: 1 to make it sequential.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name: Deploy to production environment
concurrency: deploy_production
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/master' && github.event_name == 'push'
needs: [build]
vacabor marked this conversation as resolved.
Show resolved Hide resolved
environment:
name: production-aks
url: ${{ steps.deploy.outputs.environment_url }}
outputs:
environment_url: ${{ steps.deploy.outputs.environment_url }}

steps:
- name: Checkout code
uses: actions/checkout@v4

- uses: azure/login@v2
with:
creds: ${{ secrets.AZURE_CREDENTIALS }}

- uses: ./.github/actions/deploy-environment
id: deploy
with:
environment: production-aks
docker-image: ${{ needs.build.outputs.docker-image-tag }}
azure-credentials: ${{ secrets.AZURE_CREDENTIALS }}

- name: Run migrations
shell: bash
run: |
make ci production-aks get-cluster-credentials
kubectl exec -n srtl-production deployment/claim-additional-payments-for-teaching-production-worker -- sh -c "DISABLE_DATABASE_ENVIRONMENT_CHECK=1 bin/prepare-database"
AbigailMcP marked this conversation as resolved.
Show resolved Hide resolved

- name: Install Ruby
uses: ruby/setup-ruby@v1
with:
bundler-cache: true

- name: Run smoke tests
shell: bash
run: bundle exec rspec spec/smoke -t smoke:true -b
env:
RAILS_ENV: test
SMOKE_TEST_APP_HOST: ${{ env.APP_URL }}
BASIC_AUTH_USERNAME: ${{ secrets.BASIC_AUTH_USERNAME }}
BASIC_AUTH_PASSWORD: ${{ secrets.BASIC_AUTH_PASSWORD }}

- name: Notify on failure
if: failure()
uses: rtCamp/action-slack-notify@master
env:
SLACK_COLOR: failure
SLACK_TITLE: Failure deploying release to production
SLACK_MESSAGE: Failure deploying release to production - Docker tag ${{ needs.build.outputs.docker-image-tag }}
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}
2 changes: 2 additions & 0 deletions terraform/application/application.tf
Original file line number Diff line number Diff line change
Expand Up @@ -58,4 +58,6 @@ module "worker_application" {

docker_image = var.docker_image
command = var.worker_command

replicas = var.worker_replicas
}
11 changes: 11 additions & 0 deletions terraform/application/config/production.tfvars.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"cluster": "production",
"namespace": "srtl-production",
"config": "production",
"environment": "production",
"canonical_hostname": "claim-additional-teaching-payment.service.gov.uk",
"web_replicas": 2,
"worker_replicas": 2,
"startup_command": ["/bin/sh", "-c", "bin/rails server -b 0.0.0.0"],
"worker_command": ["/bin/sh", "-c", "bin/bundle exec bin/delayed_job run -n 1"]
}
3 changes: 3 additions & 0 deletions terraform/application/config/production_Terrafile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
aks:
source: "https://github.com/DFE-Digital/terraform-modules"
version: "stable"
1 change: 1 addition & 0 deletions terraform/application/config/production_app_env.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
AbigailMcP marked this conversation as resolved.
Show resolved Hide resolved
2 changes: 1 addition & 1 deletion terraform/application/config/review.tfvars.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@
"deploy_azure_backing_services": false,
"enable_postgres_ssl": false,
"startup_command": ["/bin/sh", "-c", "bin/rails server -b 0.0.0.0"],
"worker_command": ["/bin/sh", "-c", "bin/bundle exec bin/delayed_job run -n 4"]
"worker_command": ["/bin/sh", "-c", "bin/bundle exec bin/delayed_job run -n 1"]
}
5 changes: 5 additions & 0 deletions terraform/application/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,11 @@ variable "web_replicas" {
default = 1
}

variable "worker_replicas" {
description = "Number of replicas of the worker"
default = 1
}

locals {
postgres_ssl_mode = var.enable_postgres_ssl ? "require" : "disable"
canonical_hostname = var.canonical_hostname != null ? var.canonical_hostname : "${var.service_name}-${var.environment}-web.test.teacherservices.cloud"
Expand Down