Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BCP early stages #1226

Closed
wants to merge 50 commits into from
Closed
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
72c33eb
reviewed and fixed BCP steps 1 through 4 and need to push iam to stag…
P0NDER0SA Mar 26, 2024
adb345d
Update images.tf
P0NDER0SA Mar 26, 2024
cac0e68
Delete env/pond/README.md
P0NDER0SA Mar 26, 2024
8fd19f5
Merge branch 'main' into scratch-env
ben851 Mar 27, 2024
d351582
Moving VPN to eks folder
ben851 Mar 27, 2024
a779d32
vpn premerge
ben851 Mar 27, 2024
5d0e36d
Merge branch 'scratch-env' of github.com:cds-snc/notification-terrafo…
ben851 Mar 27, 2024
0a311d9
Merge branch 'main' into scratch-env
ben851 Mar 27, 2024
e68feb1
Worked through 6 more steps for BCP
P0NDER0SA Mar 27, 2024
b32898b
Adding sentinel change
P0NDER0SA Mar 27, 2024
39ce5d1
Worked through a few more steps -- a few more to go. Minor adjustmen…
P0NDER0SA Mar 28, 2024
a0a09c2
fixing
P0NDER0SA Mar 28, 2024
a9ab6c3
fixing checkov recommendations
P0NDER0SA Mar 28, 2024
911923b
checkov recommendations
P0NDER0SA Mar 28, 2024
4cba696
Merge branch 'main' into scratch-env
P0NDER0SA Mar 28, 2024
2c2967a
fixing checkov and formatting
P0NDER0SA Mar 28, 2024
929073a
formatting
P0NDER0SA Mar 28, 2024
cb21493
Merge branch 'main' into scratch-env
P0NDER0SA Mar 28, 2024
f93db4e
Merge branch 'scratch-env' of github.com:cds-snc/notification-terrafo…
P0NDER0SA Mar 28, 2024
d9dac75
fixing duplicate iam definitions
P0NDER0SA Mar 28, 2024
5225120
Merge branch 'main' into scratch-env
P0NDER0SA Apr 3, 2024
e3b9314
including nuke file to clean up scratch and fixing paths for docker s…
P0NDER0SA Apr 3, 2024
271e0f3
Merge branch 'main' into scratch-env
P0NDER0SA Apr 4, 2024
b02405d
Merge branch 'main' into scratch-env
P0NDER0SA Apr 8, 2024
b1a18d8
minor adjustments for variables for bcp
P0NDER0SA Apr 8, 2024
28283da
variable updates
P0NDER0SA Apr 8, 2024
0920c99
adding dependency for aws_acm_certificate
P0NDER0SA Apr 9, 2024
44ee1ec
cert dependency
P0NDER0SA Apr 9, 2024
efda6ea
merging
ben851 Apr 10, 2024
1e52274
removing vpn dependencies
ben851 Apr 10, 2024
a83fee7
adding docker build for blazer/database tools
P0NDER0SA Apr 17, 2024
3c5aa56
format
P0NDER0SA Apr 17, 2024
b995579
missing curly bracket
P0NDER0SA Apr 23, 2024
aee41a6
nuke updates
ben851 Apr 23, 2024
59f880d
deleting activity stream
P0NDER0SA Apr 23, 2024
4c8dffc
Merge branch 'scratch-env' of github.com:cds-snc/notification-terrafo…
P0NDER0SA Apr 23, 2024
78ef647
updating dns files
P0NDER0SA Apr 24, 2024
5ab9ce1
updating dns entries
P0NDER0SA Apr 24, 2024
5a75567
cleaning up file
P0NDER0SA Apr 24, 2024
95b5765
adding provider for the other two
P0NDER0SA Apr 24, 2024
c12ba82
adding provider
P0NDER0SA Apr 24, 2024
cf167da
fixing gitignore
ben851 Apr 24, 2024
c8f7e00
fixing gitignore
ben851 Apr 24, 2024
ea435f0
fixes for quicksight
P0NDER0SA Apr 24, 2024
e993386
formatted fixes for quicksight
P0NDER0SA Apr 24, 2024
eca015c
creating the required quicksight role via a null resource script -- p…
P0NDER0SA Apr 25, 2024
4a8a58c
formatted
P0NDER0SA Apr 25, 2024
117fcfc
adding a force delete variable
P0NDER0SA Apr 30, 2024
fe8dfba
variable
P0NDER0SA Apr 30, 2024
863316b
adding the k8s subnets section for eks
P0NDER0SA Apr 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 5 additions & 9 deletions aws/common/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ output "vpc_id" {
value = aws_vpc.notification-canada-ca.id
}

output "vpc_private_subnet_cidr_blocks" {
value = aws_subnet.notification-canada-ca-private.*.cidr_block
}

output "vpc_private_subnets" {
value = aws_subnet.notification-canada-ca-private.*.id
}
Expand Down Expand Up @@ -177,12 +181,4 @@ output "sqs_send_email_medium_queue_name" {

output "sqs_send_email_high_queue_name" {
value = var.sqs_send_email_high_queue_name
}

output "client_vpn_cloudwatch_log_group_name" {
value = module.vpn.client_vpn_cloudwatch_log_group_name
}

output "client_vpn_security_group_id" {
value = module.vpn.client_vpn_security_group_id
}
}
4 changes: 4 additions & 0 deletions aws/common/s3.tf
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ resource "aws_s3_bucket" "csv_bucket" {

#tfsec:ignore:AWS077 - Versioning is not enabled
logging {
target_prefix = var.env
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had to add this in order to get the s3 buckets to deploy - this must be a new requirement by SRE.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Followed up with SRE, it's actually a new requirement from the AWS API.

target_bucket = module.csv_bucket_logs.s3_bucket_id
}

Expand Down Expand Up @@ -155,6 +156,7 @@ resource "aws_s3_bucket" "document_bucket" {

#tfsec:ignore:AWS077 - Versioning is not enabled
logging {
target_prefix = var.env
target_bucket = module.document_download_logs.s3_bucket_id
}

Expand Down Expand Up @@ -188,6 +190,7 @@ resource "aws_s3_bucket" "scan_files_document_bucket" {

#tfsec:ignore:AWS077 - Versioning is not enabled
logging {
target_prefix = var.env
target_bucket = module.document_download_logs.s3_bucket_id
}

Expand Down Expand Up @@ -329,6 +332,7 @@ resource "aws_s3_bucket" "athena_bucket" {

#tfsec:ignore:AWS077 - Versioning is not enabled
logging {
target_prefix = var.env
target_bucket = module.athena_logs_bucket.s3_bucket_id
}

Expand Down
17 changes: 0 additions & 17 deletions aws/common/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -278,23 +278,6 @@ variable "eks_cluster_name" {
type = string
}

variable "client_vpn_access_group_id" {
description = "IAM Identity Center group ID that will be allowed access to the VPN."
type = string
sensitive = true
}

variable "client_vpn_saml_metadata" {
description = "IAM Identity Center application SAML metadata. Users that want to connect to the VPN must be granted access to this app."
type = string
sensitive = true
}

variable "client_vpn_self_service_saml_metadata" {
description = "IAM Identity Center self-service application SAML metadata. This allows users to download the VPN client and configuration."
type = string
sensitive = true
}

variable "account_budget_limit" {
description = "The dollar amount in USD that this AWS account should be budgeted to"
Expand Down
6 changes: 3 additions & 3 deletions aws/common/vpc.tf
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ data "aws_availability_zones" "available" {
}

resource "aws_vpc" "notification-canada-ca" {
cidr_block = "10.0.0.0/16"
cidr_block = var.vpc_cidr_block
enable_dns_hostnames = true

tags = {
Expand Down Expand Up @@ -74,7 +74,7 @@ resource "aws_subnet" "notification-canada-ca-private" {
count = 3

vpc_id = aws_vpc.notification-canada-ca.id
cidr_block = cidrsubnet("10.0.0.0/16", 8, count.index)
cidr_block = cidrsubnet(var.vpc_cidr_block, 8, count.index)
availability_zone = element(data.aws_availability_zones.available.names, count.index)

tags = {
Expand All @@ -91,7 +91,7 @@ resource "aws_subnet" "notification-canada-ca-public" {
count = 3

vpc_id = aws_vpc.notification-canada-ca.id
cidr_block = cidrsubnet("10.0.0.0/16", 8, count.index + 3)
cidr_block = cidrsubnet(var.vpc_cidr_block, 8, count.index + 3)
availability_zone = element(data.aws_availability_zones.available.names, count.index)

tags = {
Expand Down
2 changes: 1 addition & 1 deletion aws/database-tools/rds.tf
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ resource "aws_db_instance" "database-tools" {
allocated_storage = 10
db_name = "database_tools"
engine = "postgres"
engine_version = "14.3"
engine_version = "15.5"
instance_class = "db.t3.micro"
username = "postgres"
password = var.dbtools_password
Expand Down
44 changes: 44 additions & 0 deletions aws/dns/iam.tf
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,50 @@ resource "aws_iam_role" "scratch_dns_manager" {
EOF
}

resource "aws_iam_role_policy" "pond_dns_manager_policy" {
count = var.env == "staging" ? 1 : 0
name = "pond_dns_manager_policy"
role = aws_iam_role.pond_dns_manager[0].id

policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Action = ["route53:GetHostedZone",
"route53:ListResourceRecordSets",
"route53:ListHostedZones",
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets",
"route53:GetHostedZoneCount",
"route53:ListHostedZonesByName",
"route53:GetChange"
]
Resource = ["*"]
}]
})
}

resource "aws_iam_role" "pond_dns_manager" {
count = var.env == "staging" ? 1 : 0
name = "pond_dns_manager_role"

assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"AWS": "339712868793"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}

resource "aws_iam_role_policy" "staging_dns_manager_policy" {
count = var.env == "staging" ? 1 : 0
name = "staging_dns_manager_policy"
Expand Down
12 changes: 6 additions & 6 deletions aws/ecr/images.tf
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ resource "null_resource" "build_heartbeat_docker_image" {
]

provisioner "local-exec" {
command = "docker build -t ${aws_ecr_repository.heartbeat.repository_url}:bootstrap -f /var/tmp/notification-lambdas/heartbeat/Dockerfile /var/tmp/notification-lambdas/"
command = "pushd /var/tmp/notification-lambdas/heartbeat && docker build -t ${aws_ecr_repository.heartbeat.repository_url}:bootstrap -f /var/tmp/notification-lambdas/heartbeat/Dockerfile . && popd"
}

}
Expand All @@ -121,7 +121,7 @@ resource "null_resource" "build_google_cidr_docker_image" {
]

provisioner "local-exec" {
command = "docker build -t ${aws_ecr_repository.google-cidr.repository_url}:bootstrap -f /var/tmp/notification-lambdas/google-cidr/Dockerfile /var/tmp/notification-lambdas/google-cidr"
command = "pushd /var/tmp/notification-lambdas/google-cidr && docker build -t ${aws_ecr_repository.google-cidr.repository_url}:bootstrap -f /var/tmp/notification-lambdas/google-cidr/Dockerfile . && popd"
}

}
Expand All @@ -145,7 +145,7 @@ resource "null_resource" "build_ses_receiving_emails_docker_image" {
]

provisioner "local-exec" {
command = "docker build -t ${aws_ecr_repository.ses_receiving_emails.repository_url}:bootstrap -f /var/tmp/notification-lambdas/sesreceivingemails/Dockerfile /var/tmp/notification-lambdas"
command = "pushd /var/tmp/notification-lambdas/sesreceivingemails && docker build -t ${aws_ecr_repository.ses_receiving_emails.repository_url}:bootstrap -f /var/tmp/notification-lambdas/sesreceivingemails/Dockerfile . && popd"
}

}
Expand All @@ -169,7 +169,7 @@ resource "null_resource" "build_ses_to_sqs_email_callbacks_docker_image" {
]

provisioner "local-exec" {
command = "docker build -t ${aws_ecr_repository.ses_to_sqs_email_callbacks.repository_url}:bootstrap -f /var/tmp/notification-lambdas/sesemailcallbacks/Dockerfile /var/tmp/notification-lambdas"
command = "pushd /var/tmp/notification-lambdas/sesemailcallbacks && docker build -t ${aws_ecr_repository.ses_to_sqs_email_callbacks.repository_url}:bootstrap -f /var/tmp/notification-lambdas/sesemailcallbacks/Dockerfile . && popd"
}

}
Expand All @@ -193,7 +193,7 @@ resource "null_resource" "build_sns_to_sqs_sms_callbacks_docker_image" {
]

provisioner "local-exec" {
command = "docker build -t ${aws_ecr_repository.sns_to_sqs_sms_callbacks.repository_url}:bootstrap -f /var/tmp/notification-lambdas/sesemailcallbacks/Dockerfile /var/tmp/notification-lambdas"
command = "pushd /var/tmp/notification-lambdas/sesemailcallbacks && docker build -t ${aws_ecr_repository.sns_to_sqs_sms_callbacks.repository_url}:bootstrap -f /var/tmp/notification-lambdas/sesemailcallbacks/Dockerfile . && popd"
}

}
Expand All @@ -217,7 +217,7 @@ resource "null_resource" "build_system_status_docker_image" {
]

provisioner "local-exec" {
command = "docker build -t ${aws_ecr_repository.system_status.repository_url}:bootstrap -f /var/tmp/notification-lambdas/system_status/Dockerfile /var/tmp/notification-lambdas/"
command = "pushd /var/tmp/notification-lambdas/system_status && docker build -t ${aws_ecr_repository.system_status.repository_url}:bootstrap -f /var/tmp/notification-lambdas/system_status/Dockerfile . && popd"
}
}

Expand Down
4 changes: 2 additions & 2 deletions aws/eks/securitygroups.tf
Original file line number Diff line number Diff line change
Expand Up @@ -359,7 +359,7 @@ resource "aws_security_group_rule" "client-vpn-ingress-database" {
from_port = 5432
to_port = 5432
protocol = "tcp"
source_security_group_id = var.client_vpn_security_group_id
source_security_group_id = module.vpn.client_vpn_security_group_id
security_group_id = data.aws_security_group.eks-securitygroup-rds.id
}

Expand All @@ -369,7 +369,7 @@ resource "aws_security_group_rule" "client-vpn-ingress-redis" {
from_port = 6379
to_port = 6379
protocol = "tcp"
source_security_group_id = var.client_vpn_security_group_id
source_security_group_id = module.vpn.client_vpn_security_group_id
security_group_id = data.aws_security_group.eks-securitygroup-rds.id
}

Expand Down
9 changes: 5 additions & 4 deletions aws/eks/sentinel.tf
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
locals {
application_log_group_arn = "arn:aws:logs:${var.region}:${var.account_id}:log-group:${local.eks_application_log_group}"
client_vpn_log_group_arn = "arn:aws:logs:${var.region}:${var.account_id}:log-group:${var.client_vpn_cloudwatch_log_group_name}"
client_vpn_log_group_arn = "arn:aws:logs:${var.region}:${var.account_id}:log-group:${module.vpn.client_vpn_cloudwatch_log_group_name}"
blazer_log_group_arn = "arn:aws:logs:${var.region}:${var.account_id}:log-group:blazer"
}

Expand Down Expand Up @@ -45,9 +45,10 @@ resource "aws_cloudwatch_log_subscription_filter" "blazer_logging" {
}

resource "aws_cloudwatch_log_subscription_filter" "client_vpn_connections" {
count = var.enable_sentinel_forwarding ? 1 : 0
name = "Client VPN connections"
log_group_name = var.client_vpn_cloudwatch_log_group_name
count = var.enable_sentinel_forwarding ? 1 : 0
name = "Client VPN connections"
log_group_name = module.vpn.client_vpn_cloudwatch_log_group_name

filter_pattern = "[w1=\"*\"]" # All logs
destination_arn = module.sentinel_forwarder[0].lambda_arn
distribution = "Random"
Expand Down
32 changes: 22 additions & 10 deletions aws/eks/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -240,16 +240,6 @@ variable "celery_queue_prefix" {
description = "Celery queue prefix"
}

variable "client_vpn_cloudwatch_log_group_name" {
type = string
description = "Client VPN CloudWatch log group name. This is used by the Sentinel forwarder to send logs to Sentinel."
}

variable "client_vpn_security_group_id" {
type = string
description = "Client VPN security group ID"
}

variable "eks_addon_ebs_driver_version" {
type = string
description = "Version for EBS driver addon for EKS (Persistence)"
Expand Down Expand Up @@ -294,3 +284,25 @@ variable "pr_bot_installation_id" {
sensitive = true
}

variable "client_vpn_access_group_id" {
description = "IAM Identity Center group ID that will be allowed access to the VPN."
type = string
sensitive = true
}
variable "client_vpn_saml_metadata" {
description = "IAM Identity Center application SAML metadata. Users that want to connect to the VPN must be granted access to this app."
type = string
sensitive = true
}

variable "client_vpn_self_service_saml_metadata" {
description = "IAM Identity Center self-service application SAML metadata. This allows users to download the VPN client and configuration."
type = string
sensitive = true
}

variable "vpc_private_subnet_cidr_blocks" {
description = "CIDR blocks of the subnets where the VPN will be deployed."
type = list(string)
}

9 changes: 4 additions & 5 deletions aws/common/vpn.tf → aws/eks/vpn.tf
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ module "vpn" {
endpoint_name = "private-subnets"
access_group_id = var.client_vpn_access_group_id

vpc_id = aws_vpc.notification-canada-ca.id
vpc_cidr_block = aws_vpc.notification-canada-ca.cidr_block
subnet_cidr_blocks = aws_subnet.notification-canada-ca-private.*.cidr_block
subnet_ids = aws_subnet.notification-canada-ca-private.*.id
vpc_id = var.vpc_id
vpc_cidr_block = var.vpc_cidr_block
subnet_cidr_blocks = var.vpc_private_subnet_cidr_blocks
subnet_ids = var.vpc_private_subnets
acm_certificate_arn = aws_acm_certificate.client_vpn.arn

# Only create a self-service portal in prod
Expand All @@ -27,7 +27,6 @@ module "vpn" {
}

#
# Certificate used for VPN communication
#
resource "tls_private_key" "client_vpn" {
algorithm = "RSA"
Expand Down
2 changes: 1 addition & 1 deletion aws/heartbeat/lambda.tf
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
locals {
image_tag = var.env == "production" ? var.heartbeat_docker_tag : "latest"
image_tag = var.env == "production" ? var.heartbeat_docker_tag : (var.bootstrap == true ? "bootstrap" : "latest")
}

module "heartbeat" {
Expand Down
5 changes: 5 additions & 0 deletions aws/heartbeat/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,9 @@ variable "heartbeat_docker_tag" {
type = string
description = "Set this to specify the image version"
default = "bootstrap"
}
variable "bootstrap" {
description = "Boolean value to decide whether or not to build images"
type = bool
default = false
}
43 changes: 43 additions & 0 deletions aws/lambda-api/api_gateway.tf
Original file line number Diff line number Diff line change
Expand Up @@ -176,3 +176,46 @@
resource_arn = aws_api_gateway_stage.api.arn
web_acl_arn = aws_wafv2_web_acl.api_lambda.arn
}


resource "aws_api_gateway_method_settings" "api_settings" {
rest_api_id = aws_api_gateway_rest_api.api.id
stage_name = aws_api_gateway_stage.api.stage_name
method_path = "*/*"
settings {
logging_level = "INFO"
data_trace_enabled = false
metrics_enabled = true
caching_enabled = true
}
}
Fixed Show fixed Hide fixed

# Allow API Gateway to push logs to CloudWatch
resource "aws_api_gateway_account" "main" {
cloudwatch_role_arn = aws_iam_role.main.arn
}

resource "aws_iam_role" "main" {
name = "api-gateway-logs-role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "apigateway.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOF

}

resource "aws_iam_role_policy_attachment" "main" {
role = aws_iam_role.main.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs"
}
Loading
Loading