Skip to content

Commit

Permalink
feat: imagepullsecret config
Browse files Browse the repository at this point in the history
  • Loading branch information
johanneswuerbach committed Jun 12, 2024
1 parent 83c7cea commit 77d4cfb
Show file tree
Hide file tree
Showing 13 changed files with 483 additions and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ You can find more details about Resource Packs and how to use them [inside the H

The following resources are included:

* [config/imagepullsecret](./humanitec-resource-defs/config/imagepullsecret): A `config` resource that configures [imagePullSecret](https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod) injection.
* [dns/basic](./humanitec-resource-defs/dns/basic): A `dns` resource using Route 53.
* [iam-policy/ecr-create-repository](./humanitec-resource-defs/iam-policy/ecr-create-repository): IAM policy for to create ECR repositories.
* [iam-policy/s3](./humanitec-resource-defs/iam-policy/s3): IAM policy for for an S3 bucket.
Expand Down
71 changes: 71 additions & 0 deletions examples/config/imagepullsecret/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
features:
- humanitec-operator
- imagepullsecret
---

# Example: config resource the configures ECR image pull secrets

## Configuration

This example configures a [config](https://developer.humanitec.com/platform-orchestrator/reference/resource-types/#config) Resource Definition, which injects an [imagePullSecret](https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod) into workloads. An `imagePullSecret` is required when the k8s cluster runs outside AWS and workload use private ECR images.

The image used to fetch and periodically renew the secrets can be found here <https://github.com/humanitec-architecture/aws-ecr-credentials-refresh> and the configs are heavily inspired by <https://skryvets.com/blog/2021/03/15/kubernetes-pull-image-from-private-ecr-registry/>.

## Orchestrator setup

```mermaid
graph LR;
workload_1 --> config["imagepullsecret, resource_type: config"]
workload_2 --> config["imagepullsecret, resource_type: config"]
```

## Terraform docs

<!-- BEGIN_TF_DOCS -->
### Requirements

| Name | Version |
|------|---------|
| terraform | >= 1.3.0 |
| aws | ~> 5.0 |
| humanitec | ~> 1.0 |
| random | ~> 3.5 |

### Providers

| Name | Version |
|------|---------|
| aws | ~> 5.0 |
| humanitec | ~> 1.0 |

### Modules

| Name | Source | Version |
|------|--------|---------|
| imagepullsecret | ../../../humanitec-resource-defs/config/imagepullsecret | n/a |

### Resources

| Name | Type |
|------|------|
| [aws_iam_access_key.cluster_ecr_pull](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) | resource |
| [aws_iam_user.cluster_ecr_pull](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) | resource |
| [aws_iam_user_policy_attachment.cluster_ecr_pull](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) | resource |
| [aws_secretsmanager_secret.ecr_pull](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret) | resource |
| [aws_secretsmanager_secret_version.ecr_pull](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret_version) | resource |
| [humanitec_application.example](https://registry.terraform.io/providers/humanitec/humanitec/latest/docs/resources/application) | resource |
| [humanitec_resource_definition.workload](https://registry.terraform.io/providers/humanitec/humanitec/latest/docs/resources/resource_definition) | resource |
| [humanitec_resource_definition_criteria.imagepullsecret](https://registry.terraform.io/providers/humanitec/humanitec/latest/docs/resources/resource_definition_criteria) | resource |
| [humanitec_resource_definition_criteria.workload](https://registry.terraform.io/providers/humanitec/humanitec/latest/docs/resources/resource_definition_criteria) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |

### Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| humanitec\_secret\_store\_id | Humanitec Secret Store ID that points to AWS Secrets Manager | `string` | n/a | yes |
| region | AWS Region | `string` | n/a | yes |
| name | Name of the example application | `string` | `"hum-rp-ips-example"` | no |
| prefix | Prefix of the created resources | `string` | `"hum-rp-ips-ex-"` | no |
<!-- END_TF_DOCS -->
109 changes: 109 additions & 0 deletions examples/config/imagepullsecret/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# AWS IAM user used by the k8s-cluster to pull images from ECR

resource "aws_iam_user" "cluster_ecr_pull" {
name = "cluster_ecr_pull"
}

resource "aws_iam_user_policy_attachment" "cluster_ecr_pull" {
user = aws_iam_user.cluster_ecr_pull.name
# https://docs.aws.amazon.com/AmazonECR/latest/userguide/security-iam-awsmanpol.html#security-iam-awsmanpol-AmazonEC2ContainerRegistryReadOnly
policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
}

resource "aws_iam_access_key" "cluster_ecr_pull" {
user = aws_iam_user.cluster_ecr_pull.name

# Ensure that the policy is not deleted before the access key
depends_on = [aws_iam_user_policy_attachment.cluster_ecr_pull]
}

# Store the access key and secret in AWS Secrets Manager for the Humanitec Operator to be able to fetch them.

locals {
ecr_pull_secrets = {
aws-access-key-id = aws_iam_access_key.cluster_ecr_pull.id
aws-secret-access-key = aws_iam_access_key.cluster_ecr_pull.secret
}

ecr_pull_secret_refs = {
for key, value in local.ecr_pull_secrets : key => {
ref = aws_secretsmanager_secret.ecr_pull[key].name
store = var.humanitec_secret_store_id
version = aws_secretsmanager_secret_version.ecr_pull[key].version_id
}
}
}

resource "aws_secretsmanager_secret" "ecr_pull" {
for_each = local.ecr_pull_secrets
name = "humanitec-ecr-pull-secret-${each.key}"
}

resource "aws_secretsmanager_secret_version" "ecr_pull" {
for_each = local.ecr_pull_secrets

secret_id = aws_secretsmanager_secret.ecr_pull[each.key].id
secret_string = each.value
}

# Example application and resource definition criteria

resource "humanitec_application" "example" {
id = var.name
name = var.name
}

# Current AWS Account ID
data "aws_caller_identity" "current" {}

locals {
imagepullsecret_config_res_id = "imagepullsecret"
}

module "imagepullsecret" {
source = "../../../humanitec-resource-defs/config/imagepullsecret"

prefix = var.prefix

account_id = data.aws_caller_identity.current.account_id
region = var.region
access_key_id_ref = local.ecr_pull_secret_refs["aws-access-key-id"]
secret_access_key_ref = local.ecr_pull_secret_refs["aws-secret-access-key"]
}

resource "humanitec_resource_definition_criteria" "imagepullsecret" {
resource_definition_id = module.imagepullsecret.id
app_id = humanitec_application.example.id
res_id = local.imagepullsecret_config_res_id
class = "default"

force_delete = true
}

resource "humanitec_resource_definition" "workload" {
driver_type = "humanitec/template"
id = "${var.prefix}workload"
name = "${var.prefix}workload"
type = "workload"

driver_inputs = {
values_string = jsonencode({
templates = {
outputs = <<EOL
update:
- op: add
path: /spec/imagePullSecrets
value:
- name: $${resources["config.default#${local.imagepullsecret_config_res_id}"].outputs.secret_name}
EOL
}
})
}
}

resource "humanitec_resource_definition_criteria" "workload" {
resource_definition_id = humanitec_resource_definition.workload.id
app_id = humanitec_application.example.id

force_delete = true
}
31 changes: 31 additions & 0 deletions examples/config/imagepullsecret/providers.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
humanitec = {
source = "humanitec/humanitec"
version = "~> 1.0"
}
random = {
source = "hashicorp/random"
version = "~> 3.5"
}
}

required_version = ">= 1.3.0"
}

provider "aws" {
default_tags {
tags = {
"managed_by" = "terraform"
"source" = "github.com/humanitec-architecture/resource-pack-aws"
}
}
}

provider "humanitec" {}

provider "random" {}
12 changes: 12 additions & 0 deletions examples/config/imagepullsecret/terraform.tfvars.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

# Humanitec Secret Store ID that points to AWS Secrets Manager
humanitec_secret_store_id = ""

# Name of the example application
name = "hum-rp-ips-example"

# Prefix of the created resources
prefix = "hum-rp-ips-ex-"

# AWS Region
region = ""
21 changes: 21 additions & 0 deletions examples/config/imagepullsecret/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
variable "region" {
description = "AWS Region"
type = string
}

variable "name" {
description = "Name of the example application"
type = string
default = "hum-rp-ips-example"
}

variable "prefix" {
description = "Prefix of the created resources"
type = string
default = "hum-rp-ips-ex-"
}

variable "humanitec_secret_store_id" {
description = "Humanitec Secret Store ID that points to AWS Secrets Manager"
type = string
}
40 changes: 40 additions & 0 deletions humanitec-resource-defs/config/imagepullsecret/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Humanitec Resource Definition: config/imagepullsecrets

## Terraform docs

<!-- BEGIN_TF_DOCS -->
### Requirements

| Name | Version |
|------|---------|
| terraform | >= 1.3.0 |
| humanitec | ~> 1.0 |

### Providers

| Name | Version |
|------|---------|
| humanitec | ~> 1.0 |

### Resources

| Name | Type |
|------|------|
| [humanitec_resource_definition.main](https://registry.terraform.io/providers/humanitec/humanitec/latest/docs/resources/resource_definition) | resource |

### Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| access\_key\_id\_ref | AWS Access Key ID (Secret Store reference) | <pre>object({<br> ref = optional(string)<br> store = optional(string)<br> value = optional(string)<br> version = optional(string)<br> })</pre> | n/a | yes |
| account\_id | The id of the hosted zone in which this record set will reside. | `string` | n/a | yes |
| prefix | Prefix for all resources | `string` | n/a | yes |
| region | AWS Region | `string` | n/a | yes |
| secret\_access\_key\_ref | AWS Secret Access Key (Secret Store reference) | <pre>object({<br> ref = optional(string)<br> store = optional(string)<br> value = optional(string)<br> version = optional(string)<br> })</pre> | n/a | yes |

### Outputs

| Name | Description |
|------|-------------|
| id | n/a |
<!-- END_TF_DOCS -->
32 changes: 32 additions & 0 deletions humanitec-resource-defs/config/imagepullsecret/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
locals {
secret_name = "ecr-pull-secret"
}

resource "humanitec_resource_definition" "main" {
id = "${var.prefix}config-imagepullsecret"
name = "${var.prefix}config-imagepullsecret"
type = "config"
driver_type = "humanitec/template"

driver_inputs = {
secret_refs = jsonencode({
"AWS_ACCESS_KEY_ID" = var.access_key_id_ref
"AWS_SECRET_ACCESS_KEY" = var.secret_access_key_ref
})

values_string = jsonencode({
secret_name = local.secret_name
server = "${var.account_id}.dkr.ecr.${var.region}.amazonaws.com"
aws_account_id = var.account_id
aws_region = var.region
namespace = "$${resources[\"k8s-namespace.default#k8s-namespace\"].outputs.namespace}"

templates = {
manifests = file("${path.module}/templates/manifests.yaml")
outputs = <<EOL
secret_name: {{ .driver.values.secret_name }}
EOL
}
})
}
}
3 changes: 3 additions & 0 deletions humanitec-resource-defs/config/imagepullsecret/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
output "id" {
value = humanitec_resource_definition.main.id
}
10 changes: 10 additions & 0 deletions humanitec-resource-defs/config/imagepullsecret/providers.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
terraform {
required_providers {
humanitec = {
source = "humanitec/humanitec"
version = "~> 1.0"
}
}

required_version = ">= 1.3.0"
}
Loading

0 comments on commit 77d4cfb

Please sign in to comment.