feat: load secrets for OpenStack Helm via a values file

components/10-keystone/README.md

@@ -1,28 +1,5 @@
 # OpenStack Keystone
 
-So unfortunately OpenStack Helm doesn't publish helm charts that can be consumed like
-regular helm charts. You must instead clone two of their git repos side by side and
-build the dependencies manually. They additionally don't split out secrets but instead
-template them into giant config files or even executable scripts that then get stored
-as secrets, a clear violation of <https://12factor.net>. As a result we cannot store
-a declarative config of Keystone and allow users to supply their own secrets.
-
-Due to the above issues, for now we'll skip the ArgoCD ability for this deployment.
-
-## Get OpenStack Helm Ready
-
-You may have done this for another OpenStack component and can share the same
-git clones. This assumes you're doing this from the top level of this repo.
-
-```bash
-# clone the two repos because they reference the infra one as a relative path
-# so you can't use real helm commands
-git clone https://github.com/openstack/openstack-helm
-git clone https://github.com/openstack/openstack-helm-infra
-# update the dependencies cause we can't use real helm references
-./scripts/openstack-helm-depend-sync.sh keystone
-```
-
 ## Label the node(s)
 
 In order to deploy Openstack control plane, at least one of the Kubernetes
@@ -48,16 +25,14 @@ Secrets Reference:
 - keystone-rabbitmq-password is the RabbitMQ password for the keystone user.
 
 ```bash
-helm --namespace openstack template \
-    keystone \
-    ./openstack-helm/keystone/ \
-    -f components/openstack-2023.1-jammy.yaml \
-    -f components/10-keystone/aio-values.yaml \
-    --set endpoints.identity.auth.admin.password="$(kubectl --namespace openstack get secret keystone-admin -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.oslo_db.auth.keystone.password="$(kubectl --namespace openstack get secret keystone-db-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.oslo_messaging.auth.keystone.password="$(kubectl --namespace openstack get secret keystone-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --post-renderer $(git rev-parse --show-toplevel)/scripts/openstack-helm-sealed-secrets.sh \
-    | kubectl -n openstack apply -f -
+# create secrets yaml file if you're not already storing or providing it differently
+./scripts/gen-os-secrets.sh secret-openstack.yaml
+
+kubectl kustomize \
+  --enable-helm \
+  --load-restrictor LoadRestrictionsNone \
+  components/10-keystone \
+  | kubectl -n openstack apply -f -
 ```
 
 At this point Keystone will go through some initialization and start up.

components/10-keystone/kustomization.yaml

@@ -5,3 +5,16 @@ kind: Kustomization
 resources:
   - keystone-mariadb-db.yaml
   - keystone-rabbitmq-queue.yaml
+
+helmGlobals:
+  chartHome: ../../charts/
+helmCharts:
+  - name: keystone
+    namespace: openstack
+    releaseName: keystone
+    repo: https://tarballs.opendev.org/openstack/openstack-helm/
+    version: 0.3.7
+    valuesFile: aio-values.yaml
+    additionalValuesFiles:
+    - ../openstack-2023.1-jammy.yaml
+    - ../../secret-openstack.yaml

components/13-ironic/README.md

@@ -1,28 +1,5 @@
 # OpenStack Ironic
 
-So unfortunately OpenStack Helm doesn't publish helm charts that can be consumed like
-regular helm charts. You must instead clone two of their git repos side by side and
-build the dependencies manually. They additionally don't split out secrets but instead
-template them into giant config files or even executable scripts that then get stored
-as secrets, a clear violation of <https://12factor.net>. As a result we cannot store
-a declarative config of Keystone and allow users to supply their own secrets.
-
-Due to the above issues, for now we'll skip the ArgoCD ability for this deployment.
-
-## Get OpenStack Helm Ready
-
-You may have done this for another OpenStack component and can share the same
-git clones. This assumes you're doing this from the top level of this repo.
-
-```bash
-# clone the two repos because they reference the infra one as a relative path
-# so you can't use real helm commands
-git clone https://github.com/openstack/openstack-helm
-git clone https://github.com/openstack/openstack-helm-infra
-# update the dependencies cause we can't use real helm references
-./scripts/openstack-helm-depend-sync.sh ironic
-```
-
 ## Deploy Ironic
 
 NOTE: The PXE service currently has the host network devices mapped into
@@ -45,17 +22,14 @@ Secrets Reference:
   is created by the ks-user job using the keystone-admin credential.
 
 ```bash
-helm --namespace openstack template \
-    ironic \
-    ./openstack-helm/ironic/ \
-    -f components/openstack-2023.1-jammy.yaml \
-    -f components/13-ironic/aio-values.yaml \
-    --set endpoints.identity.auth.admin.password="$(kubectl --namespace openstack get secret keystone-admin -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.oslo_db.auth.ironic.password="$(kubectl --namespace openstack get secret ironic-db-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.oslo_messaging.auth.ironic.password="$(kubectl --namespace openstack get secret ironic-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.identity.auth.ironic.password="$(kubectl --namespace openstack get secret ironic-keystone-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --post-renderer $(git rev-parse --show-toplevel)/scripts/openstack-helm-sealed-secrets.sh \
-    | kubectl -n openstack apply -f -
+# create secrets yaml file if you're not already storing or providing it differently
+./scripts/gen-os-secrets.sh secret-openstack.yaml
+
+kubectl kustomize \
+  --enable-helm \
+  --load-restrictor LoadRestrictionsNone \
+  components/13-ironic \
+  | kubectl -n openstack apply -f -
 ```
 
 At this point Ironic will go through some initialization and start up.

components/13-ironic/kustomization.yaml

@@ -5,3 +5,16 @@ kind: Kustomization
 resources:
   - ironic-mariadb-db.yaml
   - ironic-rabbitmq-queue.yaml
+
+helmGlobals:
+  chartHome: ../../charts/
+helmCharts:
+  - name: ironic
+    namespace: openstack
+    releaseName: ironic
+    repo: https://tarballs.opendev.org/openstack/openstack-helm/
+    version: 0.2.10
+    valuesFile: aio-values.yaml
+    additionalValuesFiles:
+    - ../openstack-2023.1-jammy.yaml
+    - ../../secret-openstack.yaml

components/openstack-secrets.tpl.yaml

@@ -0,0 +1,39 @@
+# The purpose of this file is to serve as a template for OpenStack Helm
+# based secrets values that are necessary for OpenStack Helm to populate
+# it's configmap-etc, which is really a secret with connection strings
+---
+
+endpoints:
+
+  # 'identity' endpoints are for keystone access
+  identity:
+    auth:
+      # this is the 'admin' user created in keystone by the initial start
+      # and used by the other services to create their service accounts
+      # and endpoint in the service catalog.
+      admin:
+        password: "${KEYSTONE_ADMIN_PASSWORD}"
+      # this user is the service account that ironic uses
+      ironic:
+        password: "${IRONIC_KEYSTONE_PASSWORD}"
+
+  # 'oslo_db' is for MariaDB
+  oslo_db:
+    auth:
+      # this is what the keystone service uses to connect to MariaDB
+      keystone:
+        password: "${KEYSTONE_DB_PASSWORD}"
+      # this is what the ironic service uses to connect to MariaDB
+      ironic:
+        password: "${IRONIC_DB_PASSWORD}"
+
+  # 'oslo_messaging' is for RabbitMQ
+  oslo_messaging:
+    auth:
+      # this is what the keystone service uses to connect to RabbitMQ
+      keystone:
+        password: "${KEYSTONE_RABBITMQ_PASSWORD}"
+      # this is what the ironic service uses to connect to RabbitMQ
+      ironic:
+        password: "${IRONIC_RABBITMQ_PASSWORD}"
+...

docs/install-understack-ubuntu-k3s.md

@@ -323,6 +323,12 @@ git clone https://github.com/openstack/openstack-helm-infra
 ./scripts/openstack-helm-depend-sync.sh ironic
 ```
 
+Load the secrets values file from the cluster:
+
+```bash
+./scripts/gen-os-secrets.sh secret-openstack.yaml
+```
+
 Label the kubernetes nodes as being openstack enabled:
 ```bash
 kubectl label node $(kubectl get nodes -o 'jsonpath={.items[*].metadata.name}') openstack-control-plane=enabled
@@ -337,10 +343,7 @@ helm --namespace openstack template \
     keystone \
     ./openstack-helm/keystone/ \
     -f components/10-keystone/aio-values.yaml \
-    --set endpoints.identity.auth.admin.password="$(kubectl --namespace openstack get secret keystone-admin -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.oslo_db.auth.keystone.password="$(kubectl --namespace openstack get secret keystone-db-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.oslo_messaging.auth.keystone.password="$(kubectl --namespace openstack get secret keystone-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --post-renderer $(git rev-parse --show-toplevel)/scripts/openstack-helm-sealed-secrets.sh \
+    -f secret-openstack.yaml \
     | kubectl -n openstack apply -f -
 ```
 
@@ -382,11 +385,7 @@ helm --namespace openstack template \
     ironic \
     ./openstack-helm/ironic/ \
     -f components/13-ironic/aio-values.yaml \
-    --set endpoints.identity.auth.admin.password="$(kubectl --namespace openstack get secret keystone-admin -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.oslo_db.auth.ironic.password="$(kubectl --namespace openstack get secret ironic-db-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.oslo_messaging.auth.ironic.password="$(kubectl --namespace openstack get secret ironic-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --set endpoints.identity.auth.ironic.password="$(kubectl --namespace openstack get secret ironic-keystone-password -o jsonpath='{.data.password}' | base64 -d)" \
-    --post-renderer $(git rev-parse --show-toplevel)/scripts/openstack-helm-sealed-secrets.sh \
+    -f secret-openstack.yaml \
     | kubectl -n openstack apply -f -
 ```
 
@@ -410,4 +409,4 @@ If everything is working, you should see output similar to the following:
 
 References:
 
-* [https://github.com/rackerlabs/understack/blob/main/components/13-ironic/README.md](https://github.com/rackerlabs/understack/blob/main/components/13-ironic/README.md)
+* [https://github.com/rackerlabs/understack/blob/main/components/13-ironic/README.md](https://github.com/rackerlabs/understack/blob/main/components/13-ironic/README.md)

scripts/gen-os-secrets.sh

@@ -0,0 +1,47 @@
+#!/bin/sh
+
+if [ $# -ne 1 ]; then
+    echo "$(basename "$0") <output-file>" >&2
+    exit 1
+fi
+
+set -o pipefail
+
+if ! type -p yq > /dev/null; then
+    echo "You must have yq installed to use this script" >&2
+    exit 1
+fi
+
+if ! type -p kubectl > /dev/null; then
+    echo "You must have kubectl installed to use this script" >&2
+    exit 1
+fi
+
+KUSTOMIZE_VERSION=$(kubectl version --client -o yaml | yq .kustomizeVersion)
+if ! (echo -e "v5.0.0\n$KUSTOMIZE_VERSION" | sort -V -C); then
+  echo "kustomize needs to be at version 5.0.0 or newer (comes with kubectl 1.27+)"
+  exit 1
+fi
+
+SCRIPTS_DIR="$(dirname "$0")"
+
+echo "This script will attempt to look up the existing values this repo used"
+echo "or will generate new values. The output below will be related to that."
+
+# keystone admin
+export KEYSTONE_ADMIN_PASSWORD=$(kubectl -n openstack get secret keystone-admin -o jsonpath='{.data.password}' | base64 -d || "${SCRIPTS_DIR}/pwgen.sh")
+# keystone mariadb
+export KEYSTONE_DB_PASSWORD=$(kubectl -n openstack get secret keystone-db-password -o jsonpath='{.data.password}' | base64 -d || "${SCRIPTS_DIR}/pwgen.sh")
+# keystone rabbitmq
+export KEYSTONE_RABBITMQ_PASSWORD=$(kubectl -n openstack get secret keystone-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d || "${SCRIPTS_DIR}/pwgen.sh")
+
+# ironic keystone service account
+export IRONIC_KEYSTONE_PASSWORD=$(kubectl -n openstack get secret ironic-keystone-password -o jsonpath='{.data.password}' | base64 -d || "${SCRIPTS_DIR}/pwgen.sh")
+# ironic mariadb
+export IRONIC_DB_PASSWORD=$(kubectl -n openstack get secret ironic-db-password -o jsonpath='{.data.password}' | base64 -d || "${SCRIPTS_DIR}/pwgen.sh")
+# ironic rabbitmq
+export IRONIC_RABBITMQ_PASSWORD=$(kubectl -n openstack get secret ironic-rabbitmq-password -o jsonpath='{.data.password}' | base64 -d || "${SCRIPTS_DIR}/pwgen.sh")
+
+yq '(.. | select(tag == "!!str")) |= envsubst' \
+    "${SCRIPTS_DIR}/../components/openstack-secrets.tpl.yaml" \
+    > "$1"