Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added snapshot releaser that runs every night #311

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 136 additions & 0 deletions .github/workflows/publish-new.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# Copyright 2024 Defense Unicorns
# SPDX-License-Identifier: AGPL-3.0-or-later OR LicenseRef-Defense-Unicorns-Commercial

name: Publish UDS Nutanix SWF Bundle

on:
# triggered by tag-and-release.yaml and snapshot-release.yaml
workflow_call:
inputs:
snapshot:
description: 'true - for snapshot release'
required: true
type: boolean

jobs:
publish-uds-nutanix-swf-bundle:
strategy:
fail-fast: false
matrix:
flavor: [upstream, registry1, unicorn]
runs-on: "uds-ubuntu-big-boy-8-core"
name: Publish bundle

permissions:
contents: read
packages: write
id-token: write # This is needed for OIDC federation.

steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Environment setup
uses: ./.github/actions/setup
with:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
ghToken: ${{ secrets.GITHUB_TOKEN }}
chainguardIdentity: ${{ secrets.CHAINGUARD_IDENTITY }}

- name: Init zarf cache
uses: actions/cache@6849a6489940f00c2f30c0fb92c6274307ccb58a # v4.1.2
with:
path: "~/.zarf-cache"
key: zarf-cache

- name: (Snapshot) Get snapshot version using git commit short sha and date
if: ${{ inputs.snapshot }}
run: |
SHORT_SHA=$(git rev-parse --short HEAD)
RELEASE_DATE=$(date +'%Y-%m-%d')
echo "SNAPSHOT_VERSION=${RELEASE_DATE}-${SHORT_SHA}" >> $GITHUB_ENV
echo "PUBLISH_ARGS=--set VERSION=${RELEASE_DATE}-${SHORT_SHA}" >> $GITHUB_ENV

- name: (Snapshot) Set versions to snapshot
if: ${{ inputs.snapshot }}
run: |
yq -ei '.metadata.version=env(SNAPSHOT_VERSION), (.packages[]|select(has("ref"))|select(.name=="core")).ref=env(SNAPSHOT_VERSION)' bundles/k3d-standard/uds-bundle.yaml
yq -ei '.metadata.version=env(SNAPSHOT_VERSION), (.packages[]|select(has("ref"))|select(.name=="core-base")).ref=env(SNAPSHOT_VERSION)' bundles/k3d-slim-dev/uds-bundle.yaml
yq -ei '.metadata.version=env(SNAPSHOT_VERSION), (.packages[]|select(has("ref"))|select(.name=="core-identity-authorization")).ref=env(SNAPSHOT_VERSION)' bundles/k3d-slim-dev/uds-bundle.yaml
yq -ei '.metadata.version=env(SNAPSHOT_VERSION)' packages/standard/zarf.yaml
yq -ei '.metadata.version=env(SNAPSHOT_VERSION)' packages/base/zarf.yaml
yq -ei '.metadata.version=env(SNAPSHOT_VERSION)' packages/identity-authorization/zarf.yaml

- name: Build and publish rke2 bundle
run: |
uds run create-bundle-rke2 --set EXTRA_ARGS="--skip-sbom=false"
uds inspect --sbom build/uds-bundle-software-factory-nutanix-rke2*.tar.zst
uds publish build/uds-bundle-software-factory-nutanix-rke2-amd64-*.tar.zst oci://ghcr.io/defenseunicorns/uds-bundle --no-progress

- name: Upload SBOM to Release
id: upload-sbom-to-release
env:
GH_TOKEN: ${{ github.token }}
run: |
TAG_NAME="v${{ inputs.tag-name }}"
gh release upload "${TAG_NAME}" "$(find . -maxdepth 1 -type f -name "*-sboms.tar" -print0 | xargs -0 echo)"

- name: Cleanup
run: |
uds run clean

# Publish package and bundle to destination repository
- name: Publish Standard Package
run: uds run -f tasks/publish.yaml standard-package --set FLAVOR=${{ matrix.flavor }} --set SNAPSHOT="${{ inputs.snapshot }}" ${PUBLISH_ARGS} --no-progress

- name: Publish Upstream Flavored Bundles
if: ${{ matrix.flavor == 'upstream' }}
run: uds run -f tasks/publish.yaml bundles --set SNAPSHOT="${{ inputs.snapshot }}" ${PUBLISH_ARGS} --no-progress

- name: Save logs
if: always()
uses: defenseunicorns/uds-core/.github/actions/save-logs@3c65fe6767fe05a362ba364dd66deb572f0758a5 # 0.32.1
with:
suffix: -${{ matrix.flavor }}

publish-uds-core-layers:
if: ${{ !inputs.snapshot }}
strategy:
fail-fast: false
matrix:
flavor: [upstream, registry1, unicorn]
layer: [base, identity-authorization, runtime-security, backup-restore, logging, metrics-server, monitoring]
arch: [amd64, arm64]
exclude:
- flavor: registry1
arch: arm64
runs-on: ${{ matrix.arch == 'arm64' && 'uds-ubuntu-arm64-4-core' || 'uds-ubuntu-big-boy-4-core' }}
name: Publish package layers

permissions:
contents: read
packages: write
id-token: write # This is needed for OIDC federation.

steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Environment setup
uses: defenseunicorns/uds-core/.github/actions/setup@3c65fe6767fe05a362ba364dd66deb572f0758a5 # 0.32.1
with:
registry1Username: ${{ secrets.IRON_BANK_ROBOT_USERNAME }}
registry1Password: ${{ secrets.IRON_BANK_ROBOT_PASSWORD }}
ghToken: ${{ secrets.GITHUB_TOKEN }}
chainguardIdentity: ${{ secrets.CHAINGUARD_IDENTITY }}

- name: Test and Publish Core Package Layer
run: uds run -f tasks/publish.yaml single-layer --set FLAVOR=${{ matrix.flavor }} --set LAYER=${{ matrix.layer }} --no-progress

- name: Debug Output
if: ${{ always() && !inputs.snapshot }}
uses: defenseunicorns/uds-core/.github/actions/debug-output@3c65fe6767fe05a362ba364dd66deb572f0758a5 # 0.32.1
- name: Save logs
if: always()
uses: defenseunicorns/uds-core/.github/actions/save-logs@3c65fe6767fe05a362ba364dd66deb572f0758a5 # 0.32.1
with:
suffix: -${{ matrix.flavor }}-${{ matrix.layer }}-${{ matrix.arch }}
44 changes: 44 additions & 0 deletions .github/workflows/snapshot-release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Copyright 2024 Defense Unicorns
# SPDX-License-Identifier: AGPL-3.0-or-later OR LicenseRef-Defense-Unicorns-Commercial

# Based off of https://github.com/defenseunicorns/uds-core/blob/main/.github/workflows/snapshot-release.yaml

name: Release Bundle Snapshot

on:
schedule:
- cron: "0 11 * * *"
workflow_dispatch:

jobs:
publish-snapshot-release:
permissions:
contents: write
packages: write
id-token: write
uses: ./.github/workflows/publish.yaml
with:
snapshot: true
secrets: inherit

update-snapshot-latest-tag:
needs: publish-snapshot-release
env:
GH_TOKEN: ${{ github.token }}
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-tags: 'true'
- name: Update snapshot-latest tag
run: |
# get last release tag
LAST_RELEASE_TAG="$(gh release list --exclude-drafts --exclude-pre-releases --json tagName --jq '.[0].tagName')"
# cleanup old release
if gh release view snapshot-latest; then
gh release delete snapshot-latest --cleanup-tag -y
fi
# generate new release
gh release create snapshot-latest -t "snapshot-latest" --prerelease --notes-start-tag "${LAST_RELEASE_TAG}" --generate-notes
22 changes: 10 additions & 12 deletions .github/workflows/tag-and-release.yaml
Original file line number Diff line number Diff line change
@@ -1,37 +1,35 @@
name: Tag, Release and Publish UDS Bundle
# Copyright 2024 Defense Unicorns
# SPDX-License-Identifier: AGPL-3.0-or-later OR LicenseRef-Defense-Unicorns-Commercial

name: Release UDS Nutanix SWF Bundle

on:
push:
branches:
- main

permissions:
contents: read
packages: write
id-token: write

jobs:
tag-new-version:
permissions: write-all
runs-on: ubuntu-latest
outputs:
release_created: ${{ steps.release-flag.outputs.release_created }}
release_tag: ${{ steps.release-tag.outputs.release_tag }}
steps:
- name: Create Release Tag
- name: Create release tag
id: tag
uses: googleapis/release-please-action@7987652d64b4581673a76e33ad5e98e3dd56832f # v4.1.3
- id: release-flag
run: echo "release_created=${{ steps.tag.outputs.release_created || false }}" >> "$GITHUB_OUTPUT"
- id: release-tag
run: echo "release_tag=${{ steps.tag.outputs.version || false }}" >> "$GITHUB_OUTPUT"

# Publish the uds bundle
publish-uds-bundle-rke2:
needs: tag-new-version
permissions: write-all
if: ${{ needs.tag-new-version.outputs.release_created == 'true'}}
permissions:
contents: write
packages: write
id-token: write
uses: ./.github/workflows/publish-bundle-rke2.yaml
with:
tag-name: ${{ needs.tag-new-version.outputs.release_tag }}
snapshot: false
secrets: inherit
160 changes: 160 additions & 0 deletions tasks/pg-admin-manifests/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# https://www.enterprisedb.com/blog/how-deploy-pgadmin-kubernetes
---
apiVersion: v1
kind: Namespace
metadata:
name: pgadmin
---
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: pgadmin

Check failure on line 12 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

12:2 [indentation] wrong indentation: expected 2 but found 1
namespace: pgadmin
data:
pgadmin-password: Q2otblhGdEBaa3F6RENfS1ZONEcK

Check failure on line 15 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

15:2 [indentation] wrong indentation: expected 2 but found 1
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pgadmin-config

Check failure on line 20 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

20:2 [indentation] wrong indentation: expected 2 but found 1
namespace: pgadmin
data:
servers.json: |

Check failure on line 23 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

23:2 [indentation] wrong indentation: expected 2 but found 1
{
"Servers": {
"gitlab": {
"Name": "PostgreSQL DB",
"Group": "Servers",
"Port": 5432,
"Username": "postgres",
"Host": "gitlab-pg.mtsi-dev.bigbang.dev",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
},
"sonarqube": {
"Name": "PostgreSQL DB",
"Group": "Servers",
"Port": 5432,
"Username": "postgres",
"Host": "sonarqube-pg.mtsi-dev.bigbang.dev",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
},
"keycloak": {
"Name": "PostgreSQL DB",
"Group": "Servers",
"Port": 5432,
"Username": "postgres",
"Host": "keycloak-pg.mtsi-dev.bigbang.dev",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
},
"jira": {
"Name": "PostgreSQL DB",
"Group": "Servers",
"Port": 5432,
"Username": "postgres",
"Host": "jira-pg.mtsi-dev.bigbang.dev",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
},
"confluence": {
"Name": "PostgreSQL DB",
"Group": "Servers",
"Port": 5432,
"Username": "postgres",
"Host": "confluence-pg.mtsi-dev.bigbang.dev",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
},
"mattermost": {
"Name": "PostgreSQL DB",
"Group": "Servers",
"Port": 5432,
"Username": "postgres",
"Host": "mattermost-pg.mtsi-dev.bigbang.dev",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
},
"nexus": {
"Name": "PostgreSQL DB",
"Group": "Servers",
"Port": 5432,
"Username": "postgres",
"Host": "nexus-pg.mtsi-dev.bigbang.dev",
"SSLMode": "prefer",
"MaintenanceDB": "postgres"
}
}
}
---
apiVersion: v1
kind: Service
metadata:
name: pgadmin-service

Check failure on line 95 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

95:2 [indentation] wrong indentation: expected 2 but found 1
namespace: pgadmin
spec:
ports:

Check failure on line 98 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

98:2 [indentation] wrong indentation: expected 2 but found 1
- protocol: TCP

Check failure on line 99 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

99:2 [indentation] wrong indentation: expected 3 but found 1
port: 80
targetPort: http
selector:
app: pgadmin
type: NodePort
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: pgadmin

Check failure on line 109 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

109:2 [indentation] wrong indentation: expected 2 but found 1
namespace: pgadmin
spec:
serviceName: pgadmin-service

Check failure on line 112 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

112:2 [indentation] wrong indentation: expected 2 but found 1
podManagementPolicy: Parallel
replicas: 1
updateStrategy:
type: RollingUpdate
selector:
matchLabels:
app: pgadmin
template:
metadata:
labels:
app: pgadmin
spec:
terminationGracePeriodSeconds: 10
containers:
- name: pgadmin
image: dpage/pgadmin4:5.4
imagePullPolicy: Always
env:
- name: PGADMIN_DEFAULT_EMAIL

Check failure on line 131 in tasks/pg-admin-manifests/manifest.yaml

View workflow job for this annotation

GitHub Actions / run

131:10 [indentation] wrong indentation: expected 11 but found 9
value: [email protected]
- name: PGADMIN_DEFAULT_PASSWORD
valueFrom:
secretKeyRef:
name: pgadmin
key: pgadmin-password
ports:
- name: http
containerPort: 80
protocol: TCP
volumeMounts:
- name: pgadmin-config
mountPath: /pgadmin4/servers.json
subPath: servers.json
readOnly: true
- name: pgadmin-data
mountPath: /var/lib/pgadmin
volumes:
- name: pgadmin-config
configMap:
name: pgadmin-config
volumeClaimTemplates:
- metadata:
name: pgadmin-data
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 3Gi
Loading
Loading