Skip to content
This repository has been archived by the owner on Oct 3, 2024. It is now read-only.

Commit

Permalink
Initial setup for repo (#1)
Browse files Browse the repository at this point in the history
* Initial setup

* Add workflow for init package and add EKS package

* Add script to create IAM roles for IRSA auth in workflow

* Remove command to remove tmpdir

* Add PR as workflow trigger

* Install same version of Go that Zarf users

* Fix sed command and update github actions output syntax

* Update ecr bootstrap script

* Switch to using bash for ecr bootstrap script to use an array

* Update readme header

* Use gsed for mac in iam bootstrap script for local testing

* Fetch pepr pod logs onFailure

* Dynamically set AWS account ID in iam script

* Specify pepr-system namespace in kubectl logs

* Update the way github outputs are set

* Fix github outputs

* Update run syntax in workflow

* Remove onFailure action from package

* Rename bootstrap directory to hack to align with k8s convention

* Run tests for private and public ECR registry in parallel

* Fix zarf command in teardown step

* Create build dir before moving zarf binary to it

* Use sudo to add exec permissions to zarf binary

* Fix cluster name input

* Download latest zarf rc version instead of build from source

* Move zarf binary to build dir

* Specify multiple availability zones for EKS cluster

* Add single quotes

* Change AZ in EKS config

* Remove availability zones

* Use setup-zarf action to install zarf binary

* Import zarf-agent via init skeleton package

* Remove local zarf-agent package

* Run zarf init from build directory

* Specify availability zones in eks config file to avoid capacity error

* Add comment for context about specifying AZs in eks config

* Add zarf as prereq to README.md

* Update Pepr to v0.14.0

Refactor ECR capability to use new K8s fluent client

Split ECR capability up into separate files for private and public APIs

* Remove unused make targets for zarf agent

* Rebuild Pepr module for changes to take effect

* Use zarf vars directly in ECR bootstrap script

* Empty commit to trigger workflows

* Update docs and Makefile and format module

* Update Watch() back to Mutate()

* Remove npm ci from build and test make targets

* Add a workaround for updating webhook status in package secret

Pepr fluent client does not currently support force Apply()

Workaround is to clear managedFields and updated webhook status in a single Patch request

* Replace IAM role shell script with pulumi typescript program

* Install NodeJS and deps in workflow

* Add CODEOWNERS file

* Add issue templates

* Add pull request template

* Refactor cluster teardown in EKS package

* Update EKS package version in deploy command

* Move onRemove action for EKS cluster into single package component

Refactor directory structure for pulumi program

* Fix filepath to IAM JSON files in pulumi module

Add make targets for deploying and removing EKS package

* Update Pepr to 0.14.1

Replace Patch with Force Apply to update package secret in webhook
Ensure Pepr module is rebuilt in the build job

* Specify package-lock.json in root of repo as cache path

* Include ts files only in capabilities dir in tsconfig to fix error in CI

Remove ./ syntax in npm cache path

* Add git-server as optional component to init package

* Add workflow to run unit tests for Pepr module

Refactor getRepositoryNames() in Pepr module to handle images with digest ref
Add test case to unit tests to validate handling of images with digest

* Add renovate.json config file

* Bump eksctl version to v0.162.0

* Add ADR for using Pulumi TypeScript SDK

* Add OpenSSF Scorecard workflow

* Downgrade eksctl to v0.160.0 due to goroutine race condition bug

* Add option to locally build credential helper image

* Install Go in build job to build credential helper binary

* Rename openssf scorecard workflow to scorecard.yml

* Fix numbered list in ADR

* Move zarf init options/vars to zarf-config.toml

Update README.md

* Uncomment log_level in zarf-config.toml

* Run zarf init command for public ECR test from root of repo

* Update README.md

* Remove docker build command comment from Dockerfile

* Remove components from zarf config file to fix cluster teardown

* Update dependencies and add workflow to scan CVEs

* Install Zarf binary in scan-cves workflow to generate SBOM

* Add revive lint workflow for Go CronJob

* Add comment to .gype.yaml about false positive for x/net pkg

* Fix linting errors

* Add scan-labels workflow

* Add codeql workflow and config file

* Add step to build Go binary in scan-codeql workflow

* Update @babel/core package to 7.23.2 to patch critical CVE-2023-45133

* Rebuilt pepr module after updating deps

* Moves credential-helper src code to root of repo

Exclude ./binaries directory from grype scan
Grype was detecting CVEs in the eksctl binaries in ./binaries directory
Since the scan now excludes the directory, there are no vulns found
This allows us to be able to comment out the .grype.yaml file

Adds cve-report make target

* Add release workflow

* Update development.md doc

* Expose Cron schedule as deploy-time variable

Default set to run once every hour at the beginning of the hour

* Remove version field from zarf.yaml

* Remove slack notification steps from aws init package workflow

This workflow is set to run on every PR commit, which would make for a noisy slack channel

* Fix publish package make target in release workflow

* Update permissions needed for AWS CLI in README prereqs

* Remove step to deploy workfloads into the cluster from README

* Remove setup go step from scan-cves workflow

* Add setup-go and build binary steps to release workflow

Update publish pkg make target to use zarf version for init pkg name

Comment out step to sign the image and add a TODO to setup repo secrets for cosign

* Update .github/ISSUE_TEMPLATE/bug_report.md

Co-authored-by: razzle <[email protected]>

* Update .github/ISSUE_TEMPLATE/feature_request.md

Co-authored-by: razzle <[email protected]>

* Update .github/ISSUE_TEMPLATE/tech_debt.md

Co-authored-by: razzle <[email protected]>

* Update .github/ISSUE_TEMPLATE/ux_test.md

Co-authored-by: razzle <[email protected]>

* Update .github/workflows/scan-lint.yml

Co-authored-by: razzle <[email protected]>

* Update README.md

Co-authored-by: razzle <[email protected]>

* Add typescript to codql workflow language matrix

* Update Node.js to version 20 in workflows

* Update pulumi and pulumi/aws packages

Remove .grype.yaml file due to no CVEs being ignored
Update test-cves make target to not exclude iam/ dir from scanning

* Include all package.json and package-lock.json paths in scan-cves workflow

* Remove needs-tutorial label from banned labels in scan-labels workflow

* Add cons to pulumi typescript SDK in ADR

* Move zarf init command and delete repos commands to make targets

Use ZARF_CONFIG env var to use config file instead of moving init package

* Fix make targets for deleting ECR repos

Makefiles use /bin/sh by default which does not support arrays

* Remove array assignment from make target

* Add make target for linting typescript code in the repo

Add step to scan-lint workflow to lint typescript code

* Update development.md doc

* Rename var in ECRPublic class to be less redundant

* Add return statement to end execution in isECRregistry()

Previously we were only logging a warning message if Zarf is configured to use an internal registry.
This behavior has been changed to log a warning a message and return if using an internal registry

* Throw an error if input images array is null in getRepositoryNames()

* Call readFileSync() method directly in IaC

* Use context.WithTimeout() to prevent binary from waiting forever

* Exclude binaries directory from CVE scan

* Remove images field from DeployedComponent interface

* Add build/ dir to ignorePatterns in .eslintrc.json

* Add comments to typescript code and update package.json description

* Split logic into functions and separate into modules

* Update Pepr to v0.14.2

* Update node dependencies

* Update pepr manifest

* Update node deps for pulumi IaC

* Add link to docs and regex101 for public ECR URL pattern

* Add link to docs and regex101 for private ECR pattern

* Format and rebuild module

* Change zarf-config.toml to zarf-config.yaml

Part of the local dev and ci workflow is to programatically update this config file
There are much better parsing libs for yaml than toml for node so switched to yaml
Updates the update-zarf-config.mjs script to use yaml parser lib instead of regex

* Update README.md to use yaml config file

* Update README.md

* Update eksctl version and ebs driver version in EKS package

* Update zarf config file

* Update eks package and makefile

* Update update-zarf-config.mjs script

Use parseDocument and toString methods to preserve blank lines and comments

Use has, get, and set methods to safely update the YAML config file

* Add a package.json file to manage yaml dep in .mjs script

Move to hack/update-zarf-config dir

* Add make target to install node deps in every package.json in the repo

Run this make target in both validate ecr jobs

* Stop tracking zarf-config.yaml in git history

* Update update-zarf-config script to make a new copy of the update file

git ignore the generated zarf-config.yaml file
This reduces the likelihood of commiting an auto-updated config file

* Add a system/workflow to keep ts interfaces in sync with Zarf structs

* Add root go.mod to workspace and rename nested go.mod

* Update pepr manifest

* Add make target to ensure build dir exists for cve report

* Update indirect deps to resolve CVEs

* Update pulumi deps

* Add Go grpc CVE to grype ignore file

* Simplify gen-schema script

Removes cobra as a dependency
We are running go run main.go so cobra is not needed

* Fix lint errors in gen-schema script

* Fix go run command in gen-schema.sh

* Change let to const in update-zarf-config script

* Use handlebars to update IAM role placeholders instead of regex

* Run make format-ts

* Add comments to explain update-zarf-config script

* Update image ref parsing

* Update Zarf version to v0.31.0

Update Go deps

* Update Pepr to v0.15.0

* Run make gen-schema and rebuild module

* Fix Zarf config file in README.md

* Use us-west-2 region for EKS clusters

* Update AWS region in IAM roles to us-west-2 for EKS OIDC

* Specify AWS region in eksctl write-kubeconfig cmd in EKS pkg

* Specify us-west-2 region in Pulumi.yaml project config

* Add metadata.version to zarf.yaml

* Update README.md to not hard code the version in the init pkg name

* Remove metadata.version from zarf.yaml

* Add ZARF_CONFIG env var to release-aws-init-package make target

* Add zarf-sbom dir to .gitignore

* Update .github/ISSUE_TEMPLATE/tech_debt.md

Co-authored-by: Wayne Starr <[email protected]>

* Remove extra char in .grype.yaml

* Add contributing guide that points to the Zarf contributing guide

---------

Co-authored-by: razzle <[email protected]>
Co-authored-by: Wayne Starr <[email protected]>
  • Loading branch information
3 people authored Nov 15, 2023
1 parent c0dd525 commit b3b7a59
Show file tree
Hide file tree
Showing 78 changed files with 16,273 additions and 2 deletions.
25 changes: 25 additions & 0 deletions .eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"env": {
"browser": false,
"es2021": true
},
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/recommended"
],
"parser": "@typescript-eslint/parser",
"parserOptions": {
"ecmaVersion": 2022
},
"plugins": [
"@typescript-eslint"
],
"ignorePatterns": [
"node_modules",
"dist",
"hack",
"build",
"capabilities/zarf-types.ts"
],
"root": true
}
30 changes: 30 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: possible-bug
assignees: ''
---

### Environment

Device and OS:
App version:
Kubernetes distro being used:
Other:

### Steps to reproduce

1.

### Expected result

### Actual Result

### Visual Proof (screenshots, videos, text, etc)

### Severity/Priority

### Additional Context

Add any other context or screenshots about the technical debt here.
25 changes: 25 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: 'enhancement'
assignees: ''
---

### Is your feature request related to a problem? Please describe.

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

### Describe the solution you'd like

- **Given** a state
- **When** an action is taken
- **Then** something happens

### Describe alternatives you've considered

(optional) A clear and concise description of any alternative solutions or features you've considered.

### Additional context

Add any other context or screenshots about the feature request here.
19 changes: 19 additions & 0 deletions .github/ISSUE_TEMPLATE/tech_debt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
name: Tech debt
about: Record something that should be investigated or refactored in the future.
title: ''
labels: 'tech-debt'
assignees: ''
---

### Describe what should be investigated or refactored

A clear and concise description of what should be changed/researched. Ex. This piece of the code is not DRY enough [...]

### Links to any relevant code

(optional) i.e. - <https://github.com/defenseunicorns/zarf-init-aws/blob/main/README.md?plain=1#L1>

### Additional context

Add any other context or screenshots about the technical debt here.
26 changes: 26 additions & 0 deletions .github/ISSUE_TEMPLATE/ux_test.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
name: UX Test
about: Record something that should be investigated to test User Experience
title: ''
labels: 'ux'
assignees: ''
---

## Driving Questions

What are we hoping to validate?

## Testing Plan

User Persona:
Sample Group:

- [ ] Use Checklist for Tasks

## Additional context

Add any other context or screenshots about the UX test here.

Related to issue: #

## Link to Test & Results
6 changes: 6 additions & 0 deletions .github/codeql.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
paths-ignore:
- build/**

query-filters:
- exclude:
id: go/path-injection
20 changes: 20 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## Description

...

## Related Issue

Fixes #
<!-- or -->
Relates to #

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Other (security config, docs update, etc)

## Checklist before merging

- [ ] Test, docs, adr added or updated as needed
- [ ] [Contributor Guide Steps](https://github.com/defenseunicorns/zarf-init-aws/blob/main/CONTRIBUTING.md) followed
75 changes: 75 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Publish Zarf Init Package for AWS on Tag

permissions:
contents: read

on:
push:
tags:
- "v*"

jobs:
release:
runs-on: ubuntu-latest
permissions:
packages: write
contents: write
steps:
# Checkout the repo and setup the tooling for this job
- name: Checkout
uses: actions/checkout@3df4ab11eba7bda6032a0b82a6bb43b11571feac # v4.0.0
with:
fetch-depth: 0

- name: Install latest version of Zarf
uses: defenseunicorns/setup-zarf@main

- name: Install tools
uses: defenseunicorns/zarf/.github/actions/install-tools@main

- name: Setup Go
uses: defenseunicorns/zarf/.github/actions/golang@main

- name: Build ECR credential-helper binary
run: make build-credential-helper-linux-amd

- name: "ECR Credential Helper: Login to GHCR"
uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3.0.0
with:
registry: ghcr.io
username: dummy
password: ${{ github.token }}

- name: "ECR Credential Helper: Build and Publish the Image"
run: docker buildx build --push --platform linux/amd64 --tag ghcr.io/defenseunicorns/zarf-init-aws/ecr-credential-helper:$GITHUB_REF_NAME .

# TODO@jeff-mccoy: Setup cosign signing key secrets in repo
# - name: "ECR Credential Helper: Sign the Image"
# run: cosign sign --key awskms:///${{ secrets.COSIGN_AWS_KMS_KEY }} -a release-engineer=https://github.com/${{ github.actor }} -a version=$GITHUB_REF_NAME ghcr.io/defenseunicorns/zarf-init-aws/ecr-credential-helper:$GITHUB_REF_NAME
# env:
# COSIGN_EXPERIMENTAL: 1
# AWS_REGION: ${{ secrets.COSIGN_AWS_REGION }}
# AWS_ACCESS_KEY_ID: ${{ secrets.COSIGN_AWS_KEY_ID }}
# AWS_SECRET_ACCESS_KEY: ${{ secrets.COSIGN_AWS_ACCESS_KEY }}

- name: Build AWS init package for release
run: make release-aws-init-package CREDENTIAL_HELPER_IMAGE_TAG=$GITHUB_REF_NAME

- name: Publish AWS Init Package as OCI and Skeleton
run: make publish-aws-init-package ARCH=amd64 REPOSITORY_URL=ghcr.io/defenseunicorns/packages

# Create a CVE report based on this build
- name: Create release time CVE report
run: make cve-report

- name: Save CVE report
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
with:
name: cve-report
path: build/zarf-known-cves.csv

# Create GitHub release and upload the AWS init package as a release artifact
- name: Create GitHub release and upload AWS init package as release artifact
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN}}
run: gh release create "$GITHUB_REF_NAME" ./build/zarf-init-*.tar.zst --generate-notes --verify-tag
66 changes: 66 additions & 0 deletions .github/workflows/scan-codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
name: Analyze CodeQL

permissions:
contents: read

on:
push:
branches: ["main"]
pull_request:
paths-ignore:
- "**.md"
- "**.jpg"
- "**.png"
- "**.gif"
- "**.svg"
- "adr/**"
- "docs/**"
- "CODEOWNERS"
schedule:
- cron: "32 2 * * 5"

jobs:
validate:
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: ["go", "javascript", "typescript"]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
- name: Checkout
uses: actions/checkout@3df4ab11eba7bda6032a0b82a6bb43b11571feac # v4.0.0

- name: Setup Go
uses: defenseunicorns/zarf/.github/actions/golang@main

- name: Setup NodeJS
uses: actions/setup-node@5e21ff4d9bc1a8cf6de233a3057d20ec6b3fb69d # v3.8.1
with:
node-version: 20
cache: "npm"
cache-dependency-path: "package-lock.json"

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@04daf014b50eaf774287bf3f0f1869d4b4c4b913 # v2.21.7
env:
CODEQL_EXTRACTOR_GO_BUILD_TRACING: on
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql.yaml

- name: Build
run: make build-credential-helper-linux-amd

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@04daf014b50eaf774287bf3f0f1869d4b4c4b913 # v2.21.7
with:
category: "/language:${{matrix.language}}"
30 changes: 30 additions & 0 deletions .github/workflows/scan-cves.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Analyze CVEs

permissions:
contents: read

on:
schedule:
- cron: "0 10 * * *"
pull_request:
paths:
- "**/package.json"
- "**/package-lock.json"
- "go.mod"
- "go.sum"

jobs:
validate:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@3df4ab11eba7bda6032a0b82a6bb43b11571feac # v4.0.0

- name: Install tools
uses: defenseunicorns/zarf/.github/actions/install-tools@main

- name: Install latest version of Zarf
uses: defenseunicorns/setup-zarf@main

- name: Check for CVEs in Dependencies
run: make test-cves
23 changes: 23 additions & 0 deletions .github/workflows/scan-gen-schema.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Validate Schema Generation
on:
pull_request:

permissions:
contents: read

jobs:
validate:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@3df4ab11eba7bda6032a0b82a6bb43b11571feac # v4.0.0

- name: Setup Go
uses: defenseunicorns/zarf/.github/actions/golang@main

- name: Check that 'make gen-schema' was ran
run: make test-gen-schema

- name: Save logs
if: always()
uses: defenseunicorns/zarf/.github/actions/save-logs@main
15 changes: 15 additions & 0 deletions .github/workflows/scan-labels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Validate Labels
on:
pull_request:
types: [labeled, unlabeled, opened, edited, synchronize]

permissions:
contents: read

jobs:
enforce:
runs-on: ubuntu-latest
steps:
- uses: yogevbd/enforce-label-action@a3c219da6b8fa73f6ba62b68ff09c469b3a1c024 # 2.2.2
with:
BANNED_LABELS: "needs-docs,needs-tests,needs-adr,needs-git-sign-off"
Loading

0 comments on commit b3b7a59

Please sign in to comment.