Add YOLO mode (#1105)

Provides the ability to create Zarf packages intended to work in online-only environments, i.e., without the internal Zarf airgap components typically provided by `zarf init`. Also adds early FAQ. ## Related Issue Fixes #1051 Fixes #1046 Related to #1134 Co-authored-by: unclegedd <[email protected]> Co-authored-by: Wayne Starr <[email protected]> Co-authored-by: MxNxPx <[email protected]>
zarf-dev · Dec 16, 2022 · 79b432f · 79b432f
1 parent d2babf3
commit 79b432f
Show file tree

Hide file tree

Showing 20 changed files with 348 additions and 86 deletions.
diff --git a/Makefile b/Makefile
@@ -145,6 +145,8 @@ build-examples: ## Build all of the example packages
 
 	@test -s ./build/zarf-package-test-helm-wait-$(ARCH).tar.zst || $(ZARF_BIN) package create examples/helm-no-wait -o build -a $(ARCH) --confirm
 
+	@test -s ./build/zarf-package-yolo-$(ARCH).tar.zst || $(ZARF_BIN) package create examples/yolo -o build -a $(ARCH) --confirm
+
 ## Run e2e tests. Will automatically build any required dependencies that aren't present.
 ## Requires an existing cluster for the env var APPLIANCE_MODE=true
 .PHONY: test-e2e

diff --git a/adr/0009-yolo-mode.md b/adr/0009-yolo-mode.md
@@ -0,0 +1,19 @@
+# 9. YOLO Mode
+
+Date: 2022-12-14
+
+## Status
+
+Accepted
+
+## Context
+
+Zarf was rooted in the idea of declarative K8s deployments for disconnected environments. Many of the design decisions made in Zarf are based on this idea. However, in certain connected environments, Zarf can still be leveraged as a way to define declarative deployments and upgrades without the constraints of disconnected environments. To that end, providing a declarative way to deploy Zarf packages without the need for a Zarf init package would be useful in such environments.
+
+## Decision
+
+YOLO mode is an optional boolean config set in the `metadata` section of the Zarf package manifest. Setting `metadata.yolo=true` will deploy the Zarf package "as is" without needing the Zarf state to exist or the Zarf Agent mutating webhook. Zarf packages with YOLO mode enabled are not allowed to specify components with container images or Git repos and validation will prevent the package from being created.
+
+## Consequences
+
+YOLO mode provides a way for existing, connected clusters to use Zarf for declarative deployments and upgrades because there is no need to perform any Zarf bootstrapping in order to deploy Zarf-packaged workloads. The addition of the `metadata.yolo` config should not affect existing Zarf users as it is entirely optional. Additionally, requiring the `metadata.yolo` config to be set to `true` and not allowing a runtime flag to override it makes it very clear both in `package create` and `package deploy` the intent and usage of the package.
diff --git a/docs/13-walkthroughs/0-using-zarf-package-create.md b/docs/13-walkthroughs/0-using-zarf-package-create.md
@@ -9,8 +9,7 @@ When creating a Zarf package, you will need to have Internet connection so that
 
 1. The [Zarf](https://github.com/defenseunicorns/zarf) repository cloned: ([git clone instructions](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository))
 2. Zarf binary installed on your $PATH: ([install instructions](../3-getting-started.md#installing-zarf))
-3. A copy of the injector's `zarf-registry` binary in the `build/` directory: ([build instructions](../3-getting-started.md#building-the-cli-from-scratch))
-4. The Zarf agent image name and tag you would like to use (the default can be found in the project's [`Makefile`](https://github.com/defenseunicorns/zarf/blob/master/Makefile) under `AGENT_IMAGE ?= `)
+
 
 ## Building the init-package
 

diff --git a/docs/4-user-guide/2-zarf-packages/2-zarf-components.md b/docs/4-user-guide/2-zarf-packages/2-zarf-components.md
@@ -46,7 +46,7 @@ components:
      name: flux-v1.0.0
 ```
 
-> Note: When importing a component, Zarf will copy all of the values from the original component expect for the `required` key. In addition, while Zarf will copy the values, you have the ability to override the value for the `description` key.
+> Note: When importing a component, Zarf will copy all of the values from the original component except for the `required` key. In addition, while Zarf will copy the values, you have the ability to override the value for the `description` key.
 
  Checkout the [composable-packages](https://github.com/defenseunicorns/zarf/blob/master/examples/composable-packages/zarf.yaml) example to see this in action.
 

diff --git a/docs/4-user-guide/2-zarf-packages/3-the-zarf-init-package.md b/docs/4-user-guide/2-zarf-packages/3-the-zarf-init-package.md
@@ -25,7 +25,7 @@ In addition to those that are always installed, Zarf's optional components provi
 
 These optional components for the init package are listed below along with the "magic strings" you pass to `zarf init --components` to pull them in:
 
-| --components | Description                                                                                                                                                       |
+| components   | Description                                                                                                                                                       |
 | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | k3s          | REQUIRES ROOT. Installs a lightweight Kubernetes Cluster on the local host&mdash;[k3s](https://k3s.io/)&mdash;and configures it to start up on boot.                             |
 | logging      | Adds a log monitoring stack&mdash;[promtail / loki / graphana (a.k.a. PLG)](https://github.com/grafana/loki)&mdash;into the cluster.                              |
@@ -39,6 +39,6 @@ There are two ways to deploy optional components, you can either pass a comma se
 
 # What Makes the Init Package Special
 
-Deploying onto air-gapped environments is a [hard problem](../../1-understand-the-basics.md#what-is-the-air-gap), especially when the k8s environment you're deploying to doesn't have a container registry running for you to put your images into. This leads to a classic 'chicken or the egg' problem since the container registry image needs to make its way into the cluster but there is on container registry running on the cluster to push to yet because the image isn't in the cluster yet. In order to remain distro agnostic, we had to come up with a unique solution to seed the container registry into the cluster.
+Deploying onto air-gapped environments is a [hard problem](../../1-understand-the-basics.md#what-is-the-air-gap), especially when the k8s environment you're deploying to doesn't have a container registry running for you to put your images into. This leads to a classic 'chicken or the egg' problem since the container registry image needs to make its way into the cluster but there is no container registry running on the cluster to push to yet because the image isn't in the cluster yet. In order to remain distro agnostic, we had to come up with a unique solution to seed the container registry into the cluster.
 
 The `zarf-injector` [component](https://github.com/defenseunicorns/zarf/blob/master/packages/zarf-injector/zarf.yaml) within the init-package solves this problem by injecting a single rust binary (statically compiled) and a series of configmap chunks of a `registry:2` image into an ephemeral pod based on an existing image in the cluster.
diff --git a/docs/4-user-guide/3-zarf-schema.md b/docs/4-user-guide/3-zarf-schema.md
@@ -164,6 +164,22 @@ Must be one of:
 </blockquote>
 </details>
 
+<details>
+<summary><strong> <a name="metadata_yolo"></a>yolo</strong>
+
+</summary>
+&nbsp;
+<blockquote>
+
+**Description:** Yaml OnLy Online (YOLO): True enables deploying a Zarf package without first running zarf init against the cluster. This is ideal for connected environments where you want to use existing VCS and container registries.
+
+|          |           |
+| -------- | --------- |
+| **Type** | `boolean` |
+
+</blockquote>
+</details>
+
 </blockquote>
 </details>
 
@@ -923,7 +939,7 @@ Must be one of:
 
 ![Required](https://img.shields.io/badge/Required-red)
 
-**Description:** The name of the chart to deploy
+**Description:** The name of the chart to deploy; this should be the name of the chart as it is installed in the helm repo
 
 |          |          |
 | -------- | -------- |
@@ -939,7 +955,7 @@ Must be one of:
 &nbsp;
 <blockquote>
 
-**Description:** The name of the release to create
+**Description:** The name of the release to create; defaults to the name of the chart
 
 |          |          |
 | -------- | -------- |
@@ -973,7 +989,7 @@ Must be one of:
 
 ![Required](https://img.shields.io/badge/Required-red)
 
-**Description:** The version of the chart to deploy
+**Description:** The version of the chart to deploy; for git-based charts this is also the tag of the git repo
 
 |          |          |
 | -------- | -------- |
@@ -1007,7 +1023,7 @@ Must be one of:
 &nbsp;
 <blockquote>
 
-**Description:** List of values files to include in the package
+**Description:** List of values files to include in the package; these will be merged together
 
 |          |                   |
 | -------- | ----------------- |
@@ -1037,7 +1053,7 @@ Must be one of:
 &nbsp;
 <blockquote>
 
-**Description:** If using a git repo
+**Description:** The path to the chart in the repo if using a git repo instead of a helm repo
 
 |          |          |
 | -------- | -------- |
@@ -1117,7 +1133,7 @@ Must be one of:
 
 ![Required](https://img.shields.io/badge/Required-red)
 
-**Description:** A name to give this collection of manifests
+**Description:** A name to give this collection of manifests; this will become the name of the dynamically-created helm chart
 
 |          |          |
 | -------- | -------- |

diff --git a/docs/6-developer-guide/2-testing.md b/docs/6-developer-guide/2-testing.md
@@ -72,7 +72,8 @@ Due to resource constraints in public github runners, K8s tests are only perform
 - 20 is reserved for `zarf init`
 - 21 is reserved for logging tests so they can be removed first (they take the most resources in the cluster)
 - 22 is reserved for tests required the git-server, which is removed at the end of the test
-- 23-99 are for the remaining tests that only require a basic zarf cluster without logging for the git-server
+- 23-98 are for the remaining tests that only require a basic zarf cluster without logging for the git-server
+- 99 is reserved for the `zarf destroy` and [YOLO Mode](../../examples/yolo/README.md) test
 
 ## Running Unit Tests Locally
 

diff --git a/docs/6-developer-guide/3-nerd-notes.md b/docs/6-developer-guide/3-nerd-notes.md
@@ -20,7 +20,6 @@ Zarf is written entirely in [go](https://go.dev/), except for a single 868Kb bin
   - The container then starts and runs the rust binary to host the registry image in an static docker registry
   - After this, the main docker registry chart is deployed, pulls the image from the ephemeral pod, and finally destroys the created configmaps, pod, and service
 
-<!-- TODO: @JPERRY I think this svg is out of date now that the 'Zarf agent' PR got merged.. -->
 ## Zarf Architecture
 
 ![Architecture Diagram](../.images/architecture.drawio.svg)
diff --git a/docs/9-faq.md b/docs/9-faq.md
@@ -1,5 +1,35 @@
 # FAQ
 
-:::caution Hard Hat Area
-This page is still being developed. More content will be added soon!
-:::
+## Do I have to use [Homebrew](https://brew.sh/) to install Zarf?
+
+No, the Zarf binary and init package can be downloaded from the [Releases Page](https://github.com/defenseunicorns/zarf/releases). Zarf does not need to be installed or available to all users on the system, but it does need to be executable for the current user (i.e. `chmod +x zarf` for Linux/Mac).
+
+## What dependencies does Zarf have?
+
+Zarf is statically compiled and written in [Go](https://golang.org/) and [Rust](https://www.rust-lang.org/), so it has no external dependencies. For Linux, Zarf can bring a Kubernetes cluster using [K3s](https://k3s.io/). For Mac and Windows, Zarf can leverage any avaiilable local or remote cluster the user has access to. Currently, the K3s installation Zarf performs does require a [Systemd](https://en.wikipedia.org/wiki/Systemd) based system and root access.
+
+## What is the Zarf Agent?
+
+The Zarf Agent is a [Kubernetes Mutating Webhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook) that is installed into the cluster during the `zarf init` operation. The Agent is responsible for modifying [Kubernetes PodSpec](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec) objects [Image](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#Container.Image) fields to point to the Zarf Registry. This allows the cluster to pull images from the Zarf Registry instead of the internet without having to modify the original image references. The Agent also modifies [Flux GitRepository](https://fluxcd.io/docs/components/source/gitrepositories/) objects to point to the local Git Server.
+
+## Why doesn't the Zarf Agent create secrets it needs in the cluster?
+
+During early discussions and [subsequent decision](../adr/0005-mutating-webhook.md) to use a Mutating Webhook, we decided to not have the Agent create any secrets in the cluster. This is to avoid the Agent having to have more priveleges than it needs as well as avoid collisions with Helm. The Agent today simply repsonds to requests to patch PodSpec and GitRepository objects. 
+
+The Agent does not need to create any secrets in the cluster. Instead, during `zarf init` and `zarf package deploy`, secrets are automatically created as  [Helm Postrender Hook](https://helm.sh/docs/topics/advanced/#post-rendering) for any namespaces Zarf sees. If you have resources managed by [Flux](https://fluxcd.io/) that are not in a namespace managed by Zarf, you can either create the secrets manually or include a manifest to create the namespace in your package and let Zarf create the secrets for you.
+
+## How can a Kubernetes resource be excluded from the Zarf Agent?
+
+Resources can be exluded at the namespace or resources level by adding the `zarf.dev/agent: ignore` label.
+
+## What happens to resources that exist in the cluster before `zarf init`?
+
+During the `zarf init` operation, the Zarf Agent will patch any existing namespaces with the `zarf.dev/agent: ignore` label to prevent the Agent from modifying any resources in that namespace. This is done because there is no way to guarantee the images used by pods in existing namespaces are available in the Zarf Registry.
+
+## What is YOLO Mode and why would I use it?
+
+YOLO Mode is a special package metatdata designation that be added to a package prior to `zarf package create` to allow the package to be installed without the need for a `zarf init` operation. In most cases this will not be used, but it can be useful for testing or for environments that manage their own registries and Git servers completely outside of Zarf. This can also be used as a way to transition slowly to using Zarf without having to do a full migration.
+
+:::note
+Typically you should not deploy a Zarf package in YOLO mode if the cluster has already been initialized with Zarf. This could lead to an [ImagePullBackOff](https://kubernetes.io/docs/concepts/containers/images/#imagepullbackoff) if the resources in the package do not include the `zarf.dev/agent: ignore` label and are not already available in the Zarf Registry.
+:::
diff --git a/examples/yolo/README.md b/examples/yolo/README.md
@@ -0,0 +1,43 @@
+# YOLO Mode
+This example demonstrates YOLO mode, an optional mode for using Zarf in a fully connected environment where users can bring their own external container registry and Git server.
+
+:::info
+
+To view the example source code, select the `Edit this page` link below the article and select the parent folder.
+
+:::
+
+
+## Prerequisites
+- A running K8s cluster. _Note that the cluster does not need to have the Zarf init package installed or any other Zarf-related bootstrapping._
+
+## Instructions
+Create the package:
+```sh
+zarf package create
+```
+
+### Deploy the package
+
+```sh
+# Run the following command to deploy the created package to the cluster
+zarf package deploy
+
+# Choose the yolo package from the list
+? Choose or type the package file [tab for suggestions] 
+> zarf-package-yolo-<ARCH>.tar.zst
+
+# Confirm the deployment
+? Deploy this Zarf package? (y/N) 
+
+# Wait a few seconds for the cluster to deploy the package; you should
+# see the following output when the package has been finished deploying:
+  Connect Command    | Description
+  zarf connect doom  | Play doom!!!
+  zarf connect games | Play some old dos games 🦄
+
+# Run the specified `zarf connect <game>` command to connect to the deployed 
+# workload (ie. kill some demons). Note that the typical Zarf registry, 
+# Gitea server and Zarf agent pods are not present in the cluster. This means 
+# that the game's container image was pulled directly from the public registry and the URL was not mutated by Zarf.
+```
diff --git a/examples/yolo/zarf.yaml b/examples/yolo/zarf.yaml
@@ -0,0 +1,15 @@
+kind: ZarfPackageConfig
+metadata:
+  name: yolo
+  yolo: true
+  description: "Game example in YOLO (online-only) mode that can be deployed without a Zarf cluster"
+
+components:
+  - name: yolo-games
+    required: true
+    manifests:
+      - name: multi-games
+        namespace: zarf-yolo-example
+        files:
+          - ../game/manifests/deployment.yaml
+          - ../game/manifests/service.yaml