Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add section for custom image registry options #2386

Merged
merged 13 commits into from
Mar 14, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Creating a content bundle provides several benefits that may address common use

## Create Content Bundle

1. Download Palette Edge Content CLI and assign the executable bit to the CLI. <br />
1. Download Palette Edge Content CLI and assign the executable bit to the CLI.

```shell
VERSION=4.1.2
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -670,7 +670,8 @@ EOF
If you need to pull images from a private image registry, you can supply the credentials for the registry in the user
data file in the `registryCredentials` field or in the cluster profile. Credentials specified in **user-data**
overwrites the credentials provided in the cluster profile. To learn how to provide credentials in cluster profiles,
refer to [Deploy Cluster with a Private Registry](../../site-deployment/deploy-private-registry.md).
refer to
[Deploy Cluster with a Private Registry](../../site-deployment/deploy-custom-registries/deploy-private-registry.md).

:::

Expand Down
1 change: 0 additions & 1 deletion docs/docs-content/clusters/edge/networking/networking.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ ensure that you can keep your Edge clusters and their services operational and a

- [Publish Cluster Services with Kube-vip](kubevip.md)
- [Enable Overlay Network](vxlan-overlay.md)
- [Enable Local Harbor Image Registry](local-registry.md)
- [Connect Intel NUC Edge Host to Wifi](./connect-wifi.md)
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
sidebar_label: "Deployment with Custom Image Registries"
title: "Deployment with Custom Image Registries"
description: "Registry options for deploying Edge clusters."
hide_table_of_contents: false
sidebar_position: 60
tags: ["edge"]
---

During Edge cluster deployment, the Palette agent must download the necessary images required by the cluster. The
Palette agent divides the images into two categories in terms of when and from where to download them:

- Provider images. These images are Kairos-based container images for each supported Operating System (OS) and
Kubernetes combination built during EdgeForge and pushed to an image registry. Provider images are always downloaded
first.

- All other images. These images are for the network, storage, and application layers of the cluster.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Application' instead of 'application'.


The Palette agent decides on where to download the images from depending on whether you have specified an external
registry and whether you have enabled a local Harbor registry on your cluster.

## Provider Image Registry

Provider images are always downloaded first by the Palette agent. You are required to provide the location of the
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
provider image in the OS pack of your cluster profile. If an external registry is not specified in the
[user data](../../edgeforge-workflow/prepare-user-data.md) during EdgeForge, provider images will be downloaded from the
provider image registry specified in the OS pack of the cluster profile.

Palette supports downloading provider images from authenticated registries. If your cluster needs to download provider
images from a authenticated registry, you need to provide the credentials to access the registry in the OS pack of the
cluster profile. For more information, refer to
[Deploy Cluster with a Private Provider Registry](./deploy-private-registry.md).

## External Registry

After the provider images are downloaded, the Palette agent proceeds to download all other images required for the
cluster from registries that are specified in each pack's YAML configuration. However, you can specify an _external
registry_ as the registry from which all images used by the cluster will be downloaded, including the provider images.

When an external registry is provided in the user data, the Palette agent will replace the registry URL of every image
used by the cluster with the external registry. For example, if your OS pack specified that the provider images be
downloaded from `quay.io/kairos/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1`, but in your user data, you have specified an
external registry `10.10.254.254:8000/spectro-images`. The Palette agent will automatically download the image using the
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
tag `10.10.254.254:8000/spectro-images/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1` instead of looking for the image in the
`quay.io/kairos` registry.

If you want to use a private image registry for applications on your Edge cluster, you can instruct the Palette agent to
download images from an _authenticated external registry_. You can specify an external registry in the user-data used to
build your Edge Installer ISO. For more information on how to deploy a cluster with an authenticated external registry,
refer to [Deploy Cluster with a Private External Registry](./deploy-external-registry.md).

## Local Harbor Registry

A local Harbor registry is a local registry that runs on your cluster. After the initial download of images from other
registries, all images except for infrastructure layer images will be uploaded to the Harbor registry. Subsequently, all
requests for application images from the cluster will be made to the Harbor registry. For more information, refer to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Application' instead of 'application'.

[Enable Local Harbor Image Registry](./local-registry.md).

## Limitations

There are limitations to using the local Harbor image registry and authenticated registries. The following table shows
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[write-good.ThereIs] Don't start a sentence with 'There are'.

you which combinations are supported.

| Feature | Can be used with | Cannot be used with |
| ------------------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------- |
| Local Harbor registry | - Authenticated external registry <br/> - Public provider registry | Authenticated provider registry |
| Authenticated external registry | - Local Harbor registry <br/> - Public provider registry | Authenticated provider registry. |
| Authenticated provider registry | N/A | - Authenticated external registry <br/> - Local Harbor registry |
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
---
sidebar_label: "Deploy Cluster with a Private External Registry"
title: "Deploy Cluster with a Private External Registry"
description: "Instructions for deploying an Edge cluster with a private external registry."
hide_table_of_contents: false
sidebar_position: 20
tags: ["edge"]
---

Palette Edge provides support for downloading images from authenticated external registries. You can instruct the
Palette agent to download images from an authenticated external registry by specifying the address and the credentials
for the registry in the user data used to build your Edge Installer ISO.

Once you specify an external registry, images for all elements of the cluster are expected to be in the external
registry. This includes the provider images, images for the network and storage layer, and images for all application
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Application' instead of 'application'.

layers. All images specified in the cluster profile will have their registry URL replaced by the registry URL of the
external image registry. For example, if your OS pack specified that the provider images be downloaded from
`quay.io/kairos/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1`, but in your user data, you have specified an external registry
`10.10.254.254:8000/spectro-images`. The Palette agent will automatically download the image using the tag
`10.10.254.254:8000/spectro-images/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1` instead of looking for the image in the
`quay.io/kairos` registry.

:::tip

You can use a private external registry together with a local Harbor image registry by adding the Harbor Edge-Native
Config pack to your cluster profile. All images for add-on layers of the cluster will be stored in the local Harbor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'config' instead of 'Config'.

registry after the initial download, which allows you to reduce the bandwidth use and protect against outages. For more
information, refer to [Enable Local Harbor Registry](./local-registry.md).

:::

## Limitations

- Palette Edge supports basic username/password authentication. Token authentication schemes used by services such as
AWS ECR and Google Artifact Registry are not supported.

## Prerequisites

- Specifying the external registry and providing credentials happens during the EdgeForge process. You should become
familiar with EdgeForge before following this guide. Refer to
[Build Edge Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md) to learn how to build Edge Installer
ISO and provider images.

- A physical or virtual Linux machine with _AMD64_ (also known as _x86_64_) processor architecture to build the Edge
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
artifacts. You can issue the following command in the terminal to check your processor architecture.

```bash
uname -m
```

- Minimum hardware configuration of the Linux machine:

- 4 CPU
- 8 GB memory
- 50 GB storage

- [Git](https://cli.github.com/manual/installation). You can ensure git installation by issuing the `git --version`
command.

- [Docker Engine](https://docs.docker.com/engine/install/) version 18.09.x or later. You can use the `docker --version`
command to view the existing Docker version.

- You should have root-level or `sudo` privileges on your Linux machine to create privileged containers.

- A [Spectro Cloud](https://console.spectrocloud.com) account. If you have not signed up, you can sign up for a
[free trial](https://www.spectrocloud.com/free-tier/).

- Palette registration token for pairing Edge hosts with Palette. You will need tenant admin access to Palette to
generate a new registration token. For detailed instructions, refer to the
[Create Registration Token](/clusters/edge/site-deployment/site-installation/create-registration-token) guide.

- A private external registry that stores all images required by your cluster.

## Deploy Cluster with a Private External Registry

1. Check out the [CanvOS](https://github.com/spectrocloud/CanvOS) GitHub repository containing the starter code.

```bash
git clone https://github.com/spectrocloud/CanvOS.git
```

2. Change to the **CanvOS/** directory.

```bash
cd CanvOS
```

3. In the user data file, provide the URL and the credentials in `stylus.registryCredentials`. The following is an
example:

```yaml
#cloud-config
stylus:
registryCredentials:
domain: 10.10.254.254:8000/spectro-images
username: ubuntu
password: *******
insecure: true
```

Refer to [Installer Configuration](../../edge-configuration/installer-reference.md#external-registry) for a
description of each field.

4. Follow the rest of the [Build Edge Artifact](../../edgeforge-workflow/palette-canvos/palette-canvos.md) guide and
build the Installer ISO with the user data containing the registry credentials.

5. Follow the [Perform Site Install](../site-installation/site-installation.md) guide to perform the installation.

6. Log in to [Palette](https://console.spectrocloud.com).

7. From the left **Main Menu**, click on **Profiles**. Then select the profile you are using to deploy the cluster.

8. Go through each layer of the profile and ensure that all images referenced in the profile are present in the external
registry. If you do not want to do this manually image by image, refer to
[Upload Cluster Images to External Registry with Palette Edge CLI](./upload-images-to-registry.md) to learn how to
use the Palette Edge CLI to upload all images in a cluster profile to an external registry.

9. Follow the [Create Cluster Definition](../site-installation/cluster-deployment.md) guide and deploy your cluster.

## Validate

1. Log in to [Palette](https://console.spectrocloud.com).

2. From the left **Main Menu**, click on **Clusters**.

3. Verify that the cluster you provisioned is in running status. Since your cluster profile only references images in
the private external registry, you can confirm that the images were downloaded successfully.
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
---
sidebar_label: "Deploy Cluster with a Private Registry"
title: "Deploy Cluster with a Private Registry"
description: "Instructions for creating an Edge Native Cluster Profile"
sidebar_label: "Deploy Cluster with a Private Provider Registry"
title: "Deploy Cluster with a Private Provider Registry"
description: "Instructions for deploying an Edge cluster with a private provider registry."
hide_table_of_contents: false
sidebar_position: 60
sidebar_position: 40
tags: ["edge"]
---

Palette Edge supports authentication with private image registries, which allows your cluster to pull images from a
private registry during deployment. You can configure your cluster to pull images from a private registry for both
cluster creation and cluster updates. To configure a cluster to pull images from a private image registry, provide the
registry URL and the credentials needed to authenticate with the registry in the cluster profile.
Palette Edge supports authentication with private image registries, which allows your cluster to pull provider images
from a private registry during deployment. You can configure your cluster to pull provider images from a private
registry for both cluster creation and cluster updates.

:::tip
To configure a cluster to pull images from a private image registry, provide the registry URL and the credentials needed
to authenticate with the registry in the cluster profile. This registry supplies the provider images only. If you want
to use a private registry for images other the provider images, refer to
[Deploy Cluster with Private External Registry](./deploy-external-registry.md).

You can use the [Harbor Edge-Native Config](../../../integrations/harbor-edge.md) pack to configure a local registry in
your Edge cluster. After the initial download of artifacts from the private registry, the cluster will redirect
subsequent image pulls to the local Harbor registry. This allows you to save network bandwidth and provision clusters
without a connection to external networks. For more information, refer to
[Enable Local Harbor Registry](../networking/local-registry.md).
## Limitations

:::
- A cluster cannot pull provider images from more than one private registry.

## Limitations
- If you have already specified an external registry. the provider registry will be ignored and the provider images will
be pulled from the external registry instead.

- A cluster cannot pull images from more than one private registry.
- You cannot use private provider registries for clusters with a local Harbor registry. For more information, refer to
[Enable Local Harbor Registry](./local-registry.md).

- If your private registry has TLS enabled, you can only configure a _new_ cluster to use a TLS certificate with a
private registry. You cannot configure an existing cluster with a TLS certificate to communicate with your private
Expand All @@ -40,7 +40,7 @@ without a connection to external networks. For more information, refer to
- A private image registry.

- A provider image you created in the EdgeForge process stored in your private image registry. For more information,
refer to [Build Artifacts](../edgeforge-workflow/palette-canvos/palette-canvos.md).
refer to [Build Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md).

## Enablement

Expand All @@ -58,7 +58,7 @@ without a connection to external networks. For more information, refer to
pack for your OS layer.

5. Update the `system.uri` parameter in the pack editor for your OS layer. Use the custom OS image you created in the
EdgeForge process. Refer to the EdgeForge [Build Images](../edgeforge-workflow/palette-canvos/palette-canvos.md)
EdgeForge process. Refer to the EdgeForge [Build Images](../../edgeforge-workflow/palette-canvos/palette-canvos.md)
guide if you are missing a custom OS image. The following is an example configuration using the BYOOS pack with a
custom OS image.

Expand All @@ -78,17 +78,17 @@ without a connection to external networks. For more information, refer to

If you have specified registry credentials in the `registryCredentials` field in the user data file during the
EdgeForge process, the credentials provided in the cluster profile will be ignored. For more information, refer to
[EdgeForge - Build Artifacts](../edgeforge-workflow/palette-canvos/palette-canvos.md) and
[Installer Configuration](../edge-configuration/installer-reference.md#external-registry).
[EdgeForge - Build Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md) and
[Installer Configuration](../../edge-configuration/installer-reference.md#external-registry).

:::

6. At the root level of YAML for your OS layer, add the `providerCredentials` field to provide the credentials you need
to authenticate with your registry. For more information about the `providerCredentials` field, refer to
[Bring Your Own OS (BYOOS)](../../../integrations/byoos.md) pack page. The `providerCredentials.password` field will
be masked when you provide it in the YAML file. You can also use a macro to store your credentials instead of
providing it directly in the YAML file. For more information, refer to
[Macros Support](../../cluster-management/macros.md):
[Bring Your Own OS (BYOOS)](/docs/docs-content/integrations/byoos.md) pack page. The `providerCredentials.password`
field will be masked when you provide it in the YAML file. You can also use a macro to store your credentials instead
of providing it directly in the YAML file. For more information, refer to
[Macros Support](/docs/docs-content/clusters/cluster-management/macros.md):

```yaml {7-8}
pack:
Expand Down Expand Up @@ -119,9 +119,10 @@ without a connection to external networks. For more information, refer to

8. If you already have an active cluster that is using the original version of the cluster profile, update the cluster
so that it uses the new version of the cluster profile you just published. For more information about updating
clusters, refer to [Update a Cluster](../../cluster-management/cluster-updates.md). This will trigger a full cluster
repave since it includes an update to the OS layer of the cluster. To learn more about cluster repave behavior, refer
to [Repave Behavior and Configuration](../../cluster-management/node-pool.md#repave-behavior-and-configuration).
clusters, refer to [Update a Cluster](../../../cluster-management/cluster-updates.md). This will trigger a full
cluster repave since it includes an update to the OS layer of the cluster. To learn more about cluster repave
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Repave' instead of 'repave'.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Repave' instead of 'repave'.

behavior, refer to
[Repave Behavior and Configuration](../../../cluster-management/node-pool.md#repave-behavior-and-configuration).

If you don't have an active cluster yet, deploy a new cluster with the profile you just created, and the cluster will
pull images from the private registry you specified.
Expand Down
Loading