Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add section for custom image registry options #2386

Merged
merged 13 commits into from
Mar 14, 2024
Original file line number Diff line number Diff line change
Expand Up @@ -670,7 +670,8 @@ EOF
If you need to pull images from a private image registry, you can supply the credentials for the registry in the user
data file in the `registryCredentials` field or in the cluster profile. Credentials specified in **user-data**
overwrites the credentials provided in the cluster profile. To learn how to provide credentials in cluster profiles,
refer to [Deploy Cluster with a Private Registry](../../site-deployment/deploy-private-registry.md).
refer to
[Deploy Cluster with a Private Registry](../../site-deployment/deploy-custom-registries/deploy-private-registry.md).

:::

Expand Down
1 change: 0 additions & 1 deletion docs/docs-content/clusters/edge/networking/networking.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ ensure that you can keep your Edge clusters and their services operational and a

- [Publish Cluster Services with Kube-vip](kubevip.md)
- [Enable Overlay Network](vxlan-overlay.md)
- [Enable Local Harbor Image Registry](local-registry.md)
- [Connect Intel NUC Edge Host to Wifi](./connect-wifi.md)
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
sidebar_label: "Deployment with Custom Image Registries"
title: "Deployment with Custom Image Registries"
description: "Registry options for deploying Edge clusters."
hide_table_of_contents: false
sidebar_position: 60
tags: ["edge"]
---

During Edge cluster deployment, the Palette agent must download the necessary images required by the cluster. The
Palette agent divides the images into two categories in terms of when and from where to download them:

- Provider images. These images are Kairos-based container images for each supported Operating System (OS) and
Kubernetes combination built during EdgeForge and pushed to an image registry. Provider images are always downloaded
first.

- All other images. These images are for the network, storage, and application layers of the cluster. These images
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Application' instead of 'application'.

lennessyy marked this conversation as resolved.
Show resolved Hide resolved

The location of where the images are downloaded from depends on whether you have specified an external registry and
whether you have enabled a local Harbor registry on your cluster.
lennessyy marked this conversation as resolved.
Show resolved Hide resolved

## Provider Image Registry

Provider images are always downloaded first by the Palette agent. You are required to provide the location of the
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
provider image in the OS pack of your cluster profile. If an external registry is not specified in the user data,
provider images will be downloaded from the provider image registry specified in the OS pack of the cluster profile.
lennessyy marked this conversation as resolved.
Show resolved Hide resolved

Palette supports downloading provider images from authenticated registries. If your cluster needs to download provider
images from a authenticated registry, you need to provide the credentials to access the registry in the OS pack of the
cluster profile. For more information, refer to
[Deploy Cluster with a Private Provider Registry](./deploy-private-registry.md).

## External Registry

After the provider images are downloaded, the Palette agent proceeds to download all other images required for the
cluster from registries that are specified in each pack's YAML configuration. However, you can specify an _external
registry_ as the registry from which all images used by the cluster will be downloaded, including the provider images.

When an external registry is provided in the user data, the Palette agent will replace the registry URL of every images
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
used by the cluster with the external registry. For example, if your OS pack specified that the provider images be
downloaded from `quay.io/kairos/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1`, but in your user data, you have specified an
external registry `10.10.254.254:8000/spectro-images`. The Palette agent will automatically download the image using the
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
tag `10.10.254.254:8000/spectro-images/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1` instead of looking for the image in the
`quay.io/kairos` registry.

If you want to use a private image registry for applications on your Edge cluster, you can instruct the Palette agent to
download images from an _authenticated external registry_. You can specify an external registry in the user-data used to
build your Edge Installer ISO. For more information on how to deploy a cluster with an authenticated external registry,
refer to [Deploy Cluster with a Private External Registry](./deploy-external-registry.md).

## Local Harbor Registry

A local Harbor registry is a local registry that runs on your cluster. After the initial download of images from other
registries, all images except for infrastructure layer images will be uploaded to the Harbor registry. Subsequently, all
requests for application images from the cluster will be made to the Harbor registry. For more information, refer to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Application' instead of 'application'.

[Enable Local Harbor Image Registry](./local-registry.md).

## Limitations

There are limitations to using the local Harbor image registry and authenticated registries. The following table shows
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[write-good.ThereIs] Don't start a sentence with 'There are'.

you which combinations are supported or unsupported.
lennessyy marked this conversation as resolved.
Show resolved Hide resolved

| Feature | Can be used with | Cannot be used with |
| ------------------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------- |
| Local Harbor registry | - Authenticated external registry <br/> - Public provider registry | Authenticated provider registry |
| Authenticated external registry | - Local Harbor registry <br/> - Public provider registry | Authenticated provider registry. |
| Authenticated provider registry | Public external registry | - Authenticated external registry <br/> - Local Harbor registry |
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
---
sidebar_label: "Deploy Cluster with a Private External Registry"
title: "Deploy Cluster with a Private External Registry"
description: "Instructions for deploying an Edge cluster with a private external registry."
hide_table_of_contents: false
sidebar_position: 60
tags: ["edge"]
---

Palette Edge provides support for downloading images from authenticated external registries. You can instruct the
Palette agent to download images from an authenticated external registry by specifying the address and the credentials
for the registry in the user data used to build your Edge Installer ISO.

Once you specify an external registry, images for all elements of the cluster are expected to be in the external
registry. This includes the provider images, images for the network and storage layer, and images for all application
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Application' instead of 'application'.

layers. All images specified in the cluster profile will have their registry URL replaced by the registry URL of the
external image registry.

:::tip

You can use a private external registry together with a local Harbor image registry. All images from the external
private registry will be stored in the local Harbor registry after the initial download, which allows you to reduce
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
bandwidth use and protect against outages. For more information, refer to
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
[Enable Local Harbor Registry](./local-registry.md).

:::

## Limitations

- Palette Edge supports basic username/password authentication. Token authentication schemes used by services such as
AWS ECR and Google Artifact Registry are not supported.

## Prerequisites

- Specifying the external registry and providing credentials happens during the EdgeForge process. You should become
familiar with EdgeForge before following this guide. Refer to
[Build Edge Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md) to learn how to build Edge Installer
ISO and provider images.

- A physical or virtual Linux machine with _AMD64_ (also known as _x86_64_) processor architecture to build the Edge
lennessyy marked this conversation as resolved.
Show resolved Hide resolved
artifacts. You can issue the following command in the terminal to check your processor architecture.

```bash
uname -m
```

- Minimum hardware configuration of the Linux machine:

- 4 CPU
- 8 GB memory
- 50 GB storage

- [Git](https://cli.github.com/manual/installation). You can ensure git installation by issuing the `git --version`
command.

- [Docker Engine](https://docs.docker.com/engine/install/) version 18.09.x or later. You can use the `docker --version`
command to view the existing Docker version. You should have root-level or `sudo` privileges on your Linux machine to
create privileged containers.
lennessyy marked this conversation as resolved.
Show resolved Hide resolved

- A [Spectro Cloud](https://console.spectrocloud.com) account. If you have not signed up, you can sign up for a
[free trial](https://www.spectrocloud.com/free-tier/).

- Palette registration token for pairing Edge hosts with Palette. You will need tenant admin access to Palette to
generate a new registration token. For detailed instructions, refer to the
[Create Registration Token](/clusters/edge/site-deployment/site-installation/create-registration-token) guide.

- A private external registry that stores all images except for provider images required by your cluster.

## Deploy Cluster with a Private External Registry

1. Check out the [CanvOS](https://github.com/spectrocloud/CanvOS) GitHub repository containing the starter code.

```bash
git clone https://github.com/spectrocloud/CanvOS.git
```

2. Change to the **CanvOS/** directory.

```bash
cd CanvOS
```

3. In the user data file, provide the URL and the credentials in `stylus.registryCredentials`. The following is an
example:

```yaml
#cloud-config
stylus:
registryCredentials:
domain: 10.10.254.254:8000/spectro-images
username: ubuntu
password: *******
insecure: true
```

Refer to [Installer Configuration](../../edge-configuration/installer-reference.md#external-registry) for a
description of each field.

4. Follow the rest of the [Build Edge Artifact](../../edgeforge-workflow/palette-canvos/palette-canvos.md) guide and
build the Installer ISO with the user data containing the registry credentials.

5. Follow the [Perform Site Install](../site-installation/site-installation.md) guide to perform the installation.

6. Log in to [Palette](https://console.spectrocloud.com).

7. From the left **Main Menu**, click on **Profiles**. Then select the profile you are using to deploy the cluster.

8. Go through each layer of the profile and ensure that all images referenced in the profile refer to the registry you
provided and that all images are present in the external registry.

9. Follow the [Create Cluster Definition](../site-installation/cluster-deployment.md) and deploy your cluster.

## Validate

1. Log in to [Palette](https://console.spectrocloud.com).

2. From the left **Main Menu**, click on **Clusters**.

3. Verify that the cluster you provisioned is in running status. Since your cluster profile only references images in
the private external registry, you can confirm that the images were downloaded successfully.
Original file line number Diff line number Diff line change
@@ -1,30 +1,27 @@
---
sidebar_label: "Deploy Cluster with a Private Registry"
title: "Deploy Cluster with a Private Registry"
description: "Instructions for creating an Edge Native Cluster Profile"
sidebar_label: "Deploy Cluster with a Private Provider Registry"
title: "Deploy Cluster with a Private Provider Registry"
description: "Instructions for deploying an Edge cluster with a private provider registry."
hide_table_of_contents: false
sidebar_position: 60
tags: ["edge"]
---

Palette Edge supports authentication with private image registries, which allows your cluster to pull images from a
private registry during deployment. You can configure your cluster to pull images from a private registry for both
cluster creation and cluster updates. To configure a cluster to pull images from a private image registry, provide the
registry URL and the credentials needed to authenticate with the registry in the cluster profile.
Palette Edge supports authentication with private image registries, which allows your cluster to pull provider images
from a private registry during deployment. You can configure your cluster to pull provider images from a private
registry for both cluster creation and cluster updates.

:::tip

You can use the [Harbor Edge-Native Config](../../../integrations/harbor-edge.md) pack to configure a local registry in
your Edge cluster. After the initial download of artifacts from the private registry, the cluster will redirect
subsequent image pulls to the local Harbor registry. This allows you to save network bandwidth and provision clusters
without a connection to external networks. For more information, refer to
[Enable Local Harbor Registry](../networking/local-registry.md).

:::
To configure a cluster to pull images from a private image registry, provide the registry URL and the credentials needed
to authenticate with the registry in the cluster profile. This registry supplies the provider images only. If you want
to use a private registry for images other the provider images, refer to
[Deploy Cluster with Private External Registry](./deploy-external-registry.md).

## Limitations

- A cluster cannot pull images from more than one private registry.
- A cluster cannot pull provider images from more than one private registry.

- You cannot use private provider registries for clusters with a local Harbor registry. For more information, refer to
[Enable Local Harbor Registry](./local-registry.md).

- If your private registry has TLS enabled, you can only configure a _new_ cluster to use a TLS certificate with a
private registry. You cannot configure an existing cluster with a TLS certificate to communicate with your private
Expand All @@ -40,7 +37,7 @@ without a connection to external networks. For more information, refer to
- A private image registry.

- A provider image you created in the EdgeForge process stored in your private image registry. For more information,
refer to [Build Artifacts](../edgeforge-workflow/palette-canvos/palette-canvos.md).
refer to [Build Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md).

## Enablement

Expand All @@ -58,7 +55,7 @@ without a connection to external networks. For more information, refer to
pack for your OS layer.

5. Update the `system.uri` parameter in the pack editor for your OS layer. Use the custom OS image you created in the
EdgeForge process. Refer to the EdgeForge [Build Images](../edgeforge-workflow/palette-canvos/palette-canvos.md)
EdgeForge process. Refer to the EdgeForge [Build Images](../../edgeforge-workflow/palette-canvos/palette-canvos.md)
guide if you are missing a custom OS image. The following is an example configuration using the BYOOS pack with a
custom OS image.

Expand All @@ -78,17 +75,17 @@ without a connection to external networks. For more information, refer to

If you have specified registry credentials in the `registryCredentials` field in the user data file during the
EdgeForge process, the credentials provided in the cluster profile will be ignored. For more information, refer to
[EdgeForge - Build Artifacts](../edgeforge-workflow/palette-canvos/palette-canvos.md) and
[Installer Configuration](../edge-configuration/installer-reference.md#external-registry).
[EdgeForge - Build Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md) and
[Installer Configuration](../../edge-configuration/installer-reference.md#external-registry).

:::

6. At the root level of YAML for your OS layer, add the `providerCredentials` field to provide the credentials you need
to authenticate with your registry. For more information about the `providerCredentials` field, refer to
[Bring Your Own OS (BYOOS)](../../../integrations/byoos.md) pack page. The `providerCredentials.password` field will
be masked when you provide it in the YAML file. You can also use a macro to store your credentials instead of
providing it directly in the YAML file. For more information, refer to
[Macros Support](../../cluster-management/macros.md):
[Bring Your Own OS (BYOOS)](/docs/docs-content/integrations/byoos.md) pack page. The `providerCredentials.password`
field will be masked when you provide it in the YAML file. You can also use a macro to store your credentials instead
of providing it directly in the YAML file. For more information, refer to
[Macros Support](/docs/docs-content/clusters/cluster-management/macros.md):

```yaml {7-8}
pack:
Expand Down Expand Up @@ -119,9 +116,10 @@ without a connection to external networks. For more information, refer to

8. If you already have an active cluster that is using the original version of the cluster profile, update the cluster
so that it uses the new version of the cluster profile you just published. For more information about updating
clusters, refer to [Update a Cluster](../../cluster-management/cluster-updates.md). This will trigger a full cluster
repave since it includes an update to the OS layer of the cluster. To learn more about cluster repave behavior, refer
to [Repave Behavior and Configuration](../../cluster-management/node-pool.md#repave-behavior-and-configuration).
clusters, refer to [Update a Cluster](../../../cluster-management/cluster-updates.md). This will trigger a full
cluster repave since it includes an update to the OS layer of the cluster. To learn more about cluster repave
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Repave' instead of 'repave'.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Repave' instead of 'repave'.

behavior, refer to
[Repave Behavior and Configuration](../../../cluster-management/node-pool.md#repave-behavior-and-configuration).

If you don't have an active cluster yet, deploy a new cluster with the profile you just created, and the cluster will
pull images from the private registry you specified.
Expand Down
Loading