-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: add section for custom image registry options #2386
Changes from all commits
ad6b029
6a0b8fe
cc5d5a8
424582d
aed220b
ad28dbd
451da77
242e8c6
1c88f4f
7efac3d
d4ffc68
7b065f6
1b47378
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
--- | ||
sidebar_label: "Deployment with Custom Image Registries" | ||
title: "Deployment with Custom Image Registries" | ||
description: "Registry options for deploying Edge clusters." | ||
hide_table_of_contents: false | ||
sidebar_position: 60 | ||
tags: ["edge"] | ||
--- | ||
|
||
During Edge cluster deployment, the Palette agent must download the necessary images required by the cluster. The | ||
Palette agent divides the images into two categories in terms of when and from where to download them: | ||
|
||
- Provider images. These images are Kairos-based container images for each supported Operating System (OS) and | ||
Kubernetes combination built during EdgeForge and pushed to an image registry. Provider images are always downloaded | ||
first. | ||
|
||
- All other images. These images are for the network, storage, and application layers of the cluster. | ||
|
||
The Palette agent decides on where to download the images from depending on whether you have specified an external | ||
registry and whether you have enabled a local Harbor registry on your cluster. | ||
|
||
## Provider Image Registry | ||
|
||
Provider images are always downloaded first by the Palette agent. You are required to provide the location of the | ||
lennessyy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
provider image in the OS pack of your cluster profile. If an external registry is not specified in the | ||
[user data](../../edgeforge-workflow/prepare-user-data.md) during EdgeForge, provider images will be downloaded from the | ||
provider image registry specified in the OS pack of the cluster profile. | ||
|
||
Palette supports downloading provider images from authenticated registries. If your cluster needs to download provider | ||
images from a authenticated registry, you need to provide the credentials to access the registry in the OS pack of the | ||
cluster profile. For more information, refer to | ||
[Deploy Cluster with a Private Provider Registry](./deploy-private-registry.md). | ||
|
||
## External Registry | ||
|
||
After the provider images are downloaded, the Palette agent proceeds to download all other images required for the | ||
cluster from registries that are specified in each pack's YAML configuration. However, you can specify an _external | ||
registry_ as the registry from which all images used by the cluster will be downloaded, including the provider images. | ||
|
||
When an external registry is provided in the user data, the Palette agent will replace the registry URL of every image | ||
used by the cluster with the external registry. For example, if your OS pack specified that the provider images be | ||
downloaded from `quay.io/kairos/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1`, but in your user data, you have specified an | ||
external registry `10.10.254.254:8000/spectro-images`. The Palette agent will automatically download the image using the | ||
lennessyy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
tag `10.10.254.254:8000/spectro-images/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1` instead of looking for the image in the | ||
`quay.io/kairos` registry. | ||
|
||
If you want to use a private image registry for applications on your Edge cluster, you can instruct the Palette agent to | ||
download images from an _authenticated external registry_. You can specify an external registry in the user-data used to | ||
build your Edge Installer ISO. For more information on how to deploy a cluster with an authenticated external registry, | ||
refer to [Deploy Cluster with a Private External Registry](./deploy-external-registry.md). | ||
|
||
## Local Harbor Registry | ||
|
||
A local Harbor registry is a local registry that runs on your cluster. After the initial download of images from other | ||
registries, all images except for infrastructure layer images will be uploaded to the Harbor registry. Subsequently, all | ||
requests for application images from the cluster will be made to the Harbor registry. For more information, refer to | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚫 [vale] reported by reviewdog 🐶 |
||
[Enable Local Harbor Image Registry](./local-registry.md). | ||
|
||
## Limitations | ||
|
||
There are limitations to using the local Harbor image registry and authenticated registries. The following table shows | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚫 [vale] reported by reviewdog 🐶 |
||
you which combinations are supported. | ||
|
||
| Feature | Can be used with | Cannot be used with | | ||
| ------------------------------- | ------------------------------------------------------------------ | --------------------------------------------------------------- | | ||
| Local Harbor registry | - Authenticated external registry <br/> - Public provider registry | Authenticated provider registry | | ||
| Authenticated external registry | - Local Harbor registry <br/> - Public provider registry | Authenticated provider registry. | | ||
| Authenticated provider registry | N/A | - Authenticated external registry <br/> - Local Harbor registry | |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
--- | ||
sidebar_label: "Deploy Cluster with a Private External Registry" | ||
title: "Deploy Cluster with a Private External Registry" | ||
description: "Instructions for deploying an Edge cluster with a private external registry." | ||
hide_table_of_contents: false | ||
sidebar_position: 20 | ||
tags: ["edge"] | ||
--- | ||
|
||
Palette Edge provides support for downloading images from authenticated external registries. You can instruct the | ||
Palette agent to download images from an authenticated external registry by specifying the address and the credentials | ||
for the registry in the user data used to build your Edge Installer ISO. | ||
|
||
Once you specify an external registry, images for all elements of the cluster are expected to be in the external | ||
registry. This includes the provider images, images for the network and storage layer, and images for all application | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚫 [vale] reported by reviewdog 🐶 |
||
layers. All images specified in the cluster profile will have their registry URL replaced by the registry URL of the | ||
external image registry. For example, if your OS pack specified that the provider images be downloaded from | ||
`quay.io/kairos/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1`, but in your user data, you have specified an external registry | ||
`10.10.254.254:8000/spectro-images`. The Palette agent will automatically download the image using the tag | ||
`10.10.254.254:8000/spectro-images/core-ubuntu-20-lts-rke2:v1.25.2-rke2r1` instead of looking for the image in the | ||
`quay.io/kairos` registry. | ||
|
||
:::tip | ||
|
||
You can use a private external registry together with a local Harbor image registry by adding the Harbor Edge-Native | ||
Config pack to your cluster profile. All images for add-on layers of the cluster will be stored in the local Harbor | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚫 [vale] reported by reviewdog 🐶 |
||
registry after the initial download, which allows you to reduce the bandwidth use and protect against outages. For more | ||
information, refer to [Enable Local Harbor Registry](./local-registry.md). | ||
|
||
::: | ||
|
||
## Limitations | ||
|
||
- Palette Edge supports basic username/password authentication. Token authentication schemes used by services such as | ||
AWS ECR and Google Artifact Registry are not supported. | ||
|
||
## Prerequisites | ||
|
||
- Specifying the external registry and providing credentials happens during the EdgeForge process. You should become | ||
familiar with EdgeForge before following this guide. Refer to | ||
[Build Edge Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md) to learn how to build Edge Installer | ||
ISO and provider images. | ||
|
||
- A physical or virtual Linux machine with _AMD64_ (also known as _x86_64_) processor architecture to build the Edge | ||
lennessyy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
artifacts. You can issue the following command in the terminal to check your processor architecture. | ||
|
||
```bash | ||
uname -m | ||
``` | ||
|
||
- Minimum hardware configuration of the Linux machine: | ||
|
||
- 4 CPU | ||
- 8 GB memory | ||
- 50 GB storage | ||
|
||
- [Git](https://cli.github.com/manual/installation). You can ensure git installation by issuing the `git --version` | ||
command. | ||
|
||
- [Docker Engine](https://docs.docker.com/engine/install/) version 18.09.x or later. You can use the `docker --version` | ||
command to view the existing Docker version. | ||
|
||
- You should have root-level or `sudo` privileges on your Linux machine to create privileged containers. | ||
|
||
- A [Spectro Cloud](https://console.spectrocloud.com) account. If you have not signed up, you can sign up for a | ||
[free trial](https://www.spectrocloud.com/free-tier/). | ||
|
||
- Palette registration token for pairing Edge hosts with Palette. You will need tenant admin access to Palette to | ||
generate a new registration token. For detailed instructions, refer to the | ||
[Create Registration Token](/clusters/edge/site-deployment/site-installation/create-registration-token) guide. | ||
|
||
- A private external registry that stores all images required by your cluster. | ||
|
||
## Deploy Cluster with a Private External Registry | ||
|
||
1. Check out the [CanvOS](https://github.com/spectrocloud/CanvOS) GitHub repository containing the starter code. | ||
|
||
```bash | ||
git clone https://github.com/spectrocloud/CanvOS.git | ||
``` | ||
|
||
2. Change to the **CanvOS/** directory. | ||
|
||
```bash | ||
cd CanvOS | ||
``` | ||
|
||
3. In the user data file, provide the URL and the credentials in `stylus.registryCredentials`. The following is an | ||
example: | ||
|
||
```yaml | ||
#cloud-config | ||
stylus: | ||
registryCredentials: | ||
domain: 10.10.254.254:8000/spectro-images | ||
username: ubuntu | ||
password: ******* | ||
insecure: true | ||
``` | ||
|
||
Refer to [Installer Configuration](../../edge-configuration/installer-reference.md#external-registry) for a | ||
description of each field. | ||
|
||
4. Follow the rest of the [Build Edge Artifact](../../edgeforge-workflow/palette-canvos/palette-canvos.md) guide and | ||
build the Installer ISO with the user data containing the registry credentials. | ||
|
||
5. Follow the [Perform Site Install](../site-installation/site-installation.md) guide to perform the installation. | ||
|
||
6. Log in to [Palette](https://console.spectrocloud.com). | ||
|
||
7. From the left **Main Menu**, click on **Profiles**. Then select the profile you are using to deploy the cluster. | ||
|
||
8. Go through each layer of the profile and ensure that all images referenced in the profile are present in the external | ||
registry. If you do not want to do this manually image by image, refer to | ||
[Upload Cluster Images to External Registry with Palette Edge CLI](./upload-images-to-registry.md) to learn how to | ||
use the Palette Edge CLI to upload all images in a cluster profile to an external registry. | ||
|
||
9. Follow the [Create Cluster Definition](../site-installation/cluster-deployment.md) guide and deploy your cluster. | ||
|
||
## Validate | ||
|
||
1. Log in to [Palette](https://console.spectrocloud.com). | ||
|
||
2. From the left **Main Menu**, click on **Clusters**. | ||
|
||
3. Verify that the cluster you provisioned is in running status. Since your cluster profile only references images in | ||
the private external registry, you can confirm that the images were downloaded successfully. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,30 +1,30 @@ | ||
--- | ||
sidebar_label: "Deploy Cluster with a Private Registry" | ||
title: "Deploy Cluster with a Private Registry" | ||
description: "Instructions for creating an Edge Native Cluster Profile" | ||
sidebar_label: "Deploy Cluster with a Private Provider Registry" | ||
title: "Deploy Cluster with a Private Provider Registry" | ||
description: "Instructions for deploying an Edge cluster with a private provider registry." | ||
hide_table_of_contents: false | ||
sidebar_position: 60 | ||
sidebar_position: 40 | ||
tags: ["edge"] | ||
--- | ||
|
||
Palette Edge supports authentication with private image registries, which allows your cluster to pull images from a | ||
private registry during deployment. You can configure your cluster to pull images from a private registry for both | ||
cluster creation and cluster updates. To configure a cluster to pull images from a private image registry, provide the | ||
registry URL and the credentials needed to authenticate with the registry in the cluster profile. | ||
Palette Edge supports authentication with private image registries, which allows your cluster to pull provider images | ||
from a private registry during deployment. You can configure your cluster to pull provider images from a private | ||
registry for both cluster creation and cluster updates. | ||
|
||
:::tip | ||
To configure a cluster to pull images from a private image registry, provide the registry URL and the credentials needed | ||
to authenticate with the registry in the cluster profile. This registry supplies the provider images only. If you want | ||
to use a private registry for images other the provider images, refer to | ||
[Deploy Cluster with Private External Registry](./deploy-external-registry.md). | ||
|
||
You can use the [Harbor Edge-Native Config](../../../integrations/harbor-edge.md) pack to configure a local registry in | ||
your Edge cluster. After the initial download of artifacts from the private registry, the cluster will redirect | ||
subsequent image pulls to the local Harbor registry. This allows you to save network bandwidth and provision clusters | ||
without a connection to external networks. For more information, refer to | ||
[Enable Local Harbor Registry](../networking/local-registry.md). | ||
## Limitations | ||
|
||
::: | ||
- A cluster cannot pull provider images from more than one private registry. | ||
|
||
## Limitations | ||
- If you have already specified an external registry. the provider registry will be ignored and the provider images will | ||
be pulled from the external registry instead. | ||
|
||
- A cluster cannot pull images from more than one private registry. | ||
- You cannot use private provider registries for clusters with a local Harbor registry. For more information, refer to | ||
[Enable Local Harbor Registry](./local-registry.md). | ||
|
||
- If your private registry has TLS enabled, you can only configure a _new_ cluster to use a TLS certificate with a | ||
private registry. You cannot configure an existing cluster with a TLS certificate to communicate with your private | ||
|
@@ -40,7 +40,7 @@ without a connection to external networks. For more information, refer to | |
- A private image registry. | ||
|
||
- A provider image you created in the EdgeForge process stored in your private image registry. For more information, | ||
refer to [Build Artifacts](../edgeforge-workflow/palette-canvos/palette-canvos.md). | ||
refer to [Build Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md). | ||
|
||
## Enablement | ||
|
||
|
@@ -58,7 +58,7 @@ without a connection to external networks. For more information, refer to | |
pack for your OS layer. | ||
|
||
5. Update the `system.uri` parameter in the pack editor for your OS layer. Use the custom OS image you created in the | ||
EdgeForge process. Refer to the EdgeForge [Build Images](../edgeforge-workflow/palette-canvos/palette-canvos.md) | ||
EdgeForge process. Refer to the EdgeForge [Build Images](../../edgeforge-workflow/palette-canvos/palette-canvos.md) | ||
guide if you are missing a custom OS image. The following is an example configuration using the BYOOS pack with a | ||
custom OS image. | ||
|
||
|
@@ -78,17 +78,17 @@ without a connection to external networks. For more information, refer to | |
|
||
If you have specified registry credentials in the `registryCredentials` field in the user data file during the | ||
EdgeForge process, the credentials provided in the cluster profile will be ignored. For more information, refer to | ||
[EdgeForge - Build Artifacts](../edgeforge-workflow/palette-canvos/palette-canvos.md) and | ||
[Installer Configuration](../edge-configuration/installer-reference.md#external-registry). | ||
[EdgeForge - Build Artifacts](../../edgeforge-workflow/palette-canvos/palette-canvos.md) and | ||
[Installer Configuration](../../edge-configuration/installer-reference.md#external-registry). | ||
|
||
::: | ||
|
||
6. At the root level of YAML for your OS layer, add the `providerCredentials` field to provide the credentials you need | ||
to authenticate with your registry. For more information about the `providerCredentials` field, refer to | ||
[Bring Your Own OS (BYOOS)](../../../integrations/byoos.md) pack page. The `providerCredentials.password` field will | ||
be masked when you provide it in the YAML file. You can also use a macro to store your credentials instead of | ||
providing it directly in the YAML file. For more information, refer to | ||
[Macros Support](../../cluster-management/macros.md): | ||
[Bring Your Own OS (BYOOS)](/docs/docs-content/integrations/byoos.md) pack page. The `providerCredentials.password` | ||
field will be masked when you provide it in the YAML file. You can also use a macro to store your credentials instead | ||
of providing it directly in the YAML file. For more information, refer to | ||
[Macros Support](/docs/docs-content/clusters/cluster-management/macros.md): | ||
|
||
```yaml {7-8} | ||
pack: | ||
|
@@ -119,9 +119,10 @@ without a connection to external networks. For more information, refer to | |
|
||
8. If you already have an active cluster that is using the original version of the cluster profile, update the cluster | ||
so that it uses the new version of the cluster profile you just published. For more information about updating | ||
clusters, refer to [Update a Cluster](../../cluster-management/cluster-updates.md). This will trigger a full cluster | ||
repave since it includes an update to the OS layer of the cluster. To learn more about cluster repave behavior, refer | ||
to [Repave Behavior and Configuration](../../cluster-management/node-pool.md#repave-behavior-and-configuration). | ||
clusters, refer to [Update a Cluster](../../../cluster-management/cluster-updates.md). This will trigger a full | ||
cluster repave since it includes an update to the OS layer of the cluster. To learn more about cluster repave | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚫 [vale] reported by reviewdog 🐶 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚫 [vale] reported by reviewdog 🐶 |
||
behavior, refer to | ||
[Repave Behavior and Configuration](../../../cluster-management/node-pool.md#repave-behavior-and-configuration). | ||
|
||
If you don't have an active cluster yet, deploy a new cluster with the profile you just created, and the cluster will | ||
pull images from the private registry you specified. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚫 [vale] reported by reviewdog 🐶
[Vale.Terms] Use 'Application' instead of 'application'.