From 52403c4828b697d3db042a6a4afa2c65fc6822f2 Mon Sep 17 00:00:00 2001 From: Ben Radstone Date: Tue, 10 Dec 2024 17:24:03 +0000 Subject: [PATCH] docs: further hybrid node pool template additions --- .../aws/eks-hybrid-nodes/architecture.md | 78 +++++++++++-------- .../create-hybrid-node-pools.md | 68 ++++++++++++++++ 2 files changed, 113 insertions(+), 33 deletions(-) diff --git a/docs/docs-content/clusters/public-cloud/aws/eks-hybrid-nodes/architecture.md b/docs/docs-content/clusters/public-cloud/aws/eks-hybrid-nodes/architecture.md index 5536c3843f..75591b7005 100644 --- a/docs/docs-content/clusters/public-cloud/aws/eks-hybrid-nodes/architecture.md +++ b/docs/docs-content/clusters/public-cloud/aws/eks-hybrid-nodes/architecture.md @@ -49,57 +49,69 @@ for guidance on all available options. If using a VPN or AWS Direct Connect between AWS and your on-premises and edge environments, review the following configuration requirements. -- Configure your EKS cluster with static placement so that your nodes are assigned to specific Availability Zones (AZs) - and fixed networking configurations. This is required because of the following reasons: +#### AWS - - The VPN configuration must be set up with predefined routes and IP ranges. - - Node placement cannot change dynamically across AZs. - - Network paths need to remain consistent for VPN tunnels to function properly. +Configure your EKS cluster with static placement so that your nodes are assigned to specific Availability Zones (AZs) and fixed networking configurations. This is required because of the following reasons: -- Traffic routing in the Amazon EKS VPC requires the following mapping for hybrid nodes: +- The VPN configuration must be set up with predefined routes and IP ranges. +- Node placement cannot change dynamically across AZs. +- Network paths need to remain consistent for VPN tunnels to function properly. - - Route table entries mapping hybrid node CIDR ranges to VPN endpoint. - For example, Hybrid Node CIDR 10.200.0.0/16 → VPN endpoint 172.16.0.1. +Traffic routing in the Amazon EKS VPC requires the following mapping for hybrid nodes: - - Route table entries mapping hybrid pod CIDR ranges to VPN endpoint. - For example, Hybrid Pod CIDR 192.168.0.0/16 → VPN endpoint 172.16.0.1. +- Route table entries mapping hybrid node CIDR ranges to VPN endpoint. + For example, Hybrid Node CIDR 10.200.0.0/16 → VPN endpoint 172.16.0.1. - - For AWS Direct Connect, map traffic to appropriate private subnet CIDR. - For example, Both CIDRs 10.200.0.0/16 & 192.168.0.0/16 → Private subnet 172.16.1.0/24. +- Route table entries mapping hybrid pod CIDR ranges to VPN endpoint. + For example, Hybrid Pod CIDR 192.168.0.0/16 → VPN endpoint 172.16.0.1. -- For AWS VPNs, configure two static routes for each of the following connections: +- For AWS Direct Connect, map traffic to appropriate private subnet CIDR. + For example, Both CIDRs 10.200.0.0/16 & 192.168.0.0/16 → Private subnet 172.16.1.0/24. - - Hybrid Node CIDR block. - For example, Hybrid Node CIDR 10.200.0.0/16 → VPN endpoint 172.16.0.1. +For AWS VPNs, configure two static routes for each of the following connections: - - Hybrid Pod CIDR block. - For example, Hybrid Pod CIDR 192.168.0.0/16 → VPN endpoint 172.16.0.1. +- Hybrid Node CIDR block. + For example, Hybrid Node CIDR 10.200.0.0/16 → VPN endpoint 172.16.0.1. - If you're using a Virtual Private Gateway or Transit Gateway, route propagation can be enabled to automatically - populate your VPC route tables. Ensure you verify your route tables after propagation. +- Hybrid Pod CIDR block. + For example, Hybrid Pod CIDR 192.168.0.0/16 → VPN endpoint 172.16.0.1. -- For on-premises and edge VPNs, set up IPsec Phase 1 tunnels with Phase 2 security associations for the following: +If you're using a Virtual Private Gateway or Transit Gateway, route propagation can be enabled to automatically populate your VPC route tables. Ensure you verify your route tables after propagation. - - Hybrid Node subnet to EKS VPC CIDR. - For example, Hybrid Node Subnet 10.201.0.0/16 → EKS VPC CIDR 10.100.0.0/16. +#### On-Premises and Edge Locations - - Hybrid Node pod CIDR to EKS VPC CIDR. - For example, Hybrid Node Pod CIDR 192.168.0.0/16 → EKS VPC CIDR 10.100.0.0/16. +For on-premises and edge VPNs, set up IPsec Phase 1 tunnels with Phase 2 security associations for the following: - You should also enable either Border Gateway Protocol (BGP) routing or static routes to ensure proper traffic flow - through VPN tunnels. +- Hybrid Node subnet to EKS VPC CIDR. + For example, Hybrid Node Subnet 10.201.0.0/16 → EKS VPC CIDR 10.100.0.0/16. -- For non-primary VPN servers, either broadcast routes via BGP or configure static routes to redirect EKS VPC CIDR - traffic appropriately. +- Hybrid Node pod CIDR to EKS VPC CIDR. + For example, Hybrid Node Pod CIDR 192.168.0.0/16 → EKS VPC CIDR 10.100.0.0/16. + +You should also enable either Border Gateway Protocol (BGP) routing or static routes to ensure proper traffic flow through VPN tunnels. + +For non-primary VPN servers, either broadcast routes via BGP or configure static routes to redirect EKS VPC CIDR traffic appropriately. ## Operating System Compatibility -Palette supports the same operating systems as AWS. Refer to -[Prepare operating system for hybrid nodes](https://docs.aws.amazon.com/eks/latest/userguide/hybrid-nodes-os.html) for -guidance. +Palette supports the operating systems available to edge hosts registered through [Agent Mode](../../../../deployment-modes/agent-mode/agent-mode.md) or by using [Provider Images](../../../edge/edgeforge-workflow/palette-canvos/build-provider-images.md). + +## Supported Edge Hosts + +Palette allows you to use your edge hosts as your Amazon EKS Hybrid Nodes. Your edge hosts need to be registered with Palette before you can add them to your node pools. + +If you want to use your edge hosts as Amazon EKS Hybrid Nodes, they must have been registered through [Agent Mode](../../../../deployment-modes/agent-mode/agent-mode.md) or by using [Provider Images](../../../edge/edgeforge-workflow/palette-canvos/build-provider-images.md). + +:::warning + +If using Provider Images, you must include the following in your `.arg` file during the [build steps](../../../edge/edgeforge-workflow/palette-canvos/build-provider-images.md#build-provider-images). + +```shell +K8S_DISTRIBUTION=nodeadm +K8S_VERSION=1.29.0 # supported versions: [ 1.29.0 | 1.30.0 ] +``` -Edge hosts require additional dependencies and you can build these into provider images using the -[EdgeForge Workflow](../../../edge/edgeforge-workflow/edgeforge-workflow.md). +::: ## Authentication and Access Management diff --git a/docs/docs-content/clusters/public-cloud/aws/eks-hybrid-nodes/create-hybrid-node-pools.md b/docs/docs-content/clusters/public-cloud/aws/eks-hybrid-nodes/create-hybrid-node-pools.md index 4e757c5ef8..504c064948 100644 --- a/docs/docs-content/clusters/public-cloud/aws/eks-hybrid-nodes/create-hybrid-node-pools.md +++ b/docs/docs-content/clusters/public-cloud/aws/eks-hybrid-nodes/create-hybrid-node-pools.md @@ -7,7 +7,75 @@ tags: ["public cloud", "aws", "eks hybrid nodes"] sidebar_position: 30 --- +This section guides you on how to create a cluster profile to collectively manage your hybrid nodes. You can then create hybrid node pools and add your edge hosts to them. + ## Limitations - Overall cluster health is determined by the Amazon EKS cluster's status. While unhealthy edge hosts will appear as unhealthy nodes in Palette, this does not change the Amazon EKS cluster's overall health status. + +## Create Cluster Profile for Hybrid Nodes + +### Prerequisites + +- Edge hosts have been registered with Palette through [Agent Mode](../../../../deployment-modes/agent-mode/agent-mode.md) or by using [Provider Images](../../../edge/edgeforge-workflow/palette-canvos/build-provider-images.md). + + :::warning + + If using Provider Images, you must include the following in your `.arg` file during the [build steps](../../../edge/edgeforge-workflow/palette-canvos/build-provider-images.md#build-provider-images). + + ```shell + K8S_DISTRIBUTION=nodeadm + K8S_VERSION=1.29.0 # supported versions: [ 1.29.0 | 1.30.0 ] + ``` + + ::: + +- Your Palette account role must have the `clusterProfile.create` permission to create a cluster profile. Refer to the + [Cluster Profile](../../../../user-management/palette-rbac/project-scope-roles-permissions.md#cluster-profile) + permissions for guidance. + +### Create Profile + +OS: +* Agent Mode = `byoi-agent-mode 1.0.0` +* Provider Images = `Kairos Edge OS Pack` + +Kubernetes = `edge-nodeadm 1.29.x` or `edge-nodeadm 1.30.x` + +Network = `cni-custom 0.1.0` (dummy) + +### Validate + + +## Hybrid Network ConfigMap (placeholder) + +Before creating any hybrid pools, the user may need to create a `kube-system/hybrid-config` ConfigMap containing `serviceCIDR` and/or `vpcCIDR` keys, whose values are the service CIDR and VPC CIDR for the imported EKS hybrid cluster, respectively. See bullets below for additional detail. + +* The `serviceCIDR` key is only required if one or more edge hosts require proxy configuration. If the ConfigMap is not found (or the key is not found), `10.96.0.0/12` will be used by default. The service CIDR is used to compute the value of `NO_PROXY` to ensure that traffic destined for Kubernetes service endpoints is not sent to the proxy server. + +* The `vpcCIDRkey` is only required if a VPN Server IP is configured for one or more edge hosts in a hybrid pool. In this scenario, the ConfigMap must exist and the `vpcCIDRkey` must be defined. + +* Example: + + ```yaml + apiVersion: v1 + kind: ConfigMap + metadata: + name: hybrid-config + namespace: kube-system + data: + serviceCIDR: "192.168.0.0/16" + vpcCIDR: "10.200.0.0/16" + ``` + +## Create Hybrid Node Pool + +### Prerequisites + +- TBA + +### Create Node Pool + + +### Validate