Skip to content

Commit

Permalink
update: Various language fixes to EFS lab (#1173)
Browse files Browse the repository at this point in the history
  • Loading branch information
niallthomson authored Nov 1, 2024
1 parent 42ccbc9 commit 07125bf
Show file tree
Hide file tree
Showing 4 changed files with 41 additions and 39 deletions.
24 changes: 12 additions & 12 deletions website/docs/fundamentals/storage/efs/deployment-with-efs.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,25 @@ title: Dynamic provisioning using EFS
sidebar_position: 30
---

Now that we understand the EFS storage class for Kubernetes let's create a [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) and change the `assets` container on the assets deployment to mount the Volume created.
Now that we understand the EFS storage class for Kubernetes, let's create a [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) and modify the `assets` container deployment to mount this volume.

First inspect the `efspvclaim.yaml` file to see the parameters in the file and the claim of the specific storage size of 5GB from the Storage class `efs-sc` we created in the earlier step:
First, let's examine the `efspvclaim.yaml` file which defines a PersistentVolumeClaim requesting 5GB of storage from the `efs-sc` storage class we created earlier:

```file
manifests/modules/fundamentals/storage/efs/deployment/efspvclaim.yaml
```

We'll also modify the assets service in two ways:
We'll update the assets service to:

- Mount the PVC to the location where the assets images are stored
- Add an [init container](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) to copy the initial images to the EFS volume
- Mount the PVC at the location where assets images are stored
- Include an [init container](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that copies the initial images to the EFS volume

```kustomization
modules/fundamentals/storage/efs/deployment/deployment.yaml
Deployment/assets
```

We can apply the changes by running the following command:
Apply these changes with the following command:

```bash
$ kubectl apply -k ~/environment/eks-workshop/modules/fundamentals/storage/efs/deployment
Expand All @@ -34,7 +34,7 @@ deployment.apps/assets configured
$ kubectl rollout status --timeout=130s deployment/assets -n assets
```

Now look at the `volumeMounts` in the deployment, notice that we have our new `Volume` named `efsvolume` mounted on`volumeMounts` named `/usr/share/nginx/html/assets`:
Let's examine the `volumeMounts` in the deployment. Notice that our new volume named `efsvolume` is mounted at `/usr/share/nginx/html/assets`:

```bash
$ kubectl get deployment -n assets \
Expand All @@ -45,15 +45,15 @@ $ kubectl get deployment -n assets \
name: tmp-volume
```

A PersistentVolume (PV) has been created automatically for the PersistentVolumeClaim (PVC) we had created in the previous step:
A PersistentVolume (PV) has been automatically created to fulfill our PersistentVolumeClaim (PVC):

```bash
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-342a674d-b426-4214-b8b6-7847975ae121 5Gi RWX Delete Bound assets/efs-claim efs-sc 2m33s
```

Also describe the PersistentVolumeClaim (PVC) created:
Let's examine the details of our PersistentVolumeClaim (PVC):

```bash
$ kubectl describe pvc -n assets
Expand All @@ -80,15 +80,15 @@ Events:
Normal ProvisioningSucceeded 33s efs.csi.aws.com_efs-csi-controller-6b4ff45b65-fzqjb_7efe91cc-099a-45c7-8419-6f4b0a4f9e01 Successfully provisioned volume pvc-342a674d-b426-4214-b8b6-7847975ae121
```

Now create a new file `newproduct.png` under the assets directory in the first Pod:
To demonstrate the shared storage functionality, let's create a new file `newproduct.png` in the assets directory of the first Pod:

```bash
$ POD_NAME=$(kubectl -n assets get pods -o jsonpath='{.items[0].metadata.name}')
$ kubectl exec --stdin $POD_NAME \
-n assets -c assets -- bash -c 'touch /usr/share/nginx/html/assets/newproduct.png'
```

And verify that the file now also exists in the second Pod:
Now verify that this file exists in the second Pod:

```bash
$ POD_NAME=$(kubectl -n assets get pods -o jsonpath='{.items[1].metadata.name}')
Expand All @@ -104,4 +104,4 @@ test.txt
wood_watch.jpg
```

Now as you can see even though we created a file through the first Pod the second Pod also has access to this file because of the shared EFS file system.
As you can see, even though we created the file through the first Pod, the second Pod has immediate access to it because they're both using the same EFS file system.
20 changes: 10 additions & 10 deletions website/docs/fundamentals/storage/efs/deployments.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ title: Persistent network storage
sidebar_position: 10
---

On our ecommerce application, we have a deployment already created as part of our assets microservice. The assets microservice utilizes a webserver running on EKS. Web servers are a great example for the use of deployments because they **scale horizontally** and **declare the new state** of the Pods.
Our ecommerce application includes a deployment for the assets microservice, which runs a webserver on EKS. Web servers are an excellent use case for deployments as they can **scale horizontally** and **declare the new state** of the Pods.

Assets component is a container which serves static images for products, these product images are added as part of the container image build. However with this setup every time the team wants to update the product images they have to recreate and redeploy the container image. In this exercise we'll utilize [EFS File System](https://docs.aws.amazon.com/efs/latest/ug/whatisefs.html) and Kubernetes [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) to update old product images and add new product images without the need to rebuild the containers images.
The assets component serves static product images that are currently bundled into the container image during build time. This means that whenever the team needs to update product images, they must rebuild and redeploy the container image. In this exercise, we'll use [Amazon EFS File System](https://docs.aws.amazon.com/efs/latest/ug/whatisefs.html) and Kubernetes [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) to enable updating existing product images and adding new ones without rebuilding container images.

We can start by describing the Deployment to take a look at its initial volume configuration:
Let's start by examining the Deployment's initial volume configuration:

```bash
$ kubectl describe deployment -n assets
Expand Down Expand Up @@ -38,13 +38,13 @@ Namespace: assets
[...]
```
As you can see the [`Volumes`](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir-configuration-example) section of our Deployment shows that we're only using an [EmptyDir volume type](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) which "shares the Pod's lifetime".
The [`Volumes`](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir-configuration-example) section shows we're only using an [EmptyDir volume type](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) which is tied to the Pod's lifetime.

![Assets with emptyDir](./assets/assets-emptydir.webp)

An `emptyDir` volume is first created when a Pod is assigned to a node, and exists as long as that Pod is running on that node. As the name says, the emptyDir volume is initially empty. All containers in the Pod can read and write the same files in the emptyDir volume, though that volume can be mounted at the same or different paths in each container. **When a Pod is removed from a node for any reason, the data in the emptyDir is deleted permanently.** This means that if we want to share data between multiple Pods in the same Deployment and make changes to that data then EmptyDir is not a good fit.
An `emptyDir` volume is created when a Pod is assigned to a node and exists only while that Pod runs on that node. As the name suggests, the emptyDir volume is initially empty. While all containers in the Pod can read and write files in the emptyDir volume, **when a Pod is removed from a node for any reason, the data in the emptyDir is deleted permanently.** This makes EmptyDir unsuitable for sharing data between multiple Pods in the same Deployment when that data needs to be modified.

The container has some initial product images copied to it as part of the container build under the folder `/usr/share/nginx/html/assets`, we can check by running the below command:
The container includes some initial product images copied during the build process to `/usr/share/nginx/html/assets`. We can verify this with:

```bash
$ kubectl exec --stdin deployment/assets \
Expand All @@ -57,29 +57,29 @@ smart_2.jpg
wood_watch.jpg
```

First lets scale up the `assets` Deployment so it has multiple replicas:
Let's scale up the `assets` Deployment to multiple replicas:
```bash
$ kubectl scale -n assets --replicas=2 deployment/assets
$ kubectl rollout status -n assets deployment/assets --timeout=60s
```
Now let us try to put a new product image named `newproduct.png` in the directory `/usr/share/nginx/html/assets` of the first Pod using the below command:
Now let's try creating a new product image file `newproduct.png` in the `/usr/share/nginx/html/assets` directory of the first Pod:

```bash
$ POD_NAME=$(kubectl -n assets get pods -o jsonpath='{.items[0].metadata.name}')
$ kubectl exec --stdin $POD_NAME \
-n assets -- bash -c 'touch /usr/share/nginx/html/assets/newproduct.png'
```

Now confirm the new product image `newproduct.png` isn't present on the file system of the second Pod:
Let's verify if the new product image `newproduct.png` exists in the second Pod's file system:

```bash
$ POD_NAME=$(kubectl -n assets get pods -o jsonpath='{.items[1].metadata.name}')
$ kubectl exec --stdin $POD_NAME \
-n assets -- bash -c 'ls /usr/share/nginx/html/assets'
```

As you see the newly created image `newproduct.png` does not exist on the second Pod. In order to help solve this issue we need a file system that can be shared across multiple Pods if the service needs to scale horizontally while still making updates to the files without re-deploying.
As we can see, the newly created image `newproduct.png` doesn't exist on the second Pod. To address this limitation, we need a file system that can be shared across multiple Pods when the service scales horizontally while allowing file updates without redeployment.
![Assets with EFS](./assets/assets-efs.webp)
24 changes: 12 additions & 12 deletions website/docs/fundamentals/storage/efs/efs-csi-driver.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,55 +3,55 @@ title: EFS CSI Driver
sidebar_position: 20
---

Before we dive into this section, make sure to familiarized yourself with the Kubernetes storage objects (volumes, persistent volumes (PV), persistent volume claim (PVC), dynamic provisioning and ephemeral storage) that were introduced on the [Storage](../index.md) main section.
Before diving into this section, you should be familiar with the Kubernetes storage objects (volumes, persistent volumes (PV), persistent volume claims (PVC), dynamic provisioning and ephemeral storage) that were introduced in the [Storage](../index.md) main section.

The [Amazon Elastic File System Container Storage Interface (CSI) Driver](https://github.com/kubernetes-sigs/aws-efs-csi-driver) helps you run stateful containerized applications. Amazon EFS Container Storage Interface (CSI) driver provide a CSI interface that allows Kubernetes clusters running on AWS to manage the lifecycle of Amazon EFS file systems.
The [Amazon Elastic File System Container Storage Interface (CSI) Driver](https://github.com/kubernetes-sigs/aws-efs-csi-driver) enables you to run stateful containerized applications by providing a CSI interface that allows Kubernetes clusters running on AWS to manage the lifecycle of Amazon EFS file systems.

In order to utilize Amazon EFS file system with dynamic provisioning on our EKS cluster, we need to confirm that we have the EFS CSI Driver installed. The [Amazon Elastic File System Container Storage Interface (CSI) Driver](https://github.com/kubernetes-sigs/aws-efs-csi-driver) implements the CSI specification for container orchestrators to manage the lifecycle of Amazon EFS file systems.
To utilize Amazon EFS with dynamic provisioning on our EKS cluster, we first need to confirm that we have the EFS CSI Driver installed. The driver implements the CSI specification which allows container orchestrators to manage Amazon EFS file systems throughout their lifecycle.

To improve security and reduce the amount of work, you can manage the Amazon EFS CSI driver as an Amazon EKS add-on. The IAM role needed by the add-on was created for us so we can go ahead and install the add-on:
For improved security and simplified management, you can run the Amazon EFS CSI driver as an Amazon EKS add-on. Since the required IAM role has already been created for us, we can proceed with installing the add-on:

```bash timeout=300 wait=60
$ aws eks create-addon --cluster-name $EKS_CLUSTER_NAME --addon-name aws-efs-csi-driver \
--service-account-role-arn $EFS_CSI_ADDON_ROLE
$ aws eks wait addon-active --cluster-name $EKS_CLUSTER_NAME --addon-name aws-efs-csi-driver
```

Now we can take a look at what has been created in our EKS cluster by the addon. For example, a DaemonSet will be running a pod on each node in our cluster:
Let's examine what the add-on has created in our EKS cluster. For example, a DaemonSet that runs a pod on each node in our cluster:

```bash
$ kubectl get daemonset efs-csi-node -n kube-system
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
efs-csi-node 3 3 3 3 3 kubernetes.io/os=linux 47s
```

The EFS CSI driver supports dynamic and static provisioning. Currently dynamic provisioning creates an access point for each PersistentVolume. This mean an AWS EFS file system has to be created manually on AWS first and should be provided as an input to the StorageClass parameter. For static provisioning, AWS EFS file system needs to be created manually on AWS first. After that it can be mounted inside a container as a volume using the driver.
The EFS CSI driver supports both dynamic and static provisioning. For dynamic provisioning, the driver creates an access point for each PersistentVolume, but requires an existing AWS EFS file system that must be specified in the StorageClass parameters. Static provisioning also requires a pre-created AWS EFS file system, which can then be mounted as a volume inside a container using the driver.

We have provisioned an EFS file system, mount targets and the required security group pre-provisioned with an inbound rule that allows inbound NFS traffic for your Amazon EFS mount points. Let's retrieve some information about it that will be used later:
An EFS file system has been provisioned for us, along with mount targets and the required security group that includes an inbound rule allowing NFS traffic to the EFS mount points. Let's get its ID which we'll need later:

```bash
$ export EFS_ID=$(aws efs describe-file-systems --query "FileSystems[?Name=='$EKS_CLUSTER_NAME-efs-assets'] | [0].FileSystemId" --output text)
$ echo $EFS_ID
fs-061cb5c5ed841a6b0
```

Now, we'll need to create a [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) object configured to use the pre-provisioned EFS file system as part of this workshop infrastructure and use [EFS Access points](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html) in provisioning mode.
Next, we'll create a [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) object configured to use our pre-provisioned EFS file system and [EFS Access points](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html) in provisioning mode.

We'll be using Kustomize to create for us the storage class and to ingest the environment variable `EFS_ID` in the parameter `filesystemid` value in the configuration of the storage class object:
Using Kustomize, we'll create the storage class and inject the `EFS_ID` environment variable into the `filesystemid` parameter:

```file
manifests/modules/fundamentals/storage/efs/storageclass/efsstorageclass.yaml
```

Let's apply this kustomization:
Apply the kustomization:

```bash
$ kubectl kustomize ~/environment/eks-workshop/modules/fundamentals/storage/efs/storageclass \
| envsubst | kubectl apply -f-
storageclass.storage.k8s.io/efs-sc created
```

Now we'll get and describe the StorageClass using the below commands. Notice that the provisioner used is the EFS CSI driver and the provisioning mode is EFS access point and ID of the file system as exported in the `EFS_ID` environment variable.
Let's examine the StorageClass. Note that it uses the EFS CSI driver as the provisioner and is configured for EFS access point provisioning mode with the file system ID we exported earlier:

```bash
$ kubectl get storageclass
Expand All @@ -71,4 +71,4 @@ VolumeBindingMode: Immediate
Events: <none>
```

Now that we have a better understanding of EKS StorageClass and EFS CSI driver. On the next page, we'll focus on modifying the asset microservice to leverage the EFS `StorageClass` using Kubernetes dynamic volume provisioning and a PersistentVolume to store the product images.
Now that we understand EKS StorageClass and the EFS CSI driver, we'll proceed to modify the assets microservice to use the EFS `StorageClass` with Kubernetes dynamic volume provisioning and a PersistentVolume for storing product images.
12 changes: 7 additions & 5 deletions website/docs/fundamentals/storage/efs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@ You can view the Terraform that applies these changes [here](https://github.com/

:::

[Amazon Elastic File System](https://docs.aws.amazon.com/efs/latest/ug/whatisefs.html) is a simple, serverless, set-and-forget elastic file system for use with AWS Cloud services and on-premises resources. It's built to scale on demand to petabytes without disrupting applications, growing and shrinking automatically as you add and remove files, eliminating the need to provision and manage capacity to accommodate growth.
[Amazon Elastic File System](https://docs.aws.amazon.com/efs/latest/ug/whatisefs.html) (Amazon EFS) provides a serverless, fully elastic file system that automatically scales on demand to petabytes without disrupting applications. It eliminates the need to provision and manage capacity as you add and remove files, making it ideal for use with AWS Cloud services and on-premises resources.

In this lab, we'll learn about the following concepts:
In this lab, you will:

- Assets microservice deployment
- EFS CSI Driver
- Dynamic provisioning using EFS and a Kubernetes deployment
- Learn about persistent network storage with the assets microservice
- Configure and deploy the EFS CSI Driver for Kubernetes
- Implement dynamic provisioning using EFS in a Kubernetes deployment

This hands-on experience will demonstrate how to effectively use Amazon EFS with Amazon EKS for scalable, persistent storage solutions.

0 comments on commit 07125bf

Please sign in to comment.