From dc6d12f830dcd4fc909e3d3cfc780421b15cacfc Mon Sep 17 00:00:00 2001 From: FireFrozen94 <136081311+FireFrozen94@users.noreply.github.com> Date: Tue, 20 Feb 2024 16:42:11 +0100 Subject: [PATCH 01/14] Add files via upload --- Blog_Docker_volume.md | 51 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 Blog_Docker_volume.md diff --git a/Blog_Docker_volume.md b/Blog_Docker_volume.md new file mode 100644 index 0000000..e44fbed --- /dev/null +++ b/Blog_Docker_volume.md @@ -0,0 +1,51 @@ +# Using bind mounts to build a persistent log system with Docker + +_FireFrozen - 2023/09/19_ + +Before reading this article, it's recommended to have a minimum knowledge of docker. See the previous blog on docker : [Get Started with Docker](https://iscsc.fr/blog/64601d94963f4a68d30f5795) by _ctmbl_ + +## Why do we need bind mounts? + +Docker containerization is very useful on lots of points for development but it's very annoying if your application need to produce persistent data like logs. Because if you kill your container (to update it for example) or if it crash, you lose everything that where inside and understanding why your application crashed without its logs is very complicated. + +So we need to find a way to keep certain files after the container's death and that can be done with a bind mounts. + +## What is a bind mounts? + +A bind mounts is the action of mounting an existing file or directory of the host (the computer running the container) machine into a container. To rephrase, it enables a container to share a memory space with the host. So both the container and the user can access and modify it in real time and because it's not "contained" in the container but only mounted, it persists after container's death. + +## How to make a bind mounts? + +To make a bind mounts, you need to add a specific argument for the "docker run" command. There is two different arguments to make a bind mounts which are really similar in their uses. Those are "--mount" and "--volume". Both can be used to make bind mounts or to create volumes (more advanced way to create and use persistent data with docker). For the bind mounts part, they only differs in one behavior : if you enter a non existing file to mount, "--volume" will create it when "--mount" will return an error. Moreover "--volume" needs less arguments so that's the solution I'm going to describe here. + +The exact syntax is : + +``` +docker run (other args) --volume PATH_TO_HOST_FILE:PATH_TO_CONTAINER_FILE:(optional args) +``` + +* Note that you can use "-v" instead of "--volume" to make it shorter. +* Note that you can use mount a directory instead of a file with the same method (just replace PATH_TO_xxx_FILE with the PATH_TO_xxx_DIRECTORY) + +Optional args must be separated by comas, they are not used very often but lets detail the most useful one quickly : +* ro : if used, the mounts will be in read only mode for the containers. + +Concrete example with the code of the site bot: +``` +HOST_LOG_FILE=$(realpath -P ${LOG_FILE}) + +[...] + +docker run --detach --rm --interactive --tty \ + --name ${NAME} \ + --volume ${HOST_LOG_FILE}:/opt/iscsc.fr-notify-bot/${LOG_FILE} \ + ${NAME}:latest +``` +In this example LOG_FILE is a environment variable defined in the .env + +> **WARNING : Be careful that the bot/application running in the container have the right permission to modify the file/directory** + +## Other utility of bind mounts + +* The same volume can be shared by multiple containers. It can be useful if you want to share a database with applications running on different containers or to centralized logs of multiple application in the same log file. +* In development you can use bind mount to enable code modification of the application without having to restart the container. \ No newline at end of file From d53d454d0731c52944689b82a848c6e0957f4f91 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Fri, 12 Apr 2024 21:43:50 +0200 Subject: [PATCH 02/14] Move article to content folder --- Blog_Docker_volume.md => src/content/posts/Blog_Docker_volume.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename Blog_Docker_volume.md => src/content/posts/Blog_Docker_volume.md (100%) diff --git a/Blog_Docker_volume.md b/src/content/posts/Blog_Docker_volume.md similarity index 100% rename from Blog_Docker_volume.md rename to src/content/posts/Blog_Docker_volume.md From 9e7f876066fc088811f31c6f4259c6b590692dce Mon Sep 17 00:00:00 2001 From: ctmbl Date: Fri, 12 Apr 2024 21:51:53 +0200 Subject: [PATCH 03/14] Rename to docker_volume_introduction.md --- .../{Blog_Docker_volume.md => docker_volume_introduction.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename src/content/posts/{Blog_Docker_volume.md => docker_volume_introduction.md} (100%) diff --git a/src/content/posts/Blog_Docker_volume.md b/src/content/posts/docker_volume_introduction.md similarity index 100% rename from src/content/posts/Blog_Docker_volume.md rename to src/content/posts/docker_volume_introduction.md From 9cc89bb6f3ab853bd940aafe7cd55674b2b8a5ef Mon Sep 17 00:00:00 2001 From: ctmbl Date: Fri, 12 Apr 2024 23:01:44 +0200 Subject: [PATCH 04/14] Update post header --- src/content/posts/docker_volume_introduction.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index e44fbed..90ec2c3 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -1,6 +1,13 @@ -# Using bind mounts to build a persistent log system with Docker +--- +title: "Use Docker's volumes to build a persistent log system" +summary: "Persist data after container's death or share data accross containers through Docker's volume feature" +date: 2023-09-19T12:00:00+0200 +lastUpdate: 2024-04-12T20:00:00+0200 +tags: ["docker", "volume", "log"] +author: FireFrozen +draft: false +--- -_FireFrozen - 2023/09/19_ Before reading this article, it's recommended to have a minimum knowledge of docker. See the previous blog on docker : [Get Started with Docker](https://iscsc.fr/blog/64601d94963f4a68d30f5795) by _ctmbl_ From 8e8397b32360484d5b73fa1f8ba921b1df38ebb3 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Fri, 12 Apr 2024 23:02:02 +0200 Subject: [PATCH 05/14] Update core article see commit body - add links to docker doc - focus the artcile on volumes instead of bind mounts, while giving key resources about their differences - highlight key information --- .../posts/docker_volume_introduction.md | 82 ++++++++++++------- 1 file changed, 54 insertions(+), 28 deletions(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index 90ec2c3..000366f 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -8,51 +8,77 @@ author: FireFrozen draft: false --- +Before reading this article, it's recommended to have a minimum knowledge of `docker`. +See, for example, the previous blog on docker: [Get Started with Docker](https://iscsc.fr/posts/short-docker-introduction/) by `ctmbl`. -Before reading this article, it's recommended to have a minimum knowledge of docker. See the previous blog on docker : [Get Started with Docker](https://iscsc.fr/blog/64601d94963f4a68d30f5795) by _ctmbl_ +> **Disclaimer**: +> Note that in Docker, **two ways exist to persist/share data** with the host: **volumes** and **bind mounts**. +> While some key differences exist between both, they are not relevant in most of our basic use cases. +> We'll then **not distinguish one from the other**, some of what we say might better applied to bind mounts but **we'll only speak of volumes**. +> For more information see [Manage data in Docker](https://docs.docker.com/storage/). -## Why do we need bind mounts? +## What is a Docker's volume? -Docker containerization is very useful on lots of points for development but it's very annoying if your application need to produce persistent data like logs. Because if you kill your container (to update it for example) or if it crash, you lose everything that where inside and understanding why your application crashed without its logs is very complicated. +A docker's volume is the action of **mounting an existing file or directory** of the host (the computer running the container) **into a container**. To rephrase, it enables a container to **share a memory space with the host**. So both the container and the user can access and modify it in real time and because it's not "contained" in the container but only mounted, it'll **persist after the container's death**. -So we need to find a way to keep certain files after the container's death and that can be done with a bind mounts. +## Why do we need volumes? -## What is a bind mounts? +Docker containerization is very useful on lots of points for development but it's very annoying if your application **need to produce persistent data**, like logs. Indeed if you **kill** your container (to update it for example) or if it **crashes**, you **lose everything that where inside** and understanding why your application crashed without its logs can be very complicated or even impossible. -A bind mounts is the action of mounting an existing file or directory of the host (the computer running the container) machine into a container. To rephrase, it enables a container to share a memory space with the host. So both the container and the user can access and modify it in real time and because it's not "contained" in the container but only mounted, it persists after container's death. +So we need to find a way to **keep certain files after the container's death** and that can be **achieved with volumes**!. -## How to make a bind mounts? +## How to create a volume? -To make a bind mounts, you need to add a specific argument for the "docker run" command. There is two different arguments to make a bind mounts which are really similar in their uses. Those are "--mount" and "--volume". Both can be used to make bind mounts or to create volumes (more advanced way to create and use persistent data with docker). For the bind mounts part, they only differs in one behavior : if you enter a non existing file to mount, "--volume" will create it when "--mount" will return an error. Moreover "--volume" needs less arguments so that's the solution I'm going to describe here. +To create a volume, you need to add a specific argument to the `docker run` command. -The exact syntax is : +> You can also create volumes outside of containers with [`docker volume`](https://docs.docker.com/storage/volumes/#create-and-manage-volumes) -``` -docker run (other args) --volume PATH_TO_HOST_FILE:PATH_TO_CONTAINER_FILE:(optional args) +There is two different arguments which are really similar in their use. Those are `--mount` and `--volume`, both can be used to create volumes (or bind mounts, see Disclaimer above). +However they slightly differ: if you enter a non existing file to mount, `--volume` will create it when `--mount` will return an error, also `--volume` needs fewer arguments. For these reason and because we focused on volumes in this post we'll describe the `--volume` argument. + +The exact syntax is: +```bash +docker run --volume PATH_TO_HOST_FILE:PATH_TO_CONTAINER_FILE: ``` -* Note that you can use "-v" instead of "--volume" to make it shorter. -* Note that you can use mount a directory instead of a file with the same method (just replace PATH_TO_xxx_FILE with the PATH_TO_xxx_DIRECTORY) +> Note that you can use `-v` instead of `--volume`. -Optional args must be separated by comas, they are not used very often but lets detail the most useful one quickly : -* ro : if used, the mounts will be in read only mode for the containers. +> Note that you can mount a directory instead of a file with the same method. -Concrete example with the code of the site bot: +> A similar syntax exists for [`docker-compose.yml`](https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes) + +Optional args must be separated by comas, they are not used very often but lets detail the most useful one quickly : +- `ro` : if used, the mounted files/folders will be in read only mode inside the container + +A concrete example with the [code of our RootMe discord bot](https://github.com/iScsc/RootPythia/blob/2681ca26286ea5063371536e995a5e3cf39734a5/run.sh#L12): +```bash + source ./.env.prod + + docker run --rm --interactive --tty \ + --detach \ + --volume $(realpath -P ${LOG_FOLDER}):/opt/${NAME}/logs \ + --env-file .env.prod \ + --name ${NAME} \ + ${NAME}:latest ``` -HOST_LOG_FILE=$(realpath -P ${LOG_FILE}) +In this example `LOG_FOLDER` is an environment variable defined in the `.env.prod` file. +The code will written transparently to `/opt/${NAME}/logs/` but because this folder is shared with the host, logs will **be available in the host** and survive potential application crashes or updates. -[...] +> **WARNING : Be careful that the bot/application running in the container must have the right permission to modify the file/directory** -docker run --detach --rm --interactive --tty \ - --name ${NAME} \ - --volume ${HOST_LOG_FILE}:/opt/iscsc.fr-notify-bot/${LOG_FILE} \ - ${NAME}:latest -``` -In this example LOG_FILE is a environment variable defined in the .env +## Other important use cases of volumes -> **WARNING : Be careful that the bot/application running in the container have the right permission to modify the file/directory** +* The same volume can be shared by multiple containers. + It can be useful if you want to share a database with applications running on different containers or to centralized logs of multiple application in the same log file/folder. +* In development you can use volumes to share code with the container (where it runs) while editing it in your IDE in the host. +* Volumes exist and can be created/managed outside of a container (contrary to bind mounts) with `docker volume` +* Volumes are entirely managed by Docker and are then OS-agnostic -## Other utility of bind mounts +## Resources to go further -* The same volume can be shared by multiple containers. It can be useful if you want to share a database with applications running on different containers or to centralized logs of multiple application in the same log file. -* In development you can use bind mount to enable code modification of the application without having to restart the container. \ No newline at end of file +- [Key differences between volumes and bind mounts](https://docs.docker.com/storage/#volumes) + - [Differences between `--volume` and `--mount`](https://docs.docker.com/storage/bind-mounts/#differences-between--v-and---mount-behavior) + - [Volumes use cases](https://docs.docker.com/storage/#good-use-cases-for-volumes) + - [Bind mounts use cases](https://docs.docker.com/storage/#good-use-cases-for-bind-mounts) +- [`docker volume` syntax](https://docs.docker.com/storage/volumes/#create-and-manage-volumes) +- [Volumes in Docker compose file](https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes) \ No newline at end of file From 9ff9c6474f45d0158cfe76218d6d932b68a02cd9 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Tue, 16 Apr 2024 22:03:42 +0200 Subject: [PATCH 06/14] fix typos --- src/content/posts/docker_volume_introduction.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index 000366f..2548b9c 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -23,7 +23,7 @@ A docker's volume is the action of **mounting an existing file or directory** of ## Why do we need volumes? -Docker containerization is very useful on lots of points for development but it's very annoying if your application **need to produce persistent data**, like logs. Indeed if you **kill** your container (to update it for example) or if it **crashes**, you **lose everything that where inside** and understanding why your application crashed without its logs can be very complicated or even impossible. +Docker containerization is very useful on lots of points for development but it's very annoying if your application **needs to produce persistent data**, like logs. Indeed if you **kill** your container (to update it for example) or if it **crashes**, you **lose everything that was inside** and understanding why your application crashed without its logs can be very complicated or even impossible. So we need to find a way to **keep certain files after the container's death** and that can be **achieved with volumes**!. @@ -62,7 +62,7 @@ A concrete example with the [code of our RootMe discord bot](https://github.com/ ${NAME}:latest ``` In this example `LOG_FOLDER` is an environment variable defined in the `.env.prod` file. -The code will written transparently to `/opt/${NAME}/logs/` but because this folder is shared with the host, logs will **be available in the host** and survive potential application crashes or updates. +The code will write transparently to `/opt/${NAME}/logs/` but because this folder is shared with the host, logs will **be available in the host** and survive potential application crashes or updates. > **WARNING : Be careful that the bot/application running in the container must have the right permission to modify the file/directory** From ec9834e62f5f0bf781529cf5c332dc80ade84cda Mon Sep 17 00:00:00 2001 From: ctmbl Date: Sat, 4 May 2024 19:07:16 +0200 Subject: [PATCH 07/14] Update article's title and summary --- src/content/posts/docker_volume_introduction.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index 2548b9c..6403f96 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -1,8 +1,8 @@ --- -title: "Use Docker's volumes to build a persistent log system" -summary: "Persist data after container's death or share data accross containers through Docker's volume feature" +title: "Use Docker's bind mounts to build a persistent log system" +summary: "Persist data after container's death and share data with the host or accross containers through Docker's bind mounts feature" date: 2023-09-19T12:00:00+0200 -lastUpdate: 2024-04-12T20:00:00+0200 +lastUpdate: 2024-05-03T20:00:00+0200 tags: ["docker", "volume", "log"] author: FireFrozen draft: false From 66970abb15c37ed9fb760c7930fe1467474f5f04 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Sat, 4 May 2024 19:09:04 +0200 Subject: [PATCH 08/14] BIND MOUNTS FIX: Replace volumes by bind mounts, place a lot of TODO --- .../posts/docker_volume_introduction.md | 51 ++++++++++--------- 1 file changed, 28 insertions(+), 23 deletions(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index 6403f96..4bab576 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -11,44 +11,47 @@ draft: false Before reading this article, it's recommended to have a minimum knowledge of `docker`. See, for example, the previous blog on docker: [Get Started with Docker](https://iscsc.fr/posts/short-docker-introduction/) by `ctmbl`. -> **Disclaimer**: -> Note that in Docker, **two ways exist to persist/share data** with the host: **volumes** and **bind mounts**. -> While some key differences exist between both, they are not relevant in most of our basic use cases. -> We'll then **not distinguish one from the other**, some of what we say might better applied to bind mounts but **we'll only speak of volumes**. -> For more information see [Manage data in Docker](https://docs.docker.com/storage/). -## What is a Docker's volume? -A docker's volume is the action of **mounting an existing file or directory** of the host (the computer running the container) **into a container**. To rephrase, it enables a container to **share a memory space with the host**. So both the container and the user can access and modify it in real time and because it's not "contained" in the container but only mounted, it'll **persist after the container's death**. +## A brief disclaimer -## Why do we need volumes? +Note that in Docker, **two ways exist to persist/share data** with the host: **volumes** and **bind mounts**. +While some key differences exist between both, they are not relevant in most of our basic use cases. +We'll then **not distinguish one from the other**, some of what we say might better applied to bind mounts but **we'll only speak of volumes**. +For more information see [Manage data in Docker](https://docs.docker.com/storage/). + +## What is a Docker's bind mount? + +A docker's bind mount is the action of **mounting an existing file or directory** of the host (the computer running the container) **into a container**. To rephrase, it enables a container to **share a memory space with the host**. So both the container and the user can access and modify it in real time and because it's not "contained" in the container but only mounted, it'll **persist after the container's death**. + +## Why do we need bind mounts? Docker containerization is very useful on lots of points for development but it's very annoying if your application **needs to produce persistent data**, like logs. Indeed if you **kill** your container (to update it for example) or if it **crashes**, you **lose everything that was inside** and understanding why your application crashed without its logs can be very complicated or even impossible. -So we need to find a way to **keep certain files after the container's death** and that can be **achieved with volumes**!. +So we need to find a way to **keep certain files after the container's death** and that can be **achieved with bind mounts**!. -## How to create a volume? +## How to create a bind mount? -To create a volume, you need to add a specific argument to the `docker run` command. +To create a bind mount, you need to add a specific argument to the `docker run` command *([similar syntax](https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes) exists for `docker-compose.yml`)*. -> You can also create volumes outside of containers with [`docker volume`](https://docs.docker.com/storage/volumes/#create-and-manage-volumes) +TO BE DELETED: > You can also create volumes outside of containers with [`docker volume`](https://docs.docker.com/storage/volumes/#create-and-manage-volumes) -There is two different arguments which are really similar in their use. Those are `--mount` and `--volume`, both can be used to create volumes (or bind mounts, see Disclaimer above). -However they slightly differ: if you enter a non existing file to mount, `--volume` will create it when `--mount` will return an error, also `--volume` needs fewer arguments. For these reason and because we focused on volumes in this post we'll describe the `--volume` argument. +There is two different arguments which are really similar in their use. Those are `--mount` and `--volume`, both can be used to create bind mounts (but volumes too, see [Disclaimer](#a-brief-disclaimer) above). +However they slightly differ: if you enter a non existing file to mount, `--volume` will create it when `--mount` will return an error, also `--volume` needs fewer arguments. For these reason and because we focused on simplicity in this post we'll describe the `--volume` argument. The exact syntax is: ```bash docker run --volume PATH_TO_HOST_FILE:PATH_TO_CONTAINER_FILE: ``` -> Note that you can use `-v` instead of `--volume`. +> Note that you can use `-v` instead of `--volume` -> Note that you can mount a directory instead of a file with the same method. +> Note that you can mount a directory instead of a file with the same method -> A similar syntax exists for [`docker-compose.yml`](https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes) +> A [similar syntax](https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes) exists for `docker-compose.yml` Optional args must be separated by comas, they are not used very often but lets detail the most useful one quickly : -- `ro` : if used, the mounted files/folders will be in read only mode inside the container +- `ro` : if used, the mounted file/folder will be in read-only mode inside the container A concrete example with the [code of our RootMe discord bot](https://github.com/iScsc/RootPythia/blob/2681ca26286ea5063371536e995a5e3cf39734a5/run.sh#L12): ```bash @@ -66,16 +69,18 @@ The code will write transparently to `/opt/${NAME}/logs/` but because > **WARNING : Be careful that the bot/application running in the container must have the right permission to modify the file/directory** -## Other important use cases of volumes +## Other important use cases of bind mounts + +> See [Bind mounts use cases](https://docs.docker.com/storage/#good-use-cases-for-bind-mounts) * The same volume can be shared by multiple containers. - It can be useful if you want to share a database with applications running on different containers or to centralized logs of multiple application in the same log file/folder. -* In development you can use volumes to share code with the container (where it runs) while editing it in your IDE in the host. -* Volumes exist and can be created/managed outside of a container (contrary to bind mounts) with `docker volume` -* Volumes are entirely managed by Docker and are then OS-agnostic + It can be useful if you want to share a database with applications running on different containers or to centralized logs of multiple application in the same log file/folder. +* In development you can use bind mounts to share code with the container (where it runs) while editing it in your IDE in the host. ## Resources to go further +TO BE REWORKED: + - [Key differences between volumes and bind mounts](https://docs.docker.com/storage/#volumes) - [Differences between `--volume` and `--mount`](https://docs.docker.com/storage/bind-mounts/#differences-between--v-and---mount-behavior) - [Volumes use cases](https://docs.docker.com/storage/#good-use-cases-for-volumes) From 13246e68e6b82435ab747b86e65b4f92adcb6729 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Sun, 5 May 2024 01:43:28 +0200 Subject: [PATCH 09/14] Update tags --- src/content/posts/docker_volume_introduction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index 4bab576..3784587 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -3,7 +3,7 @@ title: "Use Docker's bind mounts to build a persistent log system" summary: "Persist data after container's death and share data with the host or accross containers through Docker's bind mounts feature" date: 2023-09-19T12:00:00+0200 lastUpdate: 2024-05-03T20:00:00+0200 -tags: ["docker", "volume", "log"] +tags: ["docker", "volume", "bind mounts", "logs"] author: FireFrozen draft: false --- From 1a1c124c78d06b20e7d72101759b494753c50be9 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Sun, 5 May 2024 01:44:10 +0200 Subject: [PATCH 10/14] Rework the whole article formulation --- .../posts/docker_volume_introduction.md | 61 +++++++++++-------- 1 file changed, 34 insertions(+), 27 deletions(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index 3784587..75144a6 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -11,37 +11,52 @@ draft: false Before reading this article, it's recommended to have a minimum knowledge of `docker`. See, for example, the previous blog on docker: [Get Started with Docker](https://iscsc.fr/posts/short-docker-introduction/) by `ctmbl`. - +Also I **strongly** recommend you to read [the Disclaimer](#a-brief-disclaimer). ## A brief disclaimer -Note that in Docker, **two ways exist to persist/share data** with the host: **volumes** and **bind mounts**. -While some key differences exist between both, they are not relevant in most of our basic use cases. -We'll then **not distinguish one from the other**, some of what we say might better applied to bind mounts but **we'll only speak of volumes**. -For more information see [Manage data in Docker](https://docs.docker.com/storage/). +**TLDR**: +Volumes and bind mounts aren't the same thing. +We can **confuse `--volume` and `--mount`** BUT **we can't** confuse the **concepts of volumes and bind mounts**. +Volumes aren't only managed by `--volume`, bind mounts aren't only managed by `--mount`. + +This article addresses the concept of bind mounts. + +--- + +In Docker, two ways exist to persist/share data: **volumes** (named and anonymous) and **bind mounts**. + +What we're talking about in this blog aren't volumes at all, they are bind mounts. +We'll name them *bind mounts* **but be aware that** often on Internet you'll encounter the word *volume* while the author is actually speaking of a bind mount. +This is certainly due to the fact that **options `-volume` and `--mount` can be used almost interchangeably**. + +> Even in [docker's doc the naming is confusing](https://docs.docker.com/storage/volumes/#differences-between--v-and---mount-behavior) + +Key differences exist between bind mounts and volumes, for more information see [Manage data in Docker](https://docs.docker.com/storage/). ## What is a Docker's bind mount? -A docker's bind mount is the action of **mounting an existing file or directory** of the host (the computer running the container) **into a container**. To rephrase, it enables a container to **share a memory space with the host**. So both the container and the user can access and modify it in real time and because it's not "contained" in the container but only mounted, it'll **persist after the container's death**. +A docker's *bind mount* is the action of **mounting an existing file or directory** of the host (the computer running the container) **into a container**. To rephrase, it enables a container to **share a memory space with the host**. So both the container and the user can access and modify it in real time and because it's not "contained" in the container but only mounted, it'll **persist after the container's death**. ## Why do we need bind mounts? Docker containerization is very useful on lots of points for development but it's very annoying if your application **needs to produce persistent data**, like logs. Indeed if you **kill** your container (to update it for example) or if it **crashes**, you **lose everything that was inside** and understanding why your application crashed without its logs can be very complicated or even impossible. -So we need to find a way to **keep certain files after the container's death** and that can be **achieved with bind mounts**!. +So we need to find a way to **keep certain files after the container's death** and that can be **achieved with bind mounts**. + +It's also interesting to notice another really good usecase for *bind mounts*: during development you can share source code to execute it within the container while editing it in your IDE in the host ## How to create a bind mount? To create a bind mount, you need to add a specific argument to the `docker run` command *([similar syntax](https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes) exists for `docker-compose.yml`)*. -TO BE DELETED: > You can also create volumes outside of containers with [`docker volume`](https://docs.docker.com/storage/volumes/#create-and-manage-volumes) - There is two different arguments which are really similar in their use. Those are `--mount` and `--volume`, both can be used to create bind mounts (but volumes too, see [Disclaimer](#a-brief-disclaimer) above). -However they slightly differ: if you enter a non existing file to mount, `--volume` will create it when `--mount` will return an error, also `--volume` needs fewer arguments. For these reason and because we focused on simplicity in this post we'll describe the `--volume` argument. +However they slightly differ: if you enter a non existing file to mount, `--volume` will create it when `--mount` will return an error, also `--volume` needs fewer arguments. +For these reason and because we focus on simplicity in this post we'll describe the `--volume` argument. The exact syntax is: ```bash -docker run --volume PATH_TO_HOST_FILE:PATH_TO_CONTAINER_FILE: +docker run --volume path/in/host:path/in/container: ``` > Note that you can use `-v` instead of `--volume` @@ -55,28 +70,20 @@ Optional args must be separated by comas, they are not used very often but lets A concrete example with the [code of our RootMe discord bot](https://github.com/iScsc/RootPythia/blob/2681ca26286ea5063371536e995a5e3cf39734a5/run.sh#L12): ```bash - source ./.env.prod - - docker run --rm --interactive --tty \ - --detach \ - --volume $(realpath -P ${LOG_FOLDER}):/opt/${NAME}/logs \ - --env-file .env.prod \ - --name ${NAME} \ - ${NAME}:latest +source ./.env.prod + +docker run --rm --interactive --tty \ + --detach \ + --volume $(realpath -P ${LOG_FOLDER}):/opt/${NAME}/logs \ + --env-file .env.prod \ + --name ${NAME} \ + ${NAME}:latest ``` In this example `LOG_FOLDER` is an environment variable defined in the `.env.prod` file. The code will write transparently to `/opt/${NAME}/logs/` but because this folder is shared with the host, logs will **be available in the host** and survive potential application crashes or updates. > **WARNING : Be careful that the bot/application running in the container must have the right permission to modify the file/directory** -## Other important use cases of bind mounts - -> See [Bind mounts use cases](https://docs.docker.com/storage/#good-use-cases-for-bind-mounts) - -* The same volume can be shared by multiple containers. - It can be useful if you want to share a database with applications running on different containers or to centralized logs of multiple application in the same log file/folder. -* In development you can use bind mounts to share code with the container (where it runs) while editing it in your IDE in the host. - ## Resources to go further TO BE REWORKED: From f8229095b4efc77f33c82289d548bfcad7f38817 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Sun, 5 May 2024 01:44:24 +0200 Subject: [PATCH 11/14] Rework sources --- .../posts/docker_volume_introduction.md | 22 ++++++++++++------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index 75144a6..129e62d 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -86,11 +86,17 @@ The code will write transparently to `/opt/${NAME}/logs/` but because ## Resources to go further -TO BE REWORKED: - -- [Key differences between volumes and bind mounts](https://docs.docker.com/storage/#volumes) - - [Differences between `--volume` and `--mount`](https://docs.docker.com/storage/bind-mounts/#differences-between--v-and---mount-behavior) - - [Volumes use cases](https://docs.docker.com/storage/#good-use-cases-for-volumes) - - [Bind mounts use cases](https://docs.docker.com/storage/#good-use-cases-for-bind-mounts) -- [`docker volume` syntax](https://docs.docker.com/storage/volumes/#create-and-manage-volumes) -- [Volumes in Docker compose file](https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes) \ No newline at end of file +About *volumes* VS *bind mounts*: +- [Manage data in Docker](https://docs.docker.com/storage/) +- [Choose the right type of mount](https://docs.docker.com/storage/#choose-the-right-type-of-mount) + +About *bind mounts* specifically: +- [Start a container with a bind mount](https://docs.docker.com/storage/bind-mounts/#start-a-container-with-a-bind-mount) +- [Bind mounts use cases](https://docs.docker.com/storage/#good-use-cases-for-bind-mounts) + +Other sources: +- [Volumes in Docker compose file](https://docs.docker.com/compose/compose-file/compose-file-v3/#volumes) +- https://dev.to/doziestar/a-comprehensive-guide-to-docker-volumes-4d9h +- https://www.baeldung.com/ops/docker-volumes +- [Never mess with `/var/lib/docker` docker forum](https://forums.docker.com/t/write-from-host-to-volume/47274) +- [Volumes use cases](https://docs.docker.com/storage/#good-use-cases-for-volumes) From 0f9c7cab0b44159be4d3c5131aba0ad174cadb61 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Sun, 5 May 2024 01:44:38 +0200 Subject: [PATCH 12/14] Add myself as co-author --- src/content/posts/docker_volume_introduction.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index 129e62d..dbe816b 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -8,6 +8,8 @@ author: FireFrozen draft: false --- +*Co-Authored with [ctmbl](https://iscsc.fr/author/ctmbl/)* + Before reading this article, it's recommended to have a minimum knowledge of `docker`. See, for example, the previous blog on docker: [Get Started with Docker](https://iscsc.fr/posts/short-docker-introduction/) by `ctmbl`. From 5828f003b8384ffe5f1fcb99a9304c5109723c03 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Tue, 28 May 2024 08:27:37 +0200 Subject: [PATCH 13/14] Fix typo --- src/content/posts/docker_volume_introduction.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_volume_introduction.md index dbe816b..877a6c5 100644 --- a/src/content/posts/docker_volume_introduction.md +++ b/src/content/posts/docker_volume_introduction.md @@ -30,7 +30,7 @@ In Docker, two ways exist to persist/share data: **volumes** (named and anonymou What we're talking about in this blog aren't volumes at all, they are bind mounts. We'll name them *bind mounts* **but be aware that** often on Internet you'll encounter the word *volume* while the author is actually speaking of a bind mount. -This is certainly due to the fact that **options `-volume` and `--mount` can be used almost interchangeably**. +This is certainly due to the fact that **options `--volume` and `--mount` can be used almost interchangeably**. > Even in [docker's doc the naming is confusing](https://docs.docker.com/storage/volumes/#differences-between--v-and---mount-behavior) From 20bc9f8e4e1b9c0dd49f373300dc7fa1213e80a4 Mon Sep 17 00:00:00 2001 From: ctmbl Date: Wed, 29 May 2024 13:58:13 +0200 Subject: [PATCH 14/14] Rename file --- ..._volume_introduction.md => docker_bind_mounts_introduction.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename src/content/posts/{docker_volume_introduction.md => docker_bind_mounts_introduction.md} (100%) diff --git a/src/content/posts/docker_volume_introduction.md b/src/content/posts/docker_bind_mounts_introduction.md similarity index 100% rename from src/content/posts/docker_volume_introduction.md rename to src/content/posts/docker_bind_mounts_introduction.md