Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dev-containers and their environment #33

Merged
merged 10 commits into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM python:3.11-slim
Copy link
Contributor

@MBerguer MBerguer Nov 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw in the other PR @marianogigli is trying to update it to 3.13 https://github.com/xmartlabs/python-template/pull/32/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557L1

did you have a chance to try it here, too?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can include the upgrade for python in this PR and check if everything works as expected.
What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, sounds good


RUN apt-get update && apt-get install -y curl libpq-dev gcc git

ENV POETRY_HOME="/usr/local" \
POETRY_NO_INTERACTION=1 \
POETRY_VIRTUALENVS_CREATE=false \
POETRY_VERSION=1.8.3

RUN curl -sSL https://install.python-poetry.org | python3 -
5 changes: 5 additions & 0 deletions .devcontainer/aliases-devcontainer
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
alias format='scripts/format.sh'
alias makemigrations='scripts/makemigrations.sh'
alias migrate='scripts/migrate.sh'
alias shell='python src/helpers/shell.py'
alias test='pytest src'
8 changes: 8 additions & 0 deletions .devcontainer/devcontainer-start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/bash


# Run start commads such as poetry install to keep dependecies updates and in sync with your lock file.

set -xeo pipefail

poetry install --no-ansi --no-root
69 changes: 69 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
{
"name": "Python Template",
"dockerComposeFile": "docker-compose.yaml",
"service": "devcontainer",
"runServices": [
"devcontainer",
"postgres"
],
"workspaceFolder": "/workspace",
"containerEnv": {
"PYTHONPATH": "/workspace",
"USER": "${localEnv:USER}",
"DATABASE_URL": "postgresql://dev:dev@postgres:5432/dev",
"LOG_LEVEL": "DEBUG",
"SERVER_URL": "example.com",
"ACCESS_TOKEN_EXPIRE_MINUTES": "15",
"JWT_SIGNING_KEY": ""
},
"customizations": {
"vscode": {
"extensions": [
"ms-azuretools.vscode-docker",
"ms-python.black-formatter",
"ms-python.flake8",
"ms-python.isort",
"ms-python.python",
"ms-python.mypy-type-checker"
],
"settings": {
"editor.formatOnSave": true,
"editor.rulers": [
120
],
"editor.tabSize": 4,
"files.insertFinalNewline": true,
"files.trimFinalNewlines": true,
"files.trimTrailingWhitespace": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
},
"python.editor.defaultFormatter": "ms-python.black-formatter"
}
},
"black-formatter.args": [
"--line-length=120"
],
"unwantedRecommendations": []
},
"postCreateCommand": "cat /workspace/.devcontainer/aliases-devcontainer >> ~/.bashrc",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

"postStartCommand": "/workspace/.devcontainer/devcontainer-start.sh",
"forwardPorts": [
// Backend API:
// localhost:8000
"devcontainer:8000",
// Postgres:
// localhost:5432 for accessing postgres via local dbeaver/psql client
"postgres:5432"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"forwardPorts": [
// Backend API:
// localhost:8000
"devcontainer:8000",
// Postgres:
// localhost:5432 for accessing postgres via local dbeaver/psql client
"postgres:5432"
"forwardPorts": [
// Backend API:
// localhost:8000 for accessing on your host
"devcontainer:8000",
// Postgres:
// localhost:5432 for accessing postgres via local dbeaver/psql client
"postgres:5432"

],
"portsAttributes": {
"5432": {
"label": "Postgres",
"onAutoForward": "notify"
},
"8000": {
"label": "Backend API",
"onAutoForward": "notify"
}
}
}
26 changes: 26 additions & 0 deletions .devcontainer/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
services:
devcontainer:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about include the env_file attribute in case someone whats to add new variables in their project?

Copy link
Contributor Author

@gastonsalgueiro gastonsalgueiro Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the env vars using the property containerEnv of the devcontainers.json file. New ones can be added there, or we can use "runArgs": ["--env-file",".devcontainer/env_file.env"] to add them from a file (or in the docker-compose), or we can do both to have an example. What do you prefer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest defining it here as more people will know where to look for it. Also one question, is env variables interpolation available if you define it in the json file? That may be another reason for having it in an .env file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

envfile added. If the same variable is defined in both the .env file and the .json file, for example, the devcontainer will take the value from the json file

command: sleep infinity
build: .
ports:
- '8000:8000'
volumes:
- source: ..
target: /workspace
type: bind
jrg091 marked this conversation as resolved.
Show resolved Hide resolved

postgres:
image: postgres:16
pull_policy: always
restart: unless-stopped
environment:
POSTGRES_USER: dev
POSTGRES_DB: dev
POSTGRES_PASSWORD: dev
volumes:
- source: postgres
target: /var/lib/postgresql/data
type: volume

volumes:
postgres: {}
16 changes: 0 additions & 16 deletions .dockerignore
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn´t we include the .dockerignore file in the .devcontainer file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the context of devcontainers, I'm not sure if it's worth it because we're mounting the entire repo as a volume, and we're not doing any COPY or anything similar. Maybe when we add other Dockerfiles it will be worth it, but I'm not sure if I explained myself well or what do you think?

This file was deleted.

5 changes: 0 additions & 5 deletions .env.example

This file was deleted.

25 changes: 0 additions & 25 deletions Dockerfile

This file was deleted.

35 changes: 24 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,24 @@
- PostgreSQL database

## Project setup
You only need to install [Docker](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/).
To start the containers, just run `docker-compose up` (or `docker-compose up -d` if you want to run the containers in background); or `docker-compose create` and `docker-compose start` if you don't want to see the logs.
Once the containers are running, you can go to `http://localhost:8000/docs` to see the automatic interactive API documentation.

The only things you need are [Docker](https://docs.docker.com/engine/install/), [Docker Compose](https://docs.docker.com/compose/install/), and [Visual Studio Code](https://code.visualstudio.com/download). Once you open the template with VS Code, it will recommend that you install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) if you don’t have it already. Then, a pop-up will appear to reopen the template in the devcontainer, or you can use `Ctrl / Cmd + shift + P` -> `Dev Containers: Open Folder in Container…`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The only things you need are [Docker](https://docs.docker.com/engine/install/), [Docker Compose](https://docs.docker.com/compose/install/), and [Visual Studio Code](https://code.visualstudio.com/download). Once you open the template with VS Code, it will recommend that you install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) if you don’t have it already. Then, a pop-up will appear to reopen the template in the devcontainer, or you can use `Ctrl / Cmd + shift + P` -> `Dev Containers: Open Folder in Container…`.
The only things you need are [Docker](https://docs.docker.com/engine/install/), [Docker Compose](https://docs.docker.com/compose/install/), and a code editor with devcontainer support like [Visual Studio Code](https://code.visualstudio.com/download). Once you open the template with VS Code, it will recommend that you install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) if you don’t have it already. Then, a pop-up will appear to reopen the template in the devcontainer, or you can use `Ctrl / Cmd + shift + P` -> `Dev Containers: Open Folder in Container…`.


And that's it, everything is ready to use. By using the VS Code terminal with `Ctrl / Cmd + J`, you'll be inside the container to run any command or start the server with `uvicorn src.main:app --reload --host 0.0.0.0 --port 8000`.

You can connect to the container once it's running using `scripts/exec.sh bash` and execute commands from another terminal outside of VS Code. Also, for all the following commands that can be run inside the dev container, you can run `scripts/exec.sh {command}` from outside the container.

Once the containers and server are running, you can go to `http://localhost:8000/docs` to see the automatic interactive API documentation.

In case you don't to use VS Code and dev containers, or you want to set up the environment in a different way. You must have `Python >3.11`, [Poetry](https://python-poetry.org/docs/#installation) (don't forget to install the dependencies from the lock file), and a [PostgreSQL](https://www.postgresql.org/) database, setting the corresponding environment variables for the database connection.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit.... lists help me to follow steps 😅

Suggested change
In case you don't to use VS Code and dev containers, or you want to set up the environment in a different way. You must have `Python >3.11`, [Poetry](https://python-poetry.org/docs/#installation) (don't forget to install the dependencies from the lock file), and a [PostgreSQL](https://www.postgresql.org/) database, setting the corresponding environment variables for the database connection.
In case you don't to use VS Code and dev containers, or you want to set up the environment in a different way. You must have
- `Python >3.11`
- [Poetry](https://python-poetry.org/docs/#installation) (don't forget to install the dependencies from the lock file)
- [PostgreSQL](https://www.postgresql.org/) database, setting the corresponding environment variables for the database connection.


## Migrations
We use Alembic as database migration tool. To run its commands you can open an interactive shell inside the backend container (you can use `./exec.sh bash` shortcut), or use the following shortcuts under the `/scripts` directory:
- `./exec.sh migrate` -> runs all the migrations
- `./exec.sh makemigrations` -> compares the actual status of the DB against the table metadata, and generates the migrations based on the comparison

We use Alembic as database migration tool. You can run migration commands directly inside the dev container or use the provided shortcut in the `exec.sh` script.

- `migrate` – Runs all migrations.
- `makemigrations` – Compares the current database state with the table metadata and generates the necessary migration files.


## Code tools
Linters, formatters, etc.
Expand All @@ -27,15 +37,20 @@ Linters, formatters, etc.
- **mypy**: Static type checker
- **black**: PEP 8 compliant opinionated formatter

There is a shortcut under the `/scripts` directory that runs all this tools for you (`./exec.sh format`).
There is a shortcut under the `/scripts` directory that runs all this tools for you (`./exec.sh format`) or just run `format` inside the dev container.

![Screenshot](.docs/images/format.png)

## Tests
We use FastAPI's `TestClient` and `pytest` for testing. `./exec.sh test` shortcut can be used to run all tests.
We use FastAPI's `TestClient` and `pytest` for testing. `./exec.sh test` shortcut can be used to run all tests or just `test` inside the dev container.

## Shell
There is a shortcut under the `/scripts` directory that opens a python interactive shell inside the docker container. It's `./exec.sh shell` and has some useful pre-imported stuff:
You can start an interactive Python shell inside the dev container in two ways:

1. Simply run `shell` inside the container.
2. Alternatively, use the `exec.sh shell` command.

The shell provides some useful pre-imported stuff:

- `session`: A SQLAlchemy `Session` object
- `settings`: An instance of the app settings class
Expand All @@ -49,5 +64,3 @@ The template includes an admin interface via [SQLAdmin](https://github.com/amina
*One note: You should be careful when adding relationships to the list or detail pages (specially large many-to-many / one-to-many relationships), because it's not very optimal in terms of DB querys in those cases (all the related objects would be loaded in memory).*

![Screenshot](.docs/images/admin.png)


11 changes: 0 additions & 11 deletions docker-compose.override.yaml

This file was deleted.

12 changes: 0 additions & 12 deletions docker-compose.test.yaml

This file was deleted.

18 changes: 0 additions & 18 deletions docker-compose.yaml

This file was deleted.

Loading