Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dev-containers and their environment #33

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .env.example → .devcontainer/.env.example
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
DATABASE_URL=postgresql://dev:dev@db:5432/dev
DATABASE_URL=postgresql://dev:dev@postgres:5432/dev
LOG_LEVEL=DEBUG
SERVER_URL=example.com
ACCESS_TOKEN_EXPIRE_MINUTES=15
JWT_SIGNING_KEY=
JWT_SIGNING_KEY=
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
JWT_SIGNING_KEY=
JWT_SIGNING_KEY=

10 changes: 10 additions & 0 deletions .devcontainer/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM python:3.13-slim

RUN apt-get update && apt-get install -y curl libpq-dev gcc git

ENV POETRY_HOME="/usr/local" \
POETRY_NO_INTERACTION=1 \
POETRY_VIRTUALENVS_CREATE=false \
POETRY_VERSION=1.8.3

RUN curl -sSL https://install.python-poetry.org | python3 -
5 changes: 5 additions & 0 deletions .devcontainer/aliases-devcontainer
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
alias format='scripts/format.sh'
alias makemigrations='scripts/makemigrations.sh'
alias migrate='scripts/migrate.sh'
alias shell='python src/helpers/shell.py'
alias test='pytest src'
8 changes: 8 additions & 0 deletions .devcontainer/devcontainer-start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/usr/bin/bash


# Run start commads such as poetry install to keep dependecies updates and in sync with your lock file.

set -xeo pipefail

poetry install --no-ansi --no-root
64 changes: 64 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"name": "Python Template",
"dockerComposeFile": "docker-compose.yaml",
"service": "devcontainer",
"runServices": [
"devcontainer",
"postgres"
],
"workspaceFolder": "/workspace",
"containerEnv": {
"PYTHONPATH": "/workspace",
"USER": "${localEnv:USER}"
},
"customizations": {
"vscode": {
"extensions": [
"ms-azuretools.vscode-docker",
"ms-python.black-formatter",
"ms-python.flake8",
"ms-python.isort",
"ms-python.python",
"ms-python.mypy-type-checker"
],
"settings": {
"editor.formatOnSave": true,
"editor.rulers": [
120
],
"editor.tabSize": 4,
"files.insertFinalNewline": true,
"files.trimFinalNewlines": true,
"files.trimTrailingWhitespace": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
},
"python.editor.defaultFormatter": "ms-python.black-formatter"
}
},
"black-formatter.args": [
"--line-length=120"
],
"unwantedRecommendations": []
},
"postCreateCommand": "cat /workspace/.devcontainer/aliases-devcontainer >> ~/.bashrc",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

"postStartCommand": "/workspace/.devcontainer/devcontainer-start.sh",
"forwardPorts": [
// Backend API:
// localhost:8000
"devcontainer:8000",
// Postgres:
// localhost:5432 for accessing postgres via local dbeaver/psql client
"postgres:5432"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"forwardPorts": [
// Backend API:
// localhost:8000
"devcontainer:8000",
// Postgres:
// localhost:5432 for accessing postgres via local dbeaver/psql client
"postgres:5432"
"forwardPorts": [
// Backend API:
// localhost:8000 for accessing on your host
"devcontainer:8000",
// Postgres:
// localhost:5432 for accessing postgres via local dbeaver/psql client
"postgres:5432"

],
"portsAttributes": {
"5432": {
"label": "Postgres",
"onAutoForward": "notify"
},
"8000": {
"label": "Backend API",
"onAutoForward": "notify"
}
}
}
31 changes: 31 additions & 0 deletions .devcontainer/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
services:
devcontainer:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about include the env_file attribute in case someone whats to add new variables in their project?

Copy link
Author

@gastonsalgueiro gastonsalgueiro Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the env vars using the property containerEnv of the devcontainers.json file. New ones can be added there, or we can use "runArgs": ["--env-file",".devcontainer/env_file.env"] to add them from a file (or in the docker-compose), or we can do both to have an example. What do you prefer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest defining it here as more people will know where to look for it. Also one question, is env variables interpolation available if you define it in the json file? That may be another reason for having it in an .env file

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

envfile added. If the same variable is defined in both the .env file and the .json file, for example, the devcontainer will take the value from the json file

command: sleep infinity
build: .
ports:
- '8000:8000'
volumes:
- source: ..
target: /workspace
type: bind
jrg091 marked this conversation as resolved.
Show resolved Hide resolved
- source: cache
target: /root/.cache
type: volume
env_file: .env

postgres:
image: postgres:16
pull_policy: always
restart: unless-stopped
environment:
POSTGRES_USER: dev
POSTGRES_DB: dev
POSTGRES_PASSWORD: dev
volumes:
- source: postgres
target: /var/lib/postgresql/data
type: volume

volumes:
postgres: {}
cache: {}
Empty file modified .dockerignore
100755 → 100644
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn´t we include the .dockerignore file in the .devcontainer file?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the context of devcontainers, I'm not sure if it's worth it because we're mounting the entire repo as a volume, and we're not doing any COPY or anything similar. Maybe when we add other Dockerfiles it will be worth it, but I'm not sure if I explained myself well or what do you think?

Empty file.
Empty file modified Dockerfile
100755 → 100644
Empty file.
44 changes: 32 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Xmartlabs Python Template

![python version](https://img.shields.io/badge/python-3.11-brightgreen)
![python version](https://img.shields.io/badge/python-3.13-brightgreen)
![fastAPI version](https://img.shields.io/badge/fastapi-0.95.2-brightgreen)


Expand All @@ -9,14 +9,31 @@
- PostgreSQL database

## Project setup
You only need to install [Docker](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/).
To start the containers, just run `docker-compose up` (or `docker-compose up -d` if you want to run the containers in background); or `docker-compose create` and `docker-compose start` if you don't want to see the logs.
Once the containers are running, you can go to `http://localhost:8000/docs` to see the automatic interactive API documentation.

The only things you need are [Docker](https://docs.docker.com/engine/install/), [Docker Compose](https://docs.docker.com/compose/install/), and a code editor with devcontainer support like [Visual Studio Code](https://code.visualstudio.com/download). Once you open the template with VS Code, it will recommend that you install the [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) if you don’t have it already. Then, a pop-up will appear to reopen the template in the devcontainer, or you can use `Ctrl / Cmd + shift + P` -> `Dev Containers: Open Folder in Container…`. Remember to add the `.env` file; you can use `.env.example` as a reference.

And that's it, everything is ready to use. By using the VS Code terminal with `Ctrl / Cmd + J`, you'll be inside the container to run any command or start the server with `uvicorn src.main:app --reload --host 0.0.0.0 --port 8000`.

You can connect to the container once it's running using `scripts/exec.sh bash` and execute commands from another terminal outside of VS Code. Also, for all the following commands that can be run inside the dev container, you can run `scripts/exec.sh {command}` from outside the container.

Once the containers and server are running, you can go to `http://localhost:8000/docs` to see the automatic interactive API documentation.

In case you don't to use VS Code and dev containers, or you want to set up the environment in a different way. You can use the `Dockerfile` in the root of the repository to create the image with everything needed to run the project. The `docker-compose.yaml` and `.env.example` files in the `.devcontainer` folder serve as references for recreating other services like the database. Also, you will need to run the `poetry install --no-ansi --no-root` command manually to install all the required dependencies.

Alternatively, you must have:

- `Python >3.13`
- [Poetry](https://python-poetry.org/docs/#installation) (don't forget to install the dependencies from the lock file)
- [PostgreSQL](https://www.postgresql.org/) database, setting the corresponding environment variables for the database connection.


## Migrations
We use Alembic as database migration tool. To run its commands you can open an interactive shell inside the backend container (you can use `./exec.sh bash` shortcut), or use the following shortcuts under the `/scripts` directory:
- `./exec.sh migrate` -> runs all the migrations
- `./exec.sh makemigrations` -> compares the actual status of the DB against the table metadata, and generates the migrations based on the comparison

We use Alembic as database migration tool. You can run migration commands directly inside the dev container or use the provided shortcut in the `exec.sh` script.

- `migrate` – Runs all migrations.
- `makemigrations` – Compares the current database state with the table metadata and generates the necessary migration files.


## Code tools
Linters, formatters, etc.
Expand All @@ -27,15 +44,20 @@ Linters, formatters, etc.
- **mypy**: Static type checker
- **black**: PEP 8 compliant opinionated formatter

There is a shortcut under the `/scripts` directory that runs all this tools for you (`./exec.sh format`).
There is a shortcut under the `/scripts` directory that runs all this tools for you (`./exec.sh format`) or just run `format` inside the dev container.

![Screenshot](.docs/images/format.png)

## Tests
We use FastAPI's `TestClient` and `pytest` for testing. `./exec.sh test` shortcut can be used to run all tests.
We use FastAPI's `TestClient` and `pytest` for testing. `./exec.sh test` shortcut can be used to run all tests or just `test` inside the dev container.

## Shell
There is a shortcut under the `/scripts` directory that opens a python interactive shell inside the docker container. It's `./exec.sh shell` and has some useful pre-imported stuff:
You can start an interactive Python shell inside the dev container in two ways:

1. Simply run `shell` inside the container.
2. Alternatively, use the `exec.sh shell` command.

The shell provides some useful pre-imported stuff:

- `session`: A SQLAlchemy `Session` object
- `settings`: An instance of the app settings class
Expand All @@ -49,5 +71,3 @@ The template includes an admin interface via [SQLAdmin](https://github.com/amina
*One note: You should be careful when adding relationships to the list or detail pages (specially large many-to-many / one-to-many relationships), because it's not very optimal in terms of DB querys in those cases (all the related objects would be loaded in memory).*

![Screenshot](.docs/images/admin.png)


11 changes: 0 additions & 11 deletions docker-compose.override.yaml

This file was deleted.

12 changes: 0 additions & 12 deletions docker-compose.test.yaml

This file was deleted.

18 changes: 0 additions & 18 deletions docker-compose.yaml

This file was deleted.

Loading