# required for psycopg2 (postgres)
sudo apt-get update -y && sudo apt-get install -y libpq-dev python3-devSupport for vscode is provided, with a sensible configuration.
A docker-compose.yml file is provided for development.
# Copy the template .env
cp .template.env .env
# Edit the content of .env file to your needs
# It should work as is, except for the `AIRFLOW_UID=` variable that must be set
# to the value returned by `echo $(id -u)`
# Start the stack
docker compose up -dAfter a few seconds, the services should be avaible as follow:
| default location | default credentials | |
|---|---|---|
| airflow UI | http://localhost:8080 | user: airflow pass: airflow |
| data.inclusion | http://localhost:8000 | token must be generated |
This is optional but allows you to interact with the datalake locally from the command line.
Cf DEPLOYMENT.md if you also wich to interact with staging and prod bucket.
See installation instructions here.
Project uses python3.11
pyenv is a clean and easy way to manage multiple python versions on your computer. Installation instructions are available here.
pipx install pre-commit# Clone this repository
git clone [email protected]:gip-inclusion/data-inclusion.git
# Setup code quality tools
pre-commit install
# Open workspace in vscode
code -n data-inclusionEach subdirectory (./pipeline, ./api, etc.) has its own contributing guidelines on how to setup an environment for development.
Issues and PRs are welcome.