- python >= 3.8.*
- virutalenv or virtualenvwrapper
- docker == 20.10.18
- docker compose (plugin)
cd $HOME && git clone https://github.com/mnemonico/ovapi_data_extraction.git && cd ovapi_data_extraction && export WORKSPACE=\`pwd\`
virtualenv venv
ormkvirtualenv venv && workon venv
if you've virtualenvwrapperpip install -r requirements.txt
docker compose -f docker-compose.yml up -d
docker ps
(make sure containers are up)- to execute the api call and extarction with database load
python extract_code/dag_executor.py
- after the end of the previous execution, pass to the browser then enter the url
- http://0.0.0.0:8080/
- username: [email protected]
- password: SuperSecret
- once connected to pgadmin; connect to the local database instance
- username: postgres
- password: postgres
- database: postgres
- http://0.0.0.0:8080/
- check table
public.ods_line
onpostgres
database - to test the simple dag with (3 tasks) without any setup (without usage via webserver and scheduler up)
- cd $WORKSPACE && python extract_code/dag_executor.py