- hydra: Manage configuration files - article
- pre-commit plugins: Automate code reviewing formatting - article
- DVC: Data version control - article
- pdoc: Automatically create an API documentation for your project
To set up the environment run:
make activate
make setup
To start the application run:
make run_app
To run a python script from the virtual environment run:
make run_python script="<name_of_script>"
To install new PyPI packages, run:
pip install <package-name>
To run the entire pipeline, type:
dvc repo
Read this article on how to use DVC to version your data.
Basically, you start with setting up a remote storage. The remote storage is where your data is stored. You can store your data on DagsHub, Google Drive, Amazon S3, Azure Blob Storage, Google Cloud Storage, Aliyun OSS, SSH, HDFS, and HTTP.
dvc remote add -d remote <REMOTE-URL>
Commit the config file:
git commit .dvc/config -m "Configure remote storage"
Push the data to remote storage:
dvc push
Add and push all changes to Git:
git add .
git commit -m 'commit-message'
git push origin <branch>
To auto-generate API document for your project, run:
make docs