Skip to content

Development Guide

Jump to bottom
rkaufman13 edited this page Jun 30, 2022 · 73 revisions

Local Deployment with Scripts

The first time you pull from main after ethan/refreshtoken has been incorporated, rm flag.db and rm any files in migrations/versions before you pull from main. If you have already pulled from main do git status and remove any untracked files in migratons/versions.

Step 0 - Install Postgres and Python/Conda

  • Install Postgres
  • Install Python or Conda
  • Install sqlite

Step 1 - git clone and/or checkout branch

  • Clone this repository
  • Checkout a new branch if you are making changes

Step 2 - Configure .env

Step 2a - Configure .env for sqlite

  • If you have customized .env look to see if there are any changes to .env.example.sqlite

  • If you have not customized .env then copy .env.example to .env

    cp .env.example.sqlite .env

Step 2b - Configure .env for Postgres

  • If you have customized .env look to see if there are any changes to .env.example.postgres

  • If you have not customized .env then copy .env.example to .env

    cp .env.example.postgres .env

  • If you have not initialized the postgres database or want to recreate db:

    psql postgres_setup.sql

Step 3b - Configure frontend .env for the db

Step 3 - Set Permissions of Shell Scripts

From the Mac or git bash terminal:

chmod -R 777 *.sh

Step 4 - Run Setup Script

source run_setup.sh

Manual Steps

To do the same without the scripts:

======================
source myenv/bin/activate
python3 -m venv myenv

        OR

conda create --name myenv
conda activate myenv
=======================
pip install -r requirements.txt
DEBUG=True FLASK_APP=app.py flask db upgrade
python3 populate_seed_data.py
rm flag.db ## required if any schema changes
FLASK_APP=app.py FLASK_ENV=development flask db migrate
FLASK_APP=app.py FLASK_ENV=development flask db upgrade
FLASK_APP=app.py FLASK_ENV=development flask db migrate

Logging In

Accounts are auto populated with default passwords. Accounts for federal offices include their accounts include FED-HOSS, FED-ADMIN, FED-AOC, FED-MAIL, FED-HOSS-ADMIN, FED-AOC-ADMIN, and FED-MAIL-ADMIN. Accounts for state offices are in the form NY-02 and NY-02-ADMIN. The password is the username plus -1010 (e.g., NY-02-1010). See security for what these accounts are authorized to do.

Migrating/Making Model Changes

After making changes to one or more database models you have two optoins:

  1. Regenerate Schema from Scratch This is the preferred method until the application goes into development so that all schema is defined in a single file.
source setup/init_schema.sh
  1. Create Schema Upgrade Scripts Once the application is production you need to create schema upgrade scripts that upgrade the schema from the previous model to the current model. You may also need to write scripts to populate the new columns, unless you can provide default values for the columns or make them optional. Follow these steps
  • source db_schema.sh
  • if required:
    • create scripts to populate new columns
    • look in migrations/versions directory for most recent python script
    • add call to those scripts from that script
    • if you renamed or dropped tables or columns you may need to adjust the script to do this correctly

Security

When fully implemented:

  • Only FED accounts can add orders and view and update all orders regardless of the associated home office
  • State office accounts can view and update orders associated with their home office
  • Only accounts ending in ADMIN can change passwords for users associated with their home office, e.g. NY-02-ADMIN can change passwords for NY-02 and NY-02-ADMIN
  • Accounts not ending in ADMIN cannot change passwords
  • FED-ADMIN is unrestricted from doing any task. As of when this is written, the only task that is restricted to just FED-ADMIN is maintaining users.

Running Tests

pytest -s --verbose

Test Coverage

 pytest --cov --cov-report html:coverage
 open coverage/index.html

Viewing Data

pip install datasette first if needed, then datasette flag.db

click the link (typically port 8001 on localhost)

or run sqlite

Heroku

Heroku Account Setup

  • Install Heroku if not done
  • If you do not have a heroku account, create an account using heroku login
  • Request permission to push to heroku repository

Using Existing Heroku Environment

Below instructions assume you are using codefordc-flag repository and app. If you are using a different repository and app, substitute codefordc-flag with the appropriate name.

heroku login
heroku git:remote -a codefordc-flag
heroku config:set FLASK_APP=app.py FLASK_ENV=development

Pushing Changes in Main branch to Heroku

Follow instructions above to use existing Heroku environment if not done.

git checkout main
git pull origin main
git push heroku main

Resetting and Repopulating Heroku DB

Follow instructions above to use existing Heroku environment if not done.

heroku pg:reset
heroku run flask db upgrade
heroku run python populate_seed_data.py

Using Heroku db with Netlify app

URL for current Netlify app is https://codefordc-flag.netlify.app/

Creating Heroku Environment First Time

This is for first time installation if you need to rebuild or create a new Heroku environment different from the existing codefordc-flag. If you want to use codefordc-flag, see Configuring Heroku Environment

Step 1 - Create Heroku Repository / App

heroku create

=====================

Heroku will create a repository for you with a random name. The app is the same as the repository without the .git. You can rename it. For the shared dev heroku, the name is codefordc-flag

======================

Step #2 For , use the app created in step 1.

heroku git:remote -a <app name>
heroku config:set FLASK_APP=app.py FLASK_ENV=development
heroku addons:create heroku-postgresql:hobby-dev --app <app name>

Step 2 - push to heroku

git checkout main
git pull origin main
git push heroku main

Step 3 - Remaining Steps

heroku run flask db upgrade
heroku run python populate_seed_data.py

Step 4 (OPTIONAL) - Fix DATABASE_URL

DATABASE_URL in Heroku defaults to start with postgres: but it should start with postgresql to work with Flask. There is code that will make the change when running the app, but it would be better if Heroku had the right values. I (Ethan) was able to get the change to stick once but after trying again, I ran into various problems, so below instructions will need to change.

  • Check DATABASE_URL created in Heroku (from heroku.com, navigate to App, click on Settings gear, click on Reveal Config Vars)
  • If DATABASE_URL starts with postgres: follow the below steps to change it to postgresql: otherwise continue to step 3
  • Copy and paste the DATABASE_URL from Heroku
  • will be the same URL but replace postgres: with postgresql:
  • Open terminal
# I get error with below command
heroku addons:detach DATABASE --app codefordc-flag
heroku config:add DATABASE_URL=<new URL>
heroku restart
heroku pg:reset

Other potentially useful commands:

heroku addons:add heroku-postgresql:hobby-dev DATABASE
heroku addons:attach heroku-postgresql:hobby-dev --app codefordc-flag
# useful if you accidentally create a second DB that does not use DATABASE_URL
heroku addons:destroy HEROKU_POSTGRESQL_NAVY 




# Guidelines
## Proposed
Backend
- When returning an error, use format  {"message": "<error message>", <error_number>).  For example, ???.  See ???.
- Use a parameter class for passing info
- Use table_record_to_json or table_to_json to convert from sqlalchemy to json in util.py
- Convert in the controller, not the actions
- If a function requires three or  more parameters, create a class to hold those params.  See UserParams
as an example.  This simplifies a bunch of code so you don't need to repeat.



## Accepted
Clone this wiki locally