Dependency Updates

This wiki page describes how to update the JWQL repository with dependency updates provided by pyup.

On the first day of each month, the pyup tool automatically opens up a new pull request in the jwql repository with suggested dependency updates. Since pyup only works on requirements.txt files and not conda environment files (see the currently open GitHub issue about this), someone must manually go into that branch/pull request and update the appropriate environment files. The files of interest are:

• environment_python_3_7.yml
• environment_python_3_8.yml
• requirements.txt

Ultimately, the goal of this exercise is to (1) make sure the requirements.txt and environment_python_<version>.yml files are all in sync with each other (i.e. all have the same pinned version numbers) and (2) make sure the version numbers reflect that of the latest version available in conda.

Below is a step-by-step procedure for processing dependencies:

Find the pyup pull request and add appropriate assignees and labels.

The pull request will be named something like "Scheduled monthly dependency update for <month>". Open up the pull request on GitHub. Add yourself as an assignee. Add the GitHub label for "environment".

Check out the pyup branch to your local clone

The pyup tool opens the PR on a branch called pyup/scheduled-update-YYYY-MM-DD. Grab this branch in your local clone of the jwql repository with:

• git fetch upstream (This assumes you have a remote called upstream pointing to https://github.com/spacetelescope/jwql.git)
• git checkout -b pyup/scheduled-update-YYYY-MM-DD upstream/pyup/scheduled-update-YYYY-MM-DD

Check which dependency updates are actually updated in conda, update the conda environment files/requirements.txt file as necessary

Some updates that pyup suggests may not yet be available in conda yet, and thus should not be updated in the environment/requirements files. To check this, initialize the jwql-3.7 conda environment (this assumes you have done something like conda env create --name=jwql-3.7 --file=environment_python_3_7.yml), and try to conda install <dependency> for each suggested dependency. If conda does indeed install an update, make the appropriate update to the corresponding environment_python_3_7.yml file. If conda cannot find the updated dependency, do not make any changes to the conda environment files, and revert the dependency in the requirements.txt file back to what it was before.

Repeat this process for the Python 3.8 environment too.

Commit the changes to the environment/requirement files, push back to upstream

Once you have finished identifying all of the dependency updates and have updated the conda environment files and requirements.txt file, commit those changes to the branch, and push the branch back to upstream using e.g.:

• git push upstream pyup/scheduled-update-YYYY-MM-DD

  This will automatically update the pull request.

Ensure the GitHub Actions builds pass

Updating the pull request will trigger new GitHub action builds. Make sure that the changes you (and pyup) have made to the environment/requirement files result in a successful build of the GitHub Actions CI workflow. If the build fails, make any appropriate changes to the files until they pass.

Assign a reviewer and (eventually) merge

Once the GitHub Actions builds pass, the PR is ready to be reviewed. Assign a reviewer. Once the reviewer has approved, or you have iterated with the reviewer with any necessary changes, the PR can be merged.