debt: clean up python dependency management

[shrouxm]

Description

Currently we manage our python dependencies in a somewhat adhoc way:

• there are .in files (base.in, deploy.in, and dev.in) which declare our direct dependencies without associated versions
• there's a Makefile with pip-compile commands that generate requirements.txt and requirements-dev.txt with pinned versions of our direct and transitive dependencies
• when dependabot runs, it updates requirements.txt and requirements-dev.txt, which are essentially our lock files, but not the .in files, which are our actual dependency specifications

This has the following problems:

• make lock and make lock-dev, which generate our lockfiles, are not deterministic (!!)
• there is no easy way to view which versions of our direct dependencies we have pinned

I don't have any experience with the python dependency management ecosystem, so I'm not sure if better solutions exist or what the effort might be to adopt them.

It's possible that just manually copying over the current pinned versions of our direct dependencies from the requirements files to the .in files might work, but I'm not sure if dependabot will then start updating the .in files automatically or if it'd break that part of our workflow.