Add logging to API

[anth-volk]

Fixes #1947.

This PR adds logging to both the API and attached workers, as well as testing to the base Logger class (but not yet WorkerLogger). I'm opening this in draft because there are still a few drawbacks (below) that I'm stuck on, and I'd love any assistance the requested reviewers might have. This is very much a work in progress, and as I've delved into some of the complexities of logging, I realize that there are a lot of considerations this code may not be handling, even at a design level. Any and all feedback is welcomed.

Drawbacks:

• When running Flask in debug mode, Flask starts a server process, triggering the start of the logger. However, it then uses this process to start a child process also running the API, but with Werkzeug debugging tools, creating yet another log file. It's this child process that will ultimately be in use. I've tried checking for particular env variables and only launching when the right combo is present, but this feels hackish and has negative ramifications for the worker. I've also tried using the Flask before_request feature, but that's not a solution.
• I was unable to properly launch memory usage logging. My preferred implementation merely copied what I had done previously in Add logging to worker #1948, but this either crashed (if I used a weak ref to store the logger) or improperly logged memory stats for all workers (if using a strong one). I am unsure how to proceed on this.
• The reform impact service is typically only run alongside a job, which is itself run on a worker. However, the service interfaces with portions of the code that are not isolated to the job itself, creating challenges when choosing a logging file to write to. If logging with the more basic Logger class, the service will actually create a new log file, perhaps because it is run within the worker thread, not the main API one. However, if logging with the WorkerLogger class, the service has no access to the proper worker ID (at least, not that I found), so it will create a file with the wrong name and log to a separate file from the worker itself.

[mikesmit]

Pausing here because I think the main comment about "why aren't we doing this the usual log4x way" is the most relevant and impacts the rest of the review.

[mikesmit]

discussion, blocking With the qualifier I am looking at this as a fairly standard Log4X-style logger, but there may be aspects /conventions in python that make this different...

Generally I would expect you to just use logging.getLogger in your classes like this

And configure your logs globally at the root logger level rather than doing it on child loggers, generally with something like logging.config.fileConfig or command line arguments or whatever).

Where you might have different settings for the main app thread vs. the worker thread and you could tune specific loggers

suggestion I think this is worth a discussion. You may have gone this way for specific reasons that I'm not aware of.

[mikesmit]

This makes more sense to me in terms of standardizing your message string. you don't necessarily need to wrap logger to do this though, you could just write a 'format_context' function or whatever:

logger.info(format_context(message, context))

[mikesmit]

I believe if we configure as discussed above you don't have to worry about this.

[mikesmit]

Similarly you could configure this at the application level rather than in code at each logger declaration. Maybe there are reasons to only cloud log some things, but I'd generally expect it to be on/off for the whole app.

[mikesmit]

nitt, documentation Arguments in documentation not in the constructor. suggestion remove them

[mikesmit]

question, blocking - who/what is this for? Google app engine looks at either stdout/stderr or the cloud logging API.

For that reason I'm surprised we write this by default even in the cloud.

suggestion I would not do this by default I would only do it in dev mode. In the context of the suggestion above I would pass a different config in dev than in prod.