CI setup for AWS

[phbnf]

• provide basic backend storage setup
  • eventually, use Aurora Serverless instead of setting up a new AuroraDB every time
• integrate with ECS + Fargate to run the conformance test (I haven't tested this yet with lambda functions, but did so from a VM)
  • add a VPC between the lambda function and S3
  • set-up service accounts
  • use IAM for database authentication
• integrate with a build pipeilne
• integrate with a deployment pipeline
• connect the build pipeline to git

[phbnf]

I spent a bunch of time looking into (integrate with a build pipeline, integrate with a deployment pipeline), and spoke a bit with @patflynn about various options.

We can:

1. do everything in AWS, using CodeBuild, CodePipeline. It seems like a good idea since everything will be deployed on AWS anyways.
2. do everything in GCP, in the same way that we do this for the GCP deployment (modulo that the conformance binary and hammer would run on AWS). There would be some wrinkles about a. which container registry we push the AWS conformance binary, b. cross cloud authentication (probably requires using secret keys, etc)
3. do everything with GitHub action. Same wrinkles as for 2., but there's a lot open source bits that we can re-use to make this work.

I'm going to move forward with 3.:

• authentication with AWS is greatly simplified by https://github.com/aws-actions/configure-aws-credentials, and https://github.com/aws-actions/amazon-ecr-login
• after playing with the three options, I come to the conclusion that GithubAction is by far the most frictionless option: a single .yaml file to rule them all, can easily pass data between steps, lowest learning curve, the team is already familiar with this
• it's different than the GCP setup, but we already use Github Actions for a few things including some integration tests, so it's not breaking ground either.
• we can always move the GCP setup to Github Actions later if we want consistency

[phbnf]

For authentication, we'll start by storing a secret key in GitHub for a user defined with clickops that has the right permissions. We'll move to a sleeker form of authentication later.