Epic: Create new Usage Service to support tiered pricing

[jameshochadel]

What we're after

The new pricing model for Cloud.gov relies on gathering information about customer usage of brokered services. Today, we gather information manually on a monthly basis using scripts. The new system must:

1. Gather information about services usage more frequently than today - at least daily.
2. Store that information for some period of time for auditing purposes.
3. Calculate the credit equivalent of the raw usage, and store that information.
4. Provide the credit usage information to customers via the new Web UI (dashboard) so customers are aware of their usage.
5. Provide an administrative interface that allows setting the pricing tier of each customer. This might also be exposed through the web UI, but only to Cloud.gov team members. Or, it could be a different, simpler interface that's internal-only.

Hypothesized benefit(s)/why:

• For the CG team, fewer or no manual audits of usage
• CG team: Simpler and less error-prone administration of customer billing status. (Currently tracked in a spreadsheet)
• Transparency for customers, who will not be able to see this information otherwise

Potential metrics

• How many of our current brokered services the Usage Service can collect data on. Must be 100% for pricing to go live.
• Performance. The service should respond quickly to requests. It should have automated benchmarks for response latency. Usage information gathering should be sensitive to AWS API rate limits, so it may need to be spread throughout the day.
• Customer satisfaction with the new billing section of the dashboard will help inform Service Usage API design.

---

Further context for those unfamiliar with what we're doing

GDrive for new pricing: https://drive.google.com/drive/folders/1DRCp1WkWHdlbeu2aLYEVqIY-zoMCEZub

The pricing model in that folder enumerates the usage dimensions for current service plans, each of which will need to be accounted for in the Usage Service.

Security considerations

The service will probably expose a public-facing API for use by the Dashboard which will have to be secured. It will have to accept Cloud.gov user credentials and only allow users authorized with the correct role to view billing information. Similarly, the administrative API will require authentication, and only authorized Cloud.gov team members will be permitted to use it. The service will likely use a PostgreSQL database backend which will need to be secured.

As a new service in boundary, it will require an SCR.

Notes for implementers

• I plan to implement the service in Go. We will need to work closely with the Dashboard team to design an API that is useful to the dashboard, and the billing team at CG to determine their needs. We have some other, smaller Go HTTP services in use on the platform today.

Related issues/sub-projects

• None yet.

[jameshochadel]

It may make sense to have a contingency for serving static HTML pages to customers in case the Dashboard pricing pages are not done in time. But, this is far from ideal; there is otherwise no HTML templating in this service, and authentication would work differently to a web app vs an API.