Writing an HTTP client for dstopic

[zkat]

As part of development on dstopic (the Rust side of the client), we need to have a shiny new http client with certain features:

• http-compliant caching, to cacache
• retry support
• keepalive support
• async/await support
• (optional, unnecessary, low-priority) proxy support

No existing http clients on crates.io currently support all these features (or even most of them), so we need to write one from scratch!

(original thread)

Blocked by

• Land surf-middleware-cache
  • Needs http-cache-semantics to land. (Rust port of http-cache-semantics #16)
• Get surf 2.0 over the finish line. The planned API changes are gonna be important to implement this client. (2.0 release tracking http-rs/surf#107)
• Implement a baseline client API wrapper around surf
• (Optional) Integrate backoff to automatically retry http requests. Maybe this needs to be implemented as a surf middleware?

[zkat]

@yoshuawuyts wrote the baseline for this http client, in the form of surf, which takes care of async/await and keepalive support on its own (on top of being a generally excellent http client). Besides that, it has a middleware system that can be used to add caching, retry, and possibly proxy support (eventually).

[zkat]

Status: I'm currently working on caching middleware for surf, now that cacache-rs has mostly stabilized.

[zkat]

Update: I've added a Status section to the OP to track the requirements to get this written. We're still a ways away from being able to write this.