Can the async stuff be feasibly upstreamed?

[svenstaro]

Currently you maintain a bunch of downstream forks (which is great!) but I think it'd be even better if async with esp32 could just work out of the box with upstream libraries. Do you think there's a chance to get your work integrated upstream?

[ivmarkov]

Yes, there is. But I think it is a bit too early. For one, I've yet to see something meaningful implemented either by the community, or by me which is using these patches. For example, a pure rust async mqtt client. Or http client. Or http server. Once some of these things start popping up, I'll upstream.

[ivmarkov]

By the way - and if it is not clear - the "async" patches revolve around asynchronous networking. They basically (a) patch the socket2 crate to be ESP-IDF compatible, and (b) the polling crate (the smol "reactor") to be - ditto - ESP-IDF compatible. Hence my comment that until somebody starts using async networking from Rust (as in async TCP sockets), I don't feel comfortable upstreaming those.

If you don't explicitly require/use async networking (as in instead just using the available HTTP client and MQTT client from esp-idf-svc - note that the latter can be "asyncified" - haven't checked the former yet) - you actually don't even need these patches.

For example, async-task (the async executor used by smol and async-std) as well as the executor of futures-rs work just fine, unpatched. I've even implemented and I'm using an async executor (on top of async-task) that can be notified directly from an ISR context. E.g. a button click or a timer firing. But that's another story.

[ivmarkov]

With edge-net on the horizon, I've started upstreaming the async stuff.

First patch that needs to happen is in libc itself: rust-lang/libc#2864

The patches to polling, socket2 and smol itself will follow after that.

[SimonSapin]

Hence my comment that until somebody starts using async networking from Rust (as in async TCP sockets), I don't feel comfortable upstreaming those.

Hi! I recently got a couple ESP32-C3 boards. The project I have in mind would involve an HTTPS client (to send sensor data to a server, and i2c and BTLE to collect said data). I was thinking of using esp-idf for the wifi driver and the TCP implementation, but use Rust async + Hyper + rustls for HTTPS. I’m happy to try out patches.

[jasta]

I'm also attempting upstreaming similar changes but for tokio support: tokio-rs/tokio#5867. The socket2 changes are the same, though.

[ivmarkov]

Both tokio and smol changes are now upstreamed. Moreover, the changes to the async-io echosystem are also released as async-io 2.

Closed.

[svenstaro]

That's great news!