How bad would it be to ignore async features completely?

[bubelov]

I've been experimenting with Rocket and different SQLite crates and it looks like there are no good ways to use SQLite in async context. SQLite doesn't offer an async API and existing attempts to create async wrappers come with severe performance hits and other issues.

What should probably work fine for SQLite is a thread pool. It allows concurrent reads and it should scale fine, as long as each read has it's own connection. The problem is, this approach doesn't seem to benefit from async runtime and I don't see any sensible ways of utilizing it.

What would be the implications of treating Rocket as a traditional synchronous web framework? By that I mean calling strictly sync code from async handlers. The number of worker treads is configurable, so I can start Rocket with a lot of worker threads and give each thread a database connection. Would it be any worse than using synchronous web framework with the same number of working threads?

[the10thWiz]

This would be awful for performance. The runtime (tokio) expects async tasks to return quickly. There are ways to execute blocking code in an async-safe way, such as tokio::spawn_blocking, which spawns an async task (which can be awaited, and return a value) where blocking is okay. This allows the runtime to schedule around your blocking code, and keep the server running normally.

However, I would recommend taking a look at sync_db_pools, one of the replacement crates for rocket-contrib. I believe it does exactly what you want (interact with a sync db client via a thread pool), from an async context, with the smallest performance hit. I would also take a look at db_pools to see if it supports SQLite, since that should (in theory) be the perfect solution.

[the10thWiz]

Also, I'm not sure if the number of worker threads is directly configurable anymore, since Rocket needs less of them now.

[bubelov]

This would be awful for performance. The runtime (tokio) expects async tasks to return quickly. There are ways to execute blocking code in an async-safe way, such as tokio::spawn_blocking, which spawns an async task (which can be awaited, and return a value) where blocking is okay. This allows the runtime to schedule around your blocking code, and keep the server running normally.

AFAIK, spawn_blocking just pushes the task to a bigger pool. I see how making futures too greedy and non-cooperative can degrade async performance but that's the whole point: I don't have async code and SQLite simply can't benefit from an async runtime anyway.

To make the example a bit more specific, let's say we have a traditional thread-pool based web server with 16 worker threads. Let's also say our handles simply call a repository which is responsible for talking to our DB. That would give as up to 16 concurrent calls to SQLite, right? If all those calls are writes, there is no benefit in concurrency, we're getting bottle-necked and there are no ways around it, but if they're reads, performance would scale pretty nicely with the number of CPUs, wouldn't it?

Now, let's say we have an exact setup with an async web server backed by the same number of worker threads. No code changes at all, except the fact that controller-level handlers are now async. Would this setup be any slower than the previous one?

To put it in other words, is scheduling non-cooperative futures less efficient than scheduling threads in a thread pool? Basically, by ignoring async runtime, we're making every handler non-cooperative. That's exactly what traditional web servers do, so I wonder if it leads to comparable performance. If it does, ignoring async runtime might be the best course of action when using SQLite.

[bubelov]

However, I would recommend taking a look at sync_db_pools, one of the replacement crates for rocket-contrib. I believe it does exactly what you want (interact with a sync db client via a thread pool), from an async context, with the smallest performance hit. I would also take a look at db_pools to see if it supports SQLite, since that should (in theory) be the perfect solution.

I tried both options. Async one doesn't support SQLite except the crate which does it with a huge performance hit. Sync one is fine though, because it supports rusqlite. I can probably use it, but it would force me to use closures which would make the code more verbose and it would also trigger painful scope moves. I'm also a bit hesitant to expose connection handlers to controllers. I'm gravitating towards common Controller -> Service -> Repository kind of structure, and connections/pools feel like the implementation detail which belongs to layer 3. Lifting it 2 layers up is a thing I'd prefer to avoid.

[the10thWiz]

AFAIK, spawn_blocking just pushes the task to a bigger pool.

I believe it's slightly more complicated (specifically, tokio also reserves a thread to run the scheduler and non-blocking futures).

Async can provide a performance boost to IO bound tasks. Webservers, like Rocket, are typically IO bound, since they spend the majority of their time sending and receiving data. I think the biggest issue you might run into is actually DOS - since it would then be possible to block every thread in the pool with the right set of requests. If this isn't a concern, I would test the performance and see if it works as desired.

Async one doesn't support SQLite except the crate which does it with a huge performance hit.

I am interested in taking a closer look at this, since it really shouldn't have a huge performance hit.

I'm also a bit hesitant to expose connection handlers to controllers.

Did you know that request guards can invoke other request guards? I would see if you can leverage that to move the connection pool further down. This could also help alleviate the scope issues, since the closures would be much smaller and simpler.

[bubelov]

Thanks for you feedback! A few comments and clarifications:

I think the biggest issue you might run into is actually DOS - since it would then be possible to block every thread in the pool with the right set of requests

Resources are always finite, no abstraction would help with that. I believe DOS prevention and mitigation has little to do with async (because it should be dealt with at a much lower layer, perhaps some fancy and insanely performant firewall rules). Anyway, my question is not really about sync vs async, it's about the overhead of async runtime in a situation when all the code except the handlers is sync. Let's say I used "sync" framework and just switched to Rocket. I just copied my service and repo layers and connected them to Rocket's async handlers. Given that underlying thread pool size stays the same, should I expect poor performance, compared to my previous generic thread-pool based web framework? If so, why? There will be the same number of threads hitting my services and repos concurrently, right? Shouldn't performance be comparable, unless async scheduler fails to utilize all of it's worker threads somehow?

I am interested in taking a closer look at this, since it really shouldn't have a huge performance hit.

You can take a look here and in the linked issues: rusqlite/rusqlite#188. Async might make sense where there are async APIs available, SQLite isn't that so I doubt there could be any benefit in using it in async contexts, and there are plenty of unpleasant side effects

Did you know that request guards can invoke other request guards? I would see if you can leverage that to move the connection pool further down. This could also help alleviate the scope issues, since the closures would be much smaller and simpler.

I use State for that, which is a kind of request guard, except I don't need to write any code or instantiate anything. I do have custom guards for auth checks, but services/repos are easier to instantiate once and simply add them to Rocket state before it lifts off

[jebrosen]

What would be the implications of treating Rocket as a traditional synchronous web framework? By that I mean calling strictly sync code from async handlers. The number of worker treads is configurable, so I can start Rocket with a lot of worker threads and give each thread a database connection. Would it be any worse than using synchronous web framework with the same number of working threads?

The best answer I can come up with is "It will be fine, until maybe it isn't". If active_db_operations is always less than worker_threads, I would not expect too many problems. As soon as you have more active database operations than the number of worker threads, you will start to see performance degradation since every worker thread will be waiting (probably on disk I/O) and unable to accept connections, transfer data, etc. This is the main reason to use spawn_blocking: the blocking operations are sent to a separate thread pool from the worker threads, which will be able to continue doing non-database work.

(edited to add:) A different angle on this: one of the main benefits of going from sync to async is that the amount of in-flight I/O (disk, network) is no longer coupled 1:1 to OS-level threads. This means that operations that don't need I/O are not impeded, and that you need much, much fewer OS-level threads and context switches to accomplish roughly the same amount of work. Yes, there is some degree of overhead to using an async runtime, but in many applications this overhead is more than made up for by being able to handle more than worker_thread_count simultaneous pending I/O operations. Using an async framework as if it were synchronous will at least revert to sync-style performance, and will bring back some of the other difficulties with synchronous APIs (e.g. around cancellability and timeouts) that async tries to avoid.

[bubelov]

As soon as you have more active database operations than the number of worker threads, you will start to see performance degradation since every worker thread will be waiting (probably on disk I/O) and unable to accept connections, transfer data, etc

So, basically, the same kind of behavior one might expect from a thread-pool web server without an async runtime? Looks like that might be the best option in my situation. Using spawn_blocking makes a lot of sense in "mostly async" code which needs to fit a few "exceptions". If the whole codebase is sync, starting with a big enough main pool should probably give the same result as wrapping all endpoints into spawn_blocking, but without the boilerplate

[jebrosen]

In principle, probably - but of course you should always test your own application under load to see the exact consequences.

Note that this is less risky with disk I/O such as SQLite, which generally should complete in a somewhat timely manner. Longer synchronous operations, like network I/O or waiting for a mutex, are more concerning. For further reading I highly recommend Alice Ryhl's post What is blocking? in case you haven't already read it - it goes into more detail on the subject and has further helpful links. It is primarily tokio-oriented, but the general concepts apply to some degree to nearly every library or framework for asynchronous I/O.

[bubelov]

This means that operations that don't need I/O are not impeded, and that you need much, much fewer OS-level threads and context switches to accomplish roughly the same amount of work

Yea, I get that. I have some Kotlin background and Kotlin Coroutines are pretty close to async runtimes in Rust. A typical Rocket web server should benefit from being async tremendously because it would receive requests from clients in an async way (via OS API) and it would send it's own requests to a database server (again, truly async, via OS primitives). The problem is: there are no "primitives" for SQLite, it's radically different from dealing with TCP sockets which are async all the way down. It doesn't have an async API and it scales in a different way instead. Well, writes don't scale at all, and read throughput improves with the number of threads, assuming each thread has a separate connection. Rocket doc states that its thread pool size is configurable, so it might be the cleanest way to reconcile Rocket and SQLite