TThreadedServer vs. TNonblockingServer
Which Thrift RPC server should MapKeeper use, TThreadedServer or TNonblockingServer? This benchmark compares 2 Thrift C++ RPC servers using StubServer. The focus of this benchmark is to test these 2 servers on a multi-core servers with a limited number (<1000) of concurrent client connections.
TThreadedServer spawns a new thread for each client connection, and each thread remains alive until the client connection is closed. This means that if there are 1000 concurrent client connections, TThreadedServer needs to run 1000 threads simultaneously.
TNonblockingServer has one thread dedicated for network I/O. The same thread can also process requests, or you can create a separate pool of worker threads for request processing. The server can handle many concurrent connections with a small number of threads since it doesn’t need to spawn a new thread for each connection.
TThreadPoolServer (not benchmarked here)
TThreadPoolServer is similar to TThreadedServer; each client connection gets its own dedicated server thread. It’s different from TThreadedServer in 2 ways:
- Server thread goes back to the thread pool after client closes the connection for reuse.
- There is a limit on the number of threads. The thread pool won’t grow beyond the limit.
Client hangs if there is no more thread available in the thread pool. It’s much more difficult to use compared to the other 2 servers.
- 2 x Xeon E5620 2.40GHz (HT enabled, 8 cores, 16 threads)
- RHEL Server 5.4, Linux 2.6.18-164.2.1.el5 x86_64, 64-bit
- Thrift 0.6.1
- TNonblockingServer thread pool size: 32 threads
- Client and server run on the same box.
- Number of client threads: 300
- Number of requests: 10 million
- Request size: ~60 bytes
- Response size: ~30 bytes
In this benchmark, TThreadedServer performs much better than TNonblockingServer. CPU is maxed out with TThreadedServer, while TNonblockingServer only uses about 20% of CPU time. I’m guessing it’s because the I/O thread is being the bottleneck and worker threads are not getting enough things to do .
TThreadedServer seems like a better fit for MapKeeper since I’m not planning to support thousands of concurrent connections (yet). TNonblockingServer might be a better choice when you face the C10K problem, but you need to make sure the I/O thread doesn’t become the bottleneck. It would be an interesting project to add a new type of Thrift server with a single accept() thread and multiple worker threads handling network I/O and request processing. There is already an open JIRA for this feature in Java. Is anybody interested in working on a similar feature in C++?
Send me email at [email protected] or post your comments here if you have any questions/suggestions.