Connection pool monitoring

[inikolaev]

Hello!

Can you advise some way to monitor connection pool, specifically for the AsyncClient?

What I would like to know is the following:

• How many request are awaiting a connection
• How many connections are being used right now (in-flight requests)
• Some statistics on connection open/close

I could potentially get some numbers using existing request and response hook, what I don't know is when request hook is fired - does it fire before connection is acquired or after - it's not very clear to me from the documentation. But this should be easy to check.

Perhaps there could be added hooks into the connection pool or maybe someone from the community does this in a different way somehow already?

[tomchristie]

Hiya!

I like this question. 😀

---

How many request are awaiting a connection

We don't currently have a public API exposing this, but you can get it by digging into the implementation. (Make sure to strictly version-pin if you're doing this in live code!)

All the handled requests are stored as state here...

httpcore/httpcore/_async/connection_pool.py

Line 106 in cad62bf

self._requests: List[RequestStatus] = []

The status has two attributes .request and .connection.

If connection is None then the request is queued waiting. If the connection is not None then the request/response is currently in-flight.

---

How many connections are being used right now

Connections are stored as state here...

httpcore/httpcore/_async/connection_pool.py

Line 105 in cad62bf

self._pool: List[AsyncConnectionInterface] = []

A connection might either be idle (keep alive) or active. Active connections will only ever have one request for HTTP/1.1, but may have one or more requests for HTTP/2. The object repr will give you a bit of information about the connection state.

---

Some statistics on connection open/close

You'd need to figure that one out yourself.

Something that would help would be if the 'trace' extension included callbacks for events in the connection pool. We don't currently do that in the connection pool. Take a look at eg. connection.py to see examples of where we do use the 'trace' extension to provide fine-grained event details...

httpcore/httpcore/_async/connection.py

Lines 108 to 112 in cad62bf

	async with Trace(
	"connection.connect_tcp", request, kwargs
	) as trace:
	stream = await self._network_backend.connect_tcp(**kwargs)
	trace.return_value = stream

---

If you're interested in helping work on public API towards any of these as new features, I'd be very happy to help guide you through the process.

[inikolaev]

The trace extension indeed looks like a good place to add this and I'm happy to help if you already have some ideas of where I would start.

[inikolaev]

I've been looking at RequestStatus and it looks like a good place to track when connection was acquired/released:

class RequestStatus:
    def __init__(self, request: Request):
        self.request = request
        self.connection: Optional[AsyncConnectionInterface] = None
        self._connection_acquired = AsyncEvent()

    def set_connection(self, connection: AsyncConnectionInterface) -> None:
        async with Trace("pool.connection.acquire", self.request):
            assert self.connection is None
            self.connection = connection
            self._connection_acquired.set()

    def unset_connection(self) -> None:
        async with Trace("pool.connection.release", self.request):
            assert self.connection is not None
            self.connection = None
            self._connection_acquired = AsyncEvent()

    async def wait_for_connection(
        self, timeout: float = None
    ) -> AsyncConnectionInterface:
        async with Trace("pool.connection.wait", self.request):
            await self._connection_acquired.wait(timeout=timeout)
            assert self.connection is not None
            return self.connection

Looks too easy though or what do you think?

I haven't really tried to run any code with this to see how it works, but I'll create an example using httpcore to try.

[inikolaev]

Turned out to be more involving, but I need more guidance here.

Instead I added similar extensions to the connection pool, because in some cases request object is not available, e.g. in the code which closes idle connections.

Then with the following example:

import asyncio
import httpcore


async def log(event_name, info):
    print(event_name, info)


async def request():
    print("Making a request")

    async with httpcore.AsyncConnectionPool(extensions={"trace": log}) as http:
        response = await http.request("GET", "https://www.google.com/")
        await response.aclose()
        print("Received response", response)

    print("Exiting")


def main():
    asyncio.run(request())


main()

I got the following results:

Making a request
pool.connection.wait {}
pool.connection.created {}
pool.connection.acquired {}
pool.connection.released {}
pool.connection.wait {}
pool.connection.acquired {}
pool.connection.released {}
Received response <Response [200]>
pool.connection.closed {}
Exiting

Which is close enough to what I want to achieve.

It will probably be a bit more productive to create a PR.

@tomchristie Should we convert this discussion into an issue?

[tomchristie]

Great yes - I've opened issue #508 to address this.

I'd recommend that a good place to start would be sketching out what events we want to see, and what we think that'd look like in the relevant part of the docs.