IPIP-484: Opt-in Filtering on Routing V1 HTTP API

src/ipips/ipip-0484.md

@@ -0,0 +1,103 @@
+---
+title: 'IPIP-0484: Delegated Routing V1 Support for Querying Specific Network Transport and Transfer Protocols'
+date: 2024-09-03
+ipip: proposal
+editors:
+  - name: Daniel Norman
+    github: 2color
+    affiliation:
+      name: Shipyard
+      url: https://ipshipyard.com
+relatedIssues:
+  - https://github.com/ipfs-shipyard/someguy/issues/13
+order: 484
+tags: ['ipips']
+---
+
+## Summary
+
+Add opt-in support for filtering specific network transports and/or transfer protocols to the Delegated Routing v1 HTTP endpoint via HTTP GET parameters.
+
+## Motivation
+
+IPFS aims to allow ubiquitous data exchange across different runtimes and platforms. One of the most challenging aspects of this goal is the diversity of network conditions and capabilities across different environments. Web browsers have a very limited network stack, and most web browsers do not support the full range of network transport protocols that are commonly used in other IPFS implementations.
+
+The Delegated Routing v1 API empowers resource constrained clients like web browsers by significantly reducing the number of network connections necessary to fetch content addressed data directly from provider peers.
+
+However, there are many cases where most of the results from the Delegated Routing v1 API are not actionable by clients, because the client does not support either the **network transport protocol** or the **transfer protocol** of the provider.
+
+For instance, web browsers are limited to a specific set of network transport protocols, namely HTTPS, Secure WebSockets, WebTransport (emerging), and WebRTC. Consequently, providing information about providers that exclusively support TCP and/or UDP is not beneficial for browser-based clients, as they are unable to utilize such connections.
+
+Moreover, [Helia](https://github.com/ipfs/helia/), the most actively maintained browser IPFS implementation, supports block retrieval by CID with Bitswap and Trustless Gateways, but does not support GraphSync.
+
+This means that returning providers that only support raw TCP, raw UDP/QUIC, or GraphSync from the Delegated Routing API is not useful for browser clients, and results in unnecessary network traffic for browser clients.
+
+## Note on terminology
+
+The term **"transport"** is overloaded in the IPFS ecosystem.
+
+In the context of this IPIP, we refer to the network layer transport protocol, e.g. TCP, QUIC, WebTransport, as **"network transport protocol"** to avoid ambiguity.
+
+**"Transfer protocol"** refers to data transfer protocols, i.e. content-addressed block retrieval protocols, e.g. Bitswap, GraphSync, HTTP.
+
+## Detailed design
+
+### Network Transport Protocol Filtering
+
+The proposed change is to add a `?filter-addrs` parameter to the `GET /routing/v1/providers/{cid}` endpoint of :cite[http-routing-v1]:
+
+- Add a `?filter-addrs=<comma-separated-list>` optional parameter to `GET /routing/v1/providers/{CID}` that indicates which network transports to return by filtering the multiaddrs in the `Addrs` field of the [Peer schema].
+- The value of the `filter-addrs` parameter is a comma-separated list of network transport protocol _name strings_ as defined in the [multiaddr protocol registry](https://github.com/multiformats/multiaddr/blob/master/protocols.csv), e.g. `?filter-addrs=webtransport`.
+- `unknown` can be be passed to include providers whose multiaddrs are unknown, e.g. `?filter-addrs=unknown`. This allows filtering providers whose multiaddrs are unknown at the time of filtering.
+- Multiaddrs are filtered by checking if the protocol name appears in any of the multiaddrs (logical OR).
+- Negative filtering is done by prefixing the protocol name with `!`, e.g. `?filter-addrs=!quic-v1,!tcp`. Note that negative filtering is done by checking if the protocol name does not appear in any of the multiaddrs (logical AND).
+- For an address to pass the filter, it must pass all negative filters AND match at least one positive filter.
+- Only multiaddrs that pass the filter are included in the response.
+- If there are no multiaddrs that match the passed transports, the provider is omitted from the response.
+- Filtering is case-insensitive.
+- If no parameter is passed, the default behavior is to not apply filtering by network transports.
+
+### Transfer Protocol Filtering
+
+The proposed change is to add a `?filter-protocols` parameter to the `GET /routing/v1/providers/{cid}` endpoint of :cite[http-routing-v1]:
+
+- Add a `?filter-protocols=<comma-separated-list>` optional parameter to `GET /routing/v1/providers/{CID}` to filter providers based on the `Protocol` field of the [Peer schema] .
+- The `filter-protocols` parameter is a comma-separated list of transfer protocol names, e.g. `?filter-protocols=transport-bitswap`.
+- Transfer protocols names should be treated as opaque strings and have a max length of 63 characters. A non-exhaustive list of transfer protocols are defined per convention in the [multicodec registry](https://github.com/multiformats/multicodec/blob/3b7b52deb31481790bc4bae984d8675bda4e0c82/table.csv#L149-L151). Implementations should not break when encountering unknown transfer protocol names.
+- `unknown` can be be passed to include providers whose transfer protocol is unknown, e.g. `?filter-protocols=unknown`. This allows filtering providers returned from the DHT that do not contain explicit transfer protocol information.
+- Providers are filtered by checking if the transfer protocol name appears in the `Protocols` array (logical OR).
+- If the provider doesn't match any of the passed transfer protocols, the provider is omitted from the response.
+- If a provider passes the filter, it is returned unchanged, i.e. the full set of protocols is returned including protocols that not included in the filter. (note that this is different from `filter-addrs` where only the multiaddrs that pass the filter are returned)
+- Filtering is case-insensitive.
+- If no parameter is passed, the default behavior is to not filter by transfer protocol.
+
+:::note
+Even though existing transfer protocol names start with `transport`, e.g. `transport-bitswap`, `transport-graphsync-filecoinv1`, and `transport-ipfs-gateway-http`. This is not to be confused with the network transport protocols, which are filtered using the `filter-addrs` parameter.
+:::
+
+## Design rationale
+
+- Using these query parameters improves cache efficiency, as the response will be smaller and more specific to the client's needs.
+- Backward compatibility is maintained by not changing the default behavior of the API.
+- Use of protocol name rather than codes makes it easier for human debugging.
+- DHT providers currently do not contain any transfer protocol information. `unknown` can be passed to `filter-protocols` to include such providers.
+- Since provider records are independent of peer records, and it's pretty common to have provider records without up-to-date multiaddrs for that peer, `unknown` can be passed to `filter-addrs` to include such providers.
+- Combining transfer protocol and transport protocol filters is done by ANDing the results of the filters, e.g. `?filter-addrs=webtransport&filter-protocols=transport-bitswap` will return providers that support bitswap and have a webtransport multiaddr.
+
+### User benefit
+
+By filtering out providers that do not support the desired network transport protocol and/or transfer protocol, the client can reduce the traffic necessary in order to fetch the data.
+
+Moreover, it makes it much easier to determine whether there are any browser-usable providers for a given CID, which is a common use case for clients.
+
+### Compatibility
+
+This should not effect existing clients or servers.
+
+The default behavior when `?filter-addrs` and `?filter-protocols` is not passed is left unspecified, this IPIP is limited to opt-in behavior.
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+[Peer schema]: https://specs.ipfs.tech/routing/http-routing-v1/#peer-schema