[QST] Evaluate nanoarrow as part of any future Arrow C Data Interface exploration

[beckernick]

We've had some discussions in the past about exploring Arrow C Data Interface (cc @shwina @vyasr @GregoryKimball ) for a variety of reasons. If we used the C Data Interface, it might open the door to using the new, much lighter weight Arrow project nanoarrow:

The vision of nanoarrow is that it should be trivial for a library or application to implement an Arrow-based interface: if a library consumes or produces tabular data, Arrow should be the first place developers look. Developers shouldn’t have to be familiar with the details of the columnar specification—nor should they have to take on any build-time dependencies—to get started.

• Nanoarrow release blog

Nanoarrow is <350KB when compiled and can in theory safely interact with any recent PyArrow versions. If we return to exploring the Arrow C Data Interface, we should also explore Nanoarrow.

[shwina]

Related: #5097

[paleolimbot]

Just a note if you aren't aware that there's now a Device extension of the C Data interface ( https://arrow.apache.org/docs/dev/format/CDeviceDataInterface.html ) and proof-of-concept nanoarrow implementation of it ( https://github.com/apache/arrow-nanoarrow/tree/main/extensions/nanoarrow_device#readme )!

[vyasr]

We are now using nanoarrow in #15047 and I expect the usage to increase to cover the other relevant use cases as discussed in #15193.