When to use rmm_async

[charlesbluca]

With #566, we added the ability to use RMM's asynchronous allocator for cluster setup, which is mutually exclusive with RMM pools and managed memory. I am wondering if there are any specific benefits for using this, or use cases where it would be important? These could potentially be added somewhere in docs if they are prominent enough.

cc @pentschev

[pentschev]

cc @beckernick who originally asked about this feature.

[beckernick]

The async memory resource will not currently perform as well as the default pool resource. However, this feature has one core benefit relative to a standard RMM pool: it makes it easier to share GPU memory with libraries that don't expose a pluggable external memory interface (while still getting performance gains relative to standard cudaMalloc).

The standard pool resource will exhaust the available memory as needed and never free it. The async resource will free it.

[pentschev]

Seems like the use case where this is helpful is when using external libraries at the moment. Do you happen to have any such examples @beckernick ? Perhaps we could document that's the intended use case and, if possible, add an example of the intended usage.