Disable default cuDF pinned pool (#10868)

* Disabling the cuDF default pinned pool for 24.06 Signed-off-by: Alessandro Bellina <[email protected]> * Add a warning in case we can't configure the cuDF default pool --------- Signed-off-by: Alessandro Bellina <[email protected]>
NVIDIA · May 28, 2024 · 91bdb92 · 91bdb92
1 parent d1e986d
commit 91bdb92
Showing 1 changed file with 6 additions and 0 deletions.
diff --git a/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuDeviceManager.scala b/sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuDeviceManager.scala
@@ -447,6 +447,12 @@ object GpuDeviceManager extends Logging {
     } else {
       (conf.pinnedPoolSize, -1L)
     }
+    // disable the cuDF provided default pinned pool for now
+    if (!PinnedMemoryPool.configureDefaultCudfPinnedPoolSize(0L)) {
+      // This is OK in tests because they don't unload/reload our shared
+      // library, and in prod it would be nice to know about it.
+      logWarning("The default cuDF host pool was already configured")
+    }
     if (!PinnedMemoryPool.isInitialized && pinnedSize > 0) {
       logInfo(s"Initializing pinned memory pool (${pinnedSize / 1024 / 1024.0} MiB)")
       PinnedMemoryPool.initialize(pinnedSize, gpuId, setCuioDefaultResource)