Java thread pool leak bug in idp-plugin-privacyIDEA-1.1.0

[equbaykiflay]

I opened this bug on Shibboleth but it was found that the bug is actually in the privacyIDEA plugin. I verified that by disabling the plugin and checking if the problem disappears, which it did. So I am posting it here

Description

Thread pools are not being released resulting in thread and memory leak. Here is the output of jstack with a threads aged over 11 hours and less than 10 second stuck waiting on condition.

root@shibserver:~# jstack 1115417 | grep -A 14 thread-1 | head -30
"pool-3-thread-1" #105 prio=5 os_prio=0 cpu=36.14ms elapsed=42650.98s tid=0x00007f52f819fe80 nid=0x1106a7 waiting on condition [0x00007f52d6ffe000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000005aeab6258> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506)
at java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3465)
at java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3436)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1625)
at java.util.concurrent.ArrayBlockingQueue.take([email protected]/ArrayBlockingQueue.java:420)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1062)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1122)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635)
at java.lang.Thread.run([email protected]/Thread.java:840)

"pool-4-thread-1" #109 prio=5 os_prio=0 cpu=0.62ms elapsed=42649.61s tid=0x00007f52f81a2ff0 nid=0x1106af waiting on condition [0x00007f52d69f8000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000005aeab64b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506)
at java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3465)
at java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3436)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1625)
at java.util.concurrent.ArrayBlockingQueue.take([email protected]/ArrayBlockingQueue.java:420)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1062)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1122)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635)
at java.lang.Thread.run([email protected]/Thread.java:840)

root@shibserver:~# jstack 1115417 | grep -A 14 thread-1 | tail -32
"pool-6929-thread-1" #32245 prio=5 os_prio=0 cpu=0.57ms elapsed=7.75s tid=0x00007f535e314060 nid=0x122c78 waiting on condition [0x00007f5122ad4000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000004acb9a2d8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506)
at java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3465)
at java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3436)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1625)
at java.util.concurrent.ArrayBlockingQueue.take([email protected]/ArrayBlockingQueue.java:420)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1062)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1122)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635)
at java.lang.Thread.run([email protected]/Thread.java:840)

--
"pool-6930-thread-1" #32248 prio=5 os_prio=0 cpu=0.21ms elapsed=6.41s tid=0x00007f52f4724470 nid=0x122c7b waiting on condition [0x00007f51249f3000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
- parking to wait for <0x00000004acdb1000> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506)
at java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3465)
at java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3436)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1625)
at java.util.concurrent.ArrayBlockingQueue.take([email protected]/ArrayBlockingQueue.java:420)
at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1062)
at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1122)
at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635)
at java.lang.Thread.run([email protected]/Thread.java:840)

Environment

OS: Debian 12.7
Container: Jetty 12.0.13
JVM: amazon-corretto 17.0.12.7.1 (build 17.0.12+7-LTS, mixed mode, sharing)
Idp: Shibboleth 5.1.3
PI plugin: idp-plugin-privacyIDEA 1.1.0

[nilsbehlen]

@lukasmatusiewicz does this plugin in v1.1.0 use the javaclient in v1.2.2?
I have made changes to the java client to fix this problem. it was reported previously with keycloak.
If not, we need to build a new version of this plugin with the current javaclient.

[lukasmatusiewicz]

@nilsbehlen it uses v1.2.0 - 17 Jan 2023, so we need to update it