ONNX Runtime Multithreading Issue Preventing NIST FRVT 11 Submission

[AwaisPF]

Reporting a Multithreading Issue in ONNX Runtime to NIST FRVT GitHub Repository

Body:

I am encountering an issue with ONNX Runtime that prevents my submission to the NIST FRVT 11 evaluation.

Describe the problem:

• I am using ONNX Runtime for inference in my submission package.
• I have attempted to restrict the number of threads used by ONNX Runtime to 1 using the provided GUID.
• However, when my submission package is executed, the NIST FRVT script detects 17 threads instead of the expected 2.
• This discrepancy becomes apparent when using the top -H command, which reveals 17 threads are created, with only 1 in a running state and the remaining 16 in a sleeping state.
• Due to this multithreading behavior, my package crashes when executed by NIST FRVT on multiple cores, violating their single-threaded requirement.

Here is screen shot of htop of same package

• Only one thread is running but why there are so many threads launched

Possible impact:

• This issue prevents my package from being successfully evaluated in the NIST FRVT 11 program.

• I kindly request the NIST FRVT developers to investigate this multithreading issue in ONNX Runtime and provide any potential solutions or workarounds.

• Additionally, I would appreciate guidance on overcoming this limitation and ensuring successful submission compliance with the single-threaded requirement.

Additional information:

• Version of ONNX Runtime==1.10.0

Closing:

Thank you for your time and consideration. I look forward to your response and assistance in resolving this issue.

[deeperlearner]

Did you close the docker and re-open one?

[AwaisPF]

there is no usage of docker here. It ran directly on host machine as recommended.