Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose NNG_MAX_EXPIRE_THREADS via CMake #1619

Closed
JochenBaier opened this issue Sep 19, 2022 · 5 comments
Closed

Expose NNG_MAX_EXPIRE_THREADS via CMake #1619

JochenBaier opened this issue Sep 19, 2022 · 5 comments

Comments

@JochenBaier
Copy link

The define NNG_EXPIRE_THREADS not set in CMakeLists.txt. Define is used in nni_aio_sys_init().

Fix in CMakeLists.txt:

if (NNG_EXPIRE_THREADS)
add_definitions(-DNNG_EXPIRE_THREADS=${NNG_EXPIRE_THREADS})
endif ()
mark_as_advanced(NNG_EXPIRE_THREADS)

@LowLevelMahn
Copy link

so what does this patch help?

@JochenBaier
Copy link
Author

so what does this patch help?

Related to "nng creates a massive amount of threads on high core count machines"
#1572:

@gdamore
Copy link
Contributor

gdamore commented Oct 17, 2022

Yes, it would be good to have this as a tunable in the CMakeLists.

@gdamore gdamore changed the title Define NNG_EXPIRE_THREADS not set in CMakeLists.txt Expose NNG_MAX_EXPIRE_THREADS via CMake Apr 20, 2023
@phsilva
Copy link
Contributor

phsilva commented Aug 23, 2023

@gdamore at the end only the NNG_MAX_EXPIRE_THREADS was added. Code still uses NNG_EXPIRE_THREADS that effectively overrides NNG_MAX_EXPIRE_THREADS.

Should we apply something similar to how taskq limits are set?

Having a max value (NNG_MAX_EXPIRE_THREADS) that only caps the number of tasks, default to 8 or 256 (CMake default is 8 today but code imposes 256 limit).

And having a NNG_NUM_EXPIRE_THREADS that overrides the value, but still obeys the limit?

I have an MR for that if you want.

@gdamore
Copy link
Contributor

gdamore commented Aug 23, 2023

Go ahead and submit the PR. Cannot promise it will merge, but it might, and cannot hurt to look.

The taskqs needs to have a minimum of 2, or else deadlocks may occur.

phsilva added a commit to phsilva/nng that referenced this issue Aug 23, 2023
This change makes expire threads tunable follows the same strategy as
taskq threads tunables.

Add NNG_NUM_EXPIRE_THREADS to override the default behaviour (`n_cpu`
expire threads).

The NNG_MAX_EXPIRE_THREADS limit is always applied if > 0, even
if you specifiy the desired number of threads using NNG_NUM_EXPIRE_THREADS.

NNG_EXPIRE_THREADS is not used anymore. This was only referenced in code
but never defined on CMake.

The logic to cap expire threads between 1 and 256 was removed. If user
set no limits, whatever value they choose will be used instead of being
silently overriden by us.
phsilva added a commit to phsilva/nng that referenced this issue Aug 23, 2023
This change makes expire threads tunable follows the same strategy as
taskq threads tunables.

Add NNG_NUM_EXPIRE_THREADS to override the default behavior (`n_cpu`
expire threads).

The NNG_MAX_EXPIRE_THREADS limit is always applied if > 0, even if you
specify the desired number of threads using NNG_NUM_EXPIRE_THREADS.

NNG_EXPIRE_THREADS is not used anymore. This was only referenced in the
code but never defined on CMake.

The logic to cap expire threads between 1 and 256 was removed. If users
set no limits, whatever value they choose will be used instead of being
silently overridden by us.
gdamore pushed a commit that referenced this issue Aug 28, 2023
This change makes expire threads tunable follows the same strategy as
taskq threads tunables.

Add NNG_NUM_EXPIRE_THREADS to override the default behavior (`n_cpu`
expire threads).

The NNG_MAX_EXPIRE_THREADS limit is always applied if > 0, even if you
specify the desired number of threads using NNG_NUM_EXPIRE_THREADS.

NNG_EXPIRE_THREADS is not used anymore. This was only referenced in the
code but never defined on CMake.

The logic to cap expire threads between 1 and 256 was removed. If users
set no limits, whatever value they choose will be used instead of being
silently overridden by us.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants