WSL 2 only using half the cores of the host machine

[prenaux]

WSL 2 is only using up to 50% of the CPUs on my 8c/16t machine.

I currently use an Hyper-V Ubuntu VM and had the same issue initially, as that was the default setting after creating the VM. With Hyper-V I can use the Hyper-V manager to allocate more cores, but I dont see any way of doing that with WSL2.

If WSL2 are Hyper-V VM it would be convenient to be able to configure them in the Hyper-V manager.

Is there anyway to setup the resource allocation of the WSL2 VM's?

[benhillis]

@prenaux - Do your machine have 2 CPUs?

We have some changes in progress for configuring various properties of the WSL2 lightweight VM. These are not traditional VM's. If it isn't running, it doesn't exist. They are created on-the-fly so they do not appear in Hyper-V manager.

[prenaux]

I'm testing WSL2 on a Ryzen7 1700 CPU, single socket.

I build the same java project from the command line, on the same machine. Using an HyperV VM where I allocated 15 of the 16 threads to the VM, and then in WSL2. WSL2 never uses more than 50% of the CPU, the HyperV VM uses the 15threads, so close to 100% CPU usage.

I doubt it's a contention issue on WSL2 - although I suppose it possible - the reason I say that is because I noticed the same issue the first time I setup the HyperV VM, and I had to go change the default configuration. Maybe the default uses the core count instead of the thread count, or something of the sort, the usage profile is really stuck at 50%, if it was contention I wouldnt expect it to stay so close to 50% usage for the 10-20mins WSL2 build...

I use Process Explorer to look at the graph of the CPU usage.

[benhillis]

Looks like I'm not properly calculating how many cores to give to the WSL2 lightweight VM. Working on a fix now.

[tr1ckymast3r]

dual 2690v2 - 20 core 40 threads ... and only 8 core on WSL 2 ... setup nginx, php running too slow ...

[therealkenc]

Looks like I'm not properly calculating how many cores to give to the WSL2 lightweight VM

In specific setups? I only ask because I've got a (lowly?) i7-6700K and my bog-typical make -j8 workloads peg all 4 cores and 8 threads. /proc/cpuinfo shows all 8.

[benhillis]

@therealkenc - Yes, it seems only configs are affected.

[benhillis]

Figured it out, the kernel is being built to only allow 8, I'll fix this.

root@BENHILL-DELL:/mnt/c/Users/benhill# zcat /proc/config.gz | grep NR_CPUS=
CONFIG_NR_CPUS=8

[prenaux]

@benhillis Please let us know when the update is live, I'll try it out.

[g0blue]

@benhillis Not sure what your fix is (increase cap or remove it). I do have some questions:

• Will NUMA be reported correctly by the kernel / VM?
• Will logical processors from all Windows processor groups be reported? (i.e. more than 64 processors support?)
  • WSL 1 "reported" only one processor group, but had no cap on the actual number of concurrent processes.
  • Since WSL 2 enforces the concurrent process limit, this is a much bigger issue.
• Will thread affinitization work correctly? This is mostly useful for HPC work, less so for normal server or compilation workloads.

Trying to use a dual SkyLake Xeon (112 total logical processors) with WSL 2 and am currently severely constrained by the 8 proc limit.

[onomatopellan]

@g0blue You can compile the latest kernel to try it out. Following the steps from #4203 but getting the sources from here instead: https://github.com/microsoft/WSL2-Linux-Kernel.
There you can see the changes they did like increasing the cap from 8 to 256 cores/threads.

[g0blue]

Thanks @onomatopellan! This worked great.

It seems that it still only uses 1 processor group, so even though the max cores is bumped to 256, you'll never actually use more than 64. My dual SkyLake is now limited to 1 socket (56 logical processors). Much better than 8, but less than what can be used with WSL 1.

[g0blue]

Taskmanager during WSL 2 software compilation:

[benhillis]

@g0blue - Do you happen to know how many processors GetSystemInfo returns for your machine? I'm assuming 56.

[g0blue]

Do you happen to know how many processors GetSystemInfo returns for your machine? I'm assuming 56.

Yes, 56.

Relevant links:

• https://docs.microsoft.com/en-us/windows/desktop/procthread/processor-groups
• https://docs.microsoft.com/en-us/windows/desktop/procthread/what-s-new-in-processes-and-threads

[g0blue]

gnome-system-monitor running in WSL 2 during sofware compilation. :)

[prenaux]

@benhillis any update on this? I just updated the Windows machine and reinstalled WSL2 from scratch and its still only using 8 cores, is the only option really to recompile the kernel somehow? If so how do you go about doing that?

[benhillis]

Should already be fixed. What version are you running?

[benhillis]

Or specifically does your system have multiple processor groups?

[prenaux]

@benhillis The test is on a Ryzen7 1700, so that's a single processor group (8c/16t). Do I need to re-download the Ubuntu appx? I've reused the one I downloaded a while back (because the Microsoft Store is broken again and refused to dl it...)

I also have a Threadripper 2950, are the 2 NUMA (unit/core?) 2 processor groups?

[benhillis]

@prenaux - No you do not need to redownload any apps. Can you share the output of the following commands?

uname -r
zcat /proc/config.gz | grep -i NR_CPU
dmesg | grep 'Command line'

[bjarnirafn]

I'm having the same issue on a Threadripper 2950X.

uname -r
4.19.43-microsoft-standard

zcat /proc/config.gz | grep -i NR_CPU
CONFIG_NR_CPUS_RANGE_BEGIN=2
CONFIG_NR_CPUS_RANGE_END=512
CONFIG_NR_CPUS_DEFAULT=64
CONFIG_NR_CPUS=8

dmesg | grep 'Command line'
[ 0.000000] Command line: initrd=\initrd.img panic=-1 pty.legacy_count=0

[benhillis]

@bjarnirafn - you are running an old Insider version with the # of cores capped at 8. You should upgrade to a new build.

[prenaux]

@benhillis I can confirm that the issue is fixed - I spent some time today to unstuck Windows Updates (reset caches... etc) and it updated to 19002. So I probably wasn't running the latest build. Tested on a Ryzen 1700.

[gregdwilson]

@benhillis I am having a similar issue. I have a multi socket system using two Intel Xeon E5-2699 v3 CPUs. In windows 10 it reports 2 Sockets, 36 cores and 72 logical processors. Those commands in WSL 2 listed above give,

4.19.84-microsoft-standard
CONFIG_NR_CPUS_RANGE_BEGIN=2
CONFIG_NR_CPUS_RANGE_END=512
CONFIG_NR_CPUS_DEFAULT=64
CONFIG_NR_CPUS=256
[ 0.000000] Command line: initrd=\initrd.img panic=-1 pty.legacy_count=0 nr_cpus=36

If I type in lscpu | grep -E '^Thread|^Core|^Socket|^CPU(' it gives
CPU(s): 36
Thread(s) per core: 2
Core(s) per socket: 18
Socket(s): 1

So it appears that only one socket is being recognized. What can I do to make sure that it recognizes both CPUs? Do I need to build a kernal with NUMA support like mentioned here? (#5025 (comment)) Let me know if I need to open a different ticket somewhere else. Thank you.

[AAlMutairi]

I also have the same issue with AMD threadripper. It detects the CPU as a 2 socket system but only uses one of the sockets. Is there anyway to fix it?

[therealkenc]

Let me know if I need to open a different ticket somewhere else.

It is worth opening new issue if people still aren't seeing all their cores on two-socket rigs, because the resolution for this issue (CONFIG_NR_CPUS stuck at =8) is on a one-way track administratively to closed/fixedin2004 (because "thing fixed" even if "different thing not fixed"). #5025 can stand on its own separately as a yet different ask. [It is tempting to repurpose circa-2016 #1115 but that is WSL1 which is another kettle of fish.]

[g0blue]

It is worth opening new issue if people still aren't seeing all their cores on two-socket rigs...

So, I went to open a new issue, and I found #3314 - which is exactly the issue that needs to be opened and was closed as a duplicate of #1115 by you.

To be clear, the issue has nothing directly to do with number of sockets, it's purely an issue with multiple processor groups in Windows. Should I file a bug (feature?) requesting support for multiple processor groups? Will this also get closed as a duplicate of #1115?

[g0blue]

OK, looking at the date for #3314 I see that this was still WSL 1. So, I suppose open a new feature explicitly asking for WSL 2 support for multiple processor groups?

[therealkenc]

To be clear, the issue has nothing directly to do with number of socket

To be clear the issue is "WSL 2 is only using up to 50% of the CPUs on my 8c/16t [Ryzen 1700] machine", which has been fixinbound (really fixedininsiderbuilds) since June 2019 (and confirmed by the OP in October).

Should I file a bug (feature?) requesting support for multiple processor groups?

Can your issue (a single-socket Xeon 8180) and the question I was addressing (Greg's dual Intel Xeon E5-2699) be two new issues instead of one? Sure, whynot.

[edit] I see now you are dual socket too. Same question with same answer as Greg.

[benhillis]

I would suggest a new issue for multi processor groups. I am already looking into this, AMD brought this to my attention as well.

[sanastasiou]

@benhillis @therealkenc is this fixed or is there some issue to subscribe somewhere? Thanks!

[AAlMutairi]

@sanastasiou , Did you find any issue posted? because even when I gave up on WSL 2 and moved back to hyper V, the issue still persists. :(

[therealkenc]

No one has submitted a new issue for the multi processor groups scenario AFAIK. Doing one is somewhat academic since the devs are aware of the limitation, but it could act as a "like" target.

[sanastasiou]

@therealkenc anything against re-opening this issue? Or simply referencing it from a new issue?

[AAlMutairi]

@sanastasiou @therealkenc , I opened a new issue #5423 for the AMD cpu issue.

[parthdevbundela]

hi @benhillis,
In one of your comment u asked whether u have 2 processors
In my workstaion i've 2 processors each with 20 cores, butt typing lscpu on wsl only detects only one processor
pls help

[uniartisan]

lscpu

lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         46 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  56
  On-line CPU(s) list:   0-55
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
    CPU family:          6
    Model:               79
    Thread(s) per core:  2
    Core(s) per socket:  28
    Socket(s):           1
    Stepping:            1
    BogoMIPS:            4788.90
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse ss                         e2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid p                         ni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c
                         rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow
                         vnmi ept vpid ept_ad fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt                          flush_l1d arch_capabilities
Virtualization features:
  Virtualization:        VT-x
  Hypervisor vendor:     Microsoft
  Virtualization type:   full
Caches (sum of all):
  L1d:                   896 KiB (28 instances)
  L1i:                   896 KiB (28 instances)
  L2:                    7 MiB (28 instances)
  L3:                    35 MiB (1 instance)
Vulnerabilities:
  Gather data sampling:  Not affected
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                   Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
  Retbleed:              Not affected
  Spec rstack overflow:  Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS
                         Not affected
  Srbds:                 Not affected
  Tsx async abort:       Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown

I have 2 sockets (2 cpus), each with 14 cores and 28 threads, now I can recognize all cores well in wsl2, but numa can't recognize 2. This also caused all numa2 cores to be identified as numa1 hyperthreads. This has a significant performance impact.

htop

[jinzzasol]

The same issue still persists with WSL2 in 2024. See the attached image above. I was running multiple ML model with Pytorch and Tensorflow.

lscpu

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         46 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  40
  On-line CPU(s) list:   0-39
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Xeon(R) Gold 6242R CPU @ 3.10GHz
    CPU family:          6
    Model:               85
    Thread(s) per core:  2
    Core(s) per socket:  20
    Socket(s):           1
    Stepping:            7
    BogoMIPS:            6185.46
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse ss
                         e2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology tsc_rel
                         iable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe p
                         opcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid
                         _single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust
                         bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx5
                         12bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512_vnni md_clear flush_l1d arch_capabilities
Virtualization features:
  Virtualization:        VT-x
  Hypervisor vendor:     Microsoft
  Virtualization type:   full
Caches (sum of all):
  L1d:                   640 KiB (20 instances)
  L1i:                   640 KiB (20 instances)
  L2:                    20 MiB (20 instances)
  L3:                    35.8 MiB (1 instance)
Vulnerabilities:
  Gather data sampling:  Unknown: Dependent on hypervisor status
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Mitigation; Clear CPU buffers; SMT Host state unknown
  Retbleed:              Mitigation; Enhanced IBRS
  Spec rstack overflow:  Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
  Srbds:                 Not affected
  Tsx async abort:       Mitigation; TSX disabled

I was thinking of editing .wslconfig but Microsoft claims that it should use the same number of processors as native Windows:
Microsoft - main-wsl-settings

From WSL2, top -i shows that approx. 70% of CPU was being used as shown below. It looks like WSL2 only utilises 50% of CPU as well as RAM (*WSL2 allocates 50% of RAM as default).

[muratyurdakul75]

Hi

Unfortunately, there is no development on this issue. There are constant requests on this issue, but they are marked as "duplicate requests" and directed to old notifications. Is there any development?

Actually, there are multiple expectations regarding this NUMA issue;
1> More than one NUMA needs to be supported.
2> It needs to be able to work with more than 64 processors.
3> We need to be able to decide which NUMA it will work on. (There may be those who want it to work on only one NUMA)

It would be really great if such a development could be made.

Thank you in advance for your efforts.

[benhillis]

Thanks for your patience on this - we have identified a fix for this and will be releasing it as an update soon.