Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vcpu hotplug arm 5.15.12 #13

Open
wants to merge 5 commits into
base: ch-5.15.12
Choose a base branch
from
Open

Vcpu hotplug arm 5.15.12 #13

wants to merge 5 commits into from

Commits on Jun 30, 2022

  1. arm64: kernel: Handle disabled[(+)present] cpus in MADT/GICC during init 

    With ACPI enabled, cpus get identified by the presence of the GICC
entry in the MADT Table. Each GICC entry part of MADT presents cpu as
enabled or disabled. As of now, the disabled cpus are skipped as
physical cpu hotplug is not supported. These remain disabled even after
the kernel has booted.

To support virtual cpu hotplug(in which case disabled vcpus could be
hotplugged even after kernel has booted), QEMU will populate MADT Table
with appropriate details of GICC entry for each possible(present+disabled)
vcpu. Now, during the init time vcpus will be identified as present or
disabled. To achieve this, below changes have been made with respect to
the present/possible vcpu handling along with the mentioned reasoning:

1. Identify all possible(present+disabled) vcpus at boot/init time
   and set their present mask and possible mask. In the existing code,
   cpus are being marked present quite late within smp_prepare_cpus()
   function, which gets called in context to the kernel thread. Since
   the cpu hotplug is not supported, present cpus are always equal to
   the possible cpus. But with cpu hotplug enabled, this assumption is
   not true. Hence, present cpus should be marked while MADT GICC entries
   are bring parsed for each vcpu.
2. Set possible cpus to include disabled. This needs to be done now
   while parsing MADT GICC entries corresponding to each vcpu as the
   disabled vcpu info is available only at this point as for hotplug
   case possible vcpus is not equal to present vcpus.
3. We will store the parsed madt/gicc entry even for the disabled vcpus
   during init time. This is needed as some modules like PMU registers
   IRQs for each possible vcpus during init time. Therefore, a valid
   entry of the MADT GICC should be present for all possible vcpus.
4. Refactoring related to DT/OF is also done to align it with the init
   changes to support vcpu hotplug.

Signed-off-by: Salil Mehta <[email protected]>
Signed-off-by: Xiongfeng Wang <[email protected]>
    @jongwu
    Salil Mehta authored and jongwu committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    0867205 View commit details
    Browse the repository at this point in the history

  2. arm64: kernel: Bound the total(present+disabled) cpus with nr_cpu_ids 

    Bound the total number of identified cpus(including disabled cpus) by
maximum allowed limit by the kernel. Max value is either specified as
part of the kernel parameters 'nr_cpus' or specified during compile
time using CONFIG_NR_CPUS.

Signed-off-by: Salil Mehta <[email protected]>
Signed-off-by: Xiongfeng Wang <[email protected]>
    @jongwu
    Salil Mehta authored and jongwu committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    6f70bd5 View commit details
    Browse the repository at this point in the history

  3. arm64: kernel: Init cpu operations for all possible vcpus 

    Currently, cpu-operations are only initialized for the cpus which
already have logical cpuid to hwid assoication established. And this
only happens for the cpus which are present during boot time.

To support virtual cpu hotplug, we shall initialize the cpu-operations
for all possible(present+disabled) vcpus. This means logical cpuid to
hwid/mpidr association might not exists(i.e. might be INVALID_HWID)
during init. Later, when the vcpu is actually hotplugged logical cpuid
is allocated and associated with the hwid/mpidr.

This patch does some refactoring to support above change.

Signed-off-by: Salil Mehta <[email protected]>
Signed-off-by: Xiongfeng Wang <[email protected]>
    @jongwu
    Salil Mehta authored and jongwu committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    8d06116 View commit details
    Browse the repository at this point in the history

  4. arm64: kernel: Arch specific ACPI hooks(like logical cpuid<->hwid etc.) 

    To support virtual cpu hotplug, some arch specific hooks must be
facilitated. These hooks are called by the generic ACPI cpu hotplug
framework during a vcpu hot-(un)plug event handling. The changes
required involve:

1. Allocation of the logical cpuid corresponding to the hwid/mpidr
2. Mapping of logical cpuid to hwid/mpidr and marking present
3. Removing vcpu from present mask during hot-unplug
4. For arm64, all possible cpus are registered within topology_init()
   Hence, we need to override the weak ACPI call of arch_register_cpu()
   (which returns -ENODEV) and return success.
5. NUMA node mapping set for this vcpu using SRAT Table info during init
   time will be discarded as the logical cpu-ids used at that time
   might not be correct. This mapping will be set again using the
   proximity/node info obtained by evaluating _PXM ACPI method.

Note, during hot unplug of vcpu, we do not unmap the association between
the logical cpuid and hwid/mpidr. This remains persistent.

Signed-off-by: Salil Mehta <[email protected]>
Signed-off-by: Xiongfeng Wang <[email protected]>
    @jongwu
    Salil Mehta authored and jongwu committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    744b6fb View commit details
    Browse the repository at this point in the history

  5. cpu/numa: fix failure when hot-remove cpu 

    When hot-remove cpu, the map from cpu to numa will set to NUMA_NO_NODE
which will lead to failure as the map is used by others. Thus we need a
specific map to descrip the unpluged cpu.

Here we introduce a new map to descrip the unpluged cpu map.

Signed-off-by: Jianyong Wu <[email protected]>
    @jongwu
    jongwu committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    13c5067 View commit details
    Browse the repository at this point in the history