Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preliminary SMP support #46

Merged
merged 11 commits into from
Jul 10, 2024
Merged

Preliminary SMP support #46

merged 11 commits into from
Jul 10, 2024

Conversation

ranvd
Copy link
Collaborator

@ranvd ranvd commented Jun 15, 2024

In this commit, semu is able to simulate SMP architecture running on the Linux kernel.

Before simulating SMP on semu, we need to enable SMP support in the Linux kernel. Please cross-compile the Linux kernel with the configuration file located at configs/linux.config.

After recompiling the Linux kernel with SMP support enabled, simply execute make check SMP=1 to simulate a quad-hart RISC-V CPU.

processor	: 0
hart		: 0
isa		: rv32ima_zicntr_zicsr_zifencei_zihpm
mmu		: sv32
mvendorid	: 0x12345678
marchid		: 0x80000001
mimpid		: 0x1
hart isa	: rv32ima_zicntr_zicsr_zifencei_zihpm

processor	: 1
hart		: 1
isa		: rv32ima_zicntr_zicsr_zifencei_zihpm
mmu		: sv32
mvendorid	: 0x12345678
marchid		: 0x80000001
mimpid		: 0x1
hart isa	: rv32ima_zicntr_zicsr_zifencei_zihpm

processor	: 2
hart		: 2
isa		: rv32ima_zicntr_zicsr_zifencei_zihpm
mmu		: sv32
mvendorid	: 0x12345678
marchid		: 0x80000001
mimpid		: 0x1
hart isa	: rv32ima_zicntr_zicsr_zifencei_zihpm

processor	: 3
hart		: 3
isa		: rv32ima_zicntr_zicsr_zifencei_zihpm
mmu		: sv32
mvendorid	: 0x12345678
marchid		: 0x80000001
mimpid		: 0x1
hart isa	: rv32ima_zicntr_zicsr_zifencei_zihpm

@ranvd ranvd force-pushed the PR_SMP branch 2 times, most recently from 1e7abb0 to 1561811 Compare June 15, 2024 07:04
Copy link
Collaborator

@jserv jserv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not submit a single pull request that involves various independent commits. Instead, this pull request should be divided into at least three distinct parts as follows.

  1. Modify the Linux kernel configurations to be SMP-aware, ensuring compatibility with both uniprocessor setups and implementations preceding SMP-related changes.
  2. Manage RISC-V harts and hart state, along with CLINT support and peripheral supports. Separate these into as many distinct git commits as feasible. Of course, the new option "smp=" should be supported by semu command line.
  3. Implement SMP-aware tests. The command make SMP=n check should be expanded where n represents the number of harts for system emulation. Here, SMP=1 should be the default configuration, but it should be possible to specify SMP=2 or more for testing purposes. Instead of using a file named minimal-quad.dts, generate 'minimal.dtsi' based on the specified SMP configurations using bash or Python scripts. This minimal.dtsi file will include definitions from cpu@0 to cpu@(n-1) and can be included in the minimal.dts file.

@jserv jserv changed the title Implement SMP simulation for semu Preliminary SMP support Jun 15, 2024
@ranvd
Copy link
Collaborator Author

ranvd commented Jun 16, 2024

Got it. I will separate this PR into more pull requests, and leave this pull request with the actual implementation of preliminary SMP implementation .

main.c Outdated Show resolved Hide resolved
@ranvd ranvd mentioned this pull request Jun 24, 2024
Copy link
Collaborator

@jserv jserv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase the latest master branch.

ranvd added 2 commits June 25, 2024 02:26
By changing `ie` from `uint32_t` to an array of `uint32_t`, this can
simulate different interrupt enable registers for different contexts.
For example, `ie[0]` simulates the interrupt enable register for the
first context, which is at 0x002000 in the address map, and `ie[1]`
simulates the second context with the address map at 0x002080.
Replace emu.timer with the CLINT device. This CLINT device
preliminarily supports sending 4095 individual timer and
software interrupts to different harts.
@ranvd
Copy link
Collaborator Author

ranvd commented Jun 24, 2024

In the commit log this time, I separate the commits of PLIC, CLINT implementation and LR/SC correction. Does it needed to create new PR for each individual commit?

@jserv
Copy link
Collaborator

jserv commented Jun 25, 2024

In the commit log this time, I separate the commits of PLIC, CLINT implementation and LR/SC correction. Does it needed to create new PR for each individual commit?

You can gather them in one pull request, but you should utilize git rebase -i to rework the commits, making them concise.

Makefile Outdated Show resolved Hide resolved
clint.c Outdated Show resolved Hide resolved
clint.c Outdated Show resolved Hide resolved
clint.c Outdated Show resolved Hide resolved
scripts/dtsi-gen.py Outdated Show resolved Hide resolved
@ranvd
Copy link
Collaborator Author

ranvd commented Jun 25, 2024

You can gather them in one pull request, but you should utilize git rebase -i to rework the commits, making them concise.

Sorry, I can't understand what exactly I should do. The current commit is already rebased to the latest master branch. Does "Rework the commits" mean I need to rewrite the commit message because my commit message is not clear enough to convey the idea of changes?

@jserv
Copy link
Collaborator

jserv commented Jun 25, 2024

Does "Rework the commits" mean I need to rewrite the commit message because my commit message is not clear enough to convey the idea of changes?

The requested changes are involved in several functional and minimal commits which refer to their individual change lists. e.g.,

  • Adapt Linux kernel configurations for SMP-awareness
  • Introduce CLINT for multi-harts
  • Fix atomic instructions for SMP
  • Introduce SMP-aware option and runtime support

While the following changes should be amended into the above. e.g.,

  • Change device tree related file name
  • Enhance coding sytle in clint.c

Each commit should be fully traceable and well-understood.

main.c Outdated Show resolved Hide resolved
main.c Outdated Show resolved Hide resolved
@@ -1,5 +1,7 @@
/dts-v1/;

#include "riscv-harts.dtsi"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some comments such as "RISC-V hart descriptions."

riscv.c Outdated Show resolved Hide resolved
riscv.h Outdated Show resolved Hide resolved
common.h Outdated
@@ -17,6 +17,11 @@ static inline int ilog2(int x)
return 31 - __builtin_clz(x | 1);
}

static inline int ffs(int x)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ffs() function appeared in 4.3BSD. Its prototype existed previously in <string.h> before it was moved to <strings.h> for IEEE Std 1003.1-2001 ("POSIX.1") compliance.

You can include <strings.h> when needed.

riscv.h Outdated
};

struct __vm_internel {
uint32_t hart_number;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to n_harts for consistency.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to rename it to hart_count? Since the variable name storing the --smp argument value is hart_count, which will be used to initialize vm.hart_number.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Count is to enumerate one by one. Number is either total or status of anything in a series. So, we would "number" or the short form "n_" in structures. Meanwhile, we use count during initialization to emphasize the enumeration.

@@ -386,7 +386,7 @@ static bool virtio_blk_reg_write(virtio_blk_state_t *vblk,
#undef _
}

void virtio_blk_read(vm_t *vm,
void virtio_blk_read(hart_t *vm,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the sake of consistency, the first parameter should be exactly vm which points to the instance of virtual machine regardless of its hart configuration.

hart->sip &= ~RV_INT_SSI_BIT;
}

static bool clint_reg_read(clint_state_t *clint, uint32_t addr, uint32_t *value)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specify const qualifier when possible. i.e., const clint_state_t *clint.

@ranvd
Copy link
Collaborator Author

ranvd commented Jun 30, 2024

I found that some implementations are not consistent with the specifications. I will update this branch afterward.

Copy link
Collaborator

@jserv jserv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use git rebase -i to consolidate the commits and classify them properly.

@jserv
Copy link
Collaborator

jserv commented Jul 8, 2024

Can you integrate SMP aware tests in GitHub Actions?

@ranvd
Copy link
Collaborator Author

ranvd commented Jul 8, 2024

Okay, I will integrate it by creating a new pull request afterward.

@ranvd ranvd force-pushed the PR_SMP branch 2 times, most recently from a29be25 to 3091d8c Compare July 8, 2024 14:21
ranvd added 9 commits July 8, 2024 23:55
In this commit, semu is able to simulate SMP architecture on Linux
kernel. The original `vm_t` has been changed to `hart_t`, and now
`vm_t` represents the entire virtual machine. Additionally, HSM,
RFENCE, and IPI SBI extensions have been implemented with rough
implementations. Also, the interrupt signal in PLIC is required to
be sent to every hart. Lastly, ensure the LR/SC instructions in
semu are handled properly.

Before simulation, we need to enable SMP support in the Linux
kernel. Please cross-compile the Linux kernel with the configuration
file located at configs/linux.config

After recompiling the Linux kernel with SMP support enabled, simply
execute `make check SMP=n`, where n means the number of hart you want to
simulate, and SMP=1 is the default setting. If you want to execute semu
by yourself, the --smp argument can be used to specify the hart to
simulate, and you are responsible for modifying the device tree.
Use -c for short representation of SMP option, and Leave --smp for
long representation.
The mmu-type should be sv32 not rv32.
Enhance coding style by replacing "else if" with "if". Since it will
return true if it fulfills the condition of the "if" statement.
@jserv jserv merged commit 914d14e into sysprog21:master Jul 10, 2024
2 checks passed
@jserv
Copy link
Collaborator

jserv commented Jul 10, 2024

Thank @ranvd for contributing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants