Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drivers: irqchip: irq-bcm2835: Concurrency fix #5589

Merged
merged 1 commit into from
Sep 1, 2023

Conversation

pelwell
Copy link
Contributor

@pelwell pelwell commented Aug 31, 2023

The commit shown in Fixes: aims to improve interrupt throughput by getting the handlers invoked on different CPU cores. It does so (*) by using an irq_ack hook to change the interrupt routing.

Unfortunately, the IRQ status bits must be cleared at source, which only happens once the interrupt handler has run - there is no easy way for one core to claim one of the IRQs before sending the remainder to the next core on the list, so waking another core immediately results in a race with a chance of both cores handling the same IRQ. It is probably for this reason that the routing change is deferred to irq_ack, but that doesn't guarantee no clashes - after irq_ack is called, control returns to bcm2836_chained_handler_irq which proceeds to check for other pending IRQs at a time when the next core is probably doing the same thing.

Since the whole point of the original commit is to distribute the IRQ handling, there is no reason to attempt to handle multiple IRQs in one interrupt callback, so the problem can be solved (or at least made much harder to reproduce) by changing a "while" into an "if", so that each invocation only handles one IRQ.

(*) I'm not convinced it's as effective as claimed since irq_ack is called after the interrupt handler, but the author thought it made a difference.

See: #5214
#1794

Fixes: fd4c978 ("ARM64: Round-Robin dispatch IRQs between CPUs.")

The commit shown in Fixes: aims to improve interrupt throughput by
getting the handlers invoked on different CPU cores. It does so (*) by
using an irq_ack hook to change the interrupt routing.

Unfortunately, the IRQ status bits must be cleared at source, which only
happens once the interrupt handler has run - there is no easy way for
one core to claim one of the IRQs before sending the remainder to the
next core on the list, so waking another core immediately results in a
race with a chance of both cores handling the same IRQ. It is probably
for this reason that the routing change is deferred to irq_ack, but that
doesn't guarantee no clashes - after irq_ack is called, control returns
to bcm2836_chained_handler_irq which proceeds to check for other pending
IRQs at a time when the next core is probably doing the same thing.

Since the whole point of the original commit is to distribute the IRQ
handling, there is no reason to attempt to handle multiple IRQs in one
interrupt callback, so the problem can be solved (or at least made much
harder to reproduce) by changing a "while" into an "if", so that each
invocation only handles one IRQ.

(*) I'm not convinced it's as effective as claimed since irq_ack is
called _after_ the interrupt handler, but the author thought it made a
difference.

See: raspberrypi#5214
     raspberrypi#1794

Fixes: fd4c978 ("ARM64: Round-Robin dispatch IRQs between CPUs.")
Signed-off-by: Phil Elwell <[email protected]>
@popcornmix
Copy link
Collaborator

Okay with me.

Reverting the round robin commit is another option (in an ideal world combined with a an arm64 fiq solution for dwc).

@P33M
Copy link
Contributor

P33M commented Aug 31, 2023

BCM2836/7 never had any sort of automatic interrupt dispatching to not-busy cores, so round-robin servicing would be a useful compromise in the general sense. Fiq_fsm is just the canary in the coalmine since it needs <100us servicing latency or things start to break.

@pelwell pelwell merged commit e804bd1 into raspberrypi:rpi-6.1.y Sep 1, 2023
10 of 11 checks passed
popcornmix added a commit to raspberrypi/firmware that referenced this pull request Sep 5, 2023
kernel: configs: Raise 8250 UART limit to 5 on BCM2711
See: raspberrypi/linux#5590

kernel: drivers: irqchip: irq-bcm2835: Concurrency fix
See: raspberrypi/linux#5589

kernel: TinyDRM ILI9486 overlay, and minor bcm2835-codec logging tweak
See: raspberrypi/linux#5588
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants