Skip to main content

Milestone 2

Milestone 2 Status​

This milestone was completed February 5, 2026

Milestone 2 was completed during this reporting period. The work focused on enabling external interrupt support using the APLIC.

In milestone M2, a lightweight and extensible interrupt controller abstraction was introduced into the IRQ chip layer within OpenSBI, allowing drivers to register claim, completion, mask, and unmask hooks for wired interrupt lines. Using this abstraction, support for the APLIC was implemented. The abstraction is designed to be extensible, allowing for future support of additional wired interrupt sources other than the APLIC without requiring changes to trap-level logic.

In parallel, APLIC initialization logic was implemented based on device tree configuration, with internal data structures populated from the device tree to support interrupt handling at runtime. APLIC configuration was extended to account for different OpenSBI domain configurations, ensuring correct initialization and behavior within the current M-mode interrupt handling model.

The external interrupt handling path was validated in M-Mode using a UART as a wired interrupt source. End-to-end testing exercised the complete interrupt flow: WFI → UART (wired IRQ) → APLIC → IDC.CLAIMI → OpenSBI trap → registered INTC provider → handler → clean UART → complete → return to WFI. This validation confirmed that external interrupts can be correctly claimed, handled, and completed in M-Mode.

Following completion of the core M2 functionality, system-level validation was carried out across both Linux and bare-metal environments. All APLIC implementations within the project scope were exercised, and the OpenSBI changes were validated to have no observed side effects on Linux’s existing interrupt handling mechanisms. This validation included configuration and verification steps to ensure that Linux interrupt enable and pending conditions remained intact under the updated M-mode APLIC setup.

During the validation, an unexpected APLIC register behavior was observed in QEMU. The issue was reported to RISE, and a workaround was implemented in OpenSBI to allow continued validation of the M2 functionality. A fix for QEMU was developed by Samuel Holland of SiFive but this hasn’t been upstreamed or provided to RISCstar at the time of writing this report.

A review meeting was held with RISE to collect feedback on the M2. Based on this feedback, the designs for milestones M2 and M3 were updated and aligned, and an RFC for the interrupt controller abstraction and virtual IRQ layer was posted to the OpenSBI mailing list to validate the accepted design direction with the broader community. Following review comments, an updated RFC was subsequently posted after refining the design and integrating the interrupt controller abstraction into the existing OpenSBI IRQ chip layer framework, with all review comments on the RFC from RISE addressed, to better align with the existing OpenSBI interrupt infrastructure.

To fully close M2, the team then consolidated and finalized the OpenSBI changes for release, including squashing and amending the patch series, updating the buildroot project and associated documentation to support RISE validation, and refining the build, run, and test procedures.

Milestone Description: APLIC M-mode Wired Interrupt Support in OpenSBI​

This milestone is about supporting external interrupts using the APLIC. OpenSBI requires extensions to service external interrupts in M-Mode. OpenSBI presently only has stubs for APLIC external interrupt processing. These stubs need to be extended to allow external interrupts via the APLIC to be minimally handled in OpenSBI. This is needed to enable interrupt driven software domain context switching.

Overview​

The deliverable is a new hierarchical abstraction of interrupt handling is introduced with implementation for APLIC. The extended IRQ chip hierarchy is flexible enough for other wired interrupt sources other than APLIC to be modelled without modifying trap-level logic. A new IRQ chip provider interface was introduced for OpenSBI drivers to register IDC claim, complete, mask and unmask function hooks per wired interrupt line. A test routine was added with full coverage for a demo through:

WFI → UART (wired IRQ) → APLIC → IDC.CLAIMI → OpenSBI trap → registered INTC provider → handler → clean UART → complete → return to WFI

to prove a complete interrupt handling path in M-Mode is working.

The implementation was validated on QEMU (-M virt,aia=aplic)

Background and Motivation​

RISC-V Advanced Interrupt Architecture (AIA) introduces the APLIC (Advanced Platform-Level Interrupt Controller) to manage wired (platform) interrupts and routes them to harts or MSI (Message Signaled Interrupt) endpoints.

In the current OpenSBI implementation, APLIC support primarily focuses on initialization and delegation, while M-mode external interrupt handling for wired interrupts remains largely stubbed. As a result:

  • Real wired interrupts cannot be handled end-to-end in M-mode.
  • There is no generic mechanism for OpenSBI drivers or platforms to register handlers for wired interrupt lines.
  • Trap-level interrupt dispatch remains tightly coupled to specific IRQ chip implementations. The goal of this work is to flesh out minimal first-level wired interrupt support for APLIC in M-mode, while introducing a small, extensible abstraction that avoids hard-coding APLIC-specific logic into the trap handler.

Design Goals​

  1. Enable real wired interrupt handling in M-mode
    • Support end-to-end delivery of platform interrupts (e.g. UART RX).
  2. Keep the design minimal and generic
    • Avoid embedding APLIC-specific details in trap handling.
  3. Provide a reusable abstraction
    • Allow future interrupt controllers (PLIC, SoC-private IRQs, etc.) to integrate with the same model.
  4. Validate using real hardware behavior
    • Use UART RX on QEMU virt for testing.

The following were not goals for milestone M2

  • MSI (Message Signaled Interrupt) / IMSIC (Incoming Message Signaled Interrupt Controller) support
  • S-mode interrupt couriering is the focus of milestone M3

Terminology​

  • Wired IRQ: A physical interrupt line asserted by a device (e.g. UART RX), as opposed to message-signaled interrupts (MSI).
  • Root APLIC: The M-mode APLIC domain responsible for delivering wired interrupts directly to harts.
  • IDC (Interrupt Delivery Controller): Per-hart APLIC component responsible for final interrupt delivery and interrupt claiming.

High-Level Architecture​

Design overview​

  1. Introducing a minimal, generic abstraction (claim/complete/mask/unmask) for wired interrupt handling in OpenSBI.
  2. Using this abstraction to implement APLIC wired interrupt support in M-mode.
  3. Providing a QEMU virt-specific test based on UART RX to validate the complete interrupt lifecycle.

Interrupt Handling Abstraction​

A new provider abstraction is introduced to represent wired interrupt controllers:

struct sbi_irqchip_provider_ops {
	/*
	 * Claim a pending wired interrupt on current hart.
	 * Returns:
	 *   SBI_OK      : *hwirq is valid
	 *   SBI_ENOENT  : no pending wired interrupt
	 *   <0          : error
	 */
	int (*claim)(void *ctx, u32 *hwirq);

	/*
	 * Complete/acknowledge a previously claimed wired interrupt
	 * (if required by HW).
	 * Some HW may not require an explicit completion.
	 */
	void (*complete)(void *ctx, u32 hwirq);

	/*
	 * mask/unmask a wired interrupt line.
	 *
	 * These are required for reliable couriering of
	 * level-triggered device interrupts to S-mode:
	 * mask in M-mode before enqueueing, and unmask
	 * after S-mode has cleared the device interrupt source.
	 */
	void (*mask)(void *ctx, u32 hwirq);
	void (*unmask)(void *ctx, u32 hwirq);
};

Key properties:

  • claim() returns a hardware IRQ ID (hwirq).
  • complete() signals end-of-interrupt.
  • mask() and unmask() enable/disable the interrupt source.
  • Independent of the underlying controller (APLIC, PLIC, etc.).

A central dispatcher:

  • Maps hwirq to the registered handler.
  • Invokes handler.
  • Calls complete().

This abstraction allows OpenSBI to support multiple interrupt controllers without modifying trap-level logic.

APLIC Wired Interrupt Provider​

APLIC is integrated by implementing the irqchip provider interface.

Claim Semantics​

For wired interrupts, APLIC provides the IDC.CLAIMI register:

  • Reading IDC.CLAIMI:
    • Returns a non-zero hwirq if pending.
    • Atomically marks the interrupt as “in service”.
  • Returning 0 indicates no pending interrupt (spurious).

Implementation:

  • claim() reads IDC.CLAIMI and extracts the hwirq.
  • If no pending interrupt exists, the claim fails gracefully.
Completion Semantics​

After handler execution: -The interrupt source must be cleared at the device level.

  • complete() finalizes the interrupt lifecycle (EOI bookkeeping).

For QEMU APLIC: -Writing CLAIMI is not required. -Correct device-side clearing is sufficient.

Mask/Unmask Semantics​

For wired interrupts, APLIC provides SETIENUM and CLRIENUM registers.

  • mask() enables the interrupt source by writing the hwirq to SETIENUM.
  • unmask() disables the interrupt source by writing the hwirq to CLRIENUM.

Trap Handling Integration​

CPU-Level Conditions​

A wired interrupt is delivered to M-mode when:

  • mstatus.MIE == 1
  • mie.MEIE == 1
  • mip.MEIP == 1

This results in: mcause = 0x8000_0000_0000_000b (Machine External Interrupt)

OpenSBI Trap Flow​

The trap handler decodes mcause and dispatches machine external interrupts to the registered external interrupt handler.

With our design:

  • The external interrupt path invokes sbi_irqchip_handle_external_irq()
  • The irqchip registered dispatcher performs: 1, provider->claim() 2, handler lookup and invocation 3. provider->complete()

This removes the need for APLIC-specific logic in the trap handler.

UART-Based Testing Code​

UART RX was selected as an ideal validation source because:

  • It is a real wired interrupt
  • It is level-triggered
  • Its interrupt behavior is easy to observe and reason about
Test Setup (QEMU virt)​
  • Platform: qemu-system-riscv64 -M virt,aia=aplic
  • UART IRQ line: hwirq = 10
  • Root APLIC domain targets M-mode

Key setup steps:

  1. Route UART interrupt-parent to the root APLIC domain.
  2. Configure APLIC:
    • sourcecfg[10] = LEVEL_HIGH
    • target[10] = hart0
    • DOMAINCFG.DM = 0 (direct delivery)
  3. Enable IDC delivery (IDELIVERY, ITHRESHOLD)
  4. Enable CPU MEIP (mstatus.MIE, mie.MEIE)
Handler Responsibilities​

The UART interrupt handler must drain the RX FIFO by reading RBR until empty, which clears the interrupt source and prevents interrupt storms for level-triggered IRQs.

Expected behavior:

  • One interrupt per key press.
  • CPU returns to WFI after handling.

End-to-End Interrupt Flow​

UART RX
  → APLIC (root, wired)
    → IDC.CLAIMI
      → MEIP asserted
        → OpenSBI trap handler
          → irqchip IRQ dispatcher
            → registered UART handler
              → clear RX FIFO
            → complete

This demonstrates a full interrupt lifecycle: assert → claim → handle → clear → complete.

Results and Validation​

  • Wired UART interrupts are successfully delivered to M-mode
  • hwirq is correctly claimed via IDC.CLAIMI
  • Handlers are invoked exactly once per interrupt

No interrupt storms occur after proper device-side clearing. This validates:

  • The new IRQ chip abstraction
  • The APLIC wired provider implementation
  • The correctness of the trap-level integration

Build Steps and Test Instructions​

Get Buildroot source code:

$ git clone https://gitlab.com/riseproject/riscv-optee/buildroot.git -b rp016_m2_aplic_v2

Configure Buildroot:

$ cd buildroot
$ make qemu_riscv64_virt_optee_defconfig

Build:

$ make -j$(nproc)

To avoid build errors due to outdated Buildroot native CMakeLists.txt files, if you have a CMAKE version > 3.30 on your host, build using the following command:

$ make -j$(nproc) CMAKE_POLICY_VERSION_MINIMUM=3.5

This will build all of the required components. All build artifacts can be found under output/build.

Running OpenSBI and Linux​

Start QEMU and launch the kernel:

$ ./output/images/start-qemu-kernel.sh

When the following output appears on the console, QEMU is waiting for a pending connection.

qemu-system-riscv64: -chardev
socket,id=vc0,host=127.0.0.1,port=64321,server=on,wait=on: info:
QEMU waiting for connection on:
disconnected:tcp:127.0.0.1:64321,server=on`

Connect to QEMU via a new console by using telnet to port 64321:

$ telnet 127.0.0.1 64321

The Linux logs will appear on the new Linux console, while the OpenSBI logs will appear on the original OpenSBI console.

By typing any key in the OpenSBI console, you should see the logs below, which indicates successful wired interrupt handling (claimed/handled/completed) in M-mode.

[IRQCHIP] claim hwirq <IRQ_NUM>
[IRQCHIP] calling handler for hwirq <IRQ_NUM>
[APLIC TEST] UART got '<KEY_NAME>'(<KEY_ASCII>)
[IRQCHIP] complete hwirq <IRQ_NUM>

For example, if you press ‘a’, you should see:

[IRQCHIP] claim hwirq 10
[IRQCHIP] calling handler for hwirq 10
[APLIC TEST] UART got 'a'(0x61)
[IRQCHIP] complete hwirq 10

Linux IRQ Tests​

In Linux console, after logging in as root, confirm the APLIC is registered properly:

$ dmesg | egrep -i "aplic|imsic|aia|irqchip|riscv-intc"
[    0.000000] riscv-intc: 64 local interrupts mapped
[    0.591472] riscv-aplic d000000.interrupt-controller: 96 interrupts directly connected to 2 CPUs

Check the IRQ status:

$ cat /proc/interrupts
           CPU0       CPU1
 10:        974       1041 RISC-V INTC   5 Edge      riscv-timer
 12:         20          0 APLIC-DIRECT  33 Level     virtio2
 14:        560          0 APLIC-DIRECT   7 Level     virtio1
 15:        284          0 APLIC-DIRECT   8 Level     virtio0
 16:          0          0 APLIC-DIRECT  11 Level     101000.rtc
IPI0:        66         56  Rescheduling interrupts
IPI1:       244        323  Function call interrupts
IPI2:         0          0  CPU stop interrupts
IPI3:         0          0  CPU stop (for crash dump) interrupts
IPI4:         0          0  IRQ work interrupts
IPI5:         0          0  Timer broadcast interrupts
IPI6:         0          0  CPU backtrace interrupts
IPI7:         0          0  KGDB roundup interrupts

This shows that the APLIC M-mode wired interrupt implementation does not affect the Linux IRQ mechanisms.

Try with a few more test steps to confirm IRQ affinity has no side effects. In the Linux console, enable hvc1:

$ setsid sh </dev/hvc1 >/dev/hvc1 2>&1

Connect to QEMU via a new console by using telnet to port 64322:

$ telnet 127.0.0.1 64322

Start tracking the IRQ status in this new monitor console:

$ watch -n 1 cat /proc/interrupts

The counters of APLIC-DIRECT are incrementing. Keep testing in OpenSBI console by typing keys while monitoring the counters in the monitor console, the IRQ status should not be affected.

Test the IRQ affinity on the source which is triggered most frequently, for example, IRQ13 or IRQ14.

Bind the IRQ to CPU1:

$ echo 1 > /proc/irq/14/smp_affinity_list

The CPU0 counter stops incrementing and the CPU1 counter starts incrementing.

Bind the IRQ to CPU0:

$ echo 0 > /proc/irq/14/smp_affinity_list

The CPU1 counter stops incrementing and the CPU0 counter resumes incrementing.

This proves that OpenSBI APLIC changes have no side effects on the Linux IRQ affinity. More IRQ affinity tests can be performed, please reference this documentation on tuning IRQ affinity.