Milestone 6

Milestone 6 Status

The last M6 patch series was sent to the OpenSBI mailing list on May 16, 2026 and accepted. This is now upstream.

Milestone 6 focused on the design and implementation of context switch management. A lazy context switching design was considered for floating-point and vector state handling. In this approach, access to the floating-point or vector registers on first use triggers a trap, after which the relevant processor state is enabled so that the floating-point and vector context can then be saved and restored as needed.

Unfortunately, the lazy context switching approach for floating-point and vector state handling was determined to have limitations and could not be accepted upstream. The design was changed to adopt an eager context switch model. The patch series implements floating-point and vector register state save and restore, along with the associated CSR handling. This implementation enables context switching between S-mode domains using the OpenSBI domain framework.

Milestone Description

Overview

Milestone 6 (M6) is a comprehensive design and implementation of eager floating-point (FP) and RISC-V vector (V) context switching within OpenSBI. The solution is engineered to provide strict state isolation, deterministic execution, and robust multi-domain support.

In multi-tenant or partitioned environments, the architectural state must remain fully isolated. This implementation ensures that all floating-point and vector register states are preserved and restored across every domain transition, eliminating undefined behaviour and improving system reliability:

Per-domain FP and vector context structures

Save/restore routines for FP and vector registers
Integration into the domain context switching path
Validation of vector length (VLENB)
Eager switching model

The implementataion is complete and validated on QEMU (-M virt -cpu rv64,v=true,zkr=on)

Background and Motivation

With the increasing adoption of RISC-V in:

Secure enclaves
Hypervisors and virtualization stacks
Embedded multi-domain systems

There is a growing need for hardware state isolation guarantees. Floating-point and vector extensions introduce large, complex register files that:

Are shared across execution contexts
Can leak or corrupt data if unmanaged

Multi-domain systems require strict isolation of architectural state. Without proper FP and vector context switching the following behaviour may occur:

Cross-domain data leakage (security vulnerability)
Non-deterministic execution behaviour
Debugging complexity and instability
Violation of isolation guarantees required in safety-critical systems

The eager model ensures that every context switch saves and restores the full FP and vector state, avoiding corruption of registers and CSR.

Eager Context Switch

The initial RFC and proposal for a lazy context sSwitch was submitted to the OpenSBI community. This approach for floating-point and vector state handling was determined to have limitations and was rejected by the upstream community. Specifically, machine mode cannot reliably detect floating-point or vector state usage across privilege levels, as the FS and VS bits can be modified in supervisor mode without accurately reflecting actual register usage. This lack of reliable state visibility introduces correctness and security risks, including the possibility of hidden state changes across privilege boundaries and potential data leakage or side-channel exposure.

In addition, the expected performance benefits of a lazy context switch are diminished in practice, as modern workloads more frequently use floating-point and vector instructions, leading to frequent traps that offset any gains.

The current eager context switch is designed to provide deterministic and secure handling of the register state. This implementation covers a complete floating point and vector register state save and restore, along with the associated CSR handling. This enables context switching between S-mode domains using the OpenSBI domain framework. The implementation has been reviewed by the OpenSBI community.

Design Goals

The implementation is guided by the following principles:

Determinism First – Every domain switch produces predictable results
Full Isolation – No shared architectural state between domains
Simplicity Over Optimization – Avoid complex lazy mechanisms
Hardware Awareness – Adapt to platform-specific vector capabilities
Extensibility – Clean structure for future ISA extensions

Terminology

FLD/FSD : 32 bit FP registers for double precision.
FLW/FSW : 32 bit FP registers for single precision.
Vlenb : Configurable vector length
VL8R/VS8R : 32 bit vector registers.

High-Level Architecture

The solution introduces three major architectural components: OpenSBI itself doesn’t use either floating point or vector state, however, support is needed to context switch from one S-mode domain to another S-mode domain in OpenSBI.

Context Structures

Dedicated per-domain FP context
Dedicated per-domain vector context

Save/Restore Engine

Low-level routines to capture and reinstate register state
CSR (Control and Status Register) preservation

Scheduler Integration Layer

Hooks into OpenSBI domain switching
Ensures correct sequencing of operations

Floating-Point (FP) Context Management

Data Structure

Each domain is assigned a complete FP state container:

32 floating-point registers (f0–f31)
Floating-point control/status register (fcsr)

Save Workflow

Sequentially store all FP registers
Capture fcsr
Use architecture-specific instructions (fsd/fsw)

Restore Workflow

Reload all FP registers
Restore fcsr

Key Considerations

Supports both RV64F and RV64D
Ensures no partial or stale register values

Vector Context Management

Data Structure

Each domain contains:

Vector CSRs (vcsr, vstart)
Full vector register file (v0–v31)
Memory sized dynamically using SBI_MAX_VLENB

Save Workflow

Save control registers (vcsr, vstart)
Dump full vector register file using vector instructions (vs8r.v)

Restore Workflow

Restore vector registers (vl8r.v)
Restore control registers

VLENB Validation

Read hardware vector length at runtime
Compare with configured maximum
Trigger error if mismatch exceeds limits

Design Implications

Prevents buffer overruns
Ensures portability across implementations

Domain Switching Integration

Enhanced Switching Flow

Enable FP and vector execution units
Save outgoing domain FP state
Save outgoing domain vector state
Restore incoming domain FP state
Restore incoming domain vector state
Proceed with standard trap/context switch

Control Register Handling

MSTATUS_FS and MSTATUS_VS bits explicitly managed
Ensures hardware units are active when required

Outcome

Fully deterministic transitions
Zero residual state leakage

Performance Considerations

Overhead Analysis

Eager switching introduces consistent overhead per context switch
Cost scales with vector length (VLENB)

Why This Is Acceptable

Predictable performance is preferred over variable latency
Avoids trap overhead from lazy switching
Better suited for real-time and secure systems

Optimization Opportunities (Future Work)

Selective register tracking
Hybrid eager-lazy models
Hardware-assisted context switching

Security Analysis

Threat Mitigation

Prevents register state leakage across domains
Eliminates side-channel exposure via residual state

Isolation Guarantees

Each domain operates with fully independent FP/vector state
No implicit sharing of hardware resources

Validation & Testing

Test Environment

QEMU RISC-V (virt platform)
Vector-enabled CPU configuration

Validation Scope

Functional correctness of save/restore
Cross-domain isolation verification
Stress testing with repeated switches

Results

Deterministic execution confirmed
No observed state corruption

Build Steps and Test Instructions

Build via Buildroot Project

Get Buildroot source code:

$ git clone https://gitlab.com/riseproject/riscv-optee/buildroot.git -b rp016_m6_v2

Configure Buildroot:

$ cd buildroot
$ make qemu_riscv64_virt_optee_defconfig

Build:

$ make -j$(nproc)

To avoid building errors due to outdated Buildroot native CMakeLists.txt files, if you have a CMAKE version > 3.30 on your host, build with:

$ make -j$(nproc) CMAKE_POLICY_VERSION_MINIMUM=3.5

This will build all of the required components. All build artifacts can be found under output/build.

Running OpenSBI and Bare Metal

Start QEMU and launch the kernel:

$ ./output/images/start-qemu-bm.sh

When the following output appears on the console, QEMU has finished testing and has gone through 10 exchanges of context switching between secure and non-secure S-mode app.
OpenSBI v1.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name               : riscv-virtio,qemu
Platform Features           : medeleg
Platform HART Count         : 2
Platform HART Protection    : pmp
Platform IPI Device         : aclint-mswi
Platform Timer Device       : aclint-mtimer @ 10000000Hz
Platform Console Device     : uart8250
Platform HSM Device         : ---
Platform PMU Device         : ---
Platform Reboot Device      : syscon-reboot
Platform Shutdown Device    : syscon-poweroff
Platform Suspend Device     : ---
Platform CPPC Device        : ---
Firmware Base               : 0x80000000
Firmware Size               : 357 KB
Firmware RW Offset          : 0x40000
Firmware RW Size            : 101 KB
Firmware Heap Offset        : 0x4f000
Firmware Heap Size          : 41 KB (total), 1 KB (reserved), 31 KB (used), 8 KB (free)
Firmware Scratch Size       : 4096 B (total), 1464 B (used), 2632 B (free)
Runtime SBI Version         : 3.0
Standard SBI Extensions     : time,rfnc,ipi,base,hsm,srst,pmu,dbcn,fwft,legacy,dbtr,sse,context
Experimental SBI Extensions : none

Domain0 Name                : root
Domain0 Boot HART           : 0
Domain0 HARTs               : 0,1
Domain0 Region00            : 0x0000000080040000-0x000000008005ffff M: (F,R,W) S/U: ()
Domain0 Region01            : 0x0000000080000000-0x000000008003ffff M: (F,R,X) S/U: ()
Domain0 Region02            : 0x0000000000100000-0x0000000000100fff M: (I,R,W) S/U: (R,W)
Domain0 Region03            : 0x0000000010000000-0x0000000010000fff M: (I,R,W) S/U: (R,W)
Domain0 Region04            : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain0 Region05            : 0x000000000c400000-0x000000000c5fffff M: (I,R,W) S/U: (R,W)
Domain0 Region06            : 0x000000000c000000-0x000000000c3fffff M: (I,R,W) S/U: (R,W)
Domain0 Region07            : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X)
Domain0 Next Address        : 0x0000000080200000
Domain0 Next Arg1           : 0x0000000082200000
Domain0 Next Mode           : S-mode
Domain0 SysReset            : yes
Domain0 SysSuspend          : yes

Domain1 Name                : trusted-domain
Domain1 Boot HART           : 0
Domain1 HARTs               : 0*,1*
Domain1 Region00            : 0x0000000080040000-0x000000008005ffff M: (F,R,W) S/U: ()
Domain1 Region01            : 0x0000000080000000-0x000000008003ffff M: (F,R,X) S/U: ()
Domain1 Region02            : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain1 Region03            : 0x0000000000000000-0xffffffffffffffff M: (R,W,X) S/U: (R,W,X)
Domain1 Next Address        : 0x0000000080a00000
Domain1 Next Arg1           : 0x0000000081f80000
Domain1 Next Mode           : S-mode
Domain1 SysReset            : no
Domain1 SysSuspend          : no

Domain2 Name                : untrusted-domain
Domain2 Boot HART           : 0
Domain2 HARTs               : 0,1
Domain2 Region00            : 0x0000000080040000-0x000000008005ffff M: (F,R,W) S/U: ()
Domain2 Region01            : 0x0000000080000000-0x000000008003ffff M: (F,R,X) S/U: ()
Domain2 Region02            : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain2 Region03            : 0x0000000080800000-0x0000000080ffffff M: () S/U: ()
Domain2 Region04            : 0x0000000000000000-0xffffffffffffffff M: (R,W,X) S/U: (R,W,X)
Domain2 Next Address        : 0x0000000080200000
Domain2 Next Arg1           : 0x00000000bfe00000
Domain2 Next Mode           : S-mode
Domain2 SysReset            : no
Domain2 SysSuspend          : no

Boot HART ID                : 0
Boot HART Domain            : trusted-domain
Boot HART Priv Version      : v1.12
Boot HART Base ISA          : rv64imafdcvh
Boot HART ISA Extensions    : sstc,zicntr,zihpm,zicboz,zicbom,sdtrig,svadu
Boot HART PMP Count         : 16
Boot HART PMP Granularity   : 2 bits
Boot HART PMP Address Bits  : 54
Boot HART MHPM Info         : 16 (0x0007fff8)
Boot HART Debug Triggers    : 2 triggers
Boot HART MIDELEG           : 0x0000000000001666
Boot HART MEDELEG           : 0x0000000000f4b509

OpenSBI S-Mode App: Secure
[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.239999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT

OpenSBI S-Mode App: Non-Secure
[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['1.340000']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.171428']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['3.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.133333']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['5.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.109090']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['7.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.092307']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['9.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.080000']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['11.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.070588']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['13.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.063157']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['15.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.057142']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['17.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.052173']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['19.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

[Secure] Hello Secure World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Secure F] value is ['3.048000']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_EXIT
[S-Mode Non-Secure V] Vector context success

[Non-Secure] Hello Normal World.
OpenSBI S-Mode App: FPU instruction done
[S-Mode Non-Secure F] value is ['21.339999']
sbi_ecall_context_handler: SBI_EXT_TEST_CONTEXT_ENTER
[S-Mode Secure V] Vector context success

End-to-End Context Switch Flow and F and V updates

S-Mode Secure App: Entry
  → F and V enabled
    → FP test loaded with a value and arithmetic operation is performed.
    → Vector test load v8 to v15 with vector pattern
       → ecall to domain context switch SBI_EXT_TEST_CONTEXT_XX
        → FP and V are saved and restored for context switch
          → Switch to Next domain context Non Secure App
            → FP and V enabled, and FP and V operation performed in non-secure domain and context switched again (ecall).
              → Again FP and V are saved and restored for context switch
                → Secure App executed from last context and both FP and V retains its values

Results and Validation

Both Secure and Normal Application performs both F and V operations and does context switching and the values are retained between the context, which can also be seen in the debug log above.

This validates the full cycle of both F and V context switches for save and restore of the registers and csr for the current domain, next domain.

Milestone 6 Status​

Milestone Description​

Overview​

Background and Motivation​

Eager Context Switch​

Design Goals​

Terminology​

High-Level Architecture​

Context Structures​

Save/Restore Engine​

Scheduler Integration Layer​

Floating-Point (FP) Context Management​

Data Structure​

Save Workflow​

Restore Workflow​

Key Considerations​

Vector Context Management​

Data Structure​

Save Workflow​

Restore Workflow​

VLENB Validation​

Design Implications​

Domain Switching Integration​

Enhanced Switching Flow​

Control Register Handling​

Outcome​

Performance Considerations​

Overhead Analysis​

Why This Is Acceptable​

Optimization Opportunities (Future Work)​

Security Analysis​

Threat Mitigation​

Isolation Guarantees​

Validation & Testing​

Test Environment​

Validation Scope​

Results​

Build Steps and Test Instructions​

Build via Buildroot Project​

Running OpenSBI and Bare Metal​

End-to-End Context Switch Flow and F and V updates​

Results and Validation​

Milestone 6 Status

Milestone Description

Overview

Background and Motivation

Eager Context Switch

Design Goals

Terminology

High-Level Architecture

Context Structures

Save/Restore Engine

Scheduler Integration Layer

Floating-Point (FP) Context Management

Data Structure

Save Workflow

Restore Workflow

Key Considerations

Vector Context Management

Data Structure

Save Workflow

Restore Workflow

VLENB Validation

Design Implications

Domain Switching Integration

Enhanced Switching Flow

Control Register Handling

Outcome

Performance Considerations

Overhead Analysis

Why This Is Acceptable

Optimization Opportunities (Future Work)

Security Analysis

Threat Mitigation

Isolation Guarantees

Validation & Testing

Test Environment

Validation Scope

Results

Build Steps and Test Instructions

Build via Buildroot Project

Running OpenSBI and Bare Metal

End-to-End Context Switch Flow and F and V updates

Results and Validation