RFC: Xhctx -- an experimental extension replacing trap/ISR/mret with event-scheduled persistent hart contexts

[martin ribelotta]

Hi all,

I'd like to get early feedback on Xhctx ("Hart Context"), an experimental, non-ratified extension I've been drafting. This is an individual proposal, not backed by a Task Group -- I'm posting here first specifically to find out whether the idea is sound before investing in an RTL reference implementation.

THE IDEA. Replace the classical interrupt model (trap -> ISR -> mret) with persistent execution contexts, activated by events, that behave architecturally as RISC-V harts:

Event -> Event Fabric -> Persistent hart -> Abandonment -> Next hart

Instead of one hart moving between privilege levels and trapping in and out of ISRs, each context IS a hart -- its own PC, register file, mhartid, mepc/mcause -- and all of that state is fully banked in hardware. A "context switch" is hardware selecting a different bank, not software saving/restoring registers. Hart dispatch is arbitrated entirely by a fixed-priority hardware Event Fabric, not a programmable scheduler.

WHY. This was originally motivated by two demanding domains that, it turns out, want the same thing from the hardware: mission-critical aerospace systems, and high-speed network processing in FPGA fabrics.

- Aerospace / mission-critical: What the spec calls "recovery is not architecturally defined" for a faulted hart (Section 8.1) is, from this angle, an ARINC-653-style partition boundary enforced by hardware rather than by a hypervisor or partitioning kernel -- fewer software layers to certify to sustain isolation between subsystems of different criticality on the same core. Bubble-free 1-cycle activation and a precise preemption boundary (Chapter 6) give a WCET argument that doesn't depend on "how many registers does this ISR happen to touch." And a fixed-priority hardware arbiter is a much smaller, more exhaustively verifiable artifact than a general RTOS scheduler -- it shifts scheduling from DO-178C software certification territory into DO-254 hardware territory.

- High-speed network processing in FPGA fabric: Mandatory bypass (Chapter 5) dispatches on the cycle after an event with no software polling -- needed to react to "descriptor ready" / "packet arrived" at line rate. Independent flows, DMA rings, or virtual channels map naturally onto separate hart-contexts, arbitrated by fixed priority (e.g. QoS class as ctx_id priority). Specifically for FPGA targets: this is one physical pipeline plus N banked register files (cheap in BRAM), instead of N fully duplicated soft-core pipelines (expensive in LUTs/DSPs) to handle N independent handlers -- with the ergonomics of C instead of hand-written FSMs.

WHAT'S GENUINELY DIFFERENT:

- mret does not exist as a control-transfer mechanism. No operation anywhere in this model restores a precisely-suspended computation and continues it -- every reactivation is a fresh dispatch driven by an event, including exception "recovery" and inter-hart request/response.

- No new instruction encoding. Everything (mctxctl, mevent, meventdata) goes through existing CSRRW/CSRRS/CSRRC on new CSR addresses. Cross-hart introspection (mepc[ctx_id], mcause[ctx_id], etc.) is built on Smcsrind instead of burning a raw CSR address per hart per array.

- U-mode can still request service from an M-mode hart: ecall from U-mode is a defined, resumable, firmware-routed request channel, distinct from the terminal path used for genuine synchronous exceptions.

- Fully preemptive, fixed-priority scheduling in hardware, with the fairness trade-off (lowest-priority hart has no forward-progress guarantee under sustained load) stated explicitly rather than implied away.

STATUS. Draft and development. Several things are explicitly unresolved -- CSR address numbering (currently placeholder addresses in the legitimate Machine custom range), the exact miselect bit layout for the Smcsrind window, S-mode support, and access-control granularity for software-addressed events. See the Open Issues chapter linked below.

LINKS:
- Spec, chapter by chapter: https://martinribelotta.github.io/xhctx/
- Latest PDF release: https://github.com/martinribelotta/xhctx/releases/latest
- Repository: https://github.com/martinribelotta/xhctx

WHAT I'D SPECIFICALLY LIKE FEEDBACK ON:

1. Is replacing the hart's own local trap/mret flow with delivery to a different hart (via the event mechanism) a reasonable model, or does it break some assumption I'm not seeing?

2. Does depending on Smcsrind for cross-hart CSR introspection seem like a sound use of that mechanism, or is there a better-fitting precedent I should look at?

3. Is this better suited to the Fast-Track process given its scope, or does it need a full Task Group?

4. Any prior art I'm missing that already covers this ground?

Happy to answer questions or be told this has an obvious flaw -- that's exactly what I'm here to find out before going further.

Thanks,
Martin Ribelotta

[L Peter Deutsch]

Disclaimer: I'm at an age where my memory misses and garbles quite a lot, so
some of the details below are undoubtedly wrong!

I believe there is quite a lot of similar prior art on this since the 1970s:
what little I remember faintly comes from the telecom world. The
computational model has the flavor of Hoare's Communicating Sequential
Processes, and I believe a model like this gave rise to a language or
languages used by telecom companies since that era. I also recall the BB&N
Butterfly (?), a multi-processor system that used a similar scheme of short
hardware-scheduled tasks. But this was never a research interest of mine, so
I'm sure others will have much better pointers.

The most significant issue, in my opinion, is that what's on the table here
is not just a hardware design: it's a computational paradigm that also has
deep interactions with source language facilities and software system
design. So IMO it's important to cast a pretty wide net in looking for
relevant material.

What about communication between tasks / threads / ...? Is the intent that
this is a "reduction to a previously solved problem" using the existing
RISC-V memory access model?

--

L Peter Deutsch :: Aladdin Enterprises :: Healdsburg, CA & Burnaby, BC

[BGB]

I can't comment on the particular idea as I am lacking some context here.


In general, I have found that I prefer mostly sticking to a fairly
minimal interrupt handler mechanism. Adding a more complex interrupt
handler rarely results in much benefit, but may add significant
implementation cost (particularly if adding things like "bank switched
register files" or other such craziness).

So, a sane sort of interrupt handling mechanism is, say:
You have a CR / CSR that holds the interrupt handler entry point;
Interrupt saves off PC and other core state into special CSRs;
Sets CPU into Interrupt / M mode, and does a branch to the entry point.

The interrupt handler then does the register save and restore handling.

While it may be tempting to consider things like banking out the
registers or offloading the save/restore mechanism to HW, IMHO, this can
easily become a foot gun. Say, for example, one may easily that working
around such a mechanism, say to perform a context switch, ends up
costing more than had all the interrupts just saved/restored the
registers to memory themselves.

...



In my projects I had approached things in several different layers.
Software / Kernel level;
and HW communication level via inter-processor interrupts.


Software layer:
You use something resembling COM objects;
Intraprocess:
These are normal method calls;
Interprocess:
These are routed over a system call (*1).

*1: Method call interface, at an ECALL:
X10: Object Handle
X11: Method Slot
X12: Return Value Pointer
X13: Argument List Pointer
X14/X15/X16: MBZ
X17: -1

Where, X17:
-1: Method call or TestKern style syscall (basically a method call)
Bare SYSCALL case is essentially a method call to a NULL object.
-2: Return to caller
Puts task in a state expecting to receive/handle next call.
>= 0: Linux style Syscall Number
X10..X15: Arguments
X16: MBZ

For a normal inter-process method call, it pulls up the handler task
associated with the object, and then copies the relevant argument
registers into that task and switches to it. At present, normal system
calls work in a similar way.

Note that control flow doesn't "necessarily" flow in this way, as the
scheduler may decide that rather than proceed directly down that path,
it may instead set things up for the context switch, but then switch to
something else instead (in this case, the callee or caller is instead
run at some later point).


In my case, rescheduling things on SYSCALL was preferable to timer
interrupts, as timer-driven scheduling was more prone to introducing
implicit race conditions. Seemingly it was better to only reschedule on
a timer interrupt if a task has failed to be rescheduled on a SYSCALL
within a reasonable time period.


Granted, one could argue that COM-like objects are ugly and awkward,
granted, but not like there is *that* much better at least in C land
IME. They can be wrapped though in a C API wrapper to make things less
ugly though (say, as opposed to bare objects like in DirectX or similar).

Also, still less awkward for some uses (and lower overhead) than shoving
things over "send()"/"recv()" or through "ioctl()"...



In a multi-core setup, another mechanism could be used:
An inter-processor interrupt.

In this case, the interrupt takes parameters that indicate the
destination core and an address (within the physical memory map) that
gives the address to the message parameters (mostly, again, taking a
form resembling the argument a COM-like SYSCALL).

In my implementation, this message would pass along the bus in a similar
way to a hardware IRQ. On arrival, it would be handled similar to an
IRQ, but then essentially added to a queue visible to the scheduler
(with the call being dispatched when the scheduler does its thing).

It performing a method call to a task pinned to another core, the caller
is then put to sleep, and the request can be delivered via this
interrupt. The return path can be delivered via another inter-processor
interrupt.


Typically, in terms of initial boot, only Core 0 initiates the boot
process. The other cores go into an immediate idle state where they
await to be "woken" by an inter-processor interrupt, which in this case
would give the address of the bootstrap code (supplied by the main OS).


The use of multi-core hasn't really been developed much, but largely
this is because on many of the FPGA boards I have, the FPGA is only
really large enough to fit a single core (multi-core processors are only
really viable on the larger end of the Artix-7 line and similar).

Can go multi-core on something like an XC7A200T, but this is a fairly
large and expensive FPGA.



Well, in my case, I am stuck here for now.

I guess they changed stuff, so can no longer post to any of the SIG
lists (then again, maybe this isn't a huge loss).

Did recently write up a "Why XG3?" spec:

https://github.com/cr88192/bgbtech_btsr1arch/blob/master/docs/2026-09-28_WhyXG3.adoc


Where XG3 is basically RV64G and a custom ISA essentially glued
together, intended to coexist on the same CPU with RV64GC.
Note that as I see it, it does not "replace" RV64GC, rather both would
serve different use-cases.

This being more intended as a sort of higher-level overview.

Though, yes, still likely all a bit niche, and likely unable to gain
wider acceptance. But still I found it an interesting path, and it does
see some nice performance advantages in my testing.

Granted, if the main goal is code density or widespread compatibility,
RV64GC is still a good option.



Though, recently did realize that my LR/RA tagging scheme may have been
related to the long standing "ld-linux.so and similar don't work" issue.

As-in, they expect the link register to be a bare untagged address (and
actually notice/care). I have recently gone and modified my
implementation so that calls within RV64GC mode will use bare/untagged
link registers (not yet posted the code changes for this though). In
this case, they would still be tagged in XG3 Mode, and for any inter-ISA
calls.


I have yet to retest to see if this changes the situation (even if this
part works, would still need to climb the debugging hill and resume
implementing Linux style syscalls).

My own binaries worked differently (native ones are PE based), and I
could do ELF PIE binaries with my own C library. Arguably, there would
be a merit of being able to use binaries built against glibc or musl,
but previously this was not working, and didn't go the uphill battle of
trying to debug it.


At present, trying to port the Linux kernel looks like a bigger pain
though (pretty much all of the system-level stuff and hardware level
interfaces are nonstandard; not particularly inclined to poke around in
the Linux kernel).


Also, in case it matters to anyone:
Don't trust my former X account, it got stolen by scammers...

So, annoyingly, my former X profile no longer reflects me, even if they
left my name and similar on the profile (it is a somewhat annoying
situation).

...

[martin ribelotta]

Peter, and on the register-banking point below -- thanks both, this is exactly the kind of feedback I posted here to get.

On prior art: my own closest reference points are XMOS's xCORE and Infineon's TriCore/AURIX line. xCORE in particular is close in spirit -- hardware-scheduled threads, each with its own full register set, near-zero-cost switching, dispatch triggered by hardware events on channels/ports rather than a classical interrupt -- probably the single closest published architecture to what Xhctx is doing. TriCore's angle is related but distinct: its CSA (Context Save Area) mechanism does fast, hardware-assisted context save/restore through a linked list of save areas in memory, which is still memory-backed rather than a pure register bank.

That said, the register-banking idea itself goes back further and I don't want to claim it's new: the Z80's alternate register set (EXX / EX AF,AF') is a single software-switched shadow bank built for fast interrupt handling; the MCS-51 has four hardware register banks selected by PSW bits, used the same way -- give an interrupt priority level its own bank instead of paying push/pop; SPARC's register windows bank for procedure calls rather than interrupts, but it's the same "banking beats save/restore" hardware bet; and several MIPS implementations have Shadow Register Sets (SRS) -- banked GPR sets selected on interrupt entry via SRSCtl specifically to skip the software prologue. That last one is probably the closest standardized-ISA precedent for the core mechanism, even though it's still layered onto a classical trap/ISR flow rather than replacing it the way Xhctx tries to.

Peter, if the CSP/telecom lineage you mentioned (or the BB&N Butterfly) has something to add on top of that list -- especially anything addressing the failure-recovery question -- I'd still like to hear it. Since posting, working through fault recovery, I ended up landing on something closer to Erlang's actor-model "let it crash" philosophy than to classical OS recovery -- a faulted hart's only recovery path restarts its actor from a known-good entry point rather than resuming the exact computation that failed. If CSP has an answer to that specific question, that's exactly the precedent I'm missing.

On your specific question -- yes, this is meant as a reduction to a previously-solved problem, not a new communication model. Inter-hart IPC is ordinary shared memory: a single-word payload rides in event_data, anything larger travels as a pointer/index into a firmware-defined in-memory descriptor. No new coherence story, no new primitive beyond "dispatch an event, then go read memory the caller already wrote." The part I'm least confident about is the software-facing side -- how a language runtime, or a compiler, should think about "an actor" as the unit of compilation -- and I'd welcome pushback there specifically.

On banked register files being a foot gun: that matches my own instinct for the general case. For an occasional interrupt, software save/restore is cheaper than paying hardware area for a full bank, and I wouldn't reach for this mechanism there either. Xhctx targets the opposite regime deliberately -- switches frequent enough (line-rate packet processing, hard real-time partition boundaries) that software save/restore cost is the thing being optimized away, not the rare case. So I think we agree on the general principle and differ on which workloads justify the area cost.

Also -- the ecall convention you described (fixed x10-x17 argument registers, -1/-2 sentinels for method-call vs. return) is structurally close to how Case E routes ecall here: a fixed request/reply shape over ecall, no new instruction. Did your scheduler end up living in hardware, or is it software dispatching off a queue? That's the piece of your design I'd compare most directly against this one.

One more round, now for the list generally, since it's been quiet a few days:

1. mret vs. the current ecall -> resume design (Case E). A U-mode ecall today freezes the caller's PC and resumes it via an ordinary event once serviced -- functionally a call/return, implemented with no mret and no hardware "reply-to" register. Reviving mret for just this path was considered and dropped: it would need an implicit single-slot reply register that a second ecall arriving before the first is answered would silently clobber. Is the event-based resume actually the better trade here, or is the extra hardware/complexity of a real mret worth it to get a conventional call/return primitive back? Genuinely undecided on this one.

2. Where this is headed: something closer to io_uring, or C++20/Rust coroutines, than to classical interrupts -- submit work, get notified on completion, no thread-per-request. Rust in particular seems like an interesting target for a processor shaped like this (async/await mapping onto persistent, hardware-scheduled contexts instead of a software executor). Has anyone looked at compiling coroutine-style async code down onto hardware-scheduled contexts like these, or is that a dead end I'm not seeing?

3. Normative language: the spec currently uses SHALL/SHALL NOT/SHOULD but never defines them anywhere, and there's no Conventions/Terminology section in this repo. Is there a de facto RISC-V convention I should be citing (RFC 2119, ISO/IEC style, something else), or does this need its own section?

4. Scope beyond RV32E: the reference target is RV32E with a 3-stage pipeline specifically because that combination is what guaranteed the 1-cycle context switch the originating project needed (nanosecond-jitter-free event capture). The model itself doesn't depend on RV32E or a 3-stage pipeline -- RV32I/RV64I with any extension set should work, at the cost of a bigger bank per hart and whatever pipeline depth the target needs. Worth saying explicitly, since I think a couple of replies assumed this is microcontroller-only by design rather than by reference-implementation choice.

5. Multi-core scope: everything described so far is one physical pipeline with N banked harts. Nothing in the spec says what happens with multiple Xhctx cores on one SoC -- is cross-core event delivery in scope at all, or is this strictly single-core, with cross-core communication left entirely to platform/IPI convention?

6. Interrupt-controller coexistence: the Rationale chapter compares Xhctx to PLIC/CLIC conceptually but doesn't say whether a real PLIC can sit upstream of the Event Fabric aggregating external sources, or whether every external IRQ line is expected to get its own routing entry directly. Practically, which one is intended?

One thing that I don't think needs a question, but is worth stating up front: no new instruction encoding means no compiler/binutils work -- everything is CSR intrinsics on existing CSRRW/CSRRS/CSRRC. Wanted to rule that concern out in case "this needs its own backend" was anyone's assumption.

Thanks again -- keep it coming, including "this doesn't work because X."

Martin

--
You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
To view this discussion visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/32b38d38-87d5-4936-ac14-303f4f181459%40gmail.com.

[BGB]

OK.

> On banked register files being a foot gun: that matches my own instinct
> for the general case. For an occasional interrupt, software save/restore
> is cheaper than paying hardware area for a full bank, and I wouldn't
> reach for this mechanism there either. Xhctx targets the opposite regime
> deliberately -- switches frequent enough (line-rate packet processing,
> hard real-time partition boundaries) that software save/restore cost is
> the thing being optimized away, not the rare case. So I think we agree
> on the general principle and differ on which workloads justify the area
> cost.
>

I had tried experimenting with register banking before, but had noted a
problem:
Making the register file any bigger than it is already would come with a
significant cost increase.

Had tried a mechanism where the registers were banked in hardware to an
internal SRAM (or "Block RAM"), but the mechanism for swapping out the
registers via this SRAM ended up being significantly slower than the
more naive "just do it all in software" option.


Mostly this was because the normal Load/Store path is fully pipelined,
but the mechanism within the register file had to stall the pipeline and
do a non-pipelined move for every GPR.

So, effectively, the fastest path between registers and memory here was
via the normal Load/Store instructions.


Not that on an FPGA (at least, Xilinx 7-series), doing the register file
via BRAM would be prohibitively expensive. Seemingly, the Intel/Altera
FPGAs used an intermediate option (between LUTRAM and BRAM).

Say options for a register file:
Flip-Flops: High area cost and poor timing;
Most flexible though.
Register ports cost mostly in terms of MUX'ing.
MUX'ing cost is very steep for these.
LUTRAMs: Best option on Xilinx (N/A on Altera)
Need to mirror arrays for every register port though.
Written on clock edges, but support combinatorial read
Limited to a 1R1W pattern.
Generally support a 5 or 6 bit address.
5b address: 3 data bits per LUT
6b address: 2 data bits per LUT
Has per-port MUX costs, but avoids per-register MUX cost.
BRAM: Bad option on Xilinx
Too large and too finite.
Read and Written on clock edges.

Altera has an intermediate option, larger than LUTRAMs but smaller than
BRAMs, but still requires a similar access pattern to BRAMs (read/write
only on clock edges).

On the Xilinx FPGAs, doing GPRs arrays via BRAM would effectively eat
all of the BRAM (which is better, and much more efficiently, used for
things like the L1 and L2 caches and similar).


In my register file:
Had used LUTRAMs for the main GPR file, 6R3W;
Had used FF's for the CRs (Control Registers).
Many CR's need side-channels to CPU logic.
The CR register ports are 1R1W though.


The L1/L2 caches and main TLB use BRAM though, Say (on XC7A100T):
L1-D: 32K, direct-mapped
L1-I: 16K, direct-mapped
Main TLB: 256 x 4-way (set-associative)
L2: 128K, direct-mapped

And, LUTRAM:
L1 I/D TLBs, 32 entry direct-mapped
64 x 4-way (4K), victim cache
Sits between L1 and L2 caches to absorb conflict misses (*1).

*1: In this case, a 32K direct-mapped L1 and 4-way victim cache being
cheaper than a 2-way L1 cache. Likewise, the efficiency gains from the
small 4-way victim cache were larger than those from a set-associative
L1. Likewise, it serves to some extent to shield the L2 cache from
conflict misses, and the (comparably slow) path to external RAM.

An 8-way victim cache would be better than 4-way here, but 8-way is
significantly more expensive.

Note that one "could" just hold the last 8 cache lines that passed by,
vs cache arrays, but from FPGA quirks, it doesn't add much cost to have
LUTRAM arrays (main costs are mostly MUX related).


Basically, it seems like capacity and conflict misses represent two
mostly orthogonal axes.


The TLB is set associative mostly because associativity is required for
it to work correctly (otherwise conflict misses will ruin ones' day).
The TLBs lean a little more towards conflict misses, whereas the L1 I/D
caches are dominated by capacity misses. However, higher associativity
here is effectively too expensive.


Situation might be different with a smaller register file, or with an
implementation on different underlying hardware.

> Also -- the ecall convention you described (fixed x10-x17 argument
> registers, -1/-2 sentinels for method-call vs. return) is structurally
> close to how Case E routes ecall here: a fixed request/reply shape over
> ecall, no new instruction. Did your scheduler end up living in hardware,
> or is it software dispatching off a queue? That's the piece of your
> design I'd compare most directly against this one.
>

Pretty much all in software here...


Can note I am primarily running RV64GC as a userland ISA, where kernel
space is running a slightly different hybrid ISA (XG3), so the
privileged ISA is somewhat nonstandard.


Can note that in my CPU, the interrupt mechanism is roughly:
Copies PC to SPC;
Copies SR(31:0) to EXSR(63:32);
Copies status code into EXSR(15:0);
Copies exception address into TEA (as relevant);
For exception types that produce an address component.
Copies ISA Mode from VBR into SR;
Sets SR bits for ISR (M) Mode;
Causes SP and SSP to swap places in the decoder.
Jumps to an entry point address, loosely speaking:
{ VBR[47:6], ISR_Offset[EXSR[14:12]], 3'b000 };

Where, EXSR(15:12)
8: General Interrupt
A: TLB Miss
C: IRQ
E: SYSCALL

9: Reserved
B/D: Inter-processor Interrupts
F: Critical Fault or Reset
0-7: Non-Interrupt Status (CPU internal signals, eg, state flushes)

Where, for offsets (from high 4 bits of EXSR):
8: 3'h1 (0x08), 9: 3'h7 (0x38),
A: 3'h3 (0x18), B: 3'h5 (0x28),
C: 3'h2 (0x10), D: 3'h6 (0x30),
E: 3'54 (0x20), F: 3'h0 (0x00)
Generally, the offset 0 entry is used for a CPU reset/critical-fault
vector (what could be called "Double Fault" in x86 terms).


Doesn't use RISC-V CSR names mostly because this part wasn't originally
designed for RISC-V (and, if going back in time far enough, my project
originally evolved out of an ISA known as "Hitachi SuperH SH-4"; though
is at this point unrecognizable as such).

So, in this case, internal CR's being like:
SR: Status Register (CPU Control)
VBR: Vector Base Register (ISR Entry)
SPC: Saved PC
TEA: Trap Exception Address
TBR: Task Base Register;
TTB: Translation Table Base (Page Table)
MMCR: MMU Control Register
EXSR: Exception Status Register
LR: (SPR) Link Register (aliased to X1)
SP: (SPR) Link Register (aliased to X2/SP when not in an ISR)
GBR: (SPR) Global Base Register (aliased to GP/X3)
SSP: (SPR) Saved Stack Pointer (aliased to SP in ISRs)
...

The SPRs are special in that they may be aliased to GPRs, and are
handled differently from the true CRs. In the CPU, there are also
"Pseudo Registers" like PC and IMM/JIMM/etc (they have IDs, but don't
actually have a register behind them; they). ZZR (X0/Zero) also being a
Pseudo Register.

TBR was originally also an SPR, but in the current ISA is effectively a
CR only. In my own binaries, its value is copied into X4/TP, but the
core part of the structure is Read-Only in user-mode (it does have
pointers for some usermode writable parts though, like the TLS variables).


But, the newest form of the project is moving towards running RV64GC or
a hybrid ISA mode.

At present, actually the CSR instructions either map to a MOV to/from
one of the internal CR's, or are turned into an emulation trap (if it
expresses anything other than a MOV to/from a CR).



So, ECALL here basically just triggers a SYSCALL interrupt.


As for what the ISR does:
Saves off registers to the TaskInfo structure (held in TBR);
Figures out which task it needs to dispatch to;
Copies over the relevant registers (Eg: X10..X17);
Evaluates if it should schedule a different task instead;
Usually on the return path from a SYSCALL;
Loads the new task into TBR;
Restores the registers from TBR;
Return from interrupt.

Does mean that effectively every system call is a context switch
(actually, two of them, both for entry and return).


Calling to objects basically works similar, though thus far most of the
existing objects just go into kernel space (typically implementing OS
level APIs).

In the future, more U -> U calls may exist (vs just U -> S).

Note that S -> U calls don't currently exist, mostly because implicitly
the existence of direct S -> U calls would put the integrity of the
kernel at risk.

Instead, most communication from the kernel back to userland is through
event polling. While Unix signals are possible, my kernel doesn't
currently implement these, and it is unclear the best approach. The most
likely strategy though would be to handle it similar to method calls,
and basically (as-needed) spawn a (potentially temporary) signal-handler
thread within the context of the usermode process. Or, maybe a global
"signal" thread, whose identity jumps from one process to another,
wearing the hat of each process whenever a signal is sent to it.

> One more round, now for the list generally, since it's been quiet a few
> days:
>
> 1. mret vs. the current ecall -> resume design (Case E). A U-mode ecall
> today freezes the caller's PC and resumes it via an ordinary event once
> serviced -- functionally a call/return, implemented with no mret and no
> hardware "reply-to" register. Reviving mret for just this path was
> considered and dropped: it would need an implicit single-slot reply
> register that a second ecall arriving before the first is answered would
> silently clobber. Is the event-based resume actually the better trade
> here, or is the extra hardware/complexity of a real mret worth it to get
> a conventional call/return primitive back? Genuinely undecided on this one.
>

As noted, ECALL in my case effectively goes:
U -> M -> S
Then:
S -> M -> U

There is generally only a single syscall handler task, which is
basically the stand-in for all system-call like scenarios.

Except for Page-Faults, which need their own handler, because it is
possible for a TLB Miss -> Page Fault to happen within the Syscall.


Where, say:
TLB Misses (in both L1 and Main TLB);
Goes to TLB Miss handler;
TLB miss handler looks at page table;
Sees that page exists but is paged out;
Needs to context switch to Page Fault handler to load it in.
As with ISRs, PF handler needs to operate No-MMU.

Note that the VFS code needs to poke at the extents of buffers used for
IO before doing IO, mostly because the IO device (such as an SDcard) can
only deal with a single request at a time.

So, if any TLB misses or page faults are going to happen, you want them
to happen before starting an IO operation.

Implicitly, this also means that the pagefile does not have access to
the VFS proper (and the current implementation requires the pagefile to
be fully contiguous).

> 2. Where this is headed: something closer to io_uring, or C++20/Rust
> coroutines, than to classical interrupts -- submit work, get notified on
> completion, no thread-per-request. Rust in particular seems like an
> interesting target for a processor shaped like this (async/await mapping
> onto persistent, hardware-scheduled contexts instead of a software
> executor). Has anyone looked at compiling coroutine-style async code
> down onto hardware-scheduled contexts like these, or is that a dead end
> I'm not seeing?
>

I had before considered CPS (or "Continuation Passing Style"), but
lacked a good way to implement CPS in a way that wouldn't have worked
out as significantly more expensive than a model based on traditional C
threads (and context-switching via COM-like calls).

Well, because while threads have a high per-context cost, they have a
low per-function cost.

Whereas CPS would have been the reverse:
Cheap contexts, but a comparably steep per-function cost.


Hybrid approaches are also possible, such as having threads which mostly
serve to pull and process items from work queues and go back to sleep
once the queue is empty (or a sufficient time-slice has passed).

In this case, this avoids the high per-function cost of CPS, also
without needing to spawn a new thread for each task. I had used this
sort of structure before to deal with things like motor-control tasks.


Could be integrated with an OS, but would get a little tricky, say:
You can't schedule normal user code if it is less than 2ms until a
scheduled event (assuming a 1kHz timer IRQ);
At < 2ms, it would need to schedule the event handler, wait for the
time, run the event, and then return to being idle once this event has
passed.

However, this would mean events could only be scheduled coarsely.

Something like trying to fire off pulse events (like motor steps) at
2kHz would effectively eat the CPU (and stop normal user code from running).


However, I had noted before that my CPU design doesn't deal that well
with rapid-fire interrupts, at least not at 50MHz.

This is one area where an MSP430 has an advantage:
The MSP430 can deal with firing off interrupts at 32kHz on a 16MHz MCU;
Whereas, try that with my CPU at 50MHz, and it grinds to a halt.

Though, this is one case where it can be useful to keep something like a
small 8/16 microcontroller style core around, mostly to do these sorts
of "strobe GPIO pins" style operations (well, and/or use something like
an MSP430).

These sorts of timed pin-pulsing events being needed for controlling
things like steppers and servomotors.

Though, some people had instead taken the approach of structuring it
like a big blob of data to be sent out over a parallel port; in which
case it turns into a similar sort of task as audio mixing, and a 1kHz
timer is enough to drive a 64kHz pin-strobe by pushing out the data
stream to an MMIO device 64 bytes at a time.

> 3. Normative language: the spec currently uses SHALL/SHALL NOT/SHOULD
> but never defines them anywhere, and there's no Conventions/Terminology
> section in this repo. Is there a de facto RISC-V convention I should be
> citing (RFC 2119, ISO/IEC style, something else), or does this need its
> own section?
>

Errm, I think most people can figure this part out themselves?...

> 4. Scope beyond RV32E: the reference target is RV32E with a 3-stage
> pipeline specifically because that combination is what guaranteed the 1-
> cycle context switch the originating project needed (nanosecond-jitter-
> free event capture). The model itself doesn't depend on RV32E or a 3-
> stage pipeline -- RV32I/RV64I with any extension set should work, at the
> cost of a bigger bank per hart and whatever pipeline depth the target
> needs. Worth saying explicitly, since I think a couple of replies
> assumed this is microcontroller-only by design rather than by reference-
> implementation choice.
>

Dunno.


In my core, it is an 8-stage 64-bit CPU, in-order 3-wide superscalar.

Though 3 wide mostly as an excuse for a 6R3W register file and not
needing to drop down to scalar operation to deal with indexed-store (the
3-wide core can still remain dual-issue in the face of indexed store
operations; whereas a design with a 4R2W register file would need to
drop to a single instruction for 3R0W or 4R0W instructions).

At present, there are no actual instructions that use all 6 read ports
though, as while theoretically a Binary128 FMA does exist in the ISA,
all of the Binary128 instructions are trap-and-emulate.



And, most likely, XG3 ISA would not make sense in a small
microcontroller use case.

When I originally designed my CPU, I originally intended it for
real-time motor control, but mostly it ended up shifting towards
"experimental thing I mostly use for running Doom and similar" and me
trying to maximize performance.

Then ended up running GLQuake and similar.

It can sort of run Quake 3 Arena, though Q3A is painfully slow on this
at 50 MHz. Seemingly, would need a CPU running at 200+ MHz to make Q3A
not painful.


Where, in practice, for motor control tasks, it was harder to beat, say,
using a RasPi for high-level stuff, and an MSP430 or AVR8 for the actual
motor driving, with some sort of serial link for sending commands to the
microcontroller.


Can note that I have a cell-phone, and what does it have:
8 cores, Cortex A55
So, dual-issue superscalar, NEON SIMD

Seemingly, there is a fair bit of life left for these sorts of in-order
designs. Even if neither giving the best IPC, nor being cheap for
microcontroller tasks.


I aim a little lower than NEON though, in that in my case all of the
SIMD instructions operate on register pairs.

Say, for example, one is limited strictly to even pairs in the 128-bit
cases, say:
F11:F10, F13:F12, ...
With each register holding two Binary32 values, so a register pair can
represent a 4 element SIMD vector in this case.


No RV-V support in my case, and current plans are explicitly not to
support RV-V in hardware (may be emulated in software though).

Issue being basically that there isn't really a viable way to fit a CPU
core with RV-V onto an Artix-7 class FPGA (similar issue for OoO designs).

One major thing this side-steps, is the need to expand the register
space vs what was already needed to run RV64GC on this thing.

> 5. Multi-core scope: everything described so far is one physical
> pipeline with N banked harts. Nothing in the spec says what happens with
> multiple Xhctx cores on one SoC -- is cross-core event delivery in scope
> at all, or is this strictly single-core, with cross-core communication
> left entirely to platform/IPI convention?
>

Dunno.

> 6. Interrupt-controller coexistence: the Rationale chapter compares
> Xhctx to PLIC/CLIC conceptually but doesn't say whether a real PLIC can
> sit upstream of the Event Fabric aggregating external sources, or
> whether every external IRQ line is expected to get its own routing entry
> directly. Practically, which one is intended?
>
> One thing that I don't think needs a question, but is worth stating up
> front: no new instruction encoding means no compiler/binutils work --
> everything is CSR intrinsics on existing CSRRW/CSRRS/CSRRC. Wanted to
> rule that concern out in case "this needs its own backend" was anyone's
> assumption.
>
> Thanks again -- keep it coming, including "this doesn't work because X."
>

I think the usual motive for banked ISRs on microcontrollers is the
motivation to push for high frequency IRQ handling.

But, this only really makes sense if one is seeing interrupt densities
sufficient to justify having a large register file specifically for
dealing with interrupt handling.

Or, conversely, limiting the number of registers as in RV42E.

For example, you could have 4 bank-swapped RV32E contents within 64
physical registers. So, bank swapping RV32E or RV32EC makes more sense
than bank-swapping RV64GC.


Ironically, despite my CPU core using software managed TLB, still
doesn't have the interrupt density to justify a more expensive banked
ISR mechanism. Well, mostly because the 256 x 4-way TLB seems to be at
least moderately effective...

> docs/2026-09-28_WhyXG3.adoc <https://github.com/cr88192/
> bgbtech_btsr1arch/blob/master/docs/2026-09-28_WhyXG3.adoc>

> send an email to isa-dev+u...@groups.riscv.org <mailto:isa-
> dev%2Bunsu...@groups.riscv.org>.

> To view this discussion visit https://groups.google.com/a/
> groups.riscv.org/d/msgid/isa-dev/32b38d38-87d5-4936-

> ac14-303f4f181459%40gmail.com <https://groups.google.com/a/
> groups.riscv.org/d/msgid/isa-dev/32b38d38-87d5-4936-
> ac14-303f4f181459%40gmail.com>.
>