RFC: RISC-V microcontroller profile

[Liviu Ionescu]

(I apologise to the Linux audience, but currently I could not find an
email group dedicated to embedded RISC-V devices).


I did a study on using the RISC-V ISA in embedded devices and I
managed to identify several issues together with possible solutions.

The result is a proposal for a 'RISC-V microcontroller profile',
intended to complement the existing privileged profile:

https://github.com/emb-riscv/specs-markdown/blob/master/README.md


The main issues identified are interrupt latency and lack of C/C++ friendliness.

For microcontrollers, the main solutions to minimise latency are:

- a lighter EABI (preferably designed for only 16 registers, and
making use of the other 16 as a bonus, to look similar for both E and
I cores) together with
- hardware stacking/unstacking the interrupt context (which also
greatly improves C friendliness).

Otherwise RISC-V cores will never be able to compete with ARM Cortex-M
cores, which not only have nested interrupts, but advertise maximum 12
cycles from the interrupt to the final C handler (!) and 10 cycles to
fully exit interrupts (not to mention tail-chaining and, when FP is
present, lazy saving of the FP registers).


Regards,

Liviu


p.s. feel free to reply directly to my email address; if this topic
triggers any interest for embedded devices, I suggest to create a
separate group, like embe...@groups.riscv.org, and move the
discussions there.

[Alex Bradbury]

On 14 March 2018 at 10:29, Liviu Ionescu <i...@livius.net> wrote:
> (I apologise to the Linux audience, but currently I could not find an
> email group dedicated to embedded RISC-V devices).
>
>
> I did a study on using the RISC-V ISA in embedded devices and I
> managed to identify several issues together with possible solutions.
>
> The result is a proposal for a 'RISC-V microcontroller profile',
> intended to complement the existing privileged profile:
>
> https://github.com/emb-riscv/specs-markdown/blob/master/README.md

Thanks for sharing your thoughts here Liviu.

>
> The main issues identified are interrupt latency and lack of C/C++ friendliness.
>
> For microcontrollers, the main solutions to minimise latency are:
>
> - a lighter EABI (preferably designed for only 16 registers, and
> making use of the other 16 as a bonus, to look similar for both E and
> I cores) together with

If this is desirable, I'd be strongly in favour of this being done as
a modification of the not-yet-finalised RV32E ABI rather than having
it be yet another ABI to live alongside ilp32, ilp32e, ilp32f, ilp32d,
lp64, lp64f and lp64d.

I think it's important to get some performance numbers on the options
here. How were you hoping to evaluate them - are there real-world
workloads we might analyse, or do you think that targeted
microbenchmarks are sufficient?

Best,

Alex

[Liviu Ionescu]

On 14 March 2018 at 13:04:22, Alex Bradbury (a...@asbradbury.org) wrote:

> ... strongly in favour of this being done as

> a modification of the not-yet-finalised RV32E ABI

fully agree.

> rather than having
> it be yet another ABI to live alongside ilp32, ilp32e, ilp32f, ilp32d,
> lp64, lp64f and lp64d.

that would be nice, but I'm afraid not enough.

my proposal for a RV32E ABI would be to save no more than 6 registers
(and no less than 4), plus the return address, the status register and
minimal status.

then preferably save the same registers by RV32I/RV64I. at the limit
we can save only 4 registers for RV32E, and 6 for the larger cores,
but 16 registers, as are now marked as 'saved by the caller', are way
too many.

> I think it's important to get some performance numbers on the options
> here. How were you hoping to evaluate them - are there real-world
> workloads we might analyse, or do you think that targeted
> microbenchmarks are sufficient?

that's a good question.

as prequisites, I would assume that the stacking/unstaking mechanism
uses only internal fast RAM, that idealy requires 1 cycle per word,
otherwise results are not comparable.

if Cortex-M can stack/unstack a total of 8 (eight) 32-bits words in
only 12/10 cycles; matching it (at least) would be a good challenge to
start with.

right now, looking at the entry.S code used in Linux, I see that it
saves all 32 registers plus 6 CSRs. I don't have actual measurements,
but I would expect this to take at least 38 cycles, plus some good
more cycles in the assembly logic used before/after calling the C
code. an inacurate estimate would be that now we are somewhere in the
50-60 range, without handling nesting and interrupt pre-emption.
adding an extra software stack for nesting might take a few more good
cycles, probably raising the total stacking time to 80-100 cycles, and
slightly less for unstacking.


regards,

Liviu


p.s. any chance to get a separate embe...@groups.riscv.org group?

[Watson Ladd]

On Wed, Mar 14, 2018 at 5:04 AM, Liviu Ionescu <i...@livius.net> wrote:
> On 14 March 2018 at 13:04:22, Alex Bradbury (a...@asbradbury.org) wrote:
>
>> ... strongly in favour of this being done as
>> a modification of the not-yet-finalised RV32E ABI
>
> fully agree.
>
>> rather than having
>> it be yet another ABI to live alongside ilp32, ilp32e, ilp32f, ilp32d,
>> lp64, lp64f and lp64d.
>
> that would be nice, but I'm afraid not enough.
>
> my proposal for a RV32E ABI would be to save no more than 6 registers
> (and no less than 4), plus the return address, the status register and
> minimal status.
>
> then preferably save the same registers by RV32I/RV64I. at the limit
> we can save only 4 registers for RV32E, and 6 for the larger cores,
> but 16 registers, as are now marked as 'saved by the caller', are way
> too many.

If the Cortex-M0 can save 8 32 bit words in 12 cycles, we should be
able to save 16 in 24. This is the only work that needs to be done in
an interrupt in hardware: now we are ready to jump to
an interrupt handler that looks just like a C function.

[Liviu Ionescu]

On 14 March 2018 at 22:12:31, Watson Ladd (watso...@gmail.com) wrote:

> If the Cortex-M0 can save 8 32 bit words in 12 cycles,

I double checked and for Cortex-M3/M4/M7 the latency is 12/10
(entry/exit). Cortex-M0+ is specified at 15 cycles and Cortex-M0 at 16
cycles. the non-FP stack frame is 8 words in all cases. Cortex-M0/M0+
also have an optional zero jitter feature.

> we should be able to save 16 in 24.

with the current ABI, I estimated at least 18 registers to be saved.
the same rule gives 27 cycles.

> This is the only work that needs to be done in
> an interrupt in hardware: now we are ready to jump to
> an interrupt handler that looks just like a C function.

that would be nice.

for compatibility reasons, my proposal also supports the current ABI,
but a ligher ABI would further reduce latency to values comparable
with Cortex-M.


regards,

Liviu

[Torbjørn Viem Ness]

Hi,

This is probaby an unconventional suggestion but I'm still an unexperienced student, so too young to know better I guess. =)

What if we added one more set of registers "behind" the ones that need to be saved upon entering an interrupt handler?
This way the entire status could just be "shifted out" in one cycle before jumping to the routine, and propagating the data to RAM or cache could be handled in the background to prepare for another context swap if necessary, and the latency would be invisible to the user (unless a new call occurs before it's done saving the data from the previous one).
Then after the handler completes, the previous context can simply be shifted back and be ready to go one cycle later.

Does this sound like a good idea (or even feasible), or would it be too expensive in terms of area and complexity seeing as we're talking about microcontrollers?

--
Torbjørn Ness
M.Sc. student, NTNU

[Rogier Brussee]

Op woensdag 14 maart 2018 21:42:38 UTC+1 schreef Liviu Ionescu:

If I understand correctly, to save ra + N additional (word sized) registers a0, a1, a2, .. a5, t1 t2  on a 16 byte boundary should do

(assuming the C extension is defined[1])

C.add16isp ((N +3)>>2)                  # i.e. addi sp sp ((N +3)>>2 <<4)       

jalr   t0 zero  Csavew - (N<<1)

where Csavew is the bit of milicode 

Csavew-16   C.swsp t2   (+32)

Csavew-14   C.swsp t1   (+28)     

Csavew-12   C.swsp a5  (+24)

Csavew-10   C.swsp a4  (+20)

Csavew -8    C.swsp a3  (+16)

Csavew -6    C.swsp a2  (+12)

Csavew -4    C.swsp a1  (+8)

Csavew -2    C.swsp a0  (+4)

Csavew:       C.swsp ra   (0)

                     j t0

If one architecturally fixes the adres of Csavew just like you defined adresses for mmaped CSR's (representable in 12 bits to avoid 

an additional lui or 11 bits if one does not want to run into top negative range to avoid) 

then that jalr  t0 zero  Csavew - (N<<1) could be _allowed_ to be implemented in hardware without demanding it. 

[1] Mutatis mutandis the same can be done without the C extension but the adresses corresponding to registers would be different.

regards,

Liviu

[Liviu Ionescu]

On 14 March 2018 at 23:20:30, Torbjørn Viem Ness (tbn...@gmail.com) wrote:

> This is probably an unconventional suggestion but I'm still an unexperienced

> student, so too young to know better I guess. =)

don't worry, creativity has no age ;-)

> What if we added one more set of registers "behind" the ones that need to
> be saved upon entering an interrupt handler?

if I'm not terribly wrong, this technique is called 'shadow register
set', and it is used by some other architectures more concerned with
latency (MIPS, PIC32, maybe SPARC, possibly others).

it is probably the fastest solution.

> Does this sound like a good idea (or even feasible), or would it be too
> expensive in terms of area and complexity seeing as we're talking about
> microcontrollers?

I would say it is not cheap.


regards,

Liviu

[Liviu Ionescu]

On 14 March 2018 at 23:30:22, Rogier Brussee (rogier....@gmail.com) wrote:

> ... to save ra + N additional (word sized) registers ... on a 16 byte boundary

in my proposal I considered an 8 byte alignment enough, but if a 16
bytes boundary simplifies the logic or makes things faster, I see no
problem to update the specs to support it (the number of extra words
added must be preserved somewhere to de-adjust the stack pointer on
exit).

regards,

Liviu

[Jacob Bachmeyer]

Torbjørn Viem Ness wrote:
> This is probaby an unconventional suggestion but I'm still an
> unexperienced student, so too young to know better I guess. =)
>
> What if we added one more set of registers "behind" the ones that need
> to be saved upon entering an interrupt handler?

You are asking for shadow registers.

> This way the entire status could just be "shifted out" in one cycle
> before jumping to the routine, and propagating the data to RAM or
> cache could be handled in the background to prepare for another
> context swap if necessary, and the latency would be invisible to the
> user (unless a new call occurs before it's done saving the data from
> the previous one).
> Then after the handler completes, the previous context can simply be
> shifted back and be ready to go one cycle later.
>
> Does this sound like a good idea (or even feasible), or would it be
> too expensive in terms of area and complexity seeing as we're talking
> about microcontrollers?

I am working on a similar proposal (there are some points of
disagreement between Liviu Ionescu and myself) that uses a shadow
register bank almost exactly as you suggest, including
spilling/reloading the inactive shadow registers into stack frames. The
main difference is that I will also propose an EABI with a minimum of
caller-saved registers, and only those EABI caller-saved registers are
shadowed.



-- Jacob

[Liviu Ionescu]

On 14 March 2018 at 23:41:20, Jacob Bachmeyer (jcb6...@gmail.com) wrote:

> ... a similar proposal ... that uses a shadow register bank

any solution that provides a better latency and/or ease of use, and
has an acceptable cost is welcomed.

from my point of view, for a real life device, that includes flash,
ram and lots of complex peripherals, architecture C/C++ friendliness
and performance (like latency) are more important than the number of
transistors in the core (within reasonable limits).


regards,

Liviu

[Jacob Bachmeyer]

Liviu Ionescu wrote:
> (I apologise to the Linux audience, but currently I could not find an
> email group dedicated to embedded RISC-V devices).
>

This is the RISC-V ISA mailing list; while there may be many people here
interested primarily in Linux, I am quite certain that wider discussions
are appropriate.

> I did a study on using the RISC-V ISA in embedded devices and I
> managed to identify several issues together with possible solutions.
>

I have been working on a similar proposal; taking a different approach
to some of the issues Liviu Ionescu has raised. I have attached a
working draft and also seek comments and comparisons between our proposals.


-- Jacob

[Rogier Brussee]

Op woensdag 14 maart 2018 22:39:40 UTC+1 schreef Liviu Ionescu:

The 16 byte alignment is not essential but allows to use addi16sp.

Essentially the same idea can be used with 4 byte allignment

addi sp sp N << 2

jalr   t0 zero  Csavew - (N<<1)

or 8 byte alignment

addi sp sp (((N +1)>>1) << 3)                       

jalr   t0 zero  Csavew - (N<<1)

Regards,

Rogier

 

regards,

Liviu

[Richard Herveille]

The problem with shadow registers is that you always run out and you still need to spill to main memory.

For an RVE implementation, which reduces the RF in half to save gates, it would be weird to double the memory now, just to implement a shadow register.

 

Richard

 

 

 

Richard Herveille

Managing Director

Phone +31 (45) 405 5681

Cell +31 (6) 5207 2230

richard....@roalogic.com

 

--

You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.

To post to this group, send email to isa...@groups.riscv.org.

Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/CAG7hfcJRmvBgfMCg2rtnAh-3xfoahRiF225Yvu9k%3DBbNjy_xDA%40mail.gmail.com.

 

[Liviu Ionescu]

On 15 March 2018 at 10:55:11, Richard Herveille

(richard....@roalogic.com) wrote:

> The problem with shadow registers is that you always run out and you still need to spill
> to main memory.

you run out when the interrupt nesting gets deeper than the available
register banks; in most cases, the depth is 1, rarely 2, even rarely
3, and so on.

spilling can be done in parallel, while starting the handler.

but this method reveals a possible latency problem: if a high priority
interrupt occurs right after a series of other interrupts, and there
are no more register banks, it must wait for a previous spill to
complete, to free a register bank, leading to a jitter on the high
priority interrupt latency.

most applications tolerate a small jitter, but for applications that
implement control loops we might need a way to disable this mechanism
and provide constant latency (even if it is slightly higher).
Cortex-M0 has such a configuration bit to prevent jitter.

> For an RVE implementation, which reduces the RF in half to save gates, it would be weird
> to double the memory now, just to implement a shadow register.

yes, this mechanism is not cheap. however, as Jacob suggested, only
the ABI caller registers need to be shadowed/spilled, so, with a
lighter EABI, the extra cost may be kept to a minimum.

---

using shadow registers seems attractive at first sight, but I'm afraid
it brings more problems that is solves.

for the moment, regardless the implementation, my conclusion is that a
light EABI with a small number of caller registers, plus a fast
hardware stacking/unstacking seem required anyway.


regards,

Liviu

[Michael Clark]

> On 14/03/2018, at 2:41 PM, Jacob Bachmeyer <jcb6...@gmail.com> wrote:
>
> Torbjørn Viem Ness wrote:
>> This is probaby an unconventional suggestion but I'm still an unexperienced student, so too young to know better I guess. =)
>>
>> What if we added one more set of registers "behind" the ones that need to be saved upon entering an interrupt handler?
>
> You are asking for shadow registers.

I had asked for a similar feature called “Privileged Register Windows” that unlike “Register Windows” on SPARC, are only windowed on privileged mode changes, not on all procedure calls. Only a0-a7 would be shared between different privileged modes and the remainder of the registers would be in a per mode window. On an OoO, this could be handled in the renamer, and in a single issue, access to the larger register file could be muxed by privilege mode.

This is very different from “Register Windows” that have fallen out of favour due to the fact that compiler Register allocators can do the job better than using Register Windows on regular procedure calls, rather “Privileged Register Windows” are a feature to minimise save restore when handing synchronous or asynchronous traps between Privilege levels.

Of course they don’t help the case of a nested trap in the same mode, but they could reduce syscall and asynchronous trap latency in a system where code tends to be operating in U mode and the Interrupt bottom half is running in S mode.

>> This way the entire status could just be "shifted out" in one cycle before jumping to the routine, and propagating the data to RAM or cache could be handled in the background to prepare for another context swap if necessary, and the latency would be invisible to the user (unless a new call occurs before it's done saving the data from the previous one).
>> Then after the handler completes, the previous context can simply be shifted back and be ready to go one cycle later.
>>
>> Does this sound like a good idea (or even feasible), or would it be too expensive in terms of area and complexity seeing as we're talking about microcontrollers?
>
> I am working on a similar proposal (there are some points of disagreement between Liviu Ionescu and myself) that uses a shadow register bank almost exactly as you suggest, including spilling/reloading the inactive shadow registers into stack frames. The main difference is that I will also propose an EABI with a minimum of caller-saved registers, and only those EABI caller-saved registers are shadowed.
>
>
>
> -- Jacob
>

> --
> You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
> To post to this group, send email to isa...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/5AA996FD.8040807%40gmail.com.

[Rogier Brussee]

Op donderdag 15 maart 2018 10:41:50 UTC+1 schreef Liviu Ionescu:

On 15 March 2018 at 10:55:11, Richard Herveille
(richard....@roalogic.com) wrote:

> The problem with shadow registers is that you always run out and you still need to spill
> to main memory.

you run out when the interrupt nesting gets deeper than the available
register banks; in most cases, the depth is 1, rarely 2, even rarely
3, and so on.

spilling can be done in parallel, while starting the handler.

but this method reveals a possible latency problem: if a high priority
interrupt occurs right after a series of other interrupts, and there
are no more register banks, it must wait for a previous spill to
complete, to free a register bank, leading to a jitter on the high
priority interrupt latency.

most applications tolerate a small jitter, but for applications that
implement control loops we might need a way to disable this mechanism
and provide constant latency (even if it is slightly higher).
Cortex-M0 has such a configuration bit to prevent jitter.

> For an RVE implementation, which reduces the RF in half to save gates, it would be weird
> to double the memory now, just to implement a shadow register.

yes, this mechanism is not cheap. however, as Jacob suggested, only
the ABI caller registers need to be shadowed/spilled, so, with a
lighter EABI, the extra cost may be kept to a minimum.

_If_  you have all 31 + 1 registers available, what stops you from defining an ABI that that sets aside, say, registers 22--31 

exclusively for the highest level interrupts e.g. using

 

x22 -> irq_ra, 

x23 -> irq_sp, 

x24 -> irq_tp,

x25 -> irq_s0 

x26 -> irq_t0,

x27 -> irq_a0

x28 -> irq_a1

..

x31-> irq_a4

 

(this assumes x3 = gp can still be used as a global pointer)

[Liviu Ionescu]

On 15 March 2018 at 12:32:07, Rogier Brussee (rogier....@gmail.com) wrote:

> If_ you have all 31 + 1 registers available, what stops you from
> defining an ABI that that sets aside, say, registers 22--31

> exclusively for the highest level interrupts e.g. using ...

can you elaborate?

how would this work with nested interrupts?


regards,

Liviu

[Rogier Brussee]

Op donderdag 15 maart 2018 11:38:07 UTC+1 schreef Liviu Ionescu:

It would not work, at least not without spilling the (fewer) reserved registers, exactly as with all other shadow schemes. 

I wrote highest level (highest priority) interrupts assuming the interrupt necessarily

runs uninterrupted to completion. In any case I just wanted to point out that I think you could use the 32 register ISA as 16 registers and 16 "shadowy" registers, or with 

any other split like 22 registers and 10 "shadowy" registers for interrupts or for that matter, 16 registers for normal use, 10 regs for nestable interrupts that have to spill on entry and 

6 for nonnestable, uninterruptable highest priority interrupts. 

 

regards,

Liviu

[Liviu Ionescu]

On 15 March 2018 at 13:23:45, Rogier Brussee (rogier....@gmail.com) wrote:

> ... interrupts assuming the interrupt necessarily
> runs uninterrupted to completion.

real-time systems **need** nested interrupts, this is one of the main
requirements for the microcontroller profile.

> ... you could use the 32 register ISA as 16 registers and 16
> "shadowy" registers

I still think we should design the basic RISC-V EABI with a set of 16
registers (for very small RV32E devices), and, then extend it to a
sibling that has more registers (for RV32I/RV64I), but be sure the
extra registers have no special meaning, so the compiler can use them
only for more local variables.

regards,

Liviu

[kr...@berkeley.edu]

I'll shortly be sending out an invite to a new Foundation Task Group
we have formed to address adding fast interrupts to RISC-V.

Germane to this thread, one feature of the proposal under development
is to standardize interrupt attribute annotations so C compilers can
generate interrupt handlers that only save registers as needed. This
effectively changes the calling conventions just for the handlers but
leaves the rest of the ABI unchanged.

/* Not real code, just a sketch. */
extern volatile int *DEVICE;
extern volatile int *COUNT;

void __attribute__ ((interrupt))
foo() {
*DEVICE = 0;
*COUNT++;
}

A rough sketch of what a generated handler looks like is:

# Small ISR that pokes device to clear interrupt, and increments in-memory counter.

.align 3 # Has to be 8-byte aligned.
foo:
addi sp, sp, -16 # Create a frame on stack.
sw s0, 0(sp) # Save working register.
sw x0, DEVICE, s0 # Clear interrupt flag.
sw s1, 4(sp) # Save working register.
la s0, COUNT # Get counter address.
li s1, 1
amoadd.w x0, (s0), s1 # Increment counter in memory.
lw s1, 4(sp) # Restore registers.
lw s0, 0(sp)
addi sp, sp, 16 # Free stack frame.
mret # Return from handler using saved mepc.

This change will be useful even with existing interrupt architecture,
but TG will be looking at a new design that supports nested
interrupts. Our initial studies show a small core can take interrupt,
enter, execute, and exit the handler above in less than 20 cycles,
while supporting preemption on any clock cycle (i.e., only a few cycles ~3
to get to first instruction).

Krste

| --
| You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
| To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
| To post to this group, send email to isa...@groups.riscv.org.
| Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

| To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/CAG7hfcJ7gWvOW8hGnykJ-NHi8i%2BT6Vd39hm%3D7KFZyygqyoaHHw%40mail.gmail.com.

[Tommy Thorn]

> On Mar 15, 2018, at 16:48 , kr...@berkeley.edu wrote:
> I'll shortly be sending out an invite to a new Foundation Task Group
> we have formed to address adding fast interrupts to RISC-V.
>
> Germane to this thread, one feature of the proposal under development
> is to standardize interrupt attribute annotations so C compilers can
> generate interrupt handlers that only save registers as needed. This
> effectively changes the calling conventions just for the handlers but
> leaves the rest of the ABI unchanged.
>
> /* Not real code, just a sketch. */
> extern volatile int *DEVICE;
> extern volatile int *COUNT;
>
> void __attribute__ ((interrupt))
> foo() {
> *DEVICE = 0;
> *COUNT++;
> }
>
> A rough sketch of what a generated handler looks like is:
>
> # Small ISR that pokes device to clear interrupt, and increments in-memory counter.
>
> .align 3 # Has to be 8-byte aligned.
> foo:
> addi sp, sp, -16 # Create a frame on stack.

If the ABI had included a stack "red zone" with a small reservation for interrupts,
then the two "addi sp, " instructions could have been avoided in most cases.

> sw s0, 0(sp) # Save working register.

Presumedly you meant to load s0 with a global pointer?

> sw x0, DEVICE, s0 # Clear interrupt flag.
> sw s1, 4(sp) # Save working register.
> la s0, COUNT # Get counter address.
> li s1, 1
> amoadd.w x0, (s0), s1 # Increment counter in memory.
> lw s1, 4(sp) # Restore registers.
> lw s0, 0(sp)
> addi sp, sp, 16 # Free stack frame.
> mret # Return from handler using saved mepc.

Tommy

[Andrew Waterman]

On Thu, Mar 15, 2018 at 4:58 PM, Tommy Thorn
<tommy...@esperantotech.com> wrote:
>> On Mar 15, 2018, at 16:48 , kr...@berkeley.edu wrote:
>> I'll shortly be sending out an invite to a new Foundation Task Group
>> we have formed to address adding fast interrupts to RISC-V.
>>
>> Germane to this thread, one feature of the proposal under development
>> is to standardize interrupt attribute annotations so C compilers can
>> generate interrupt handlers that only save registers as needed. This
>> effectively changes the calling conventions just for the handlers but
>> leaves the rest of the ABI unchanged.
>>
>> /* Not real code, just a sketch. */
>> extern volatile int *DEVICE;
>> extern volatile int *COUNT;
>>
>> void __attribute__ ((interrupt))
>> foo() {
>> *DEVICE = 0;
>> *COUNT++;
>> }
>>
>> A rough sketch of what a generated handler looks like is:
>>
>> # Small ISR that pokes device to clear interrupt, and increments in-memory counter.
>>
>> .align 3 # Has to be 8-byte aligned.
>> foo:
>> addi sp, sp, -16 # Create a frame on stack.
>
> If the ABI had included a stack "red zone" with a small reservation for interrupts,
> then the two "addi sp, " instructions could have been avoided in most cases.

In the non-preemptible case, the addi instructions can be elided
as-is. This example works for the (forthcoming) preemptible case, as
well.

>
>> sw s0, 0(sp) # Save working register.
>
> Presumedly you meant to load s0 with a global pointer?

That's what's going on here. "sw x0, DEVICE, s0" is "store x0 to
global symbol DEVICE using s0 as a temporary", i.e., it's syntactic
sugar for "1: auipc s0, %pcrel_hi(DEVICE); sw x0, %pcrel_lo(1b)(s0)"

>
>> sw x0, DEVICE, s0 # Clear interrupt flag.
>> sw s1, 4(sp) # Save working register.
>> la s0, COUNT # Get counter address.
>> li s1, 1
>> amoadd.w x0, (s0), s1 # Increment counter in memory.
>> lw s1, 4(sp) # Restore registers.
>> lw s0, 0(sp)
>> addi sp, sp, 16 # Free stack frame.
>> mret # Return from handler using saved mepc.
>
> Tommy
>
>
>>
>> This change will be useful even with existing interrupt architecture,
>> but TG will be looking at a new design that supports nested
>> interrupts. Our initial studies show a small core can take interrupt,
>> enter, execute, and exit the handler above in less than 20 cycles,
>> while supporting preemption on any clock cycle (i.e., only a few cycles ~3
>> to get to first instruction).
>>
>> Krste
>

> --
> You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
> To post to this group, send email to isa...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/995D7241-2798-4754-B2D0-866B910C4B02%40esperantotech.com.

[Bruce Hoult]

On Thu, Mar 15, 2018 at 4:58 PM, Tommy Thorn <tommy...@esperantotech.com> wrote:

> On Mar 15, 2018, at 16:48 , kr...@berkeley.edu wrote:
> I'll shortly be sending out an invite to a new Foundation Task Group
> we have formed to address adding fast interrupts to RISC-V.
>
> Germane to this thread, one feature of the proposal under development
> is to standardize interrupt attribute annotations so C compilers can
> generate interrupt handlers that only save registers as needed.  This
> effectively changes the calling conventions just for the handlers but
> leaves the rest of the ABI unchanged.
>
> /* Not real code, just a sketch. */
> extern volatile int *DEVICE;
> extern volatile int *COUNT;
>
> void __attribute__ ((interrupt))
> foo() {
>      *DEVICE = 0;
>      *COUNT++;
> }
>
> A rough sketch of what a generated handler looks like is:
>
> # Small ISR that pokes device to clear interrupt, and increments in-memory counter.
>
>        .align 3 # Has to be 8-byte aligned.
>     foo:
>        addi sp, sp, -16             # Create a frame on stack.

If the ABI had included a stack "red zone" with a small reservation for interrupts,
then the two "addi sp, " instructions could have been avoided in most cases.

I believe you've got that backwards.

A "red zone" is stack space below the Stack Pointer that may be used by normal leaf functions without adjusting the Stack Pointer.

If the ABI has a Red Zone then all interrupt service routines must subtract the size of the Red Zone from the Stack Pointer *in addition* to whatever space the interrupt routine will use.

 

 

>        sw s0, 0(sp)                 # Save working register.

Presumedly you meant to load s0 with a global pointer?

>        sw x0, DEVICE, s0            # Clear interrupt flag.
>        sw s1, 4(sp)                 # Save working register.
>        la s0, COUNT                 # Get counter address.
>        li s1, 1
>        amoadd.w x0, (s0), s1        # Increment counter in memory.
>        lw s1, 4(sp)                 # Restore registers.
>        lw s0, 0(sp)
>        addi sp, sp, 16              # Free stack frame.
>        mret                         # Return from handler using saved mepc.

Tommy


>
> This change will be useful even with existing interrupt architecture,
> but TG will be looking at a new design that supports nested
> interrupts.  Our initial studies show a small core can take interrupt,
> enter, execute, and exit the handler above in less than 20 cycles,
> while supporting preemption on any clock cycle (i.e., only a few cycles ~3
> to get to first instruction).
>
> Krste

--
You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to isa...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/995D7241-2798-4754-B2D0-866B910C4B02%40esperantotech.com.

[Alex Bradbury]

Why is 8-byte function alignment required?

Standardising an interrupt attribute similar to those supported by
compilers for most other targets would definitely be worthwhile.
Liviu's document raises the concern that you have to spill the
caller-saved registers in the case where your interrupt handler calls
a function compiled for the standard calling convention. Of course if
your ISR is calling functions where inlining can't be justified, your
interrupt handling is fairly heavyweight already, meaning the extra
overhead of saving caller-saved registers should be a smaller
percentage of execution time.

Better understanding and characterising the workloads people are
struggling with would really help in defining the best solution here.

Best,

Alex

[Krste Asanovic]

That follows from the trap vector alignment constraint in a particular proposal.

> Standardising an interrupt attribute similar to those supported by
> compilers for most other targets would definitely be worthwhile.
> Liviu's document raises the concern that you have to spill the
> caller-saved registers in the case where your interrupt handler calls
> a function compiled for the standard calling convention. Of course if
> your ISR is calling functions where inlining can't be justified, your
> interrupt handling is fairly heavyweight already, meaning the extra
> overhead of saving caller-saved registers should be a smaller
> percentage of execution time.

Yes. Also, at some point regardless of calling convention, you should schedule long-running compute to a background thread scheduled at a more appropriate time, both to reduce interrupt latency and to avoid wasting processor time shuffling registers in ISR routines.

> Better understanding and characterising the workloads people are
> struggling with would really help in defining the best solution here.

Of course. The difficult part is persuading owners to share their workload.
Any offers?

Krste

> Best,
>
> Alex

[Liviu Ionescu]

On 16 March 2018 at 01:48:53, kr...@berkeley.edu (kr...@berkeley.edu) wrote:

> > void __attribute__ ((interrupt))
> foo() {
> *DEVICE = 0;
> *COUNT++;
> }

...

> > Our initial studies show a small core can take interrupt,
> enter, execute, and exit the handler above in less than 20 cycles,
> while supporting preemption on any clock cycle (i.e., only a
> few cycles ~3
> to get to first instruction).

you can make a design lile this and claim less than 20 cycles latency,
but most real applications need to call a plain C function from the
interrupt handler.

to be correct, you must measure latency from the interrupt to the
moment execution enters the plain C function.

can you estimate latency in this case? both entry and exit latencies
are important.


for the final design you must also consider cases like tail chaining
and late arrival.


regards,

Liviu

[Albert Cahalan]

On 3/15/18, Liviu Ionescu <i...@livius.net> wrote:

> real-time systems **need** nested interrupts, this is one of the
> main requirements for the microcontroller profile.

No they don't. I've supported real-time systems as an OS developer
for two different OSes, MC/OS and RedHawk. Customers had a
fondness for running with interrupts disabled entirely. This caused
all sorts of fun for the OS's internal housekeeping tasks. There could
be no clock tick, yet the customer expects the clock to keep working!

In one case, the customer was trying to warp a mirror to aim a laser.
This had to overcome turbulent air bending the beam away from the
intended target. Failure is literally fatal, due to an incoming missile.

For one of those OSes, real-time tasks would run on cores that were
being babysat by the other cores. The cores with real-time tasks would
simply not take any interrupts at all.

The fastest way to get data is to spin waiting for it. That is, you poll
for just one thing continuously. You don't mess around with interrupts.

> I still think we should design the basic RISC-V EABI with a set of 16
> registers (for very small RV32E devices), and, then extend it to a

Normal compilers hardly even use 16 when that is what is available.
I think going past 16 registers was not good, but this is water under
the bridge now. Dropping to 16 is a completely different architecture.
There comes a time to push ahead with what you have, flaws and all.

[Liviu Ionescu]

On 16 March 2018 at 09:45:06, Albert Cahalan (acah...@gmail.com) wrote:

> real-time tasks would run on cores that were
> being babysat by the other cores. The cores with real-time tasks
> would simply not take any interrupts at all.

yes, for hard real-time tasks this is probably the case, but I'm not
sure all applications can afford multi-core devices plus the added
cost of writing multi-core software; I would estimate that only the
top 10% of real-time applications are that extreme, and the rest of
them can still use simpler solutions, if properly designed.


regards,

Liviu

[Liviu Ionescu]

On 16 March 2018 at 05:41:49, Alex Bradbury (a...@asbradbury.org) wrote:

> Standardising an interrupt attribute similar to those supported
> by
> compilers for most other targets would definitely be worthwhile.

interrupt attributes are common to the architectures of yesterday, if
RISC-V wants to be the architecture of the future, it should not look
only to the past.

modern microcontroller architectures use no attributes at all, the
hardware is able to call plain C functions directly.

for the privileged profile you can invent any attributes you like and
try to enforce inlining for the entire interrupt handler, but for the
microcontroller profile I think that hardware stacking/unstaking
coupled with a lite ABI can provide the best performace (with a target
of 12+10 cycles for the total entry/exit to a C function). plus that
it is unbeatable in terms of ease of use.


regards,

Liviu

[kr...@berkeley.edu]

>>>>> On Fri, 16 Mar 2018 00:37:45 -0700, Liviu Ionescu <i...@livius.net> said:
| you can make a design lile this and claim less than 20 cycles latency,
| but most real applications need to call a plain C function from the
| interrupt handler.

It is less than 20 cycles for this use case. There are many different
use cases, including common patterns represented by this example that
explicitly avoid complex code in the ISR itself.

| to be correct, you must measure latency from the interrupt to the
| moment execution enters the plain C function.

No, that is not the standard definition of interrupt latency. The
measure you propose is only interesting for a particular use case
where ISRs are compiled as standard C functions. I understand this
use cases exists, but it is not the only one.

| can you estimate latency in this case? both entry and exit latencies
| are important.

Obviously this depends on the ABI when calling a standard compiled C
function, and for the standard ABI you will have to save/restore a lot
of registers.

| for the final design you must also consider cases like tail chaining
| and late arrival.

Yes - by providing comparable performance for the situations that led
to these architecture-specific optimizations, but not by necessarily
copying an existing design.

Krste


| regards,

| Liviu

[Liviu Ionescu]

On 16 March 2018 at 07:48:33, Krste Asanovic (kr...@berkeley.edu) wrote:

> at some point regardless of calling convention, you should
> schedule long-running compute to a background thread scheduled
> at a more appropriate time, both to reduce interrupt latency
> and to avoid wasting processor time shuffling registers in ISR
> routines.

yes, two-tiered interrupt processing is the ideal textbook solution,
but few RTOSes/applications do it.

how do you suggest to notify the background thread to wakeup and pick
up the job from where the ISR left it?


regards,

Liviu

[kr...@berkeley.edu]

If the background thread was sleeping on WFI, it will now wake up and
check for work when control returns from ISR. If it was not sleeping,
it'll find work on end of its queue.

Krste

[kr...@berkeley.edu]

It is common to throw hardware at a problem to save software design
effort.

Krste

| regards,

| Liviu

| --
| You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
| To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
| To post to this group, send email to isa...@groups.riscv.org.
| Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

| To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/CAG7hfcK4cK3FuFR9EKZo5Zg9VtNZ6eBG1CYPD2mpFntSqxc%2B0A%40mail.gmail.com.

[kr...@berkeley.edu]

I agree this model is convenient to use, but there is a cost for the
convenience, and some would prefer to have the option to choose faster
interrupts, simpler hardware, and faster regular code instead in some
cases.

Krste


| regards,

| Liviu

[kr...@berkeley.edu]

>>>>> On Fri, 16 Mar 2018 03:45:04 -0400, Albert Cahalan <acah...@gmail.com> said:
|| I still think we should design the basic RISC-V EABI with a set of 16
|| registers (for very small RV32E devices), and, then extend it to a

| Normal compilers hardly even use 16 when that is what is available.
| I think going past 16 registers was not good, but this is water under
| the bridge now. Dropping to 16 is a completely different
| architecture.

Yes, the E variant exists.

| There comes a time to push ahead with what you have, flaws and all.

More than 16 registers is noticeably superior for high-performance
code using floating-point or a vector unit, and will also be very
useful in smaller systems using the P extension out of the x
registers.

Krste

[Liviu Ionescu]

On 16 March 2018 at 10:38:58, kr...@berkeley.edu (kr...@berkeley.edu) wrote:


> No, that is not the standard definition of interrupt latency. The
> measure you propose is only interesting for a particular use case
> where ISRs are compiled as standard C functions. I understand this
> use cases exists, but it is not the only one.

agree, it is not the only one.

but in today microcontroller world it is by far the most common one.

> ... but not by necessarily copying an existing design.

sure, we should struggle to do better than existing designs.

but when this is not achievable, we should do at least as good as
existing designs, not go one generation behind (yes, the current
M-only privileged profile with the current non-nested PLIC, is one
generation behind Cortex-M, both in terms of interrupt performance and
ease of use).

On 16 March 2018 at 10:50:43, kr...@berkeley.edu (kr...@berkeley.edu) wrote:

> If the background thread was sleeping on WFI, it will now wake
> up and
> check for work when control returns from ISR. If it was not sleeping,
> it'll find work on end of its queue.

right. so you enqueued an element to a queue.

well, in all RTOSes that I had to do, this is done with a plain C
function, complicated enough to be non-inlineable. and requiring some
kind of protection, a critical section implemented via modifying the
interrupt priority on single cores, or atomics on multi core.

you might ask all RTOS maintainers to modify their designs to run all
this code in an inlined handler, but I doubt this will happen any time
soon.

(the same as you might ask all debug and IDE maintainers to add
special processing and views for the many RISC-V CSRs, but I also
doubt this will happen any time soon)

On 16 March 2018 at 10:53:31, kr...@berkeley.edu (kr...@berkeley.edu) wrote:

> It is common to throw hardware at a problem to save software design
> effort.

hardware stacking/unstaking is one such case. a little bit of hardware
greatly helps the software.

On 16 March 2018 at 11:01:06, kr...@berkeley.edu (kr...@berkeley.edu) wrote:

> I agree this model is convenient to use, but there is a cost for the
> convenience, and some would prefer to have the option to choose faster
> interrupts, simpler hardware, and faster regular code instead in some
> cases.

that's why I suggested a separate microcontroller profile. one of the
main design requirements is ease of use via C/C++ friendliness.

if the privileged profile has other requirements, like minimising
hardware, fine, but don't enforce it to all, and allow  for other
profiles to exist.

---

for those who did not take the time to read my proposal, I copy here
that this is actually the first step to be considered by the
foundation, to acknowledge that the world does not turn around Linux
and privileged profiles, and to make possible the existence of other
profiles too (first and foremost by not mandating the privileged
profile).


regards,

Liviu

[Alex Bradbury]

On 16 March 2018 at 09:38, Liviu Ionescu <i...@livius.net> wrote:
> for those who did not take the time to read my proposal, I copy here
> that this is actually the first step to be considered by the
> foundation, to acknowledge that the world does not turn around Linux
> and privileged profiles, and to make possible the existence of other
> profiles too (first and foremost by not mandating the privileged
> profile).

I thought the intent was that implementers could pick and choose which
standards to implement, as long as they are clear on what they _do_
conform to. e.g. an implementer might ship a device implementing the
standard RV32IMA instruction set (as described in
https://content.riscv.org/wp-content/uploads/2017/05/riscv-spec-v2.2.pdf),
plus their own non-standard privileged implementation, their own
non-standard debug, and other non-standard extensions. I understood
they would still be able to advertise as something like "Compliant
with the RV32IMA ISA described in version 2.2 of the RISC-V User-Level
ISA", despite not being compliant with other specifications published
by the RISC-V Foundation.

Perhaps someone could confirm whether I am correct in this?

Thanks,

Alex

[Liviu Ionescu]

On 16 March 2018 at 12:50:58, Alex Bradbury (a...@asbradbury.org) wrote:

> I thought the intent was that implementers could pick and choose which
> standards to implement, as long as they are clear on what they _do_
> conform to. e.g. an implementer might ship a device implementing the
> standard RV32IMA instruction set (as described in
> https://content.riscv.org/wp-content/uploads/2017/05/riscv-spec-v2.2.pdf),
> plus their own non-standard privileged implementation, their own
> non-standard debug, and other non-standard extensions. I understood
> they would still be able to advertise as something like "Compliant
> with the RV32IMA ISA described in version 2.2 of the RISC-V User-Level
> ISA", despite not being compliant with other specifications published
> by the RISC-V Foundation.
>
> Perhaps someone could confirm whether I am correct in this?

ISA Volume I, v2.2, at page 3, states:

"The RISC-V manual is structured in two volumes. This volume covers
the user-level ISA design, including optional ISA extensions. The
second volume provides the privileged architecture."

I suggested Krste to make Volume II optional, but he insisted Volume
II is always needed.

https://github.com/riscv/riscv-isa-manual/commit/a439dada57fe6c1ed426351742a5ba7dd2cace37#commitcomment-27447508

my microcontroller profile does not use 'mstatus' and can do very well
without any of the requirements of the privileged specs.


it is an irony that the RISC-V specs allow any weird custom
extensions, but does not allow designs to deviate from the privileged
specs.


de-entangling the privileged specs from the instruction set is the
main request addressed by my microcontroller profile proposal.

https://github.com/emb-riscv/specs-markdown/blob/master/improvements-upon-privileged.md#de-entangle-the-privileged-specs


even the name itself is biased towards applications running an
operating system, "User-level ISA" automatically implies a non-user
level ISA.

no, the first volume, the only one that should be mandatory for
compliance reasons (Allen?), should not be related to any user or
privileged mode, and should cover only the instruction set, as neutral
as possible.

I suggest the name: "The RISC-V Architecture Manual: Volume I: The
Instruction Set".

(if anyone from the Foundation board reads this, please consider this
name suggestion a formal proposal to be analysed by the board).


regards,

Liviu

[Alex Bradbury]

I think this is important, and actually would make the RISC-V
Foundation's job much easier. If compliance with the privileged spec
is required to badge something as 'RISC-V', this puts additional
pressure on defining something that can keep every potential user
happy. As you point out, the contrast with the flexibility available
to implementers for ISA extensions is jarring. You could produce a
core that implements RV32IM but deviates hugely from the standard
extensions in other ways, maybe providing a more application-tuned
alternative to the compressed extension, a novel take on atomics, and
posits from that IEEE floating point. Such a core could be marketed as
a RISC-V implementation with no problems, but you'd lose that ability
if you felt that the privileged spec didn't meet your requirements and
deviated in some areas?

It seems inconsistent to embrace and encourage innovation in the
user-level ISA, but discourage it for the privileged spec. I'd point
out that custom user-level extensions can have just as much impact on
compatibility with 'standard' RISC-V software stacks as deviations
from the privileged spec. e.g. if an extension introduces new
relocation types, or introduces extra user state which must be saved
when context switching.

For anyone who didn't click through and read Liviu's work at
https://github.com/emb-riscv/specs-markdown/blob/master/README.md I'd
strongly recommend doing so - I think Liviu perhaps undersold the
amount of content there in the opening to this thread.

Best,

Alex

[Alex Bradbury]

"posits rather than IEEE floating point".

Sorry, that sentence got mangled.

[Guy Lemieux]

Krste,

I'm glad to see a new task group being spun out. However, I believe
the focus should be broader than just fast interrupts. The
microcontroller environment needs to be better specified, as the
current Privileged ISA Specification is inappropriate for that domain.

My small RISC-V processor survey done for DAC2017 was able to capture
26+ RISC-V CPU designs, of which 35% were low-performance micro
controllers and 54% were high-performance microcontrollers/embedded
CPUs. Only 3 of these designs were RV32E.

In this email thread, we have already seen two entirely different
proposals for a microcontroller environment. This will quickly
fracture even more as users are forced to "roll their own". The RISC-V
Foundation needs to address this immediately to protect the brand from
excessive variation at the microcontroller level, which arguably is
the most popular level and the one to the quickest impact.

Guy

[Liviu Ionescu]

On 16 March 2018 at 16:46:46, Guy Lemieux (glem...@vectorblox.com) wrote:


> I'm glad to see a new task group being spun out. However, I believe
> the focus should be broader than just fast interrupts.

+1

> The
> microcontroller environment needs to be better specified, as the
> current Privileged ISA Specification is inappropriate for that domain.

Fully agree.

I mentioned this in many of my previous posts, some times with
arguments, but the mainly Linux audience was generally annoyed and
dismissed them.

> My small RISC-V processor survey done for DAC2017 was able to capture
> 26+ RISC-V CPU designs, of which 35% were low-performance micro
> controllers and 54% were high-performance microcontrollers/embedded
> CPUs. Only 3 of these designs were RV32E.

In my proposal I identified three sub-classes of microcontrollers:

https://github.com/emb-riscv/specs-markdown/blob/master/introduction.md#sub-profiles

RV32E is tentatively used only in the S (small) sub-profile.

> In this email thread, we have already seen two entirely different
> proposals for a microcontroller environment. This will quickly
> fracture even more as users are forced to "roll their own". The RISC-V
> Foundation needs to address this immediately to protect the brand from
> excessive variation at the microcontroller level, which arguably is
> the most popular level and the one to the quickest impact.

... and received the less amount of attention... so far...

Thank you, Guy.


Liviu

[Samuel Falvo II]

On Fri, Mar 16, 2018 at 1:46 AM, Liviu Ionescu <i...@livius.net> wrote:
> yes, two-tiered interrupt processing is the ideal textbook solution,
> but few RTOSes/applications do it.

Citation needed? This is such a common pattern that I find this
statement questionable. Perhaps its rare for a certain profile of
embedded applications; however, in my embedded experience, I've *only*
used two-tiered interrupt processing systems.

> how do you suggest to notify the background thread to wakeup and pick
> up the job from where the ISR left it?

AmigaOS and VMS both use "event flags" (AmigaOS calls them "signals",
so if I use this term, please understand that I'm not referring to
POSIX signals). (DISCLAIMER: Neither of these OSes are hard
real-time; however, they have nonetheless been used in hard real-time
projects.) L4 treats interrupts as messages delivered to message
queues; if a task waits on an interrupt queue, it'll block until the
interrupt fires. Then, part of the kernel's job is to reschedule any
waiting tasks. If no task is waiting, it sets a "interrupt has
happened" flag, so that next time a task tries to wait, it just
doesn't bother. I could be wrong, but if memory serves me right, QNX
maps interrupts to messages sent to message queues as well.

I'm thinking that Liviu's concern will manifest most when projects
cannot drive the processor clock at a faster rate to compensate for a
scheduler's overhead. This could be due to battery life concerns,
etc. However, I've personally never worked on an embedded project
where this was an issue. What kinds of projects would cause this to
become important?

--
Samuel A. Falvo II

[Liviu Ionescu]

On 16 March 2018 at 20:58:13, Samuel Falvo II (sam....@gmail.com) wrote:

> On Fri, Mar 16, 2018 at 1:46 AM, Liviu Ionescu wrote:
> > yes, two-tiered interrupt processing is the ideal textbook solution,
> > but few RTOSes/applications do it.
>
> Citation needed?

sure.

the venerable eCos calls them ISRs and DSRs; the µC/OS-III calls them
direct and deferred interrupts; FreeRTOS has an optional Deferred
Interrupt Handling.

> This is such a common pattern that I find this
> statement questionable.

I'm not sure we are talking about the same thing.

the traditional interrupt processing use case is to register an ISR,
which does all the operations required by the peripheral, then
notifies an associated user thread (via a semaphore, queue, etc), to
continue processing.

the two-tired interrupt processing, also called deferred interrupt
processing, uses **two** routines, one direct and one deferred.

the direct routine (some times shared by multiple peripherals)
practically does the absolute minimum, possibly does not even touch
the peripheral, it only identifies the peripheral, arranges for the
associated DSR to be executed and returns as soon as possible.

immediately after the interrupt returns, the system executes the DSR,
usually on the context of a special system thread, running with a
priority higher than all application threads.

the DSR accesses the peripheral and does all first hand related
processing, then notifies the user thread, similarly as in the
traditional use case.

the big difference is that the DSR does not run on an interrupt
context, but on a regular thread context. in other words, the core is
free to accept further interrupts.

this mechanism was probably fashionable with some legacy architectures
(MIPS?), that used a single interrupt handler, and executed the
interrupt handlers with interrupts disabled (I know it since the 90s).

modern architectures went for nested interrupts and running the
handlers with the interrupts enabled, so the need for inovative tricks
to return from the handler as soon as possible was no longer so
paramount.

when it came in 2004, already Cortex-M adopted this modern mechanism
and eclipse most other microcontrollers in terms of ease of use (since
then I don't remember hearing about the need for deferred interrupts).

imagine my surprise in 2017 to see that RISC-V opted for a single trap
handler and running with interrupts disabled... :-(

> Perhaps its rare for a certain profile of
> embedded applications; however, in my embedded experience, I've *only*
> used two-tiered interrupt processing systems.

if you really used two-tiered interrupts, congratulations!

> I'm thinking that Liviu's concern will manifest most when projects
> cannot drive the processor clock at a faster rate to compensate for a
> scheduler's overhead. This could be due to battery life concerns,
> etc. However, I've personally never worked on an embedded project
> where this was an issue. What kinds of projects would cause this to
> become important?

I'm not sure I understand the question, but I had a project where the
top priority event was hooked to NMI.


regards,

Liviu

[Alex Elsayed]

On Mar 16, 2018 13:57, "Liviu Ionescu" <i...@livius.net> wrote:

On 16 March 2018 at 20:58:13, Samuel Falvo II (sam....@gmail.com) wrote:

> On Fri, Mar 16, 2018 at 1:46 AM, Liviu Ionescu wrote:
> > yes, two-tiered interrupt processing is the ideal textbook solution,
> > but few RTOSes/applications do it.
>
> Citation needed?

sure.

the venerable eCos calls them ISRs and DSRs; the µC/OS-III calls them
direct and deferred interrupts; FreeRTOS has an optional Deferred
Interrupt Handling.

I believe you misunderstood the question - I'm quite sure Samuel wanted you to justify your claim that they are _rare_. He's quite aware of what they are, as can be seen in the rest of his mail.

[Samuel Falvo II]

On Fri, Mar 16, 2018 at 2:01 PM, Alex Elsayed <etern...@gmail.com> wrote:
> I believe you misunderstood the question - I'm quite sure Samuel wanted you
> to justify your claim that they are _rare_. He's quite aware of what they
> are, as can be seen in the rest of his mail.

Correct; that these alternative interrupt handling techniques are
supported by a given RTOS is not, prima facia, evidence that they are
widely used; only that they have proven valuable to "enough" clients
of the RTOSes to be a useful addition to their API.

I'm not intending to say direct interrupt handling is a bad design
choice, or that it is somehow "better" to choose two-level interrupt
structures. Rather, I just want to know when one would want to choose
to go that route, and see if there are general patterns between
projects that influences that decision accordingly. I think it's a
valuable data-point that would contribute to the discussion, and it's
not something I've seen mentioned before.

Hope that clarifies my position.

[Liviu Ionescu]

On 16 March 2018 at 23:01:04, Alex Elsayed (etern...@gmail.com) wrote:

> I believe you misunderstood the question - I'm quite sure Samuel wanted you
> to justify your claim that they are _rare_.

ah, sorry for the misunderstanding.

maybe there are other RTOSes that, for historical reasons
(compatibility with legacy architectures), still implement deferred
interrupts, but I don't remember seeing this feature in the young
RTOSes, that came to market in the Cortex-M age, and, for RTOSes that
implement it, I don't remember seeing Cortex-M projects using it.

I accept that I may be wrong, and on other architectures this feature
might be still in use, but in my world these other architectures
became irrelevant.


regards,

Liviu

[Liviu Ionescu]

On 16 March 2018 at 23:14:53, Samuel Falvo II (sam....@gmail.com) wrote:

> ... only that they have proven valuable to "enough" clients

> of the RTOSes to be a useful addition to their API.

yes, having "enough" client requests is a good reason for extending an
API, but my personal guess is that these features were added when the
RTOSes were ported on architectures with non-preemptive interrupts.

this might also be one reason why deferred interrupts are not present
in recent RTOSes, because their designers focused on Cortex-M and
decided to ignore legacy architectures.


regards,

Liviu

[kr...@berkeley.edu]

I want to clear up the misconception that we don't encourage
experimentation or standardization of alternative privileged
architectures.

Part of the docs clearly states that people can completely replace the
privileged architecture, and in fact, one of our goals in writing
specs this way was to enable experimentation with new OS models.
While I agree microcontrollers are a big near-term market for RISC-V,
RISC-V Unix cores are also a large upcoming market that has a vastly
larger software base that needs time to port and mature, and so a lot
of effort has gone into stable standards for conventional OS ports.

I agree we also need a standard "rich" microcontroller profile and
that this should support C ISRs and preemption/nesting efficiently, as
one of several use cases. I disagree that hardware stacking is the
only solution for this, and prefer more flexible primitives to achieve
the same goal while meeting other goals. I agree modifications to the
ABI can also help when interrupt latency is more important than
straight line performance.

In my github thread response to Liviu, what I am trying to get across
is that for standard profiles, you want to minimize changes to
existing mstatus/privileged, not that they can never be extended. The
new task group is looking at extending interrupt behavior, but with a
view to maintaining backwards compatibility and to support dual-use
cores that run either real-time or virtual-memory code.

Krste

[Jacob Bachmeyer]

Guy Lemieux wrote:
> In this email thread, we have already seen two entirely different
> proposals for a microcontroller environment.

Part of the reason that I wrote a separate proposal is an expectation
that the two (or more?) microcontroller proposals will "cross-pollinate"
to some extent and yield a better proposal than either Liviu Ionescu or
myself could have produced alone.


-- Jacob

[Liviu Ionescu]

On 17 March 2018 at 03:34:32, Jacob Bachmeyer (jcb6...@gmail.com) wrote:


> ... an expectation

> that the two (or more?) microcontroller proposals will "cross-pollinate"
> to some extent and yield a better proposal than either Liviu Ionescu or
> myself could have produced alone.

which, I think, is a resonable expectation.

I already included some of Jacob's comments in my pages, and I'm open
to any comments and new proposals.

I think that the goal of designing a performant and easy to use RISC-V
microcontroller profile is too important to stumble on personal
preferences.

any proposals that come with use cases that show they are simpler to
use, and performance estimates that show better latency or better
performance in general, are welcomed.

regards,

Liviu

[Guy Lemieux]

On Sat, Mar 17, 2018 at 12:44 AM Liviu Ionescu <i...@livius.net> wrote:

any proposals that come with use cases that show they are simpler to
use, and performance estimates that show better latency or better
performance in general, are welcomed.

I'd like to add that cost of implementation must also be considered, particularly for fpga targets.

hence, I would support a modular approach, much like how the whole ISA is designed, where there is a minimal core and then optional extensions.

for example, adding counters and exception support adds considerable area to FPGA implementations. adding more CSRs makes it worse. yet, the reduced register file size of RV32E produces no savings on FPGAs.

Guy

[Liviu Ionescu]

On 17 March 2018 at 10:05:31, Guy Lemieux (glem...@vectorblox.com) wrote:

> On Sat, Mar 17, 2018 at 12:44 AM Liviu Ionescu wrote:
>
> > any proposals that come with use cases that show they are simpler to
> > use, and performance estimates that show better latency or better
> > performance in general, are welcomed.
>
>
> I'd like to add that cost of implementation must also be considered,
> particularly for fpga targets.

yes, sure, the cost is important, and if any of my proposals prove
unreasonably expensive to implement, we'll redesign them.

> hence, I would support a modular approach, much like how the whole ISA is
> designed, where there is a minimal core and then optional extensions.

yes, if possible, this would be a reasonable and consistent approach.

> for example, adding counters and exception support adds considerable area
> to FPGA implementations.

if I remember right, there are two counters in my proposal, the system
counter and the rtc counter, both having the same behaviour as the
mtimer in the privileged specs. are those counters too heavy for you?

also, since you obviously have experience with those things, can you
estimate how much extra space would require the logic to automatically
stack/unstack registers?

> adding more CSRs makes it worse.

in my proposal I reduced the CSRs to a minimum of 10. maybe not all
are needed as CSRs, we can reconsider some of them.

> yet, the reduced
> register file size of RV32E produces no savings on FPGAs.

you mean the reduction is not significative compared to the total
size? since, in my naive understanding of these things, 16 registers
of 32-bits should save some space anyway, like at least some 512
simple cells (again, I'm totally out-of-date with new FPGAs, I only
used some old Xilinx chips more than ten years ago, so I might be
terribly wrong).

anyway, my proposal includes three sub-profiles:

https://github.com/emb-riscv/specs-markdown/blob/master/introduction.md

for highly optimised softcores I would prioritise footprint
optimisations only on the 'small' sub-profile (similarly to ARM
Cortex-M1, which is optimised for synthesised cores).

for the medium sub-profile, although we obviously should not waste
unnecessary resources, I would accept to trade a few resources for
some more gains in terms of 'ease of use'.

ARM had a similar approach with Cortex-M0/M0+/M1 vs Cortex-M3/M4/M7, I
think this two-fold approach is a good starting point, and, given the
RISC-V modularity, we should try to do it even better.


regards,

Liviu

[Liviu Ionescu]

On 17 March 2018 at 02:16:23, kr...@berkeley.edu (kr...@berkeley.edu) wrote:

> I want to clear up the misconception that we don't encourage
> experimentation or standardization of alternative privileged
> architectures.

That's actually the point, you encourage alternate 'privileged'
architectures and completely prevent experimentation or
standardisation on non-privileged architectures.

> Part of the docs clearly states that people can completely replace the
> privileged architecture, and in fact, one of our goals in writing
> specs this way was to enable experimentation with new OS models.

Based on the prerequisites in my proposal, microcontrollers are not
expected to not run any 'new OS models', they are not expected to run
any true OS at all, they should run bare-metal or at most they should
run a multi-threaded scheduler.

https://github.com/emb-riscv/specs-markdown/blob/master/introduction.md#limitations

> While I agree microcontrollers are a big near-term market for RISC-V,
> RISC-V Unix cores are also a large upcoming market that has a vastly
> larger software base that needs time to port and mature, and so a lot
> of effort has gone into stable standards for conventional OS ports.

I think that everyone on this list will agree that the results are
outstanding and these efforts should continue.

However, those with a minimum experience with microcontrollers, will
agree with Guy's statement:

"The microcontroller environment needs to be better specified, as the
current Privileged ISA Specification is inappropriate for that domain."

As someone who actually put a lot of effort to write software
components for the HiFive1 and S31Arty/S51Arty devices (see the
Eclipse RISC-V project template), I would say that the current
privileged ISA specifications are totally inapropriate for wide usage
in microcontroller applications.

> I agree we also need a standard "rich" microcontroller profile and
> that this should support C ISRs and preemption/nesting efficiently, as
> one of several use cases.

Ok, great, we'll further explore this line.

> I disagree that hardware stacking is the
> only solution for this, and prefer more flexible primitives to achieve
> the same goal while meeting other goals.

It is not the only solution, we also have a proposal to use shadow registers.

If your team comes with a feasible solution to use a special compiler
semantic for interrupt routines, which saves only used registers as
long as everything is inlined, I see no problem to support this
mechanism too in the microcontroller profile.

In practical terms, I would add a bit in the interrupt status
registers (per each interrupt source), and allow the user to select
this alternate mode for really fast interrupts, that have very simple
handlers.

Personally I doubt that many applications will be able to inline
everything in the handler, but if someone takes the time to do it, I
think he/she deserves a prize, and this prize can be to have this
feature available, especially since it should be no problem to
implement it.

However, the default interrupt behaviour should use C handlers,
regardless how we decide to implement this.

> I agree modifications to the
> ABI can also help when interrupt latency is more important than
> straight line performance.

Ok, great.

We'll work on a proposal for a RISC-V EABI. Actually two, one for
RV32E and the other for RV32I/RV64I.

> The
> new task group is looking at extending interrupt behavior, but
> with a
> view to maintaining backwards compatibility and to support
> dual-use
> cores that run either real-time or virtual-memory code.

Sure, feel free to extend the privileged profile in Volume II as you
think appropriate, but please do not enforce it on the whole RISC-V
community.

The microcontroller profile simply cannot be a subset of the privileged profile.

Ideally the Foundation should acknowledge this, and possibly support
the design of a new microcontroller profile, complementary to the
privileged profile.

As the first step, I repeat the proposal sent a few messages ago:

---
The first volume, the only one that should be mandatory for compliance
reasons, should not be related to any user or privileged mode, and

should cover only the instruction set, as neutral as possible.

I suggest the name: "The RISC-V Architecture Manual: Volume I: The
Instruction Set".

---

If the Foundation decides to not acknowledge that the microcontroller
profile should be a separate profile, it is fine too, we can always
make it a community effort, as I already started with my proposal, and
hopefully get some support from companies interested in creating
microcontrollers based on RISC-V cores.

But this would not be a fortunate situation...


Regards,

Liviu

[Liviu Ionescu]

On 17 March 2018 at 20:27:52, Liviu Ionescu (i...@livius.net) wrote:

> microcontrollers are not expected to not run any 'new OS models'

microcontrollers are not expected to run any 'new OS models'

[Michael Clark]

> On 17/03/2018, at 11:27 AM, Liviu Ionescu <i...@livius.net> wrote:
>
> On 17 March 2018 at 02:16:23, kr...@berkeley.edu (kr...@berkeley.edu) wrote:
>
>> I want to clear up the misconception that we don't encourage
>> experimentation or standardization of alternative privileged
>> architectures.
>
> That's actually the point, you encourage alternate 'privileged'
> architectures and completely prevent experimentation or
> standardisation on non-privileged architectures.

I think that there is a general expectation that a considered design would minimize unnecessary deltas from the current privileged architecture, versus making radical changes by defining a completely new privileged architecture. There is a difference between privileged architecture experimentation and a wholesale replacement of the existing privileged architecture.

There are some valuable insights from your proposal with respect to lightweight recursive interrupts, however there are also many unnecessary changes where existing lightweight mechanisms are already provided for by the current privileged architecture. These unnecessary changes will only lead to fragmentation and unnecessary complexity for common code. Well, in fact, common code is somewhat thrown out the window with your proposal.

There are some very clear misconceptions in your section on “RISC-V compatibility CSRs”. The specification is quite clear regarding unsupported CSRs (which can be hardwired to zero). Defining alternate mechanisms for well-defined existing mechanisms is what I would call pointless complexity, unless there is a genuine advantage from the alternate approach.

The crux of the issue is mstatus ie and pie bits, mip and mie CSRs, and defining a new mechanism for lightweight recursive interrupts. The other changes are unnecessary. I think baking the stack into the ISA with respect to interrupt handling is a big mistake. You’re adding unnecessary load/store latency where an alternative considered approach would allow an ISR to recurse while only performing register operations. The RISC-V architecture is carefully designed such that one instruction results in one unit of work. i.e. an instruction is a micro-op. Your proposal in its current form is very CISCy. I suspect you might find it challenging to get your proposal adopted as it stands currently.

This is not to say that an alternative experimental lightweight recursive interrupt system, as a considered delta to the current privileged ISA, wouldn’t achieve your high-level objective, and that is compiler annotated irq handlers with support for recursion. i.e. minimize assembly, minimize interrupt latency, and support recursive interrupts.

A better approach may be a requirements or use-case driven approach, to help instruction set architects make a considered delta based on your feedback. e.g.

- no assembly needed to define IRQs. i.e. compiler attributes
- handle prioritized recursive interrupts
- reduce interrupt latency (save/restore overhead)

Adding a complex interrupt state machine for save/restore that bakes a stack into an architecture when the stack is in the most part an ABI convention is perhaps an antithesis to an improvement to interrupt handling. The lowest possible latency approach will perform register swaps, and may use register aliasing or other technique.

My personal opinion is that while Register Windows have proven to be less than optimal as a mechanism for procedure calls, because compilers handle register allocation better, they may very well have some specific use cases with respect to changes in privilege modes or interrupt recursion level. This however would need some research and experimentation…

Liviu, my advice, build a simulator and modify the tools (binutils/gcc) and then you’ll see that some of the unnecessary changes will only hurt (you, if you are the one modifying binutils/gcc, the simulators and RTL, etc)