RISC-V Privileged Architecture Proposal v1.9

[Krste Asanovic]

We've just released version 1.9 of the RISC-V Privileged Architecture
proposal. Attached below, and available at:

http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-129.html

In RISC-V land, v2.0 means "frozen," and v1.9 means "what we hope to
freeze, barring good reason not to." So hopefully something very
close to this version will become the standard.

Notable features since the last version include a standardized
platform-level interrupt controller; simpler exception handling; a
virtual memory scheme that allows execute-only paging; and an outline
of a standard way of describing the hardware platform to the low-level
software.

Look forward to seeing everyone at the RISC-V workshop next week in
Boston,

Krste

[Prashanth Mundkur]

On 02:21 Fri 08 Jul, Krste Asanovic wrote:
>
> We've just released version 1.9 of the RISC-V Privileged Architecture
> proposal. Attached below, and available at:
>
> http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-129.html
>
> In RISC-V land, v2.0 means "frozen," and v1.9 means "what we hope to
> freeze, barring good reason not to." So hopefully something very
> close to this version will become the standard.

Thanks!

> Notable features since the last version include a standardized
> platform-level interrupt controller; simpler exception handling; a
> virtual memory scheme that allows execute-only paging; and an outline
> of a standard way of describing the hardware platform to the low-level
> software.

Would it be possible to include this information in more detailed form
in the document itself? The user-level ISA spec does this really
nicely in the Preface.

--prashanth

[Jonathan Neuschäfer]

On Fri, Jul 08, 2016 at 02:21:13AM -0700, Krste Asanovic wrote:
>
> We've just released version 1.9 of the RISC-V Privileged Architecture
> proposal. Attached below, and available at:
>
> http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-129.html

That's good to hear.


I found a minor error: Table 9.1, the list of SBI calls, mentions
sbi_num_harts, but doesn't list it.

Besides, I wonder why the SBI calls are described in terms of calling a
particular address, instead of using the ecall instruction, which seems
more straight-forward to me. What's the reasoning there?

> In RISC-V land, v2.0 means "frozen," and v1.9 means "what we hope to
> freeze, barring good reason not to." So hopefully something very
> close to this version will become the standard.
>
> Notable features since the last version include a standardized
> platform-level interrupt controller; simpler exception handling; a
> virtual memory scheme that allows execute-only paging; and an outline
> of a standard way of describing the hardware platform to the low-level
> software.

What is the execute-only paging scheme? The page-based virtual memory
schemes don't support execute-only pages (AFAICS), and the Mbbid scheme
arguably isn't "paging", because it doesn't use pages.


Jonathan

[Andrew Waterman]

On Fri, Jul 8, 2016 at 2:14 PM, Jonathan Neuschäfer
<j.neus...@gmx.net> wrote:
> On Fri, Jul 08, 2016 at 02:21:13AM -0700, Krste Asanovic wrote:
>>
>> We've just released version 1.9 of the RISC-V Privileged Architecture
>> proposal. Attached below, and available at:
>>
>> http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-129.html
>
> That's good to hear.
>
>
> I found a minor error: Table 9.1, the list of SBI calls, mentions
> sbi_num_harts, but doesn't list it.

Thanks for pointing that out.

>
> Besides, I wonder why the SBI calls are described in terms of calling a
> particular address, instead of using the ecall instruction, which seems
> more straight-forward to me. What's the reasoning there?

The additional level of indirection gives flexibility to the SEE
implementation, which can improve performance. For example, some SBI
calls in some systems can be handled without an upcall to H-mode or
M-mode.

>
>> In RISC-V land, v2.0 means "frozen," and v1.9 means "what we hope to
>> freeze, barring good reason not to." So hopefully something very
>> close to this version will become the standard.
>>
>> Notable features since the last version include a standardized
>> platform-level interrupt controller; simpler exception handling; a
>> virtual memory scheme that allows execute-only paging; and an outline
>> of a standard way of describing the hardware platform to the low-level
>> software.
>
> What is the execute-only paging scheme? The page-based virtual memory
> schemes don't support execute-only pages (AFAICS), and the Mbbid scheme
> arguably isn't "paging", because it doesn't use pages.

We changed the VM page table entry format slightly to enable X-only permissions.

>
>
> Jonathan
>
> --
> You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
> To post to this group, send email to isa...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.
> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/20160708211429.GG23571%40latitude.

[kr...@berkeley.edu]

>>>>> On Fri, 8 Jul 2016 11:34:01 -0700, Prashanth Mundkur <pmundk...@gmail.com> said:

| On 02:21 Fri 08 Jul, Krste Asanovic wrote:
||
|| We've just released version 1.9 of the RISC-V Privileged Architecture
|| proposal. Attached below, and available at:

...

|| Notable features since the last version include a standardized
|| platform-level interrupt controller; simpler exception handling; a
|| virtual memory scheme that allows execute-only paging; and an outline
|| of a standard way of describing the hardware platform to the low-level
|| software.

| Would it be possible to include this information in more detailed form
| in the document itself? The user-level ISA spec does this really
| nicely in the Preface.

This would have been a good idea to do as we went along as we did with
user-level, but a little difficult to recreate now. The logs for
Spike provide some idea, though mixed in with other unrelated changes.

Hopefully things are mostly settled down, so future changes will be
easier to track.

Krste

[Samuel Falvo II]

On Fri, Jul 8, 2016 at 4:00 PM, <kr...@berkeley.edu> wrote:
> Hopefully things are mostly settled down, so future changes will be
> easier to track.

Too late to change things now, especially for someone still struggling
to release anything like me. But, the whole xRET thing doesn't make
sense to me. It seems to me that ERET should still be sufficient to
cover all the exception handling needs. Removing MRTS, MRTH, and HRTS
also seems to be a strange thing to me. According to V1.9, you can
use xRET to emulate them; however, I can't see how this can be done
without incurring large overheads bit-banging the mstatus register:
you need to manually set the xIE, xPIE, and xPP fields.

Speaking of which, there's no means that I can identify of looking at
the mstatus and telling what operating mode the computer is actually
in, and no other register which could possibly tell me this
information. If, for example, the M-mode trap handler wants to
down-call to S-mode, and since there's no MRTS instruction, I have to
fake it by:

1. LET SPP = MPP[0]. I sure hope sstatus was saved prior to the trap.
2. LET SPIE = MPIE.
3. LET SEPC = MEPC.
4. LET MPP = %01.
5. LET MPIE = %0.
6. LET MEPC = STVEC.
7. Execute MRET. With any luck, this will dispatch the S-mode trap handler.

This might be doable in a handful of instructions, and so maybe it's
not terribly all that slow. But, let's suppose something happens
between steps 6 and 7, where we end up printing a diagnostic to the
screen. Printing the value of mstatus to the console in a debugging
dump will yield a status register that is in an inconsistent state.
There's no way to tell that the CPU is still in M-mode that I can see.

I'm pretty sure I'm missing something obvious here, so if someone can
help explain these design changes, that'd be awesome.

--
Samuel A. Falvo II

[Andrew Waterman]

In most systems, the vast majority of traps from {U,S,H} to {S,H} will
be handled with the interrupt/exception delegation mechanism,
bypassing M-mode altogether. This is much faster than the MRTx
approach encouraged by the previous version.

> --
> You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
> To post to this group, send email to isa...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/CAEz%3Dso%3DDegRnckHeb5%2BU1AovAcd_y2j6cSqHvsmYG-f_D6uGGA%40mail.gmail.com.

[Samuel Falvo II]

On Fri, Jul 8, 2016 at 4:51 PM, Andrew Waterman <and...@sifive.com> wrote:
> In most systems, the vast majority of traps from {U,S,H} to {S,H} will
> be handled with the interrupt/exception delegation mechanism,
> bypassing M-mode altogether. This is much faster than the MRTx
> approach encouraged by the previous version.

If you have the logic cells/transistors to spare for that, sure. But,
that seems like a bit of complex logic to implement, and in the
interest of simplicity, I don't plan on supporting this facility right
away. Getting back to the crux of the issue I was pointing out, my
point wasn't that it was fast or slow, but rather, fickle -- implicit,
inaccessible state inside the CPU tracks the current operating mode,
with no means for inspection *except* for maintaining a shadow
variable in RAM somewhere (e.g., statements like "extern volatile char
cpu_state; cpu_state = M_MODE;" appear early inside M-mode handler
pointed to by mtvec). I would prefer that the current operating mode
be made available at least in a read-only field in a M-mode CSR,
though I prefer such fields being read-write.

[Andrew Waterman]

On Fri, Jul 8, 2016 at 4:59 PM, Samuel Falvo II <sam....@gmail.com> wrote:
> On Fri, Jul 8, 2016 at 4:51 PM, Andrew Waterman <and...@sifive.com> wrote:
>> In most systems, the vast majority of traps from {U,S,H} to {S,H} will
>> be handled with the interrupt/exception delegation mechanism,
>> bypassing M-mode altogether. This is much faster than the MRTx
>> approach encouraged by the previous version.
>
> If you have the logic cells/transistors to spare for that, sure. But,
> that seems like a bit of complex logic to implement, and in the
> interest of simplicity, I don't plan on supporting this facility right
> away. Getting back to the crux of the issue I was pointing out, my

It is a little more complex, but not grossly so. In Rocket, it took
about 20 more LOC to describe than the old MRTx approach.

The hardware cost is negligible for systems that already support
S-mode (it adds 12 flops to rocket, and no more datapath than the MRTx
design).

> point wasn't that it was fast or slow, but rather, fickle -- implicit,
> inaccessible state inside the CPU tracks the current operating mode,
> with no means for inspection *except* for maintaining a shadow
> variable in RAM somewhere (e.g., statements like "extern volatile char
> cpu_state; cpu_state = M_MODE;" appear early inside M-mode handler
> pointed to by mtvec). I would prefer that the current operating mode
> be made available at least in a read-only field in a M-mode CSR,
> though I prefer such fields being read-write.

We consciously excluded a current-privilege-mode register, because
it's a virtualization hole if exposed to less-privileged modes.

If it's only visible to M-mode, it's not terribly useful, since it
will always hold the value M.

[Samuel Falvo II]

On Fri, Jul 8, 2016 at 5:19 PM, Andrew Waterman <and...@sifive.com> wrote:
> The hardware cost is negligible for systems that already support
> S-mode (it adds 12 flops to rocket, and no more datapath than the MRTx
> design).

This actually surprises me. I'll need to study this in greater detail.

> We consciously excluded a current-privilege-mode register, because
> it's a virtualization hole if exposed to less-privileged modes.

Hmm...OK, that's helpful. That puts some useful context in place that
I didn't consider before.

On a related thought, in a Github comment to me some time ago, you'd
mentioned that MRET, SRET, and HRET instructions also patched a
potential exploit as well, but didn't go into details. Is it possible
to elucidate here? Is there a page that describes the exploit and how
this can be a useful work-around?

I ask, only because it seems like having mode-aware xRET instructions
would appear to violate the Popek and Goldberg conditions for
virtualization, making it counter-productive. OTOH, you wouldn't have
gone this route without good reason behind it. I'm just trying to
reconcile my two world-views. :)

[Andrew Waterman]

On Fri, Jul 8, 2016 at 6:01 PM, Samuel Falvo II <sam....@gmail.com> wrote:
> On Fri, Jul 8, 2016 at 5:19 PM, Andrew Waterman <and...@sifive.com> wrote:
>> The hardware cost is negligible for systems that already support
>> S-mode (it adds 12 flops to rocket, and no more datapath than the MRTx
>> design).
>
> This actually surprises me. I'll need to study this in greater detail.
>
>> We consciously excluded a current-privilege-mode register, because
>> it's a virtualization hole if exposed to less-privileged modes.
>
> Hmm...OK, that's helpful. That puts some useful context in place that
> I didn't consider before.
>
> On a related thought, in a Github comment to me some time ago, you'd
> mentioned that MRET, SRET, and HRET instructions also patched a
> potential exploit as well, but didn't go into details. Is it possible
> to elucidate here? Is there a page that describes the exploit and how
> this can be a useful work-around?

No exploit, just a virtualization hole resulting from an instruction's
semantics change with privilege mode (rather than simply trapping at
lower privilege mode).

With ERET, this was initially benign, until we decided to spec
user-level interrupt handling. If ERET were to have become a legal
user-level instruction, we could no longer have intercepted it to
support classic virtualization of S-mode software in U-mode.

Once we decided to add a separate opcode for URET, it seemed more
orthogonal (and slightly lower cost) to have all xRET, rather than
URET for U-mode and ERET for S/H/M. This new design also supports
virtualizing H-mode software in S-mode, though it is unclear whether
anyone will take advantage of this property.

>
> I ask, only because it seems like having mode-aware xRET instructions
> would appear to violate the Popek and Goldberg conditions for
> virtualization, making it counter-productive. OTOH, you wouldn't have
> gone this route without good reason behind it. I'm just trying to
> reconcile my two world-views. :)

Yeah, it's the other way around. Since the semantics no longer vary
with privilege mode, and trap when executed with insufficient
privilege, they are virtualizable.

[Michael Clark]

What happened to sasid? (address space tagged TLB)

I guess the address space switch mechanism needs to be atomic? (guessing interrupts in-between setting sptbr and sasid?)

-- 
You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
To post to this group, send email to isa...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/CA%2B%2B6G0BzFtSXjFeN4VhZVSZRT_6_O2cWrh0Jf-_U7_-xoMbDwg%40mail.gmail.com.

[Angelo Bulfone]

With the privleged spec mostly finalized, is there any chance we might get a http://wiki.osdev.org/Barebones style guide to a hello world supervisor mode application? We have the newlib + proxy kernel, but that hides much of the functioning behind a standard library and drops the programmer off in user mode.

[Samuel Falvo II]

On Fri, Jul 8, 2016 at 7:03 PM, Angelo Bulfone <mbul...@gmail.com> wrote:
> With the privleged spec mostly finalized, is there any chance we might get a
> http://wiki.osdev.org/Barebones style guide to a hello world supervisor mode
> application? We have the newlib + proxy kernel, but that hides much of the
> functioning behind a standard library and drops the programmer off in user
> mode.

Does the tutorial have to use C, or can a pure assembly language
example suffice?

I'm just wondering because I built my own toolchain entirely from
specs; I don't use GCC or related tooling at all in my Kestrel-3
project. And while I don't have the eForth environment documented in
a literate sense yet, that is a longer-term goal of the project.

[Angelo Bulfone]

The advantage to making the example in C is it's easier to read than Assembly. The idea is to show how the kernel initializes itself and how to get basic output. If a part of the tutorial is in C, it's easy to translate into Assembly or some other language.

[Andrew Waterman]

We merged sasid into the MSBs of sptbr.

Changing the ptbr and asid atomically avoids polluting the old (or
new) ASID's translations with the new (or old) PTBR's translations.

[Michael Clark]

That makes a lot of sense. Similar to segmentation (Burroughs in the 50’ and Intel in the late 70’s).

If the page tables fit in the low 4GB then we have a lot of address space ids on RV64.

PTE is 64bits so that’s ~500 million PTE’s in 4GB. Enough for 1PiB RAM (on one node) with 2MiB super pages while still leaving 32-bits for ASID.

Or optimise for more memory with smaller ASID e.g. 16:48 (for operating systems that use a lot of 4KiB pages)

Segmentation is still a good method for fast virtualised address space switching, even if it is decades old.

I sometimes wish Intel and AMD hadn’t crippled it on x86 (we have those two remaining segment registers: fs/gs).

~mc

[Monte Dalrymple]

Page 6: In Table 2.1 I think there is a typo in the send-from-the-last entry in the Hex column.

I think it should read 0xF00 - 0xFBF

Page 7: "Machine-mode standard read-write CSRs 0xFA0 - 0xFBF..."

Unless I am mistaken CSR addresses 0xFA0-0xFBF are read-only.



Page 52: Table 4.1: I think that the last line for interrupt exception codes

is incorrect.

Isn't a code of 9, for a Supervisor external interrupt valid?

On Friday, July 8, 2016 at 2:21:29 AM UTC-7, krste wrote:

[Andrew Waterman]

On Mon, Jul 11, 2016 at 11:31 AM, Monte Dalrymple <mon...@systemyde.com> wrote:
> Page 6: In Table 2.1 I think there is a typo in the send-from-the-last entry
> in the Hex column.
>
> I think it should read 0xF00 - 0xFBF

Yes, good catch.

>
>
>
> Page 7: "Machine-mode standard read-write CSRs 0xFA0 - 0xFBF..."
>
> Unless I am mistaken CSR addresses 0xFA0-0xFBF are read-only.

The range should read 0x7A0 - 0x7BF (which is read-write). The debug
register range is correctly listed in Table 2.1. Thanks for pointing
out the inconsistency.

>
>
>
> Page 52: Table 4.1: I think that the last line for interrupt exception codes
> is incorrect.
>
> Isn't a code of 9, for a Supervisor external interrupt valid?

Yes, supervisor external interrupts are missing from that table. Thanks.

>
>
>
>
> On Friday, July 8, 2016 at 2:21:29 AM UTC-7, krste wrote:
>>
>>
>> We've just released version 1.9 of the RISC-V Privileged Architecture
>> proposal. Attached below, and available at:
>>
>> http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-129.html
>>
>> In RISC-V land, v2.0 means "frozen," and v1.9 means "what we hope to
>> freeze, barring good reason not to." So hopefully something very
>> close to this version will become the standard.
>>
>> Notable features since the last version include a standardized
>> platform-level interrupt controller; simpler exception handling; a
>> virtual memory scheme that allows execute-only paging; and an outline
>> of a standard way of describing the hardware platform to the low-level
>> software.
>>
>> Look forward to seeing everyone at the RISC-V workshop next week in
>> Boston,
>>
>> Krste
>

> --
> You received this message because you are subscribed to the Google Groups
> "RISC-V ISA Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isa-dev+u...@groups.riscv.org.
> To post to this group, send email to isa...@groups.riscv.org.
> Visit this group at
> https://groups.google.com/a/groups.riscv.org/group/isa-dev/.
> To view this discussion on the web visit

> https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/acd461b2-fb34-4165-9712-24add4b22c8f%40groups.riscv.org.

[Monte Dalrymple]

On p15: "M-mode code can access all CSRs at lower privilege levels."

Yes, I know it _can_ but does it _have_ to in all cases? If a machine is M-mode

only is it required to decode and return all of the different-privilege views of the

various CSRs? Obviously it is required for the timers and counters, because of

the RDCYCLE, etc pseudo-instructions, but I am specifically wondering about

things like status, ip, and ie.

On Friday, July 8, 2016 at 2:21:29 AM UTC-7, krste wrote:

[Andrew Waterman]

On Fri, Jul 15, 2016 at 1:53 PM, Monte Dalrymple <mon...@systemyde.com> wrote:
> On p15: "M-mode code can access all CSRs at lower privilege levels."
>
> Yes, I know it _can_ but does it _have_ to in all cases? If a machine is
> M-mode
> only is it required to decode and return all of the different-privilege
> views of the
> various CSRs? Obviously it is required for the timers and counters, because
> of
> the RDCYCLE, etc pseudo-instructions, but I am specifically wondering about
> things like status, ip, and ie.

The statement only applies to the privilege modes that are actually
implemented, so for M-only, there's no need to provide the
less-privileged views.

Actually, even RDCYCLE can be illegal in M-only implementations;
M-mode software should read the mcycle CSR.

>
>
>
> On Friday, July 8, 2016 at 2:21:29 AM UTC-7, krste wrote:
>>
>>
>> We've just released version 1.9 of the RISC-V Privileged Architecture
>> proposal. Attached below, and available at:
>>
>> http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-129.html
>>
>> In RISC-V land, v2.0 means "frozen," and v1.9 means "what we hope to
>> freeze, barring good reason not to." So hopefully something very
>> close to this version will become the standard.
>>
>> Notable features since the last version include a standardized
>> platform-level interrupt controller; simpler exception handling; a
>> virtual memory scheme that allows execute-only paging; and an outline
>> of a standard way of describing the hardware platform to the low-level
>> software.
>>
>> Look forward to seeing everyone at the RISC-V workshop next week in
>> Boston,
>>
>> Krste
>

> --
> You received this message because you are subscribed to the Google Groups
> "RISC-V ISA Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isa-dev+u...@groups.riscv.org.
> To post to this group, send email to isa...@groups.riscv.org.
> Visit this group at
> https://groups.google.com/a/groups.riscv.org/group/isa-dev/.
> To view this discussion on the web visit

> https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/afcfac3e-21ed-41b2-a759-9470fbc22512%40groups.riscv.org.

[Monte Dalrymple]

Table 2.4: hip CSR is missing

On Friday, July 8, 2016 at 2:21:29 AM UTC-7, krste wrote:

[Stefan O'Rear]

(sending to the list this time. Sorry Krste.)

Would it be possible to get a direct link to the document from the
riscv.org site? I'm probably in the minority but I prefer pdfium over
pdf.js.

Have the following questions after reading it. These are mostly with
my assembly and kernel programming hat; others have commented on
hardware aspects, and I'm weak in that area anyway.

> (p16) We require that lower privilege levels execute environment calls instead of reading CPU registers to determine features available at each privilege level. This enables virtualization layers to alter the ISA observed at any level, and supports a much richer command interface without burdening hardware designs.

Will the "environment calls" be specified in a future revision of the
RISC-V calling convention, or will they be left up to each OS
implementor? Will things like crypto kernels that rely on runtime
feature detection need OS-specific code to do so?

(Should there be a way to query if the integer multiplier has
data-dependent timing or not? That's also a highly relevant question
for cryptographic work, and known RISC-V implementations differ on
it.)

> (p21) Virtualization Management Field in mstatus Register

In principle, a kernel could host multiple processes with different
requirements for virtual address mapping, and those requirements could
change over the lifetime of the process. (Imagine a Sv39 process
being automatically promoted to Sv48 after attempting to mmap more
than 256GiB of user address space.) Is that a supported use case?

> (p24) The floating-point unit state is always initialized, saved, and restored using standard instructions (F, D, and/or Q), and privileged code must be aware of FLEN to determine the appropriate space

Does this create an ABI hazard? It seems difficult to support adding
the Q extension to existing hardware-software systems, since code
compiled for the standard calling convention will only save/restore
the low 64 bits. If the kernel does not support Q, is there any
provision to prevent user code from trying to use Q?

> (p33) The mepc register can never hold a PC value that would cause an instruction-address-misaligned exception.

What precisely happens if a JAL or JALR instruction not divisible by 4
is executed on a hart which requires 32-bit alignment (not RV*C)? Is
the trap with mepc pointing to the jump instruction itself?

> (p54) In addition to the ECALL, EBREAK, and SRET instructions defined in Section 3.2.1, one new supervisor-level instruction is provided.

Editorial erratum: §3.2.1 does not actually mention ECALL or EBREAK.

> (p54) The supervisor memory-management fence instruction SFENCE.VM is used to synchronize updates to in-memory memory-management data structures with current execution.

What is the effect of SFENCE.VM on ITLB operations? My reading of
this is that if the mapping of the page of instructions currently
being executed is changed followed by a SFENCE.VM, no instruction
after the SFENCE.VM will be executed from the original page.

What are the ordering semantics of writes to SPTBR? If a write to
SPTBR is immediately followed by a memory reference, is the memory
reference guaranteed to use the new value of SPTBR, or is a SFENCE.VM
required to force the ordering? Does any write to SPTBR have implicit
SFENCE.VM semantics? (These are not the same question; suppose two
page table bases are in use, and supervisor code modifies the inactive
one, switches to it, and then immediately attempts to use the new
mapping.)

> (p73) Keys comprise most printable characters (including ASCII 35–58 and 60–122 and printable UTF-8 characters).

I assume that early boot code that parses configuration strings is not
required to have up-to-date Unicode General_Category tables. What
does "printable" mean operationally, if you don't have unicode tables?

> (p73) Values are whitespace-delimited sequences of the following three types

Same question: is there a specific list of allowed whitespace
characters, or does it change with the current Unicode version?

> (p78) Table 9.1: SBI calls

Several calls in the draft SBI take a hart bitmask with no explicit
length. How should the length be determined, and what happens in a
CPU hotplug situation?

Thank you,

-sorear

[David Horner]

I believe a minimalist implementation validation and optimization would be helpful.

Specifically, targeting RV32E with machine only state in two sub-cases A) only 1 hart and B) more than 1 hart.
and then, the major case is RV32E with M and U, also with the thwo sub-cases; A) only 1 hart and B) more than 1 hart.

The objective would be that these two machine implementations (4 sub cases) would
      1) have the fewest non-zero bits for their encoding and
      2) the number of active bits required would be minimized both in total state information and location in the CSR registers.

Although these criteria do not guarantee low design effort and minimal die use, it likely will help most minimalist implementations.

Specifically targeting RV32E appears logical as the intent of that architecture is the minimal die size required.
    (although it could result in the fastest implementation due to the optimizations that could be achieved because of less decoding and register set addressing.)
    ( I can envision a massively parallel implementation of thousands of (virtual? shared process blocks?) cores/thread/harts).
 
Lastly, the effort is likely to smoke out some assumptions that were made and not readily apparent without such a validation effort.
    (and it is a fun thing that I would like to help with.)

As an example of such considerations that are already in place:

mvendorid   Vendor ID.
marchid       Architecture ID.
mimpid        Implementation ID.
mhartid        Hardware thread ID.

can each be zero indicating the register is not implemented in the first 3 examples, and is a required value for mhartid (for at least on hart, and if only one is present than no confusion)

One such proposal that I would make to reduce set bit state:

change the base encoding in mesa CSR to:

32 bit     - 0 ("00"b)
64 bit     - 2 ("10"b
128 bit   - 3 ("11"b)
reserved - 1 ("01"b)

Thus, RV32E will not have to set any bits to identify its base size.
Which also means that the location of these bits is irrelevant to an implementation that defaults all non-active bits to zero.
 (that being the goal of this minimization).

Also "01"b is explicitly reserved for future expansion.
That reserved mode will likely have to, in the general case, operating in 32bit until such time as it can determine what it really is and determine that the running software (original rom ) did not assume positive mesa CSR meant 32 bit.

I plan to start a separate thread on this effort.

I see re-encoding the values in the CSR as a similar optimization as was done for the base ISA and calling conventions to support RVC; to optimize its encoding and simplify the mapping for low end hardware.

I hope it will be as well received.

 

 

On Friday, 8 July 2016 05:21:29 UTC-4, krste wrote:

We've just released version 1.9 of the RISC-V Privileged Architecture
proposal.  Attached below, and available at:

http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-129.html

In RISC-V land, v2.0 means "frozen," and v1.9 means "what we hope to
freeze, barring good reason not to."  So hopefully something very
close to this version will become the standard.

...

Krste

[kr...@berkeley.edu]

Hi Stefan,

>>>>> On Fri, 15 Jul 2016 21:59:52 -0700, "Stefan O'Rear" <sor...@gmail.com> said:

| On Fri, Jul 8, 2016 at 2:21 AM, Krste Asanovic <kr...@berkeley.edu> wrote:
||
|| We've just released version 1.9 of the RISC-V Privileged Architecture
|| proposal. Attached below, and available at:
|| http://www.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-129.html
||

| Would it be possible to get a direct link to the document from the
| riscv.org site? I'm probably in the minority but I prefer pdfium over
| pdf.js.

We're planning to release incremental versions on arkiv from now on,
including comments on what's changed.

|| (p16) We require that lower privilege levels execute environment calls instead of reading CPU registers to determine features available at each privilege level. This enables virtualization layers to alter the ISA observed at any level, and supports a much richer command interface without burdening hardware designs.

| Will the "environment calls" be specified in a future revision of the
| RISC-V calling convention, or will they be left up to each OS
| implementor? Will things like crypto kernels that rely on runtime
| feature detection need OS-specific code to do so?

There is an ongoing debate on how to pass platform-specific
information through the SBI.

| (Should there be a way to query if the integer multiplier has
| data-dependent timing or not? That's also a highly relevant question
| for cryptographic work, and known RISC-V implementations differ on
| it.)

I don't believe this will truly help with security, given the
multitude of other channels, and the fact the OS can lie to you.
Security needs to be handled a bit more holisticly.

|| (p21) Virtualization Management Field in mstatus Register

| In principle, a kernel could host multiple processes with different
| requirements for virtual address mapping, and those requirements could
| change over the lifetime of the process. (Imagine a Sv39 process
| being automatically promoted to Sv48 after attempting to mmap more
| than 256GiB of user address space.) Is that a supported use case?

No - the intent is the OS only ever runs with one VM. We are
planning to supports optimizations that remove unneeded levels of the
page table when running with larger virtual address spaces.

|| (p24) The floating-point unit state is always initialized, saved, and restored using standard instructions (F, D, and/or Q), and privileged code must be aware of FLEN to determine the appropriate space

| Does this create an ABI hazard? It seems difficult to support adding
| the Q extension to existing hardware-software systems, since code
| compiled for the standard calling convention will only save/restore
| the low 64 bits. If the kernel does not support Q, is there any
| provision to prevent user code from trying to use Q?

The plan for now is to pass Q arguments by reference, and have no
callee-saved Q registers. This keeps the ABI constant on machines with
and without Q hardware, and avoids penalizing current systems by
requiring they support Q with increased stack size.

There can be a future ABI that has Q as first-class citizen when Q
hardware becomes commonplace.

|| (p33) The mepc register can never hold a PC value that would cause an instruction-address-misaligned exception.

| What precisely happens if a JAL or JALR instruction not divisible by 4
| is executed on a hart which requires 32-bit alignment (not RV*C)? Is
| the trap with mepc pointing to the jump instruction itself?

mepc points to the start of the erroneous JAL/JALR instruction, while
badaddr points to the faulting address.

|| (p54) In addition to the ECALL, EBREAK, and SRET instructions defined in Section 3.2.1, one new supervisor-level instruction is provided.

| Editorial erratum: §3.2.1 does not actually mention ECALL or EBREAK.

Thanks for catching the bug - will be fixed in next draft.

|| (p54) The supervisor memory-management fence instruction SFENCE.VM is used to synchronize updates to in-memory memory-management data structures with current execution.

| What is the effect of SFENCE.VM on ITLB operations? My reading of
| this is that if the mapping of the page of instructions currently
| being executed is changed followed by a SFENCE.VM, no instruction
| after the SFENCE.VM will be executed from the original page.

Yes.

| What are the ordering semantics of writes to SPTBR? If a write to
| SPTBR is immediately followed by a memory reference, is the memory
| reference guaranteed to use the new value of SPTBR, or is a SFENCE.VM
| required to force the ordering? Does any write to SPTBR have implicit
| SFENCE.VM semantics? (These are not the same question; suppose two
| page table bases are in use, and supervisor code modifies the inactive
| one, switches to it, and then immediately attempts to use the new
| mapping.)

Modifications to the page tables require an sfence.vm to be sure to be
seen. Writes to sptbr do not cause an automatic sfence.vm, as most
won't need to, as page tables won't usually have been changed in
memory as part of a context swap.

|| (p73) Keys comprise most printable characters (including ASCII 35–58 and 60–122 and printable UTF-8 characters).

| I assume that early boot code that parses configuration strings is not
| required to have up-to-date Unicode General_Category tables. What
| does "printable" mean operationally, if you don't have unicode tables?

|| (p73) Values are whitespace-delimited sequences of the following three types

| Same question: is there a specific list of allowed whitespace
| characters, or does it change with the current Unicode version?

We'll work on tightening the definition for Unicode.

|| (p78) Table 9.1: SBI calls

| Several calls in the draft SBI take a hart bitmask with no explicit
| length. How should the length be determined, and what happens in a
| CPU hotplug situation?

The current design assumes the number of harts can't increase after
the OS is started. We could make the hart set be represented by a
list.

Krste

| Thank you,

| -sorear

[Jacob Bachmeyer]

kr...@berkeley.edu wrote:
>> On Fri, 15 Jul 2016 21:59:52 -0700, "Stefan O'Rear" <sor...@gmail.com> said:
>>
> | On Fri, Jul 8, 2016 at 2:21 AM, Krste Asanovic <kr...@berkeley.edu> wrote:
> || (p21) Virtualization Management Field in mstatus Register
>
> | In principle, a kernel could host multiple processes with different
> | requirements for virtual address mapping, and those requirements could
> | change over the lifetime of the process. (Imagine a Sv39 process
> | being automatically promoted to Sv48 after attempting to mmap more
> | than 256GiB of user address space.) Is that a supported use case?
>
> No - the intent is the OS only ever runs with one VM. We are
> planning to supports optimizations that remove unneeded levels of the
> page table when running with larger virtual address spaces.
>

I had a similar idea when I first read the privileged ISA draft, except
that I thought of it in the context of a user program (such as a Lisp
runtime) being able to trade address space for tag bits and gain a bit
of efficiency along the way. In a nutshell, I propose combining all Sv*
VM modes into three variants of an Spage ("supervisor-controlled
paging") VM mode. The three variants would be Spage32/Spage64/Spage128
for running RV32/RV64/RV128 supervisors, respectively. An
implementation that supports Spage need only support the mode
corresponding to its native width, but may support narrower widths for
compatibility with narrower supervisors.

An address space in the current RISC-V model can be thought of as a
paging depth (LEVELS in sec. 4.5.2) and a root physical page number.
For implementations that support multiple base architectures, an
additional field is needed for width selection, but this field need only
be one bit on RV64 (to select running RV32 user code) and two bits on
RV128 (to choose between RV128/RV64/RV32). Further, width selection
should probably use one of the reserved bits in the RV64 PTE to allow
interworking between RV32 and RV64 code, so I will set that issue aside
for now.

Since only one paging depth is practical on RV32, the new field is zero
bits wide and the RV32 sptbr is unchanged. For RV64, I propose
inserting a two bit paging depth field (exact name to be determined:
"paging depth" (PD), "table count" (TC), "paging levels" (PL), or
something else if someone else has a good idea for it) that stores the
number of page tables to use, minus 3, between ASID and root PPN in
sptbr, reducing the width of ASID by two bits. This should not be
problem, as even a 16 mebislot TLB (needed to make full use of a 24-bit
ASID--and that if only one entry from each ASID is stored, otherwise the
TLB needs to be even larger) seems impractical.

If each page table handles 9 bits of the virtual address in RV64, then
this approach provides virtual address spaces from 39 to 66 bits with
the top two bits in a "66-bit" virtual address space sign-extended from
bit 63. Following the pattern, if each RV128 page table handles 8 bits
of the virtual address, a four bit paging depth field with no offset
enables virtual address spaces from 12 to 132 bits, with the top four
bits in a "132-bit" virtual address space sign-extended from bit 127.

Since the general convention is to map the supervisor at the top of the
address space, varying paging depth will have no effect on the
supervisor mappings provided that the supervisor correctly maps itself
into every root page table. User code that uses tag bits need only
ensure that all-1s is not a valid tag, since such a pointer is
indistinguishable from a supervisor address. To fully support tagged
pointers, a flag needs to be added somewhere that relaxes address
checking and makes the unused bits don't-care in user mode.

A further step is to have separate ssptbr and suptbr registers for
supervisor and user page table bases. I have a rough outline that I'll
fill in if there is interest in the concept.

On a somewhat related matter, PUM should have no effect on instruction
fetch and S-mode should not be allowed to execute code from user pages
under any circumstance. Likewise, H-mode should never be allowed to
execute code from pages belonging to supervisors. What uses, other than
exploits, are there for running S-mode code from user pages or H-mode
code from supervisor pages?

On this note, how are M-mode instruction fetches translated in paging
modes? It seems to be unspecified. Am I overlooking something? Do
supervisors really have the ability to replace the M-mode trap handler
by mapping their own code at that address?

-- Jacob

[Paolo Bonzini]

On Saturday, July 9, 2016 at 3:12:29 AM UTC+2, andrew wrote:

Once we decided to add a separate opcode for URET, it seemed more
orthogonal (and slightly lower cost) to have all xRET, rather than
URET for U-mode and ERET for S/H/M.  This new design also supports
virtualizing H-mode software in S-mode, though it is unclear whether
anyone will take advantage of this property.

As the KVM maintainer in Linux, I'm sure that people will.   HRET is 
a good idea.

I read the specification and it seems sane from a virtualization point 
of view.  I have two objections though.

1) the delta registers.  A hypervisor would probably want to control
the guest's time registers; I'd rather see each level have a CSR for
the delta to the immediately inferior privilege level.  It seems better
from the point of view of the writer of S level code, too.

For example:
* if the H level is present:
** mcycle_delta would be the delta to the H level
** mcycle_delta+hcycle_delta would be the delta to the S level if present
** mcycle_delta+hcycle_delta+scycle_delta would be the delta to the U level

* if it is absent:
** mcycle_delta would be the delta to the S level
** mcycle_delta+scycle_delta would be the delta to the U level


2) interception of environment calls

Currently it seems like ECALL will only allow an environment call to the
above level.  Without a specification for ECALL, this makes the machine
"a bit less virtualizable" (e.g. a hypervisor's hypercall interface might
conflict with a platform's machine call interface).

It would be preferrable IMHO to separate SCALL, HCALL and MCALL.
This would let a hypervisor trap and emulate M calls (e.g. "shut down" a
virtual machine could use MCALL as on a physical machine, instead of HCALL).
It would also match xRET nicely.

There is no need for additional registers for now: HCALL and MCALL handlers
can consult HPP/MPP to fail environment calls from whoever is "not privileged
enough" to invoke them.  However, an "intercept" mechanism (dual to delegation)
can be added to let hypervisors trap SCALL and MCALL in addition to HCALL, and
for M mode to trap SCALL and HCALL in addition to MCALL.  This can use two
new CSRs mintercept and hintercept (consulted in this order; that is, if a bit is
set in both, the processor traps to M mode) together with the existing exception
causes for environment calls.

Is there still time to make these changes?  Apart from the intercept mechanism,
it would be very nice for them to be in there.
 
I have some more reflections on changes to efficiently implement Type-2 hypervisors
or nested Type-1 hypervisors, but I will post them to a separate thread.

Thanks,

Paolo

[Andrew Waterman]

On Mon, Aug 29, 2016 at 10:03 AM, Paolo Bonzini <paolo....@gmail.com> wrote:
>
>
> On Saturday, July 9, 2016 at 3:12:29 AM UTC+2, andrew wrote:
>>
>> Once we decided to add a separate opcode for URET, it seemed more
>> orthogonal (and slightly lower cost) to have all xRET, rather than
>> URET for U-mode and ERET for S/H/M. This new design also supports
>> virtualizing H-mode software in S-mode, though it is unclear whether
>> anyone will take advantage of this property.
>
>
> As the KVM maintainer in Linux, I'm sure that people will. HRET is
> a good idea.
>
> I read the specification and it seems sane from a virtualization point
> of view. I have two objections though.
>
> 1) the delta registers. A hypervisor would probably want to control
> the guest's time registers; I'd rather see each level have a CSR for
> the delta to the immediately inferior privilege level. It seems better
> from the point of view of the writer of S level code, too.
>
> For example:
> * if the H level is present:
> ** mcycle_delta would be the delta to the H level
> ** mcycle_delta+hcycle_delta would be the delta to the S level if present
> ** mcycle_delta+hcycle_delta+scycle_delta would be the delta to the U level
>
> * if it is absent:
> ** mcycle_delta would be the delta to the S level
> ** mcycle_delta+scycle_delta would be the delta to the U level

We are reconsidering the counter-delta strategy altogether and will
have a new proposal soon.

The intent is that a supervisor won't knowingly make requests of
H-mode or M-mode, so there shouldn't be a need to distinguish HCALL
from MCALL. Rather, it makes SBI calls to whatever happens to be
running at the next-highest privilege level. This design allows
S-mode to be agnostic to the existence of H-mode and M-mode.

>
> I have some more reflections on changes to efficiently implement Type-2
> hypervisors
> or nested Type-1 hypervisors, but I will post them to a separate thread.
>
> Thanks,
>
> Paolo
>

> --
> You received this message because you are subscribed to the Google Groups
> "RISC-V ISA Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to isa-dev+u...@groups.riscv.org.
> To post to this group, send email to isa...@groups.riscv.org.
> Visit this group at
> https://groups.google.com/a/groups.riscv.org/group/isa-dev/.
> To view this discussion on the web visit

> https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/ca07c728-7ff7-4455-b377-0e27f300deb5%40groups.riscv.org.

[Paolo Bonzini]

On 29/08/2016 21:07, Andrew Waterman wrote:
> On Mon, Aug 29, 2016 at 10:03 AM, Paolo Bonzini <paolo....@gmail.com> wrote:
>> 2) interception of environment calls
>>
>> Currently it seems like ECALL will only allow an environment call to the
>> above level. Without a specification for ECALL, this makes the machine
>> "a bit less virtualizable" (e.g. a hypervisor's hypercall interface might
>> conflict with a platform's machine call interface).
>>
>> It would be preferrable IMHO to separate SCALL, HCALL and MCALL.
>> This would let a hypervisor trap and emulate M calls (e.g. "shut down" a
>> virtual machine could use MCALL as on a physical machine, instead of HCALL).
>> It would also match xRET nicely.
>

> The intent is that a supervisor won't knowingly make requests of
> H-mode or M-mode, so there shouldn't be a need to distinguish HCALL
> from MCALL. Rather, it makes SBI calls to whatever happens to be
> running at the next-highest privilege level. This design allows
> S-mode to be agnostic to the existence of H-mode and M-mode.

But can it really be? Without any spec-defined MCALLs or generic MCALL
interface (similar to the ARM PSCI for example), a hypervisor cannot
really define its own in a way that will certainly not conflict with a
platform's MCALLs.

Based on my experience with x86, the differentiation between HCALL and
MCALL would actually help the implementation of hypervisors.

Also, what would happen if hedeleg included H-mode environment calls?
Would they be reflected up to M-mode or (more likely) down to S-mode?
Perhaps that's the part that is missing.

Paolo

[Andrew Waterman]

ECALLs from different privilege levels are separately delegable (since
they result in unique CAUSE values, they get unique bits in the edeleg
registers). So, the system can be rigged up so that H-mode ECALLs go
to M-mode, S-mode ECALLs go to H-mode, and U-mode ECALLs go to S-mode.

[Paolo Bonzini]

On 29/08/2016 23:05, Andrew Waterman wrote:
> ECALLs from different privilege levels are separately delegable (since
> they result in unique CAUSE values, they get unique bits in the edeleg
> registers). So, the system can be rigged up so that H-mode ECALLs go
> to M-mode, S-mode ECALLs go to H-mode, and U-mode ECALLs go to S-mode.

Fair enough. I guess it's not a big deal if the SBI/HBI are implemented
through JALR rather than direct ECALLs.

Paolo

[Stefan O'Rear]

On Mon, Aug 15, 2016 at 6:50 PM, <kr...@berkeley.edu> wrote:
>
> Hi Stefan,

>
> | (Should there be a way to query if the integer multiplier has
> | data-dependent timing or not? That's also a highly relevant question
> | for cryptographic work, and known RISC-V implementations differ on
> | it.)
>
> I don't believe this will truly help with security, given the
> multitude of other channels, and the fact the OS can lie to you.
> Security needs to be handled a bit more holisticly.

Fair, although I'm now wondering what such a "holistic approach to
security" would look like. One-size-fits-all security mechanisms do
not seem to be possible.

> | What are the ordering semantics of writes to SPTBR? If a write to
> | SPTBR is immediately followed by a memory reference, is the memory
> | reference guaranteed to use the new value of SPTBR, or is a SFENCE.VM
> | required to force the ordering? Does any write to SPTBR have implicit
> | SFENCE.VM semantics? (These are not the same question; suppose two
> | page table bases are in use, and supervisor code modifies the inactive
> | one, switches to it, and then immediately attempts to use the new
> | mapping.)
>
> Modifications to the page tables require an sfence.vm to be sure to be
> seen. Writes to sptbr do not cause an automatic sfence.vm, as most
> won't need to, as page tables won't usually have been changed in
> memory as part of a context swap.

So it sounds like, if switching between two page tables neither of
which has been modified since the last sfence.vm, sptbr changes are
required to take effect immediately, and in particular for the
immediately following instruction fetch.

-s

[Andrew Waterman]

Yep.

>
> -s

>
> --
> You received this message because you are subscribed to the Google Groups "RISC-V ISA Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to isa-dev+u...@groups.riscv.org.
> To post to this group, send email to isa...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/isa-dev/.

> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/CADJ6UvOACRYMY%3D3xjs8xmFrNieM26p-Etzk7gDHvh0hzKGY9hQ%40mail.gmail.com.