Proposal: RVSdas -- Supervisor disjoint address space

[Jacob Bachmeyer]

The recent announcement of Spectre and Meltdown exploits has changed my
view on this topic from "would be a nice feature" to "we probably need
this". This has been discussed on isa-dev in the past, and was most
recently raised by Christoph Hellwig in message-id
<2017112921...@lst.de>
<URL:https://groups.google.com/a/groups.riscv.org/d/msgid/isa-dev/20171129215426.GB6121%40lst.de>
and discussed in that thread.

I propose an extension RVSdas that adds support for disjoint user and
supervisor address spaces. RVSdas provides full backwards compatibility
with legacy supervisors and partial backwards compatibility with legacy
hypervisors (a non-RVSdas hypervisor cannot run RVSdas supervisors as
guests on RVSdas hardware; the guests will crash). RVSdas adds no new
instructions, but adds two CSRs and compatibly redefines an existing
CSR. These CSR changes are mirrored in RVH.

I have editorial questions that I will need answered before I can write
this as a pull request: where should supervisor extensions be placed in
the privileged ISA spec? Should RVSdas get a new chapter or be placed
in the supervisor-level ISA chapter? RVSdas expands the supervisor
state that RVH must manage. On the other hand, RVSdas is completely
backwards compatible for software, should it be merged into baseline RVS?

-- Jacob




====================

RVSdas -- Supervisor disjoint address space extension


This extension allows supervisor and user translations to use entirely
separate page tables. Two new supervisor CSRs are added and one
existing supervisor CSR is redefined. A small amount of additional
logic provides backwards compatibility.


Supervisor CSRs
========

Renamed CSR 0x180 "satp" to "ssatp" ("Supervisor-controlled Supervisor
Address Translation and Protection")
--------
The ssatp CSR controls address translation in S-mode and retains its
format from baseline RVS. For backwards compatibility, writes to CSR
0x180 "ssatp" also write the same value to CSR 0x181 "suatp" and a
neutral value to CSR 0x182 "sats". The old "satp" name remains valid
for use in software that does not support RVSdas.

New CSR 0x181 "suatp" ("Supervisor-controlled User Address Translation
and Protection")
--------
The suatp CSR controls address translation in U-mode and shares the same
format as the ssatp CSR. The MODE field in suatp must be written as
zero by RVSdas-aware supervisors; non-zero values different from
ssatp.MODE are reserved for future extensions that permit user and
supervisor address spaces to use different paging depths.
Non-RVSdas-aware supervisors will implicitly set suatp.MODE = ssatp.MODE
when they write to the "satp" CSR at 0x180.

[Commentary: The choice of retaining CSR 0x180 as ssatp was driven by
backwards compatibility concerns. While the alternative of assigning
CSR 0x180 as suatp and allocating a new ssatp CSR simplifies handling of
ASIDs if fewer ASID bits are available for the supervisor, it
complicates handling of the backwards compatibility case: an old
supervisor will write only to CSR 0x180, and that write must place the
system into a state where user and supervisor share an address space.
Such a supervisor also will write to CSR 0x180 on every task switch. On
the other hand, an RVSdas-aware supervisor will write to ssatp only
rarely and to suatp on every task switch. Placing suatp at CSR 0x180
means that every user task switch puts the system into backwards
compatibility mode, which forces the supervisor task switch code to be
mapped in every user address space in order for execution to continue
and reactivate disjoint addressing mode.
Nor can the presence of a write to the new CSR be used to select
disjoint addressing mode, because an RVSdas-aware hypervisor will reload
both ssatp and suatp when switching into or out of a guest VM task. A
guest VM running a non-RVSdas supervisor will use the same value for
both CSRs, but the hypervisor still performs both writes. The guest
supervisor, however, will write only to CSR 0x180, which must update
both ssatp and suatp. The solution chosen in this proposal is to
instead require implementations to support equivalent but disjoint user
and supervisor ASID spaces. This avoids the need to map any part of the
supervisor code in the user address space.]

New CSR 0x182 "sats" ("Supervisor Address Translation Selection")
--------
The sats CSR allows the supervisor to efficiently access the user
address space. The sats CSR is XLEN-bits, but only bits 3:0 are
currently defined; the rest of the register is reserved WPRI. Setting
bits in this CSR alters the procedure used for S-mode memory access and
a "neutral value" is therefore defined as a value in this CSR that does
not affect S-mode address translation.

Implementations supporting RVSdas must implement SUM, SLU, and SSU.
Implementations supporting both RVSdas and RVA must implement SAU.
Similar requirements apply to future extensions that have bits in the
sats CSR.

CSR 0x182 "sats" bit 0 "SUM"
----
The least-significant bit in the sats CSR is an alias of sstatus.SUM,
placed within the reach of the immediate-form CSR write instructions.
If sstatus.SUM is clear, the value of the sats CSR is ignored.
Implicitly writing a "neutral value" to the sats CSR does not change
this bit because it actually lives in the sstatus CSR. Explicitly
setting or clearing this bit sets or clears sstatus.SUM. Reading this
bit returns the value of sstatus.SUM.

CSR 0x182 "sats" bit 1 "SLU" ("Supervisor Load from User memory")
----
The SLU bit in the sats CSR causes RVI loads executed in S-mode to use
the user address translations with supervisor permissions. If SLU is
clear or ineffective, RVI loads executed in S-mode use the supervisor
translations with supervisor permissions.

CSR 0x182 "sats" bit 2 "SSU" ("Supervisor Store to User memory")
----
The SSU bit in the sats CSR causes RVI stores executed in S-mode to use
the user address translations with supervisor permissions. If SSU is
clear or ineffective, RVI loads executed in S-mode use the supervisor
translations with supervisor permissions.

CSR 0x182 "sats" bit 3 "SAU" ("Supervisor Atomic access to User memory")
----
The SAU bit in the sats CSR causes RVA operations executed in S-mode to
use the user address translations with supervisor permissions. If SAU
is clear or ineffective, RVA operations in S-mode use the supervisor
translations with supervisor permissions.

CSR 0x182 "sats" bit 4 "STU" ("Supervisor Transactional access to User
memory") (planned, pending standardization of RVT)
----
The STU bit in the sats CSR causes RVT operations executed in S-mode to
use the user address translations with supervisor permissions. If STU
is clear or ineffective, RVT operations in S-mode use the supervisor
translations with supervisor permissions.

CSR 0x182 "sats" bit 5 "SVU" ("Supervisor Vector access to User memory")
(planned, pending standardization of RVV)
----
The SVU bit in the sats CSR causes RVV memory access (or similar
operations of other vector extensions) to use the user address
translations with supervisor permissions, If SVU is clear or
ineffective, vector memory accesses in S-mode use the supervisor address
translations with supervisor permissions.
[Commentary: SVU may be dropped if including vector accesses in SLU/SSU
seems like a better idea. SVU is also the first bit beyond the reach of
the immediate-form CSR write instructions, and should not be moved down,
since it will be less frequently used. Bulk data transfers large enough
to make the vector unit worth using should also be large enough to
amortize page remapping overhead to directly map the relevant user pages
into the supervisor address space.]


Implementation Requirements
========

Implementations supporting RVSdas must bifurcate their ASID space into
disjoint user and supervisor sections. User ASID "N" and supervisor
ASID "N" are unrelated and independent for all "N". Further,
microarchitectural address translation resources must be partitioned
between user ASIDs such that activity in or presence of one user ASID
has no effect whatsoever on activity in other user ASIDs, excepting
memory bus contention and other effects also observable with
fully-independent processors. Each user ASID must appear to have an
independent MMU. Supervisor ASIDs are not required to have similar
isolation, since an RVSdas-aware supervisor will probably use only one
or a few ASIDs and the other supervisor ASID resources should remain
available. A non-RVSdas-aware supervisor will effectively replicate its
mappings across all supervisor ASIDs. The RVH proposal uses
second-level translation with guest accesses translated as user accesses
in the hypervisor; these second-level translations are considered to
occur in a user ASID and therefore guest VMs benefit from the user ASID
isolation guarantees.

The scope of an ASID is defined as the set of harts where loading an
{ASID,PPN} tuple into an address translation and protection CSR can
conflict with cached translations associated with an {ASID,PPN'} tuple.
The narrowest possible scope is "hart-local": two harts may have
different root pages for the same ASID without conflict. The widest
scope is global: ASIDs must be unique within the system. User ASIDs
have implementation-defined scope and may have global scope provided
that the implementation provides at least as many ASIDs as S-mode
harts. Otherwise, user ASIDs must have narrower scope either limited to
some group of harts or be local to a single hart. Supervisor ASIDs are
always hart-local. Supervisors are explicitly permitted to reduce
remapping overhead by using per-hart root page tables with a single ASID.


Effect of Disjoint Address Spaces on Page Translations
========

The PTE U bit is honored in both address spaces, even though the
supervisor address space is entirely inaccessible to user mode. This
allows supervisors that maintain direct-mapped windows into physical
memory to mark pages allocated to user space as user pages, gaining the
benefit of sstatus.SUM protections without making those mappings visible
to user programs.

All user translations are performed using the page tables indicated by
suatp. Except when modified by the flags in the sats CSR, all
supervisor translations are performed from the page tables indicated by
ssatp. The rule that a write to CSR 0x180 writes both ssatp and suatp
is sufficient to preserve backwards compatibility.


Hypervisor CSRs
========

Implementations supporting RVH and RVSdas must also implement background
CSR versions of the suatp and sats CSRs. The bssatp CSR is at the same
address as the bsatp CSR, and the bsuatp and bsats CSRs are at analogous
addresses for the suatp and sats CSRs as bssatp is to ssatp.


====================