SBI - taking a step back

433 views
Skip to first unread message

Alex Bradbury

unread,
Nov 10, 2018, 4:21:50 PM11/10/18
to sw-...@groups.riscv.org
I've been following the recent discussions on SBI extensions with
great interest. Firstly, thank you to everyone who has been
contributing SBI proposals and documenting the current de facto SBI
standard.

I do have one request - could we all take a step back and try to
define the role and driving principles of the SBI? What problems
should it be solving? What problems shouldn't be solved via the SBI? I
understand once the framework is in place there's scope for people to
use it in a wide variety of ways, but it's really hard to judge the
various proposals without any clear definition of the SBI
requirements. Proposing incremental ABI additions seems like the wrong
way to go about development without agreeing this first.

The nearest thing we have to the guiding principles of the SBI is the
stated aim in the current privspec draft "Using a single SBI across
all SEE implementations allows a single OS binary image to run on any
SEE". The document currently proposes SBI calls for:
* Initialising, saving, and restoring user-mode state outside of the
standard FDQ extensions
* Inter-processor interrupts
* Clear timer interrupts
* Mask, unmask, and query the cause of external interrupts
(apparently, all access to the PLIC should be through the SBI?)
* Modifying the time/cycle/instret CSRs
* Scheduling timer interrupts
* TLB shootdown

This is quite an expansion over the current SBI (which seems to offer
a rather arbitrary subset of platform-specific functionality). I would
observe that none of the proposals to extend the SBI so far actually
cover all of these cases. Is that because there is disagreement over
whether the SBI should serve all these purposes, or are the proposals
simply incomplete?

Based on a recent software working group call, it seems there is a
desire for the SBI to ensure that a single kernel works across all
RISC-V implementations, including support for custom extensions (e.g.
save/restore of user-visible state as mentioned above). I'm concerned
that reaching this goal will require an increasing number of hooks
into the kernel effectively making the SBI a mini-OS. e.g. for the
tagged memory mechanism in lowRISC, saving/restoring the tagctrl
register is a necessary component of kernel support but it's also
necessary to modify the virtual memory system to support (at a
minimum) clearing tags on pages and propagating tags upon
copy-on-write.

Other questions/concerns/thoughts:
* Is there actually consensus amongst developers of S-mode code
(kernels, bootloaders etc) that the SBI is a desirable abstraction?
* Shouldn't there be an interface to allow S-mode code to control
delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
supply its own floating point emulation code. Or maybe an alternative
mechanism to swap out the M-mode handler code (as Ron has been
advocating)
* We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
intended to work in the case where a hart supports switching between
these? Surely an S-mode OS may be running with a different ISA variant
to the M-mode SBI/SEE? How is this handled?
* There are ideas floating around about SBI calls being more like
function calls (rather than the current ecall mechanism). Has anyone
written down how this would work?

How about we start by agreeing and enumerating the requirements, then
work our way up?

Thanks,

Alex

Bin Meng

unread,
Nov 10, 2018, 9:34:21 PM11/10/18
to a...@lowrisc.org, sw-...@groups.riscv.org
Hi Alex,

On Sun, Nov 11, 2018 at 5:21 AM Alex Bradbury <a...@lowrisc.org> wrote:
>
> I've been following the recent discussions on SBI extensions with
> great interest. Firstly, thank you to everyone who has been
> contributing SBI proposals and documenting the current de facto SBI
> standard.
>
> I do have one request - could we all take a step back and try to
> define the role and driving principles of the SBI? What problems
> should it be solving? What problems shouldn't be solved via the SBI? I

I agree. Maybe one more question: why is SBI a must-have for RISC-V?
To me current SBI is not absolutely necessary. All these
functionalities can be implemented by the S-mode OS. Why do they have
to be M-mode? One may say some hardware components can only be
accessed by M-mode (eg: IPI) but why should hardware expose such
limitation?

> understand once the framework is in place there's scope for people to
> use it in a wide variety of ways, but it's really hard to judge the
> various proposals without any clear definition of the SBI
> requirements. Proposing incremental ABI additions seems like the wrong
> way to go about development without agreeing this first.
>
> The nearest thing we have to the guiding principles of the SBI is the
> stated aim in the current privspec draft "Using a single SBI across
> all SEE implementations allows a single OS binary image to run on any
> SEE". The document currently proposes SBI calls for:

Shouldn't we clarify why such goal that "Using a single SBI across all
SEE implementations allows a single OS binary image to run on any SEE"
is absolutely necessary? Eventually we will move all platform-specific
stuff into the SBI implementation and that makes SBI another mini-OS
(just as you pointed out below).

> * Initialising, saving, and restoring user-mode state outside of the
> standard FDQ extensions
> * Inter-processor interrupts
> * Clear timer interrupts
> * Mask, unmask, and query the cause of external interrupts
> (apparently, all access to the PLIC should be through the SBI?)

I hope we will not go this way. See below.

> * Modifying the time/cycle/instret CSRs
> * Scheduling timer interrupts
> * TLB shootdown
>

Basically I think RISC-V should allow S-mode OS to handle things like
timer interrupts, IPI, etc, to reduce interrupt latency. I hope S-mode
can directly handle these hardware components instead of going through
the M-mode firmware. There is needs for real-time performance and
determinism.

> This is quite an expansion over the current SBI (which seems to offer
> a rather arbitrary subset of platform-specific functionality). I would
> observe that none of the proposals to extend the SBI so far actually
> cover all of these cases. Is that because there is disagreement over
> whether the SBI should serve all these purposes, or are the proposals
> simply incomplete?
>
> Based on a recent software working group call, it seems there is a
> desire for the SBI to ensure that a single kernel works across all
> RISC-V implementations, including support for custom extensions (e.g.
> save/restore of user-visible state as mentioned above). I'm concerned
> that reaching this goal will require an increasing number of hooks
> into the kernel effectively making the SBI a mini-OS. e.g. for the
> tagged memory mechanism in lowRISC, saving/restoring the tagctrl
> register is a necessary component of kernel support but it's also
> necessary to modify the virtual memory system to support (at a
> minimum) clearing tags on pages and propagating tags upon
> copy-on-write.
>
> Other questions/concerns/thoughts:
> * Is there actually consensus amongst developers of S-mode code
> (kernels, bootloaders etc) that the SBI is a desirable abstraction?

I don't think so :)

> * Shouldn't there be an interface to allow S-mode code to control
> delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
> supply its own floating point emulation code. Or maybe an alternative
> mechanism to swap out the M-mode handler code (as Ron has been
> advocating)

Agreed. We should allow some flexibility.

> * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> intended to work in the case where a hart supports switching between
> these? Surely an S-mode OS may be running with a different ISA variant
> to the M-mode SBI/SEE? How is this handled?

Do you mean switch between 32-bit and 64-bit as defined in the draft
priv spec v1.11?

> * There are ideas floating around about SBI calls being more like
> function calls (rather than the current ecall mechanism). Has anyone
> written down how this would work?
>
> How about we start by agreeing and enumerating the requirements, then
> work our way up?

Regards,
Bin

Luke Kenneth Casson Leighton

unread,
Nov 10, 2018, 9:42:07 PM11/10/18
to Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 9:21 PM Alex Bradbury <a...@lowrisc.org> wrote:

> requirements. Proposing incremental ABI additions seems like the wrong
> way to go about development without agreeing this first.

hmmm, indeed.

> The nearest thing we have to the guiding principles of the SBI is the
> stated aim in the current privspec draft "Using a single SBI across
> all SEE implementations allows a single OS binary image to run on any
> SEE".

section 1.1, V1.11-draft of privspec, SEE="Supervisor Execution Environment"

> The document currently proposes SBI calls for:

_whoa_ whoa, whoa, that's an alarmingly-large set of functionality to
suddenly encounter.


> This is quite an expansion over the current SBI (which seems to offer
> a rather arbitrary subset of platform-specific functionality).

with zero access to critical information behind the unethical
ITU-style closed doors of the RISC-V Foundation, basically from the
sparse and paltry information here we need to reverse-engineer the
intent.

"...... Using a single SBI across all SEE implementations allows a
single OS binary
image to run on any SEE. The SEE can be a simple boot loader and
BIOS-style IO system in a
low-end hardware platform, or a hypervisor-provided virtual machine in
a high-end server, or a
thin translation layer over a host operating system in an architecture
simulation environment."

from that statement, and from the list that you gave, alex, it looks
to me that the intent here is to re-create XEN.

without any kind of wider consultation.

> I would
> observe that none of the proposals to extend the SBI so far actually
> cover all of these cases. Is that because there is disagreement over
> whether the SBI should serve all these purposes, or are the proposals
> simply incomplete?

who knows? certain people do, and they're lobbing stuff over the
fence, ramming it down everyone's throats in a fait-accomplit fashion
and then claiming it's "open".

which is basically the definition of unethical behaviour.

> Based on a recent software working group call, it seems there is a
> desire for the SBI to ensure that a single kernel works across all
> RISC-V implementations, including support for custom extensions (e.g.
> save/restore of user-visible state as mentioned above). I'm concerned
> that reaching this goal will require an increasing number of hooks
> into the kernel effectively making the SBI a mini-OS.

likewise. the clue is in the "recursive" requirement: this looks
very much like a XEN style hypervisor.

"The ABI, SBI, and HBI are still a work-in-progress, but we are now
prioritizing support for
Type-2 hypervisors where the SBI is provided recursively by an S-mode OS."

where is _that_ work and discussion being published??

i'm getting really alarmed that the RISC-V Foundation is operating
extremely unethcially, by isolating everyone from discussion and
forcing the rest of the community to have to work out what the hell is
going on... by the time things are actually released it's far, far too
late,

much worse, anyone OUTSIDE of this unethically-developed process who
submits upstream patches which adds anything that they actually *want*
is quite likely to have their patches REJECTED out of hand because
they're "not submitted through the proper [unethical] channels".

> e.g. for the
> tagged memory mechanism in lowRISC, saving/restoring the tagctrl
> register is a necessary component of kernel support but it's also
> necessary to modify the virtual memory system to support (at a
> minimum) clearing tags on pages and propagating tags upon
> copy-on-write.

that's similar to what IIT Madras are doing: they have some
memory-tracking that helps detect invalid malloc / free, and i talked
to them about the idea of also adding ref-counters for languages like
python, c++ and rust, particularly those with garbage-collection.


> Other questions/concerns/thoughts:
> * Is there actually consensus amongst developers of S-mode code
> (kernels, bootloaders etc) that the SBI is a desirable abstraction?

if it was being properly publicly discussed in an OPEN fashion, and
the code being PROPERLY developed as an organic project, i would not
be concerned.

as it is, we know that the RISC-V Foundation has a consistent track
record of operating completely unethically by shoving code at people.

do we *really* want something as mission-critical as a full
Hypervisor to be developed and then controlled by a known-unethical
organisation, running on our machines?

will we be told "well if you don't like it you can always fork it",
when we *know* that's totally financially unrealistic for small
organisations and companies to even contemplate.

> How about we start by agreeing and enumerating the requirements, then
> work our way up?

that would be a good idea.

l.

Alex Bradbury

unread,
Nov 10, 2018, 11:11:14 PM11/10/18
to Luke Kenneth Casson Leighton, sw-...@groups.riscv.org
Hi Luke. I appreciate your frustration but I'd really like to keep
this thread focused on the specific technical topics I raised.

You've pointed out previously that the RISC-V mailing lists are often
not very welcoming. I think you should consider:
1) The individuals working on standards within the RISC-V Foundation
processes are doing so with good intent, doing the best they can with
the resources they have.
2) Criticism of this intensity, even when directed at "the Foundation"
rather than personally might put such individuals off communicating on
the public lists.

Thanks,

Alex

Luke Kenneth Casson Leighton

unread,
Nov 11, 2018, 1:46:00 AM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
i'd like to see that happen as well, in an open fashion, where
everyone is invited to participate, and feels welcome. it would be
absolutely fantastic, to be able to have up-to-date information and
access to the resources and discussions that allow everyone to
participate.

except that's not the case, is it? and denying it and making excuses
is not going to help the situation get any better, is it? and
expecting people to WASTE THEIR TIME *reverse-engineering* private
closed discussions... and then because it's flat-out impossible to get
it right when the information and context isn't avaiable, i have
bullies fucking well REPEATEDLY poisoning discussions by saying
"you're wrong, you're wrong, but you don't know that because WE talked
about it on a PRIVATE list that YOU don't have access to, because
you're such an oh-so-high-and-mighty ethical little tosspot, har har
we had a real laugh about that one, at your expense, loserr"


right now there's dozens of people over the years - long before i got
involved - who have been made to feel so unwelcome that they have
completely given up. one organisation is even looking to set up a
complete fork of the entire RISC-V infrastructure, they're so pissed
off.

i mean, for f***'s sake, it's right here, the words used by the
director, rick o'connor, in the barcelona workshop announcement:

" Hear from from the major players in the RISC-V ecosystem, including
the leading technology companies and research institutions driving the
RISC-V ISA specification"

hear from the companies and the research institutes, give them credit
where it's due... oh and anyone involved in the libre and open world
who contributed and put hard work into making RISC-V a success can go
f*** themselves, because we can't be bothered to mention them, at all.

the systemic laws of organisations are systemic LAWS for a reason,
and failing to acknowledge credit for peoples' efforts is a systemic
law violation, plain and simple.



> You've pointed out previously that the RISC-V mailing lists are often
> not very welcoming. I think you should consider:
> 1) The individuals working on standards within the RISC-V Foundation
> processes are doing so with good intent, doing the best they can with
> the resources they have.

repeatedly telling me and others who have completely given up that
"they're worthless", or "they're wrong" or "they're wasting
everybody's time" or failing to acknowledge their needs and
constructive input is not even *remotely* "good intent".

telling someone that their constructive criticism is an "ad hominem
attack" is not even remotely "good intent": it's a way to deflect and
deny the truth of an ongoing long-running damaging behavioural
pattern.

what's going on is called absolute arrogant power-abusive bullying,
plain and simple.


> 2) Criticism of this intensity, even when directed at "the Foundation"
> rather than personally might put such individuals off communicating on
> the public lists.

they'll continue to do what is useful to them. that includes
bullying. i've been targetted by bullies my entire life, for speaking
up when nobody else will, because they're too afraid or too vested to
do so.

i'll speak the truth, living to the integrity that i've set, and
nobody's going to tell me i can't.

when people do something good, i'll say so. when people do something
bad, i'll say so. you can't ask fairer than that.

where and when the RISC-V Foundation and the people behind it start
doing *GOOD*, i'll be their loudest, most vocal and most loyal
supporter. and you've already seen me do that: i've made it clear,
repeatedly, that i am deeply impressed and respectful of what they've
achieved.

however i cannot sit by and watch unethical behaviour unfold right in
front of my eyes, and i can NOT support unethical behaviour, or fund
it, or provide resources to organisations or individuals that operate
unethically, because to do so it itself unethical, and i absolutely
cannot do that.

it may sound weird to be both respectful of someone's accomplishments
*and* have some criticism of someone's behaviour: that's just how it
is. there's no conflict there, at all. and i say that because i've
seen people already twist my words and focus on the constructive
criticism.

you want me to stop? HELP get the unethical behaviour of the RISC-V
Foundation FIXED, by acknowledging it, discussing it, and recommending
alternatives and solutions.

if those alternatives and solutions are ethical, and properly adopted,
i'll become the RISC-V Foundation's most loyal supporter and advocate,
you can absolutely count on it.

l.

Bruce Hoult

unread,
Nov 11, 2018, 1:59:31 AM11/11/18
to Luke Kenneth Casson Leighton, Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 10:45 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
On Sun, Nov 11, 2018 at 4:11 AM Alex Bradbury <a...@lowrisc.org> wrote:
> Hi Luke. I appreciate your frustration but I'd really like to keep
> this thread focused on the specific technical topics I raised.

 i'd like to see that happen as well, in an open fashion, where
everyone is invited to participate, and feels welcome.  it would be
absolutely fantastic, to be able to have up-to-date information and
access to the resources and discussions that allow everyone to
participate.

 except that's not the case, is it?  and denying it and making excuses
is not going to help the situation get any better, is it?  and
expecting people to WASTE THEIR TIME *reverse-engineering* private
closed discussions... and then because it's flat-out impossible to get
it right when the information and context isn't avaiable, i have
bullies fucking well REPEATEDLY poisoning discussions by saying
"you're wrong, you're wrong, but you don't know that because WE talked
about it on a PRIVATE list that YOU don't have access to, because
you're such an oh-so-high-and-mighty ethical little tosspot, har har
we had a real laugh about that one, at your expense, loserr"

Luke, Alex is a very reasonable person who is trying to help.

When you reply to him thus, as a bully and an abuser, it would take a saint to remain sympathetic to you.

You're actually capable of staying within the bounds of technical discussion. I've seen it in the last week or so, in comp.arch, where you managed to reply to me in a civilised way, and even quote me approvingly from time to time.

Please try to do the same here in future.

Luke Kenneth Casson Leighton

unread,
Nov 11, 2018, 2:47:33 AM11/11/18
to Bruce Hoult, Alex Bradbury, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 6:59 AM Bruce Hoult <bruce...@sifive.com> wrote:

> Luke, Alex is a very reasonable person who is trying to help.

i know. he's acting with good intent and is following some of the
guidelines here:
http://www.crnhq.org/content.aspx?file=66138|37449y#mediation

i'm just so distressed by the constant adversarial behaviour that i'm
beginning to give up, just like the other libre/open developers who,
after years of not being listened to, are no longer actively
participating.

unfortunately, alex, whilst i fully appreciate that you were
endeavouring to act in a mediatorial role, unfortunately i did not
feel that i was being fully listened to:
http://www.crnhq.org/content.aspx?file=66138|37449y#Empathy


> You're actually capable of staying within the bounds of technical discussion.

that's right - i am!

> I've seen it in the last week or so, in comp.arch, where you managed to reply
> to me in a civilised way, and even quote me approvingly from time to time.

that's right - i did. so you know it's perfectly possible. and, if
you've reviewed the libre-risc-dev (a low-traffic mailing list as it's
only just started), you'll see more evidence of that.

if you go further back than that, you'll find evidence that i've been
the technical lead on large software libre projects for over two
decades, with in some cases over 20 simultaneous part-time
contributors.

so the question to ask is not "how* come you are civilised and
capable of staying within the bounds of technical discussion on other
forums: the real question is, what's different about *these* forums?

that's the *real* question.

> Please try to do the same here in future.

i would absolutely love to see these forums be welcoming and
productive technical forums where *everyone* is welcome, where they
can get fully up-to-date discussions and information (so that effort
is not duplicated or wasted), and can be directly involved in the
development and direction of RISC-V, where they can express their
needs and requirements clearly and be listened to, and their
contributions are appreciated, respected and encouraged.

does that sound reasonable?

l.

Bruce Hoult

unread,
Nov 11, 2018, 3:00:14 AM11/11/18
to Luke Kenneth Casson Leighton, Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 11:46 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
 so the question to ask is not "how* come you are civilised and
capable of staying within the bounds of technical discussion on other
forums: the real question is, what's different about *these* forums?

In the last few hours I've seen you being adversarial with not only Alex Bradbury, but also Nick Kossifidis (who you for unknown reason persist in addressing as "Mick"), and Tommy Murphy.  And adopting a derisory tone with others such as Ron Minnich.

There's a common thread here. These people don't talk to each other in the manner in which you reply to them.

Luke Kenneth Casson Leighton

unread,
Nov 11, 2018, 4:46:55 AM11/11/18
to Bruce Hoult, Alex Bradbury, sw-...@groups.riscv.org
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Sun, Nov 11, 2018 at 8:00 AM Bruce Hoult <bruce...@sifive.com> wrote:
>
> On Sat, Nov 10, 2018 at 11:46 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
>>
>> so the question to ask is not "how* come you are civilised and
>> capable of staying within the bounds of technical discussion on other
>> forums: the real question is, what's different about *these* forums?
>
>
> In the last few hours I've seen you being adversarial with not only
> Alex Bradbury,

if you look carefully, alex was trying to be reasonable, and play a
mediatorial role... except if you look carefully at the advice given
on crhnq, a mediator needs to respect *both* sides, and relay (reflect
back) the position of *both* parties.

regrettably, he didn't do that, and unfortunately, all it did was
make it look he wasn't listening to what i'm saying. as i was
writing, i was thinking, "damn, damn damn, i really like alex, how the
hell can i make this look any better, so it doesn't reflect badly on
him?" and, sadly, as he (you, hello, yes, sorry to be talking _about_
you, alex) hadn't summarised and acknowledged both perspectives as a
mediator normally would, there really wasn't a way to do that.

> but also Nick Kossifidis

if you look carefully at what nick's writing, he made good points
from the field in which he has experience.. unfortunately he laced it
with hyperbolic adversarial questions that were very carefully
designed to ridicule rather than critique not only the proposal that i
made, they were also very very carefully crafted to ridicule me, as a
person, as well.

you'll notice that he dodged the positive aspects that were supported
by other people's constructive input, and focussed on generating as
much negativity as he could, through the use of what can only be
termed "hysteria" questions, hoping to assign them to me (without my
consent) and use them rhetorically for the purposes of ridicule.

the initial (thinly-veiled) adversarial attacks that he launched were
sufficiently over the top that someone else had to step in, and hint
that he was going a little too far. nick made a conscious choice to
ignore those hints.

the question here i have to ask, is, can you see that that's what
nick did, and, if you can, why do you feel that it's acceptable
behaviour on a technical forum?

> (who you for unknown reason persist in addressing as "Mick"),

sorry, nick: small high-resolution screen here, and i run the
brightness way down and the room lights off, most of the time.

> and Tommy Murphy.

tommy, bless him, i know he's trying to help. you may be confusing
the pain that i am experiencing through having to deal with the
ongoing bullying and expressing that, and thus interpreting my
responses to him as "this must be an attack on tommy". categorically
and absolutely it is not.

> And adopting a derisory tone with others such as Ron Minnich.

what?? not at all! ron makes a lot of sense, and i've said so quite
clearly!! what on *earth* gave you the impression that i've been
"derisory" with ron?? i've made it quite clear that the points he's
made about being able to operate fully in M-Mode are good ones!

> There's a common thread here.

there is, indeed. the challenge is, then, if the opportunity to
attack arises, if someone appears to be vulnerable, in pain, or
appears to not be knowledgeable, is for people to step up to the plate
and *not* react in a derisory fashion, *not* attack or denigrate them,
make them feel so unwelcome and so despised that it actually
deteriorates their physical health.

you know i speak my mind, and i made a decision 20 years ago to speak
truth, even when it's uncomfortable truth. that's what people face
here, and i know they don't like it.

i told you: ethical's a bitch. truth, awareness, love and creativity
aren't "nice", they're damn uncomfortable, and dealing with this is so
hurtful it's *literally* making me physically ill.

yet... i have a goal to achieve, the target's been set, to create a
completely libre, open and freely-available 3D-capable processor, and
to *develop* it in a truly libre and open fashion, and i am not going
to go away until that goal is achieved.

so, we have at least two more years of this. that's the reality.
so, we can use these forums for the benefit of RISC-V, or not. which
is it going to be?

we can work together, to improve RISC-V and make a conscious decision
to make the forums a welcoming place... or we can turn it into a
hate-filled forum where students, corporate employees and researchers
treat it - and RISC-V - as an absolute "No Go Zone", and psychology
researchers use the public archives in case-studies over the next
couple of decades, on how open communities go into denial when faced
with challenges.

that's what we're looking at, here. it's up to everyone to make a
conscious decision which direction they want to go.

> These people don't talk to each other in the manner in which you reply to them.

people don't talk to me on comp.arch or libre-riscv-dev in the manner
in which they reply to me on here. why?

l.

Alex Bradbury

unread,
Nov 11, 2018, 5:05:01 AM11/11/18
to Luke Kenneth Casson Leighton, sw-...@groups.riscv.org
Hi Luke,

I've changed the email subject to split this out to a separate thread.
I totally accept that your concerns about process are intertwined with
the technical discussion, but I'd much rather keep that separate to
the thread I started (more on that below). I'm not telling you what to
do, just asking.

For anyone who wants to follow the precursor to this split thread, see
<https://groups.google.com/a/groups.riscv.org/d/msg/sw-dev/dO1gR0bKfKs/z9wrlyTJAgAJ>
and <https://groups.google.com/a/groups.riscv.org/d/msg/sw-dev/dO1gR0bKfKs/zq8zanPWAgAJ>.
I don't think that the closed working groups used by the RISC-V
Foundation are the right model. This is an opinion I've expressed
repeatedly, going back to before the formation of the Foundation.
Others obviously feel differently. I think the pragmatic way forwards
is to enthusiastically embrace opportunities for more open discussion
and development. I'll note that:

1) the privileged spec source is on github with public issue tracking
<https://github.com/riscv/riscv-isa-manual>
2) the draft SBI spec is public on GitHub with public issue tracking
<https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md>
3) the de facto SBI reference implementation is upon on GitHub with
public issue tracking <https://github.com/riscv/riscv-pk>
4) The proposal from Atish on extending the SBI and all subsequent
discussions have been on public mailing lists

Given all of the above, wouldn't it make sense to work hard to keep
SBI-related technical discussions on track, with the hope of pointing
to what is hopefully a positive and focused discussion as an example
of the benefits of developing + discussing these standards out in the
open?

I'm sorry you feel that I didn't fully listen to your concerns. Mostly
there's just not much I can say in response - I can't speak for the
the RISC-V Foundation or its collaboration/development model. I was
careful not to label your concerns as invalid - I'm simply suggesting
that it would be more appropriate to raise them in a different thread,
and that you might reflect on whether the approach you're taking to
raising them is the best strategy to achieve your desired outcome.
Just a thought, and you're welcome to disagree.

Best,

Alex

Richard W.M. Jones

unread,
Nov 11, 2018, 6:31:51 AM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
> Based on a recent software working group call, it seems there is a
> desire for the SBI to ensure that a single kernel works across all
> RISC-V implementations,

Not sure what the "software working group call" is, but if this is in
any way related to what I said in my talk in Barcelona then I'd like
to clarify that I only meant for server platforms. Embedded folk can
and most likely will go their own way.

BTW the way you described the full scope of the (potential) SBI made
it sound like UEFI.

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

Christoph Hellwig

unread,
Nov 11, 2018, 7:40:55 AM11/11/18
to Richard W.M. Jones, Alex Bradbury, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 11:31:47AM +0000, Richard W.M. Jones wrote:
> On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
> > Based on a recent software working group call, it seems there is a
> > desire for the SBI to ensure that a single kernel works across all
> > RISC-V implementations,
>
> Not sure what the "software working group call" is,

I have no idea either. But more importantly having important points
only on a call is always a bad idea for a truely open project. Time
zones will get in the way of attendance, nevermind that a of people
who could contribute on mailing lists simply don't have the time to
attend regularly scheduled calls for everything they are involved in.

> but if this is in
> any way related to what I said in my talk in Barcelona then I'd like
> to clarify that I only meant for server platforms. Embedded folk can
> and most likely will go their own way.
>
> BTW the way you described the full scope of the (potential) SBI made
> it sound like UEFI.

The SBI has always been described as for Unix class systems, which
was to entail more than servers, but basically everything running
Linux, FreeBSD, etc.

Christoph Hellwig

unread,
Nov 11, 2018, 7:49:32 AM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
Hi Alex,

as someone who spent a fair time with the SBI interface and the
Supervsior (OS) side of it I'll try to chime in.

The SBI seems to be trying to archive two things, but in the few
bits actually implemented, and even more so in the grand unified
scheme of things.

First it seems to avoid having to architecturally specific things that
in other modern architectures is part of the privileged instruction set.

In the current "stub" SBI this includes TLB flushing, instruction
cache flushing on others than the local core, IPI handling, time CSR
handling. This part is specific to the actual cpu microarchitecture
used in a given setup.

The other bit is just general platform wire up that is handled by
a firmware interface (like UEFI) elsewhere. Example include the SBI
console or the shutdown handling in the current "stub" SBI. This part
is generally board specific.

In general the first category seems like a really bad idea in
retrospective, as we could have just made the sbi interfaces instruction
and CSRs instead, leading to an actually well defined and standalone
usable supervisor isa. The other bits could then be left to the firmware
interface of choice.

Now do we need the SBI? Without having a compiler and/or assembler that
knows about the superset of all vendor specific instructions and CSRs
we very much do unless we move more of the fast (or fast-ish) path
bits the SBI currently handles into the privilged ISA as architectural
features.


[and now I'll try to find some time to read the big flame ware. Or maybe
not and enjoy the sunday afternoon instead]

Luke Kenneth Casson Leighton

unread,
Nov 11, 2018, 7:57:09 AM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 10:05 AM Alex Bradbury <a...@lowrisc.org> wrote:
>
> Hi Luke,
>
> I've changed the email subject to split this out to a separate thread.
> I totally accept that your concerns about process are intertwined with
> the technical discussion, but I'd much rather keep that separate to
> the thread I started (more on that below). I'm not telling you what to
> do, just asking.

no it makes a lot of sense. i get caught up in these things.

> I don't think that the closed working groups used by the RISC-V
> Foundation are the right model. This is an opinion I've expressed
> repeatedly, going back to before the formation of the Foundation.
> Others obviously feel differently.

when i spoke to krste in barcelona about this, he explained that
there's people who just simply would not come forward - at all - if
the discussions were public. mostly, they're people who have worked
in U.S. Classified environments, and their knowledge and expertise is
too valuable to ignore.

i understand this... i see the logic... and then i'm faced with a
massive goal and the worry of keeping my family fed and housed when i
have absolutely no idea, month to month, whether i'll make the next
rent payment, living under the radar in a Republic of China.... and
i'm locked out of access to critical information and resources that
would *literally* save months if not years of development effort.

also, samuel's concern about the ITU-style standards development
process bleeding over into software development as well *has*
basically already happened [
https://groups.google.com/a/groups.riscv.org/d/msg/sw-dev/0fU8_ZBRcm4/49tMtyyrAwAJ
]

> I think the pragmatic way forwards
> is to enthusiastically embrace opportunities for more open discussion
> and development. I'll note that:
>
> 1) the privileged spec source is on github with public issue tracking
> <https://github.com/riscv/riscv-isa-manual>
> 2) the draft SBI spec is public on GitHub with public issue tracking
> <https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md>
> 3) the de facto SBI reference implementation is upon on GitHub with
> public issue tracking <https://github.com/riscv/riscv-pk>
> 4) The proposal from Atish on extending the SBI and all subsequent
> discussions have been on public mailing lists
>
> Given all of the above, wouldn't it make sense to work hard to keep
> SBI-related technical discussions on track, with the hope of pointing
> to what is hopefully a positive and focused discussion as an example
> of the benefits of developing + discussing these standards out in the
> open?

yeah, that sounds like a great plan. what you say reminds me of
"don't complain, do it". my friend today sent me a message, "you hear
of any vision that succeeded which was a massive of 'don'ts'???" :)

> I'm sorry you feel that I didn't fully listen to your concerns. Mostly
> there's just not much I can say in response -

it's ok, i'm sorry, i just... flipped.

> I can't speak for the
> the RISC-V Foundation or its collaboration/development model. I was
> careful not to label your concerns as invalid - I'm simply suggesting
> that it would be more appropriate to raise them in a different thread,
> and that you might reflect on whether the approach you're taking to
> raising them is the best strategy to achieve your desired outcome.
> Just a thought, and you're welcome to disagree.

appreciated. it's just... i've spent literally years of my life
doing reverse-engineering, cleaning up after the proprietary, secret,
closed-door messes of microsoft, google, and chinese criminal
copyright infringers, then having people sponge off of my effforts,
make a ton of money, and claim all the credit (that's *free software*
people claiming credit for my work). i'm absolutely sick of it,
figuratively *and* literally.

so whenever i am faced with closed doors and having to face the
prospect of reverse-engineering the intent behind a document, i'm
vividly - so vividly it *really* scares me - reminded of the times
when i've quite literally been homeless, and not being able to turn
the key in the lock of my house because i had RSI so badly as i was
down to under 65kg because i couldn't buy enough food, and too many
other instances to list, burned into my memory that i'm never, ever
going to forget.

i'm sharing this so that you don't just _think_ i'm motivated to work
towards the goal of a welcoming and thriving open RISC-V community,
you really *know* it.

so yeah, let's see what happens. those are useful links to start from, alex.

l.

Alex Bradbury

unread,
Nov 11, 2018, 8:16:33 AM11/11/18
to bmen...@gmail.com, sw-...@groups.riscv.org
On Sun, 11 Nov 2018 at 02:34, Bin Meng <bmen...@gmail.com> wrote:
> > * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> > intended to work in the case where a hart supports switching between
> > these? Surely an S-mode OS may be running with a different ISA variant
> > to the M-mode SBI/SEE? How is this handled?
>
> Do you mean switch between 32-bit and 64-bit as defined in the draft
> priv spec v1.11?

Yes, for a SoC that allows switching between RV32E/RV32I/RV64I, how
does the SBI support S-mode operating systems compiled for
RV32E/RV32I/RV64I? Presumably the SBI will be running with MXLEN=64
but the S-mode OS and user-space might run with SXLEN=32 and UXLEN=32.
The SBI/SEE would need to support these three different ABIs.

Similarly, the instruction trap+emulation code should be checking
SXLEN/UXLEN depending on the privilege mode which encountered the
illegal instruction. Is it currently possible to do this? I can't see
a way to differentiate a S-mode illegal instruction vs a U-mode
illegal instruction based on mcause.

Best,

Alex

Bin Meng

unread,
Nov 11, 2018, 8:26:56 AM11/11/18
to a...@lowrisc.org, sw-...@groups.riscv.org
Hi Alex,

On Sun, Nov 11, 2018 at 9:16 PM Alex Bradbury <a...@lowrisc.org> wrote:
>
> On Sun, 11 Nov 2018 at 02:34, Bin Meng <bmen...@gmail.com> wrote:
> > > * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> > > intended to work in the case where a hart supports switching between
> > > these? Surely an S-mode OS may be running with a different ISA variant
> > > to the M-mode SBI/SEE? How is this handled?
> >
> > Do you mean switch between 32-bit and 64-bit as defined in the draft
> > priv spec v1.11?
>
> Yes, for a SoC that allows switching between RV32E/RV32I/RV64I, how
> does the SBI support S-mode operating systems compiled for
> RV32E/RV32I/RV64I? Presumably the SBI will be running with MXLEN=64
> but the S-mode OS and user-space might run with SXLEN=32 and UXLEN=32.
> The SBI/SEE would need to support these three different ABIs.
>

Yes, I think we will need support such configuration in SBI when priv
spec v1.11 complaint silicon comes out.

> Similarly, the instruction trap+emulation code should be checking
> SXLEN/UXLEN depending on the privilege mode which encountered the
> illegal instruction. Is it currently possible to do this? I can't see
> a way to differentiate a S-mode illegal instruction vs a U-mode
> illegal instruction based on mcause.
>

We can check mstatus.mpp to see which privilege level the illegal
instruction came from, no?

Regards,
Bin

Alex Bradbury

unread,
Nov 11, 2018, 8:37:41 AM11/11/18
to bmen...@gmail.com, sw-...@groups.riscv.org
Ah yes, thanks. I was a bit confused by the fact there are separate
exception codes for environment calls from U-mode, S-mode and M-mode.
But I see the reasoning for this is explained in commentary for the
ECALL instruction (to allow selective delegation).

Best,

Alex

Alex Bradbury

unread,
Nov 11, 2018, 8:43:08 AM11/11/18
to h...@lst.de, sw-...@groups.riscv.org
On Sun, 11 Nov 2018 at 12:49, Christoph Hellwig <h...@lst.de> wrote:
>
> Hi Alex,
>
> as someone who spent a fair time with the SBI interface and the
> Supervsior (OS) side of it I'll try to chime in.

Thanks for the summary as you see things.

> The SBI seems to be trying to archive two things, but in the few
> bits actually implemented, and even more so in the grand unified
> scheme of things.
>
> First it seems to avoid having to architecturally specific things that
> in other modern architectures is part of the privileged instruction set.
>
> In the current "stub" SBI this includes TLB flushing, instruction
> cache flushing on others than the local core, IPI handling, time CSR
> handling. This part is specific to the actual cpu microarchitecture
> used in a given setup.
>
> The other bit is just general platform wire up that is handled by
> a firmware interface (like UEFI) elsewhere. Example include the SBI
> console or the shutdown handling in the current "stub" SBI. This part
> is generally board specific.
>
> In general the first category seems like a really bad idea in
> retrospective, as we could have just made the sbi interfaces instruction
> and CSRs instead, leading to an actually well defined and standalone
> usable supervisor isa. The other bits could then be left to the firmware
> interface of choice.

Your summary matches my current understanding. Though the scope of the
SBI as defined in the current privileged spec appears to go beyond
these two purposes. e.g. save/restore of user-visible state introduced
by custom extensions, and providing a virtual interface to the PLIC
(or presumably an alternative interrupt controller?).

Best,

Alex

Christoph Hellwig

unread,
Nov 11, 2018, 8:51:05 AM11/11/18
to Alex Bradbury, h...@lst.de, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 01:46:49PM +0000, Alex Bradbury wrote:
> Your summary matches my current understanding. Though the scope of the
> SBI as defined in the current privileged spec appears to go beyond
> these two purposes. e.g. save/restore of user-visible state introduced
> by custom extensions,

Well, this is clearly the fast path in the supervisor case. And although
architectures have come up with ways to save arbitrary extension state
(x86 XSAVE/XSAVEOPT) that approach seems rather CISC-y and might have
a a difficult standing in RISC-V. So it might need something SBI-ish
or kernel and compiler support for extensions with state (which doesn't
sound like an all bad idea to me).

> and providing a virtual interface to the PLIC
> (or presumably an alternative interrupt controller?).

And this seems like a horrible idea. We should rather make an actual
PLIC implementation architectural and let the generaly purpose OS rely
on it (ala x86 API and ARM GIC).

Nick Kossifidis

unread,
Nov 11, 2018, 9:44:18 AM11/11/18
to Luke Kenneth Casson Leighton, Bruce Hoult, Alex Bradbury, sw-...@groups.riscv.org
Hello Luke,

Στις 2018-11-11 11:46, Luke Kenneth Casson Leighton έγραψε:
>
>> but also Nick Kossifidis
>
> if you look carefully at what nick's writing, he made good points
> from the field in which he has experience.. unfortunately he laced it
> with hyperbolic adversarial questions that were very carefully
> designed to ridicule rather than critique not only the proposal that i
> made, they were also very very carefully crafted to ridicule me, as a
> person, as well.
>
> you'll notice that he dodged the positive aspects that were supported
> by other people's constructive input, and focussed on generating as
> much negativity as he could, through the use of what can only be
> termed "hysteria" questions, hoping to assign them to me (without my
> consent) and use them rhetorically for the purposes of ridicule.
>
> the initial (thinly-veiled) adversarial attacks that he launched were
> sufficiently over the top that someone else had to step in, and hint
> that he was going a little too far. nick made a conscious choice to
> ignore those hints.
>
> the question here i have to ask, is, can you see that that's what
> nick did, and, if you can, why do you feel that it's acceptable
> behaviour on a technical forum?
>

This discussion is way off topic and definitely non-technical but
since you mentioned me and what a bad person I am according to
you I guess I'll answer...

I don't know you and you don't know me. If my technical questions
offend you I can't do anything about it. That was definitely not
my intention as I don't have time to spent bullying, offending
and "carefully design" attacks to other people I don't know.
I've had enough flame wars in my 20s, I don't plan on starting
again. If you want to feel offended and present yourself as a
victim it's your bug, not mine.

If you expect people to always agree with you and support every
proposal you make, a technical discussion forum is not a place
where this is going to happen. We all want RISC-V to succeed and
each one of us here will try to convince the others why the X or
Y proposal is good or bad. That's totally normal.

As far as I'm concerned I never made a personal comment to you,
never characterized you, never judged your intentions and
never claimed that you make "hysteria" questions or anything
like that. What I've received from my intention to keep
the SBI clean (a technical reason), was a full-blown personal
attack from you. If you can't see it take a step back, read your
message and what you've written about me in public, and go back
and compare it with our other discussion.

Now I could start writing about how I've been hurt and the pain
of having someone portraying me as an evil person that wants to
ridicule him etc in public. I could go on and brag about my years
of experience and the fights I've given for open source and open
networks with zero resources, from a country with not even half
the opportunities your country provides, but that's not the point.
You see I didn't come here to make friends, I didn't come here to
feel liked or welcome, I don't expect from people to encourage me
and tell me how good my proposals are. I came here to work on
technical issues and make RISC-V fly, as it should fly ! Damn we
have the opportunity to make a difference and we waste time on this
stuff ! I could be writing a mail with my comments on Atish's SBI
proposal instead of this...

Anyway I hope we meet in person at some point, drink a beer and
laugh about this. Seriously I never meant to insult you or ridicule
you, I don't tend to spend time answering to people I think are
ridiculous.

>> (who you for unknown reason persist in addressing as "Mick"),
>
> sorry, nick: small high-resolution screen here, and i run the
> brightness way down and the room lights off, most of the time.
>

Mick is my username (long story), I'm ok with either Nick or mick.


Regards,
Nick



Jonathan Neuschäfer

unread,
Nov 11, 2018, 12:14:35 PM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
> What problems shouldn't be solved via the SBI?

I'd like to argue against sbi_shutdown, or for its clarification.
Sorry if anyone has already made these arguments; I have not thoroughly
checked the other thread.

The current definition in the riscv-sbi-doc repo[1] is:

void sbi_shutdown(void)

Puts all the harts to shut down state from supervisor point of view.
This SBI call doesn't return.

I can see two possible interpretations of this SBI call:

(a) Halt all harts (for example with the WFI instruction), and that's it.

(b) Halt all harts and also initiate a SoC-wide or board-wide power-off
sequence. Maybe I'm reading too much into the name, but "shutdown"
seems to imply this, for me.


Under interpretation (a),

- I'd prefer if it was simply possible to halt each hart directly from
S-mode, instead of going through the SBI (e.g. with WFI).
- If sbi_shutdown stays, I think it would be better to make it halt not
all harts, so that code running one one hart can halt the other harts
and then initiate a board power-off sequence[2]. I guess an interface
which only halts one specified hart would be the most flexible one.

If this is the correct interpretation, I think the SBI call should be
renamed to something more obvious, such as "sbi_halt_all_harts", or
"sbi_shutdown_harts".


Under interpretation (b),

- Board level power-off is a complex thing that might require various
drivers (I²C, SPI, IPMI, ...) in order to talk to off-chip power
management ICs. Moving these drivers into M-mode firmware is a bad
idea IMHO, for the following reasons:
- More M-mode code *potentially* means more M-mode attack surface,
weakening the security of M-mode implementations.
- Bugs in M-mode code may be harder to patch than bugs in the S-mode
operating system, because the M-mode firmware might be stored on
special flash, or signed with the SoC vendor's key, etc.
- Having drivers in M-mode and S-mode for the same hardware may cause
some conflicts and race-conditions when both drivers access the same
hardware at the same time.


Kind regards,
Jonathan Neuschäfer


[1]: https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md
[2]: S-mode shutdown code *could* be written like this:
start_poweroff_sequence();
sbi_shutdown();
// sbi_shutdown halts all harts
// the power-off sequence completes and the board is powered off
But that seems prone to timing problems, to me.
signature.asc

Samuel Falvo II

unread,
Nov 11, 2018, 1:31:39 PM11/11/18
to Bin Meng, Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 6:34 PM Bin Meng <bmen...@gmail.com> wrote:
> I agree. Maybe one more question: why is SBI a must-have for RISC-V?
> To me current SBI is not absolutely necessary. All these
> functionalities can be implemented by the S-mode OS. Why do they have
> to be M-mode? One may say some hardware components can only be
> accessed by M-mode (eg: IPI) but why should hardware expose such
> limitation?

I think it boils down to having some manner of hardware abstraction
just sufficient enough to bootstrap that S-mode OS. To boot a typical
computer, you'll need the following things:

1) Keyboard driver, so that you can customize boot-loader options on
next reboot.
2) Display driver, so you can present said boot-loader options to the
user on next reboot.
3) Mass storage driver, so you can load binary blobs into RAM for
subsequent execution.

Note that by "driver", I'm of course not suggesting a complete
"schedule/interrupt" interface with I/O request queues and the like.
I'm just talking about thin, purely-synchronous, relatively high-level
library routines that know how to translate "read sector X from drive
Y" into a corresponding block of data that a bootloader can work with.
That's about all that you need to get a system up and running
(everything else is nice-to-have gravy, at least in the home computer
context, and from personal experience, in a data-center environment as
well).

Playing a bit of devil's advocate, there's other things that might
crop up as well which tend to be hardware implementation specific yet
are common across operating systems. I'm pretty sure my timer
hardware will be quite different from Sifive's. How IRQs are mapped
are another consideration. Having a common SBI functionality for that
then becomes valuable, if the interface is thin enough. Otherwise,
you'll have to write drivers not just for supported external
peripherals, but also for entire motherboards.

As I mentioned in another message on the topic, a good BIOS (nee a
good SBI) will serve as a hardware abstraction layer, *just* thick
enough to support booting into the preferred OS environment, and
optionally, for operation of the OS as well. Just as CP/M still
needed a BIOS to be independent of concrete computer design issues, so
too would Linux, BSD, Plan 9, et. al. need similar services. Well,
OK, not *need*; but I would imagine the overhead of maintaining so
many different configurations of the kernel would certainly make it
strongly preferable, especially if SBI performance were on par with
calling any normal C function, or can be made to be called relatively
infrequently.

> is absolutely necessary? Eventually we will move all platform-specific
> stuff into the SBI implementation and that makes SBI another mini-OS

That is the crux right here, I think -- there are three groups of
people vying for influence on how system firmware should be handled:
(1) the pro-UEFI(-like) folks who thinks that system firmware should
include the kitchen sink, (2) the pro-BIOS folks (like myself) which
thinks it should be a thin abstraction over the available hardware,
and then (3) there's the anti-firmware folks who thinks that the
firmware should be opaque to any operating system you load, with
complete and total hand-off to the kernel once the bootloader has
completed its job.

> the M-mode firmware. There is needs for real-time performance and
> determinism.

There's also a real need for OS authors to have some manner of
reliable interface to a wider variety of system implementations during
the bootstrap process, and if I had my preference, during OS operation
as well.

Put bluntly, I'd much rather have a kernel which is slower but runs on
wider hardware than hyper-optimized kernels that can only run on one
motherboard's processor and/or chipset. Computers are fast enough for
me to not have to care about break-neck performance anymore. The only
exception I'd make for that is in a data center context, where now I'm
managing entire fleets of like-configured systems. Here, it makes
sense to maintain local kernel images, software repositories, et. al.
Ditto as well for deep-embedded environments.

But, for laptop use? Phone use? Tablet use? TV set-top box use?
The idea of maintaining a crap-ton of kernels and chipset/motherboard
drivers just does not appeal to me at all.

> > * Is there actually consensus amongst developers of S-mode code
> > (kernels, bootloaders etc) that the SBI is a desirable abstraction?
>
> I don't think so :)

Linux has a HAL. Windows uses a HAL. Most modern kernels rely upon a
HAL-like "thing" at some level. That's an SBI but by a different
name.

It's clear that it's a valuable abstraction. The question is, I
*think*, whether having a common interface across OSes is valuable.

--
Samuel A. Falvo II

Alex Bradbury

unread,
Nov 11, 2018, 1:57:42 PM11/11/18
to Samuel Falvo II, bmen...@gmail.com, sw-...@groups.riscv.org
On Sun, 11 Nov 2018 at 18:31, Samuel Falvo II <sam....@gmail.com> wrote:
> Linux has a HAL. Windows uses a HAL. Most modern kernels rely upon a
> HAL-like "thing" at some level. That's an SBI but by a different
> name.
>
> It's clear that it's a valuable abstraction. The question is, I
> *think*, whether having a common interface across OSes is valuable.

I think you've hit the nail on the head. Assuming agreement on the
need for some sort of SBI, should it be scoped to _only_ provide code
necessary for first boot of a kernel and maybe providing some basic
platform-specific functionality (tlb shootdown etc) that isn't
(currently) defined in the spec. Or should it actively seek to
increase code reuse across different OSes and provide a level of
abstraction that reduces platform-specific code? e.g. abstracting the
PLIC or other platform interrupt controller. If you go that far, why
stop there? You could provide helper functionality for handling page
tables, or implement a range of other device drivers in M-mode
firmware.

Speccing the complete desired interface is the only way to go I think.
As Christoph suggests, having a RISC-V way of saving user-visible
extension state isn't a bad idea (though of course this may just be
one small part of supporting a custom extension). But someone needs to
define what the interface for doing that would look like. Requiring an
extra ecall to provide this functionality sounds expensive.

Best,

Alex

Nick Kossifidis

unread,
Nov 11, 2018, 9:21:12 PM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
Hello Alex,

Στις 2018-11-10 23:25, Alex Bradbury έγραψε:
> I've been following the recent discussions on SBI extensions with
> great interest. Firstly, thank you to everyone who has been
> contributing SBI proposals and documenting the current de facto SBI
> standard.
>
> I do have one request - could we all take a step back and try to
> define the role and driving principles of the SBI? What problems
> should it be solving? What problems shouldn't be solved via the SBI? I
> understand once the framework is in place there's scope for people to
> use it in a wide variety of ways, but it's really hard to judge the
> various proposals without any clear definition of the SBI
> requirements. Proposing incremental ABI additions seems like the wrong
> way to go about development without agreeing this first.
>

+1

> The nearest thing we have to the guiding principles of the SBI is the
> stated aim in the current privspec draft "Using a single SBI across
> all SEE implementations allows a single OS binary image to run on any
> SEE". The document currently proposes SBI calls for:
> * Initialising, saving, and restoring user-mode state outside of the
> standard FDQ extensions
> * Inter-processor interrupts
> * Clear timer interrupts
> * Mask, unmask, and query the cause of external interrupts
> (apparently, all access to the PLIC should be through the SBI?)
> * Modifying the time/cycle/instret CSRs
> * Scheduling timer interrupts
> * TLB shootdown
>
> This is quite an expansion over the current SBI (which seems to offer
> a rather arbitrary subset of platform-specific functionality). I would
> observe that none of the proposals to extend the SBI so far actually
> cover all of these cases. Is that because there is disagreement over
> whether the SBI should serve all these purposes, or are the proposals
> simply incomplete?
>

Regarding IPI it can't be done by the supervisor, it needs to happen
on M mode (see privilege spec 1.11-draft, sec 3.1.14, p29), so it makes
sense to me to go through the SBI. The same goes for the remote fence.

Regarding timers, setting mtimecmp also needs M mode access so we have
to go through SBI for using it.

Regarding external interrupts, I guess we could have an SBI call for
handling mideleg but other than that the supervisor/userspace just use
their own sie/uie CSRs right now, so we don't use SBI for that and I
think the recommendation on the privilege spec should go away as well.

As for the cycle and instret CSRs they are handled directly by the
supervisor as well (see arch/riscv/kernel/perf_event.c). So again
maybe we should remove that recommendation from the privilege spec
at some point. By the way I don't think they can be modified, they
are marked as URO (privilege spec 1.11-draft sec. 2.2).

So to me what we have right now makes sense, we don't use the SBI as
an alternative to the supervisor or as an abstraction layer. It's
mostly an interface to the firmware for handling higher-privileged
functions the supervisor can't handle.

When it comes to saving/restoring user-mode etc state outside of
the standard extensions, I think it's related to the other issue
you mention, see below...

> Based on a recent software working group call, it seems there is a
> desire for the SBI to ensure that a single kernel works across all
> RISC-V implementations, including support for custom extensions (e.g.
> save/restore of user-visible state as mentioned above). I'm concerned
> that reaching this goal will require an increasing number of hooks
> into the kernel effectively making the SBI a mini-OS. e.g. for the
> tagged memory mechanism in lowRISC, saving/restoring the tagctrl
> register is a necessary component of kernel support but it's also
> necessary to modify the virtual memory system to support (at a
> minimum) clearing tags on pages and propagating tags upon
> copy-on-write.
>

Just so that everyone is on the same page here, the discussion started
on [1] and the problem at hand is that we don't want custom CSRs or
instructions of the various vendor-specific extensions ending up on
the Linux kernel since we want to keep the kernel clean and
independent of custom toolchains (I've explained the issues on [2]).
The proposed approach we discussed on the working group call and on
that thread, was to be able to call custom/vendor-specific code
through the SBI so that vendors can keep their extensions on the
firmware, instead of messing up the kernel (also see [3]). As an
example we talked about having a table of function pointers of the
hooks you mention, that will be initialized on early boot. If a hook
is not used it'll be a nop, if it is, a pointer to the function that
does the SBI call will be used. It's not an ideal solution but it's
better than the alternative.

> Other questions/concerns/thoughts:
> * Is there actually consensus amongst developers of S-mode code
> (kernels, bootloaders etc) that the SBI is a desirable abstraction?

Again, to be clear, right now SBI is not an abstraction, the
discussion for using it as an abstraction to "hide" custom extension
assembly code from the kernel is still ongoing (latest update was [3]
AFAIK).

I can speak only for myself here, the code that handles arch-specific
things within the OS/bootloader should be clean and simple. On RISC-V
this is a challenge to achieve. The ISA covers only the operation of
a hart (which to me it makes perfect sense), there are areas not
covered by the ISA that in contrast with other archs we don't know
what to expect ! There will be systems out there that have coherent
caches for example and don't need remote fence to e.g. update their
page tables, there will be systems that don't provide any coherency
and we should always force fence, there will be systems without caches
or with caches that cover only part of the memory etc. We should be
open for new things to come and ready to handle them in a clean way.
Software should be designed to support hardware's modularity, the
same way we allow extensions on hw, we should allow them on software.
Without a hooking mechanism in place the code will get chaotic with
if checks everywhere etc. Not all hooks need to use the SBI obviously,
but in case custom assembly code or M mode access etc is needed, I
don't see any other way than using the SBI.

> * Shouldn't there be an interface to allow S-mode code to control
> delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
> supply its own floating point emulation code. Or maybe an alternative
> mechanism to swap out the M-mode handler code (as Ron has been
> advocating)

I guess it's a valid request to use the SBI for controlling mideleg,
feel free to suggest it to Atish for inclusion. To me it should be
optional since minimizing / eliminating the barrier between M mode
and the rest is a security control bypass. I can see the benefit
though in some cases.

> * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> intended to work in the case where a hart supports switching between
> these? Surely an S-mode OS may be running with a different ISA variant
> to the M-mode SBI/SEE? How is this handled?

For the current calling convention, check out
arch/riscv/include/asm/sbi.h and [4]. There is a weak spot IMHO:

Passing 32bit values to/from M mode through a* registers is straight
forward and should be compatible across different base ISAs. However
passing pointers is another beast. According to the spec Sv32/39/48
are part of the supervisor-level ISA, there is nothing about what kind
of virtual memory mapping scheme the firmware on M mode should use. I
guess from the code that the assumption is that M mode will keep the
same scheme and page table etc that the supervisor has configured, but
there is no documentation on that. At least we know that the sbi
call will happen on the same hart as the caller so it should be
fine.

> * There are ideas floating around about SBI calls being more like
> function calls (rather than the current ecall mechanism). Has anyone
> written down how this would work?
>

On [2] I wrote "If we see that the SBI has too much overhead or is not
enough we can design it in a better way or extend it". One way of
doing that which I suggested on the call and Palmer also wrote about
on [3] is to export SBI as a library, sort of how the vDSO works on
Linux -> http://man7.org/linux/man-pages/man7/vdso.7.html. It seems
that an early version of the SBI used a similar approach instead
of ecalls. Apart from the idea, no one has come up so far with a
document on how this can be implemented, I guess the way Linux
does it should be a start.

Hope I helped !

Regards,
Nick

> How about we start by agreeing and enumerating the requirements, then
> work our way up?
>
> Thanks,
>
> Alex

[1]: https://lkml.org/lkml/2018/10/31/431
[2]: https://lkml.org/lkml/2018/11/6/2
[3]: https://lkml.org/lkml/2018/11/7/189
[4]: https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md

Bruce Hoult

unread,
Nov 11, 2018, 9:26:39 PM11/11/18
to Alex Bradbury, Samuel Falvo II, bmen...@gmail.com, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 11:01 AM, Alex Bradbury <a...@lowrisc.org> wrote:
On Sun, 11 Nov 2018 at 18:31, Samuel Falvo II <sam....@gmail.com> wrote:
> Linux has a HAL.  Windows uses a HAL.  Most modern kernels rely upon a
> HAL-like "thing" at some level.  That's an SBI but by a different
> name.
>
> It's clear that it's a valuable abstraction.  The question is, I
> *think*, whether having a common interface across OSes is valuable.

I think you've hit the nail on the head. Assuming agreement on the
need for some sort of SBI, should it be scoped to _only_ provide code
necessary for first boot of a kernel and maybe providing some basic
platform-specific functionality (tlb shootdown etc) that isn't
(currently) defined in the spec. Or should it actively seek to
increase code reuse across different OSes and provide a level of
abstraction that reduces platform-specific code? e.g. abstracting the
PLIC or other platform interrupt controller. If you go that far, why
stop there? You could provide helper functionality for handling page
tables, or implement a range of other device drivers in M-mode
firmware.

The absolutely essential is the kind of thing that might be included in hardware in a CISC architecture, but we do in low level software on RISC-V. The same kind of idea as DEC Alpha "PALCode".

However I think I'd like to see enough there that a basic OS *could* choose to use nothing but SBI calls to talk to the world. But would not be forced to, and could use its own drivers if it wants.

So I'd include something to enumerate character and block devices and read and write them. *Without* any interpretation of the contents. The OS itself would have to provide the code that knows how to identify and work with partition tables, FAT or EXT filesystems etc.

Manipulating page table layouts I'm probably against. They are well defined in the privileged spec, so it's only things that follow a different privileged spec that would benefit. I think the only divergence we should see there is machines too small to run a proper OS anyway.

However TLB design, functioning, and management on the other hand is completely undefined in the spec. Technically, you don't even have to have one, and every memory reference could walk the page tables. Calls to manage local and remote TLBs would be useful. I'd include that.

Abstracting interrupts doesn't seem too useful. PLIC is already pretty simple to deal with and the newer CLIC is backwards compatible with it. I'd imagine that would continue in any future enhancement, though a lot of thought has gone into CLIC and it will probably serve well for a long time.

The SBI code shouldn't be very big. Single figure KB I guess. The current BBL is 27 KB, but includes things an SBI probably should have such as a floating point emulation library.

 
 
Speccing the complete desired interface is the only way to go I think.
As Christoph suggests, having a RISC-V way of saving user-visible
extension state isn't a bad idea (though of course this may just be
one small part of supporting a custom extension). But someone needs to
define what the interface for doing that would look like. Requiring an
extra ecall to provide this functionality sounds expensive.

An ecall the provides a service for a process, rather than switching processes, doesn't have to be much more expensive than a standard function call.

Alan Kao

unread,
Nov 11, 2018, 10:43:34 PM11/11/18
to Nick Kossifidis, Alex Bradbury, sw-...@groups.riscv.org
Hi Nick,
I would like to add some comment here since I am resiponsible for the
mentioned source file.

Perf (which performance counters is mainly for) needs some HW features
that are not currently available. Please also check
Documentations/riscv/pmu.txt if you want to know more details. You already
mentioned one instance here that kernel cannot modify these counters,
so a SBI to enable writing will be a must.

I will keep a lite proposal about perf until RISC-V MC on Thursday unless
any of you are urged to know perf-related issues.

Alan

atish patra

unread,
Nov 11, 2018, 11:38:47 PM11/11/18
to j.neus...@gmx.net, a...@lowrisc.org, sw-...@groups.riscv.org
As per my understanding, this is the correct interpretation.
Once, we have power management in place, we can decide on correct power-off
sequence and board specific shutdown interface.
 

Under interpretation (b),

- Board level power-off is a complex thing that might require various
  drivers (I²C, SPI, IPMI, ...) in order to talk to off-chip power
  management ICs. Moving these drivers into M-mode firmware is a bad
  idea IMHO, for the following reasons:
  - More M-mode code *potentially* means more M-mode attack surface,
    weakening the security of M-mode implementations.
  - Bugs in M-mode code may be harder to patch than bugs in the S-mode
    operating system, because the M-mode firmware might be stored on
    special flash, or signed with the SoC vendor's key, etc.
  - Having drivers in M-mode and S-mode for the same hardware may cause
    some conflicts and race-conditions when both drivers access the same
    hardware at the same time.


Kind regards,
Jonathan Neuschäfer


[1]: https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md
[2]: S-mode shutdown code *could* be written like this:
       start_poweroff_sequence();
       sbi_shutdown();
       // sbi_shutdown halts all harts
       // the power-off sequence completes and the board is powered off
     But that seems prone to timing problems, to me.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/20181111171431.GA2219%40latitude.


--
Regards,
Atish

atish patra

unread,
Nov 11, 2018, 11:42:49 PM11/11/18
to a...@lowrisc.org, h...@lst.de, sw-...@groups.riscv.org
I am not sure why SBI is required for PLIC either. IIRC, it was mentioned in a very old version of privileged spec (1.9.x).
But it got removed eventually. As per current implementation, all PLIC handling is happening directly through kernel.
Even current SBI implementation doesn't have anything related to that.
 
Regards,
Atish
Best,

Alex


--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

atish patra

unread,
Nov 12, 2018, 12:55:31 AM11/12/18
to ala...@andestech.com, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
I believe you are talking about writing to performance counters.
What's the common practice in other architectures? If it can be added to the privileged spec, 
we probably should. Otherwise, we can discuss in details during Plumbers. 
But, my general suggestion is that SBI shouldn't be used as a kitchen sink.

I will keep a lite proposal about perf until RISC-V MC on Thursday unless
any of you are urged to know perf-related issues.


Great looking forward to it.
 
Alan


--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

atish patra

unread,
Nov 12, 2018, 1:31:18 AM11/12/18
to a...@lowrisc.org, sw-...@groups.riscv.org
Hi Alex,
Thanks for starting this topic. I believe you raised a very valid point.
IMHO, SBI should not become UEFI or a "mini" OS. I view SBI should provide
an interface for S mode that helps in maintaining a clear separation between privileged
mode & machine mode. It should include features that are absolutely necessary and would
need a hack to implement in supervisor mode. This is just my opinion. 

As RISC-V is modern architecture and we all envision it to be used in various types of computing
devices, the idea with the proposal is to introduce minimal functions and keep them optional.
For example, if any embedded/iot devices doesn't need to use hart power management, they
can skip it and same supervisor OS will continue to work.

However, the primary goal is to have a proper function ID scheme & introduce version/feature check
capabilities at this point. That will help in adding any feature (i.e. absolutely necessary) in future.
Currently, it is not possible at all.

I am not sure if it will be helpful. But I can maintain a section of list of new feature suggestions in the
proposal. We can take a call if that feature is going to be added to SBI or not after it has been discussed
enough in public forums. But we need a continuous feedback/discussion on this so that we can finalize
a version of SBI at least.

On Sat, Nov 10, 2018 at 1:21 PM Alex Bradbury <a...@lowrisc.org> wrote:
I've been following the recent discussions on SBI extensions with
great interest. Firstly, thank you to everyone who has been
contributing SBI proposals and documenting the current de facto SBI
standard.

I do have one request - could we all take a step back and try to
define the role and driving principles of the SBI? What problems
should it be solving? What problems shouldn't be solved via the SBI? I
understand once the framework is in place there's scope for people to
use it in a wide variety of ways, but it's really hard to judge the
various proposals without any clear definition of the SBI
requirements. Proposing incremental ABI additions seems like the wrong
way to go about development without agreeing this first.

The nearest thing we have to the guiding principles of the SBI is the
stated aim in the current privspec draft "Using a single SBI across
all SEE implementations allows a single OS binary image to run on any
SEE".  The document currently proposes SBI calls for:
* Initialising, saving, and restoring user-mode state outside of the
standard FDQ extensions
* Inter-processor interrupts
* Clear timer interrupts
* Mask, unmask, and query the cause of external interrupts
(apparently, all access to the PLIC should be through the SBI?)
* Modifying the time/cycle/instret CSRs
* Scheduling timer interrupts
* TLB shootdown

This is quite an expansion over the current SBI (which seems to offer
a rather arbitrary subset of platform-specific functionality). I would
observe that none of the proposals to extend the SBI so far actually
cover all of these cases. Is that because there is disagreement over
whether the SBI should serve all these purposes, or are the proposals
simply incomplete?

Based on a recent software working group call, it seems there is a
desire for the SBI to ensure that a single kernel works across all
RISC-V implementations, including support for custom extensions (e.g.
save/restore of user-visible state as mentioned above). I'm concerned
that reaching this goal will require an increasing number of hooks
into the kernel effectively making the SBI a mini-OS. e.g. for the
tagged memory mechanism in lowRISC, saving/restoring the tagctrl
register is a necessary component of kernel support but it's also
necessary to modify the virtual memory system to support (at a
minimum) clearing tags on pages and propagating tags upon
copy-on-write.

Other questions/concerns/thoughts:
* Is there actually consensus amongst developers of S-mode code
(kernels, bootloaders etc) that the SBI is a desirable abstraction?
* Shouldn't there be an interface to allow S-mode code to control
delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
supply its own floating point emulation code. Or maybe an alternative
mechanism to swap out the M-mode handler code (as Ron has been
advocating)
* We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
intended to work in the case where a hart supports switching between
these? Surely an S-mode OS may be running with a different ISA variant
to the M-mode SBI/SEE? How is this handled?
* There are ideas floating around about SBI calls being more like
function calls (rather than the current ecall mechanism). Has anyone
written down how this would work?

How about we start by agreeing and enumerating the requirements, then
work our way up?

Thanks,

Alex

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Alex Bradbury

unread,
Nov 12, 2018, 4:12:07 AM11/12/18
to mi...@ics.forth.gr, sw-...@groups.riscv.org
With the exception of sbi_console_putchar, sbi_console_getchar and
maybe sbi_shutdown perhaps. Though I know you argued elsewhere that
the serial should _only_ be used for earlyprintk or the equivalent.
I'm not quite sure how access to the serial port should be mediated,
but that makes sense.

> Just so that everyone is on the same page here, the discussion started
> on [1] and the problem at hand is that we don't want custom CSRs or
> instructions of the various vendor-specific extensions ending up on
> the Linux kernel since we want to keep the kernel clean and
> independent of custom toolchains (I've explained the issues on [2]).
> The proposed approach we discussed on the working group call and on
> that thread, was to be able to call custom/vendor-specific code
> through the SBI so that vendors can keep their extensions on the
> firmware, instead of messing up the kernel (also see [3]). As an
> example we talked about having a table of function pointers of the
> hooks you mention, that will be initialized on early boot. If a hook
> is not used it'll be a nop, if it is, a pointer to the function that
> does the SBI call will be used. It's not an ideal solution but it's
> better than the alternative.

Thanks for the summary and pointers. I understand that there's a focus
on developing a Linux kernel policy, but I think it would be
worthwhile exposing more of these discussions to the wider RISC-V
community. Although not a requirement, it seems it would be beneficial
for different RISC-V compatible OSes/kernels to arrive at a similar
policy and agree the SBI interface. Given that Palmer has labeled both
sw-dev and isa-dev as "worthless"
<https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/mhng-2e75d6c3-ec0f-42db-be43-cdba1cf52248%40palmer-si-x1c4>
I'm not really sure where that discussion should take place.

I think it's worth remembering that there's no reason custom
extensions shouldn't/wouldn't get supported in standard clang and GCC.
Also, in the cases where it it just comes down to accessing
non-standard CSRs, requiring that Linux kernel code use CSR numbers
rather than assembler support for CSR names doesn't seem too onerous.

Looking at the linked LKML threads, Arnd Bergmann suggested "If a
vendor extension requires kernel support, that support must be able to
be built into a kernel image without breaking support for CPUs that do
not have that extension, to allow building a single kernel image that
works on all CPUs." That seemed a reasonable starting point to me
<https://lkml.org/lkml/2018/11/5/283>


> > Other questions/concerns/thoughts:
> > * Is there actually consensus amongst developers of S-mode code
> > (kernels, bootloaders etc) that the SBI is a desirable abstraction?
>
> Again, to be clear, right now SBI is not an abstraction, the
> discussion for using it as an abstraction to "hide" custom extension
> assembly code from the kernel is still ongoing (latest update was [3]
> AFAIK).
>
> I can speak only for myself here, the code that handles arch-specific
> things within the OS/bootloader should be clean and simple. On RISC-V
> this is a challenge to achieve. The ISA covers only the operation of
> a hart (which to me it makes perfect sense), there are areas not
> covered by the ISA that in contrast with other archs we don't know
> what to expect ! There will be systems out there that have coherent
> caches for example and don't need remote fence to e.g. update their
> page tables, there will be systems that don't provide any coherency
> and we should always force fence, there will be systems without caches
> or with caches that cover only part of the memory etc. We should be
> open for new things to come and ready to handle them in a clean way.
> Software should be designed to support hardware's modularity, the
> same way we allow extensions on hw, we should allow them on software.
> Without a hooking mechanism in place the code will get chaotic with
> if checks everywhere etc. Not all hooks need to use the SBI obviously,
> but in case custom assembly code or M mode access etc is needed, I
> don't see any other way than using the SBI.

I'm concerned that supporting arbitrary extensions via SBI hooks is
too ambitious. Using an SBI call to provide some equivalent to X86
XSAVE has value and it should be enough to support a simple extension
that provides some user-visible state. But it's not sufficient for all
such extensions. e.g. the example of tagged memory I gave previously,
I'd imagine a range of privspec extensions. Is the intent that people
introduce new hooks whenever necessary? Or do we just accept that a
limited number of SBI hooks handles a lot of common cases, but not
everything? I'm just trying to encourage some explicitness about the
end goal here - again, it's always helpful to have a clear picture of
what problems a scheme doesn't solve (or doesn't try to solve) when
evaluating it.

> > * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> > intended to work in the case where a hart supports switching between
> > these? Surely an S-mode OS may be running with a different ISA variant
> > to the M-mode SBI/SEE? How is this handled?
>
> For the current calling convention, check out
> arch/riscv/include/asm/sbi.h and [4]. There is a weak spot IMHO:
>
> Passing 32bit values to/from M mode through a* registers is straight
> forward and should be compatible across different base ISAs. However
> passing pointers is another beast. According to the spec Sv32/39/48
> are part of the supervisor-level ISA, there is nothing about what kind
> of virtual memory mapping scheme the firmware on M mode should use. I
> guess from the code that the assumption is that M mode will keep the
> same scheme and page table etc that the supervisor has configured, but
> there is no documentation on that. At least we know that the sbi
> call will happen on the same hart as the caller so it should be
> fine.

We of course run in to problems if more complex datastructures are
passed over the SBI. e.g. the structs containing two longs proposed by
Atish will obviously differ depending on -mabi=ilp32 vs -mabi=lp64.
Given that the SBI is meant to be extensible, these concerns should be
clearly documented in order to help vendors make the right choices
when extending the SBI.

> > * There are ideas floating around about SBI calls being more like
> > function calls (rather than the current ecall mechanism). Has anyone
> > written down how this would work?
> >
>
> On [2] I wrote "If we see that the SBI has too much overhead or is not
> enough we can design it in a better way or extend it". One way of
> doing that which I suggested on the call and Palmer also wrote about
> on [3] is to export SBI as a library, sort of how the vDSO works on
> Linux -> http://man7.org/linux/man-pages/man7/vdso.7.html. It seems
> that an early version of the SBI used a similar approach instead
> of ecalls. Apart from the idea, no one has come up so far with a
> document on how this can be implemented, I guess the way Linux
> does it should be a start.
>
> Hope I helped !

You helped massively, thanks!

Bes,t

Alex

Alex Bradbury

unread,
Nov 12, 2018, 4:18:25 AM11/12/18
to atis...@gmail.com, h...@lst.de, sw-...@groups.riscv.org
On Mon, 12 Nov 2018 at 04:42, atish patra <atis...@gmail.com> wrote:
> I am not sure why SBI is required for PLIC either. IIRC, it was mentioned in a very old version of privileged spec (1.9.x).
> But it got removed eventually. As per current implementation, all PLIC handling is happening directly through kernel.
> Even current SBI implementation doesn't have anything related to that.

The current privileged spec draft states:
"""
In the expected use case, only machine mode accesses the source
priority, source pending, and target interrupt enables to configure
the interrupt subsystem. Lower-privilege modes access these features
via ABI or SBI calls. The interrupt enables act as a protection
mechanism where a target can only signal completion to an interrupt
gateway that is currently enabled for that target.
"""

Best,

Alex

Alex Bradbury

unread,
Nov 12, 2018, 4:43:11 AM11/12/18
to atis...@gmail.com, sw-...@groups.riscv.org
On Mon, 12 Nov 2018 at 06:31, atish patra <atis...@gmail.com> wrote:
>
> Hi Alex,
> Thanks for starting this topic. I believe you raised a very valid point.
> IMHO, SBI should not become UEFI or a "mini" OS. I view SBI should provide
> an interface for S mode that helps in maintaining a clear separation between privileged
> mode & machine mode. It should include features that are absolutely necessary and would
> need a hack to implement in supervisor mode. This is just my opinion.
>
> As RISC-V is modern architecture and we all envision it to be used in various types of computing
> devices, the idea with the proposal is to introduce minimal functions and keep them optional.
> For example, if any embedded/iot devices doesn't need to use hart power management, they
> can skip it and same supervisor OS will continue to work.
>
> However, the primary goal is to have a proper function ID scheme & introduce version/feature check
> capabilities at this point. That will help in adding any feature (i.e. absolutely necessary) in future.
> Currently, it is not possible at all.
>
> I am not sure if it will be helpful. But I can maintain a section of list of new feature suggestions in the
> proposal. We can take a call if that feature is going to be added to SBI or not after it has been discussed
> enough in public forums. But we need a continuous feedback/discussion on this so that we can finalize
> a version of SBI at least.

My suggestion is that it's really hard to give that feedback and see
that discussion moving forwards productively without 1) a clear but
brief description of the scope and purpose of the SBI and the problems
it's trying to solve. 2) a proposal that at least sketches out how the
full range of requirements should be handled (e.g. is XSAVE-style
functionality provided via the same ecall mechanism? or is a function
pointer provided via an SBI call that is called directly? or something
else?).

Best,

Alex

David Craven

unread,
Nov 12, 2018, 6:03:18 AM11/12/18
to a...@lowrisc.org, atis...@gmail.com, sw-...@groups.riscv.org
> Given that Palmer has labeled both sw-dev and isa-dev as "worthless"
<https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/mhng-2e75d6c3-ec0f-42db-be43-cdba1cf52248%40palmer-si-x1c4>
I'm not really sure where that discussion should take place.

Maybe someone can create a ra...@groups.riscv.org mailing list and
kindly ask users to use the rant mailing list for ranting. That way
technical discussion can still happen here.

Bin Meng

unread,
Nov 12, 2018, 7:09:17 AM11/12/18
to j.neus...@gmx.net, a...@lowrisc.org, sw-...@groups.riscv.org
Agreed. That's why I asked why do we need SBI at all in the first
place. None of services provided by current SBI spec seem to be really
necessary for M-mode. And "providing a virtual interface to the PLIC
(or presumably an alternative interrupt controller?)" seems to be on
the wrong track. I agreed with what Christoph Hellwig said, we should
leave it for S-mode OS to drive it directly. And this is not only for
PLIC, but for other hardware components like timer/IPI, etc.

> - More M-mode code *potentially* means more M-mode attack surface,
> weakening the security of M-mode implementations.
> - Bugs in M-mode code may be harder to patch than bugs in the S-mode
> operating system, because the M-mode firmware might be stored on
> special flash, or signed with the SoC vendor's key, etc.
> - Having drivers in M-mode and S-mode for the same hardware may cause
> some conflicts and race-conditions when both drivers access the same
> hardware at the same time.
>

Regards,
Bin

Bin Meng

unread,
Nov 12, 2018, 7:36:11 AM11/12/18
to Samuel Falvo II, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Samuel,

On Mon, Nov 12, 2018 at 2:31 AM Samuel Falvo II <sam....@gmail.com> wrote:
>
> On Sat, Nov 10, 2018 at 6:34 PM Bin Meng <bmen...@gmail.com> wrote:
> > I agree. Maybe one more question: why is SBI a must-have for RISC-V?
> > To me current SBI is not absolutely necessary. All these
> > functionalities can be implemented by the S-mode OS. Why do they have
> > to be M-mode? One may say some hardware components can only be
> > accessed by M-mode (eg: IPI) but why should hardware expose such
> > limitation?
>
> I think it boils down to having some manner of hardware abstraction
> just sufficient enough to bootstrap that S-mode OS. To boot a typical
> computer, you'll need the following things:
>
> 1) Keyboard driver, so that you can customize boot-loader options on
> next reboot.
> 2) Display driver, so you can present said boot-loader options to the
> user on next reboot.
> 3) Mass storage driver, so you can load binary blobs into RAM for
> subsequent execution.
>
> Note that by "driver", I'm of course not suggesting a complete
> "schedule/interrupt" interface with I/O request queues and the like.
> I'm just talking about thin, purely-synchronous, relatively high-level
> library routines that know how to translate "read sector X from drive
> Y" into a corresponding block of data that a bootloader can work with.

coreboot/U-Boot can meet such requirement very well :)

> That's about all that you need to get a system up and running
> (everything else is nice-to-have gravy, at least in the home computer
> context, and from personal experience, in a data-center environment as
> well).
>
> Playing a bit of devil's advocate, there's other things that might
> crop up as well which tend to be hardware implementation specific yet
> are common across operating systems. I'm pretty sure my timer
> hardware will be quite different from Sifive's. How IRQs are mapped
> are another consideration. Having a common SBI functionality for that
> then becomes valuable, if the interface is thin enough. Otherwise,
> you'll have to write drivers not just for supported external
> peripherals, but also for entire motherboards.

It's definitely no problem that different silicon vendors choose
different timer IP in their chipset. I am talking about SoC timers.
But from architecture perspective, given RISC-V priv spec already
defines a real-time timer, the driver can be generic as long as the
spec is clearly defined. S-mode OS can load such timer driver and talk
to the architecture defined timer directly. There is no need to go
through SBI. This is the same as x86 TSC, or PowerPC decrementer. I do
not think SBI should provide abstraction for these different SoC
timers given there is already an architecture defined timer.

>
> As I mentioned in another message on the topic, a good BIOS (nee a
> good SBI) will serve as a hardware abstraction layer, *just* thick
> enough to support booting into the preferred OS environment, and
> optionally, for operation of the OS as well. Just as CP/M still
> needed a BIOS to be independent of concrete computer design issues, so
> too would Linux, BSD, Plan 9, et. al. need similar services. Well,
> OK, not *need*; but I would imagine the overhead of maintaining so
> many different configurations of the kernel would certainly make it
> strongly preferable, especially if SBI performance were on par with
> calling any normal C function, or can be made to be called relatively
> infrequently.
>
> > is absolutely necessary? Eventually we will move all platform-specific
> > stuff into the SBI implementation and that makes SBI another mini-OS
>
> That is the crux right here, I think -- there are three groups of
> people vying for influence on how system firmware should be handled:
> (1) the pro-UEFI(-like) folks who thinks that system firmware should
> include the kitchen sink, (2) the pro-BIOS folks (like myself) which
> thinks it should be a thin abstraction over the available hardware,
> and then (3) there's the anti-firmware folks who thinks that the
> firmware should be opaque to any operating system you load, with
> complete and total hand-off to the kernel once the bootloader has
> completed its job.
>

I did not see a lot of voices from group 1. Maybe I missed them. I am
OK with either 2 or 3. But my point is we should only allow whatever
is absolutely necessary to go into SBI.

> > the M-mode firmware. There is needs for real-time performance and
> > determinism.
>
> There's also a real need for OS authors to have some manner of
> reliable interface to a wider variety of system implementations during
> the bootstrap process, and if I had my preference, during OS operation
> as well.
>

I understand this.

> Put bluntly, I'd much rather have a kernel which is slower but runs on
> wider hardware than hyper-optimized kernels that can only run on one
> motherboard's processor and/or chipset. Computers are fast enough for
> me to not have to care about break-neck performance anymore. The only

Fast enough computers cannot be the excuse for putting everything into SBI.

> exception I'd make for that is in a data center context, where now I'm
> managing entire fleets of like-configured systems. Here, it makes
> sense to maintain local kernel images, software repositories, et. al.
> Ditto as well for deep-embedded environments.
>

Yes, my concern was for deep-embedded environments. I don't think
RISC-V was invented only for mainstream desktop or high-end servers.

> But, for laptop use? Phone use? Tablet use? TV set-top box use?
> The idea of maintaining a crap-ton of kernels and chipset/motherboard
> drivers just does not appeal to me at all.
>
> > > * Is there actually consensus amongst developers of S-mode code
> > > (kernels, bootloaders etc) that the SBI is a desirable abstraction?
> >
> > I don't think so :)
>
> Linux has a HAL. Windows uses a HAL. Most modern kernels rely upon a
> HAL-like "thing" at some level. That's an SBI but by a different
> name.
>
> It's clear that it's a valuable abstraction. The question is, I
> *think*, whether having a common interface across OSes is valuable.
>

Regards,
Bin

Bin Meng

unread,
Nov 12, 2018, 8:22:09 AM11/12/18
to mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Nick,
Yes, it was unfortunately designed this way. Sigh. My question was why
does hardware expose such limitation that accessing timer/IPI should
only be done in M-mode? It could have only be one instruction of
CSR/MMIO read to get a timer counter, but now it has to call into SBI
and execute maybe one hundred instructions. This affects real-time
performance a lot.
If we could have done SBI this way, I would be much less concerned.

> of ecalls. Apart from the idea, no one has come up so far with a
> document on how this can be implemented, I guess the way Linux
> does it should be a start.
>
> Hope I helped !
>

Yes, the discussion threads you shared are really helpful. Thanks!

Regards,
Bin

Samuel Falvo II

unread,
Nov 12, 2018, 1:20:31 PM11/12/18
to Bin Meng, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 4:36 AM Bin Meng <bmen...@gmail.com> wrote:
> coreboot/U-Boot can meet such requirement very well :)

Of course; but, I think we're both starting to digress.

Logically, if you agree that these services are provided solely by
coreboot, u-boot, et. al., then it follows these products ultimately
define the SBI. This entire discussion, then, all becomes irrelevant,
at least insofar as these mailing lists are concerned. New feature
development for the SBI should then be done on the mailing lists for
coreboot, u-boot, et. al. All we would be doing here is
bike-shedding.

By contradiction, then, that we are discussing the SBI here suggests
we want something more from the SBI. Something coreboot and u-boot
and whatever else there is cannot provide on their own.

> But from architecture perspective, given RISC-V priv spec already
> defines a real-time timer, the driver can be generic as long as the
> spec is clearly defined. S-mode OS can load such timer driver and talk

It genuinely pains me that I cannot use something as simple as a timer
as an example of a class of hardware which *could* (but not
necessarily *will*) differ from platform to platform without having to
justify my argument. It was clearly a hypothetical example, intended
to highlight why someone *might* want to implement or use an SBI call
in the context of a hardware abstraction layer. I depend upon the
experiences of the reader to be able to extrapolate from this simple
example into more complex scenarios.

But, let's roll with it. While mtime/mtimecmp are adequately
specified for the purposes of getting something working on one
platform, I don't think it's adequately specified to be considered
*portable*. Open questions, from a software portability concern,
include but I'm sure aren't limited to the following:

- What is the base address of mtime and mtimecmp? By your own
admission, the hardware interface defined so far is too simple to be
considered for SBI calls. So, where do I get the base address from?
Right now, the only official answer is the device tree. But, having
to include a multi-kilobyte parser for this structure (and it will be
multi-kilobyte based on my experience just implementing *simple*
procedures in my own firmware images) is, to be blunt, thoroughly
unpalatable. It's noise for a kernel the size of Linux, but it'd be
rather significant overhead for something the size of a small
microkernel.

- Is mtime and mtimecmp obligated to be located in a page which can be
assigned to run in supervisor mode? We know a priori from the
description of PMAs and PMPs that each RISC-V processor has to expose
a flag to address decoding logic as to whether or not it's currently
running in M-mode -- it's the only way PMPs can do what they do.
Ergo, the decision of whether or not mtime and mtimecmp are S-mode
accessible at all is, ultimately, up to the SoC or system integrator.
Knowing that it *is* allowed by the standard to delegate mtimecmp
interrupts into S-mode, is it still OK for a platform to restrict
mtime and mtimecmp to M-mode only regions of memory? It is not
specified, and therefore, conservatively, we must assume the answer to
be yes! If so, then despite how simple mtime/mtimecmp logic is to use
from a purely software interface, you would have literally *no*
*choice* but to use an SBI to access mtime and/or mtimecmp. Nowhere
in the specification that I've read (maybe I missed something?)
compels an implementation to expose mtime and mtimecmp in a manner
that allows it to be accessed via S-mode.

- What frequency does mtime count at? (Again, I feel I should not
have to include an entire DT parser to find this information out.)

- What even is the *purpose* of mtime/mtimecmp? Or, put another way,
why *must* it be M-mode only in the first place? Here's something you
can't encode in a DT structure. If mtime/mtimecmp are to be used for
time-slicing, why use an absolute, one-shot timer? Why not use
relative, periodic timers instead? That at least saves the overhead
of computing the reload value of mtimecmp. I strongly suspect this
timer is *not* intended for this use-case, but rather intended more
for the kinds of software that would use alarm(2) instead. But, even
this raises some issues, because processes generally tend to have
affinities to specific harts, while mtimecmp interrupts might be
routed to any operating hart. That means an alarm could also involve
the overhead of an IPI. It's more efficient if each hart maintains
its own mtime/mtimecmp pair under these circumstances, which once
again argues strongly for moving mtimecmp back into CSR space. If it
is intended to be used to trigger alarms, what kinds of alarms would
an M-mode binary blob need? M-Mode is supposed to be used a la
PALcode on the Alpha; it should be the highest priority code, and per
any common sense kernel design rules of thumb, correspondingly the
*least likely to execute*. So, if there's no clear use-case, maybe
mtime/mtimecmp should be demoted to stime/stimecmp instead.

> I did not see a lot of voices from group 1.

I'm not skilled enough to search Google Groups and provide links, but
there have been extensive discussions on the value proposition of UEFI
versus other system firmware implementations on the RISC-V mailing
lists in months past. These discussions are not recent, but they have
happened.

> OK with either 2 or 3. But my point is we should only allow whatever
> is absolutely necessary to go into SBI.

We are not in disagreement here.

> Fast enough computers cannot be the excuse for putting everything into SBI.

Please refer to me where I advocated putting everything into an SBI,
and I'll reply with quotations of my messages advocating strongly
*against* doing this. Please do not misrepresent my intentions, and
be sure to read my text in the proper and intended context. If you
have questions about what that context may be, I'd much rather you ask
for clarification than just assume.

Samuel Falvo II

unread,
Nov 12, 2018, 1:22:39 PM11/12/18
to Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 5:22 AM Bin Meng <bmen...@gmail.com> wrote:
> Yes, it was unfortunately designed this way. Sigh. My question was why
> does hardware expose such limitation that accessing timer/IPI should
> only be done in M-mode? It could have only be one instruction of

I wish I'd seen this message earlier before I sent my last response.
This gives me increased sense that we are, in fact, more in
"frustrated agreement" than we are in opposition to each other. :)

Samuel Falvo II

unread,
Nov 12, 2018, 1:31:33 PM11/12/18
to Bin Meng, Alex Bradbury, sw-...@groups.riscv.org
Replying to myself for clarification. I wrote something in haste
which, after I just finished issuing a complaint on context
sensitivity of my text, reads entirely wrong in context. LOL!
Clarification below.

On Mon, Nov 12, 2018 at 10:23 AM Samuel Falvo II <sam....@gmail.com> wrote:
> I'm not skilled enough to search Google Groups and provide links, but

What I meant to say here is that I'm not skilled enough to issue a
query to Google Groups that yields a high enough signal to noise ratio
for me to copy links out of. I can of course issue queries in the
search box, but my experience with Google Groups is that it yields a
terribly low signal to noise ratio. Sorry for the amusing confusion.
:)

Bruce Hoult

unread,
Nov 12, 2018, 10:34:44 PM11/12/18
to Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 5:21 AM, Bin Meng <bmen...@gmail.com> wrote:
Yes, it was unfortunately designed this way. Sigh. My question was why
does hardware expose such limitation that accessing timer/IPI should
only be done in M-mode? It could have only be one instruction of
CSR/MMIO read to get a timer counter, but now it has to call into SBI
and execute maybe one hundred instructions. This affects real-time
performance a lot.

If it's 100 instructions then someone is doing something very very wrong. Try 10.
 

Samuel Falvo II

unread,
Nov 12, 2018, 10:46:15 PM11/12/18
to Bruce Hoult, Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
Including ecall function ID dispatch logic?



--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Bruce Hoult

unread,
Nov 12, 2018, 11:41:58 PM11/12/18
to Samuel Falvo II, Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
Of course.

Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:

lui t0,#ecallTableBase #or auipc
slli t1,a6,2 #3 for 64 bit
add t0,t0,t1
jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
# called function does the mret

That's five instructions.

You could do something like a "li t0,#nnn; blt a6,t0,heavyweight" to distinguish these simple calls from ones that actually need to save and restore registers. That makes it seven instructions.

I'd submit that trusted SBI calls should use the same ABI as regular function calls (i.e. a0-a7 and t0-t6 can come back trashed) rather than the Linux "all registers except designated return values come back unchanged" system call ABI.


On Mon, Nov 12, 2018 at 7:46 PM, Samuel Falvo II <sam....@gmail.com> wrote:
Including ecall function ID dispatch logic?



On Mon, Nov 12, 2018, 7:34 PM Bruce Hoult <bruce...@sifive.com wrote:
On Mon, Nov 12, 2018 at 5:21 AM, Bin Meng <bmen...@gmail.com> wrote:
Yes, it was unfortunately designed this way. Sigh. My question was why
does hardware expose such limitation that accessing timer/IPI should
only be done in M-mode? It could have only be one instruction of
CSR/MMIO read to get a timer counter, but now it has to call into SBI
and execute maybe one hundred instructions. This affects real-time
performance a lot.

If it's 100 instructions then someone is doing something very very wrong. Try 10.
 

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Samuel Falvo II

unread,
Nov 12, 2018, 11:59:08 PM11/12/18
to Bruce Hoult, Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 8:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
> lui t0,#ecallTableBase #or auipc
> slli t1,a6,2 #3 for 64 bit
> add t0,t0,t1
> jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
> # called function does the mret
>
> That's five instructions.

DUHH, of course. Not sure why I didn't see that earlier. For some
reason, I had it in my mind that you had system call overheads similar
to, e.g., Linux's system call dispatcher. Sorry for the brain-lapse.

Bin Meng

unread,
Nov 13, 2018, 12:12:16 AM11/13/18
to bruce...@sifive.com, Samuel Falvo II, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Bruce,

On Tue, Nov 13, 2018 at 12:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
>
> Of course.
>
> Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:
>
> lui t0,#ecallTableBase #or auipc

You need save t0 before using it, no?

> slli t1,a6,2 #3 for 64 bit

and t1, a6?

> add t0,t0,t1
> jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
> # called function does the mret
>
> That's five instructions.
>

Yes, for this particular part. But the called function (illegal
instruction handler in this case) to implement reading the mtimer
counter needs another bunch of instructions, including instruction
decoding, etc. Having said that, there is room for optimization if we
handcraft the handler in assembly codes carefully.

> You could do something like a "li t0,#nnn; blt a6,t0,heavyweight" to distinguish these simple calls from ones that actually need to save and restore registers. That makes it seven instructions.

Unless we write all ecall trap handlers in assembly codes, it's still
needed to do the blind register saving/restoring.

>
> I'd submit that trusted SBI calls should use the same ABI as regular function calls (i.e. a0-a7 and t0-t6 can come back trashed) rather than the Linux "all registers except designated return values come back unchanged" system call ABI.
>
>
> On Mon, Nov 12, 2018 at 7:46 PM, Samuel Falvo II <sam....@gmail.com> wrote:
>>
>> Including ecall function ID dispatch logic?
>>

Regards,
Bin

Bin Meng

unread,
Nov 13, 2018, 12:37:15 AM11/13/18
to Samuel Falvo II, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Samuel,
I was against the reason for making timer stuff abstracted by SBI is
due to different IPs. As you mentioned in the follow-up thread, you
and I are actually in "frustrated agreement". So it looks I may have
misunderstood your opinion.
Thanks a lot for the sum-up about current the RISC-V timer facility.
There are tremendously valuable questions and suggestions. I hope
someone from the RISC-V hardware design team can take a deeper look.

> > I did not see a lot of voices from group 1.
>
> I'm not skilled enough to search Google Groups and provide links, but
> there have been extensive discussions on the value proposition of UEFI
> versus other system firmware implementations on the RISC-V mailing
> lists in months past. These discussions are not recent, but they have
> happened.
>
> > OK with either 2 or 3. But my point is we should only allow whatever
> > is absolutely necessary to go into SBI.
>
> We are not in disagreement here.
>

Glad we are :)

> > Fast enough computers cannot be the excuse for putting everything into SBI.
>
> Please refer to me where I advocated putting everything into an SBI,
> and I'll reply with quotations of my messages advocating strongly
> *against* doing this. Please do not misrepresent my intentions, and
> be sure to read my text in the proper and intended context. If you
> have questions about what that context may be, I'd much rather you ask
> for clarification than just assume.
>

Sorry I misinterpret this. What I was saying was I am not in favor of
a slower kernel that was built on top of an SBI that abstracted
something not absolutely necessary. There are needs for real-time
performance and determinism and I hope such requirement does not get
ignored.

Regards,
Bin

Michael Clark

unread,
Nov 13, 2018, 1:05:39 AM11/13/18
to Luke Kenneth Casson Leighton, Bruce Hoult, Alex Bradbury, sw-...@groups.riscv.org
Luke.

Everyone has a different opinion and perspective. Almost everyone has been communicating politely. Trying to use other folks opinions or “ideas” as a rationale to act the way you do is no excuse. You make things personal. I’m also fond of the IETF model but it doesn’t work if the list is dominated by expletives, personal attacks, talk of mediation and conflict resolution. That is unnecessary for cooperative development.

Your ideas on the other hand stand on their own merit. This talk of “conflict resolution and mediation”, however, is completely unnecessary. Folk are not deciding whether or not they like ideas based on who proposed them.

No swearing from now on. PLEASE.

Michael

> On 11/11/2018, at 10:46 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
>
> ---
> crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
>
>> On Sun, Nov 11, 2018 at 8:00 AM Bruce Hoult <bruce...@sifive.com> wrote:
>>
>>> On Sat, Nov 10, 2018 at 11:46 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
>>>
>>> so the question to ask is not "how* come you are civilised and
>>> capable of staying within the bounds of technical discussion on other
>>> forums: the real question is, what's different about *these* forums?
>>
>>
>> In the last few hours I've seen you being adversarial with not only
>> Alex Bradbury,
>
> if you look carefully, alex was trying to be reasonable, and play a
> mediatorial role... except if you look carefully at the advice given
> on crhnq, a mediator needs to respect *both* sides, and relay (reflect
> back) the position of *both* parties.
>
> regrettably, he didn't do that, and unfortunately, all it did was
> make it look he wasn't listening to what i'm saying. as i was
> writing, i was thinking, "damn, damn damn, i really like alex, how the
> hell can i make this look any better, so it doesn't reflect badly on
> him?" and, sadly, as he (you, hello, yes, sorry to be talking _about_
> you, alex) hadn't summarised and acknowledged both perspectives as a
> mediator normally would, there really wasn't a way to do that.
>
>> but also Nick Kossifidis
>
> if you look carefully at what nick's writing, he made good points
> from the field in which he has experience.. unfortunately he laced it
> with hyperbolic adversarial questions that were very carefully
> designed to ridicule rather than critique not only the proposal that i
> made, they were also very very carefully crafted to ridicule me, as a
> person, as well.
>
> you'll notice that he dodged the positive aspects that were supported
> by other people's constructive input, and focussed on generating as
> much negativity as he could, through the use of what can only be
> termed "hysteria" questions, hoping to assign them to me (without my
> consent) and use them rhetorically for the purposes of ridicule.
>
> the initial (thinly-veiled) adversarial attacks that he launched were
> sufficiently over the top that someone else had to step in, and hint
> that he was going a little too far. nick made a conscious choice to
> ignore those hints.
>
> the question here i have to ask, is, can you see that that's what
> nick did, and, if you can, why do you feel that it's acceptable
> behaviour on a technical forum?
>
>> (who you for unknown reason persist in addressing as "Mick"),
>
> sorry, nick: small high-resolution screen here, and i run the
> brightness way down and the room lights off, most of the time.
>
>> and Tommy Murphy.
>
> tommy, bless him, i know he's trying to help. you may be confusing
> the pain that i am experiencing through having to deal with the
> ongoing bullying and expressing that, and thus interpreting my
> responses to him as "this must be an attack on tommy". categorically
> and absolutely it is not.
>
>> And adopting a derisory tone with others such as Ron Minnich.
>
> what?? not at all! ron makes a lot of sense, and i've said so quite
> clearly!! what on *earth* gave you the impression that i've been
> "derisory" with ron?? i've made it quite clear that the points he's
> made about being able to operate fully in M-Mode are good ones!
>
>> There's a common thread here.
>
> there is, indeed. the challenge is, then, if the opportunity to
> attack arises, if someone appears to be vulnerable, in pain, or
> appears to not be knowledgeable, is for people to step up to the plate
> and *not* react in a derisory fashion, *not* attack or denigrate them,
> make them feel so unwelcome and so despised that it actually
> deteriorates their physical health.
>
> you know i speak my mind, and i made a decision 20 years ago to speak
> truth, even when it's uncomfortable truth. that's what people face
> here, and i know they don't like it.
>
> i told you: ethical's a bitch. truth, awareness, love and creativity
> aren't "nice", they're damn uncomfortable, and dealing with this is so
> hurtful it's *literally* making me physically ill.
>
> yet... i have a goal to achieve, the target's been set, to create a
> completely libre, open and freely-available 3D-capable processor, and
> to *develop* it in a truly libre and open fashion, and i am not going
> to go away until that goal is achieved.
>
> so, we have at least two more years of this. that's the reality.
> so, we can use these forums for the benefit of RISC-V, or not. which
> is it going to be?
>
> we can work together, to improve RISC-V and make a conscious decision
> to make the forums a welcoming place... or we can turn it into a
> hate-filled forum where students, corporate employees and researchers
> treat it - and RISC-V - as an absolute "No Go Zone", and psychology
> researchers use the public archives in case-studies over the next
> couple of decades, on how open communities go into denial when faced
> with challenges.
>
> that's what we're looking at, here. it's up to everyone to make a
> conscious decision which direction they want to go.
>
>> These people don't talk to each other in the manner in which you reply to them.
>
> people don't talk to me on comp.arch or libre-riscv-dev in the manner
> in which they reply to me on here. why?
>
> l.
>
> --
> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> To post to this group, send email to sw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAPweEDwKwf4iF%2BE5sqV6wdB4%3DT%3D1rR8cAKH-Kb9g5cYmXtVtPg%40mail.gmail.com.

Bruce Hoult

unread,
Nov 13, 2018, 1:11:34 AM11/13/18
to Bin Meng, Samuel Falvo II, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 9:12 PM, Bin Meng <bmen...@gmail.com> wrote:
Hi Bruce,

On Tue, Nov 13, 2018 at 12:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
>
> Of course.
>
> Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:
>
> lui t0,#ecallTableBase #or auipc

You need save t0 before using it, no?

Not if, as I specified below, SBI ecalls have the same/similar ABI as they would if they were in a C library. 
 

> slli t1,a6,2 #3 for 64 bit

and t1, a6?

*Certainly* not a6, as that has the index of the desired SBI call, and we don't modify it.
 

> add t0,t0,t1
> jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
> # called function does the mret
>
> That's five instructions.
>

Yes, for this particular part. But the called function (illegal
instruction handler in this case) to implement reading the mtimer
counter needs another bunch of instructions, including instruction
decoding, etc.

I'm assuming here that you're making an explicit SBI call to get mtime, not handling an illegal instruction trap. Illegal instruction would definitely be a bit more heavy-weight with both decoding overheads and the necessity to save and restore everything it touches.

> You could do something like a "li t0,#nnn; blt a6,t0,heavyweight" to distinguish these simple calls from ones that actually need to save and restore registers. That makes it seven instructions.

Unless we write all ecall trap handlers in assembly codes, it's still
needed to do the blind register saving/restoring.

I would definitely write the smallest and simplest ones, that should be as fast as possible, in assembly language. Like getting timers, for example. And I'd make sure a 64 bit counter was returned on 32 bit systems (already made atomic, and possibly virtualised with an offset subtracted as well) so that only one SBI call was needed, not two.

Michael Clark

unread,
Nov 13, 2018, 3:19:30 AM11/13/18
to Richard W.M. Jones, h...@lst.de, Alex Bradbury, sw-...@groups.riscv.org


>> On 12/11/2018, at 12:31 AM, Richard W.M. Jones <rjo...@redhat.com> wrote:
>>
>> On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
>> Based on a recent software working group call, it seems there is a
>> desire for the SBI to ensure that a single kernel works across all
>> RISC-V implementations,
>
> Not sure what the "software working group call" is, but if this is in
> any way related to what I said in my talk in Barcelona then I'd like
> to clarify that I only meant for server platforms. Embedded folk can
> and most likely will go their own way.
>
> BTW the way you described the full scope of the (potential) SBI made
> it sound like UEFI.

In some ways UEFI (or a subset of) is a little clearer in scope than SBI a). because after one calls ExitBootServices() one has complete control over the machine with respect to UEFI. SMI issues aside as they are independent to UEFI. b) the UEFI API is relatively well thought through although perhaps a little heavyweight e.g. the use of GUIDs to get structures with function pointers is very Microsoft centric. Lots of optional APIs too e.g. do we really need to boot from SCSI tape?

I believe we need to separate these 3 concerns:

1). boot time services. arm’s EBBR has a minimal UEFI subset which IMHO is a good starting point (as an option)

2) runtime HAL e.g. interrupt controller virtualization, power management, etc i.e. platform device specification versus SBI driver? interfaces.

3). emulation and virtualization fixups or core support firmware e.g. trap-and-emulate in RISC-V code vs micro-code - features such as misaligned fetches and FPU emulation. TLB flushes, as Christoph mentioned.

For 2). part of the problem is that with the current Privileged ISA and SiFive’s CLINT, M-mode is necessarily involved and must stay resident to support timers and software interrupts. These are runtime services that depend on M-mode. Until we’ve fully virtualized interrupt configuration for Supervisor’s, we temporarily need to keep this interface, however as I mentioned there has been work on this. Question is whether it will go into silicon and formalised in the specs.

For 3). The kernel could adopt trap-and-emulate for misaligned loads and stores and FPU emulation like it does for other platforms and that removes the need for any stay resident firmware here. We (we as in probably someone other than me who has done a lot more work on RISC-V) have already devolved page table dirty handling to the kernel. For whoever did that, they got it for free as it’s part of Linux kernel. FPU emulation and misaligned loads and stores emulation could also move to the kernel(s). These do not need M modes. They just need illegal instruction trap: Also; the kernel should be compiled with -mstrict-align anyhow (it is?). Just because we can do something (slow misaligned accesses) doesn’t mean we should, but alas it’s in the spec.

UEFI does not provide runtime services nor does its scope cover virtualization, HALs or Secure Worlds (or I am mistaken). Of course measured boot and boot integrity services are later used by things like Windows Virtual Secure Mode to create a tier above the NT kernel that the kernel does not have access to (like arm’s Secure World) using VMX root (Extended Supervisor). RISC-V has Keystone here as well as several other solutions that folk are developing in this area that leverage PMP.

My take on this is that we should be focusing on standardising hardware interfaces or gaps in the Privileged ISA for the functions that are provided by SBI. It evolved because the hardware interfaces were still in flux. I believe they still are in flux (albeit now a relatively stable flux) if one does some analysis on colocating a Hypervisor with the current PLIC model one sees the gaps as we give M-mode interrupt configuration access to the Supervisor. While the Base ISA is frozen, the Privileged ISA is not and we can move forward in a step-wise manner keeping support for the “RISC-V legacy” until things have fully evolved, removing things with an adequate deprecation period. From my perspective, my only criticism of the Unprivileged ISA now is that FENCE.I should probably be moved to the Privileged ISA (or maybe it has). This is likely to be a harmless change due to the current User ABI (nop or trap is debatable). The Privileged ISA should be considered a still evolving target. I also think Andrew has been busy building cores with more ILP from reading SiFive announcements.

Trying to be as concise as possible while describing current issues that would require SBI interfaces either because they break an M-mode S-mode privilege boundary (like the initial SiFive PLIC revision) or alternatively require development on the hardware interfaces that SBI abstracts such that S-mode is fully autonomous. I really don’t think we should move PLIC behind SBI interfaces, instead the hardware should evolve such that an M-mode HAL is not required.

The latter being the direction I think most folk prefer from the comments on the list. i.e. the direction where folk have a standard machine, where standard boot can be layered on top.

SiFive have made some progress with their CLIC proposal whereby aspects from the model could be folded into the PLIC to give S-mode full autonomy for IPIs and real-time control over S-mode interrupts. Maybe SiFive’s new S-series can be configured with an S-mode CLIC and CLINT backwards compatibility and an updated PLIC memory layout separating M mode and S mode. I hope this made it in there...

Forgive me for speculating about SiFive hardware as I do not have all of the details, as there are a huge number of moving parts, of which just a few have been modelled in QEMU, but a fully autonomous S-mode is something that would be ideal. i.e. a situation where SBI is no longer required at runtime for Linux, and where real-time apps can coexist with the Supervisor with direct control over interrupts. There certainly is a model whereby this is feasible.

Michael

(Caveats regarding speculation with respect to details of current or future SiFive products ; this is my own speech, etc, etc)

Bin Meng

unread,
Nov 13, 2018, 4:27:20 AM11/13/18
to michae...@mac.com, rjo...@redhat.com, h...@lst.de, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Michael,

On Tue, Nov 13, 2018 at 4:19 PM 'Michael Clark' via RISC-V SW Dev
<sw-...@groups.riscv.org> wrote:
>
>
>
> >> On 12/11/2018, at 12:31 AM, Richard W.M. Jones <rjo...@redhat.com> wrote:
> >>
> >> On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
> >> Based on a recent software working group call, it seems there is a
> >> desire for the SBI to ensure that a single kernel works across all
> >> RISC-V implementations,
> >
> > Not sure what the "software working group call" is, but if this is in
> > any way related to what I said in my talk in Barcelona then I'd like
> > to clarify that I only meant for server platforms. Embedded folk can
> > and most likely will go their own way.
> >
> > BTW the way you described the full scope of the (potential) SBI made
> > it sound like UEFI.
>
> In some ways UEFI (or a subset of) is a little clearer in scope than SBI a). because after one calls ExitBootServices() one has complete control over the machine with respect to UEFI. SMI issues aside as they are independent to UEFI. b) the UEFI API is relatively well thought through although perhaps a little heavyweight e.g. the use of GUIDs to get structures with function pointers is very Microsoft centric. Lots of optional APIs too e.g. do we really need to boot from SCSI tape?
>
> I believe we need to separate these 3 concerns:
>
> 1). boot time services. arm’s EBBR has a minimal UEFI subset which IMHO is a good starting point (as an option)
>
> 2) runtime HAL e.g. interrupt controller virtualization, power management, etc i.e. platform device specification versus SBI driver? interfaces.
>
> 3). emulation and virtualization fixups or core support firmware e.g. trap-and-emulate in RISC-V code vs micro-code - features such as misaligned fetches and FPU emulation. TLB flushes, as Christoph mentioned.
>
> For 2). part of the problem is that with the current Privileged ISA and SiFive’s CLINT, M-mode is necessarily involved and must stay resident to support timers and software interrupts. These are runtime services that depend on M-mode. Until we’ve fully virtualized interrupt configuration for Supervisor’s, we temporarily need to keep this interface, however as I mentioned there has been work on this. Question is whether it will go into silicon and formalised in the specs.

Good to hear that at least someone is working on addressing the timer
and IPI M-mode headache.

>
> For 3). The kernel could adopt trap-and-emulate for misaligned loads and stores and FPU emulation like it does for other platforms and that removes the need for any stay resident firmware here. We (we as in probably someone other than me who has done a lot more work on RISC-V) have already devolved page table dirty handling to the kernel. For whoever did that, they got it for free as it’s part of Linux kernel. FPU emulation and misaligned loads and stores emulation could also move to the kernel(s). These do not need M modes. They just need illegal instruction trap: Also; the kernel should be compiled with -mstrict-align anyhow (it is?). Just because we can do something (slow misaligned accesses) doesn’t mean we should, but alas it’s in the spec.
>

I am all for moving trap-and-emulate stuff into S-mode. My opinion has
always been only keeping absolutely necessary functionalities in SBI,
yet what functionalities should be provided in SBI needs be fully
discussed and agreed on the public forum (I assume it's this mailing
list?).

> UEFI does not provide runtime services nor does its scope cover virtualization, HALs or Secure Worlds (or I am mistaken). Of course measured boot and boot integrity services are later used by things like Windows Virtual Secure Mode to create a tier above the NT kernel that the kernel does not have access to (like arm’s Secure World) using VMX root (Extended Supervisor). RISC-V has Keystone here as well as several other solutions that folk are developing in this area that leverage PMP.
>

UEFI indeed provides runtime services. However it's not mandatory for
kernel to call it. Instead it's completely kernel's call to decide
whether it should call into UEFI runtime services to accomplish some
specific tasks.

> My take on this is that we should be focusing on standardising hardware interfaces or gaps in the Privileged ISA for the functions that are provided by SBI. It evolved because the hardware interfaces were still in flux. I believe they still are in flux (albeit now a relatively stable flux) if one does some analysis on colocating a Hypervisor with the current PLIC model one sees the gaps as we give M-mode interrupt configuration access to the Supervisor. While the Base ISA is frozen, the Privileged ISA is not and we can move forward in a step-wise manner keeping support for the “RISC-V legacy” until things have fully evolved, removing things with an adequate deprecation period. From my perspective, my only criticism of the Unprivileged ISA now is that FENCE.I should probably be moved to the Privileged ISA (or maybe it has). This is likely to be a harmless change due to the current User ABI (nop or trap is debatable). The Privileged ISA should be considered a still evolving target. I also think Andrew has been busy building cores with more ILP from reading SiFive announcements.
>
> Trying to be as concise as possible while describing current issues that would require SBI interfaces either because they break an M-mode S-mode privilege boundary (like the initial SiFive PLIC revision) or alternatively require development on the hardware interfaces that SBI abstracts such that S-mode is fully autonomous. I really don’t think we should move PLIC behind SBI interfaces, instead the hardware should evolve such that an M-mode HAL is not required.
>
> The latter being the direction I think most folk prefer from the comments on the list. i.e. the direction where folk have a standard machine, where standard boot can be layered on top.
>
> SiFive have made some progress with their CLIC proposal whereby aspects from the model could be folded into the PLIC to give S-mode full autonomy for IPIs and real-time control over S-mode interrupts. Maybe SiFive’s new S-series can be configured with an S-mode CLIC and CLINT backwards compatibility and an updated PLIC memory layout separating M mode and S mode. I hope this made it in there...
>

I will need to take a look at the new CLIC model some time. Thanks for
sharing such information.

> Forgive me for speculating about SiFive hardware as I do not have all of the details, as there are a huge number of moving parts, of which just a few have been modelled in QEMU, but a fully autonomous S-mode is something that would be ideal. i.e. a situation where SBI is no longer required at runtime for Linux, and where real-time apps can coexist with the Supervisor with direct control over interrupts. There certainly is a model whereby this is feasible.
>
> Michael
>
> (Caveats regarding speculation with respect to details of current or future SiFive products ; this is my own speech, etc, etc)

Thank you. This is really helpful. I am now much reassured after
seeing such even labeled as *unofficial* statement for SiFive's next
move. :)

Regards,
Bin

Bin Meng

unread,
Nov 13, 2018, 5:01:49 AM11/13/18
to bruce...@sifive.com, Samuel Falvo II, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Bruce,

On Tue, Nov 13, 2018 at 2:11 PM Bruce Hoult <bruce...@sifive.com> wrote:
>
> On Mon, Nov 12, 2018 at 9:12 PM, Bin Meng <bmen...@gmail.com> wrote:
>>
>> Hi Bruce,
>>
>> On Tue, Nov 13, 2018 at 12:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
>> >
>> > Of course.
>> >
>> > Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:
>> >
>> > lui t0,#ecallTableBase #or auipc
>>
>> You need save t0 before using it, no?
>
>
> Not if, as I specified below, SBI ecalls have the same/similar ABI as they would if they were in a C library.
>

Ah, I see. So that means the SBI handlers are allowed to only use
t0-t6 and a0-a7, right?

>>
>>
>> > slli t1,a6,2 #3 for 64 bit
>>
>> and t1, a6?
>
>
> *Certainly* not a6, as that has the index of the desired SBI call, and we don't modify it.

Typo, I think it should be a7.

>
>>
>>
>> > add t0,t0,t1
>> > jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
>> > # called function does the mret
>> >
>> > That's five instructions.
>> >
>>
>> Yes, for this particular part. But the called function (illegal
>> instruction handler in this case) to implement reading the mtimer
>> counter needs another bunch of instructions, including instruction
>> decoding, etc.
>
>
> I'm assuming here that you're making an explicit SBI call to get mtime, not handling an illegal instruction trap. Illegal instruction would definitely be a bit more heavy-weight with both decoding overheads and the necessity to save and restore everything it touches.

The issue here is that rdtime will trigger an illegel instruction
exception from user space, meaning M-mode SBI handler will need decode
and emulate the CSR instruction first before accessing to the real
mtime register, at least this is what QEMU virt does so far. I am not
sure if the virt model reflects the SiFive U540 hardware or not ...

>
>> > You could do something like a "li t0,#nnn; blt a6,t0,heavyweight" to distinguish these simple calls from ones that actually need to save and restore registers. That makes it seven instructions.
>>
>> Unless we write all ecall trap handlers in assembly codes, it's still
>> needed to do the blind register saving/restoring.
>
>
> I would definitely write the smallest and simplest ones, that should be as fast as possible, in assembly language. Like getting timers, for example. And I'd make sure a 64 bit counter was returned on 32 bit systems (already made atomic, and possibly virtualised with an offset subtracted as well) so that only one SBI call was needed, not two.
>

Regards,
Bin

Bruce Hoult

unread,
Nov 13, 2018, 5:15:59 AM11/13/18
to Bin Meng, Samuel Falvo II, mi...@ics.forth.gr, Alex Bradbury, RISC-V SW Dev
On Tue, Nov 13, 2018 at 2:01 AM, Bin Meng <bmen...@gmail.com> wrote:
Hi Bruce,

On Tue, Nov 13, 2018 at 2:11 PM Bruce Hoult <bruce...@sifive.com> wrote:
>
> On Mon, Nov 12, 2018 at 9:12 PM, Bin Meng <bmen...@gmail.com> wrote:
>>
>> Hi Bruce,
>>
>> On Tue, Nov 13, 2018 at 12:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
>> >
>> > Of course.
>> >
>> > Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:
>> >
>> > lui t0,#ecallTableBase #or auipc
>>
>> You need save t0 before using it, no?
>
>
> Not if, as I specified below, SBI ecalls have the same/similar ABI as they would if they were in a C library.
>

Ah, I see. So that means the SBI handlers are allowed to only use
t0-t6 and a0-a7, right?

Under this proposal, without saving them, yes. Or of course any other register if it is saved and restored.
 

>>
>>
>> > slli t1,a6,2 #3 for 64 bit
>>
>> and t1, a6?
>
>
> *Certainly* not a6, as that has the index of the desired SBI call, and we don't modify it.

Typo, I think it should be a7.

It's only an example. This is SBI between S mode and M mode, not the Linux BI between U mode and S mode. It doesn't have to be the same. In particular, I am proposing that calls do not have to preserve all registers.

 

>
>>
>>
>> > add t0,t0,t1
>> > jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
>> > # called function does the mret
>> >
>> > That's five instructions.
>> >
>>
>> Yes, for this particular part. But the called function (illegal
>> instruction handler in this case) to implement reading the mtimer
>> counter needs another bunch of instructions, including instruction
>> decoding, etc.
>
>
> I'm assuming here that you're making an explicit SBI call to get mtime, not handling an illegal instruction trap. Illegal instruction would definitely be a bit more heavy-weight with both decoding overheads and the necessity to save and restore everything it touches.

The issue here is that rdtime will trigger an illegel instruction
exception from user space, meaning M-mode SBI handler will need decode
and emulate the CSR instruction first before accessing to the real
mtime register, at least this is what QEMU virt does so far. I am not
sure if the virt model reflects the SiFive U540 hardware or not ...

Illegal/unimplemented instructions from U mode are a COMPLETELY different thing to SBI, which is a call-like Binary Interface used by the Supervisor.

Bin Meng

unread,
Nov 13, 2018, 7:20:10 AM11/13/18
to bruce...@sifive.com, Samuel Falvo II, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Bruce,

On Tue, Nov 13, 2018 at 6:15 PM Bruce Hoult <bruce...@sifive.com> wrote:
>
> On Tue, Nov 13, 2018 at 2:01 AM, Bin Meng <bmen...@gmail.com> wrote:
>>
>> Hi Bruce,
>>
>> On Tue, Nov 13, 2018 at 2:11 PM Bruce Hoult <bruce...@sifive.com> wrote:
>> >
>> > On Mon, Nov 12, 2018 at 9:12 PM, Bin Meng <bmen...@gmail.com> wrote:
>> >>
>> >> Hi Bruce,
>> >>
>> >> On Tue, Nov 13, 2018 at 12:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
>> >> >
>> >> > Of course.
>> >> >
>> >> > Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:
>> >> >
>> >> > lui t0,#ecallTableBase #or auipc
>> >>
>> >> You need save t0 before using it, no?
>> >
>> >
>> > Not if, as I specified below, SBI ecalls have the same/similar ABI as they would if they were in a C library.
>> >
>>
>> Ah, I see. So that means the SBI handlers are allowed to only use
>> t0-t6 and a0-a7, right?
>
>
> Under this proposal, without saving them, yes. Or of course any other register if it is saved and restored.
>
>
>> >>
>> >>
>> >> > slli t1,a6,2 #3 for 64 bit
>> >>
>> >> and t1, a6?
>> >
>> >
>> > *Certainly* not a6, as that has the index of the desired SBI call, and we don't modify it.
>>
>> Typo, I think it should be a7.
>
>
> It's only an example. This is SBI between S mode and M mode, not the Linux BI between U mode and S mode. It doesn't have to be the same. In particular, I am proposing that calls do not have to preserve all registers.
>

OK, that doesn't matter. I think this proposal makes sense. So where
is your proposal posted?

>
>>
>>
>> >
>> >>
>> >>
>> >> > add t0,t0,t1
>> >> > jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
>> >> > # called function does the mret
>> >> >
>> >> > That's five instructions.
>> >> >
>> >>
>> >> Yes, for this particular part. But the called function (illegal
>> >> instruction handler in this case) to implement reading the mtimer
>> >> counter needs another bunch of instructions, including instruction
>> >> decoding, etc.
>> >
>> >
>> > I'm assuming here that you're making an explicit SBI call to get mtime, not handling an illegal instruction trap. Illegal instruction would definitely be a bit more heavy-weight with both decoding overheads and the necessity to save and restore everything it touches.
>>
>> The issue here is that rdtime will trigger an illegel instruction
>> exception from user space, meaning M-mode SBI handler will need decode
>> and emulate the CSR instruction first before accessing to the real
>> mtime register, at least this is what QEMU virt does so far. I am not
>> sure if the virt model reflects the SiFive U540 hardware or not ...
>
>
> Illegal/unimplemented instructions from U mode are a COMPLETELY different thing to SBI, which is a call-like Binary Interface used by the Supervisor.
>

Yes I know. I was pointing out the truth that it could have been just
one instruction to get a timer counter value in an S-mode OS, but now
it has to execute way more instructions than that.

Regards,
Bin

Bruce Hoult

unread,
Nov 13, 2018, 7:34:38 AM11/13/18
to Bin Meng, Samuel Falvo II, mi...@ics.forth.gr, Alex Bradbury, RISC-V SW Dev
On Tue, Nov 13, 2018 at 4:19 AM, Bin Meng <bmen...@gmail.com> wrote:
> It's only an example. This is SBI between S mode and M mode, not the Linux BI between U mode and S mode. It doesn't have to be the same. In particular, I am proposing that calls do not have to preserve all registers.
>

OK, that doesn't matter. I think this proposal makes sense. So where
is your proposal posted?

In this email thread.

> Illegal/unimplemented instructions from U mode are a COMPLETELY different thing to SBI, which is a call-like Binary Interface used by the Supervisor.
>

Yes I know. I was pointing out the truth that it could have been just
one instruction to get a timer counter value in an S-mode OS, but now
it has to execute way more instructions than that.

Less than 10 instructions in 64 bit mode. (a 32 bit OS should do the atomic loop dance to return a 64 bit value to avoid two calls)

If on S mode OS can use one instruction to get timer values then you can't virtualise the OS. You *have* to do one of trap-and-emulate or explicitly call an SBI function. The latter is lower overhead.

ron minnich

unread,
Nov 13, 2018, 10:13:33 AM11/13/18
to bruce...@sifive.com, Samuel Falvo II, Bin Meng, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Bruce, this is very helpful for coreboot.

I wonder if you might add to the github issue with the code?

short story: coreboot implemented a correct but very inefficient SBI
for the faster calls. This will help us make it better.

ron
On Mon, Nov 12, 2018 at 10:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
>
> Of course.
>
> Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:
>
> lui t0,#ecallTableBase #or auipc
> slli t1,a6,2 #3 for 64 bit
> add t0,t0,t1
> jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
> # called function does the mret
>
> That's five instructions.
>
> You could do something like a "li t0,#nnn; blt a6,t0,heavyweight" to distinguish these simple calls from ones that actually need to save and restore registers. That makes it seven instructions.
>
> I'd submit that trusted SBI calls should use the same ABI as regular function calls (i.e. a0-a7 and t0-t6 can come back trashed) rather than the Linux "all registers except designated return values come back unchanged" system call ABI.
>
>
> On Mon, Nov 12, 2018 at 7:46 PM, Samuel Falvo II <sam....@gmail.com> wrote:
>>
>> Including ecall function ID dispatch logic?
>>
>>
>>
>> On Mon, Nov 12, 2018, 7:34 PM Bruce Hoult <bruce...@sifive.com wrote:
>>>
>>> On Mon, Nov 12, 2018 at 5:21 AM, Bin Meng <bmen...@gmail.com> wrote:
>>>>
>>>> Yes, it was unfortunately designed this way. Sigh. My question was why
>>>> does hardware expose such limitation that accessing timer/IPI should
>>>> only be done in M-mode? It could have only be one instruction of
>>>> CSR/MMIO read to get a timer counter, but now it has to call into SBI
>>>> and execute maybe one hundred instructions. This affects real-time
>>>> performance a lot.
>>>
>>>
>>> If it's 100 instructions then someone is doing something very very wrong. Try 10.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> --
> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> To post to this group, send email to sw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAP8PnuSgTJkD0mYr-%2B%3D9%3DkDWLSDQ3_RfVuUryZT9ywtegYSPMQ%40mail.gmail.com.

Richard W.M. Jones

unread,
Nov 13, 2018, 2:32:40 PM11/13/18
to Michael Clark, h...@lst.de, Alex Bradbury, sw-...@groups.riscv.org
I asked Laszlo Ersek - who works for Red Hat upstream on edk2 - about
this topic and he had two small clarifications which I quote below.

Rich.

> In some ways UEFI (or a subset of) is a little clearer in scope than
> SBI a). because after one calls ExitBootServices() one has complete
> control over the machine with respect to UEFI. SMI issues aside as
> they are independent to UEFI.

SMI handling (on x86) is very much implemented / set up as part of the
UEFI platform firmware.

> UEFI does not provide runtime services

It does; see section "8 Services -- Runtime Services" in the UEFI 2.7
spec.




--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

ron minnich

unread,
Nov 13, 2018, 3:03:12 PM11/13/18
to Richard W.M. Jones, Michael Clark, Christoph Hellwig, a...@lowrisc.org, sw-...@groups.riscv.org
On Tue, Nov 13, 2018 at 1:32 PM Richard W.M. Jones <rjo...@redhat.com> wrote:
>
> I asked Laszlo Ersek - who works for Red Hat upstream on edk2 - about
> this topic and he had two small clarifications which I quote below.
>
> Rich.
>
> > In some ways UEFI (or a subset of) is a little clearer in scope than
> > SBI a). because after one calls ExitBootServices() one has complete
> > control over the machine with respect to UEFI. SMI issues aside as
> > they are independent to UEFI.
>
> SMI handling (on x86) is very much implemented / set up as part of the
> UEFI platform firmware.
>
> > UEFI does not provide runtime services
>
> It does; see section "8 Services -- Runtime Services" in the UEFI 2.7
> spec.

yeah. That's why it's called ExitBootServices(), not TurnOFFUefi(). :-)

And that's the problem for people who care about security. The
LinuxBoot project is working on fixing the glitch. :-)

ron

kr...@berkeley.edu

unread,
Nov 13, 2018, 3:36:06 PM11/13/18
to Luke Kenneth Casson Leighton, Alex Bradbury, sw-...@groups.riscv.org

>>>>> On Sun, 11 Nov 2018 12:56:34 +0000, Luke Kenneth Casson Leighton <lk...@lkcl.net> said:
| On Sun, Nov 11, 2018 at 10:05 AM Alex Bradbury <a...@lowrisc.org> wrote:
|| Hi Luke,
||
|| I've changed the email subject to split this out to a separate thread.
|| I totally accept that your concerns about process are intertwined with
|| the technical discussion, but I'd much rather keep that separate to
|| the thread I started (more on that below). I'm not telling you what to
|| do, just asking.

Alex - thanks for starting separate thread,

|| I don't think that the closed working groups used by the RISC-V
|| Foundation are the right model. This is an opinion I've expressed
|| repeatedly, going back to before the formation of the Foundation.
|| Others obviously feel differently.

...and let me try and remind everyone why things are the way they are
below.

| when i spoke to krste in barcelona about this, he explained that
| there's people who just simply would not come forward - at all - if
| the discussions were public.

This part is correct.

| mostly, they're people who have worked
| in U.S. Classified environments, and their knowledge and expertise
| is too valuable to ignore.

This part is not.

The important people are industry engineering teams who've built
commercial processors, and commercial companies that will provide
actual RISC-V hardware in volume at low cost. My limited experience
with teams operating in US classified environments is that they
generally don't have advanced processor design experience, and are
very happy that there is an open alternative now available to them.

The Foundation standards setting structure arose to address the patent
issues that show up in hardware industry. RISC-V is not very useful
without hardware implementations, hardware implementations require
money, and that money is a target for patent lawsuits.

All of the members have agreed not to sue based on the original
standard ISA specifications. They've also agreed to donate any
necessary IP behind any contribution they make to a standard to
prevent a standards patent ambush. Everyone realizes there are large
categories of patent attack that this does not mitigate (proprietary
licensed IP has many of the same risks, and indemnity clauses provide
only minor relief) but it is a step that was required to bring large
companies into the Foundation.

The more companies that are inside the tent agreeing to work together,
the faster RISC-V advances and the lower the probabilty of a crippling
lawsuit. You may notice we've been very successful at bringing most
of the major hardware makers into the Foundation. We can't go back
and do a control experiment where we adopted a completely open model
and saw how much progress we'd have made, but I seriously doubt RISC-V
would have gathered anywhere close to current momentum. The
Foundation "constitution" says that the specs, golden model, and
compatibility suites are always publicly available, so everyone gets
to leverage the work product.

The "closed" process is open to anyone who agrees to these conditions
when working on hardware standards, which are those that affect
hardware implementations. We have managed to get over 100 companies
to sign agreement, representing many of the leading industry processor
design teams. This is the valuable experience we want to include in
the standards process, and would not be available with a completely
open model. There is also the non-trivial observation that without
large investments, there will be no advanced RISC-V hardware.

Where the standard under discussion is a software convention on top of
the existing hardware standards, there is strictly no need for this to
be handled in the Foundation closed standards procedure (or even by
Foundation members). Obviously, it will often be helpful to have
Foundation members involved to help guide software developers, and
ideally Foundation would be able to bless a software standard,
particulary at lower levels of the stack.

For example, in this particular case, the SBI discussion is
sufficiently reliant on low-level hardware design that it makes sense
as part of a Foundation task group, but as no hardware mechanisms are
being standardized, there's no reason not to have an open working
group list.

Krste

Alex Bradbury

unread,
Nov 17, 2018, 2:46:40 AM11/17/18
to Krste Asanovic, Luke Kenneth Casson Leighton, sw-...@groups.riscv.org
On Tue, 13 Nov 2018 at 20:36, <kr...@berkeley.edu> wrote:
> The Foundation standards setting structure arose to address the patent
> issues that show up in hardware industry. RISC-V is not very useful
> without hardware implementations, hardware implementations require
> money, and that money is a target for patent lawsuits.
>
> All of the members have agreed not to sue based on the original
> standard ISA specifications. They've also agreed to donate any
> necessary IP behind any contribution they make to a standard to
> prevent a standards patent ambush. Everyone realizes there are large
> categories of patent attack that this does not mitigate (proprietary
> licensed IP has many of the same risks, and indemnity clauses provide
> only minor relief) but it is a step that was required to bring large
> companies into the Foundation.

There are multiple issues under discussion here:
1) Agreeing not to sue based on the standard ISA specs
2) CLA/patent policy on contributions
3) Membership of the RISC-V Foundation
4) Whether standards discussions are open to non-Foundation members

I agree that patents are an important issue and that it is prudent to
have an agreement on specification contributions that covers patents.
This doesn't necessarily need to be conflated with RISC-V Foundation
membership, and doesn't necessarily imply that discussions must take
place behind closed doors.

In terms of such agreements, there's prior work in at least the
agreements created by the Open Web Foundation intended to support the
development of open standards
<http://www.openwebfoundation.org/legal/the-owf-1-0-agreements>. Last
year the WHATWG sought to add clarity on IPR to its standards
development process whilst maintaining its public and inclusive
development model <https://blog.whatwg.org/working-mode-changes>.

Best,

Alex

Luke Kenneth Casson Leighton

unread,
Nov 17, 2018, 7:20:40 AM11/17/18
to Alex Bradbury, Krste Asanovic, sw-...@groups.riscv.org
On Sat, Nov 17, 2018 at 7:46 AM Alex Bradbury <a...@lowrisc.org> wrote:

> On Tue, 13 Nov 2018 at 20:36, <kr...@berkeley.edu> wrote:
> > The Foundation standards setting structure arose to address the patent
> > issues that show up in hardware industry. RISC-V is not very useful
> > without hardware implementations, hardware implementations require
> > money, and that money is a target for patent lawsuits.
> >
> > All of the members have agreed not to sue based on the original
> > standard ISA specifications. They've also agreed to donate any
> > necessary IP behind any contribution they make to a standard to
> > prevent a standards patent ambush. Everyone realizes there are large
> > categories of patent attack that this does not mitigate (proprietary
> > licensed IP has many of the same risks, and indemnity clauses provide
> > only minor relief) but it is a step that was required to bring large
> > companies into the Foundation.
>
> There are multiple issues under discussion here:
> 1) Agreeing not to sue based on the standard ISA specs
> 2) CLA/patent policy on contributions
> 3) Membership of the RISC-V Foundation
> 4) Whether standards discussions are open to non-Foundation members
>
> I agree that patents are an important issue and that it is prudent to
> have an agreement on specification contributions that covers patents.
> This doesn't necessarily need to be conflated with RISC-V Foundation
> membership, and doesn't necessarily imply that discussions must take
> place behind closed doors.

indeed.

> In terms of such agreements, there's prior work in at least the
> agreements created by the Open Web Foundation intended to support the
> development of open standards
> <http://www.openwebfoundation.org/legal/the-owf-1-0-agreements>. Last
> year the WHATWG sought to add clarity on IPR to its standards
> development process whilst maintaining its public and inclusive
> development model <https://blog.whatwg.org/working-mode-changes>.

there are also at least two or three other means and methods by which
protection from patents may be achieved:

(1) patent pool agreements. IBM kicked this off over a decade ago by
dropping literally thousands of patents into a pool, regarding the
linux kernel. deeply impressive. now i believe that initiative has
morphed into the "Linux Foundation", and even Microsoft has
contributed things like that really really annoying DOS fileformat
patent.

as Alex mentions: these patent pool agreements have *nothing* to do
with whether discussion is public or private. and, they're much more
effective than "agreements not to sue", as there is both incentive for
members to contribute *to* the pool *and* incentive for the members to
keep to the agreement.


(2) the BBC's process which was first brought to light in the
development of the Dirac Video CODEC.

https://en.wikipedia.org/wiki/Dirac_(video_compression_format)

what is *not* mentioned in the wikipedia page is two things.
ffirstly: the BBC has by far and above the largest digital archive of
video footage in the world: anyone may apply for access to it, it's
called "BBC Backstage". this next bit is speculation: i imagine that
if they used patented proprietary formats, they would be an instant
patent-troll-magnet. hence a huge financial incentive to come up with
an alternative (non-patented, non-patentable alternative)

second: the development of Dirac was done by researching *OLD AND
EXPIRED PATENTS* and deliberately stitching together an algorithm
based on work that they *knew* and could *PROVE* could never be
patented [again].

what *is* mentioned in the page is that the Dirac algorithm was
patented... *AND THEN ALLOWED TO LAPSE*. this is extremely
significant, as not only would a patent troll not be able to get a
slice of the action, the BBC would not even need a lawyer to send a
standard form letter back to the troll "here's the publicly-listed
registered prior art, goodbye".

thus, overall, they have deployed a strategy that extremely neatly
and efficiently saves massive, massive amounts of money.


(3) the strategy that i have systematically deployed, which jacob
bachmeyer mentioned about six months ago: *keep the discussion
completely public*.

you can see from the post-script paragraphs of the last three messages
that i've sent to isa-dev, i slowly outlined and emphasised that
strategy in incrementally increasing detail. namely: the timestamp of
delivery of the messages through public forums constitutes publication
of prior art.

patent lawyers always, always advise their clients to never, under any
circumstances, disclose *anything* about the invention to a third
party, and, if it is absolutely necessary to do so, to get them to
sign an NDA. the reasons are twofold:

(a) the third party may publish the material as "prior art", thus
terminating the possibility of a successful patent application [or, if
one succeeds, it may be challenged in court]

(b) the third party could potentially claim that they are one of the
co-inventors.

note that, hilariously, **BOTH THESE CHARACTERISTICS ARE ACTUALLY
DESIRABLE OUTCOMES FOR RISC-V FOUNDATION MEMBERS**!

they're *exactly* and *precisely* what the RISC-V Member Agreement wants!


by keeping discussions private and secret, actually the members are
placing themselves at risk *and* increasing costs. firstly, by having
to use patents to "protect" themselves (patent write-up costs and
patent law firm costs), secondly, by having to enter into multi-party
agreements with the RISC-V Foundation and members (legal firm
administrative costs).

and, more than that, there is the possibility of one of the Foundation
Members deciding, "oh, actually, we don't agree to be a member any
more", and surpriiise! one year later, a submarine patent application
pops up with a filing date the day after their membership lapses.


by keeping the discussions *public*, there *is* no possibility of one
of the members breaking their agreement; no member need pay any legal
costs to file patents, there is no need for multi-way multi-party
legal agreement...

... oh, and people from outside of the Foundation may freely and
*actively* contribute to the development of RISC-V, and thus may feel
like they are part of the *future* of RISC-V.

remember: no debian developer and no software libre developer will
ever, ever join the RISC-V Foundation as it stands. the future
success and stability of RISC-V critically depends on people who are
completely excluded from participating in the *development* of RISC-V.

there's something wrong with that picture.

l.

Renba

unread,
Nov 17, 2018, 10:02:48 AM11/17/18
to atish patra, a...@lowrisc.org, sw-...@groups.riscv.org

From UEFI perspective, I believe SBI would not become part of UEFI spec. But for the RISC-V ekd2 package, we would like to follow SBI spec and implement RISC-V SBI edk2 port.

 

From: atish patra [mailto:atis...@gmail.com]
Sent: Monday, November 12, 2018 2:31 PM
To: a...@lowrisc.org
Cc: sw-...@groups.riscv.org
Subject: Re: [sw-dev] SBI - taking a step back

 

Hi Alex,
Thanks for starting this topic. I believe you raised a very valid point.
IMHO, SBI should not become UEFI or a "mini" OS. I view SBI should provide
an interface for S mode that helps in maintaining a clear separation between privileged
mode & machine mode. It should include features that are absolutely necessary and would

need a hack to implement in supervisor mode. This is just my opinion. 

 

 

As RISC-V is modern architecture and we all envision it to be used in various types of computing

devices, the idea with the proposal is to introduce minimal functions and keep them optional.

For example, if any embedded/iot devices doesn't need to use hart power management, they

can skip it and same supervisor OS will continue to work.

 

However, the primary goal is to have a proper function ID scheme & introduce version/feature check

capabilities at this point. That will help in adding any feature (i.e. absolutely necessary) in future.
Currently, it is not possible at all.

I am not sure if it will be helpful. But I can maintain a section of list of new feature suggestions in the
proposal. We can take a call if that feature is going to be added to SBI or not after it has been discussed

enough in public forums. But we need a continuous feedback/discussion on this so that we can finalize

a version of SBI at least.

On Sat, Nov 10, 2018 at 1:21 PM Alex Bradbury <a...@lowrisc.org> wrote:

I've been following the recent discussions on SBI extensions with
great interest. Firstly, thank you to everyone who has been
contributing SBI proposals and documenting the current de facto SBI
standard.

I do have one request - could we all take a step back and try to
define the role and driving principles of the SBI? What problems
should it be solving? What problems shouldn't be solved via the SBI? I
understand once the framework is in place there's scope for people to
use it in a wide variety of ways, but it's really hard to judge the
various proposals without any clear definition of the SBI
requirements. Proposing incremental ABI additions seems like the wrong
way to go about development without agreeing this first.

The nearest thing we have to the guiding principles of the SBI is the
stated aim in the current privspec draft "Using a single SBI across
all SEE implementations allows a single OS binary image to run on any
SEE".  The document currently proposes SBI calls for:
* Initialising, saving, and restoring user-mode state outside of the
standard FDQ extensions
* Inter-processor interrupts
* Clear timer interrupts
* Mask, unmask, and query the cause of external interrupts
(apparently, all access to the PLIC should be through the SBI?)
* Modifying the time/cycle/instret CSRs
* Scheduling timer interrupts
* TLB shootdown

This is quite an expansion over the current SBI (which seems to offer
a rather arbitrary subset of platform-specific functionality). I would
observe that none of the proposals to extend the SBI so far actually
cover all of these cases. Is that because there is disagreement over
whether the SBI should serve all these purposes, or are the proposals
simply incomplete?

Based on a recent software working group call, it seems there is a
desire for the SBI to ensure that a single kernel works across all

RISC-V implementations, including support for custom extensions (e.g.
save/restore of user-visible state as mentioned above). I'm concerned
that reaching this goal will require an increasing number of hooks
into the kernel effectively making the SBI a mini-OS. e.g. for the
tagged memory mechanism in lowRISC, saving/restoring the tagctrl
register is a necessary component of kernel support but it's also
necessary to modify the virtual memory system to support (at a
minimum) clearing tags on pages and propagating tags upon
copy-on-write.

Other questions/concerns/thoughts:
* Is there actually consensus amongst developers of S-mode code
(kernels, bootloaders etc) that the SBI is a desirable abstraction?

* Shouldn't there be an interface to allow S-mode code to control
delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
supply its own floating point emulation code. Or maybe an alternative
mechanism to swap out the M-mode handler code (as Ron has been
advocating)

* We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
intended to work in the case where a hart supports switching between
these? Surely an S-mode OS may be running with a different ISA variant
to the M-mode SBI/SEE? How is this handled?

* There are ideas floating around about SBI calls being more like
function calls (rather than the current ecall mechanism). Has anyone
written down how this would work?

How about we start by agreeing and enumerating the requirements, then
work our way up?

Thanks,

Alex



--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Chang, Abner (HPS SW/FW Technologist)

unread,
Nov 17, 2018, 10:16:34 AM11/17/18
to atish patra, a...@lowrisc.org, sw-...@groups.riscv.org

Luke Kenneth Casson Leighton

unread,
Nov 17, 2018, 10:20:50 AM11/17/18
to Samuel Falvo II, Bin Meng, Alex Bradbury, sw-...@groups.riscv.org
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Sun, Nov 11, 2018 at 6:31 PM Samuel Falvo II <sam....@gmail.com> wrote:

> > is absolutely necessary? Eventually we will move all platform-specific
> > stuff into the SBI implementation and that makes SBI another mini-OS
>
> That is the crux right here, I think -- there are three groups of
> people vying for influence on how system firmware should be handled:
> (1) the pro-UEFI(-like) folks who thinks that system firmware should
> include the kitchen sink, (2) the pro-BIOS folks (like myself) which
> thinks it should be a thin abstraction over the available hardware,
> and then (3) there's the anti-firmware folks who thinks that the
> firmware should be opaque to any operating system you load, with
> complete and total hand-off to the kernel once the bootloader has
> completed its job.

and (4) the virtualisation people. the thing is: if virtualisation
is made a mandatory part of SBI (v2, v3, vN) the size of the SBI gets
rapidly out of hand, as it becomes basically a micro-kernel similar to
L4KA, XEN Hypervisor and so on.

with very few technical exceptions (hybrid designs with non-uniform
cores such as RF base-band DSPs with shared memory to the main cores,
in order to save on cost) an embedded system absolutely does *not*
want a full-on *mandatory* hypervisor. in fact, even a powerful
desktop doesn't want one, either.

> Put bluntly, I'd much rather have a kernel which is slower but runs on
> wider hardware than hyper-optimized kernels that can only run on one
> motherboard's processor and/or chipset. Computers are fast enough for
> me to not have to care about break-neck performance anymore. The only
> exception I'd make for that is in a data center context, where now I'm
> managing entire fleets of like-configured systems.

...and scenarios where real-time latency is an issue. this can be
professional real-time Studio-Grade desktop Audio/Video processing
just as much as it can be mission-critical systems.

> Linux has a HAL. Windows uses a HAL. Most modern kernels rely upon a
> HAL-like "thing" at some level. That's an SBI but by a different
> name.

the issue here is: there is no precedent that i am aware of where the
linux kernel has external library dependencies. the UNISW team have
quite literally been maintaining and porting L4KA patches to the linux
kernel for well over a decade, because Linus Torvalds rejects
micro-kernels. OSKit was abandoned over 15 years ago.

fortunately, SBI v1 was initially small enough to slip "under the
radar", however i can foresee efforts to turn it into a full-blown
virtualisation hypervisor microkernel being met with considerable
resistance from Linus Torvalds.

l.

Michael Clark

unread,
Nov 17, 2018, 5:37:12 PM11/17/18
to Bin Meng, bruce...@sifive.com, Samuel Falvo II, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Yes. This emulation function is distinct from SBI. However they are related because they both use “M-mode”, which is a distinguishing factor with respect to whether an SBI interface needs to exist.

Also of note, trap-and-emulate platform requirement can be fulfilled by the kernel as is the case with page table dirty bit handling, FPU and misaligned load store handling. Linux kernel has misaligned load store handling and FPU emulation for several architectures. There is no necessity for M-mode permissions and the kernel can be compiled with -mstrict-align which just makes good sense anyway even if misaligned accesses are just slow, not extremely slow.

This indeed is a function could logically be exposed by SBI given it is in this case a “runtime” vs “boot time” function. e.g. delegation for emulation is controlled by M-mode so to actually expose a “monitor/supervisor control surface” for this function then one would add something like:

enum {
sbi_emul_fpu = 1,
sbi_emul_misaligned = 2,
sbi_emul_rdtime = 4,
sbi_emul_rdhartid = 8,
sbi_emul_rdisa = 16
};

int sbi_query_emulation();
int sbi_set_emulation(int emul_flags);

So they are linked to SBI in the sense that they are M-mode functions, that currently lack control surfaces. Note: not all of these emulation functions could be done by the Supervisor e.g. a U-mode version of RDISA (rdisa/uisa is an existing encoding using the current CSR encoding scheme that contains mode bits).

A supervisor along with the SBI version and a feature discovery method (ecall number, not function pointer) could provide these interfaces to the kernel so that the function could be optionally moved to the kernel in a 100% backwards compatible manner.

It makes sense, even when we have hardware that exposes timers and IPIs to S-mode, to retain the SBI Platform Timer and SBI Platform Software Interrupts drivers in Linux and other kernels to handle simpler hardware that choose to multiplex this function onto M-mode accessible hardware function. This is “maintain current SBI” even if ... i.e. 100% backwards compatibility or migration plans with long deprecation periods so everything continues to interoperate during transitions between different versions of the Privileged ISA and future platform specs.

The question for new SBI functions is “why is this necessary?”. If there is a function that requires M-mode on specific hardware due to the function being exposed via m* registers, then there is a rationale. If there is simply no device driver, then I don’t think this is a good rationale. I guess it lets one mock a function in simulation. They may be a factor that justifies an addition, if it is essential and the simulation platform would not have a hardware interface that is suitable to expose to the kernel, as is the case with HTIF.

Adding and removing a hart seems justifiable as it’s a Hypervisor-type function (the original proposal) that may not be usually or always exposable as a hardware interface. That said, an asynchronous message interface via an MMIO aperture is an alternative to an “SBI hypercall”. i.e. VirtIO platform-cpu.

Issues around the monitor are in scope for SBI even if there is not a current control surface for one of the Hypervisor or Monitor interfaces. We don’t have an HBI, and if we did, cpu hot add and memory ballooning would be in it, thus would be “emulation control” for the Hypervisor to query and configure the platform based on what it wants. e.g. all illegal instructions.

i.e. SBI ~= HBI

Michael Clark

unread,
Nov 17, 2018, 5:52:03 PM11/17/18
to Bin Meng, bruce...@sifive.com, Samuel Falvo II, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Indeed SBI is the only control surface between the Supervisor and Monitor, so if moving illegal instruction emulation (medeleg requires M-mode) to the Supervisor would require an SBI interface to do so smoothly (besides hard-coding and a flag day). The interface to emulation “control”, a monitor function, would require an interface between the Monitor and Supervisor. Of course the interface to the monitor is different from the function of the monitor. The question is more which functions should be there and what their exposed control surface to the Supervisor should be. So in that sense they are different but intertwined.

Back to the original but related issue - An SBI version interface and call number query interface is IMHO a good idea. Even if we plan to migrate function to Supervisors, we need to maintain backwards compatibility. We also need to decide the “limit” of what is appropriate. Emulation is appropriate as mstatus.MPRV (an emulation feature) is only available to M mode. So emulation control is IMHO in scope for SBI.

Michael Clark

unread,
Nov 17, 2018, 8:36:18 PM11/17/18
to Richard W.M. Jones, h...@lst.de, Alex Bradbury, sw-...@groups.riscv.org


> On 14/11/2018, at 8:32 AM, Richard W.M. Jones <rjo...@redhat.com> wrote:
>
> I asked Laszlo Ersek - who works for Red Hat upstream on edk2 - about
> this topic and he had two small clarifications which I quote below.
>
> Rich.
>
>> In some ways UEFI (or a subset of) is a little clearer in scope than
>> SBI a). because after one calls ExitBootServices() one has complete
>> control over the machine with respect to UEFI. SMI issues aside as
>> they are independent to UEFI.
>
> SMI handling (on x86) is very much implemented / set up as part of the
> UEFI platform firmware.

That’s not exactly true. A module loaded by UEFI could interact or be a component that runs in SMM however that is all under NDA and none of the consumer interfaces to UEFI i.e. that would be used by a boot loader or OS allow one to install any code that runs in SMM. This is interface versus implementation. Yes. The firmware /may/ have code that runs in SMM. UEFI and SMM are however distinct.

>> UEFI does not provide runtime services
>
> It does; see section "8 Services -- Runtime Services" in the UEFI 2.7
> spec.

I’ve read the UEFI spec. Yes my statement was incorrect unless it was qualified with “in general”. There is a very tiny faction of calls that can be called after ExitBootServices. Shutdown, Cold and WarmReset is a relatively large part of that subset because I believe once ExitBootServices has been called, the boot time drivers cannot be used. e.g. timers and IPI are not part of the runtime services.

So “in general” EFI does not provide runtime services as is stated with respect to SBI in terms of hardware abstraction. The bulk of the Runtime services are to do with shutdown and I believe capsule update (firmware update). I know there are also pointer conversion APIs for physical memory such that an OS can remap physical memory at different locations.

So with respect to my argument, besides shutdown, reboot, firmware update, the other 95% of EFI boot services are not available at runtime. That figure might be 92% or 96%. I have not counted the APIs.

Michael Clark

unread,
Nov 17, 2018, 8:45:53 PM11/17/18
to Richard W.M. Jones, h...@lst.de, Alex Bradbury, sw-...@groups.riscv.org
Just checking the UEFI 2.6 spec. Yes, there are what I would classify as “minor edges” to limited runtime services and privileged SMM that may be part of the firmware as a whole. But the statements are correct in the sense that UEFI is not concerned with Runtime Services from the perspective of a HAL (the point I was making) and it is also not concerned with what this runtime (SMM) may later do, beyond the limited APIs for shutdown, warm/cold reset, capsule update. UEFI is primarily a boot firmware interface (or I am missing something). The bulk of the APIs in the document are to provide boot services.

I write UEFI code and read the spec. Yes I am wrong. Possibly ~3% of the API surface can be used at runtime if one calls ExitBootServices(). The driver interfaces tend to be simple and may even be polled mode interfaces. I believe interrupt support is very limited.

Michael Clark

unread,
Nov 17, 2018, 8:53:28 PM11/17/18
to Richard W.M. Jones, h...@lst.de, Alex Bradbury, sw-...@groups.riscv.org
I think the qualifying point about runtime services is that nowhere in UEFI is it concerned with whether the firmware is active or passive. i.e. whether a (real-time) OS is timesharing with SMM firmware is completely outside of the scope of UEFI; the majority of which is concerned with services required to load an OS, update firmware securely and shutdown, restart.

Just so it be known. I am fond of UEFI. It is a well written spec and it is possible to implement a relatively minimal subset, with device-tree, such as the subset in arm’s EBBR. I would like to see this happen...

> On 18/11/2018, at 2:36 PM, Michael Clark <michae...@mac.com> wrote:
>
>
>

ron minnich

unread,
Nov 19, 2018, 1:17:56 PM11/19/18
to Michael Clark, Richard W.M. Jones, Christoph Hellwig, a...@lowrisc.org, sw-...@groups.riscv.org
more info on UEFI runtime services is here:

http://wiki.phoenix.com/wiki/index.php/EFI_RUNTIME_SERVICES

Possibly of particular interest to people here is the firmware
variable access, which requires a drop to SMM (m mode for us) to
access variables. Yes. To read a variable in the "firmware volume",
you have to call UEFI runtime services.

Further, the UEFI model is that it updates itself (see: capsule
update), which has made for lots of attack surface: UEFI is a
target-rich environment for exploit writers.

Anyway it's fine by me if people want to implement UEFI for RISCV; I
don't ever intend to use it; my only concern is that it not be
mandated (as, e.g., Intel mandated it on x86, customer desires
notwithstanding). As long as UEFI existence on riscv and my existence
on riscv have no intersection, I'm happy.
> --
> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> To post to this group, send email to sw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/19153FB2-77A9-4196-B41A-55C4B52B4D16%40mac.com.

Michael Clark

unread,
Nov 19, 2018, 8:10:51 PM11/19/18
to Luke Kenneth Casson Leighton, Samuel Falvo II, Bin Meng, Alex Bradbury, sw-...@groups.riscv.org
It’s already there in the form of paravirt ops. Xen, KVM, Hyper-V and other Hypervisors support it to different degrees. It took years to get in and was spearheaded by the Cambridge Xen folk.

Linux is pretty Hypervisor agnostic. It supports itself as a Type II Hypervisor, many Type I Hypervisors via full virtualization, paravirt ops and mixtures of both. Indeed paravirt ops is faster for some functions due to the overhead of emulating MMIO configuration space for real devices (which can require running page fault exception handlers for each load or store).

SBI is an interface. I think you mean riscv-pk/bbl. Of course a Hypervisor needs a scheduler. The SBI may or may not provide interfaces that are required for VM switch. Indeed currently it is required for timers. The services to a Type II Hypervisor which runs under a Supervisor like Linux may include SBI for the guest. I guess when folk port KVM; they will discover whether there is a need for additional SBI interfaces (cpu hot add/remove and memory ballooning come to mind)...

Benjamin Herrenschmidt

unread,
Dec 2, 2018, 1:23:36 AM12/2/18
to Christoph Hellwig, Alex Bradbury, sw-...@groups.riscv.org
On Sun, 2018-11-11 at 13:49 +0100, Christoph Hellwig wrote:
>
> In general the first category seems like a really bad idea in
> retrospective, as we could have just made the sbi interfaces instruction
> and CSRs instead, leading to an actually well defined and standalone
> usable supervisor isa. The other bits could then be left to the firmware
> interface of choice.

I very very strongly concur. These things should be architected solidly
in the ISA, not left to vendor extensions with a firmware layer to
paper over. Among others it's a very fragile approach, but the way
Linux uses these facilities has room for a lot of very specific and
fine tunes optimizations that you simply cannot do if you don't have a
very precise definition of the architecture.

For things like IPIs, FW calls would destroy your performance in a lot
of very important areas.

These things just need to be properly architected in the ISA to begin
with.

> Now do we need the SBI? Without having a compiler and/or assembler that
> knows about the superset of all vendor specific instructions and CSRs
> we very much do unless we move more of the fast (or fast-ish) path
> bits the SBI currently handles into the privilged ISA as architectural
> features.
>
>
> [and now I'll try to find some time to read the big flame ware. Or maybe
> not and enjoy the sunday afternoon instead]

Cheers,
Ben.

Benjamin Herrenschmidt

unread,
Dec 2, 2018, 1:26:24 AM12/2/18
to Christoph Hellwig, Alex Bradbury, sw-...@groups.riscv.org
On Sun, 2018-11-11 at 14:51 +0100, Christoph Hellwig wrote:
> On Sun, Nov 11, 2018 at 01:46:49PM +0000, Alex Bradbury wrote:
> > Your summary matches my current understanding. Though the scope of the
> > SBI as defined in the current privileged spec appears to go beyond
> > these two purposes. e.g. save/restore of user-visible state introduced
> > by custom extensions,
>
> Well, this is clearly the fast path in the supervisor case. And although
> architectures have come up with ways to save arbitrary extension state
> (x86 XSAVE/XSAVEOPT) that approach seems rather CISC-y and might have
> a a difficult standing in RISC-V. So it might need something SBI-ish
> or kernel and compiler support for extensions with state (which doesn't
> sound like an all bad idea to me).

Even then... how do you deal with those extensions in the context of
signal frames, ptrace accessors, perf sampling, etc... ?

Fundamentally the kernel has to know something about the state.

It might be possible to construe a way to save/restore arbitrary unit
state via FIFO-style CSRs or similar, along with DT properties to
describe the size etc... but that still won't solve all the of the
above issues.

> > and providing a virtual interface to the PLIC
> > (or presumably an alternative interrupt controller?).
>
> And this seems like a horrible idea. We should rather make an actual
> PLIC implementation architectural and let the generaly purpose OS rely
> on it (ala x86 API and ARM GIC).
>

Reply all
Reply to author
Forward
0 new messages