SBI - taking a step back

259 views
Skip to first unread message

Alex Bradbury

unread,
Nov 10, 2018, 4:21:50 PM11/10/18
to sw-...@groups.riscv.org
I've been following the recent discussions on SBI extensions with
great interest. Firstly, thank you to everyone who has been
contributing SBI proposals and documenting the current de facto SBI
standard.

I do have one request - could we all take a step back and try to
define the role and driving principles of the SBI? What problems
should it be solving? What problems shouldn't be solved via the SBI? I
understand once the framework is in place there's scope for people to
use it in a wide variety of ways, but it's really hard to judge the
various proposals without any clear definition of the SBI
requirements. Proposing incremental ABI additions seems like the wrong
way to go about development without agreeing this first.

The nearest thing we have to the guiding principles of the SBI is the
stated aim in the current privspec draft "Using a single SBI across
all SEE implementations allows a single OS binary image to run on any
SEE". The document currently proposes SBI calls for:
* Initialising, saving, and restoring user-mode state outside of the
standard FDQ extensions
* Inter-processor interrupts
* Clear timer interrupts
* Mask, unmask, and query the cause of external interrupts
(apparently, all access to the PLIC should be through the SBI?)
* Modifying the time/cycle/instret CSRs
* Scheduling timer interrupts
* TLB shootdown

This is quite an expansion over the current SBI (which seems to offer
a rather arbitrary subset of platform-specific functionality). I would
observe that none of the proposals to extend the SBI so far actually
cover all of these cases. Is that because there is disagreement over
whether the SBI should serve all these purposes, or are the proposals
simply incomplete?

Based on a recent software working group call, it seems there is a
desire for the SBI to ensure that a single kernel works across all
RISC-V implementations, including support for custom extensions (e.g.
save/restore of user-visible state as mentioned above). I'm concerned
that reaching this goal will require an increasing number of hooks
into the kernel effectively making the SBI a mini-OS. e.g. for the
tagged memory mechanism in lowRISC, saving/restoring the tagctrl
register is a necessary component of kernel support but it's also
necessary to modify the virtual memory system to support (at a
minimum) clearing tags on pages and propagating tags upon
copy-on-write.

Other questions/concerns/thoughts:
* Is there actually consensus amongst developers of S-mode code
(kernels, bootloaders etc) that the SBI is a desirable abstraction?
* Shouldn't there be an interface to allow S-mode code to control
delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
supply its own floating point emulation code. Or maybe an alternative
mechanism to swap out the M-mode handler code (as Ron has been
advocating)
* We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
intended to work in the case where a hart supports switching between
these? Surely an S-mode OS may be running with a different ISA variant
to the M-mode SBI/SEE? How is this handled?
* There are ideas floating around about SBI calls being more like
function calls (rather than the current ecall mechanism). Has anyone
written down how this would work?

How about we start by agreeing and enumerating the requirements, then
work our way up?

Thanks,

Alex

Bin Meng

unread,
Nov 10, 2018, 9:34:21 PM11/10/18
to a...@lowrisc.org, sw-...@groups.riscv.org
Hi Alex,

On Sun, Nov 11, 2018 at 5:21 AM Alex Bradbury <a...@lowrisc.org> wrote:
>
> I've been following the recent discussions on SBI extensions with
> great interest. Firstly, thank you to everyone who has been
> contributing SBI proposals and documenting the current de facto SBI
> standard.
>
> I do have one request - could we all take a step back and try to
> define the role and driving principles of the SBI? What problems
> should it be solving? What problems shouldn't be solved via the SBI? I

I agree. Maybe one more question: why is SBI a must-have for RISC-V?
To me current SBI is not absolutely necessary. All these
functionalities can be implemented by the S-mode OS. Why do they have
to be M-mode? One may say some hardware components can only be
accessed by M-mode (eg: IPI) but why should hardware expose such
limitation?

> understand once the framework is in place there's scope for people to
> use it in a wide variety of ways, but it's really hard to judge the
> various proposals without any clear definition of the SBI
> requirements. Proposing incremental ABI additions seems like the wrong
> way to go about development without agreeing this first.
>
> The nearest thing we have to the guiding principles of the SBI is the
> stated aim in the current privspec draft "Using a single SBI across
> all SEE implementations allows a single OS binary image to run on any
> SEE". The document currently proposes SBI calls for:

Shouldn't we clarify why such goal that "Using a single SBI across all
SEE implementations allows a single OS binary image to run on any SEE"
is absolutely necessary? Eventually we will move all platform-specific
stuff into the SBI implementation and that makes SBI another mini-OS
(just as you pointed out below).

> * Initialising, saving, and restoring user-mode state outside of the
> standard FDQ extensions
> * Inter-processor interrupts
> * Clear timer interrupts
> * Mask, unmask, and query the cause of external interrupts
> (apparently, all access to the PLIC should be through the SBI?)

I hope we will not go this way. See below.

> * Modifying the time/cycle/instret CSRs
> * Scheduling timer interrupts
> * TLB shootdown
>

Basically I think RISC-V should allow S-mode OS to handle things like
timer interrupts, IPI, etc, to reduce interrupt latency. I hope S-mode
can directly handle these hardware components instead of going through
the M-mode firmware. There is needs for real-time performance and
determinism.

> This is quite an expansion over the current SBI (which seems to offer
> a rather arbitrary subset of platform-specific functionality). I would
> observe that none of the proposals to extend the SBI so far actually
> cover all of these cases. Is that because there is disagreement over
> whether the SBI should serve all these purposes, or are the proposals
> simply incomplete?
>
> Based on a recent software working group call, it seems there is a
> desire for the SBI to ensure that a single kernel works across all
> RISC-V implementations, including support for custom extensions (e.g.
> save/restore of user-visible state as mentioned above). I'm concerned
> that reaching this goal will require an increasing number of hooks
> into the kernel effectively making the SBI a mini-OS. e.g. for the
> tagged memory mechanism in lowRISC, saving/restoring the tagctrl
> register is a necessary component of kernel support but it's also
> necessary to modify the virtual memory system to support (at a
> minimum) clearing tags on pages and propagating tags upon
> copy-on-write.
>
> Other questions/concerns/thoughts:
> * Is there actually consensus amongst developers of S-mode code
> (kernels, bootloaders etc) that the SBI is a desirable abstraction?

I don't think so :)

> * Shouldn't there be an interface to allow S-mode code to control
> delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
> supply its own floating point emulation code. Or maybe an alternative
> mechanism to swap out the M-mode handler code (as Ron has been
> advocating)

Agreed. We should allow some flexibility.

> * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> intended to work in the case where a hart supports switching between
> these? Surely an S-mode OS may be running with a different ISA variant
> to the M-mode SBI/SEE? How is this handled?

Do you mean switch between 32-bit and 64-bit as defined in the draft
priv spec v1.11?

> * There are ideas floating around about SBI calls being more like
> function calls (rather than the current ecall mechanism). Has anyone
> written down how this would work?
>
> How about we start by agreeing and enumerating the requirements, then
> work our way up?

Regards,
Bin

Luke Kenneth Casson Leighton

unread,
Nov 10, 2018, 9:42:07 PM11/10/18
to Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 9:21 PM Alex Bradbury <a...@lowrisc.org> wrote:

> requirements. Proposing incremental ABI additions seems like the wrong
> way to go about development without agreeing this first.

hmmm, indeed.

> The nearest thing we have to the guiding principles of the SBI is the
> stated aim in the current privspec draft "Using a single SBI across
> all SEE implementations allows a single OS binary image to run on any
> SEE".

section 1.1, V1.11-draft of privspec, SEE="Supervisor Execution Environment"

> The document currently proposes SBI calls for:

_whoa_ whoa, whoa, that's an alarmingly-large set of functionality to
suddenly encounter.


> This is quite an expansion over the current SBI (which seems to offer
> a rather arbitrary subset of platform-specific functionality).

with zero access to critical information behind the unethical
ITU-style closed doors of the RISC-V Foundation, basically from the
sparse and paltry information here we need to reverse-engineer the
intent.

"...... Using a single SBI across all SEE implementations allows a
single OS binary
image to run on any SEE. The SEE can be a simple boot loader and
BIOS-style IO system in a
low-end hardware platform, or a hypervisor-provided virtual machine in
a high-end server, or a
thin translation layer over a host operating system in an architecture
simulation environment."

from that statement, and from the list that you gave, alex, it looks
to me that the intent here is to re-create XEN.

without any kind of wider consultation.

> I would
> observe that none of the proposals to extend the SBI so far actually
> cover all of these cases. Is that because there is disagreement over
> whether the SBI should serve all these purposes, or are the proposals
> simply incomplete?

who knows? certain people do, and they're lobbing stuff over the
fence, ramming it down everyone's throats in a fait-accomplit fashion
and then claiming it's "open".

which is basically the definition of unethical behaviour.

> Based on a recent software working group call, it seems there is a
> desire for the SBI to ensure that a single kernel works across all
> RISC-V implementations, including support for custom extensions (e.g.
> save/restore of user-visible state as mentioned above). I'm concerned
> that reaching this goal will require an increasing number of hooks
> into the kernel effectively making the SBI a mini-OS.

likewise. the clue is in the "recursive" requirement: this looks
very much like a XEN style hypervisor.

"The ABI, SBI, and HBI are still a work-in-progress, but we are now
prioritizing support for
Type-2 hypervisors where the SBI is provided recursively by an S-mode OS."

where is _that_ work and discussion being published??

i'm getting really alarmed that the RISC-V Foundation is operating
extremely unethcially, by isolating everyone from discussion and
forcing the rest of the community to have to work out what the hell is
going on... by the time things are actually released it's far, far too
late,

much worse, anyone OUTSIDE of this unethically-developed process who
submits upstream patches which adds anything that they actually *want*
is quite likely to have their patches REJECTED out of hand because
they're "not submitted through the proper [unethical] channels".

> e.g. for the
> tagged memory mechanism in lowRISC, saving/restoring the tagctrl
> register is a necessary component of kernel support but it's also
> necessary to modify the virtual memory system to support (at a
> minimum) clearing tags on pages and propagating tags upon
> copy-on-write.

that's similar to what IIT Madras are doing: they have some
memory-tracking that helps detect invalid malloc / free, and i talked
to them about the idea of also adding ref-counters for languages like
python, c++ and rust, particularly those with garbage-collection.


> Other questions/concerns/thoughts:
> * Is there actually consensus amongst developers of S-mode code
> (kernels, bootloaders etc) that the SBI is a desirable abstraction?

if it was being properly publicly discussed in an OPEN fashion, and
the code being PROPERLY developed as an organic project, i would not
be concerned.

as it is, we know that the RISC-V Foundation has a consistent track
record of operating completely unethically by shoving code at people.

do we *really* want something as mission-critical as a full
Hypervisor to be developed and then controlled by a known-unethical
organisation, running on our machines?

will we be told "well if you don't like it you can always fork it",
when we *know* that's totally financially unrealistic for small
organisations and companies to even contemplate.

> How about we start by agreeing and enumerating the requirements, then
> work our way up?

that would be a good idea.

l.

Alex Bradbury

unread,
Nov 10, 2018, 11:11:14 PM11/10/18
to Luke Kenneth Casson Leighton, sw-...@groups.riscv.org
Hi Luke. I appreciate your frustration but I'd really like to keep
this thread focused on the specific technical topics I raised.

You've pointed out previously that the RISC-V mailing lists are often
not very welcoming. I think you should consider:
1) The individuals working on standards within the RISC-V Foundation
processes are doing so with good intent, doing the best they can with
the resources they have.
2) Criticism of this intensity, even when directed at "the Foundation"
rather than personally might put such individuals off communicating on
the public lists.

Thanks,

Alex

Luke Kenneth Casson Leighton

unread,
Nov 11, 2018, 1:46:00 AM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
i'd like to see that happen as well, in an open fashion, where
everyone is invited to participate, and feels welcome. it would be
absolutely fantastic, to be able to have up-to-date information and
access to the resources and discussions that allow everyone to
participate.

except that's not the case, is it? and denying it and making excuses
is not going to help the situation get any better, is it? and
expecting people to WASTE THEIR TIME *reverse-engineering* private
closed discussions... and then because it's flat-out impossible to get
it right when the information and context isn't avaiable, i have
bullies fucking well REPEATEDLY poisoning discussions by saying
"you're wrong, you're wrong, but you don't know that because WE talked
about it on a PRIVATE list that YOU don't have access to, because
you're such an oh-so-high-and-mighty ethical little tosspot, har har
we had a real laugh about that one, at your expense, loserr"


right now there's dozens of people over the years - long before i got
involved - who have been made to feel so unwelcome that they have
completely given up. one organisation is even looking to set up a
complete fork of the entire RISC-V infrastructure, they're so pissed
off.

i mean, for f***'s sake, it's right here, the words used by the
director, rick o'connor, in the barcelona workshop announcement:

" Hear from from the major players in the RISC-V ecosystem, including
the leading technology companies and research institutions driving the
RISC-V ISA specification"

hear from the companies and the research institutes, give them credit
where it's due... oh and anyone involved in the libre and open world
who contributed and put hard work into making RISC-V a success can go
f*** themselves, because we can't be bothered to mention them, at all.

the systemic laws of organisations are systemic LAWS for a reason,
and failing to acknowledge credit for peoples' efforts is a systemic
law violation, plain and simple.



> You've pointed out previously that the RISC-V mailing lists are often
> not very welcoming. I think you should consider:
> 1) The individuals working on standards within the RISC-V Foundation
> processes are doing so with good intent, doing the best they can with
> the resources they have.

repeatedly telling me and others who have completely given up that
"they're worthless", or "they're wrong" or "they're wasting
everybody's time" or failing to acknowledge their needs and
constructive input is not even *remotely* "good intent".

telling someone that their constructive criticism is an "ad hominem
attack" is not even remotely "good intent": it's a way to deflect and
deny the truth of an ongoing long-running damaging behavioural
pattern.

what's going on is called absolute arrogant power-abusive bullying,
plain and simple.


> 2) Criticism of this intensity, even when directed at "the Foundation"
> rather than personally might put such individuals off communicating on
> the public lists.

they'll continue to do what is useful to them. that includes
bullying. i've been targetted by bullies my entire life, for speaking
up when nobody else will, because they're too afraid or too vested to
do so.

i'll speak the truth, living to the integrity that i've set, and
nobody's going to tell me i can't.

when people do something good, i'll say so. when people do something
bad, i'll say so. you can't ask fairer than that.

where and when the RISC-V Foundation and the people behind it start
doing *GOOD*, i'll be their loudest, most vocal and most loyal
supporter. and you've already seen me do that: i've made it clear,
repeatedly, that i am deeply impressed and respectful of what they've
achieved.

however i cannot sit by and watch unethical behaviour unfold right in
front of my eyes, and i can NOT support unethical behaviour, or fund
it, or provide resources to organisations or individuals that operate
unethically, because to do so it itself unethical, and i absolutely
cannot do that.

it may sound weird to be both respectful of someone's accomplishments
*and* have some criticism of someone's behaviour: that's just how it
is. there's no conflict there, at all. and i say that because i've
seen people already twist my words and focus on the constructive
criticism.

you want me to stop? HELP get the unethical behaviour of the RISC-V
Foundation FIXED, by acknowledging it, discussing it, and recommending
alternatives and solutions.

if those alternatives and solutions are ethical, and properly adopted,
i'll become the RISC-V Foundation's most loyal supporter and advocate,
you can absolutely count on it.

l.

Bruce Hoult

unread,
Nov 11, 2018, 1:59:31 AM11/11/18
to Luke Kenneth Casson Leighton, Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 10:45 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
On Sun, Nov 11, 2018 at 4:11 AM Alex Bradbury <a...@lowrisc.org> wrote:
> Hi Luke. I appreciate your frustration but I'd really like to keep
> this thread focused on the specific technical topics I raised.

 i'd like to see that happen as well, in an open fashion, where
everyone is invited to participate, and feels welcome.  it would be
absolutely fantastic, to be able to have up-to-date information and
access to the resources and discussions that allow everyone to
participate.

 except that's not the case, is it?  and denying it and making excuses
is not going to help the situation get any better, is it?  and
expecting people to WASTE THEIR TIME *reverse-engineering* private
closed discussions... and then because it's flat-out impossible to get
it right when the information and context isn't avaiable, i have
bullies fucking well REPEATEDLY poisoning discussions by saying
"you're wrong, you're wrong, but you don't know that because WE talked
about it on a PRIVATE list that YOU don't have access to, because
you're such an oh-so-high-and-mighty ethical little tosspot, har har
we had a real laugh about that one, at your expense, loserr"

Luke, Alex is a very reasonable person who is trying to help.

When you reply to him thus, as a bully and an abuser, it would take a saint to remain sympathetic to you.

You're actually capable of staying within the bounds of technical discussion. I've seen it in the last week or so, in comp.arch, where you managed to reply to me in a civilised way, and even quote me approvingly from time to time.

Please try to do the same here in future.

Luke Kenneth Casson Leighton

unread,
Nov 11, 2018, 2:47:33 AM11/11/18
to Bruce Hoult, Alex Bradbury, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 6:59 AM Bruce Hoult <bruce...@sifive.com> wrote:

> Luke, Alex is a very reasonable person who is trying to help.

i know. he's acting with good intent and is following some of the
guidelines here:
http://www.crnhq.org/content.aspx?file=66138|37449y#mediation

i'm just so distressed by the constant adversarial behaviour that i'm
beginning to give up, just like the other libre/open developers who,
after years of not being listened to, are no longer actively
participating.

unfortunately, alex, whilst i fully appreciate that you were
endeavouring to act in a mediatorial role, unfortunately i did not
feel that i was being fully listened to:
http://www.crnhq.org/content.aspx?file=66138|37449y#Empathy


> You're actually capable of staying within the bounds of technical discussion.

that's right - i am!

> I've seen it in the last week or so, in comp.arch, where you managed to reply
> to me in a civilised way, and even quote me approvingly from time to time.

that's right - i did. so you know it's perfectly possible. and, if
you've reviewed the libre-risc-dev (a low-traffic mailing list as it's
only just started), you'll see more evidence of that.

if you go further back than that, you'll find evidence that i've been
the technical lead on large software libre projects for over two
decades, with in some cases over 20 simultaneous part-time
contributors.

so the question to ask is not "how* come you are civilised and
capable of staying within the bounds of technical discussion on other
forums: the real question is, what's different about *these* forums?

that's the *real* question.

> Please try to do the same here in future.

i would absolutely love to see these forums be welcoming and
productive technical forums where *everyone* is welcome, where they
can get fully up-to-date discussions and information (so that effort
is not duplicated or wasted), and can be directly involved in the
development and direction of RISC-V, where they can express their
needs and requirements clearly and be listened to, and their
contributions are appreciated, respected and encouraged.

does that sound reasonable?

l.

Bruce Hoult

unread,
Nov 11, 2018, 3:00:14 AM11/11/18
to Luke Kenneth Casson Leighton, Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 11:46 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
 so the question to ask is not "how* come you are civilised and
capable of staying within the bounds of technical discussion on other
forums: the real question is, what's different about *these* forums?

In the last few hours I've seen you being adversarial with not only Alex Bradbury, but also Nick Kossifidis (who you for unknown reason persist in addressing as "Mick"), and Tommy Murphy.  And adopting a derisory tone with others such as Ron Minnich.

There's a common thread here. These people don't talk to each other in the manner in which you reply to them.

Luke Kenneth Casson Leighton

unread,
Nov 11, 2018, 4:46:55 AM11/11/18
to Bruce Hoult, Alex Bradbury, sw-...@groups.riscv.org
---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Sun, Nov 11, 2018 at 8:00 AM Bruce Hoult <bruce...@sifive.com> wrote:
>
> On Sat, Nov 10, 2018 at 11:46 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
>>
>> so the question to ask is not "how* come you are civilised and
>> capable of staying within the bounds of technical discussion on other
>> forums: the real question is, what's different about *these* forums?
>
>
> In the last few hours I've seen you being adversarial with not only
> Alex Bradbury,

if you look carefully, alex was trying to be reasonable, and play a
mediatorial role... except if you look carefully at the advice given
on crhnq, a mediator needs to respect *both* sides, and relay (reflect
back) the position of *both* parties.

regrettably, he didn't do that, and unfortunately, all it did was
make it look he wasn't listening to what i'm saying. as i was
writing, i was thinking, "damn, damn damn, i really like alex, how the
hell can i make this look any better, so it doesn't reflect badly on
him?" and, sadly, as he (you, hello, yes, sorry to be talking _about_
you, alex) hadn't summarised and acknowledged both perspectives as a
mediator normally would, there really wasn't a way to do that.

> but also Nick Kossifidis

if you look carefully at what nick's writing, he made good points
from the field in which he has experience.. unfortunately he laced it
with hyperbolic adversarial questions that were very carefully
designed to ridicule rather than critique not only the proposal that i
made, they were also very very carefully crafted to ridicule me, as a
person, as well.

you'll notice that he dodged the positive aspects that were supported
by other people's constructive input, and focussed on generating as
much negativity as he could, through the use of what can only be
termed "hysteria" questions, hoping to assign them to me (without my
consent) and use them rhetorically for the purposes of ridicule.

the initial (thinly-veiled) adversarial attacks that he launched were
sufficiently over the top that someone else had to step in, and hint
that he was going a little too far. nick made a conscious choice to
ignore those hints.

the question here i have to ask, is, can you see that that's what
nick did, and, if you can, why do you feel that it's acceptable
behaviour on a technical forum?

> (who you for unknown reason persist in addressing as "Mick"),

sorry, nick: small high-resolution screen here, and i run the
brightness way down and the room lights off, most of the time.

> and Tommy Murphy.

tommy, bless him, i know he's trying to help. you may be confusing
the pain that i am experiencing through having to deal with the
ongoing bullying and expressing that, and thus interpreting my
responses to him as "this must be an attack on tommy". categorically
and absolutely it is not.

> And adopting a derisory tone with others such as Ron Minnich.

what?? not at all! ron makes a lot of sense, and i've said so quite
clearly!! what on *earth* gave you the impression that i've been
"derisory" with ron?? i've made it quite clear that the points he's
made about being able to operate fully in M-Mode are good ones!

> There's a common thread here.

there is, indeed. the challenge is, then, if the opportunity to
attack arises, if someone appears to be vulnerable, in pain, or
appears to not be knowledgeable, is for people to step up to the plate
and *not* react in a derisory fashion, *not* attack or denigrate them,
make them feel so unwelcome and so despised that it actually
deteriorates their physical health.

you know i speak my mind, and i made a decision 20 years ago to speak
truth, even when it's uncomfortable truth. that's what people face
here, and i know they don't like it.

i told you: ethical's a bitch. truth, awareness, love and creativity
aren't "nice", they're damn uncomfortable, and dealing with this is so
hurtful it's *literally* making me physically ill.

yet... i have a goal to achieve, the target's been set, to create a
completely libre, open and freely-available 3D-capable processor, and
to *develop* it in a truly libre and open fashion, and i am not going
to go away until that goal is achieved.

so, we have at least two more years of this. that's the reality.
so, we can use these forums for the benefit of RISC-V, or not. which
is it going to be?

we can work together, to improve RISC-V and make a conscious decision
to make the forums a welcoming place... or we can turn it into a
hate-filled forum where students, corporate employees and researchers
treat it - and RISC-V - as an absolute "No Go Zone", and psychology
researchers use the public archives in case-studies over the next
couple of decades, on how open communities go into denial when faced
with challenges.

that's what we're looking at, here. it's up to everyone to make a
conscious decision which direction they want to go.

> These people don't talk to each other in the manner in which you reply to them.

people don't talk to me on comp.arch or libre-riscv-dev in the manner
in which they reply to me on here. why?

l.

Alex Bradbury

unread,
Nov 11, 2018, 5:05:01 AM11/11/18
to Luke Kenneth Casson Leighton, sw-...@groups.riscv.org
Hi Luke,

I've changed the email subject to split this out to a separate thread.
I totally accept that your concerns about process are intertwined with
the technical discussion, but I'd much rather keep that separate to
the thread I started (more on that below). I'm not telling you what to
do, just asking.

For anyone who wants to follow the precursor to this split thread, see
<https://groups.google.com/a/groups.riscv.org/d/msg/sw-dev/dO1gR0bKfKs/z9wrlyTJAgAJ>
and <https://groups.google.com/a/groups.riscv.org/d/msg/sw-dev/dO1gR0bKfKs/zq8zanPWAgAJ>.
I don't think that the closed working groups used by the RISC-V
Foundation are the right model. This is an opinion I've expressed
repeatedly, going back to before the formation of the Foundation.
Others obviously feel differently. I think the pragmatic way forwards
is to enthusiastically embrace opportunities for more open discussion
and development. I'll note that:

1) the privileged spec source is on github with public issue tracking
<https://github.com/riscv/riscv-isa-manual>
2) the draft SBI spec is public on GitHub with public issue tracking
<https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md>
3) the de facto SBI reference implementation is upon on GitHub with
public issue tracking <https://github.com/riscv/riscv-pk>
4) The proposal from Atish on extending the SBI and all subsequent
discussions have been on public mailing lists

Given all of the above, wouldn't it make sense to work hard to keep
SBI-related technical discussions on track, with the hope of pointing
to what is hopefully a positive and focused discussion as an example
of the benefits of developing + discussing these standards out in the
open?

I'm sorry you feel that I didn't fully listen to your concerns. Mostly
there's just not much I can say in response - I can't speak for the
the RISC-V Foundation or its collaboration/development model. I was
careful not to label your concerns as invalid - I'm simply suggesting
that it would be more appropriate to raise them in a different thread,
and that you might reflect on whether the approach you're taking to
raising them is the best strategy to achieve your desired outcome.
Just a thought, and you're welcome to disagree.

Best,

Alex

Richard W.M. Jones

unread,
Nov 11, 2018, 6:31:51 AM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
> Based on a recent software working group call, it seems there is a
> desire for the SBI to ensure that a single kernel works across all
> RISC-V implementations,

Not sure what the "software working group call" is, but if this is in
any way related to what I said in my talk in Barcelona then I'd like
to clarify that I only meant for server platforms. Embedded folk can
and most likely will go their own way.

BTW the way you described the full scope of the (potential) SBI made
it sound like UEFI.

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

Christoph Hellwig

unread,
Nov 11, 2018, 7:40:55 AM11/11/18
to Richard W.M. Jones, Alex Bradbury, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 11:31:47AM +0000, Richard W.M. Jones wrote:
> On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
> > Based on a recent software working group call, it seems there is a
> > desire for the SBI to ensure that a single kernel works across all
> > RISC-V implementations,
>
> Not sure what the "software working group call" is,

I have no idea either. But more importantly having important points
only on a call is always a bad idea for a truely open project. Time
zones will get in the way of attendance, nevermind that a of people
who could contribute on mailing lists simply don't have the time to
attend regularly scheduled calls for everything they are involved in.

> but if this is in
> any way related to what I said in my talk in Barcelona then I'd like
> to clarify that I only meant for server platforms. Embedded folk can
> and most likely will go their own way.
>
> BTW the way you described the full scope of the (potential) SBI made
> it sound like UEFI.

The SBI has always been described as for Unix class systems, which
was to entail more than servers, but basically everything running
Linux, FreeBSD, etc.

Christoph Hellwig

unread,
Nov 11, 2018, 7:49:32 AM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
Hi Alex,

as someone who spent a fair time with the SBI interface and the
Supervsior (OS) side of it I'll try to chime in.

The SBI seems to be trying to archive two things, but in the few
bits actually implemented, and even more so in the grand unified
scheme of things.

First it seems to avoid having to architecturally specific things that
in other modern architectures is part of the privileged instruction set.

In the current "stub" SBI this includes TLB flushing, instruction
cache flushing on others than the local core, IPI handling, time CSR
handling. This part is specific to the actual cpu microarchitecture
used in a given setup.

The other bit is just general platform wire up that is handled by
a firmware interface (like UEFI) elsewhere. Example include the SBI
console or the shutdown handling in the current "stub" SBI. This part
is generally board specific.

In general the first category seems like a really bad idea in
retrospective, as we could have just made the sbi interfaces instruction
and CSRs instead, leading to an actually well defined and standalone
usable supervisor isa. The other bits could then be left to the firmware
interface of choice.

Now do we need the SBI? Without having a compiler and/or assembler that
knows about the superset of all vendor specific instructions and CSRs
we very much do unless we move more of the fast (or fast-ish) path
bits the SBI currently handles into the privilged ISA as architectural
features.


[and now I'll try to find some time to read the big flame ware. Or maybe
not and enjoy the sunday afternoon instead]

Luke Kenneth Casson Leighton

unread,
Nov 11, 2018, 7:57:09 AM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 10:05 AM Alex Bradbury <a...@lowrisc.org> wrote:
>
> Hi Luke,
>
> I've changed the email subject to split this out to a separate thread.
> I totally accept that your concerns about process are intertwined with
> the technical discussion, but I'd much rather keep that separate to
> the thread I started (more on that below). I'm not telling you what to
> do, just asking.

no it makes a lot of sense. i get caught up in these things.

> I don't think that the closed working groups used by the RISC-V
> Foundation are the right model. This is an opinion I've expressed
> repeatedly, going back to before the formation of the Foundation.
> Others obviously feel differently.

when i spoke to krste in barcelona about this, he explained that
there's people who just simply would not come forward - at all - if
the discussions were public. mostly, they're people who have worked
in U.S. Classified environments, and their knowledge and expertise is
too valuable to ignore.

i understand this... i see the logic... and then i'm faced with a
massive goal and the worry of keeping my family fed and housed when i
have absolutely no idea, month to month, whether i'll make the next
rent payment, living under the radar in a Republic of China.... and
i'm locked out of access to critical information and resources that
would *literally* save months if not years of development effort.

also, samuel's concern about the ITU-style standards development
process bleeding over into software development as well *has*
basically already happened [
https://groups.google.com/a/groups.riscv.org/d/msg/sw-dev/0fU8_ZBRcm4/49tMtyyrAwAJ
]

> I think the pragmatic way forwards
> is to enthusiastically embrace opportunities for more open discussion
> and development. I'll note that:
>
> 1) the privileged spec source is on github with public issue tracking
> <https://github.com/riscv/riscv-isa-manual>
> 2) the draft SBI spec is public on GitHub with public issue tracking
> <https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md>
> 3) the de facto SBI reference implementation is upon on GitHub with
> public issue tracking <https://github.com/riscv/riscv-pk>
> 4) The proposal from Atish on extending the SBI and all subsequent
> discussions have been on public mailing lists
>
> Given all of the above, wouldn't it make sense to work hard to keep
> SBI-related technical discussions on track, with the hope of pointing
> to what is hopefully a positive and focused discussion as an example
> of the benefits of developing + discussing these standards out in the
> open?

yeah, that sounds like a great plan. what you say reminds me of
"don't complain, do it". my friend today sent me a message, "you hear
of any vision that succeeded which was a massive of 'don'ts'???" :)

> I'm sorry you feel that I didn't fully listen to your concerns. Mostly
> there's just not much I can say in response -

it's ok, i'm sorry, i just... flipped.

> I can't speak for the
> the RISC-V Foundation or its collaboration/development model. I was
> careful not to label your concerns as invalid - I'm simply suggesting
> that it would be more appropriate to raise them in a different thread,
> and that you might reflect on whether the approach you're taking to
> raising them is the best strategy to achieve your desired outcome.
> Just a thought, and you're welcome to disagree.

appreciated. it's just... i've spent literally years of my life
doing reverse-engineering, cleaning up after the proprietary, secret,
closed-door messes of microsoft, google, and chinese criminal
copyright infringers, then having people sponge off of my effforts,
make a ton of money, and claim all the credit (that's *free software*
people claiming credit for my work). i'm absolutely sick of it,
figuratively *and* literally.

so whenever i am faced with closed doors and having to face the
prospect of reverse-engineering the intent behind a document, i'm
vividly - so vividly it *really* scares me - reminded of the times
when i've quite literally been homeless, and not being able to turn
the key in the lock of my house because i had RSI so badly as i was
down to under 65kg because i couldn't buy enough food, and too many
other instances to list, burned into my memory that i'm never, ever
going to forget.

i'm sharing this so that you don't just _think_ i'm motivated to work
towards the goal of a welcoming and thriving open RISC-V community,
you really *know* it.

so yeah, let's see what happens. those are useful links to start from, alex.

l.

Alex Bradbury

unread,
Nov 11, 2018, 8:16:33 AM11/11/18
to bmen...@gmail.com, sw-...@groups.riscv.org
On Sun, 11 Nov 2018 at 02:34, Bin Meng <bmen...@gmail.com> wrote:
> > * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> > intended to work in the case where a hart supports switching between
> > these? Surely an S-mode OS may be running with a different ISA variant
> > to the M-mode SBI/SEE? How is this handled?
>
> Do you mean switch between 32-bit and 64-bit as defined in the draft
> priv spec v1.11?

Yes, for a SoC that allows switching between RV32E/RV32I/RV64I, how
does the SBI support S-mode operating systems compiled for
RV32E/RV32I/RV64I? Presumably the SBI will be running with MXLEN=64
but the S-mode OS and user-space might run with SXLEN=32 and UXLEN=32.
The SBI/SEE would need to support these three different ABIs.

Similarly, the instruction trap+emulation code should be checking
SXLEN/UXLEN depending on the privilege mode which encountered the
illegal instruction. Is it currently possible to do this? I can't see
a way to differentiate a S-mode illegal instruction vs a U-mode
illegal instruction based on mcause.

Best,

Alex

Bin Meng

unread,
Nov 11, 2018, 8:26:56 AM11/11/18
to a...@lowrisc.org, sw-...@groups.riscv.org
Hi Alex,

On Sun, Nov 11, 2018 at 9:16 PM Alex Bradbury <a...@lowrisc.org> wrote:
>
> On Sun, 11 Nov 2018 at 02:34, Bin Meng <bmen...@gmail.com> wrote:
> > > * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> > > intended to work in the case where a hart supports switching between
> > > these? Surely an S-mode OS may be running with a different ISA variant
> > > to the M-mode SBI/SEE? How is this handled?
> >
> > Do you mean switch between 32-bit and 64-bit as defined in the draft
> > priv spec v1.11?
>
> Yes, for a SoC that allows switching between RV32E/RV32I/RV64I, how
> does the SBI support S-mode operating systems compiled for
> RV32E/RV32I/RV64I? Presumably the SBI will be running with MXLEN=64
> but the S-mode OS and user-space might run with SXLEN=32 and UXLEN=32.
> The SBI/SEE would need to support these three different ABIs.
>

Yes, I think we will need support such configuration in SBI when priv
spec v1.11 complaint silicon comes out.

> Similarly, the instruction trap+emulation code should be checking
> SXLEN/UXLEN depending on the privilege mode which encountered the
> illegal instruction. Is it currently possible to do this? I can't see
> a way to differentiate a S-mode illegal instruction vs a U-mode
> illegal instruction based on mcause.
>

We can check mstatus.mpp to see which privilege level the illegal
instruction came from, no?

Regards,
Bin

Alex Bradbury

unread,
Nov 11, 2018, 8:37:41 AM11/11/18
to bmen...@gmail.com, sw-...@groups.riscv.org
Ah yes, thanks. I was a bit confused by the fact there are separate
exception codes for environment calls from U-mode, S-mode and M-mode.
But I see the reasoning for this is explained in commentary for the
ECALL instruction (to allow selective delegation).

Best,

Alex

Alex Bradbury

unread,
Nov 11, 2018, 8:43:08 AM11/11/18
to h...@lst.de, sw-...@groups.riscv.org
On Sun, 11 Nov 2018 at 12:49, Christoph Hellwig <h...@lst.de> wrote:
>
> Hi Alex,
>
> as someone who spent a fair time with the SBI interface and the
> Supervsior (OS) side of it I'll try to chime in.

Thanks for the summary as you see things.

> The SBI seems to be trying to archive two things, but in the few
> bits actually implemented, and even more so in the grand unified
> scheme of things.
>
> First it seems to avoid having to architecturally specific things that
> in other modern architectures is part of the privileged instruction set.
>
> In the current "stub" SBI this includes TLB flushing, instruction
> cache flushing on others than the local core, IPI handling, time CSR
> handling. This part is specific to the actual cpu microarchitecture
> used in a given setup.
>
> The other bit is just general platform wire up that is handled by
> a firmware interface (like UEFI) elsewhere. Example include the SBI
> console or the shutdown handling in the current "stub" SBI. This part
> is generally board specific.
>
> In general the first category seems like a really bad idea in
> retrospective, as we could have just made the sbi interfaces instruction
> and CSRs instead, leading to an actually well defined and standalone
> usable supervisor isa. The other bits could then be left to the firmware
> interface of choice.

Your summary matches my current understanding. Though the scope of the
SBI as defined in the current privileged spec appears to go beyond
these two purposes. e.g. save/restore of user-visible state introduced
by custom extensions, and providing a virtual interface to the PLIC
(or presumably an alternative interrupt controller?).

Best,

Alex

Christoph Hellwig

unread,
Nov 11, 2018, 8:51:05 AM11/11/18
to Alex Bradbury, h...@lst.de, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 01:46:49PM +0000, Alex Bradbury wrote:
> Your summary matches my current understanding. Though the scope of the
> SBI as defined in the current privileged spec appears to go beyond
> these two purposes. e.g. save/restore of user-visible state introduced
> by custom extensions,

Well, this is clearly the fast path in the supervisor case. And although
architectures have come up with ways to save arbitrary extension state
(x86 XSAVE/XSAVEOPT) that approach seems rather CISC-y and might have
a a difficult standing in RISC-V. So it might need something SBI-ish
or kernel and compiler support for extensions with state (which doesn't
sound like an all bad idea to me).

> and providing a virtual interface to the PLIC
> (or presumably an alternative interrupt controller?).

And this seems like a horrible idea. We should rather make an actual
PLIC implementation architectural and let the generaly purpose OS rely
on it (ala x86 API and ARM GIC).

Nick Kossifidis

unread,
Nov 11, 2018, 9:44:18 AM11/11/18
to Luke Kenneth Casson Leighton, Bruce Hoult, Alex Bradbury, sw-...@groups.riscv.org
Hello Luke,

Στις 2018-11-11 11:46, Luke Kenneth Casson Leighton έγραψε:
>
>> but also Nick Kossifidis
>
> if you look carefully at what nick's writing, he made good points
> from the field in which he has experience.. unfortunately he laced it
> with hyperbolic adversarial questions that were very carefully
> designed to ridicule rather than critique not only the proposal that i
> made, they were also very very carefully crafted to ridicule me, as a
> person, as well.
>
> you'll notice that he dodged the positive aspects that were supported
> by other people's constructive input, and focussed on generating as
> much negativity as he could, through the use of what can only be
> termed "hysteria" questions, hoping to assign them to me (without my
> consent) and use them rhetorically for the purposes of ridicule.
>
> the initial (thinly-veiled) adversarial attacks that he launched were
> sufficiently over the top that someone else had to step in, and hint
> that he was going a little too far. nick made a conscious choice to
> ignore those hints.
>
> the question here i have to ask, is, can you see that that's what
> nick did, and, if you can, why do you feel that it's acceptable
> behaviour on a technical forum?
>

This discussion is way off topic and definitely non-technical but
since you mentioned me and what a bad person I am according to
you I guess I'll answer...

I don't know you and you don't know me. If my technical questions
offend you I can't do anything about it. That was definitely not
my intention as I don't have time to spent bullying, offending
and "carefully design" attacks to other people I don't know.
I've had enough flame wars in my 20s, I don't plan on starting
again. If you want to feel offended and present yourself as a
victim it's your bug, not mine.

If you expect people to always agree with you and support every
proposal you make, a technical discussion forum is not a place
where this is going to happen. We all want RISC-V to succeed and
each one of us here will try to convince the others why the X or
Y proposal is good or bad. That's totally normal.

As far as I'm concerned I never made a personal comment to you,
never characterized you, never judged your intentions and
never claimed that you make "hysteria" questions or anything
like that. What I've received from my intention to keep
the SBI clean (a technical reason), was a full-blown personal
attack from you. If you can't see it take a step back, read your
message and what you've written about me in public, and go back
and compare it with our other discussion.

Now I could start writing about how I've been hurt and the pain
of having someone portraying me as an evil person that wants to
ridicule him etc in public. I could go on and brag about my years
of experience and the fights I've given for open source and open
networks with zero resources, from a country with not even half
the opportunities your country provides, but that's not the point.
You see I didn't come here to make friends, I didn't come here to
feel liked or welcome, I don't expect from people to encourage me
and tell me how good my proposals are. I came here to work on
technical issues and make RISC-V fly, as it should fly ! Damn we
have the opportunity to make a difference and we waste time on this
stuff ! I could be writing a mail with my comments on Atish's SBI
proposal instead of this...

Anyway I hope we meet in person at some point, drink a beer and
laugh about this. Seriously I never meant to insult you or ridicule
you, I don't tend to spend time answering to people I think are
ridiculous.

>> (who you for unknown reason persist in addressing as "Mick"),
>
> sorry, nick: small high-resolution screen here, and i run the
> brightness way down and the room lights off, most of the time.
>

Mick is my username (long story), I'm ok with either Nick or mick.


Regards,
Nick



Jonathan Neuschäfer

unread,
Nov 11, 2018, 12:14:35 PM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
> What problems shouldn't be solved via the SBI?

I'd like to argue against sbi_shutdown, or for its clarification.
Sorry if anyone has already made these arguments; I have not thoroughly
checked the other thread.

The current definition in the riscv-sbi-doc repo[1] is:

void sbi_shutdown(void)

Puts all the harts to shut down state from supervisor point of view.
This SBI call doesn't return.

I can see two possible interpretations of this SBI call:

(a) Halt all harts (for example with the WFI instruction), and that's it.

(b) Halt all harts and also initiate a SoC-wide or board-wide power-off
sequence. Maybe I'm reading too much into the name, but "shutdown"
seems to imply this, for me.


Under interpretation (a),

- I'd prefer if it was simply possible to halt each hart directly from
S-mode, instead of going through the SBI (e.g. with WFI).
- If sbi_shutdown stays, I think it would be better to make it halt not
all harts, so that code running one one hart can halt the other harts
and then initiate a board power-off sequence[2]. I guess an interface
which only halts one specified hart would be the most flexible one.

If this is the correct interpretation, I think the SBI call should be
renamed to something more obvious, such as "sbi_halt_all_harts", or
"sbi_shutdown_harts".


Under interpretation (b),

- Board level power-off is a complex thing that might require various
drivers (I²C, SPI, IPMI, ...) in order to talk to off-chip power
management ICs. Moving these drivers into M-mode firmware is a bad
idea IMHO, for the following reasons:
- More M-mode code *potentially* means more M-mode attack surface,
weakening the security of M-mode implementations.
- Bugs in M-mode code may be harder to patch than bugs in the S-mode
operating system, because the M-mode firmware might be stored on
special flash, or signed with the SoC vendor's key, etc.
- Having drivers in M-mode and S-mode for the same hardware may cause
some conflicts and race-conditions when both drivers access the same
hardware at the same time.


Kind regards,
Jonathan Neuschäfer


[1]: https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md
[2]: S-mode shutdown code *could* be written like this:
start_poweroff_sequence();
sbi_shutdown();
// sbi_shutdown halts all harts
// the power-off sequence completes and the board is powered off
But that seems prone to timing problems, to me.
signature.asc

Samuel Falvo II

unread,
Nov 11, 2018, 1:31:39 PM11/11/18
to Bin Meng, Alex Bradbury, sw-...@groups.riscv.org
On Sat, Nov 10, 2018 at 6:34 PM Bin Meng <bmen...@gmail.com> wrote:
> I agree. Maybe one more question: why is SBI a must-have for RISC-V?
> To me current SBI is not absolutely necessary. All these
> functionalities can be implemented by the S-mode OS. Why do they have
> to be M-mode? One may say some hardware components can only be
> accessed by M-mode (eg: IPI) but why should hardware expose such
> limitation?

I think it boils down to having some manner of hardware abstraction
just sufficient enough to bootstrap that S-mode OS. To boot a typical
computer, you'll need the following things:

1) Keyboard driver, so that you can customize boot-loader options on
next reboot.
2) Display driver, so you can present said boot-loader options to the
user on next reboot.
3) Mass storage driver, so you can load binary blobs into RAM for
subsequent execution.

Note that by "driver", I'm of course not suggesting a complete
"schedule/interrupt" interface with I/O request queues and the like.
I'm just talking about thin, purely-synchronous, relatively high-level
library routines that know how to translate "read sector X from drive
Y" into a corresponding block of data that a bootloader can work with.
That's about all that you need to get a system up and running
(everything else is nice-to-have gravy, at least in the home computer
context, and from personal experience, in a data-center environment as
well).

Playing a bit of devil's advocate, there's other things that might
crop up as well which tend to be hardware implementation specific yet
are common across operating systems. I'm pretty sure my timer
hardware will be quite different from Sifive's. How IRQs are mapped
are another consideration. Having a common SBI functionality for that
then becomes valuable, if the interface is thin enough. Otherwise,
you'll have to write drivers not just for supported external
peripherals, but also for entire motherboards.

As I mentioned in another message on the topic, a good BIOS (nee a
good SBI) will serve as a hardware abstraction layer, *just* thick
enough to support booting into the preferred OS environment, and
optionally, for operation of the OS as well. Just as CP/M still
needed a BIOS to be independent of concrete computer design issues, so
too would Linux, BSD, Plan 9, et. al. need similar services. Well,
OK, not *need*; but I would imagine the overhead of maintaining so
many different configurations of the kernel would certainly make it
strongly preferable, especially if SBI performance were on par with
calling any normal C function, or can be made to be called relatively
infrequently.

> is absolutely necessary? Eventually we will move all platform-specific
> stuff into the SBI implementation and that makes SBI another mini-OS

That is the crux right here, I think -- there are three groups of
people vying for influence on how system firmware should be handled:
(1) the pro-UEFI(-like) folks who thinks that system firmware should
include the kitchen sink, (2) the pro-BIOS folks (like myself) which
thinks it should be a thin abstraction over the available hardware,
and then (3) there's the anti-firmware folks who thinks that the
firmware should be opaque to any operating system you load, with
complete and total hand-off to the kernel once the bootloader has
completed its job.

> the M-mode firmware. There is needs for real-time performance and
> determinism.

There's also a real need for OS authors to have some manner of
reliable interface to a wider variety of system implementations during
the bootstrap process, and if I had my preference, during OS operation
as well.

Put bluntly, I'd much rather have a kernel which is slower but runs on
wider hardware than hyper-optimized kernels that can only run on one
motherboard's processor and/or chipset. Computers are fast enough for
me to not have to care about break-neck performance anymore. The only
exception I'd make for that is in a data center context, where now I'm
managing entire fleets of like-configured systems. Here, it makes
sense to maintain local kernel images, software repositories, et. al.
Ditto as well for deep-embedded environments.

But, for laptop use? Phone use? Tablet use? TV set-top box use?
The idea of maintaining a crap-ton of kernels and chipset/motherboard
drivers just does not appeal to me at all.

> > * Is there actually consensus amongst developers of S-mode code
> > (kernels, bootloaders etc) that the SBI is a desirable abstraction?
>
> I don't think so :)

Linux has a HAL. Windows uses a HAL. Most modern kernels rely upon a
HAL-like "thing" at some level. That's an SBI but by a different
name.

It's clear that it's a valuable abstraction. The question is, I
*think*, whether having a common interface across OSes is valuable.

--
Samuel A. Falvo II

Alex Bradbury

unread,
Nov 11, 2018, 1:57:42 PM11/11/18
to Samuel Falvo II, bmen...@gmail.com, sw-...@groups.riscv.org
On Sun, 11 Nov 2018 at 18:31, Samuel Falvo II <sam....@gmail.com> wrote:
> Linux has a HAL. Windows uses a HAL. Most modern kernels rely upon a
> HAL-like "thing" at some level. That's an SBI but by a different
> name.
>
> It's clear that it's a valuable abstraction. The question is, I
> *think*, whether having a common interface across OSes is valuable.

I think you've hit the nail on the head. Assuming agreement on the
need for some sort of SBI, should it be scoped to _only_ provide code
necessary for first boot of a kernel and maybe providing some basic
platform-specific functionality (tlb shootdown etc) that isn't
(currently) defined in the spec. Or should it actively seek to
increase code reuse across different OSes and provide a level of
abstraction that reduces platform-specific code? e.g. abstracting the
PLIC or other platform interrupt controller. If you go that far, why
stop there? You could provide helper functionality for handling page
tables, or implement a range of other device drivers in M-mode
firmware.

Speccing the complete desired interface is the only way to go I think.
As Christoph suggests, having a RISC-V way of saving user-visible
extension state isn't a bad idea (though of course this may just be
one small part of supporting a custom extension). But someone needs to
define what the interface for doing that would look like. Requiring an
extra ecall to provide this functionality sounds expensive.

Best,

Alex

Nick Kossifidis

unread,
Nov 11, 2018, 9:21:12 PM11/11/18
to Alex Bradbury, sw-...@groups.riscv.org
Hello Alex,

Στις 2018-11-10 23:25, Alex Bradbury έγραψε:
> I've been following the recent discussions on SBI extensions with
> great interest. Firstly, thank you to everyone who has been
> contributing SBI proposals and documenting the current de facto SBI
> standard.
>
> I do have one request - could we all take a step back and try to
> define the role and driving principles of the SBI? What problems
> should it be solving? What problems shouldn't be solved via the SBI? I
> understand once the framework is in place there's scope for people to
> use it in a wide variety of ways, but it's really hard to judge the
> various proposals without any clear definition of the SBI
> requirements. Proposing incremental ABI additions seems like the wrong
> way to go about development without agreeing this first.
>

+1

> The nearest thing we have to the guiding principles of the SBI is the
> stated aim in the current privspec draft "Using a single SBI across
> all SEE implementations allows a single OS binary image to run on any
> SEE". The document currently proposes SBI calls for:
> * Initialising, saving, and restoring user-mode state outside of the
> standard FDQ extensions
> * Inter-processor interrupts
> * Clear timer interrupts
> * Mask, unmask, and query the cause of external interrupts
> (apparently, all access to the PLIC should be through the SBI?)
> * Modifying the time/cycle/instret CSRs
> * Scheduling timer interrupts
> * TLB shootdown
>
> This is quite an expansion over the current SBI (which seems to offer
> a rather arbitrary subset of platform-specific functionality). I would
> observe that none of the proposals to extend the SBI so far actually
> cover all of these cases. Is that because there is disagreement over
> whether the SBI should serve all these purposes, or are the proposals
> simply incomplete?
>

Regarding IPI it can't be done by the supervisor, it needs to happen
on M mode (see privilege spec 1.11-draft, sec 3.1.14, p29), so it makes
sense to me to go through the SBI. The same goes for the remote fence.

Regarding timers, setting mtimecmp also needs M mode access so we have
to go through SBI for using it.

Regarding external interrupts, I guess we could have an SBI call for
handling mideleg but other than that the supervisor/userspace just use
their own sie/uie CSRs right now, so we don't use SBI for that and I
think the recommendation on the privilege spec should go away as well.

As for the cycle and instret CSRs they are handled directly by the
supervisor as well (see arch/riscv/kernel/perf_event.c). So again
maybe we should remove that recommendation from the privilege spec
at some point. By the way I don't think they can be modified, they
are marked as URO (privilege spec 1.11-draft sec. 2.2).

So to me what we have right now makes sense, we don't use the SBI as
an alternative to the supervisor or as an abstraction layer. It's
mostly an interface to the firmware for handling higher-privileged
functions the supervisor can't handle.

When it comes to saving/restoring user-mode etc state outside of
the standard extensions, I think it's related to the other issue
you mention, see below...

> Based on a recent software working group call, it seems there is a
> desire for the SBI to ensure that a single kernel works across all
> RISC-V implementations, including support for custom extensions (e.g.
> save/restore of user-visible state as mentioned above). I'm concerned
> that reaching this goal will require an increasing number of hooks
> into the kernel effectively making the SBI a mini-OS. e.g. for the
> tagged memory mechanism in lowRISC, saving/restoring the tagctrl
> register is a necessary component of kernel support but it's also
> necessary to modify the virtual memory system to support (at a
> minimum) clearing tags on pages and propagating tags upon
> copy-on-write.
>

Just so that everyone is on the same page here, the discussion started
on [1] and the problem at hand is that we don't want custom CSRs or
instructions of the various vendor-specific extensions ending up on
the Linux kernel since we want to keep the kernel clean and
independent of custom toolchains (I've explained the issues on [2]).
The proposed approach we discussed on the working group call and on
that thread, was to be able to call custom/vendor-specific code
through the SBI so that vendors can keep their extensions on the
firmware, instead of messing up the kernel (also see [3]). As an
example we talked about having a table of function pointers of the
hooks you mention, that will be initialized on early boot. If a hook
is not used it'll be a nop, if it is, a pointer to the function that
does the SBI call will be used. It's not an ideal solution but it's
better than the alternative.

> Other questions/concerns/thoughts:
> * Is there actually consensus amongst developers of S-mode code
> (kernels, bootloaders etc) that the SBI is a desirable abstraction?

Again, to be clear, right now SBI is not an abstraction, the
discussion for using it as an abstraction to "hide" custom extension
assembly code from the kernel is still ongoing (latest update was [3]
AFAIK).

I can speak only for myself here, the code that handles arch-specific
things within the OS/bootloader should be clean and simple. On RISC-V
this is a challenge to achieve. The ISA covers only the operation of
a hart (which to me it makes perfect sense), there are areas not
covered by the ISA that in contrast with other archs we don't know
what to expect ! There will be systems out there that have coherent
caches for example and don't need remote fence to e.g. update their
page tables, there will be systems that don't provide any coherency
and we should always force fence, there will be systems without caches
or with caches that cover only part of the memory etc. We should be
open for new things to come and ready to handle them in a clean way.
Software should be designed to support hardware's modularity, the
same way we allow extensions on hw, we should allow them on software.
Without a hooking mechanism in place the code will get chaotic with
if checks everywhere etc. Not all hooks need to use the SBI obviously,
but in case custom assembly code or M mode access etc is needed, I
don't see any other way than using the SBI.

> * Shouldn't there be an interface to allow S-mode code to control
> delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
> supply its own floating point emulation code. Or maybe an alternative
> mechanism to swap out the M-mode handler code (as Ron has been
> advocating)

I guess it's a valid request to use the SBI for controlling mideleg,
feel free to suggest it to Atish for inclusion. To me it should be
optional since minimizing / eliminating the barrier between M mode
and the rest is a security control bypass. I can see the benefit
though in some cases.

> * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> intended to work in the case where a hart supports switching between
> these? Surely an S-mode OS may be running with a different ISA variant
> to the M-mode SBI/SEE? How is this handled?

For the current calling convention, check out
arch/riscv/include/asm/sbi.h and [4]. There is a weak spot IMHO:

Passing 32bit values to/from M mode through a* registers is straight
forward and should be compatible across different base ISAs. However
passing pointers is another beast. According to the spec Sv32/39/48
are part of the supervisor-level ISA, there is nothing about what kind
of virtual memory mapping scheme the firmware on M mode should use. I
guess from the code that the assumption is that M mode will keep the
same scheme and page table etc that the supervisor has configured, but
there is no documentation on that. At least we know that the sbi
call will happen on the same hart as the caller so it should be
fine.

> * There are ideas floating around about SBI calls being more like
> function calls (rather than the current ecall mechanism). Has anyone
> written down how this would work?
>

On [2] I wrote "If we see that the SBI has too much overhead or is not
enough we can design it in a better way or extend it". One way of
doing that which I suggested on the call and Palmer also wrote about
on [3] is to export SBI as a library, sort of how the vDSO works on
Linux -> http://man7.org/linux/man-pages/man7/vdso.7.html. It seems
that an early version of the SBI used a similar approach instead
of ecalls. Apart from the idea, no one has come up so far with a
document on how this can be implemented, I guess the way Linux
does it should be a start.

Hope I helped !

Regards,
Nick

> How about we start by agreeing and enumerating the requirements, then
> work our way up?
>
> Thanks,
>
> Alex

[1]: https://lkml.org/lkml/2018/10/31/431
[2]: https://lkml.org/lkml/2018/11/6/2
[3]: https://lkml.org/lkml/2018/11/7/189
[4]: https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md

Bruce Hoult

unread,
Nov 11, 2018, 9:26:39 PM11/11/18
to Alex Bradbury, Samuel Falvo II, bmen...@gmail.com, sw-...@groups.riscv.org
On Sun, Nov 11, 2018 at 11:01 AM, Alex Bradbury <a...@lowrisc.org> wrote:
On Sun, 11 Nov 2018 at 18:31, Samuel Falvo II <sam....@gmail.com> wrote:
> Linux has a HAL.  Windows uses a HAL.  Most modern kernels rely upon a
> HAL-like "thing" at some level.  That's an SBI but by a different
> name.
>
> It's clear that it's a valuable abstraction.  The question is, I
> *think*, whether having a common interface across OSes is valuable.

I think you've hit the nail on the head. Assuming agreement on the
need for some sort of SBI, should it be scoped to _only_ provide code
necessary for first boot of a kernel and maybe providing some basic
platform-specific functionality (tlb shootdown etc) that isn't
(currently) defined in the spec. Or should it actively seek to
increase code reuse across different OSes and provide a level of
abstraction that reduces platform-specific code? e.g. abstracting the
PLIC or other platform interrupt controller. If you go that far, why
stop there? You could provide helper functionality for handling page
tables, or implement a range of other device drivers in M-mode
firmware.

The absolutely essential is the kind of thing that might be included in hardware in a CISC architecture, but we do in low level software on RISC-V. The same kind of idea as DEC Alpha "PALCode".

However I think I'd like to see enough there that a basic OS *could* choose to use nothing but SBI calls to talk to the world. But would not be forced to, and could use its own drivers if it wants.

So I'd include something to enumerate character and block devices and read and write them. *Without* any interpretation of the contents. The OS itself would have to provide the code that knows how to identify and work with partition tables, FAT or EXT filesystems etc.

Manipulating page table layouts I'm probably against. They are well defined in the privileged spec, so it's only things that follow a different privileged spec that would benefit. I think the only divergence we should see there is machines too small to run a proper OS anyway.

However TLB design, functioning, and management on the other hand is completely undefined in the spec. Technically, you don't even have to have one, and every memory reference could walk the page tables. Calls to manage local and remote TLBs would be useful. I'd include that.

Abstracting interrupts doesn't seem too useful. PLIC is already pretty simple to deal with and the newer CLIC is backwards compatible with it. I'd imagine that would continue in any future enhancement, though a lot of thought has gone into CLIC and it will probably serve well for a long time.

The SBI code shouldn't be very big. Single figure KB I guess. The current BBL is 27 KB, but includes things an SBI probably should have such as a floating point emulation library.

 
 
Speccing the complete desired interface is the only way to go I think.
As Christoph suggests, having a RISC-V way of saving user-visible
extension state isn't a bad idea (though of course this may just be
one small part of supporting a custom extension). But someone needs to
define what the interface for doing that would look like. Requiring an
extra ecall to provide this functionality sounds expensive.

An ecall the provides a service for a process, rather than switching processes, doesn't have to be much more expensive than a standard function call.

Alan Kao

unread,
Nov 11, 2018, 10:43:34 PM11/11/18
to Nick Kossifidis, Alex Bradbury, sw-...@groups.riscv.org
Hi Nick,
I would like to add some comment here since I am resiponsible for the
mentioned source file.

Perf (which performance counters is mainly for) needs some HW features
that are not currently available. Please also check
Documentations/riscv/pmu.txt if you want to know more details. You already
mentioned one instance here that kernel cannot modify these counters,
so a SBI to enable writing will be a must.

I will keep a lite proposal about perf until RISC-V MC on Thursday unless
any of you are urged to know perf-related issues.

Alan

atish patra

unread,
Nov 11, 2018, 11:38:47 PM11/11/18
to j.neus...@gmx.net, a...@lowrisc.org, sw-...@groups.riscv.org
As per my understanding, this is the correct interpretation.
Once, we have power management in place, we can decide on correct power-off
sequence and board specific shutdown interface.
 

Under interpretation (b),

- Board level power-off is a complex thing that might require various
  drivers (I²C, SPI, IPMI, ...) in order to talk to off-chip power
  management ICs. Moving these drivers into M-mode firmware is a bad
  idea IMHO, for the following reasons:
  - More M-mode code *potentially* means more M-mode attack surface,
    weakening the security of M-mode implementations.
  - Bugs in M-mode code may be harder to patch than bugs in the S-mode
    operating system, because the M-mode firmware might be stored on
    special flash, or signed with the SoC vendor's key, etc.
  - Having drivers in M-mode and S-mode for the same hardware may cause
    some conflicts and race-conditions when both drivers access the same
    hardware at the same time.


Kind regards,
Jonathan Neuschäfer


[1]: https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.md
[2]: S-mode shutdown code *could* be written like this:
       start_poweroff_sequence();
       sbi_shutdown();
       // sbi_shutdown halts all harts
       // the power-off sequence completes and the board is powered off
     But that seems prone to timing problems, to me.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/20181111171431.GA2219%40latitude.


--
Regards,
Atish

atish patra

unread,
Nov 11, 2018, 11:42:49 PM11/11/18
to a...@lowrisc.org, h...@lst.de, sw-...@groups.riscv.org
I am not sure why SBI is required for PLIC either. IIRC, it was mentioned in a very old version of privileged spec (1.9.x).
But it got removed eventually. As per current implementation, all PLIC handling is happening directly through kernel.
Even current SBI implementation doesn't have anything related to that.
 
Regards,
Atish
Best,

Alex


--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

atish patra

unread,
Nov 12, 2018, 12:55:31 AM11/12/18
to ala...@andestech.com, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
I believe you are talking about writing to performance counters.
What's the common practice in other architectures? If it can be added to the privileged spec, 
we probably should. Otherwise, we can discuss in details during Plumbers. 
But, my general suggestion is that SBI shouldn't be used as a kitchen sink.

I will keep a lite proposal about perf until RISC-V MC on Thursday unless
any of you are urged to know perf-related issues.


Great looking forward to it.
 
Alan


--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

atish patra

unread,
Nov 12, 2018, 1:31:18 AM11/12/18
to a...@lowrisc.org, sw-...@groups.riscv.org
Hi Alex,
Thanks for starting this topic. I believe you raised a very valid point.
IMHO, SBI should not become UEFI or a "mini" OS. I view SBI should provide
an interface for S mode that helps in maintaining a clear separation between privileged
mode & machine mode. It should include features that are absolutely necessary and would
need a hack to implement in supervisor mode. This is just my opinion. 

As RISC-V is modern architecture and we all envision it to be used in various types of computing
devices, the idea with the proposal is to introduce minimal functions and keep them optional.
For example, if any embedded/iot devices doesn't need to use hart power management, they
can skip it and same supervisor OS will continue to work.

However, the primary goal is to have a proper function ID scheme & introduce version/feature check
capabilities at this point. That will help in adding any feature (i.e. absolutely necessary) in future.
Currently, it is not possible at all.

I am not sure if it will be helpful. But I can maintain a section of list of new feature suggestions in the
proposal. We can take a call if that feature is going to be added to SBI or not after it has been discussed
enough in public forums. But we need a continuous feedback/discussion on this so that we can finalize
a version of SBI at least.

On Sat, Nov 10, 2018 at 1:21 PM Alex Bradbury <a...@lowrisc.org> wrote:
I've been following the recent discussions on SBI extensions with
great interest. Firstly, thank you to everyone who has been
contributing SBI proposals and documenting the current de facto SBI
standard.

I do have one request - could we all take a step back and try to
define the role and driving principles of the SBI? What problems
should it be solving? What problems shouldn't be solved via the SBI? I
understand once the framework is in place there's scope for people to
use it in a wide variety of ways, but it's really hard to judge the
various proposals without any clear definition of the SBI
requirements. Proposing incremental ABI additions seems like the wrong
way to go about development without agreeing this first.

The nearest thing we have to the guiding principles of the SBI is the
stated aim in the current privspec draft "Using a single SBI across
all SEE implementations allows a single OS binary image to run on any
SEE".  The document currently proposes SBI calls for:
* Initialising, saving, and restoring user-mode state outside of the
standard FDQ extensions
* Inter-processor interrupts
* Clear timer interrupts
* Mask, unmask, and query the cause of external interrupts
(apparently, all access to the PLIC should be through the SBI?)
* Modifying the time/cycle/instret CSRs
* Scheduling timer interrupts
* TLB shootdown

This is quite an expansion over the current SBI (which seems to offer
a rather arbitrary subset of platform-specific functionality). I would
observe that none of the proposals to extend the SBI so far actually
cover all of these cases. Is that because there is disagreement over
whether the SBI should serve all these purposes, or are the proposals
simply incomplete?

Based on a recent software working group call, it seems there is a
desire for the SBI to ensure that a single kernel works across all
RISC-V implementations, including support for custom extensions (e.g.
save/restore of user-visible state as mentioned above). I'm concerned
that reaching this goal will require an increasing number of hooks
into the kernel effectively making the SBI a mini-OS. e.g. for the
tagged memory mechanism in lowRISC, saving/restoring the tagctrl
register is a necessary component of kernel support but it's also
necessary to modify the virtual memory system to support (at a
minimum) clearing tags on pages and propagating tags upon
copy-on-write.

Other questions/concerns/thoughts:
* Is there actually consensus amongst developers of S-mode code
(kernels, bootloaders etc) that the SBI is a desirable abstraction?
* Shouldn't there be an interface to allow S-mode code to control
delegation from M-mode (the SBI/SEE)? e.g. if my OS would prefer to
supply its own floating point emulation code. Or maybe an alternative
mechanism to swap out the M-mode handler code (as Ron has been
advocating)
* We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
intended to work in the case where a hart supports switching between
these? Surely an S-mode OS may be running with a different ISA variant
to the M-mode SBI/SEE? How is this handled?
* There are ideas floating around about SBI calls being more like
function calls (rather than the current ecall mechanism). Has anyone
written down how this would work?

How about we start by agreeing and enumerating the requirements, then
work our way up?

Thanks,

Alex

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Alex Bradbury

unread,
Nov 12, 2018, 4:12:07 AM11/12/18
to mi...@ics.forth.gr, sw-...@groups.riscv.org
With the exception of sbi_console_putchar, sbi_console_getchar and
maybe sbi_shutdown perhaps. Though I know you argued elsewhere that
the serial should _only_ be used for earlyprintk or the equivalent.
I'm not quite sure how access to the serial port should be mediated,
but that makes sense.

> Just so that everyone is on the same page here, the discussion started
> on [1] and the problem at hand is that we don't want custom CSRs or
> instructions of the various vendor-specific extensions ending up on
> the Linux kernel since we want to keep the kernel clean and
> independent of custom toolchains (I've explained the issues on [2]).
> The proposed approach we discussed on the working group call and on
> that thread, was to be able to call custom/vendor-specific code
> through the SBI so that vendors can keep their extensions on the
> firmware, instead of messing up the kernel (also see [3]). As an
> example we talked about having a table of function pointers of the
> hooks you mention, that will be initialized on early boot. If a hook
> is not used it'll be a nop, if it is, a pointer to the function that
> does the SBI call will be used. It's not an ideal solution but it's
> better than the alternative.

Thanks for the summary and pointers. I understand that there's a focus
on developing a Linux kernel policy, but I think it would be
worthwhile exposing more of these discussions to the wider RISC-V
community. Although not a requirement, it seems it would be beneficial
for different RISC-V compatible OSes/kernels to arrive at a similar
policy and agree the SBI interface. Given that Palmer has labeled both
sw-dev and isa-dev as "worthless"
<https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/mhng-2e75d6c3-ec0f-42db-be43-cdba1cf52248%40palmer-si-x1c4>
I'm not really sure where that discussion should take place.

I think it's worth remembering that there's no reason custom
extensions shouldn't/wouldn't get supported in standard clang and GCC.
Also, in the cases where it it just comes down to accessing
non-standard CSRs, requiring that Linux kernel code use CSR numbers
rather than assembler support for CSR names doesn't seem too onerous.

Looking at the linked LKML threads, Arnd Bergmann suggested "If a
vendor extension requires kernel support, that support must be able to
be built into a kernel image without breaking support for CPUs that do
not have that extension, to allow building a single kernel image that
works on all CPUs." That seemed a reasonable starting point to me
<https://lkml.org/lkml/2018/11/5/283>


> > Other questions/concerns/thoughts:
> > * Is there actually consensus amongst developers of S-mode code
> > (kernels, bootloaders etc) that the SBI is a desirable abstraction?
>
> Again, to be clear, right now SBI is not an abstraction, the
> discussion for using it as an abstraction to "hide" custom extension
> assembly code from the kernel is still ongoing (latest update was [3]
> AFAIK).
>
> I can speak only for myself here, the code that handles arch-specific
> things within the OS/bootloader should be clean and simple. On RISC-V
> this is a challenge to achieve. The ISA covers only the operation of
> a hart (which to me it makes perfect sense), there are areas not
> covered by the ISA that in contrast with other archs we don't know
> what to expect ! There will be systems out there that have coherent
> caches for example and don't need remote fence to e.g. update their
> page tables, there will be systems that don't provide any coherency
> and we should always force fence, there will be systems without caches
> or with caches that cover only part of the memory etc. We should be
> open for new things to come and ready to handle them in a clean way.
> Software should be designed to support hardware's modularity, the
> same way we allow extensions on hw, we should allow them on software.
> Without a hooking mechanism in place the code will get chaotic with
> if checks everywhere etc. Not all hooks need to use the SBI obviously,
> but in case custom assembly code or M mode access etc is needed, I
> don't see any other way than using the SBI.

I'm concerned that supporting arbitrary extensions via SBI hooks is
too ambitious. Using an SBI call to provide some equivalent to X86
XSAVE has value and it should be enough to support a simple extension
that provides some user-visible state. But it's not sufficient for all
such extensions. e.g. the example of tagged memory I gave previously,
I'd imagine a range of privspec extensions. Is the intent that people
introduce new hooks whenever necessary? Or do we just accept that a
limited number of SBI hooks handles a lot of common cases, but not
everything? I'm just trying to encourage some explicitness about the
end goal here - again, it's always helpful to have a clear picture of
what problems a scheme doesn't solve (or doesn't try to solve) when
evaluating it.

> > * We currently have 3 base ISAs - RV32I, RV32E, RV64I. How is the SBI
> > intended to work in the case where a hart supports switching between
> > these? Surely an S-mode OS may be running with a different ISA variant
> > to the M-mode SBI/SEE? How is this handled?
>
> For the current calling convention, check out
> arch/riscv/include/asm/sbi.h and [4]. There is a weak spot IMHO:
>
> Passing 32bit values to/from M mode through a* registers is straight
> forward and should be compatible across different base ISAs. However
> passing pointers is another beast. According to the spec Sv32/39/48
> are part of the supervisor-level ISA, there is nothing about what kind
> of virtual memory mapping scheme the firmware on M mode should use. I
> guess from the code that the assumption is that M mode will keep the
> same scheme and page table etc that the supervisor has configured, but
> there is no documentation on that. At least we know that the sbi
> call will happen on the same hart as the caller so it should be
> fine.

We of course run in to problems if more complex datastructures are
passed over the SBI. e.g. the structs containing two longs proposed by
Atish will obviously differ depending on -mabi=ilp32 vs -mabi=lp64.
Given that the SBI is meant to be extensible, these concerns should be
clearly documented in order to help vendors make the right choices
when extending the SBI.

> > * There are ideas floating around about SBI calls being more like
> > function calls (rather than the current ecall mechanism). Has anyone
> > written down how this would work?
> >
>
> On [2] I wrote "If we see that the SBI has too much overhead or is not
> enough we can design it in a better way or extend it". One way of
> doing that which I suggested on the call and Palmer also wrote about
> on [3] is to export SBI as a library, sort of how the vDSO works on
> Linux -> http://man7.org/linux/man-pages/man7/vdso.7.html. It seems
> that an early version of the SBI used a similar approach instead
> of ecalls. Apart from the idea, no one has come up so far with a
> document on how this can be implemented, I guess the way Linux
> does it should be a start.
>
> Hope I helped !

You helped massively, thanks!

Bes,t

Alex

Alex Bradbury

unread,
Nov 12, 2018, 4:18:25 AM11/12/18
to atis...@gmail.com, h...@lst.de, sw-...@groups.riscv.org
On Mon, 12 Nov 2018 at 04:42, atish patra <atis...@gmail.com> wrote:
> I am not sure why SBI is required for PLIC either. IIRC, it was mentioned in a very old version of privileged spec (1.9.x).
> But it got removed eventually. As per current implementation, all PLIC handling is happening directly through kernel.
> Even current SBI implementation doesn't have anything related to that.

The current privileged spec draft states:
"""
In the expected use case, only machine mode accesses the source
priority, source pending, and target interrupt enables to configure
the interrupt subsystem. Lower-privilege modes access these features
via ABI or SBI calls. The interrupt enables act as a protection
mechanism where a target can only signal completion to an interrupt
gateway that is currently enabled for that target.
"""

Best,

Alex

Alex Bradbury

unread,
Nov 12, 2018, 4:43:11 AM11/12/18
to atis...@gmail.com, sw-...@groups.riscv.org
On Mon, 12 Nov 2018 at 06:31, atish patra <atis...@gmail.com> wrote:
>
> Hi Alex,
> Thanks for starting this topic. I believe you raised a very valid point.
> IMHO, SBI should not become UEFI or a "mini" OS. I view SBI should provide
> an interface for S mode that helps in maintaining a clear separation between privileged
> mode & machine mode. It should include features that are absolutely necessary and would
> need a hack to implement in supervisor mode. This is just my opinion.
>
> As RISC-V is modern architecture and we all envision it to be used in various types of computing
> devices, the idea with the proposal is to introduce minimal functions and keep them optional.
> For example, if any embedded/iot devices doesn't need to use hart power management, they
> can skip it and same supervisor OS will continue to work.
>
> However, the primary goal is to have a proper function ID scheme & introduce version/feature check
> capabilities at this point. That will help in adding any feature (i.e. absolutely necessary) in future.
> Currently, it is not possible at all.
>
> I am not sure if it will be helpful. But I can maintain a section of list of new feature suggestions in the
> proposal. We can take a call if that feature is going to be added to SBI or not after it has been discussed
> enough in public forums. But we need a continuous feedback/discussion on this so that we can finalize
> a version of SBI at least.

My suggestion is that it's really hard to give that feedback and see
that discussion moving forwards productively without 1) a clear but
brief description of the scope and purpose of the SBI and the problems
it's trying to solve. 2) a proposal that at least sketches out how the
full range of requirements should be handled (e.g. is XSAVE-style
functionality provided via the same ecall mechanism? or is a function
pointer provided via an SBI call that is called directly? or something
else?).

Best,

Alex

David Craven

unread,
Nov 12, 2018, 6:03:18 AM11/12/18
to a...@lowrisc.org, atis...@gmail.com, sw-...@groups.riscv.org
> Given that Palmer has labeled both sw-dev and isa-dev as "worthless"
<https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/mhng-2e75d6c3-ec0f-42db-be43-cdba1cf52248%40palmer-si-x1c4>
I'm not really sure where that discussion should take place.

Maybe someone can create a ra...@groups.riscv.org mailing list and
kindly ask users to use the rant mailing list for ranting. That way
technical discussion can still happen here.

Bin Meng

unread,
Nov 12, 2018, 7:09:17 AM11/12/18
to j.neus...@gmx.net, a...@lowrisc.org, sw-...@groups.riscv.org
Agreed. That's why I asked why do we need SBI at all in the first
place. None of services provided by current SBI spec seem to be really
necessary for M-mode. And "providing a virtual interface to the PLIC
(or presumably an alternative interrupt controller?)" seems to be on
the wrong track. I agreed with what Christoph Hellwig said, we should
leave it for S-mode OS to drive it directly. And this is not only for
PLIC, but for other hardware components like timer/IPI, etc.

> - More M-mode code *potentially* means more M-mode attack surface,
> weakening the security of M-mode implementations.
> - Bugs in M-mode code may be harder to patch than bugs in the S-mode
> operating system, because the M-mode firmware might be stored on
> special flash, or signed with the SoC vendor's key, etc.
> - Having drivers in M-mode and S-mode for the same hardware may cause
> some conflicts and race-conditions when both drivers access the same
> hardware at the same time.
>

Regards,
Bin

Bin Meng

unread,
Nov 12, 2018, 7:36:11 AM11/12/18
to Samuel Falvo II, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Samuel,

On Mon, Nov 12, 2018 at 2:31 AM Samuel Falvo II <sam....@gmail.com> wrote:
>
> On Sat, Nov 10, 2018 at 6:34 PM Bin Meng <bmen...@gmail.com> wrote:
> > I agree. Maybe one more question: why is SBI a must-have for RISC-V?
> > To me current SBI is not absolutely necessary. All these
> > functionalities can be implemented by the S-mode OS. Why do they have
> > to be M-mode? One may say some hardware components can only be
> > accessed by M-mode (eg: IPI) but why should hardware expose such
> > limitation?
>
> I think it boils down to having some manner of hardware abstraction
> just sufficient enough to bootstrap that S-mode OS. To boot a typical
> computer, you'll need the following things:
>
> 1) Keyboard driver, so that you can customize boot-loader options on
> next reboot.
> 2) Display driver, so you can present said boot-loader options to the
> user on next reboot.
> 3) Mass storage driver, so you can load binary blobs into RAM for
> subsequent execution.
>
> Note that by "driver", I'm of course not suggesting a complete
> "schedule/interrupt" interface with I/O request queues and the like.
> I'm just talking about thin, purely-synchronous, relatively high-level
> library routines that know how to translate "read sector X from drive
> Y" into a corresponding block of data that a bootloader can work with.

coreboot/U-Boot can meet such requirement very well :)

> That's about all that you need to get a system up and running
> (everything else is nice-to-have gravy, at least in the home computer
> context, and from personal experience, in a data-center environment as
> well).
>
> Playing a bit of devil's advocate, there's other things that might
> crop up as well which tend to be hardware implementation specific yet
> are common across operating systems. I'm pretty sure my timer
> hardware will be quite different from Sifive's. How IRQs are mapped
> are another consideration. Having a common SBI functionality for that
> then becomes valuable, if the interface is thin enough. Otherwise,
> you'll have to write drivers not just for supported external
> peripherals, but also for entire motherboards.

It's definitely no problem that different silicon vendors choose
different timer IP in their chipset. I am talking about SoC timers.
But from architecture perspective, given RISC-V priv spec already
defines a real-time timer, the driver can be generic as long as the
spec is clearly defined. S-mode OS can load such timer driver and talk
to the architecture defined timer directly. There is no need to go
through SBI. This is the same as x86 TSC, or PowerPC decrementer. I do
not think SBI should provide abstraction for these different SoC
timers given there is already an architecture defined timer.

>
> As I mentioned in another message on the topic, a good BIOS (nee a
> good SBI) will serve as a hardware abstraction layer, *just* thick
> enough to support booting into the preferred OS environment, and
> optionally, for operation of the OS as well. Just as CP/M still
> needed a BIOS to be independent of concrete computer design issues, so
> too would Linux, BSD, Plan 9, et. al. need similar services. Well,
> OK, not *need*; but I would imagine the overhead of maintaining so
> many different configurations of the kernel would certainly make it
> strongly preferable, especially if SBI performance were on par with
> calling any normal C function, or can be made to be called relatively
> infrequently.
>
> > is absolutely necessary? Eventually we will move all platform-specific
> > stuff into the SBI implementation and that makes SBI another mini-OS
>
> That is the crux right here, I think -- there are three groups of
> people vying for influence on how system firmware should be handled:
> (1) the pro-UEFI(-like) folks who thinks that system firmware should
> include the kitchen sink, (2) the pro-BIOS folks (like myself) which
> thinks it should be a thin abstraction over the available hardware,
> and then (3) there's the anti-firmware folks who thinks that the
> firmware should be opaque to any operating system you load, with
> complete and total hand-off to the kernel once the bootloader has
> completed its job.
>

I did not see a lot of voices from group 1. Maybe I missed them. I am
OK with either 2 or 3. But my point is we should only allow whatever
is absolutely necessary to go into SBI.

> > the M-mode firmware. There is needs for real-time performance and
> > determinism.
>
> There's also a real need for OS authors to have some manner of
> reliable interface to a wider variety of system implementations during
> the bootstrap process, and if I had my preference, during OS operation
> as well.
>

I understand this.

> Put bluntly, I'd much rather have a kernel which is slower but runs on
> wider hardware than hyper-optimized kernels that can only run on one
> motherboard's processor and/or chipset. Computers are fast enough for
> me to not have to care about break-neck performance anymore. The only

Fast enough computers cannot be the excuse for putting everything into SBI.

> exception I'd make for that is in a data center context, where now I'm
> managing entire fleets of like-configured systems. Here, it makes
> sense to maintain local kernel images, software repositories, et. al.
> Ditto as well for deep-embedded environments.
>

Yes, my concern was for deep-embedded environments. I don't think
RISC-V was invented only for mainstream desktop or high-end servers.

> But, for laptop use? Phone use? Tablet use? TV set-top box use?
> The idea of maintaining a crap-ton of kernels and chipset/motherboard
> drivers just does not appeal to me at all.
>
> > > * Is there actually consensus amongst developers of S-mode code
> > > (kernels, bootloaders etc) that the SBI is a desirable abstraction?
> >
> > I don't think so :)
>
> Linux has a HAL. Windows uses a HAL. Most modern kernels rely upon a
> HAL-like "thing" at some level. That's an SBI but by a different
> name.
>
> It's clear that it's a valuable abstraction. The question is, I
> *think*, whether having a common interface across OSes is valuable.
>

Regards,
Bin

Bin Meng

unread,
Nov 12, 2018, 8:22:09 AM11/12/18
to mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Nick,
Yes, it was unfortunately designed this way. Sigh. My question was why
does hardware expose such limitation that accessing timer/IPI should
only be done in M-mode? It could have only be one instruction of
CSR/MMIO read to get a timer counter, but now it has to call into SBI
and execute maybe one hundred instructions. This affects real-time
performance a lot.
If we could have done SBI this way, I would be much less concerned.

> of ecalls. Apart from the idea, no one has come up so far with a
> document on how this can be implemented, I guess the way Linux
> does it should be a start.
>
> Hope I helped !
>

Yes, the discussion threads you shared are really helpful. Thanks!

Regards,
Bin

Samuel Falvo II

unread,
Nov 12, 2018, 1:20:31 PM11/12/18
to Bin Meng, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 4:36 AM Bin Meng <bmen...@gmail.com> wrote:
> coreboot/U-Boot can meet such requirement very well :)

Of course; but, I think we're both starting to digress.

Logically, if you agree that these services are provided solely by
coreboot, u-boot, et. al., then it follows these products ultimately
define the SBI. This entire discussion, then, all becomes irrelevant,
at least insofar as these mailing lists are concerned. New feature
development for the SBI should then be done on the mailing lists for
coreboot, u-boot, et. al. All we would be doing here is
bike-shedding.

By contradiction, then, that we are discussing the SBI here suggests
we want something more from the SBI. Something coreboot and u-boot
and whatever else there is cannot provide on their own.

> But from architecture perspective, given RISC-V priv spec already
> defines a real-time timer, the driver can be generic as long as the
> spec is clearly defined. S-mode OS can load such timer driver and talk

It genuinely pains me that I cannot use something as simple as a timer
as an example of a class of hardware which *could* (but not
necessarily *will*) differ from platform to platform without having to
justify my argument. It was clearly a hypothetical example, intended
to highlight why someone *might* want to implement or use an SBI call
in the context of a hardware abstraction layer. I depend upon the
experiences of the reader to be able to extrapolate from this simple
example into more complex scenarios.

But, let's roll with it. While mtime/mtimecmp are adequately
specified for the purposes of getting something working on one
platform, I don't think it's adequately specified to be considered
*portable*. Open questions, from a software portability concern,
include but I'm sure aren't limited to the following:

- What is the base address of mtime and mtimecmp? By your own
admission, the hardware interface defined so far is too simple to be
considered for SBI calls. So, where do I get the base address from?
Right now, the only official answer is the device tree. But, having
to include a multi-kilobyte parser for this structure (and it will be
multi-kilobyte based on my experience just implementing *simple*
procedures in my own firmware images) is, to be blunt, thoroughly
unpalatable. It's noise for a kernel the size of Linux, but it'd be
rather significant overhead for something the size of a small
microkernel.

- Is mtime and mtimecmp obligated to be located in a page which can be
assigned to run in supervisor mode? We know a priori from the
description of PMAs and PMPs that each RISC-V processor has to expose
a flag to address decoding logic as to whether or not it's currently
running in M-mode -- it's the only way PMPs can do what they do.
Ergo, the decision of whether or not mtime and mtimecmp are S-mode
accessible at all is, ultimately, up to the SoC or system integrator.
Knowing that it *is* allowed by the standard to delegate mtimecmp
interrupts into S-mode, is it still OK for a platform to restrict
mtime and mtimecmp to M-mode only regions of memory? It is not
specified, and therefore, conservatively, we must assume the answer to
be yes! If so, then despite how simple mtime/mtimecmp logic is to use
from a purely software interface, you would have literally *no*
*choice* but to use an SBI to access mtime and/or mtimecmp. Nowhere
in the specification that I've read (maybe I missed something?)
compels an implementation to expose mtime and mtimecmp in a manner
that allows it to be accessed via S-mode.

- What frequency does mtime count at? (Again, I feel I should not
have to include an entire DT parser to find this information out.)

- What even is the *purpose* of mtime/mtimecmp? Or, put another way,
why *must* it be M-mode only in the first place? Here's something you
can't encode in a DT structure. If mtime/mtimecmp are to be used for
time-slicing, why use an absolute, one-shot timer? Why not use
relative, periodic timers instead? That at least saves the overhead
of computing the reload value of mtimecmp. I strongly suspect this
timer is *not* intended for this use-case, but rather intended more
for the kinds of software that would use alarm(2) instead. But, even
this raises some issues, because processes generally tend to have
affinities to specific harts, while mtimecmp interrupts might be
routed to any operating hart. That means an alarm could also involve
the overhead of an IPI. It's more efficient if each hart maintains
its own mtime/mtimecmp pair under these circumstances, which once
again argues strongly for moving mtimecmp back into CSR space. If it
is intended to be used to trigger alarms, what kinds of alarms would
an M-mode binary blob need? M-Mode is supposed to be used a la
PALcode on the Alpha; it should be the highest priority code, and per
any common sense kernel design rules of thumb, correspondingly the
*least likely to execute*. So, if there's no clear use-case, maybe
mtime/mtimecmp should be demoted to stime/stimecmp instead.

> I did not see a lot of voices from group 1.

I'm not skilled enough to search Google Groups and provide links, but
there have been extensive discussions on the value proposition of UEFI
versus other system firmware implementations on the RISC-V mailing
lists in months past. These discussions are not recent, but they have
happened.

> OK with either 2 or 3. But my point is we should only allow whatever
> is absolutely necessary to go into SBI.

We are not in disagreement here.

> Fast enough computers cannot be the excuse for putting everything into SBI.

Please refer to me where I advocated putting everything into an SBI,
and I'll reply with quotations of my messages advocating strongly
*against* doing this. Please do not misrepresent my intentions, and
be sure to read my text in the proper and intended context. If you
have questions about what that context may be, I'd much rather you ask
for clarification than just assume.

Samuel Falvo II

unread,
Nov 12, 2018, 1:22:39 PM11/12/18
to Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 5:22 AM Bin Meng <bmen...@gmail.com> wrote:
> Yes, it was unfortunately designed this way. Sigh. My question was why
> does hardware expose such limitation that accessing timer/IPI should
> only be done in M-mode? It could have only be one instruction of

I wish I'd seen this message earlier before I sent my last response.
This gives me increased sense that we are, in fact, more in
"frustrated agreement" than we are in opposition to each other. :)

Samuel Falvo II

unread,
Nov 12, 2018, 1:31:33 PM11/12/18
to Bin Meng, Alex Bradbury, sw-...@groups.riscv.org
Replying to myself for clarification. I wrote something in haste
which, after I just finished issuing a complaint on context
sensitivity of my text, reads entirely wrong in context. LOL!
Clarification below.

On Mon, Nov 12, 2018 at 10:23 AM Samuel Falvo II <sam....@gmail.com> wrote:
> I'm not skilled enough to search Google Groups and provide links, but

What I meant to say here is that I'm not skilled enough to issue a
query to Google Groups that yields a high enough signal to noise ratio
for me to copy links out of. I can of course issue queries in the
search box, but my experience with Google Groups is that it yields a
terribly low signal to noise ratio. Sorry for the amusing confusion.
:)

Bruce Hoult

unread,
Nov 12, 2018, 10:34:44 PM11/12/18
to Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 5:21 AM, Bin Meng <bmen...@gmail.com> wrote:
Yes, it was unfortunately designed this way. Sigh. My question was why
does hardware expose such limitation that accessing timer/IPI should
only be done in M-mode? It could have only be one instruction of
CSR/MMIO read to get a timer counter, but now it has to call into SBI
and execute maybe one hundred instructions. This affects real-time
performance a lot.

If it's 100 instructions then someone is doing something very very wrong. Try 10.
 

Samuel Falvo II

unread,
Nov 12, 2018, 10:46:15 PM11/12/18
to Bruce Hoult, Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
Including ecall function ID dispatch logic?



--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Bruce Hoult

unread,
Nov 12, 2018, 11:41:58 PM11/12/18
to Samuel Falvo II, Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
Of course.

Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:

lui t0,#ecallTableBase #or auipc
slli t1,a6,2 #3 for 64 bit
add t0,t0,t1
jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
# called function does the mret

That's five instructions.

You could do something like a "li t0,#nnn; blt a6,t0,heavyweight" to distinguish these simple calls from ones that actually need to save and restore registers. That makes it seven instructions.

I'd submit that trusted SBI calls should use the same ABI as regular function calls (i.e. a0-a7 and t0-t6 can come back trashed) rather than the Linux "all registers except designated return values come back unchanged" system call ABI.


On Mon, Nov 12, 2018 at 7:46 PM, Samuel Falvo II <sam....@gmail.com> wrote:
Including ecall function ID dispatch logic?



On Mon, Nov 12, 2018, 7:34 PM Bruce Hoult <bruce...@sifive.com wrote:
On Mon, Nov 12, 2018 at 5:21 AM, Bin Meng <bmen...@gmail.com> wrote:
Yes, it was unfortunately designed this way. Sigh. My question was why
does hardware expose such limitation that accessing timer/IPI should
only be done in M-mode? It could have only be one instruction of
CSR/MMIO read to get a timer counter, but now it has to call into SBI
and execute maybe one hundred instructions. This affects real-time
performance a lot.

If it's 100 instructions then someone is doing something very very wrong. Try 10.
 

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Samuel Falvo II

unread,
Nov 12, 2018, 11:59:08 PM11/12/18
to Bruce Hoult, Bin Meng, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 8:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
> lui t0,#ecallTableBase #or auipc
> slli t1,a6,2 #3 for 64 bit
> add t0,t0,t1
> jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
> # called function does the mret
>
> That's five instructions.

DUHH, of course. Not sure why I didn't see that earlier. For some
reason, I had it in my mind that you had system call overheads similar
to, e.g., Linux's system call dispatcher. Sorry for the brain-lapse.

Bin Meng

unread,
Nov 13, 2018, 12:12:16 AM11/13/18
to bruce...@sifive.com, Samuel Falvo II, mi...@ics.forth.gr, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Bruce,

On Tue, Nov 13, 2018 at 12:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
>
> Of course.
>
> Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:
>
> lui t0,#ecallTableBase #or auipc

You need save t0 before using it, no?

> slli t1,a6,2 #3 for 64 bit

and t1, a6?

> add t0,t0,t1
> jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
> # called function does the mret
>
> That's five instructions.
>

Yes, for this particular part. But the called function (illegal
instruction handler in this case) to implement reading the mtimer
counter needs another bunch of instructions, including instruction
decoding, etc. Having said that, there is room for optimization if we
handcraft the handler in assembly codes carefully.

> You could do something like a "li t0,#nnn; blt a6,t0,heavyweight" to distinguish these simple calls from ones that actually need to save and restore registers. That makes it seven instructions.

Unless we write all ecall trap handlers in assembly codes, it's still
needed to do the blind register saving/restoring.

>
> I'd submit that trusted SBI calls should use the same ABI as regular function calls (i.e. a0-a7 and t0-t6 can come back trashed) rather than the Linux "all registers except designated return values come back unchanged" system call ABI.
>
>
> On Mon, Nov 12, 2018 at 7:46 PM, Samuel Falvo II <sam....@gmail.com> wrote:
>>
>> Including ecall function ID dispatch logic?
>>

Regards,
Bin

Bin Meng

unread,
Nov 13, 2018, 12:37:15 AM11/13/18
to Samuel Falvo II, a...@lowrisc.org, sw-...@groups.riscv.org
Hi Samuel,
I was against the reason for making timer stuff abstracted by SBI is
due to different IPs. As you mentioned in the follow-up thread, you
and I are actually in "frustrated agreement". So it looks I may have
misunderstood your opinion.
Thanks a lot for the sum-up about current the RISC-V timer facility.
There are tremendously valuable questions and suggestions. I hope
someone from the RISC-V hardware design team can take a deeper look.

> > I did not see a lot of voices from group 1.
>
> I'm not skilled enough to search Google Groups and provide links, but
> there have been extensive discussions on the value proposition of UEFI
> versus other system firmware implementations on the RISC-V mailing
> lists in months past. These discussions are not recent, but they have
> happened.
>
> > OK with either 2 or 3. But my point is we should only allow whatever
> > is absolutely necessary to go into SBI.
>
> We are not in disagreement here.
>

Glad we are :)

> > Fast enough computers cannot be the excuse for putting everything into SBI.
>
> Please refer to me where I advocated putting everything into an SBI,
> and I'll reply with quotations of my messages advocating strongly
> *against* doing this. Please do not misrepresent my intentions, and
> be sure to read my text in the proper and intended context. If you
> have questions about what that context may be, I'd much rather you ask
> for clarification than just assume.
>

Sorry I misinterpret this. What I was saying was I am not in favor of
a slower kernel that was built on top of an SBI that abstracted
something not absolutely necessary. There are needs for real-time
performance and determinism and I hope such requirement does not get
ignored.

Regards,
Bin

Michael Clark

unread,
Nov 13, 2018, 1:05:39 AM11/13/18
to Luke Kenneth Casson Leighton, Bruce Hoult, Alex Bradbury, sw-...@groups.riscv.org
Luke.

Everyone has a different opinion and perspective. Almost everyone has been communicating politely. Trying to use other folks opinions or “ideas” as a rationale to act the way you do is no excuse. You make things personal. I’m also fond of the IETF model but it doesn’t work if the list is dominated by expletives, personal attacks, talk of mediation and conflict resolution. That is unnecessary for cooperative development.

Your ideas on the other hand stand on their own merit. This talk of “conflict resolution and mediation”, however, is completely unnecessary. Folk are not deciding whether or not they like ideas based on who proposed them.

No swearing from now on. PLEASE.

Michael

> On 11/11/2018, at 10:46 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
>
> ---
> crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68
>
>> On Sun, Nov 11, 2018 at 8:00 AM Bruce Hoult <bruce...@sifive.com> wrote:
>>
>>> On Sat, Nov 10, 2018 at 11:46 PM, Luke Kenneth Casson Leighton <lk...@lkcl.net> wrote:
>>>
>>> so the question to ask is not "how* come you are civilised and
>>> capable of staying within the bounds of technical discussion on other
>>> forums: the real question is, what's different about *these* forums?
>>
>>
>> In the last few hours I've seen you being adversarial with not only
>> Alex Bradbury,
>
> if you look carefully, alex was trying to be reasonable, and play a
> mediatorial role... except if you look carefully at the advice given
> on crhnq, a mediator needs to respect *both* sides, and relay (reflect
> back) the position of *both* parties.
>
> regrettably, he didn't do that, and unfortunately, all it did was
> make it look he wasn't listening to what i'm saying. as i was
> writing, i was thinking, "damn, damn damn, i really like alex, how the
> hell can i make this look any better, so it doesn't reflect badly on
> him?" and, sadly, as he (you, hello, yes, sorry to be talking _about_
> you, alex) hadn't summarised and acknowledged both perspectives as a
> mediator normally would, there really wasn't a way to do that.
>
>> but also Nick Kossifidis
>
> if you look carefully at what nick's writing, he made good points
> from the field in which he has experience.. unfortunately he laced it
> with hyperbolic adversarial questions that were very carefully
> designed to ridicule rather than critique not only the proposal that i
> made, they were also very very carefully crafted to ridicule me, as a
> person, as well.
>
> you'll notice that he dodged the positive aspects that were supported
> by other people's constructive input, and focussed on generating as
> much negativity as he could, through the use of what can only be
> termed "hysteria" questions, hoping to assign them to me (without my
> consent) and use them rhetorically for the purposes of ridicule.
>
> the initial (thinly-veiled) adversarial attacks that he launched were
> sufficiently over the top that someone else had to step in, and hint
> that he was going a little too far. nick made a conscious choice to
> ignore those hints.
>
> the question here i have to ask, is, can you see that that's what
> nick did, and, if you can, why do you feel that it's acceptable
> behaviour on a technical forum?
>
>> (who you for unknown reason persist in addressing as "Mick"),
>
> sorry, nick: small high-resolution screen here, and i run the
> brightness way down and the room lights off, most of the time.
>
>> and Tommy Murphy.
>
> tommy, bless him, i know he's trying to help. you may be confusing
> the pain that i am experiencing through having to deal with the
> ongoing bullying and expressing that, and thus interpreting my
> responses to him as "this must be an attack on tommy". categorically
> and absolutely it is not.
>
>> And adopting a derisory tone with others such as Ron Minnich.
>
> what?? not at all! ron makes a lot of sense, and i've said so quite
> clearly!! what on *earth* gave you the impression that i've been
> "derisory" with ron?? i've made it quite clear that the points he's
> made about being able to operate fully in M-Mode are good ones!
>
>> There's a common thread here.
>
> there is, indeed. the challenge is, then, if the opportunity to
> attack arises, if someone appears to be vulnerable, in pain, or
> appears to not be knowledgeable, is for people to step up to the plate
> and *not* react in a derisory fashion, *not* attack or denigrate them,
> make them feel so unwelcome and so despised that it actually
> deteriorates their physical health.
>
> you know i speak my mind, and i made a decision 20 years ago to speak
> truth, even when it's uncomfortable truth. that's what people face
> here, and i know they don't like it.
>
> i told you: ethical's a bitch. truth, awareness, love and creativity
> aren't "nice", they're damn uncomfortable, and dealing with this is so
> hurtful it's *literally* making me physically ill.
>
> yet... i have a goal to achieve, the target's been set, to create a
> completely libre, open and freely-available 3D-capable processor, and
> to *develop* it in a truly libre and open fashion, and i am not going
> to go away until that goal is achieved.
>
> so, we have at least two more years of this. that's the reality.
> so, we can use these forums for the benefit of RISC-V, or not. which
> is it going to be?
>
> we can work together, to improve RISC-V and make a conscious decision
> to make the forums a welcoming place... or we can turn it into a
> hate-filled forum where students, corporate employees and researchers
> treat it - and RISC-V - as an absolute "No Go Zone", and psychology
> researchers use the public archives in case-studies over the next
> couple of decades, on how open communities go into denial when faced
> with challenges.
>
> that's what we're looking at, here. it's up to everyone to make a
> conscious decision which direction they want to go.
>
>> These people don't talk to each other in the manner in which you reply to them.
>
> people don't talk to me on comp.arch or libre-riscv-dev in the manner
> in which they reply to me on here. why?
>
> l.
>
> --
> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> To post to this group, send email to sw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAPweEDwKwf4iF%2BE5sqV6wdB4%3DT%3D1rR8cAKH-Kb9g5cYmXtVtPg%40mail.gmail.com.

Bruce Hoult

unread,
Nov 13, 2018, 1:11:34 AM11/13/18
to Bin Meng, Samuel Falvo II, mi...@ics.forth.gr, Alex Bradbury, sw-...@groups.riscv.org
On Mon, Nov 12, 2018 at 9:12 PM, Bin Meng <bmen...@gmail.com> wrote:
Hi Bruce,

On Tue, Nov 13, 2018 at 12:41 PM Bruce Hoult <bruce...@sifive.com> wrote:
>
> Of course.
>
> Once you're in code that knows you're in an ECALL (via hardware vectoring or otherwise) that's just something like:
>
> lui t0,#ecallTableBase #or auipc

You need save t0 before using it, no?

Not if, as I specified below, SBI ecalls have the same/similar ABI as they would if they were in a C library. 
 

> slli t1,a6,2 #3 for 64 bit

and t1, a6?

*Certainly* not a6, as that has the index of the desired SBI call, and we don't modify it.
 

> add t0,t0,t1
> jr,nnn(t0) #lo bits of ecallTableBase, if not 4k aligned
> # called function does the mret
>
> That's five instructions.
>

Yes, for this particular part. But the called function (illegal
instruction handler in this case) to implement reading the mtimer
counter needs another bunch of instructions, including instruction
decoding, etc.

I'm assuming here that you're making an explicit SBI call to get mtime, not handling an illegal instruction trap. Illegal instruction would definitely be a bit more heavy-weight with both decoding overheads and the necessity to save and restore everything it touches.

> You could do something like a "li t0,#nnn; blt a6,t0,heavyweight" to distinguish these simple calls from ones that actually need to save and restore registers. That makes it seven instructions.

Unless we write all ecall trap handlers in assembly codes, it's still
needed to do the blind register saving/restoring.

I would definitely write the smallest and simplest ones, that should be as fast as possible, in assembly language. Like getting timers, for example. And I'd make sure a 64 bit counter was returned on 32 bit systems (already made atomic, and possibly virtualised with an offset subtracted as well) so that only one SBI call was needed, not two.

Michael Clark

unread,
Nov 13, 2018, 3:19:30 AM11/13/18
to Richard W.M. Jones, h...@lst.de, Alex Bradbury, sw-...@groups.riscv.org


>> On 12/11/2018, at 12:31 AM, Richard W.M. Jones <rjo...@redhat.com> wrote:
>>
>> On Sat, Nov 10, 2018 at 09:25:29PM +0000, Alex Bradbury wrote:
>> Based on a recent software working group call, it seems there is a
>> desire for the SBI to ensure that a single kernel works across all
>> RISC-V implementations,
>
> Not sure what the "software working group call" is, but if this is in
> any way related to what I said in my talk in Barcelona then I'd like
> to clarify that I only meant for server platforms. Embedded folk can
> and most likely will go their own way.
>
> BTW the way you described the full scope of the (potential) SBI made
> it sound like UEFI.

In some ways UEFI (or a subset of) is a little clearer in scope than SBI a). because after one calls ExitBootServices() one has complete control over the machine with respect to UEFI. SMI issues aside as they are independent to UEFI. b) the UEFI API is relatively well thought through although perhaps a little heavyweight e.g. the use of GUIDs to get structures with function pointers is very Microsoft centric. Lots of optional APIs too e.g. do we really need to boot from SCSI tape?

I believe we need to separate these 3 concerns:

1). boot time services. arm’s EBBR has a minimal UEFI subset which IMHO is a good starting point (as an option)

2) runtime HAL e.g. interrupt controller virtualization, power management, etc i.e. platform device specification versus SBI driver? interfaces.

3). emulation and virtualization fixups or core support firmware e.g. trap-and-emulate in RISC-V code vs micro-code - features such as misaligned fetches and FPU emulation. TLB flushes, as Christoph mentioned.

For 2). part of the problem is that with the current Privileged ISA and SiFive’s CLINT, M-mode is necessarily involved and must stay resident to support timers and software interrupts. These are runtime services that depend on M-mode. Until we’ve fully virtualized interrupt configuration for Supervisor’s, we temporarily need to keep this interface, however as I mentioned there has been work on this. Question is whether it will go into silicon and formalised in the specs.

For 3). The kernel could adopt trap-and-emulate for misaligned loads and stores and FPU emulation like it does for other platforms and that removes the need for any stay resident firmware here. We (we as in probably someone other than me who has done a lot more work on RISC-V) have already devolved page table dirty handling to the kernel. For whoever did that, they got it for free as it’s part of Linux kernel. FPU emulation and misaligned loads and stores emulation could also move to the kernel(s). These do not need M modes. They just need illegal instruction trap: Also; the kernel should be compiled with -mstrict-align anyhow (it is?). Just because we can do something (slow misaligned accesses) doesn’t mean we should, but alas it’s in the spec.

UEFI does not provide runtime services nor does its scope cover virtualization, HALs or Secure Worlds (or I am mistaken). Of course measured boot and boot integrity services are later used by things like Windows Virtual Secure Mode to create a tier above the NT kernel that the kernel does not have access to (like arm’s Secure World) using VMX root (Extended Supervisor). RISC-V has Keystone here as well as several other solutions that folk are developing in this area that leverage PMP.

My take on this is that we should be focusing on standardising hardware interfaces or gaps in the Privileged ISA for the functions that are provided by SBI. It evolved because the hardware interfaces were still in flux. I believe they still are in flux (albeit now a relatively stable flux) if one does some analysis on colocating a Hypervisor with the current PLIC model one sees the gaps as we give M-mode interrupt configuration access to the Supervisor. While the Base ISA is frozen, the Privileged ISA is not and we can move forward in a step-wise manner keeping support for the “RISC-V legacy” until things have fully evolved, removing things with an adequate deprecation period. From my perspective, my only criticism of the Unprivileged ISA now is that FENCE.I should probably be moved to the Privileged ISA (or maybe it has). This is likely to be a harmless change due to the current User ABI (nop or trap is debatable). The Privileged ISA should be considered a still evolving target. I also think Andrew has been busy building cores with more ILP from reading SiFive announcements.

Trying to be as concise as possible while describing current issues that would require SBI interfaces either because they break an M-mode S-mode privilege boundary (like the initial SiFive PLIC revision) or alternatively require development on the hardware interfaces that SBI abstracts such that S-mode is fully autonomous. I really don’t think we should move PLIC behind SBI interfaces, instead the hardware should evolve such that an M-mode HAL is not required.

The latter being the direction I think most folk prefer from the comments on the list. i.e. the direction where folk have a standard machine, where standard boot can be layered on top.

SiFive have made some progress with their CLIC proposal whereby aspects from the model could be folded into the PLIC to give S-mode full autonomy for IPIs and real-time control over S-mode interrupts. Maybe SiFive’s new S-series can be configured with an S-mode CLIC and CLINT backwards compatibility and an updated PLIC memory layout separating M mode and S mode. I hope this made it in there...

Forgive me for speculating about SiFive hardware as I do not have all of the details, as there are a huge number of moving parts, of which just a few have been modelled in QEMU, but a fully autonomous S-mode is something that would be ideal. i.e. a situation where SBI is no longer required at runtime for Linux, and where real-time apps can coexist with the Supervisor with direct control over interrupts. There certainly is a model whereby this is feasible.

Michael

(Caveats regarding speculation with respect to details of current or future SiFive products ; this is my own speech, etc, etc)

Bin Meng

unread,
Nov 13, 2018, 4:27:20 AM11/13/18