RISC-V U-Boot for HiFive Unleashed Board is available on GitHub

[Padmarao Begari]

RISC-V U-Boot(64-bit) for HiFive Unleashed Board is available on GitHub

                    

https://github.com/Microsemi-SoC-IP/HiFive_U-Boot

This repository contains the RISC-V U-Boot created by Microsemi a Microchip company for RISC-V core on HiFive Unleashed 540 device. The HiFive Unleashed Board is used for HiFive U-Boot development

Regards

Padmarao

[Christoph Hellwig]

Can you explain a bit more where U-Boot fits into the picture? Is it
started as an S-mode payload, or have you ported M-mode code to U-boot?

[atish patra]

I think the uboot runs in M mode and acts as a FSBL in this case.

The boot flow still require bbl image (bbl+vmlinux) to boot Linux.

At least. that's what I understood from the makefile changes. Padmarao can correct me if I am wrong.

https://github.com/Microsemi-SoC-IP/HiFive_U-Boot#load-uboot-image-on-sd-card 

sgdisk --clear                                                               \
           --new=1:2048:4095   --change-name=1:uboot      --typecode=1:$(FSBL)   \
           --new=2:4096:69631  --change-name=2:bootloader --typecode=2:$(BBL)   \
           --new=3:264192:     --change-name=3:root       --typecode=3:$(LINUX) \

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/20181011132455.GA7242%40lst.de.

--

Regards,

Atish

[Padmarao Begari]

The SiFive FSBL is replaced with U-Boot and stores in SD card and runs in M-mode at present.

The u-boot requires bbl image (bbl+vmlinux) to boot Linux at present.

Our next step is to change M-mode to S-mode in u-boot and removing bbl from  Linux image (bbl+vmlinux). 

Regards

Padmarao

[atish patra]

Hi Padmarao,
Thanks for the confirmation. 

On Fri, Oct 12, 2018 at 12:35 AM Padmarao Begari <padma...@gmail.com> wrote:

The SiFive FSBL is replaced with U-Boot and stores in SD card and runs in M-mode at present.

The u-boot requires bbl image (bbl+vmlinux) to boot Linux at present.

Our next step is to change M-mode to S-mode in u-boot and removing bbl from  Linux image (bbl+vmlinux). 

Awesome. Does that mean you are planning to implement SBI as well in uboot or bbl continue to exist in M mode ?
Can you please explain the boot flow design if u-boot runs in S-mode ?

Regards,
Atish

 

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/dfaca48a-b9b5-4408-94a2-c26aa8523874%40groups.riscv.org.

--

Regards,

Atish

[Anup Patel]

Yes, thats better.

U-boot should run in S-mode only so that we can later on use same
U-boot as bootloader for Guest/VM.

In fact, it should also be possible run U-boot on QEMU Virt machine
using BBL (i.e. using bbl+uboot).

Regards,
Anup

> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/dfaca48a-b9b5-4408-94a2-c26aa8523874%40groups.riscv.org.

[Anup Patel]

On Fri, Oct 12, 2018 at 1:18 PM atish patra <atis...@gmail.com> wrote:
>
> Hi Padmarao,
> Thanks for the confirmation.
>
> On Fri, Oct 12, 2018 at 12:35 AM Padmarao Begari <padma...@gmail.com> wrote:
>>
>> The SiFive FSBL is replaced with U-Boot and stores in SD card and runs in M-mode at present.
>> The u-boot requires bbl image (bbl+vmlinux) to boot Linux at present.
>>
>> Our next step is to change M-mode to S-mode in u-boot and removing bbl from Linux image (bbl+vmlinux).
>>
>
> Awesome. Does that mean you are planning to implement SBI as well in uboot or bbl continue to exist in M mode ?
> Can you please explain the boot flow design if u-boot runs in S-mode ?

Please no M-mode code in U-boot or any general purpose bootloader.

Instead of replacing SiFive FSBL with U-Boot, I suggest to boot
bbl+uboot.bin from SiFive FSBL

U-boot can use SBI services just like any other S-mode software.

U-boot should boot RAW Linux image (instead of vmlinux) using bootm.

The role of last stage bootloader (such as U-boot, UEFI, Grub, etc) is
to support booting protocol for
variety of OSes (Linux, BSD, Vxworks, etc). There should be no
restriction on what last stage
bootloader will be except that it should run in S-mode. For any M-mode
facilities, it should use
SBI calls just like Linux.

Regards,
Anup

[Christoph Hellwig]

On Fri, Oct 12, 2018 at 01:35:02PM +0530, Anup Patel wrote:
> > Awesome. Does that mean you are planning to implement SBI as well in uboot or bbl continue to exist in M mode ?
> > Can you please explain the boot flow design if u-boot runs in S-mode ?
>
> Please no M-mode code in U-boot or any general purpose bootloader.

Indeed, that is what I feared :(

> Instead of replacing SiFive FSBL with U-Boot, I suggest to boot
> bbl+uboot.bin from SiFive FSBL

That would be more useful indeed.

[ron minnich]

I'm surprised to see this idea but in any event coreboot replaces bbl and has m mode code and I don't expect that to change. I'm done with trusting vendor supplied fsbl no matter what the source.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/20181012080840.GA27426%40lst.de.

[Paul Walmsley]

Many embedded users will have no desire to run any external firmware
beyond an open-source bootloader and the kernel.

No one should be discouraging that use-case, as it reflects the needs of
a significant chunk of the user base.

That's worked fine on ARM for many years, and it's the same approach
that folks should keep in mind for RISC-V, even if people might not
understand why others want that.


- Paul

[Paul Walmsley]

Yes - for RISC-V U-boot, it's best to follow the example set by the ARM64 U-boot folks and support booting in any mode:

https://github.com/theopolis/u-boot/blob/master/doc/README.arm64

- Paul

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAP6exYKr_uHoX1FvKSTnPB8gPkYzYY0LOKB%2BbvTvYiyV%3DcfgfA%40mail.gmail.com.

[kr...@berkeley.edu]

Ron,

Please don't confuse the desire to separate M-mode functionality from
S-mode functionality for the purposes of making code more portable,
with a desire by vendors to lock down M-mode.

Having a clean separation of concerns between abstraction layers is
important for engineering reasons, and is completely orthogonal to
whether each abstraction layer is open or not.

We want to enable platforms that don't trust the OS or even
bootloaders, as much as we want to allow platforms that don't trust
vendor firmware. We also want to make it easier to build emulations
and flexible virtualization systems.

My comments are not meant to read on any of the proposals below, but
just on the sentiment that somehow specifying this abstraction
interface will prevent open implementations of the abstraction
interface, or prevent different implementations that ignore the
abstraction interface.

Krste

| CAP6exYKr_uHoX1FvKSTnPB8gPkYzYY0LOKB%2BbvTvYiyV%3DcfgfA%40mail.gmail.com.

[ron minnich]

Krste, thanks, I was only reacting to this sentence:

"Please no M-mode code in U-boot or any general purpose bootloader."

U-boot and coreboot live in the same "place" in the boot path, usually called firmware. Some people call them bootloaders, although I do not.

Since the first port in 2014, coreboot has always started in M-mode, and has always provided an SBI implementation, and will continue to do so. 

Further, we're looking at the following model for things coreboot loads (payloads)

- payload can be loaded in M mode and if it is, is allowed to replace existing M-mode code with its own (this includes Linux)

  coreboot will not write protect or lock away the M mode code area

- payload can be loaded in S mode and if it is, we assume it uses coreboot's M-Mode code. 

  coreboot might write protect the M mode code area but will never revoke execute or write permissions.

  revoking read access to M-mode is just another instance of security through obscurity and it worries me. But vendors do this

  type of thing all the time (See: SMM)

I was a bit concerned by several comments on this thread implying firmware running in M-mode is not desirable. 

I would certainly be worried should a given riscv implementation and mask rom make it impossible for coreboot to start in M-mode.

Thanks

ron

[alist...@gmail.com]

I agree that we shouldn't trust vendor firmware. It is always buggy at best.

The problem that I see with starting coreboot or u-boot (the last stage bootloaders) in M mode is that we then end up with duplicated work between the projects.

This is where a model like Arm Trusted Firmware fits in well. We can see something like this:

 1. CPU boots up and ROM does it's thing

 2. ROM boots into an open source firmware (ATF equivalent) in M mode

    - This will end up being pretty SoC specific

 3. The firmware boots into u-boot or core boot in S mode (or H mode when that exists)

    - Hopefully this can then be as generic as possible

 4. From there we boot the system

The advantage I see with this is that we only need to implement the M mode functionality once and then coreboot, u-boot and anything else can utilise it. It also keeps bigger later stage boot loaders out of M mode which has a mild security benefit of smaller attack surfaces.

The other aim is to allow a single RISC-V image to boot on a variety of different SoCs. The more generic we can make the last stage boot loaders the more achievable this is.

Alistair

[Tommy Murphy]

On Friday, 12 October 2018 22:12:08 UTC+1, alist...@gmail.com wrote:

I agree that we shouldn't trust vendor firmware. It is always buggy at best.

 

But once the source is open/available surely one can see what's going on and address problems/bugs as needed? 

Declaration/disclaimer: I work for Microchip/Microsemi.

[ron minnich]

On Fri, Oct 12, 2018 at 2:40 PM Tommy Murphy <tommy_...@hotmail.com> wrote:

On Friday, 12 October 2018 22:12:08 UTC+1, alist...@gmail.com wrote:

I agree that we shouldn't trust vendor firmware. It is always buggy at best.

 

But once the source is open/available surely one can see what's going on and address problems/bugs as needed? 

40 years of experience shows otherwise. The only thing that works is if everyone always has the ability to compile and build their firmware, and from more than one source.

A very similar argument to the one we use for open source kernels, in fact.

ron 

[ron minnich]

On Fri, Oct 12, 2018 at 2:12 PM <alist...@gmail.com> wrote:

I agree that we shouldn't trust vendor firmware. It is always buggy at best.

The problem that I see with starting coreboot or u-boot (the last stage bootloaders) in M mode is that we then end up with duplicated work between the projects.

Sure. There's lots of duplicated work between kernels (BSD, Linux, ...); compilers (clang, gcc); browsers, shells, editors ...

nobody has ever made a convincing argument that this is a problem. And it's not a problem for firmware.

 

This is where a model like Arm Trusted Firmware fits in well. We can see something like this:

... 

(interested text removed :-)

 

The advantage I see with this is that we only need to implement the M mode functionality once and then coreboot, u-boot and anything else can utilise it. It also keeps bigger later stage boot loaders out of M mode which has a mild security benefit of smaller attack surfaces.

I don't see any advantage. If M-mode is so complex that we can only have one workable implementation, something's gone horribly wrong in RISC-V.

We also learned the value of multiple  firmware implementations back in the linuxbios days, over and over again, particularly on the early opteron: an AMD engineer told us the linuxbios hypertransport enumeration code was better than what AMD had. And we knew this from experience: companies had opteron mainboards that would not boot if socket 0 was empty, and linuxbios had no trouble with that at all.

So, no, software monoculture is not a good idea for firmware, as it is not a good idea anywhere else.

ron 

[alist...@gmail.com]

I'm not saying that we have to have one implementation. I just see a lot of value in separating what runs in M mode and what runs in S mode.

If someone wanted to they can run something that handles both, nothing is stopping them from doing that. There is no enforcement of mono-culture or of vendor firmware.

I just see a lot of value in separating the firmware from the last stage bootloader. As part of this though we would need to come up with a firmware option that can be used for those who want to.

Alistair

 

ron 

[ron minnich]

On Fri, Oct 12, 2018 at 3:43 PM <alist...@gmail.com> wrote:

I just see a lot of value in separating the firmware from the last stage bootloader. As part of this though we would need to come up with a firmware option that can be used for those who want to.

I guess I don't understand the distinction. On chromebooks, for example, coreboot is the firmware and the last stage bootloader. And certainly u-boot on many systems is the firmware and last stage bootloader.  This is an area with lots of variety :-)

But all that said, if we're not closing off firmware and even kernels from having M-mode components they provide, I'm fine.

[Luke Kenneth Casson Leighton]

On Fri, Oct 12, 2018 at 10:12 PM <alist...@gmail.com> wrote:

> The problem that I see with starting coreboot or u-boot (the last stage bootloaders)
> in M mode is that we then end up with duplicated work between the projects.

yes. that has not stopped either project from continuing. it also
hasn't stopped libreboot, or grub, or the system which allows a linux
kernel to actually *be* the bootloader (which runs an initramfs and
calls kexec).

each of these bootloaders has different fans and userbases. they
don't necessarily talk to each other, and that's just how it is.

expecting each independent team to cooperate, particularly when some
of them are hard forks of other bootloader projects, is not realistic.

> This is where a model like Arm Trusted Firmware fits in well.

you mean, ARM _treacherous_ firmware? the "trust" you refer to is
directed at *manufacturers*, providing the specific goal of
*betraying* users. "Trust Us. We Will HELP You To Prevent And
Prohibit Users From Owning Their Devices".

>We can see something like this:
> 1. CPU boots up and ROM does it's thing
> 2. ROM boots into an open source firmware (ATF equivalent) in M mode
> - This will end up being pretty SoC specific

yes. and may not fit into the available memory, which may be
absolutely tiny.

the ROM bootloader, which is best made about as ridiculously dumb and
tiny as can possibly be made, is usually just about capable of loading
from a single (16k) sector of a Micro-SD or eMMC device, or 16k from
an SPI device, or 16k from an ONFI NAND, and so on. that sub-16k
executable the "first stage" bootloader.

16K is the "normal" amount of available SRAM that SoCs are prepared to
allocate for these first-stage bootloaders. that 1st stage basically
does things like "initialise the DDR3 RAM, set up IRQs"... oh and it
happens to initialise the
device-from-which-the-next-phase-is-to-be-loaded.

in u-boot, this first stage bootloader is called the "SPL", and it's
compiled from u-boot with some special #defines that *MASSIVELY* cut
down u-boot to the absolute, absolute bare minimum (surprise: under
16k).

all of this is typically entirely hidden from end-users. you don't
have to know. you don't have to care. people have *no idea* that
their system goes through a four, five or even six stage boot process
to get to an OS. even intel x86 systems.

basically i'm saying that "one size does not fit all", that creating
standards in this area is not only unrealistic but also completely
undesirable it is so heavily application-specific, and that
restricting vendors through standardisation to be not permitted to run
from the absolute lowest possible level (M-Mode) will almost certainly
cause a massive back-lash.

*on the other hand*, if there exists *informal reference source code*
out there, that's absolutely fantastic, as it will cut down massively
on development NREs. example ROM bootloaders, for example, would be
fantastic.

why would it be fantastic? because the entire process is so
proprietary, so secret, that very few people actually know what's
involved. full libre-licensed source code - preferably with
libre-licensed overview documents - would completely change how SoCs
are developed.

l.

[alist...@gmail.com]

On Friday, October 12, 2018 at 4:23:10 PM UTC-7, Luke Kenneth Casson Leighton wrote:

On Fri, Oct 12, 2018 at 10:12 PM <alist...@gmail.com> wrote:

> The problem that I see with starting coreboot or u-boot (the last stage bootloaders)
> in M mode is that we then end up with duplicated work between the projects.

 yes.  that has not stopped either project from continuing.  it also
hasn't stopped libreboot, or grub, or the system which allows a linux
kernel to actually *be* the bootloader (which runs an initramfs and
calls kexec).

 each of these bootloaders has different fans and userbases.  they
don't necessarily talk to each other, and that's just how it is.

 expecting each independent team to cooperate, particularly when some
of them are hard forks of other bootloader projects, is not realistic.

> This is where a model like Arm Trusted Firmware fits in well.

 you mean, ARM _treacherous_ firmware?  the "trust" you refer to is
directed at *manufacturers*, providing the specific goal of
*betraying* users.   "Trust Us.  We Will HELP You To Prevent And
Prohibit Users From Owning Their Devices".

That seems a little harsh. ATF is open source and you can compile it yourself from the source to deploy. You are also free to re-write it if need be.

The risk that I see with just letting everyone go their own way is that we will end up with a similar situation to the ARM world.

That is that every SoC vendor does their own thing. This can result in a less open ecosystem as every vendor writes their own boot code and keeps it closed.

The other problem I see is a fragmented boot process. Where some boards use coreboot and some use u-boot and some use something else. We will then get to a point where every board is different. x86 has this pretty much sorted as I can boot my Linux USB stick on every PC I own. ARM is the exact opposite of this. I would like to see RISC-V not go down this fragmentation route. One way to do this is just to have a "best" way to setup the boot process. Then SoC vendors will hopefully opt to follow it.

A mix-mash of ways seems like a dangerous path to lead to fragmentation and confusion and that is what I'm afraid of.

Alistair

 

l.

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Sat, Oct 13, 2018 at 12:45 AM <alist...@gmail.com> wrote:

> That seems a little harsh. ATF is open source and you can
> compile it yourself from the source to deploy. You are also free to re-write it if need be.

superb (and thank you for the additional, missing, qualifying
information). then by definition it's not treacherous.

>> why would it be fantastic? because the entire process is so
>> proprietary, so secret, that very few people actually know what's
>> involved. full libre-licensed source code - preferably with
>> libre-licensed overview documents - would completely change how SoCs
>> are developed.
>
>
> The risk that I see with just letting everyone go their own way is that we will end up with a similar situation to the ARM world.
>
> That is that every SoC vendor does their own thing. This can
> result in a less open ecosystem as every vendor writes their own
> boot code and keeps it closed.

there's a reason for that, despite in many cases the code that
vendors being used actually being GPLv2-licensed (and thus they
release criminally-infringing products). i'll back-track and post on
that from a different point in the thread.

> The other problem I see is a fragmented boot process.

this is just an unavoidable reflection of the disparate
hardware-level requirements and design constraints that implementors
face.

> x86 has this pretty much sorted as I can boot my
> Linux USB stick on every PC I own.

yah. have you tried booting it when the SPI bootloader chip on the
motherboard is empty, corrupted, or missing? have you tried asking
intel for information about how the first stage bootloader process
actually works? try those things... it's extremely illuminating.

> One way to do this is just to have a "best" way to setup
> the boot process. Then SoC vendors will hopefully opt to follow it.

i specifically avoided the use of the word "best", as that implies
that "other means and methods are worst". however you've pretty much
said exactly what i said, just in different words.

> A mix-mash of ways seems like a dangerous path to lead
> to fragmentation and confusion and that is what I'm afraid of.

like i said: fragmentation, here, is just unavoidable, and efforts to
standardise *WILL* mean with stone-walling. you cannot possibly
expect completely different hardware to have anything in common at the
*boot initialisation* level, and we know from experience that attempts
at abstraction result in bloat.

bloat when you have only 16K to play with... the first thing that
gets cut is: attempts at abstraction.

l.

[ron minnich]

On Fri, Oct 12, 2018 at 4:45 PM <alist...@gmail.com> wrote:

That seems a little harsh. ATF is open source and you can compile it yourself from the source to deploy. You are also free to re-write it if need be.

I agree. ARM has been very good about ATF, in fact it's a model of how it can be done. I'm sorry to see this type of message in what has been an interesting technical discussion. 

 

A mix-mash of ways seems like a dangerous path to lead to fragmentation and confusion and that is what I'm afraid of.

I don't see that multiple implementations necessarily lead to this case, but you make a fair point. 

Anyway, my question has been answered: the things I want will remain possible, so I'm happy.

Thanks everyone.

ron

[alist...@gmail.com]

To a certain extent it's unavoidable, but if RISC-V is ever going to take off we are going to have to at least somewhat deal with it. Booting ARM boards today is such a pain that I suspect 90% of people give up and just take the prebuilt binaries from the SoC vendor. It is a hassle and if we tackle this early on we might be able to reduce this for RISC-V.

At least the first step is are guide lines of what should be implemented and where. Just to have everyone on the same page is very helpful.

For example saying that the first stage boot loader must start in M (obviously) and the last stage in S. That means that you can have one binary that does both, or two binaries separately. Even a very basic and general best practices will help push vendors to do the right thing and make it easier for developers.This can even be done just using current implementations as reference. It just hopefully stops someone in the future doing something super crazy that doesn't work well with anyone else.

I still prefer the layered approach that ATF follows, but any standardisation is good, so we can drop that out.

Alistair

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Fri, Oct 12, 2018 at 6:42 PM <kr...@berkeley.edu> wrote:
>
>
> Ron,
>
> Please don't confuse the desire to separate M-mode functionality from
> S-mode functionality for the purposes of making code more portable,
> with a desire by vendors to lock down M-mode.
>
> Having a clean separation of concerns between abstraction layers is
> important for engineering reasons, and is completely orthogonal to
> whether each abstraction layer is open or not.
>
> We want to enable platforms that don't trust the OS or even
> bootloaders, as much as we want to allow platforms that don't trust
> vendor firmware. We also want to make it easier to build emulations
> and flexible virtualization systems.

these are reasonable statements and a good reflection of a wide range
of goals and requirements from many different sources.

the mention of abstraction however has me twitching, slightly, where
(as mentioned in another reply), the design constraints are so
disparate, and the available memory constraints so insanely low, that
putting in any kind of abstraction is almost certainly likely to fail,
as the first thing that vendors will do is look at the hardware cost
of doing so, and reject it immediately.

chain-loading on the other hand - multi-stage bootloading - not a
problem, and is extremely common:

* http://linux-sunxi.org/EGON#eGON.BRM and you can see a link to the
BROM page as well. following the reverse-engineering that is
described, there, u-boot now supports the "eGON" header format for
allwinner SoCs, such that u-boot's "SPL" first stage bootloader is
recognised.

* http://opensource.rock-chips.com/wiki_U-Boot - here you see "u-boot
SPL" which is that 1st-stage bootloader previously mentioned. the
RK3288 is a bit annoying as the 1st stage bootloader has to be CRC/RC4
encrypted and signed. and that's in the boot ROM so is completely
impossible to circumvent. or fix.

where confusion might arise from what you wrote, krste, is that
normally, u-boot's first-stage SPL (and likely the coreboot
equivalent) *absolutely* has to be run in M-Mode. where else would
the DDR3 timings, the PLL, IRQs and other absolutely essential
peripherals be initialised?

and, that being the case, if u-boot or coreboot were operating in
S-Mode, it would be impossible for those hardware-level
initialisations to be properly carried out. more to the point: what
got u-boot / coreboot *into* S-Mode in the first place??

:)

so, logically, when you refer to a desire to run "bootloaders" in
S-Mode we may *deduce* that you are NOT referring to the ENTIRE boot
process, that instead that you may be referring to the SECOND phase of
u-boot / coreboot. i.e. definitely *not* the (16-or-so-K) first phase
(known as SPL mode for u-boot)... [unless the processor's bootrom has
already done so... which would be hugely risky from a design
perspective]

here - the second phase - it _might_ be reasonable (and possible) to
run from S-Mode. binary sizes are 600k to 2MB (estimated), in main
(DDR) memory. it just depends on whether there was enough spare SRAM
on the processor for a first-stage boot loader with S-Mode
initialisation to *actually fit*.

*that may not actually be the case*... and you'd be looking instead
at triple-stage loading.

really i just want to make people aware, this area is quite involved,
and the number of people in the world who know what's going on *and we
can communicate with because they're not under NDA* is very small: the
members of the u-boot team, plus the members of the coreboot team, and
libreboot as well.

getting their input here - and making absolutely sure that they're
properly compensated for their time (reminder to Foundation Members:
please read section 2.3 of your Member Charter Agreement) - is
absolutely essential.


one other thing that has me concerned: most DDR3/4 vendors FORCE
customers to sign ridiculously-restrictive NDAs, that do not even
permit them to release the source code of the DDR3/4 initialisation
sequence.

the *only* way for fabless semiconductors to release useful product
(i.e. product that will actually boot) is:

(a) release a proprietary first stage bootloader [and then wait for
libre software engineers to reverse-engineer it. in their "spare"
time. unpaid.]

(b) criminally infringe copyright law.

Allwinner and many other Chinese Fabless semiconductor companies
systematically choose option (b). Western corporations, who
understand a little bit more about Copyright Law, typically choose
option (a).

i suspect that even SiFive has had to face this situation. and may
even be under such a restrictive NDA that they can't even talk about
it. anyone at SiFive who can confirm that, would be nice to hear from
you. of course... if we _don't_ hear from you... :)

l.

[Luke Kenneth Casson Leighton]

On Sat, Oct 13, 2018 at 1:16 AM <alist...@gmail.com> wrote:

> To a certain extent it's unavoidable, but if RISC-V is ever going to take off we are going to have to at least somewhat deal with it. Booting ARM boards today is such a pain that I suspect 90% of people give up and just take the prebuilt binaries from the SoC vendor. It is a hassle and if we tackle this early on we might be able to reduce this for RISC-V.

no. really. please. try to understand and accept the implications,
here. if you want RISC-V to be in the embedded world, and in multiple
SoCs, and in desktops, and in high-end servers, you have to accept the
reality of the absolutely massive fundamental hardware differences
between all those different scenarios, where the bootloader and
bootloader choice absolutely necessarily reflects that.

if you start talking about UEFI to low-cost tablet vendors for
example they will just laugh at you. and if you even mention ACPI to
them as a required standard as part of a boot process, they'd be quite
likely to punch your lights out (and i would cheer and congratulate
them).

x86 does not have this "problem" that ARM has because x86 has FAILED
to enter the ARM-dominated SoC market. despite intel bribing vendors,
sellers, OEMs *and customers* to buy their products, they *still could
not get market share*. yes, i've been talking to a business
development manager for an ARM-based fabless semi company, who
regularly was asked, "how much are YOU going to pay US to use your
SoC, because that's what intel does".

all that having been said: i concur that *if* full source code is
available (and available early enough), chances are indeed high that
fabless semi companies will just cookie-cut the available solutions.

unfortunately, the DDR3/4 hard macro vendors (who typically provide
proprietary initialisation source code under insane NDAs) are going to
get in the way, there.

so that's down to the early adopters - SiFive in particular as the
high-profile one - to take responsibility for that, and tell the
DDR3/4 interface vendors in no uncertain terms that proprietary
licensing on the DRAM initialisation is not acceptable.

l.

[Samuel Falvo II]

On Fri, Oct 12, 2018 at 4:23 PM Luke Kenneth Casson Leighton
<lk...@lkcl.net> wrote:
> basically i'm saying that "one size does not fit all", that creating
> standards in this area is not only unrealistic but also completely
> undesirable it is so heavily application-specific, and that
> restricting vendors through standardisation to be not permitted to run
> from the absolute lowest possible level (M-Mode) will almost certainly
> cause a massive back-lash.
>
> *on the other hand*, if there exists *informal reference source code*
> out there, that's absolutely fantastic, as it will cut down massively
> on development NREs. example ROM bootloaders, for example, would be
> fantastic.

I'm going to once again put forth the idea that it was a mistake for
the RISC-V community to settle on an ITU-style standards setting
process. While it may prove advantageous in some circumstances, I
think it sets a bad precedent overall where viable implementations
come second to the desire for an up-front, all-inclusive, frequently
self-inconsistent standard with which everyone must conform. See also
CORBA.

If standards are genuinely desirable, then I **strongly** encourage
the use of IETF-inspired standards track processes. I cannot
emphasize this enough. The process is 100% compatible with the idea
that implementation-specific boot-loaders are not only possible, but
even expected. IETF's processes say you need at least two (more are
preferred) independent, working, and interoperable implementations for
your (bootstrap) proposal to be even *considered* for RFC status.
This is a recognition that standards evolve bottom-up, and aren't
dictated top-down. It really doesn't matter how many RFCs related to
a topic exists (for example, there exists no fewer than four remote
service invokation mechanisms: XDR, NDR, XML-RPC, and SOAP); the
market will eventually sort out which one becomes dominant. And when
one (or a couple) becomes dominant, you get to say things like, "This
system complies with RFC-12345 bootstrapping standards," or "Our
latest model now supports RFC-12345."

--
Samuel A. Falvo II

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Sat, Oct 13, 2018 at 1:16 AM <alist...@gmail.com> wrote:

> For example saying that the first stage boot loader
> must start in M (obviously) and the last stage in S.
> That means that you can have one binary that does both,

sorry, i missed this: no, you can't. really, you can't. the
processor is typically booted up with the DDR3/4 RAM *COMPLETELY
DISABLED*. as in; ***SWITCHED OFF***.

the only way to get it up and running is to turn it on, then do some
scans to find out what DRAM (and what type of DRAM) is there (and in
which banks), then tell both DRAM and Controller to start doing
impedance matching... it's *horrendously* complex, and most first
stage bootloaders do the absolute bare minimum (get the DDR3/4 RAM
running at say 200-300mhz) then leave the rest to second-stage.

the only way to load programs that do that is to have *SRAM* on-board
the processor, which, given that you need a minimum of 16k,
realistically, it's quite a lot to put down in a processor, so you
want to minimise the amount. you could hypothetically put the DRAM
initialisation into ROM however you would be taking a serious risk to
do so, as future DRAM from third party vendors could be incompatible,
ending the life of the processor immediately.

> or two binaries separately.

or, as i mentioned in the message to krste, three. first stage
M-Mode and initialising DRAM (and other critical peripherals). second
stage S-Mode initialisation. *THIRD* stage "main" boot loader, what
most people think of as "u-boot" (or coreboot).

all done as chain-loaders.

there is one _hell_ of a lot going on to get a processor up and running.

l.

[Anup Patel]

I totally agree with Alistair's comments.

The intentions behind having last stage bootloader run only in S-mode are:
1. Re-use it for VS-mode. In other words, it should be possible to
re-use last stage bootloader under a Guest/VM.
2. Platform specific services for CPU hotplug, Reboot, Shutdown, Power
management, Image authentication, etc (i.e. SBI calls) should be
provided by platform specific M-mode firmware

We certainly don't want #2 to be replicated across bootloaders (Uboot,
Coreboot, UEFI, etc). The best people to maintain code related to #2
are SOC vendors themselves. Of course, code related to #2 has to open
source so that people get chance to fix bugs themselves and also have
transparency.

It is also important that we don't want separate bootloader for
Guest/VM (i.e. VS-mode) so last stage bootloader should consider
Guest/VM requirements (i.e. #1 above)

Both above points are not cooked-up ideas, these are based on failures
and learning from ARM world. There is not point of discussion if
people want to re-invent wheel here.

For RISC-V ecosystem to succeed, all good practices from other
architectures should be considered from start itself.

It is better to take the pain now and get booting flow standardised
for RISC-V systems. It will be very difficult in future when we have
lot of RISC-V systems with there own booting flow.

Regards,
Anup

[Luke Kenneth Casson Leighton]

---
crowd-funded eco-conscious hardware: https://www.crowdsupply.com/eoma68

On Sat, Oct 13, 2018 at 7:45 AM Anup Patel <an...@brainfault.org> wrote:

> On Sat, Oct 13, 2018 at 5:46 AM <alist...@gmail.com> wrote:

> > I still prefer the layered approach that ATF follows, but any standardisation is good, so we can drop that out.
> >
> > Alistair
> >
>
> I totally agree with Alistair's comments.
>
> The intentions behind having last stage bootloader run only in S-mode are:
> 1. Re-use it for VS-mode. In other words, it should be possible to
> re-use last stage bootloader under a Guest/VM.

yes, this is a reasonable aim.

just bear in mind that each boot system being developed usually finds
it easier to go through their own chain-loading system and on
subsequently to a full OS. to add support for anything other than
that actually requires speciic effort (which obviously some
corporation has to pay for).

any kind of inter-project cross-collaboration - opportunities for
standardisation - is usually an informal affair that's completely by
accident, from a slow "organic" growth.

for example, UEFI was forced onto teams through x86 BIOSes adopting
it. consequently, YEARS after it became mainstream in the x86 world,
if you now google "u-boot UEFI" it is actually possible to do, and
there is in fact a talk about "marrying u-boot, uEFI and grub2":

https://www.cnx-software.com/2016/08/11/u-boot-now-supports-uefi-on-32-bit-and-64-bit-arm-platforms/

https://www.youtube.com/watch?v=qJAkJ3nmWgM

> 2. Platform specific services for CPU hotplug, Reboot, Shutdown, Power
> management, Image authentication, etc (i.e. SBI calls) should be
> provided by platform specific M-mode firmware
>
> We certainly don't want #2 to be replicated across bootloaders (Uboot,
> Coreboot, UEFI, etc).

apologies for putting it this way, anup: i appreciate that you're not
*actually* saying this: what you've written may very unfortunately be
interpreted as an unpaid "demand" - an order that must be obeyed - to
the u-boot and coreboot developers (and to the grub developers who
have an implementation of UEFI) that they absolutely must follow and
do precisely and exactly what you have said.

the closest that you can get is as follows:

(1) create a reasonable and rational logical argument and, in NEUTRAL
TERMS, propose it (separately) to the u-boot developers and to the
coreboot developers, and the grub developers.

(2) include an offer of compensation and payment for services as part
of the contact with those respective teams

(3) if there is no take-up (no interest) implement it yourself and
*demonstrate* that it's a good idea.

you cannot in *any way* expect sovereign libre project developers -
many of whom are chronically underfunded and absolutely sick and tired
of MULTIPLE BILLION DOLLAR corporations PERSISTENTLY spongeing off of
their expertise and time - to drop everything and "do what you want".

i know you *didn't* make demands of software libre teams: please
however be extremely careful how you put things, ok?

> The best people to maintain code related to #2
> are SOC vendors themselves. Of course, code related to #2 has to open
> source so that people get chance to fix bugs themselves and also have
> transparency.

indeed. and the upstream developers adequately compensated and paid
for their time. this does not "go without saying". it's something
that corporations have to actively take responsibility for. you made
money? PAY the people on whose success that money is critically
dependent!

i'll mention the words "shellshock", "heartbleed" and leave it at that.

> It is also important that we don't want separate bootloader for
> Guest/VM (i.e. VS-mode) so last stage bootloader should consider
> Guest/VM requirements (i.e. #1 above)
>
> Both above points are not cooked-up ideas, these are based on failures
> and learning from ARM world.

indeed... so that means that a full and comprehensive analysis *of*
the ARM world is needed. all of it. right from the Cortex M-series,
through the ARM9, ARM11, Cortex A8, A7/15, to the A53 and beyond.

then studying the way that PowerPC mirrored x86, and why MIPS
architectures do not, for example.

in other words, it's not just about saying "ARM was a failure because
the boot world is a mess", you have to understand **WHY** it got that
way in the first place.

and in doing so, you may learn that there are extremely good reasons why.

> There is not point of discussion if
> people want to re-invent wheel here.

equally there is no point in discussion if there is a failure to
recognise the reality of quite how disparate the hardware world really
is.

also, again: i'm obliged to remind you that you don't get to tell
software libre developers "what to do". if they want to reinvent
wheels as spheres, squares or other shapes, that's entirely their
prerogative, and you cannot in any way stop them.

secondly: i know you are not (i'm wording it deliberately this way by
way of illustration) - are you *really* proposing, for example, that a
low-cost RISC-V tablet processor design companyy be FORCED to use UEFI
boot, for example, because it "Shall Become A Standard That Is To Be
Forced Onto People In the RISC-V World"?

> For RISC-V ecosystem to succeed, all good practices from other
> architectures should be considered from start itself.

absolutely. except i would not make it *absolutely* conditional that
"RISC-V will *FAIL* if all good practices are not considered", which
is unfortunately a possible implication of the wording that you've
chosen here.

> It is better to take the pain now and get booting flow standardised
> for RISC-V systems.

anup: you have jumped the gun, here. it is very strange. in one
sentence you say "RISC-V should look at good practices from other
architectures", and then *IMMEDIATELY* jump to, "and that process is
ABSOLUTELY going to result in standardisation for RISC-V boot
procedures".

that sounds like the proposed review of good practices from other
architectures is a loaded, foregone conclusion. you see how that
could be bad?

> It will be very difficult in future when we have
> lot of RISC-V systems with there own booting flow.

nooo, the future will reflect what individual diverse fabless
semiconductor companies with completely and utterly different hardware
and customer requirements *need* from a boot flow.

basically what i would like to get across to people here, is, that
u-boot, coreboot, libreboot and the grub developers have been at this
for a lot longer than RISC-V has been around. their expertise and
approach actually takes precedence over that of RISC-V! u-boot has to
deal with thousands of disparate systems (only a tiny handful of which
are *developer* boards with RISC-V cores)

RISC-V is the *newcomer* here. people may *want* boot
standardisation in RISC-V in order to reduce development and
maintenance costs (a desirable goal), however starting from a position
of assuming or implying that the teams who manage those projects must
be "stupid" by having gone their own separate ways and having
"reinvented wheels", is not going to fly.

also a reminder to RISC-V Foundation Members of section 2.2:
https://content.riscv.org/wp-content/uploads/2016/01/RISC-V-Foundation-Membership-Agreement-8Dec2016.pdf

"During the term of its membership in Foundation, Member is expected
to support the free and open design, development and improvement of
the RISC-V ISA, together with its software and hardware ecosystem for
use in all computing devices."

l.

[Anup Patel]

I tried to explain my view in best possible way. If this is taken as
unpaid demand then that was not the intention.

>
> the closest that you can get is as follows:
>
> (1) create a reasonable and rational logical argument and, in NEUTRAL
> TERMS, propose it (separately) to the u-boot developers and to the
> coreboot developers, and the grub developers.
>
> (2) include an offer of compensation and payment for services as part
> of the contact with those respective teams
>
> (3) if there is no take-up (no interest) implement it yourself and
> *demonstrate* that it's a good idea.
>
> you cannot in *any way* expect sovereign libre project developers -
> many of whom are chronically underfunded and absolutely sick and tired
> of MULTIPLE BILLION DOLLAR corporations PERSISTENTLY spongeing off of
> their expertise and time - to drop everything and "do what you want".
>
> i know you *didn't* make demands of software libre teams: please
> however be extremely careful how you put things, ok?

I see this getting derailed. The whole discussion about what should be
correct boot flow.

The "should" in my comment should not be interpreted unpaid demand. The
"should" here is the sense "what I think correct way is".

>
> > The best people to maintain code related to #2
> > are SOC vendors themselves. Of course, code related to #2 has to open
> > source so that people get chance to fix bugs themselves and also have
> > transparency.
>
> indeed. and the upstream developers adequately compensated and paid
> for their time. this does not "go without saying". it's something
> that corporations have to actively take responsibility for. you made
> money? PAY the people on whose success that money is critically
> dependent!
>
> i'll mention the words "shellshock", "heartbleed" and leave it at that.
>
>
> > It is also important that we don't want separate bootloader for
> > Guest/VM (i.e. VS-mode) so last stage bootloader should consider
> > Guest/VM requirements (i.e. #1 above)
> >
> > Both above points are not cooked-up ideas, these are based on failures
> > and learning from ARM world.
>
> indeed... so that means that a full and comprehensive analysis *of*
> the ARM world is needed. all of it. right from the Cortex M-series,
> through the ARM9, ARM11, Cortex A8, A7/15, to the A53 and beyond.
>
> then studying the way that PowerPC mirrored x86, and why MIPS
> architectures do not, for example.

Yes, of course.

People should certainly make suggestions/recommendations based on
their experiences with other architectures.

(Note: Please don't mis-interpret "should" here.)

>
> in other words, it's not just about saying "ARM was a failure because
> the boot world is a mess", you have to understand **WHY** it got that
> way in the first place.
>
> and in doing so, you may learn that there are extremely good reasons why.

Yes, of course.

I don't see people asking "WHY" were things done in a particular way
in some ABC architecture.

>
>
> > There is not point of discussion if
> > people want to re-invent wheel here.
>
> equally there is no point in discussion if there is a failure to
> recognise the reality of quite how disparate the hardware world really
> is.
>
> also, again: i'm obliged to remind you that you don't get to tell
> software libre developers "what to do". if they want to reinvent
> wheels as spheres, squares or other shapes, that's entirely their
> prerogative, and you cannot in any way stop them.

Again, please don't mis-interpret my statement.

>
> secondly: i know you are not (i'm wording it deliberately this way by
> way of illustration) - are you *really* proposing, for example, that a
> low-cost RISC-V tablet processor design companyy be FORCED to use UEFI
> boot, for example, because it "Shall Become A Standard That Is To Be
> Forced Onto People In the RISC-V World"?

I am not supporting any particular bootloader here.

I am trying to suggest what should be correct boot flow.

(Please don't mis-interpret "should" in above sentence)

>
>
> > For RISC-V ecosystem to succeed, all good practices from other
> > architectures should be considered from start itself.
>
> absolutely. except i would not make it *absolutely* conditional that
> "RISC-V will *FAIL* if all good practices are not considered", which
> is unfortunately a possible implication of the wording that you've
> chosen here.

I am done clarifying your mis-interpretations. Where have I implied
RISC-V will FAIL.

It was really interesting to see how simple comments can be
heavily mis-interpreted.

Regards,
Anup

[Luke Kenneth Casson Leighton]

On Sat, Oct 13, 2018 at 10:42 AM Anup Patel <an...@brainfault.org> wrote:

> It was really interesting to see how simple comments can be
> heavily mis-interpreted.

indeed it is... and not by me. i used phrases that indicated that
they *could* (third person) be misinterpreted.

l.

[Luke Kenneth Casson Leighton]

On Sat, Oct 13, 2018 at 10:42 AM Anup Patel <an...@brainfault.org> wrote:
>
> The whole discussion about what should be
> correct boot flow.

for which sub-design architecture? are you referring to the correct
boot flow for embedded systems with below 3MB of (internal) RAM? for
embedded systems with below 48k of (hard-coded, on-board) RAM?
low-cost smartphone / tablet SoC market where the mass-volume target
price of a 4-core 1.5ghz SoC is around USD $3, for use in products
where the OEM price will be under $30? chromebook/netbook style
systems? desktop systems? high-end server systems?

all of these have wildly disparate boot flow requirements, yet you
consistently refer to "one" boot flow, and also use the word
"correct", as if there shall be one and only one possible option. if
you had said, "the whole discussion is about what should be the best
recommended boot flow for each platform", i would have agreed with you
entirely.

apologies for saying this, anup: i feel that it is disingenuous to
claim that the "conversation has been derailed" when i am simply
pointing out that the points that you raised are incomplete,
oversimplified, and indicative of a lack of knowledge of this
extremely comprehensive and diverse area.

> I don't see people asking "WHY" were things done in a particular way
> in some ABC architecture.

i have. as a reverse-engineer and embedded systems engineer who has
over 20 years experience of board bring-up on dozens of systems (many
of them with zero knowledge) i've studied dozens of different
architectures.

i have indeed asked the question "why" for example is PowerPC such a
uniform and easy architecture to work with, when the 1000+ licensees
of ARM do not have it easy at all (one clue is in the "1000+").

l.

[Luke Kenneth Casson Leighton]

On Thu, Oct 11, 2018 at 12:10 PM Padmarao Begari <padma...@gmail.com> wrote:
>
> RISC-V U-Boot(64-bit) for HiFive Unleashed Board is available on GitHub
>
> https://github.com/Microsemi-SoC-IP/HiFive_U-Boot
>
> This repository contains the RISC-V U-Boot created by Microsemi a Microchip company for RISC-V core on HiFive Unleashed 540 device. The HiFive Unleashed Board is used for HiFive U-Boot development

padmarao, hi, as the discussion has changed topic, i thought it
important to say thank you and recognise the work that you and
microsemi have put in here. as someone who has worked on u-boot board
bring-up i know it is very detailed work, so thank you.

l.

[Anup Patel]

On Sat, Oct 13, 2018 at 3:37 PM Luke Kenneth Casson Leighton
<lk...@lkcl.net> wrote:
>
> On Sat, Oct 13, 2018 at 10:42 AM Anup Patel <an...@brainfault.org> wrote:
> >
> > The whole discussion about what should be
> > correct boot flow.
>
> for which sub-design architecture? are you referring to the correct
> boot flow for embedded systems with below 3MB of (internal) RAM? for
> embedded systems with below 48k of (hard-coded, on-board) RAM?
> low-cost smartphone / tablet SoC market where the mass-volume target
> price of a 4-core 1.5ghz SoC is around USD $3, for use in products
> where the OEM price will be under $30? chromebook/netbook style
> systems? desktop systems? high-end server systems?

Yap, ATF-like boot flow work perfectly fine for internal SRAM as
low as 128KB

I have seen this in both embedded and high-end server systems.

>
> all of these have wildly disparate boot flow requirements, yet you
> consistently refer to "one" boot flow, and also use the word
> "correct", as if there shall be one and only one possible option. if
> you had said, "the whole discussion is about what should be the best
> recommended boot flow for each platform", i would have agreed with you
> entirely.

Yes, we can still come-up with boot flow where certain components
of boot chain are configurable as well as optional.

>
> apologies for saying this, anup: i feel that it is disingenuous to
> claim that the "conversation has been derailed" when i am simply
> pointing out that the points that you raised are incomplete,
> oversimplified, and indicative of a lack of knowledge of this
> extremely comprehensive and diverse area.

I think so because whole discussion was about why M-mode
code should not be there in last stage bootloader and it was not
about describing a detailed boot flow.

May be I should have directly proposed a detailed boot flow to
satisfy your queries.

>
> > I don't see people asking "WHY" were things done in a particular way
> > in some ABC architecture.
>
> i have. as a reverse-engineer and embedded systems engineer who has
> over 20 years experience of board bring-up on dozens of systems (many
> of them with zero knowledge) i've studied dozens of different
> architectures.
>
> i have indeed asked the question "why" for example is PowerPC such a
> uniform and easy architecture to work with, when the 1000+ licensees
> of ARM do not have it easy at all (one clue is in the "1000+").
>

Thanks for asking why.

I have provided enough justification for why i think M-mode code should
not be part of last stage bootloader.

Regards,
Anup

[Luke Kenneth Casson Leighton]

On Sat, Oct 13, 2018 at 12:04 PM Anup Patel <an...@brainfault.org> wrote:

> May be I should have directly proposed a detailed boot flow to
> satisfy your queries.

clarity always helps.

> > i have indeed asked the question "why" for example is PowerPC such a
> > uniform and easy architecture to work with, when the 1000+ licensees
> > of ARM do not have it easy at all (one clue is in the "1000+").
> >
>
> Thanks for asking why.
>
> I have provided enough justification for why i think M-mode code should
> not be part of last stage bootloader.

nggh, anup, y'know... i face a lot of shit on this list, as you've
probably witnessed, and many people correct my mistakes, sometimes
kind and informative ways, and at other times in really quite rude
ways, which is extremely distressing, so, knowing how that feels, i'm
in absolutely no way going to do the same to you, ever, so please bear
that in mind, ok?

i do feel compelled to point out that i don't feel that you have
provided enough justifiction for why M-Mode should not be part of the
last stage bootloader. it is i feel pretty easy to come up with a
scenario (embedded barely-fast-enough low-cost tablets where the cost
of every storage component is counted down to the last $0.01) where
the cost of the extra code in terms of performance and storage
capacity of such a [non-M-mode last-stage bootloader] proposal would
result in its outright rejection by OEMs.

that said: the *option* for last stage bootloaders to not run in
M-Mode: i would agree, yes, that is indeed a desirable characteristic.

l.

[Tommy Murphy]

Huh? You're saying that if the sources are available you *can't* see and fix bugs if necessary? Doesn't make any sense to me...

[Samuel Falvo II]

On Sat, Oct 13, 2018 at 5:35 AM Tommy Murphy <tommy_...@hotmail.com> wrote:
> Huh? You're saying that if the sources are available you *can't* see and fix bugs if necessary? Doesn't make any sense to me...

You did not quote, even partially, the person you're responding to, so
I'm going to assume you are responding to me.

The sources to UEFI are readily available on Github (for at least one
vendor, at least); despite this, you don't see me reflashing my PC
every time it gets updated. First, I don't have the development
toolchain that ASUS, American Megatrends, et. al. use. Second, I
don't have the proprietary extensions my firmware vendor has lumped
into the sources (including but not limited to the proprietary
motherboard-specific blobs). Third, it's largely unnecessary, for
when Linux boots, it largely obsoletes and renders UEFI moot. Because
of (3), then, clearly I'm not referring to bootloader-related stuff.
I'm talking about M-mode code for things like emulating FPU
instructions, or emulating a hardware page table walker, or other
kinds of things for which M-mode was actually intended for: the kinds
of things which frequently reside in "opaque, binary blobs" as they're
colloquially referred to these days. The kinds of things which
hardware vendors generally doesn't distribute sources to.

[Padmarao Begari]

Thank you Luke...👍

Sent from my iPhone

[Jonathan Neuschäfer]

On Sat, Oct 13, 2018 at 01:35:29AM +0100, Luke Kenneth Casson Leighton wrote:
[...]

> so, logically, when you refer to a desire to run "bootloaders" in
> S-Mode we may *deduce* that you are NOT referring to the ENTIRE boot
> process, that instead that you may be referring to the SECOND phase of
> u-boot / coreboot.

True.

To elaborate a bit more on the different parts being discussed: I see
three somewhat separable concerns in this discussion, that a RISC-V
boot/firmware "stack" needs to address:

(a) Low-level hardware initialization, for example of the DRAM
(b) Run-time callable firmware, which (on RISC-V) implements the SBI
(c) A full-featured boot loader which can read different storage/network
media and file systems, in order to have flexibility in loading the OS.

Some examples of how these concerns are implemented:

- SiFive's FSBL implements (a), and bbl implements (b). The original
boot stack from SiFive does not implement concern (c).
- coreboot's focus is on (a), and it happens to implement* (b) on RISC-V
because that's necessary for running linux. coreboot itself is not,
and has never been a good boot loader (concern (c)). It's just good
enough to load a "payload" from its boot medium (i.e. SPI flash).
[ And on x86, coreboot provides ACPI tables, which may be considered
run-time callable firmware, and an SMI handler. ]
- ATF in the ARM world focuses on (b), but also implements (a)
sometimes, AFAIUI.
- On chromebooks, (a) is implemented by coreboot, and (c) is implemented
by [Depthcharge]. On ARM chromebooks, (b) is usually implemented by ATF.
- On coreboot+TianoCore platforms, coreboot implements (a), and
TianoCore implements (b) and (c).
- u-boot usually implements (a) and (c), and I guess it can also
implement (b)
- The [LinuxBoot] project focuses on (c), by reusing Linux's vast
support for storage/network media and file systems


Personally, I'd welcome a central or popular implementation of the SBI
(and thus, concern (b)), which can be used with coreboot or u-boot SPL,
or the vendor's proprietary thing for concern (a), and u-boot or
LinuxBoot (once kexec is ready) or something else for concern (c).
Maybe a slightly stripped-down bbl can fit that mold.

And, personally, once these three concerns are disentangled, I don't see
why a pure boot loader (concern (c)) shouldn't run in S-mode.
[ If something must run in M-mode, it's probably not a pure boot loader. ]


Thanks,
Jonathan Neuschäfer

(* In a patch that's still under review. Development/review has been
somewhat slow, sorry about that.)

[Depthcharge]: https://chromium.googlesource.com/chromiumos/platform/depthcharge/
[LinuxBoot]: https://www.linuxboot.org/

[ron minnich]

On Sat, Oct 13, 2018 at 5:35 AM Tommy Murphy <tommy_...@hotmail.com> wrote:

Huh? You're saying that if the sources are available you *can't* see and fix bugs if necessary? Doesn't make any sense to me...

tl;dr  sorry, this got too long :-(

you betcha. It makes all kinds of sense. This is the common misunderstanding about firmware: people expect that firmware problems are as easily fixed as in kernels. It's a whole higher level of pain.

There are many cases where source code is released, but the vendor has found out ways to make it impossible to build or install it. Either the build toolchain is not available, or the device is locked down in a way that it can't be reflashed, or some critical piece of "core IP" is left out. Just look at the last 20 years of Intel cpus. Or the Motorola phones, ca 12 years ago: all the source you want, no way you ever get to install it. Or look at BootGuard: you can build all you want, but you don't get to sign it, so if you use your build you can't have security. Or the one-time fuses in chipsets that let you set up Root Of Trust *once*, meaning among other things it's very hard to resell systems, since the eventual end user can't change the keys (I'm involved in this mess right now). There are so many barriers.

Source availability does not imply that you can build or fix things. Not at all. And, there's no guarantee that the supplied binary will match the supplied source, so just seeing source is of no real value. We've seen that too. 

The only real guarantee is that you run what you build.

But back to M mode.

M mode can do anything, without limit. For that reason, it must always be possible to audit, rebuild and install M-mode code from source. This could be made impossible with a few key decisions at the chipset level. Vendors over the decades have shown a tendency to make those decisions, as part of "adding value" (their phrase) or "vendor lockin" (the customer's phrase). 

The RISCV community has a chance to get this right, but we could also get it wrong, and I'd rather see us get it right. When vendors start arguing for common implementations,I worry that we're about to get it wrong, because frequently "common implementation" ends up meaning "baked in and unchangeable implementation." Because that's happened so many times. See: "adding value."

Further, given that there are still at least 1 BILLION x86 systems out there with all kinds of embedded exploits in runtime firmware, I now strongly believe that M mode code should be installed by the kernel, post boot, since that gets around a big problem: nobody ever updates firmware, even when the worst exploits appear, because they don't know, don't care, or it's almost impossible (a German friend at the BSI explained that Intel's mitigation software for one ME exploit won't run under Windows 10, due to W10 security checks; they had to downgrade systems to Windows 7, run the update, then reinstall WIndows 10. Care to guess how many people will do this?). 

A kernel can supply its own M mode code, so the path to critical security fixes is a reboot, not a reflash. Much simpler. One billion times more likely to happen. 

So not only do I argue that firmware should supply its own M mode code, so should kernels, and that argument is supported by decades of watching ODMs get firmware security wrong. I would list all the problems the LinuxBoot community has found in UEFI in just the last few months, but we don't have time. Let's just say it's turning out there's a lot of money in auditing x86 firmware binaries for exploits; they are being found; and let's further say that many x86 system vendors are pretty clueless about the firmware they're shipping.

This is not an argument against a standardized boot flow. This is an argument for ensuring that kernels can guarantee properties of their runtime environment, and recognizing that environment includes M mode. Further, multiple implementations frequently strengthen, not weaken, standards. See, for example, TCP/IP. Many implementations, all interoperable. Long ago (1980s) there were arguments for one TCP/IP code base, for the exact same reasons I'm seeing for a single M mode code base. It made no sense back then either.

Kernels should always have the option to not share a machine with a binary that is provided by a hardware vendor. Hence it should always be possible for a kernel to supply its own M mode code.

ron

[kr...@berkeley.edu]

>>>>> On Fri, 12 Oct 2018 18:00:14 -0700, Samuel Falvo II <sam....@gmail.com> said:
| I'm going to once again put forth the idea that it was a mistake for
| the RISC-V community to settle on an ITU-style standards setting
| process. While it may prove advantageous in some circumstances, I
| think it sets a bad precedent overall where viable implementations
| come second to the desire for an up-front, all-inclusive, frequently
| self-inconsistent standard with which everyone must conform. See also
| CORBA.

RISC-V standard setting process is for the ISA and closely related
issues. Software layers above can operate standards procedure as they
usually do for each software ecosystem. Some coordination between
these layers is obviously useful, but does not have to be heavyweight
(see how the calling convention was established as an example).

Setting standards for what will be baked into silicon after huge
investment is very different that setting software standards where
systems can support multiple variants and be quickly updated in the
field.

Krste

[Luke Kenneth Casson Leighton]

On Sat, Oct 13, 2018 at 5:39 PM ron minnich <rmin...@gmail.com> wrote:

> Or look at BootGuard: you can build all you want,
> but you don't get to sign it, so if you use your build
> you can't have security. Or the one-time fuses in chipsets
> that let you set up Root Of Trust *once*, meaning among
> other things it's very hard to resell systems, since the
> eventual end user can't change the keys (I'm involved
> in this mess right now).

nothing wrong with that, right? you get to create jobs by throwing
away the old device and buying a new one. that's how capitalism
works, right?

https://news.slashdot.org/story/18/10/12/233230/tech-suffers-from-lack-of-humanities-says-mozilla-head
(and other links, this is only just the latest one)

[Luke Kenneth Casson Leighton]

On Sat, Oct 13, 2018 at 6:17 PM <kr...@berkeley.edu> wrote:

> Setting standards for what will be baked into silicon after huge
> investment is very different that setting software standards where
> systems can support multiple variants and be quickly updated in the
> field.

absolutely, i mentioned the example of the DDR3/4 initialisation
sequence as just one of the examples which make it absolutely
essential to keep the boot ROM to the absolute bare minimum, and using
as generic a range of protocols as can possibly be gotten away with.

expecting a ROM-based bootloader to run SATA, PCIe, IDE or other
complex bus is absolutely asking for trouble. a boot peripheral has
to be *absolutely* dead simple, and implementable - FULLY - in a few
*HUNDREDS* of lines of code. not thousands. not tens of thousands.
a few HUNDRED lines of code.

then, also, the actual boot media has to be incredibly popular and
unlikely to change. SPI, QuadSPI, SDMMC (which has fallback to SPI),
eMMC (being ultimately all based on SPI) are the usual ones that have
tens of thousands of devices available for decades, from dozens of
manufacturers.

any processor that offers those protocols as part of its boot ROM, and
can load at least one 16k block from it, is going to be pretty safe
for decades of life. in the embedded world, I2C, UART and USB1.1 (if
the SoC actually *has* USB) are typically added as well.

*all* of those protocols are world-wide popular, stable, decades-old,
not controlled by any one corporation... implication: they're not
going away any time soon.

now for an example of where a massive company got it wrong... without
doing anything wrong.

nintendo's gameboy was a nightmare scenario (for them). to cut down
the cost they didn't use a separate SPI/QSPI bootloader IC (because
they can actually be very expensive, relatively speaking, and a
mass-volume product is extremely price-sensitive on the BOM).

so they booted directly from NAND.

TSSOP-48 ONFI NAND ICs change so fast relatively speaking that the
*entire gameboy product* had to be end-of-lifed ... why? because the
HARD-CODED bootloader in its SoC's ROM had a series of fixed
parameters, and the NAND ICs supporting those exact and precise
parameters were *no longer being manufactured* [1]

so that's what products - and fabless semiconductor companies - can
face if they get the design of the bootloader wrong.

l.

[1] those parameters include voltage, block size, checksum size,
speed, density... and no, the JEDEC standards are not properly
supported by manufacturers. every single NAND IC that has different
parameters is completely and utterly incompatible. it's pretty
insane.

[Luke Kenneth Casson Leighton]

On Sat, Oct 13, 2018 at 5:39 PM ron minnich <rmin...@gmail.com> wrote:

> Vendors over the decades have shown a tendency to make those
> decisions, as part of "adding value" (their phrase)

"trust"

> or "vendor lockin" (the customer's phrase).

"treachery".

> So not only do I argue that firmware should supply its own
> M mode code, so should kernels, and that argument is supported
> by decades of watching ODMs get firmware security wrong.

interesting. and insightful.

> This is not an argument against a standardized boot flow.

understood

> This is an argument for ensuring that kernels can guarantee
> properties of their runtime environment, and recognizing that
> environment includes M mode.

so what you're saying is: once one phase of a bootloader only permits
applications (including the next phases in a bootloader) to be run in
S-Mode (only), the only guarantees that can be made about security are
those of the S-Mode "guest"... which is to say... absolutely no
security guarantees at all.

[historically, from the examples you gave, and many more that would
be far too numerous to list, we've seen manufacturers persistently and
consistently fail for literally decades to even *comprehend* security.
because it costs money to get it right].

that in turn results in people throwing product out and buying new
ones... with no guarantee that the new product is not designed along
the exact same deeply-flawed design paradigm as the old one they just
put into landfill.

> Kernels should always have the option to not share a machine
> with a binary that is provided by a hardware vendor. Hence it
> should always be possible for a kernel to supply its own M mode code.

so this is a very insightful and reasonable argument for ensuring
that a boot flow does *not* preclude the *possibility* of running in
M-Mode.

realistically the cynic in me says that this won't stop vendors from
creating RISC-V processors that lock out M-Mode... they're not going
to learn... however if there's at least some products released early
enough, leading the way and showing that successful product does *not*
have to be treacherous to its users...

l.

[Samuel Falvo II]

On Sat, Oct 13, 2018 at 10:17 AM <kr...@berkeley.edu> wrote:
> Setting standards for what will be baked into silicon after huge
> investment is very different that setting software standards where
> systems can support multiple variants and be quickly updated in the
> field.

Hi Krste,

This is correct, which is why I clarified that ITU-style has value in
certain circumstances. I'm just expressing a fear that this approach
will bleed over, if you will, into the software world where things
that can be allowed to be made more reactive to market forces *should*
be. Apologies in advance if I'm not (or still not) expressing myself
clearly.

For example, there's a large concern over standardizing the boot flow
for everything from wristwatches to weather-predicting supercomputer
clusters and literally everything in between. This is a noble goal,
but I also feel it's something of a fools' errand, for reasons that
both Ron and Luke expressed far more gracefully than I ever could.
More frequently than not, when we end up in debates like these, a lot
of time and effort is spent justifying one's position with logic and
reason but not necessarily with concrete implementations to make
objective comparisons against. I'm much more inclined to agree with
Ron and Luke because both have their names in code repositories that I
have ready access to and whose projects are steeped in the
bootstrapping process for a plurality of architectures, not just x86
and not just ARM, and certainly not for any NDA-covered projects that
nobody can talk about.

This is why I advocate for an implementation-based, bottom-up
standards track process. I believe working code and field-deployments
are far more valuable than speculation, however well-reasoned they may
be. Except for code signing and the semantics of the GPT, much of the
benefit of UEFI existed long before it came along. Per-partition
filesystems, for example, existed in OS/2's "micro-filesystem
drivers", for instance, and these existed with a BIOS barely more
sophisticated than CP/M's. OpenFirmware gave us both an interactive
boot-time shell as well as the venerable Device Tree standard we still
use today.

EFI technically lost in the market; the only reason we got UEFI today
is because Intel and Microsoft effectively forced it up on us by
mandating all new motherboards ship with it, and Windows refused to
boot on a system without it past some cut-off date. In case few
remember, OpenFirmware was actually the leading alternative to PC BIOS
back then, and was gaining up-take *fast* in a variety of different
markets, including data-center server racks.

Bluntly, I don't want to see another UEFI happen.

However, if the community does prefer a top-down approach to
establishing some manner of standards, then I'd like to raise a point
as an independent observer. This conversation has two conflicting
points to it: the ARM world is an abject failure because its
bootstrapping conventions are a total mess, and the ARM world is to be
taken as something of a model because ARM Ltd. defined something
called ATF and is the way of the future. If I'm reading this thread
correctly, these are mutually exclusive viewpoints, both of which are
factoring into the desire for a unified boot flow for the RISC-V
community (hence my mention of self-inconsistent when referring to
top-down standards). Before we get into standardizing any kind of
boot flows, I think we need to quantitatively define what "failure"
means in the context of a processor architecture and ISA which has all
but thoroughly dominated the RISC market and has done so without the
help of trusted firmware for the majority of its existence.

[Karsten Merker]

On Fri, Oct 12, 2018 at 02:12:08PM -0700, alist...@gmail.com wrote:

> I agree that we shouldn't trust vendor firmware. It is always buggy at best.
>
> The problem that I see with starting coreboot or u-boot (the last stage
> bootloaders) in M mode is that we then end up with duplicated work between
> the projects.
>
> This is where a model like Arm Trusted Firmware fits in well. We can see
> something like this:
> 1. CPU boots up and ROM does it's thing
> 2. ROM boots into an open source firmware (ATF equivalent) in M mode
> - This will end up being pretty SoC specific
> 3. The firmware boots into u-boot or core boot in S mode (or H mode when
> that exists)
> - Hopefully this can then be as generic as possible
> 4. From there we boot the system
>
> The advantage I see with this is that we only need to implement the M mode
> functionality once and then coreboot, u-boot and anything else can utilise
> it. It also keeps bigger later stage boot loaders out of M mode which has a
> mild security benefit of smaller attack surfaces.
>
> The other aim is to allow a single RISC-V image to boot on a variety of
> different SoCs. The more generic we can make the last stage boot loaders
> the more achievable this is.

Hello,

I'd like to chime in here with some thoughts about
firmware/bootloader interfaces from a Linux distribution's point
of view. This isn't strictly about M-mode vs. S-mode, but
nonetheless related. I've changed the subject of the email to
show that this is somewhat of a side-thread, but IMHO important
nonetheless in the context of the existing discussion.

As an ideal, Linux distributions would like to see a situation
where (provided that the hardware is technicially capable of
running the distribution), a single installer image can be used
on all systems and the resulting installed system "just works".
Having to build per-machine-type installer images is a nightmare
and doesn't scale - I have been doing that for ARM-based systems
for quite some time, so I can speak from first-hand experience
here.

In reality unfortunately this ideal isn't achieved anywhere, not
even on x86 which tends to be commonly mentioned as the platform
where "things just work". Before laying out what I would like to
see standardized on RISC-V, I'll try to describe some of the
experiences that Debian developers have made on x86 and ARM as
those are usually the two platforms that come up as examples in
discussions related to boot interfaces. This hopefully helps to
explain why I would like to see certain things standardized in a
certain way.

- x86 has classic BIOS and UEFI, and the latter in 32bit and
64bit variants, sometimes in severely broken hybrid forms
(64bit CPUs combined with 32bit-only UEFI implementations that
require dirty hacks to boot a 64bit OS).

Even when declaring the classic BIOS to be legacy and just
concentrating on UEFI with ACPI, there are still lots of broken
implementations in the wild that require tons of workarounds on
multiple levels, be it the plethora of in-kernel workarounds
for broken ACPI implementations or the broken boot handling in
quite a number of existing UEFI implementations that make the
theoretical advantages of having a proper bootloader interface
defined in the UEFI specification often moot in practice. Add
to that the problem that in practice many UEFI and ACPI
implementations don't do what the spec says but instead do what
works with Windows. As a consequence, the Linux kernel had to
move to emulating the Windows behaviour instead of doing things
the way that the spec actually provides because otherwise it
wouldn't work properly on real-world systems. And then there
are of course such great things as real-world UEFI
implementations that kill themselves (as in converting a
working Laptop to a paperweight) as soon one actually uses the
interfaces that the UEFI spec provides but that aren't commonly
used by Windows. Remember the "Linux kills Laptops" uproar?
In fact that was "broken UEFI kills Laptops" - Linux called a
UEFI function to write a crash dump into a UEFI NVRAM variable
in a perfectly spec-compliant way, but a number of existing
UEFI implementations didn't properly handle large writes
resulting in a tight loop on UEFI initialization at the next
reboot, turning the Laptop into a paperweight. Windows doesn't
do large writes by default, so the problem usually didn't show
up under Windows. You can probably imagine the reactions that
Linux distributions faced with people claiming "You have killed
my laptop!".

- On armhf (32bit) platforms, there hasn't traditionally been a
standard firmware interface for a long time. The SoC bootroom
usually loads a bitstream from a vendor-specific location
(either from SPI NOR flash or from a vendor-defined MMC block
and executes that, whatever there is. In the embedded space,
vendors have usually forked u-boot and hacked in support for
their SoC, defining a vendor-specific method for loading the
kernel, and left it at being incompatible with the rest of the
world.

In mainline u-boot there have been efforts to standardize the
boot interface that is provided to the later stages since IIRC
2014. This goes under the label of "distro bootcommand" and
provides a vendor-neutral boot interface to e.g. Linux
distributions. With this, one can boot the same installer
image on all armhf platforms that use a u-boot with "distro
bootcommand" support. All platforms that are newly added to
mainline u-boot are expected to implement "distro bootcommand"
support and many existing platforms have been converted to it
in the meantime.

On arm64 there are two worlds - the embedded space, using
u-boot+device-tree (usually combined with the "ARM Trusted
Firmware" (ATF)), and the server space with either UEFI+ACPI or
UEFI+device-tree. Pure enterprise distributions like RHEL
usually only target UEFI+ACPI on arm64 and ignore everything
else. Community distributions like Debian usually target all
of the aforementioned options: u-boot+device-tree,
UEFI+device-tree and UEFI+ACPI.

ATF is published under BSD license by ARM, but there are two
big problems with ATF in practice:

- There are two incompatible ATF interface definitions, an
older and a newer one. Unfortunately some SoC vendors use
the old one while others use the new one :-(.

- The "upstream" ATF originally didn't have support for any
hardware besides some reference systems from ARM (similar to
the situation with Tianocore in the UEFI world). SoC vendors
usually used the BSD-licensed ATF as the base for a
closed-source fork that included hardware support for their
specific SoC and didn't contribute back anything to the
upstream ATF, and of course also often haven't pulled in any
later ATF bug fixes. In the meantime, open-source developers
have contributed (partially reverse-engineered) support for
some arm64 platforms to upstream ATF, but that are still
rather few.

The problem that Linux distributions have with the
closed-source vendor forks of ATF is that they a) are often
broken and b) aren't stored in some form of onboard memory
(like an SPI NOR flash) but are shipped by the vendors as
part of their SD card boot images. In consequence, Linux
distributions would have to include those proprietary ATF
builds in their boot images, which they often can't for legal
(nobody except the SoC vendor has the right to distribute the
proprietary code) and/or policy reasons (Debian doesn't ship
any software that isn't Open Source as part of the main
distribution). Exactly the same problem exists with the code
that initializes the DRAM controller. Like with ATF the
actual implementation is usually proprietary (at least until
some open-source developer has managed to reverse-engineer
the way the DRAM controller needs to be set up) and in
contrast to PCs, where the DRAM initialization is always done
by code that is stored in an onboard flash chip, on arm
platforms the DRAM init code is often shipped as part of the
SD card boot image. In consequence, the corresponding
systems cannot be supported by Debian.


Alex Graf from SUSE has been working on making it possible to
install distributions that expect UEFI, but can use device-tree
instead of ACPI (which is a configuration that is covered by
the UEFI spec) on systems using u-boot as their boot firmware
by adding a UEFI emulation layer on top of u-boot. The
important point hereby is that this layer implements the
UEFI boot services, but not the UEFI runtime services, i.e.
it can be used to load a UEFI-GRUB, but it cannot be used
with operating systems that expect to be able to call
UEFI functions during runtime, like writing to UEFI NVRAM
variables, so this is not a replacement for a full UEFI
implementation.


For the "low-level" part of system firmware (i.e. the part that
handles stuff like initial voltage regulator setup, DRAM
initialization, etc.), I would like to see the following things
standardized for RISC-V as part of the unix platform spec:

* The platform vendor must provide DRAM initialization code
with a standardized interface in an on-board memory, so that
an OS provider doesn't have to ship the code as part of the
OS image, but can call it when needed.

The reason for this is outlined above in the section about
ATF - most DRAM init code is only provided under NDA by the
DRAM controller IP vendor, so OS providers may not be able to
use it if it isn't already available on the system.

* Define a standard partition table layout that is taken into
account by anything on the system, including the boot
firmware. My proposal would be GPT because it's flexible,
supported by about everything and the kitchen sink, and
supports large volumes (which is a problem with the classic
BIOS partition table format).

* If a SoC supports loading its raw firmware bitstream from a
block device (typically SD/eMMC) by loading a certain amount
of bytes from a certain offset, define _one_ standard offset
and length where it is to be loaded from, making sure that it
is compatible with the chosen partition table layout. The
reason for that is that without such a definition, one would
again risk incompatibilities with OS, userland applications
and possible alternative firmwares.

SoCs can usually boot their firmware from multiple places,
selectable by external pins or in a fixed order. When using
a fixed order, usually removable storage (SD card) has
highest priority, then comes eMMC, then SPI NOR flash. This
is really nice because it allows to easily "unbrick" a broken
system, e.g. if flashing a new firmware into the onboard NOR
flash has failed and the system doesn't come up anymore.
Code is only loaded from a device if it has a valid header
(usually some magic number and a CRC), so if there is no
valid header on removeable media because nobody has installed
a firmware image there, the boot process simply continues
with the in-flash vendor code. If there is a valid firmware
image on the block device, it gets loaded and executed. This
allows either to fix the broken NOR flash contents or just
use this firmware as a flexible boot-time replacement for the
firmware in the NOR flash. Usually the area the SoC loads
its firmware from is somewhere after the partition table but
below 1MB (where normally the first partition starts for
BIOS-style and GPT-style partition tables). In the arm
world, a number of SoCs have placed their firmware location
as a 32kB block at 8k offset from the beginning of the
device. Unfortunately while this works for systems using
BIOS-style partition tables, it breaks for systems using
GPT-style partition tables, because those are larger. The
maximum length of the firmware boot block needs to be
specified as well, because firmware like u-boot usually uses
the space between the first-stage boot image and the start of
the first partition for its own (non-SPL) code and data, so
it needs to know from which offset onwards blocks can be used
without interfering.


For the "high-level" part of boot firmware (the part that
implements the "user-facing" part of the boot process and
implements methods to boot from SATA disks, network interfaces,
provide system configuration options, etc.), finding a reasonable
standard that covers all use cases is really difficult.

I can understand the "simply go with UEFI and ACPI" argument that
Richard Jones had made in his talk during the Barcelona workshop,
but only when limited to the premise that it has been made under,
i.e. "RedHat is only interested in the server market and nothing
else".

One can surely with good reason argue "UEFI+ACPI is used on x86
and arm64, do the same for RISC-V" as there is existing support
for UEFI-based systems in many operating systems and bootloaders.
On the other hand, there is a lot to be said against UEFI+ACPI,
including that it is way too complex (not only for the embedded
Linux use case, but also too complex for reasonably creating
secure and bug-free implementations in the server space), has a
large attack surface (see Ron Minnichs talks), and reality shows
that things that are fine in theory are way too often broken in
real-world implementations (which I believe is also to a certain
amount the fault of UEFI's enormous complexity).

ACPI is a no-go from my point of view, I've seen way too many
broken ACPI implementations in the PC world that cannot be fixed
by the user and in practice usually don't get fixed by the
manufacturers. Things are different if you "live" in the
high-end server market (both on x86 and on ARM), but for
commodity hardware my experience has been rather devastating.
If one would want to standardize on using UEFI at all, then IMHO
only in the form of UEFI+device-tree.

For the embedded Linux use case, UEFI+ACPI isn't even worth
a single thought - from my experience with the embedded Linux
world on ARM platforms I am sure that there isn't the chance of
a snowball in hell that platform vendors for the embedded Linux
market would be willing or capable of providing a reasonable
UEFI+ACPI implementation for their systems.

U-boot on the other hand is a rather nice firmware and bootloader
for embedded systems, but it is definitely not designed for the
server world. For the embedded space, u-boot can surely provide
a reasonably easy and platform-neutral interface for booting an
operating system.

The obvious compromise could indeed be an interface standard that
defines some stripped-down UEFI boot services as the common
denominator. This would allow hardware vendors to choose between
various existing solutions, i.e. between using either some
Tianocore-based UEFI implementation with their own HAL, using
coreboot for the lower layer with a Tianocore-based payload
providing the UEFI interface to the outside, or using a
u-boot-based implementation with u-boot's UEFI emulation layer.
The latter would provide the standardized interface for embedded
use cases without significant cost (the code size increase due to
the emulation layer is rather negligible).

Comments welcome :).

Regards,
Karsten
--
Gem. Par. 28 Abs. 4 Bundesdatenschutzgesetz widerspreche ich der Nutzung
sowie der Weitergabe meiner personenbezogenen Daten für Zwecke der
Werbung sowie der Markt- oder Meinungsforschung.

[kr...@berkeley.edu]

[ Not particularly responding to Sam, but more to whole thread ]

In this thread, as is common when engaging a wide community of smart
experienced people, many of the disagreements are a result of people
having different, even if possibly overlapping, subsets of the
computing universe in mind. It would be helpful if everyone qualified
their position with the systems they're assuming (microcontroller with
no external DRAM, or servers with 8 sockets and terabytes of DRAM) and
especially if we could introduce a common taxonomy/vocabulary, so
people would be less likely to talk past each other.

I think we can all agree that reducing unnecessary duplication of
effort is a good thing, and both standards and open-source code are
great and complementary ways to achieve that goal.

I think we _should_ all agree that there is necessary fragmentation
because different platforms have different needs. While some systems
might never trust vendor-supplied M-mode software, there are other
systems that would never trust a Linux kernel to install/execute
M-mode code. Similarly, in structuring code bases, some people might
want their Linux to know nothing of what's underneath to support
flexible virtualization, while others want deep control of the
hardware, e.g., for real-time concerns. Other systems will never run
any Unix, but might still need to run many of the same steps during
boot and need similar runtime support afterwards.

The big issue underlying most peoples' positions is that structuring a
single set of interfaces and a single code base to support all these
different use cases is not easy and perhaps not possible with current
system programming tools (beyond "copy-and-paste" reuse), and so
they'd like to make sure their use-case is the one supported by the
shared software effort.

I would hope the community could work towards figuring out how to
share most of the code even if each piece of code runs in different
places in a given boot/runtime system.

I would also ask to separate out concerns about "binary blob" device
drivers from defining boot flow/runtime components, as that is a
mostly orthogonal problem to be solved.

RISC-V should reduce the concerns about monopolistic hardware vendor
policies, as there is no monopoly and there can be many independent
hardware vendors catering to different markets, and using very
different CPU designs (instead of many different vendors using the
same few CPU designs). Because there will be many more RISC-V vendors
than other architectures, there is a greater danger of fragmentation,
so it is important to try to develop a small set of standards with
supporting open implementations, and hopefully open compliance suites,
that cover the common use cases.

Krste

[kr...@berkeley.edu]

Karsten,

Thanks for the detailed writeup of past experience. I'm sure many
will find this very helpful to review. Palmer is starting up an
effort to define RISC-V Unix platform standards and hopefully this can
feed into that.

Krste


>>>>> On Sat, 13 Oct 2018 22:32:14 +0200, Karsten Merker <mer...@debian.org> said:
[..]

[Luke Kenneth Casson Leighton]

from what you write below (i read down then replied bottom-up), i
agree entirely. really appreciated your input here, karsten. you
have first-hand knowledge of the areas i've covered at various
different points over the past 20 years, and haven't really been able
to descrribe to people here quite what the reality is, and you manage
to do that extremely well.

> As an ideal, Linux distributions would like to see a situation
> where (provided that the hardware is technicially capable of
> running the distribution), a single installer image can be used
> on all systems and the resulting installed system "just works".

i know... *sigh*.

> Having to build per-machine-type installer images is a nightmare
> and doesn't scale - I have been doing that for ARM-based systems
> for quite some time,

i _did_ try to warn people, back when devicetree was touted as "the
solution!!!! to all the ARM world disparate pain!!!!!", that with
literally tens of thousands of sensor and peripheral ICs used in
products, all that would happen with devicetree is that the
programming would move *from* c source to *to* devicetree source code.

the experience of PowerPC having so successfully adopted device-tree
(because it's near-identical to x86 in terms of usage, i.e. PCIe-based
motherboards) was unfortunately quoted as a "counter-example", and
nobody listened.

whoops.

so, my sincere apologies: i know the pain that you describe, all too
well, and by the very nature of what SoCs are for, what hardware
interfaces they have (non-self-describing bus architectures like UART
and SPI) and what they do, it's really *really* just not "fixable".

i cannot emphasise that enough.

> so I can speak from first-hand experience here.

cool. that's very helpful, karsten. i have a hybrid of first-hand
experience and second-hand knowledge in the areas you comprehensively
describe, below, so can chip in and give insights and also point out
where things might work, and where they definitely won't (and why).

> - x86 has classic BIOS and UEFI, and the latter in 32bit and
> 64bit variants, sometimes in severely broken hybrid forms
> (64bit CPUs combined with 32bit-only UEFI implementations that
> require dirty hacks to boot a 64bit OS).

whoops.

> - On armhf (32bit) platforms, there hasn't traditionally been a
> standard firmware interface for a long time. The SoC bootroom
> usually loads a bitstream from a vendor-specific location
> (either from SPI NOR flash or from a vendor-defined MMC block
> and executes that, whatever there is.

yep. allwinner's bootrom has been reverse-engineered, and is clearly
evidently based on u-boot. it checks for SPI, and eMMC, and SDMMC, on
various different ports, and, finally gives up and drops into USB-OTG
"Front-end Loader" mode.

> In the embedded space,
> vendors have usually forked u-boot and hacked in support for
> their SoC, defining a vendor-specific method for loading the
> kernel, and left it at being incompatible with the rest of the
> world.

yep. and it works extremely well for them. many chinese fabless
semiconductor companies are now billion-dollar operations by virtue of
this "strategy".

> For the "low-level" part of system firmware (i.e. the part that
> handles stuff like initial voltage regulator setup, DRAM
> initialization, etc.), I would like to see the following things
> standardized for RISC-V as part of the unix platform spec:
>
> * The platform vendor must provide DRAM initialization code
> with a standardized interface in an on-board memory, so that
> an OS provider doesn't have to ship the code as part of the
> OS image, but can call it when needed.

unfortunately they can't do that, for reasons i mentioned earlier, in
other messages: there may be some cross-over.

DRAM initialisation is far too complex, there's not enough
standardisation (DDR2 is different from DDR3 is different from LPDDR2
is different from LPDDR3 is different from DDR4 is different from
LPDDR4 and that's not taking the speeds, layouts (x16, x8, 2x8, x32,
2x16) into account.

you can't possibly predict whether any given *THIRD PARTY* RAM chip
supplier - in the near to immediate future - is going to work.
fabless semi companies actually have to provide a "compatibility list"
of combinations that they've tested. these lists run to over a
hundred entries!

also, if the ROM gets it wrong, on the impedance matching, because
there's some tolerance in the PCB layout due to tight board size,
thickness and number of layers constraints, the ODM will just write
the entire SoC off as a total loss and pick another one.

whilst it would indeed solve the libre-licensing issues, it's just
far, far too risky to consider.

> The reason for this is outlined above in the section about
> ATF - most DRAM init code is only provided under NDA by the
> DRAM controller IP vendor, so OS providers may not be able to
> use it if it isn't already available on the system.

i know. it's a f*****g nuisance. it takes months if not years for
reverse-engineered libre support for various hardware to be available,
by which time the product's dead as far as the manufacturer is
concerned.

this is why i said it will take fabless semi companies putting their
foot down and saying, "um, no, sorry" on this. they can't _possibly_
expect companies to violate the GPL with u-boot and the linux kernel.
despite many of their customers being chinese OEMs who don't give a
flying fuck about Copyright Law, "well our Chinese customers don't
have a problem, so why do you" isn't an acceptable answer!

> * Define a standard partition table layout that is taken into
> account by anything on the system, including the boot
> firmware. My proposal would be GPT because it's flexible,
> supported by about everything and the kitchen sink, and
> supports large volumes (which is a problem with the classic
> BIOS partition table format).

that's almost certainly not going to fly on a first-stage bootloader
(cf: u-boot SPL). as you know they're often looking at under 16k.
loading from a specific raw sector (offset) is about as good as it
gets.

plus, don't forget: with NAND and NOR, you don't have the luxury of
knowing if the IC has built-in error-correction. or, more to the
poin, you *absolutely must not* assume that it has.

this is why for example the allwinner bootrom just does a for-loop,
increasing the offset by fixed amounts, requesting a single block and
looking for the signaure "eGON" at the front. if success, execute.

basically, a standard partition at the front of a raw NAND flash
device is a sure-fire guaranteed way to have a high percentage of
returns on customer product.

> * If a SoC supports loading its raw firmware bitstream from a
> block device (typically SD/eMMC) by loading a certain amount
> of bytes from a certain offset, define _one_ standard offset
> and length where it is to be loaded from, making sure that it
> is compatible with the chosen partition table layout. The
> reason for that is that without such a definition, one would
> again risk incompatibilities with OS, userland applications
> and possible alternative firmwares.

nggggh this *might* be ok to do. i hesitate to say it's a good idea
because of the fact that some raw NAND / NOR devices have different
block sizes, and it may hypothetically be better for an SoC to read on
the boundary of the NAND flash IC, *not* a specific standards-defined
size that doesn't take the possible ICs into account.

TSSOP-48 NAND ICs are a frickin nuisance, however unfortunately the
reduced cost (and sizes they are available in) is a compelling
argument for their use over eMMC. eMMC i haven't been able to easily
find anything less than 4GB, whereas Winbond does 128k TSSOP-48 NAND
ICs and will continue to do so for a long time yet.

> SoCs can usually boot their firmware from multiple places,
> selectable by external pins or in a fixed order. When using
> a fixed order, usually removable storage (SD card) has
> highest priority, then comes eMMC, then SPI NOR flash. This
> is really nice because it allows to easily "unbrick" a broken
> system, e.g. if flashing a new firmware into the onboard NOR
> flash has failed and the system doesn't come up anymore.

*sigh* the RK3288 got this horribly wrong, sadly. the first boot
device is hard-fixed in the boot ROM as eMMC. consequently, if the
eMMC is ever corrupted, you're f*****d. brick. and no way to
recover.

on the *other* hand, if that eMMC is *empty* (zeros), *then* it will
boot from SD/MMC no problem. but.. write a single byte to it, and
you're done. landfill.

so getting this order right is *important*.

> Code is only loaded from a device if it has a valid header
> (usually some magic number and a CRC), so if there is no
> valid header on removeable media because nobody has installed

> [ ...]

> BIOS-style and GPT-style partition tables). In the arm
> world, a number of SoCs have placed their firmware location
> as a 32kB block at 8k offset from the beginning of the
> device. Unfortunately while this works for systems using
> BIOS-style partition tables, it breaks for systems using
> GPT-style partition tables, because those are larger.

yep... see above as to a possible reason why they do that. although
to be honest, it's probably more down to the fact that BIOS-style
partition tables are dead simple but also "were there first".

i have to say: maybe the people who developed the GPT-style
partitions should have consulted people in the embedded industry
before going ahead with such a layout??

also, if you only *have* a 4MB SPI NOR boot iC, if you *absolutely*
have to have a GPT-style partition table (because it's part of the
"standard"), that's ... that's... 25% of the frickin SPI NOR IC
*wasted* on a frickin *partition* table for god's sake! what OEM in
their right mind would *deliberately* design any product that was so
cost-uncompetitive, when $0.05 on the BOM could mean the difference
between market success and failure??

> The
> maximum length of the firmware boot block needs to be
> specified as well, because firmware like u-boot usually uses
> the space between the first-stage boot image and the start of
> the first partition for its own (non-SPL) code and data, so
> it needs to know from which offset onwards blocks can be used
> without interfering.

yep. seen that hard-coded into vendor releases. works great. not
nice to maintain, but "it works".

> For the "high-level" part of boot firmware (the part that
> implements the "user-facing" part of the boot process and
> implements methods to boot from SATA disks, network interfaces,
> provide system configuration options, etc.), finding a reasonable
> standard that covers all use cases is really difficult.

thank god. someone who actually understands the reality of booting
in the embedded space (and more).

> I can understand the "simply go with UEFI and ACPI" argument that
> Richard Jones had made in his talk during the Barcelona workshop,
> but only when limited to the premise that it has been made under,
> i.e. "RedHat is only interested in the server market and nothing
> else".

yyep. there aren't enough people around who actually understand how
completely and utterly different embedded SoCs really are. "surely
the x86 style solution should be enough for everybody, right???"

errr... no :)

> One can surely with good reason argue "UEFI+ACPI is used on x86
> and arm64, do the same for RISC-V" as there is existing support
> for UEFI-based systems in many operating systems and bootloaders.
> On the other hand, there is a lot to be said against UEFI+ACPI,

it's funny... the lesson that comes through loud and clear from what
you write, here, karsten, is every single implementation of a "boot
standard" that's been described has consistently been a total
nightmare. ATF, UEFI, ACPI, *every* single attempt at a standard is
bug-ridden security-unconscious bloatware!

and yet... people have been complaining "the ARM embedded world is
hell! surely it doesn't have to be that complex!!! surely it can be
fixed with a standard!!!!"

by contrast, vendor-proliferated u-boot... um... "just works"...

there's a lesson there. i'm not sure what it *is* at this point, but
there's a lesson there :)

> ACPI is a no-go from my point of view, I've seen way too many
> broken ACPI implementations in the PC world that cannot be fixed
> by the user and in practice usually don't get fixed by the
> manufacturers. Things are different if you "live" in the
> high-end server market (both on x86 and on ARM), but for
> commodity hardware my experience has been rather devastating.
> If one would want to standardize on using UEFI at all, then IMHO
> only in the form of UEFI+device-tree.
>
> For the embedded Linux use case, UEFI+ACPI isn't even worth
> a single thought - from my experience with the embedded Linux
> world on ARM platforms I am sure that there isn't the chance of
> a snowball in hell that platform vendors for the embedded Linux
> market would be willing or capable of providing a reasonable
> UEFI+ACPI implementation for their systems.

nope. or, yup, i agree: not a chance. you can't tell ingenic, for
example, who produce $2.50 1ghz MIPS processors, and now smartwatch
processors with embedded 32MB LPDDR2 RAM, all-in at under $3.50, that
they absolutely have to use ACPI. being chinese you won't find out
their true reaction: the fact that they're under enormous size and
cost pressure in the smartwatch market from their customers should
allow you to infer their response, without needing to contact them.

> U-boot on the other hand is a rather nice firmware and bootloader
> for embedded systems, but it is definitely not designed for the
> server world. For the embedded space, u-boot can surely provide
> a reasonably easy and platform-neutral interface for booting an
> operating system.

despite - or in fact _because_ of - the massive forking and
proliferation. it's a double-edged sword.

> The obvious compromise could indeed be an interface standard that
> defines some stripped-down UEFI boot services as the common
> denominator.

from experience of witnessing the proliferation of u-boot (and
understanding why that is) i'd honestly say that even this fills me
with... what's the word... alarm / skepticism / cynicism / realism /
other.

to stand a chance in the embedded space, a proposal has to be
justified in terms of code size, boot speed, and, ultimately, sale
price of the end product.

anything that increases the final sale price (not of the SoC, not of
the product to the users, but the product - the *full* product - to
the *OEMs*) is automatically going to be discared.

so, if part of the proposal to include UEFI is a mandatory
requirement that imposes an extra $1.50 on the BOM because certain
types of non-UEFI-compliant external boot peripherals are also
mandatory, and they're more expensive, or if the fabless semi company
has to shell out for hardware encryption RTL, significantly increasing
NREs in the process, you can kiss any such proposal's acceptance
goodbye.

> This would allow hardware vendors to choose between
> various existing solutions, i.e. between using either some
> Tianocore-based UEFI implementation with their own HAL, using
> coreboot for the lower layer with a Tianocore-based payload
> providing the UEFI interface to the outside, or using a
> u-boot-based implementation with u-boot's UEFI emulation layer.
> The latter would provide the standardized interface for embedded
> use cases without significant cost (the code size increase due to
> the emulation layer is rather negligible).

that's good... the real key question is: does it impose on the BOM?
does it prevent and prohibit the OEMs from choosing specific boot
storage devices? is there a hardware-related protocol that is an
absolute requirement to support u-boot's UEFI layer? is there a
hidden cost in terms of the amount or format of storage that UEFI
requires, which would impact the sale price of the product?

also, now that i've read the bits about GPT, is UEFI *only* possible
on top of GPT? and: is there expected to be encryption or signing,
*without* which firmware is not loadable? because if so, that's a
"no" because it imposes a hardware (and development) cost on the
vendor.

i trust that people reading this now have had their eyes opened a bit,
thanks to these questions, spurred by karsten's extremely valuable
response. the embedded (RV32, <48k and <1MB) world hasn't even begun
to be discussed in depth, either, yet.

i trust that people now see why i said that restrictive
standardisation in the area of boot loading and boot initialisation is
a near-impossible task, and why it is actually in most cases even
undesirable.

it really is far more complex and comprehensive than most people ever
dream it could possibly get.... and yet there are good reasons why it
is that way.

l.

[Samuel Falvo II]

Sorry, this was originally intended to be sent to the list as a whole,
not to one person.

On Sat, Oct 13, 2018 at 4:12 PM Luke Kenneth Casson Leighton
<lk...@lkcl.net> wrote:
> yep... see above as to a possible reason why they do that. although
> to be honest, it's probably more down to the fact that BIOS-style
> partition tables are dead simple but also "were there first".
>
> i have to say: maybe the people who developed the GPT-style
> partitions should have consulted people in the embedded industry
> before going ahead with such a layout??
>
> also, if you only *have* a 4MB SPI NOR boot iC, if you *absolutely*
> have to have a GPT-style partition table (because it's part of the
> "standard"), that's ... that's... 25% of the frickin SPI NOR IC
> *wasted* on a frickin *partition* table for god's sake! what OEM in
> their right mind would *deliberately* design any product that was so
> cost-uncompetitive, when $0.05 on the BOM could mean the difference
> between market success and failure??

BIOS-style partition tables are both an incredibly short-sighted
design as well as hardly the limit of the technology of the day. The
Commodore-Amiga supported more than 4 partitions, larger partition
sizes, and even OS/2-like per-partition filesystems back in 1987. I
refer you to their Rigid Disk Block structure, which formed the head
of a linked list of partitions, each defined by a single-sector
Partition Block. Since it used linked lists, you could have as many
partitions as you wanted. Partition blocks also stored preferred
device names and volume names as well, allowing the Kickstart ROM's
built-in boot loader to provide a Grub-like boot menu as well.

RDB/PBs have 32-bit limitations these days, so they're technically
obsolete as implemented on Kickstart. However, the spirit of the
technology is sound, and I believe might work better for an embedded
environment than GPT would, particularly if you're concerned with the
ratio of storage space to partition metadata.

[Luke Kenneth Casson Leighton]

On Sat, Oct 13, 2018 at 10:29 PM <kr...@berkeley.edu> wrote:

> In this thread, as is common when engaging a wide community of smart
> experienced people, many of the disagreements are a result of people
> having different, even if possibly overlapping, subsets of the
> computing universe in mind. It would be helpful if everyone qualified
> their position with the systems they're assuming (microcontroller with
> no external DRAM, or servers with 8 sockets and terabytes of DRAM) and
> especially if we could introduce a common taxonomy/vocabulary, so
> people would be less likely to talk past each other.

from so many different backgrounds, that's quite challenging. i feel
that if people genuinely share the same goal, they'll work through
issues, to find the best ways to successfully communicate.

> I think we can all agree that reducing unnecessary duplication of
> effort is a good thing,

it's... desirable, absolutely. whether time and cost constraints on
NREs make fabless semi companies actually take pre-emptive *action*,
as opposed to just forking a codebase, throwing it at some engineers
and saying "get this into shape before the VC funding runs out, or
you're fired", is another matter.

> and both standards and open-source code are
> great and complementary ways to achieve that goal.

this is where i feel that taking an early lead, and *getting it
right*, will be so absolutely critical. identifying the failures
(proprietary binary-only distribution of "open" source firmware,
resulting in the persistent endemic security-related nightmares that
ron described so well) and the reasons why (use of BSD licenses by the
original authors because of corporate sponsorship - google f****g
about re-implementing perfectly good GNU libraries for android is by
far and above the absolute worst offender here) will help lay a path
that is followed if only because it's cheaper for SoC fabless semi
companies to do so.

> I think we _should_ all agree that there is necessary fragmentation
> because different platforms have different needs.

basically. yes. there's really no point trying to assert otherwise.
standards *only work* where and when there is common ground. if there
*is* no common ground...

> While some systems
> might never trust vendor-supplied M-mode software, there are other
> systems that would never trust a Linux kernel to install/execute
> M-mode code.

yyeahh i think the case that ron gave could be equally well applied
for {insert any OS or kernel} as it does for linux.

> Similarly, in structuring code bases, some people might
> want their Linux to know nothing of what's underneath to support
> flexible virtualization, while others want deep control of the
> hardware, e.g., for real-time concerns.

ind