Upstreaming binutils/gcc RV32E support

[Jim Wilson]

Now that gcc-8 has been released, I'm planning to upstream the
binutils and gcc support for RV32E that has been in the
riscv-gnu-toolchain tree since last August of so. There won't be a
binutils release until the end of the year, and not a gcc release
until next spring, so we will have some time to fix issues if
necessary before they appear in official releases.

The newlib RV32E support is already upstream, and has already appeared
in newlib releases, which means that we can't easily make changes
without the risk of creating backward compatibility problems.

There are also gdb sim patches, but that hasn't been upstreamed yet.
And I have a small hack to qemu so that "make check" in the
riscv-gnu-toolchain tree can work.

The RV32E support introduces some new binutils and gcc options. There
is now a -march=rv32e option to specify the RV32E architecture, and a
-mabi=ilp32e option to specify the RV32E ABI.

The ilp32e ABI has minimal changes from the official ABI. RV32E has
only 6 of the 8 argument registers, so we only use 6 registers for
arguments. The stack has 4-byte alignment, so we can't give 8-byte
alignment to 8-byte arguments on the stack. System call numbers are
normally passed in a7, which doesn't exist, so they are now passed in
t0. There is a new macro defined, __riscv_32e, so you can check for
rv32e at compile time. I think that is all of the user visible
changes.

One question here is whether we should try to make this a formal ABI,
or just let it be an unofficial ABI that gcc happens to have support
for. Since there is already some discussion about an embedded ABI
(aka EABI) for RISC-V, that will likely replace this ABI in the near
future, it isn't clear how formally we need to handle this new ilp32e
ABI. If this does need to be handled more formally, then what is the
process for that?

Jim

[Liviu Ionescu]

On 10 May 2018 at 22:49:09, Jim Wilson (ji...@sifive.com) wrote:

> One question here is whether we should try to make this a formal
> ABI,
> or just let it be an unofficial ABI

I think that this should be decided in the context of a RISC-V
microcontroller profile (both RV32I/RV64I), which normally should have
an optimised ABI, with fewer registers saved by the caller.

In this context, the RV32E ABI should be consistent with the 'bigger'
microcontroller ABI.


Regards,

Liviu

[Anton Krug]

The newlib might need a revisit, some while ago there were issues found when single floating point operations had dependencies on double precision software libs only to generate exceptions. Some of them were fixed by ye SiFive guys after I reported it, bu then I noticed there are still some inefficiencies, which wouldn't matter for big cores with a lot of rom/ram, but might be significant in context of small embedded devices.

[Jim Wilson]

On Thu, May 10, 2018 at 1:05 PM, Anton Krug <anton...@gmail.com> wrote:
> The newlib might need a revisit, some while ago there were issues found when
> single floating point operations had dependencies on double precision
> software libs only to generate exceptions. Some of them were fixed by ye
> SiFive guys after I reported it, bu then I noticed there are still some
> inefficiencies, which wouldn't matter for big cores with a lot of rom/ram,
> but might be significant in context of small embedded devices.

Duly noted. But this is a code size issue, not an ABI issue, so this
isn't relevant to the RV32E ABI problem.

Jim

[Jim Wilson]

This is of course how we should handle this in the future, but doesn't
solve my immediate problem. We've got code that has been in the
github riscv-gnu-toolchain tree for 8 months that hasn't been
upstreamed yet. I believe that this code is in use, as we get the
occasional bug report against it. The longer it stays in the github
trees, the more trouble it causes for me. The RV32E support is in
fact the only binutils and gcc patches that haven't been upstreamed
yet. I'd like to fix this sooner rather than later. Waiting for an
official RV32E ABI, or waiting for some new microcontroller EABI, may
take too long.

But maybe this is a vote to consider the current support an unofficial
ABI? I suppose I could make gcc emit a warning when someone chooses
to use the rv32e/ilp32e support, as the rv32e spec isn't final yet,
and the ilp32e ABI is not an official ABI.

Jim

[Tommy Murphy]

On Friday, 11 May 2018 02:09:07 UTC+1, Jim Wilson wrote:

But maybe this is a vote to consider the current support an unofficial
ABI?  I suppose I could make gcc emit a warning when someone chooses
to use the rv32e/ilp32e support, as the rv32e spec isn't final yet,
and the ilp32e ABI is not an official ABI.

Surely there's no vote or doubt about it - it IS inherently unofficial because rv32e has not been officially standardised yet?

Or am I missing something here?

[Khem Raj]

I think if ABI is not final yet then we should wait on upstreaming it. Since
once upstreamed, its kind of unsaid ABI then onwards.

[Jim Wilson]

RV32E hasn't been frozen yet, but I'm told that there are no changes
expected at this point, except for the F/D compressed instructions,
that currently aren't valid in RV32E, but may be redefined as valid
later. Adding support for RV32E minus these instructions doesn't
create any compatibility problems so is safe.

I'm not asking for any votes here. I'm only asking for advice. We
have RV32E support sitting on a branch, and it is getting in the way
of development work. I want to solve the problem by upstreaming it.

Jim

[Jim Wilson]

On Thu, May 10, 2018 at 6:14 PM, Khem Raj <raj....@gmail.com> wrote:
> I think if ABI is not final yet then we should wait on upstreaming it. Since
> once upstreamed, its kind of unsaid ABI then onwards.

The code is a maintenance burden for me. The longer we defer
upstreaming it, the more of a burden it is. I need to fix this. I
would much rather be fixing real bugs that dealing with problems
caused by non-upstreamed RV32E code.

It is already an unsaid ABI, as it is already in the github tree, and
people are already using it from there. Upstreaming doesn't change
this, but does make the unsaid ABI more visible. Also, note that the
newlib support was already upstreamed, so there are already references
to this unsaid ABI in upstream code, though the newlib mentions aren't
very significant.

Jim

[Khem Raj]

On Thu, May 10, 2018 at 6:32 PM, Jim Wilson <ji...@sifive.com> wrote:
> On Thu, May 10, 2018 at 6:14 PM, Tommy Murphy <tommy_...@hotmail.com> wrote:
>> On Friday, 11 May 2018 02:09:07 UTC+1, Jim Wilson wrote:
>>>
>>> But maybe this is a vote to consider the current support an unofficial
>>> ABI? I suppose I could make gcc emit a warning when someone chooses
>>> to use the rv32e/ilp32e support, as the rv32e spec isn't final yet,
>>> and the ilp32e ABI is not an official ABI.
>>
>> Surely there's no vote or doubt about it - it IS inherently unofficial
>> because rv32e has not been officially standardised yet?
>> Or am I missing something here?
>
> RV32E hasn't been frozen yet, but I'm told that there are no changes
> expected at this point, except for the F/D compressed instructions,
> that currently aren't valid in RV32E, but may be redefined as valid
> later. Adding support for RV32E minus these instructions doesn't
> create any compatibility problems so is safe.
>

Thanks Jim, if this is the case then I assume there wont be any
ABI issues, probably its fine to upstream it with some warnings as
you suggested.

[Jim Wilson]

On Thu, May 10, 2018 at 7:50 PM, Khem Raj <raj....@gmail.com> wrote:
> Thanks Jim, if this is the case then I assume there wont be any
> ABI issues, probably its fine to upstream it with some warnings as
> you suggested.

Another option here is to put it on a branch in the FSF tree instead
of on mainline. This mostly solves my immediate problem which is that
the riscv-next branch in the github tree where the patch lives has
gotten so confused that I can't easily track the patch anymore. This
also gives us some time to discuss the ABI issues.

I forgot to mention at the beginning, the patch set also includes a
change to add a new ELF header flag to mark binaries using the ilp32e
ABI. There are a limited number of these bits, so we need to be
careful about this. If we run out of bits, we can always switch to
using ELF attributes instead of ELF header flags.

Jim

[Tommy Murphy]

Fair enough Jim.

I appreciate the difficulty in trying to manage ongoing development with this stuff getting in the way.

[Richard Herveille]

The current ABI seems based on the existing non-E Linux ABI, which has way to much overhead for an embedded profile (which ‘E’ stands for).

There’s no vote, but if there were I would vote for waiting until the embedded ABI is finished before continuing much work on the current RV32E ABI.

Maybe parking it as a branch is the right approach?

 

Richard

 

 

 

Richard Herveille

Managing Director

Phone +31 (45) 405 5681

Cell +31 (6) 5207 2230

richard....@roalogic.com

 

--

You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.

Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAG7hfc%2BCiGYRG%2B_WNW5tWy3mzGcx6F_rdDz3VFM98e4vVE_%3Dfg%40mail.gmail.com.

 

[Liviu Ionescu]

On 11 May 2018 at 13:10:09, Richard Herveille

(richard....@roalogic.com) wrote:

> The current ABI seems based on the existing non-E Linux ABI,
> which has way to much overhead for an embedded profile (which
> ‘E’ stands for).

agree.

to be useful, the RISC-V EABI needs a responsible redesign, and cover
all 3 cases (RV32I/RV64I/RV32E).

the current approach to bend the microcontroller profile after the
privileged profile is not beneficial.


regards,

Liviu

[Michael Clark]

The C ABI has no dependency on the privileged isa or a potential Linux ABI. Newlib, Newlib-nano, musl and GLIBC all share the same C ABI for a given ISA and ABI. The C ABI doesn’t even depend on CSRs (they are handled as inline asm).

It boils down to the calling convention, which includes soft-float (which you don’t need to use), the classification of special purpose register (ra, sp, gp, tp), the sets of callee vs caller saved registers, the ELF format differences (which you don’t need to use if you if you are extracting flat binaries), the shared library conventions as the registers available to the PLT are different (you don’t need to use shared libraries) and things like the definition of long double.

[Liviu Ionescu]

On 11 May 2018 at 16:44:41, Michael Clark (michae...@mac.com) wrote:

> ... the sets of callee vs caller saved registers

right. interrupt latency impacted by too many caller saved registers.


regards,

Liviu

[Liviu Ionescu]

On 11 May 2018 at 13:10:09, Richard Herveille
(richard....@roalogic.com) wrote:

> ... waiting until the embedded ABI is finished

for the EABI to be ever finished, it first needs... to be started.

the microcontroller profile proposal asked for an EABI, but this does
not mean it is on the agenda of any working group.


regards,

Liviu

[Richard Herveille]

Well we really need to start the platform committee.
Find some volunteers and we’ll put in a request to the foundation.

Richard


Sent from my iPhone

> --
> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> To post to this group, send email to sw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAG7hfcJQTj2R2_PXzogtpn1tOOnNpupgcArTf%2B-YmyB1C67nTw%40mail.gmail.com.

[Alex Bradbury]

Hi Jim, thanks for starting this discussion.

I have a couple of concerns with the upstream of the current RV32E ABI support:
* The RV32E ABI hasn't really been finalised, and as Krste announced
at the RISC-V workshop there seems to be an intention to define a new
embedded ABI. Using up the 'RV32E' name for an ABI that hasn't been
considered 'official' doesn't seem like the right thing to do.
* It's still the case that nobody has taken the time to document the
RV32E ABI to at least the same level as the other RISC-V ABIs (see the
tracking issue https://github.com/riscv/riscv-elf-psabi-doc/issues/62).

I did raised these concerns previously
<https://github.com/riscv/riscv-gcc/pull/77#issuecomment-338420220>
and I thought there was an agreement about the need to at least
document the ABI before seeing it committed upstream, so it's quite
unfortunate that it was committed to upstream newlib. However given
that that RV32E has never been supported in upstream GCC, hopefully
there's still enough flexibility to deprecate it in favour of a new
RV32E ABI without too much disruption.

If you're really keen on upstreaming the current rv32e implementation,
is there any chance it could use a more offputting flag at least until
an ABI is finalised. e.g. requiring -mexperimental-e-abi to be set in
order to use -mabi=ilp32e? Or requiring -mabi=ilp32e_experimental?

It seems there are two possibilities for RV32E:
1) The eventual embedded ABI is going to be essentially what is
already implemented in GCC for -mabi=ilp32e (or incredibly close). If
that's the case, let's document it properly and get it finalised as
quickly as possible.
2) The eventual embedded ABI is going to be quite different to what is
already implemented in GCC for -mabi=ilp32e. If that's the case,
pushing out the current implementation risks creating confusion and
fragmentation.

Best,

Alex

[Ronan BARZIC]

On Thursday, May 10, 2018 at 9:49:08 PM UTC+2, Jim Wilson wrote:

.....

The newlib RV32E support is already upstream, and has already appeared
in newlib releases, which means that we can't easily make changes
without the risk of creating backward compatibility problems.

There are also gdb sim patches, but that hasn't been upstreamed yet.
And I have a small hack to qemu so that "make check" in the
riscv-gnu-toolchain tree can work.

....

Is there a specific branch or tag  that needs to be used to  compile the toolchain with RV32E support ?

 I've just tried master branch (../configure --with-arch=rv32e --prefix=/opt/riscv32e) but it fails at configure-gcc in build-gcc-newlib-stage1

Regards

-Ronan

[Tommy Murphy]

On Monday, 14 May 2018 21:53:13 UTC+1, Ronan BARZIC wrote:

Is there a specific branch or tag  that needs to be used to  compile the toolchain with RV32E support ?

 I've just tried master branch (../configure --with-arch=rv32e --prefix=/opt/riscv32e) but it fails at configure-gcc in build-gcc-newlib-stage1

You mean the master branch from here?

https://github.com/riscv/riscv-tools

How did it fail - what error message(s)? 

[Jim Wilson]

On Mon, May 14, 2018 at 1:53 PM, Ronan BARZIC <rba...@gmail.com> wrote:
> Is there a specific branch or tag that needs to be used to compile the
> toolchain with RV32E support ?
> I've just tried master branch (../configure --with-arch=rv32e
> --prefix=/opt/riscv32e) but it fails at configure-gcc in
> build-gcc-newlib-stage1

The RV32E support is on the riscv-next branch, which in theory holds
patches that are meant to be upstreamed. You need to check out the
riscv-next branch in the riscv-binutils-gdb and riscv-gcc
subdirectories. I found a few minor bugs when testing it, but the
stuff on the branches should build and work. The binaries will only
run in the gdb simulator, not in qemu, as qemu doesn't have support
for the unofficial ILP32E ABI supported by these patches, but the qemu
problem can be fixed with a trivial workaround.

I'm planning to create some new branches to hold the fixed RV32E patches.

Jim

[Michael Clark]

There is a pull request in the riscv-qemu GitHub repo that adds the ‘E’ extension ABI to qemu-user however it wasn’t applied because it doesn’t update translate.c to check that registers fall within x0-x15. The patch also needs to be forward ported to latest QEMU because there have been many changes since the patch was first created.

The change in the PR adds a check for elf_flags EF_RISCV_RVE (8) and uses t0 instead of a7 for the syscall number in ECALL. The change in the PR is limited to QEMU’s linux-user emulation. Someone would need to clarify whether these are the only changes necessary for the RVE Linux ABI.

I’ve recently added more complete misa extension checking support to qemu/target/riscv/translate.c so that we now check ‘M’, ‘A’, ‘F’, ‘D’ and ‘C’ extensions and generate illegal instruction exceptions when decoding instructions whose extension is not set. Previously only the ‘C’ extension was enforced. This is one step requires for properly emulating ‘E’.

The remaining change to support ‘E’ requires adding register bounds checks to all register operands in the ‘I’ set of instructions (only the Base Integer ISA needs checks because all other extensions must be disabled if ‘E’ is present in misa). We have a placeholder for the ‘E’ checks in write_misa. We should be able to add ‘E’ support to QEMU relatively quickly if there is demand for it.

System mode is less of a worry because we don’t have to worry about ABIs. We just have to restrict the set of available registers to x0-x15.

Its possible to run RVE code in qemu-system-riscv32 now, however there won’t be any illegal instruction exceptions generated if the code accesses x16-x31.

[Jim Wilson]

On Tue, May 15, 2018 at 4:55 AM, Michael Clark <michae...@mac.com> wrote:
> The change in the PR adds a check for elf_flags EF_RISCV_RVE (8) and uses t0 instead of a7 for the syscall number in ECALL. The change in the PR is limited to QEMU’s linux-user emulation. Someone would need to clarify whether these are the only changes necessary for the RVE Linux ABI.

Yes, this is all that is necessary. This is what my quick hack does.
I didn't think to check for an outstanding pull request before writing
my own patch. Looking now, I see that the PR looks badly confused, by
merges or rebases or whatever, so it isn't clear what the patch is
doing.

Jim

[Ronan BARZIC]

On Tuesday, May 15, 2018 at 12:41:13 AM UTC+2, Jim Wilson wrote:

....

The RV32E support is on the riscv-next branch, which in theory holds
patches that are meant to be upstreamed.  You need to check out the
riscv-next branch in the riscv-binutils-gdb and riscv-gcc
subdirectories.

Thank you

I can confirm that I got a successful  build in riscv-gnu-toolchain by using riscv-next for riscv-binutils-gdb and riscv-gcc

For the record, I used the following command :

../configure --with-arch=rv32e --with-abi=ilp32e --prefix=/opt/riscv32e 

-Ronan

[Jim Wilson]

An upper respiratory viral infection slowed me down a bit, I'm now
back to looking at this.

On Mon, May 14, 2018 at 7:30 AM, Alex Bradbury <a...@lowrisc.org> wrote:
> I have a couple of concerns with the upstream of the current RV32E ABI support:
> * The RV32E ABI hasn't really been finalised, and as Krste announced
> at the RISC-V workshop there seems to be an intention to define a new
> embedded ABI. Using up the 'RV32E' name for an ABI that hasn't been
> considered 'official' doesn't seem like the right thing to do.

RV32E is the architecture name. It is called RVE ABI in the ABI docs,
to distinguish it from the RVXXI ABIs.

ilp32e is the current -mabi= option name for the ABI. Since we
already have ilp32f for the F extension and ilp32d for the D
extension, using ilp32e for the E extension makes sense.

The new embedded ABI will be parallel to the existing Linux ABI. So
both the embedded ABI and Linux ABI support all of rv32e/rv32i/rv64i.
The new ABI is not a replacement for the current ilp32e ABI, but will
be an addition to it. So we need new ELF header flags and option
names even if we wanted to drop the current RVE ABI.

Palmer is back from the RISC-V meeting, and I confirmed with him that
it was always the intent that this would be upstreamed when possible.

My main concern here is that I would like to upstream the RV32E
architecture support, to reduce the maintenance burden, and to make it
available to others. But I can't upstream it without some minimal ABI
support, and since we already have the linux RVE ABI (somewhat)
defined, and it was always the intention to upstream this code, I
don't see any major problem with upstreaming. The next gcc release
doesn't happen for a year, so we have some room to make some changes
if necessary.

> * It's still the case that nobody has taken the time to document the
> RV32E ABI to at least the same level as the other RISC-V ABIs (see the
> tracking issue https://github.com/riscv/riscv-elf-psabi-doc/issues/62).

Yes, we need better documentation for this.

> It seems there are two possibilities for RV32E:
> 1) The eventual embedded ABI is going to be essentially what is
> already implemented in GCC for -mabi=ilp32e (or incredibly close). If
> that's the case, let's document it properly and get it finalised as
> quickly as possible.
> 2) The eventual embedded ABI is going to be quite different to what is
> already implemented in GCC for -mabi=ilp32e. If that's the case,
> pushing out the current implementation risks creating confusion and
> fragmentation.

I expect that the new embedded ABI is going to be quite different,
that it will support rv32e/rv32i/rv64i, and that it will not replace
the currrent linux ABI support. It will be like when ARM changed
their ABIs, and changed their configure triplet from arm-elf to
arm-eabi. We will just have to support both ABIs moving forward.
Though it isn't clear if we want to change our configure triplet or do
something else to indicate use of the new embedded ABIs. It would be
nice if we can generate all ABIs from a single toolchain.

Jim

[Arnd Bergmann]

Would you expect any operating system to support more than
one of the two ABIs, or do you forsee that embedded ABI to only
be used on small (non-Linux) RTOSs?

The experience with two ABIs on ARM Linux was actually quite
horrible, it would be nice if we could avoid duplicating that here.

It would be good to at least limit the number of possible ABI variations
on a given hardware+OS platform to a minimum. On RV64I-Linux,
I would very much hope to never see two incompatible ELF ABIs
supported by the kernel, and if we end up supporting RV32E with
mainline Linux, that also should not need two incompatible ABIs.

For running RV32E binaries on a RV32I of RV64I Linux kernel
(if there is a need for that), the kernel would have to support both
binary formats at once, and minimizing the differences between
those two would certainly help, and as soon as you get into
different structure layouts etc it quickly becomes unmanageable.

Arnd

[Jim Wilson]

On Wed, May 16, 2018 at 8:09 PM, Arnd Bergmann <ar...@arndb.de> wrote:
> Would you expect any operating system to support more than
> one of the two ABIs, or do you forsee that embedded ABI to only
> be used on small (non-Linux) RTOSs?

I would expect that linux continues to use the current ABI. There is
no reason to change. We froze that ABI when we upstreamed glibc.

The new ABI is only meant to be used for embedded development. There
are some problems with using the current ABI in embedded environments,
and the new ABI is meant to fix that. This new ABI might be useful
for small RTOSs. It would be up to each RTOS to choose which ABI they
want. I wouldn't expect any to support both.

I am well aware that multiple ABIs are a problem. I was a MIPS GCC
maintainer for a number of years. We had lots of different ABIs. It
was a problem that later people tried to solve by adding even more
ABIs, which I thought was an odd way to solve the problem. Right now
for RISC-V we are just planning to have a set of ABIs for linux
RV32E/RV32I/RV64I, which is the current ABI, and a set of ABIs for
embbedded RV32E/RV32I/RV64I, which is in the process of being defined.

> For running RV32E binaries on a RV32I of RV64I Linux kernel
> (if there is a need for that), the kernel would have to support both
> binary formats at once, and minimizing the differences between
> those two would certainly help, and as soon as you get into
> different structure layouts etc it quickly becomes unmanageable.

There is already effectively a linux RV32E ABI, which is the same as
the linux RV32I ABI with some minor changes due to missing registers,
so only 6 argument registers instead of 8, and syscall number in t0
instead of a7. There is also the fact that the stack has only 4 byte
alignment, but while that affects alignment of structs on the stack,
and alignment of long long and long double arguments on the stack, I
don't think it affects structure sizes or layout, as the natural
alignment of types isn't changing.

Jim

[Bruce Hoult]

I think the currently-implemented RV32E ABI isn't horrible. In fact I think the syscall number in t0 is *better* than a7 and we should do that in the non-E ABIs too. Too late?

I think the only thing I'd consider changing would be to reduce the number of function arguments from six to four. ARM gets away with four, and much embedded code is already written around that. I'd re-purpose x14-15 as s3-4. Two s registers isn't enough.

This would also reduce the number of registers an interrupt handler needs to save before calling normal compiled C code from ten to eight: ra, a0-3, t0-2.

The Linux kernel allows a maximum of six arguments (in addition to the syscall number) on every architecture. I believe no current syscall actually uses more than four. If some future one needs six I think I'd pass them in t1 & t2 -- for all RISC-V, not only E. This would not affect any current code or binaries. (moving the syscall from a7 to t0 wouldn't of course be an incompatible change, but there are currently very few Linux users, so it's far easier to do now than in even six months from now)

For simplicity, I'd allow embedded code on non-E systems to use the E ABI, with x16-31 all extra s registers.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAK8P3a0yA7XSHREHUNwwrbVX8hx8mbC7E%2BVsFfM_q5OQqJBmXw%40mail.gmail.com.

[Bruce Hoult]

Actually, no, I changed my mind a little. x14 & x15 as s2 & s3 in RVE, yes (oops, I wrote s3-4 before, sorry). But pass hypothetical future 5th and 6th syscall arguments there, not in t1 & t2. Same place as RVI does now. If you've got the overhead of making a syscall anyway, you can afford to save two more of your caller's registers first.

To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

[Jim Wilson]

On Wed, May 16, 2018 at 9:05 PM, Bruce Hoult <bruce...@sifive.com> wrote:
> I think the currently-implemented RV32E ABI isn't horrible. In fact I think
> the syscall number in t0 is *better* than a7 and we should do that in the
> non-E ABIs too. Too late?

For the current linux ABI, yes, it is too late to change.

> I think the only thing I'd consider changing would be to reduce the number
> of function arguments from six to four. ARM gets away with four, and much
> embedded code is already written around that. I'd re-purpose x14-15 as s3-4.
> Two s registers isn't enough.

This has already been suggested for the new ABI.

> This would also reduce the number of registers an interrupt handler needs to
> save before calling normal compiled C code from ten to eight: ra, a0-3,
> t0-2.

For this reason.

> For simplicity, I'd allow embedded code on non-E systems to use the E ABI,
> with x16-31 all extra s registers.

This has also already been suggested for the new ABI.

Jim

[Gnanasekar R]

Hi.. in order to build RV32E should I checkout branch riscv-next under riscv-gcc and  riscv-binutils-gdb and build the riscv-gnu-toolchain? Is that enough.

Just wanted to check because I do not see any recent checkins in the riscv-next branch. So should I take any other patch to bring it up to date?

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAFyWVaY1kRdQu0HLmZgxfC2Zsi%2BTb_ROcqWT7rq2qum%3DX9-QtQ%40mail.gmail.com.

[Bruce Hoult]

On Thu, May 17, 2018 at 4:58 PM, Jim Wilson <ji...@sifive.com> wrote:

On Wed, May 16, 2018 at 9:05 PM, Bruce Hoult <bruce...@sifive.com> wrote:
> I think the currently-implemented RV32E ABI isn't horrible. In fact I think
> the syscall number in t0 is *better* than a7 and we should do that in the
> non-E ABIs too. Too late?

For the current linux ABI, yes, it is too late to change.

No entirely convinced it's too late to change just the register the syscall number is in, if there is a will to change it.

For the most part, it should affect only the kernel itself and probably one shared library. Could be done on a kernel version change.

It could if necessary be signalled as a different ABI in the ELF attributes, but if you're going to do one for EABI anyway, why not make it a completely shared ABI (as far as the kernel interface is concerned) and deprecate the old one?

 

> I think the only thing I'd consider changing would be to reduce the number

> of function arguments from six to four. ARM gets away with four, and much
> embedded code is already written around that. I'd re-purpose x14-15 as s3-4.
> Two s registers isn't enough.

This has already been suggested for the new ABI.

> This would also reduce the number of registers an interrupt handler needs to
> save before calling normal compiled C code from ten to eight: ra, a0-3,
> t0-2.

For this reason.

> For simplicity, I'd allow embedded code on non-E systems to use the E ABI,
> with x16-31 all extra s registers.

This has also already been suggested for the new ABI.

Good to know GMTA on the rest. 

[Michael Clark]

We should probably apply it. The fact that it doesn’t enforce the use of non-E registers is not strictly a problem. I’ll go ahead and merge it…

[Michael Clark]

Okay. The riscv-qemu github version now supports the RVE syscall convention for linux-user if the RVE ELF flag is present.

- https://github.com/riscv/riscv-qemu/commit/3a6776ce09f5fb0daaec71af1b00504782b14861

The patch is in the ‘qemu-2.13-for-upstream’ branch which will be posted to qemu-devel. it is also merged into the default ‘riscv-all' branch.

I haven’t back-ported the change to the riscv-qemu-2.12, so you need to be on ‘riscv-all' or ‘qemu-2.13-for-upstream'.

[Christoph Hellwig]

On Thu, May 17, 2018 at 04:05:37PM +1200, Bruce Hoult wrote:
> I think the currently-implemented RV32E ABI isn't horrible. In fact I think
> the syscall number in t0 is *better* than a7 and we should do that in the
> non-E ABIs too. Too late?

For the Linux syscall ABI: yes.

> The Linux kernel allows a maximum of six arguments (in addition to the
> syscall number) on every architecture. I believe no current syscall
> actually uses more than four.

There are quite a few 6 argument syscall. In fact we have syscalls
that logically have 7 arguments, but need to use a pointer indirection.

ppoll is an example for a "normal" 6 argument syscall, pselect is an
example for the indirection, as is io_pgetevents that I just added.

[Alex Bradbury]

On 16 May 2018 at 18:19, Jim Wilson <ji...@sifive.com> wrote:
> An upper respiratory viral infection slowed me down a bit, I'm now
> back to looking at this.

I hope you're making a rapid recovery.

> On Mon, May 14, 2018 at 7:30 AM, Alex Bradbury <a...@lowrisc.org> wrote:
>> I have a couple of concerns with the upstream of the current RV32E ABI support:
>> * The RV32E ABI hasn't really been finalised, and as Krste announced
>> at the RISC-V workshop there seems to be an intention to define a new
>> embedded ABI. Using up the 'RV32E' name for an ABI that hasn't been
>> considered 'official' doesn't seem like the right thing to do.
>
> RV32E is the architecture name. It is called RVE ABI in the ABI docs,
> to distinguish it from the RVXXI ABIs.
>
> ilp32e is the current -mabi= option name for the ABI. Since we
> already have ilp32f for the F extension and ilp32d for the D
> extension, using ilp32e for the E extension makes sense.
>
> The new embedded ABI will be parallel to the existing Linux ABI. So
> both the embedded ABI and Linux ABI support all of rv32e/rv32i/rv64i.
> The new ABI is not a replacement for the current ilp32e ABI, but will
> be an addition to it. So we need new ELF header flags and option
> names even if we wanted to drop the current RVE ABI.

Fair enough, so we'll be seeing something like -mabi=eilp32, eilp32e
etc rather than just repurposing the existing -mabi=ilp32e.

However if the existing -mabi=ilp32e is considered to be the "Unix
ABI" in contrast to the yet-to-be-defined embedded ABI that makes me
question even more why it should be upstreamed and supported more
widely instead of deprecated. RV32E was introduced for a very very
size-constrained deeply embedded system. Supporting -mabi=ilp32e makes
little sense of Unix platforms, and for embedded targets people will
surely want to use the new embedded ABI.

> Palmer is back from the RISC-V meeting, and I confirmed with him that
> it was always the intent that this would be upstreamed when possible.
>
> My main concern here is that I would like to upstream the RV32E
> architecture support, to reduce the maintenance burden, and to make it
> available to others. But I can't upstream it without some minimal ABI
> support, and since we already have the linux RVE ABI (somewhat)
> defined, and it was always the intention to upstream this code, I
> don't see any major problem with upstreaming. The next gcc release
> doesn't happen for a year, so we have some room to make some changes
> if necessary.

I appreciate that, but supporting this ABI in upstream GCC makes it
hard to ever retire and puts a pressure on the rest of the RISC-V
world to support it. Based on what you've said about the plans for the
'embedded' ABI it really doesn't feel like the current -mabi=ilp32e
ABI is something that people should ever be encouraged to use. It's
possible I've misinterpreted things, as the only information I have on
this embedded ABI effort is what has been said in this thread and in
Krste's talk at the workshop last week. I've tried to clarify where
development and discussion of this new embedded ABI will be taking
place, but am having no luck in getting any answers so far....

If we truly have another year before having to consider any
-mabi=ilp32e support frozen that does change things somewhat. In the
LLVM community it's not uncommon for downstream parties to fork trunk
at an arbitrary point so we don't have that freedom in the same way. I
assume this doesn't happen or is strongly discouraged in the GCC
world?

Best,

Alex

[Liviu Ionescu]

On 17 May 2018 at 11:31:34, Christoph Hellwig (h...@lst.de) wrote:

> On Thu, May 17, 2018 at 04:05:37PM +1200, Bruce Hoult wrote:
> > I think the currently-implemented RV32E ABI isn't horrible. In fact I think
> > the syscall number in t0 is *better* than a7 and we should do that in the
> > non-E ABIs too. Too late?
>
> For the Linux syscall ABI: yes.

Are the RV32E devices intended to run Linux?


Liviu

[Bruce Hoult]

That question is irrelevant to the function call ABI.

RV32E is intended for machines so small that implementing the full 32 registers is seen as a burden. Inevitably, someone will create a code base on such a machine, and then want to run the same code on a more powerful implementation, possibly with a Linux-like operating system.

Some people (e.g. you, as I understand it) also want an "embedded" ABI to run on machines with the full 32 registers, but with a reduced interrupt latency by having fewer registers that must be saved by an interrupt routine before calling compiled C code. The RV32E *ABI* seems suitable for that, even on a machine with 32 registers.

[Gnanasekar R]

sorry, am getting confused following the chain. I was under the impression that the toolchain with Embedded ABI for RISCV has not been developed yet that would give me an advantage over code size. My embedded target will not be running Linux and it will just be a baremetal code.

can you please clarify if the currently-implemented EABI gives any advantage over code size? If it is going to get deprecated then why even push it upstream?

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAP8PnuQjw9x8uf3Ne4WrxE87a8tUMjDWu%3DquhHNBpRh_zTJkNQ%40mail.gmail.com.

[Liviu Ionescu]

On 17 May 2018 at 12:22:33, Bruce Hoult (bruce...@sifive.com) wrote:

> That question is irrelevant to the function call ABI.

Then I rephrase:

In what context the syscall number register is relevant for a device
that is not intended to use syscalls? Why does it need to be specified
by an EABI?


Regards,

Liviu

[Cesar Eduardo Barros]

(I sent this message prematurely before finishing editing, sending it
again. Sorry for the duplicate.)

Em 17-05-2018 06:22, Bruce Hoult escreveu:
> That question is irrelevant to the function call ABI.
>
> RV32E is intended for machines so small that implementing the full 32
> registers is seen as a burden. Inevitably, someone will create a code
> base on such a machine, and then want to run the same code on a more
> powerful implementation, possibly with a Linux-like operating system.

For this particular case, I see three easy solutions:

- Add a new RISC-V specific personality(2) which looks for the system
call number in t0 instead of a7;
- Add a new RISC-V specific system call (its number passed in a7 as
usual) which looks for the system call number in t0 and does the system
call table lookup again, and when loading an E ABI ELF (or perhaps
always) initialize a7 to that system call number (this works because, on
the E ABI, a7 is in fact a saved register, and so won't be clobbered);
- If the program uses a dynamic library for the system calls, use a
special variant which copies t0 to a7.

Another (not so easy) solution would be to use ptrace(2) or similar to
intercept all system calls.

The real issue would be with the vDSO, since it uses the normal function
call ABI. The solution would be to have the E ABI use different vDSO
entry points than the normal ABI, or simply have the E ABI not use the vDSO.

--
Cesar Eduardo Barros
ces...@cesarb.eti.br

[Christoph Hellwig]

On Thu, May 17, 2018 at 08:15:26AM -0300, Cesar Eduardo Barros wrote:
> For this particular case, I see three easy solutions:
>
> - Add a new RISC-V specific personality(2) which looks for the system call
> number in t0 instead of a7;
> - Add a new RISC-V specific system call (its number passed in a7 as usual)
> which looks for the system call number in t0 and does the system call table
> lookup again, and when loading an E ABI ELF (or perhaps always) initialize
> a7 to that system call number (this works because, on the E ABI, a7 is in
> fact a saved register, and so won't be clobbered);
> - If the program uses a dynamic library for the system calls, use a special
> variant which copies t0 to a7.

And what exactly is the advantage for that game? It creates a lot of
confusion and cruft just for a "better" system call ABI where no one
hasn't even quantified "better".

[Bruce Hoult]

I wouldn't expect to see code size reductions as a result of an EABI, at least in general.

There may be a little bit fewer instructions in interrupt handlers before they can call normal C functions, because they will save fewer registers, but the other side of that is a reduced number of scratch registers (argument/return registers plus temporary registers -- the ones that an interrupt handler has to save&restore) will certainly result in some functions using more code to save&restore s (callee save) registers that code using the standard ABI will not need to.

In the standard ABI, leaf functions can use up to 15 registers without loading or storing anything to memory at all: a0-a7 plus t0-t6. Interrupt handlers need to save all of these, plus ra (i.e. 16 in total), before they can call normal C code. A major impetus to having an EABI at all is to reduce the number of such registers.

In the currently-implemented RV32E ABI (as I understand it) this is reduced to a0-a5 plus t0-t2, for a total of ten registers (adding ra) that an interrupt handler needs to save.

I proposed a few messages back that this be reduced to a0-a3 (four argument registers, the same as ARM32) plus t0-t2 for a total of eight registers (with ra) that an interrupt handler must save. Jim replied that is is already one of the proposed changes.

RV32E has to reduce the number of a, t, and s registers as a logical necessity -- there are only 16 registers instead of 32, and there are the same set of five fixed-purpose registers: zero, return address, stack pointer, globals pointer, and thread local globals pointer. It's physically impossible to use the standard ABI, so a different one is needed with the remaining 11 registers split somehow between a, s and t groups. Having 6 a, 2 s, and 3 t means that the register numbers can stay exactly the same as for the standard ABI. This is what is implemented now. However this seems imbalanced, and in particular 2 s registers (for variables that a function will use again after it calls another function e.g. loop counters, pointers, accumulators) may not be enough.

RV32I in an embedded situation doesn't have the same logical necessity as RV32E to reduce the a and t registers. It only has (we are told) the practical desire to reduce them so as to decrease the work an interrupt handler has to do before calling normal C library code. To maximise the commonality while keeping this interrupt reponse time down, one simple thing is to make the RV32I EABI the same as the RV32E ABI, and make all the other 16 registers s registers. This means interrupt handlers don't have to save them because any C code that wants to use them will save and restore them itself. 

The downside to all of this is that any function that needs to use more than seven registers will need to save and restore some of the other registers, adding code (and execution time) that would not be needed in the standard ABI. Instead of the interrupt handler saving them once and for all, you may end up with a lot of other functions all saving and restoring them.

There is no free lunch, and the standard ABI is carefully designed based on decades of experience with RISC processors to optimise overall code size and execution time. It just means however that interrupt handlers that don't do all of their work in the handler itself have to save and restore a lot of registers before calling standard C code.

Personally I think interrupt handlers should be doing as little as possible, just transferring some information to or from device registers from a buffer, and returning ASAP, and letting some other code (often called a "bottom half") process the buffer. In this case the standard ABI is fine (on RV32I).  Some people say this just shows how little I know about embedded programming and that they need a lighter weight EABI.

Since RV32E needs this *anyway*, an interesting question is whether the RV32E ABI will also suffice as the RV32I EABI (with the addition of 16 more s registers).

On Thu, May 17, 2018 at 9:40 PM, Gnanasekar R <gnanase...@gmail.com> wrote:

sorry, am getting confused following the chain. I was under the impression that the toolchain with Embedded ABI for RISCV has not been developed yet that would give me an advantage over code size. My embedded target will not be running Linux and it will just be a baremetal code.

can you please clarify if the currently-implemented EABI gives any advantage over code size? If it is going to get deprecated then why even push it upstream?

On 17 May 2018 at 14:52, Bruce Hoult <bruce...@sifive.com> wrote:

That question is irrelevant to the function call ABI.

RV32E is intended for machines so small that implementing the full 32 registers is seen as a burden. Inevitably, someone will create a code base on such a machine, and then want to run the same code on a more powerful implementation, possibly with a Linux-like operating system.

Some people (e.g. you, as I understand it) also want an "embedded" ABI to run on machines with the full 32 registers, but with a reduced interrupt latency by having fewer registers that must be saved by an interrupt routine before calling compiled C code. The RV32E *ABI* seems suitable for that, even on a machine with 32 registers.

On Thu, May 17, 2018 at 8:54 PM, Liviu Ionescu <i...@livius.net> wrote:

On 17 May 2018 at 11:31:34, Christoph Hellwig (h...@lst.de) wrote:

> On Thu, May 17, 2018 at 04:05:37PM +1200, Bruce Hoult wrote:
> > I think the currently-implemented RV32E ABI isn't horrible. In fact I think
> > the syscall number in t0 is *better* than a7 and we should do that in the
> > non-E ABIs too. Too late?
>
> For the Linux syscall ABI: yes.

Are the RV32E devices intended to run Linux?


Liviu

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

[Christoph Hellwig]

On Thu, May 17, 2018 at 11:23:31PM +1200, Bruce Hoult wrote:
> There may be a little bit fewer instructions in interrupt handlers before
> they can call normal C functions, because they will save fewer registers,
> but the other side of that is a reduced number of scratch registers

I still haven't seen any actual need why the calling convention for
these magic fast path interrupts needs to be the same as the general
C function call ABI. We have a lot of precedence for supporting multiple
calling conventions in the same overall ABI framework and compilers.
e.g. on i386 compilers generall support the Unix and Windows calling
calling convention, compilers always had special attributes to pass
more arguments in registers instead of the stack, etc.

The only thing really needed is a specification for such a "low number
of caller saved registers" ABI on RISC-V. Then users of the ABI and
compilers can decided if they want to use it, or not - either on a per
function basis (e.g. only for the fast interrupt handlers), or if they
really want on a per-project basis because e.g. their RTOS wants to
always use it. For example the Linux kernel on i386 is built with
the -mregparm=3 gcc command line to always pass the first three
arguments in registers instead of on the stack as the normal ABI
would prescribe, and things work just fine.

[Bruce Hoult]

Just because t0 is specified to be the place to put syscall numbers, if you have syscalls, doesn't impact you in any way at all if you don't have syscalls. Just go ahead and use t0 for whatever you want.

However, it's useful to have the specification for *any* situation in which you want to bottleneck a number of different function calls through a single entry point. This could be for system calls, for debugger calls, for semi-hosting, for remote function calls, for adding tracing of function calls/returns. In every case you want 1) many logically different functions to call the same entry point;  2) an indication of which function you actually want -- whether it's the actual memory address of the function or an index into a table of function pointers, or simply a meaningless integer -- and you have to store that *somewhere*; and 3) you really don't want to have to shuffle all the arguments around just to add the function selector as the first argument.

It's very common, for example, to have some regular C function _write(int stream, void *buf, size_t count) which you just want to have load some constant meaning "write" -- let's call it 17 -- somewhere and call some supervisor, whether that's using "SVC #17" or "li t0,17; SYSCALL" or "li t0,17; call _trace" or whatever.

 

[Liviu Ionescu]

On 17 May 2018 at 14:23:32, Bruce Hoult (bruce...@sifive.com) wrote:

> Personally I think interrupt handlers should be doing as little
> as possible,

that's generally agreed.

> just transferring some information to or from device
> registers from a buffer, and returning ASAP, and letting some
> other code (often called a "bottom half") process the buffer.
> In this case the standard ABI is fine (on RV32I).

the point here is not how little the interrupt does, but if it is
reasonable to expect that everything can be inlined in the interrupt
handler, avoiding to call a plain C function.

those with a minimal embedded experience know that fully inlined
handlers are rare, especially when using a RTOS and the handler must
call system functions, like posting to a semaphore, or pushing to a
queue.


regards,

Liviu

[Christoph Hellwig]

On Thu, May 17, 2018 at 01:36:26PM +0200, Christoph Hellwig wrote:
> On Thu, May 17, 2018 at 11:23:31PM +1200, Bruce Hoult wrote:
> > There may be a little bit fewer instructions in interrupt handlers before
> > they can call normal C functions, because they will save fewer registers,
> > but the other side of that is a reduced number of scratch registers
>
> I still haven't seen any actual need why the calling convention for
> these magic fast path interrupts needs to be the same as the general
> C function call ABI. We have a lot of precedence for supporting multiple
> calling conventions in the same overall ABI framework and compilers.
> e.g. on i386 compilers generall support the Unix and Windows calling
> calling convention, compilers always had special attributes to pass
> more arguments in registers instead of the stack, etc.

And btw, at least on x86 gcc already has support for the
"no_caller_saved_registers" calling convention:

"Use this attribute to indicate that the specified function has no
caller-saved registers. That is, all registers are callee-saved. For example,
this attribute can be used for a function called from an interrupt handler.
The compiler generates proper function entry and exit sequences to save and
restore any modified registers, except for the EFLAGS register. Since GCC
doesn’t preserve MPX, SSE, MMX nor x87 states, the GCC option
-mgeneral-regs-only should be used to compile functions with
no_caller_saved_registers attribute."

Sounds like we just need a RISC-V equivalent to deal with the interrupt
handler use case without disturbing the general ABI.

[Liviu Ionescu]

On 17 May 2018 at 14:35:24, Bruce Hoult (bruce...@sifive.com) wrote:

> Just because t0 is specified to be the place to put syscall numbers, if you
> have syscalls, doesn't impact you in any way at all if you don't have
> syscalls. Just go ahead and use t0 for whatever you want.

Actually I don't want to use t0 for anything, I want the compiler to
use it as any other register, and do not reserve it for any special
purpose, especially for things that are not even planned to be ever
used.

> However, it's useful to have the specification for *any* situation in which
> you want to bottleneck a number of different function calls through a
> single entry point. This could be for system calls, for debugger calls, for
> semi-hosting, for remote function calls, for adding tracing of function
> calls/returns.

ARM semihosting calls are regular calls, using R0 & R1 for input and
R0 for output.

> It's very common, for example, to have some regular C function _write(int
> stream, void *buf, size_t count) which you just want to have load some
> constant meaning "write" -- let's call it 17 -- somewhere and call some
> supervisor, whether that's using "SVC #17" or "li t0,17; SYSCALL" or "li
> t0,17; call _trace" or whatever.

"It's very common" is a relative term.

Maybe it is common in the Unix world, but embedded applications rarely
use any SVC/SYSCALL, and when they do, it is common to pass arguments
in the regular input ABI registers.

So, as long as you do not reserve it and the compiler can use it when
needed, it is ok. otherwise this is simply a lost register.


Regards,

Liviu

[Bruce Hoult]

There are already plans for an "interrupt" function attribute in C that makes the compiler save&restore *everything*. This is effectively "no_caller_saved_registers", treating the interrupt handler as a zero argument void function.

It would be possible to allow functions marked with "interrupt" to take arguments AND PRESERVE THEM (except for an argument replaced by a return value). Some machines already have such as ABI. It means if a function receives an argument that it will pass on to several other functions in the same argument position then it doesn't have to do anything with it. With the normal ABI you get such an argument copied to, say, s0 (after saving s0 in the prologue), and then copied back from s0 to a0 before every function call.

[Bruce Hoult]

t0 is not reserved or lost by saying it's somewhere you can put a syscall number, if your software has syscalls. It is usable by any code, at any time, for *temporary* purposes. Temporary means that if you put something in it and then call a function, don't expect your value to be there when the function returns.

It's the same as an argument register, except you know there will never be an argument value in it. This means that, for example, it can be used by linker-generated code as temporary register that neither the caller nor the callee expects to find anything in particular in.

t1 and t2 are the same. And t3-t6, in the rv32i and rv64i ABIs.

[Liviu Ionescu]

On 17 May 2018 at 14:56:57, Bruce Hoult (bruce...@sifive.com) wrote:

> ... Temporary means that if you put something in it and then call

> a function, don't expect your value to be there when the function

> returns. ...

Thank you, Bruce, for the explanation.

Liviu

[Jim Wilson]

On Wed, May 16, 2018 at 10:30 PM, Gnanasekar R <gnanase...@gmail.com> wrote:
> Hi.. in order to build RV32E should I checkout branch riscv-next under
> riscv-gcc and riscv-binutils-gdb and build the riscv-gnu-toolchain? Is that
> enough.
>
> Just wanted to check because I do not see any recent checkins in the
> riscv-next branch. So should I take any other patch to bring it up to date?

The RV32E patches were added last year, or maybe two years ago. This
stuff has been sitting on branches for a long time. I did some
testing as part of upstreaming work, and only found a few small bugs
which don't prevent the existing support from working.

Jim

[Jim Wilson]

On Thu, May 17, 2018 at 1:51 AM, Alex Bradbury <a...@lowrisc.org> wrote:
> However if the existing -mabi=ilp32e is considered to be the "Unix
> ABI" in contrast to the yet-to-be-defined embedded ABI that makes me
> question even more why it should be upstreamed and supported more
> widely instead of deprecated. RV32E was introduced for a very very
> size-constrained deeply embedded system. Supporting -mabi=ilp32e makes
> little sense of Unix platforms, and for embedded targets people will
> surely want to use the new embedded ABI.

I'm calling the current ABI the "Unix ABI" for convenience, because it
is the ABI that is used by Linux, and we don't want to change that. I
just need a name, any name, to differentiate from the new ABI under
discussion. That doesn't mean that it can only be used for Linux. If
someone wants to use it for embedded work, they can certainly do that.
Anyone doing embedded work currently already is. You could also use
it to run linux on an RV32E system if you wanted, though I wouldn't
recommend it.

There is a practical problem here. There are people producing RV32E
parts. They need a compiler. I can't enable architecture support
without some kind of ABI support. And they can't wait 2 years for a
committee to make a decision about ABIs. We need one now. The Unix E
ABI is already mentioned in the official ABI documents. This was
added in November. So it should be OK to use, though it would be nice
if the docs were fleshed out a little. Another point here, is that I
believe that this ABI is already in use by at least one RISC-V
Foundation member. So I'm not adding anything new. I'm just
formalizing something that is already in use.

> I appreciate that, but supporting this ABI in upstream GCC makes it
> hard to ever retire and puts a pressure on the rest of the RISC-V
> world to support it. Based on what you've said about the plans for the
> 'embedded' ABI it really doesn't feel like the current -mabi=ilp32e
> ABI is something that people should ever be encouraged to use. It's
> possible I've misinterpreted things, as the only information I have on
> this embedded ABI effort is what has been said in this thread and in
> Krste's talk at the workshop last week. I've tried to clarify where
> development and discussion of this new embedded ABI will be taking
> place, but am having no luck in getting any answers so far....

I have no problems adding docs to discourage people from using
-mabi=ilp32e for the moment until we get better docs. But anyone that
wants RV32E code has no choice but to use it, and I do not expect that
this ABI will go away, because it is the ABI used by Linux.

GCC has lots of ABI options that can get people in trouble. We have
an option to change RISC-V stack alignment which violates the ABI.
GCC has target independent option that lets you change which registers
are callee-saved/caller-saved which violates the ABI. There is the
-fshort-enums option which changes the ABI. Etc. We are always going
to have ABI problems. I don't think this RV32E support is going to
make anything worse.

> If we truly have another year before having to consider any
> -mabi=ilp32e support frozen that does change things somewhat. In the
> LLVM community it's not uncommon for downstream parties to fork trunk
> at an arbitrary point so we don't have that freedom in the same way. I
> assume this doesn't happen or is strongly discouraged in the GCC
> world?

People are strongly discouraged from making official releases from the
development tree. But GCC is free software, so we can't stop anyone
from doing anything.

ABI problems are a fact of life. You just have to deal with them.
Apple for instance used gcc as the system compiler when they switched
to powerpc. A little while later, we discovered a PPC ABI bug that
left gcc incompatible with the IBM compiler. We had to fix it for
compatibility. But since gcc was the system compiler on Apple
machine, and they didn't want an ABI break, they deliberately
maintained a compiler using the broken ABI for about a decade. This
stuff happens. It is just normal engineering work from my point of
view.

Jim

[Gnanasekar]

Ok, so if I take the riscv-next branch then I can build the toolchain with rv32e support. Is that right? I can give it a try tomorrow although I understand that it is not the official one yet

> --
> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> To post to this group, send email to sw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CAFyWVabBy%2BoxBrf8y2Mx4iVLeNziM%3DBoiuHAgsPhBc_PW8WVrQ%40mail.gmail.com.

[Jim Wilson]

On Thu, May 17, 2018 at 2:42 AM, Liviu Ionescu <i...@livius.net> wrote:
> In what context the syscall number register is relevant for a device
> that is not intended to use syscalls? Why does it need to be specified
> by an EABI?

Newlib has syscall support, and needs to know what register to put the
syscall number in.

But if you aren't using newlib syscalls, then this is irrelevant.

Jim

[Jim Wilson]

On Thu, May 17, 2018 at 8:29 AM, Gnanasekar <gnanase...@gmail.com> wrote:
> Ok, so if I take the riscv-next branch then I can build the toolchain with rv32e support. Is that right? I can give it a try tomorrow although I understand that it is not the official one yet

Yes. If you check out riscv-gnu-toolchain, and then in the
riscv-binutils-gdb and riscv-gcc directories check out the riscv-next
branch, you can get working rv32e code. It only runs on the gdb
simulator if you compile code using syscalls. The qemu here doesn't
have a patch for the rv32e syscalls, though it is probably easy to
backport.

Jim

[Jim Wilson]

On Thu, May 17, 2018 at 1:51 AM, Alex Bradbury <a...@lowrisc.org> wrote:
> Krste's talk at the workshop last week. I've tried to clarify where
> development and discussion of this new embedded ABI will be taking
> place, but am having no luck in getting any answers so far....

I don't think there is any formal proposal for the new embedded ABI as
yet. But currently, discussion is happening in the fast interrupt
handling group, which I think is a small private group at the moment
trying to create a proposal before opening it up to a wider
discussion, and the new ABI is just a side-effect of the main work
that they are doing. The real discussion will happen once they have a
proposal to discuss.

Jim

[Liviu Ionescu]

On 17 May 2018 at 18:31:32, Jim Wilson (ji...@sifive.com) wrote:

> On Thu, May 17, 2018 at 2:42 AM, Liviu Ionescu wrote:
> > In what context the syscall number register is relevant for a device
> > that is not intended to use syscalls? Why does it need to be specified
> > by an EABI?
>
> Newlib has syscall support, and needs to know what register to put the
> syscall number in.
>
> But if you aren't using newlib syscalls, then this is irrelevant.

Actually newlib itself does not use any syscalls; the RISC-V libgloss
does use syscalls, because it was designed with specific Unix-like
test environments in mind, and completely ignored bare-metal usage.

An embedded friendly libgloss should not issue syscalls at all, since
the resulting binaries simply break on a bare-metal machine. First,
libgloss should be linked only upon explicit request, not always, as
it is now. Second, to be useful, an embedded friendly libgloss should
issue semihosting calls, similarly to the newlib distributed with the
ARM Embedded GCC toolchain.


Regards,

Liviu

[Jim Wilson]

On Thu, May 17, 2018 at 4:41 AM, Christoph Hellwig <h...@lst.de> wrote:
>> I still haven't seen any actual need why the calling convention for
>> these magic fast path interrupts needs to be the same as the general
>> C function call ABI. We have a lot of precedence for supporting multiple
>> calling conventions in the same overall ABI framework and compilers.
>> e.g. on i386 compilers generall support the Unix and Windows calling
>> calling convention, compilers always had special attributes to pass
>> more arguments in registers instead of the stack, etc.
>
> And btw, at least on x86 gcc already has support for the
> "no_caller_saved_registers" calling convention:

Thanks for the info. This looks useful. I will pass it on. I was
aware of the regparm stuff, and have been experimenting with something
similar in the RISC-V gcc port, but I hadn't noticed the
no_caller_saved_registers attribute. I will have to try something
similar in the RISC-V port.

By the way, there are multiple problems that we are trying to fix with
the new embedded ABI, and the fast path interrupts support is just one
of them. So we probably still new a need ABI even if we can fix this
interrupt handler problem without an ABI change.

Jim

[Liviu Ionescu]

On 17 May 2018 at 18:54:12, Jim Wilson (ji...@sifive.com) wrote:

> ... there are multiple problems that we are trying to fix with

> the new embedded ABI, and the fast path interrupts support is
> just one
> of them. So we probably still new a need ABI even if we can fix this
> interrupt handler problem without an ABI change.

That's correct, there are several distinct issues.

- an `__attribute__((interrupt))` and the associated semantics; as far
as I understood the proposal, it should not bother with the ABI caller
registers, and save only the registers used by the handler; if a C/C++
call is made in the handler, it must save the caller registers as per
the current ABI requirements, whatever it is.

- an EABI variant to the current ABI used on Linux, actually two, one
for 32-bits (RV32I) and one for 64-bits (RB64I), probably very
similar. The main requirement is a much lower number of caller saved
registers, ideally 4.

- an ABI for the RV32E, preferably consistent with the previous EABI for RV32I.


To me, the interrupt issue is more or less standalone, I think it can
be implemented today for the current Linux ABI, and later be improved
for any future EABI(s).

As for the current ABI for RV32E, I seriously doubt it can be
compatible with the future EABs, thus its use must be strongly
discouraged until the EABIs will be finalised.

Better park the code in a branch and revive it when the right time comes.


Regards,

Liviu

[Gnanasekar R]

Hi Jim,

I built the toolchain from the riscv-next branch just as you mentioned. I am getting the following error. I am not sure how stable this repo is and do not see a reason for me to dig deep into these problems. I would like to build a toolchain that just works and use it for my primary purpose.

The file it is cribbing about has just a simple main function along with other functions, This compiles fine with riscv32gc or other arm toolchains.

temp.c:189:1: internal compiler error: in get_variable_section, at varasm.c:1150

 }

 ^

0xe6e8e7 get_variable_section(tree_node*, bool)

../.././riscv-gcc/gcc/varasm.c:1149

0xe76360 assemble_variable(tree_node*, int, int, int)

../.././riscv-gcc/gcc/varasm.c:2279

0xe7bd99 varpool_node::assemble_decl()

../.././riscv-gcc/gcc/varpool.c:590

0xe7c9b3 varpool_node::assemble_decl()

../.././riscv-gcc/gcc/cgraph.h:1960

0xe7c9b3 symbol_table::output_variables()

../.././riscv-gcc/gcc/varpool.c:756

0x72ef4f symbol_table::compile()

../.././riscv-gcc/gcc/cgraphunit.c:2624

0x73177d symbol_table::compile()

../.././riscv-gcc/gcc/cgraphunit.c:2537

0x73177d symbol_table::finalize_compilation_unit()

../.././riscv-gcc/gcc/cgraphunit.c:2716

Please submit a full bug report,

with preprocessed source if appropriate.

Please include the complete backtrace with any bug report.

See <https://gcc.gnu.org/bugs/> for instructions.

Makefile:15: recipe for target 'all' failed

make: *** [all] Error 1

[Palmer Dabbelt]

On Wed, 16 May 2018 20:09:57 PDT (-0700), Arnd Bergmann wrote:
> On Wed, May 16, 2018 at 1:19 PM, Jim Wilson <ji...@sifive.com> wrote:
>
>> On Mon, May 14, 2018 at 7:30 AM, Alex Bradbury <a...@lowrisc.org> wrote:
>>> It seems there are two possibilities for RV32E:
>>> 1) The eventual embedded ABI is going to be essentially what is
>>> already implemented in GCC for -mabi=ilp32e (or incredibly close). If
>>> that's the case, let's document it properly and get it finalised as
>>> quickly as possible.
>>> 2) The eventual embedded ABI is going to be quite different to what is
>>> already implemented in GCC for -mabi=ilp32e. If that's the case,
>>> pushing out the current implementation risks creating confusion and
>>> fragmentation.
>>
>> I expect that the new embedded ABI is going to be quite different,
>> that it will support rv32e/rv32i/rv64i, and that it will not replace
>> the currrent linux ABI support. It will be like when ARM changed
>> their ABIs, and changed their configure triplet from arm-elf to
>> arm-eabi. We will just have to support both ABIs moving forward.
>> Though it isn't clear if we want to change our configure triplet or do
>> something else to indicate use of the new embedded ABIs. It would be
>> nice if we can generate all ABIs from a single toolchain.
>
> Would you expect any operating system to support more than
> one of the two ABIs, or do you forsee that embedded ABI to only
> be used on small (non-Linux) RTOSs?

We don't plan on supporting the embedded ABIs on Linux (or maybe more
accurately, in glibc, as the Linux support for an ABI is pretty minimal). I'd
expect that embedded RTOSes pick one of the two ABI families and support that,
depending on how featured programs running on them are expected to be. The
embedded ABI family is really aimed at bare-metal systems, where things like
stack usage and interrupt latency are counted down to the byte/cycle.

The idea here is that there's no free lunch in defining the ABI. Our current
ABI has been tuned for larger systems that require long-term binary
compatibility. As a result we pay some time/space costs that aren't necessary
in embedded systems: for example, we align the stack to 16-byte boundaries in
order to ensure quad-precision floating-point values are aligned when spilled
to the stack. The embedded ABI would probably swing the other way here, and
only align the stack to the maximum type size supported natively by the
underlying ISA. The drawback would be a performance hit when calling between
RV32GC and RV32GQC codebases without a recompilation, but the benefit would be
less stack usage. The embedded use case puts a higher value on "less stack
usage" and a lower value on "add a high-performance Q implementation without
recompiling", so it seems like a sane choice.

In Linux land we put a very strong emphasis on long-term binary compatibility,
with the explicit goal that the RV64GC-based Linux userspaces being built today
will run on all RV64GC-based Linux systems forever. That's always been the
plan, and as long as I'm here that's not changing :)

> The experience with two ABIs on ARM Linux was actually quite
> horrible, it would be nice if we could avoid duplicating that here.

The two ABI families here would be in compiler land, where it should be pretty
straight-forward to provide good support for multiple ABIs. We've already set
the precedent of supporting more ABIs in the compiler than we support in glibc
-- for example we support "lp64f" (64-bit long/pointer, single-precision floats
passed in F registers but double-precision floats passed in X registers) in the
compiler (where it's used on embedded systems) while disallowing that in glibc
just to avoid fragmentation.

I'm actually in favor of pushing the multi-ABI support even farther into the
compiler, with idea being that on RISC-V systems the tuple simply defines the
default set of compiler options and having every compiler tuple support
generating any style of code. Distributions can then simply select the subset
of libraries they want to ship along with the complier, which would be picked
in a sane manner to match what's expected to be used in their execution
environments. This is in line with what we're doing now: for example Debian
and Fedora ship a GCC that can generate code for rv32gc/ilp32f (or any other
ISA/ABI pair) while only shipping a C library that supports rv64gc/lp64d.

> It would be good to at least limit the number of possible ABI variations
> on a given hardware+OS platform to a minimum. On RV64I-Linux,
> I would very much hope to never see two incompatible ELF ABIs
> supported by the kernel, and if we end up supporting RV32E with
> mainline Linux, that also should not need two incompatible ABIs.
>
> For running RV32E binaries on a RV32I of RV64I Linux kernel
> (if there is a need for that), the kernel would have to support both
> binary formats at once, and minimizing the differences between
> those two would certainly help, and as soon as you get into
> different structure layouts etc it quickly becomes unmanageable.

That's part of the reason why we're defining the "ilp32e" ABI, as it's
essentially the same as the "ilp32" ABI (aside from some differences mandated
by the different base ISA). I don't think we'll support RV32E in Linux, but if
there's one thing I've learned it's that everyone wants everything so we'll
probably have to cross that bridge at some point :).

Essentially what we're doing here is orthogonalizing the ABI style and the base
ISA. This lines up with how we orthogonalized the floating-point calling
convention from the base ISA. That last letter in the ABI is moving from "what
registers can be used to pass floating-point arguments" to "what registers can
be used to pass arguments, with an i being implicit". If we'd thought about
this before naming the ABIs we'd have documented this already, but this is one
of those cases where sometimes an interface just needs to be extended.

There's also the wrinkle that ABIs really need to be sat down and thought about
for a while, and we haven't done that yet for the planned embedded ABI family.
It's possible that all of these tradeoffs we're considering making end up being
a bad idea for some reason we haven't discovered yet, in which case we just
won't define an embedded ABI family.

An ABI is very hard to change once it's been released, so we really like to
make sure these things are well thought out before committing to one. We're
very early in this embedded ABI idea -- for example, there's not even a
proposal yet. I anticipate there'll be a lot more discussion on sw-dev, but
that's more of a question for another thread (with a concrete ABI proposal),
though.

Hopefully this all makes a bit of sense...

[Liviu Ionescu]

On 18 May 2018 at 19:36:50, Palmer Dabbelt (pal...@sifive.com) wrote:

> ... An ABI is very hard to change once it's been released,

a good reason to not upstream the current RV32E.

> ... We're

> very early in this embedded ABI idea -- for example, there's not
> even a proposal yet.

there is:

https://github.com/emb-riscv/specs-markdown/blob/develop/eabi.md

however I agree that this is a very early proposal, and more work is required.

regards,

Liviu

[Bruce Hoult]

On Sat, May 19, 2018 at 5:09 AM, Liviu Ionescu <i...@livius.net> wrote:

On 18 May 2018 at 19:36:50, Palmer Dabbelt (pal...@sifive.com) wrote:

> ... An ABI is very hard to change once it's been released,

a good reason to not upstream the current RV32E.

Evaluating an ABI is not merely a paper or thought exercise. People need to try it.

 

> ... We're
> very early in this embedded ABI idea -- for example, there's not
> even a proposal yet.

there is:

https://github.com/emb-riscv/specs-markdown/blob/develop/eabi.md

however I agree that this is a very early proposal, and more work is required.

It looks strange to me. I'm worried that two s registers isn't enough and will cause excessive code size increase and performance hit from saving/restoring registers in calling functions.But it's proposed to reduce that to one! Ouch.

Also, changing a registers to t registers does nothing to reduce interrupt latency. Those should be s, if their role is changed at all.

Well, that's my thought exercise. It needs to be tried on large code bases.

[Tommy Murphy]

On Friday, 18 May 2018 22:58:39 UTC+1, Bruce Hoult wrote:

Evaluating an ABI is not merely a paper or thought exercise. People need to try it.

Sorry but I think that is extremely dismissive of Liviu's and others' attempts to propose something different to the de facto RV32E.

Maybe you didn't mean it that way but that's how it reads to me.

By all means try out different embedded implementations but don't upstream until things have been adequately teased out and nailed down.

Seems to me that there is so much doubt about the de facto RV32E and what should be done more generally for an "embedded profile" that it's premature to upstream things - especially just in order to make source control of the wider source base easier (no offence Jim! :-) 

[Jim Wilson]

On Fri, May 18, 2018 at 4:24 PM, Tommy Murphy <tommy_...@hotmail.com> wrote:
> Seems to me that there is so much doubt about the de facto RV32E and what
> should be done more generally for an "embedded profile" that it's premature
> to upstream things - especially just in order to make source control of the
> wider source base easier (no offence Jim! :-)

I just committed the RV32E binutils and gcc support upstream today.
The existing RVE ABI is already documented in the official psabi docs,
and it has been there since November. This isn't going away. I
expect that a new "embedded" ABI will be defined to better support
rv32e/rv32i/rv64i use in embedded applications. At that point, we
will have two ABIs, one primarily used by linux, and one primarily
used by embedded development, and the embedded ABI will be the
preferred ABI for RV32E, but the current RVE ABI support won't
disappear as a result of that. It would be reasonable to discourage
use of it once we have a better ABI. Other compilers need not
implement the current RVE ABI if they would prefer to wait for the new
embedded ABI to be defined.

Jim

[Jim Wilson]

On Fri, May 18, 2018 at 1:44 AM, Gnanasekar R <gnanase...@gmail.com> wrote:
> I built the toolchain from the riscv-next branch just as you mentioned. I am
> getting the following error. I am not sure how stable this repo is and do
> not see a reason for me to dig deep into these problems. I would like to
> build a toolchain that just works and use it for my primary purpose.

We don't do any serious testing on the riscv-next branches. It is
primarily a place to park patches that are almost but not quite ready
to go upstream into the official source trees. Most of the patches on
these branches require some modification before they can go upstream,
but the versions on these branches should be usable. I tried building
a toolchain from these branches and running the gcc testsuite, and the
results look OK, only a little worse than on the supported source
trees. These trees are based off of the old binutils-2.29/gcc-7.2
branches, so they are a bit old, and may be missing bug fixes present
in newer source trees. The current recommended trees are
binutils-2.30 and gcc-7.3 and/or gcc-8.0.

> The file it is cribbing about has just a simple main function along with
> other functions, This compiles fine with riscv32gc or other arm toolchains.
>
> temp.c:189:1: internal compiler error: in get_variable_section, at
> varasm.c:1150
> }
> ^

You didn't include a testcase. It isn't clear what is wrong. I don't
see this error when I run the gcc testsuite. I just added the RV32E
support to the FSF binutils and gcc development trees today. You
could try those. I need to rebuild and recheck them after the RV32E
patch, since moving large patches around between trees might result in
accidental inconsistencies, but I'm not expecting problems.

Jim