"Standard" for ecall functions and memory layout

[Martin Schoeberl]

Dear all,

I fully understand that there is on purpose no “standard” for ecall functions and the memory layout.
As stated in the ISA document this is environment/OS dependent.
If you compile just C code and your library and OS is setup, this also does not really bother you.
Ar least with virtual memory ;-) An embedded RISC-V with 1 MB RAM is a different story.

However, I’m preparing some teaching of a processor instruction set using RISC-V and
assembly code for that. Now those issues become interesting.

Here some observations so far:

ECALL:

* Assembler tests (old, from Sodor?) use x28 with
1 as a notion of pass and 2 as a notion to fail.
* Venus, a JavaScript based simulator, implements
[environment calls](https://github.com/kvakil/venus/wiki/Environmental-Calls)
some of the [system calls](https://www.doc.ic.ac.uk/lab/secondyear/spim/node8.html)
from the MIPS simulators SPIM and MARS.
* Current ISA tests use a0 and ecall for pass/fail, including a failed test case number.
A value of 1 is pass, fail is the test number multiplied by 2 added 1.
Probably spike implements this semantic.
* What else in ecall is implemented in spike?
* In a (newlib) compiled C program clib functions (such as file IO) call
`__internal_syscall`, which itself (after rotation of registers `a0`-`a7`)
executes `ecall`. Does this go to the proxy kernel (pk)?

Memory Layout:

* Venus follows the MIPS (MARS/SPIM) convention:
* `.text` at 0x00000000
* `.data` at 0x10000000
* sp at 0x7ffffff0
* heap probably at 0x10008000
* Hex files of some (old Sodor?) test cases start at 0x200.
* ISA test cases start at 0x80000000 with a `_start` symbol
* An assembler program linked with `.start` at 0x8000000 is executed by spike.
However, spike starts at 0x1000 and executes 5 boostrap instructions.
Assume those are hardcoded into spike.
* Compiling a C program (with newlib) has a start address (`_start`) at 0x00010074.
Assume that the proxy kernel is loaded at 0x8000000 and simply gets the start address from
the .elf file.

Comments on this?

Have not yet looked what Rocket and Sodor are doing.

Cheers,
Martin

[Christopher Celio]

Hi Martin,

Preface: all of my knowledge is up to Priv 1.9, so I apologize if I'm wrong or out-of-date (or if I missed your question entirely).


So I think the challenge here is you are describing tethered executions of RISC-V systems (spike, Sodor, etc.). There needs to be some mechanism for executing things like "print this to screen" and "end test with exit code" between the RISC-V design-under-test (DUT) and the host machine (riscv-fesvr).

> * Assembler tests (old, from Sodor?) use x28 with
> 1 as a notion of pass and 2 as a notion to fail.

The pre-priv1.9 days had no External Debug spec, so the magic connection between the RISC-V DUT and the host machine was the host-target interface (HTIF). HTIF provided special CSRs -- tohost and fromhost -- to manage communications between the tethered DUT and the host (riscv-fesvr). Tests ended with the host received a 1 to signify a pass.

The assembler test use riscv-tests-envs (a repo on github/riscv) to provide macros to write abstracted tests. Priv v1.7 Rocket/Sodor tests ended with a write to the HTIF tohost CSR.

Now that we have an External Debug spec, the non-standard HTIF has been removed and the Debug spec is used in its place. The "end test" macros in riscv-tests-env now write "tohost" values to a cached memory location, and the Debug Transport Module (DTM) will periodically read the value at the tohost location looking for a non-zero value.

> * In a (newlib) compiled C program clib functions (such as file IO) call
> `__internal_syscall`, which itself (after rotation of registers `a0`-`a7`)
> executes `ecall`. Does this go to the proxy kernel (pk)?

IIRC, the syscalls will get handled by some kernel (including the proxy kernel); than that kernel may kick into proxied routines where it sends the syscall to the host machine (riscv-fesvr) to handle them (printfs for example). In the past (and maybe currently too) this was handled by the tohost register as well. Since proxied syscalls are handled by reading the arguments from an aligned memory address, the lower bit 0 is always 0. Thus, the value tohost[0]==0 means "proxied syscall" and tohost[0]==1 means "end the test" with tohost[N-1:1] holding the exit code.

> * What else in ecall is implemented in spike?

Spike is executed as a tethered DUT, with the riscv-fesvr compiled into it. The code, syscalls.c I think, would answer this question better than I could.

> * ISA test cases start at 0x80000000 with a `_start` symbol
> * An assembler program linked with `.start` at 0x8000000 is executed by spike.
> However, spike starts at 0x1000 and executes 5 boostrap instructions.
> Assume those are hardcoded into spike.

The memory map for spike/Rocket-chip has the high-order bit set to 1 to represent "physical memory". This does slightly confuse things like elf2hex that are expecting to to spit out contiguous memory starting at 0x0.

The memory map also puts 0x0 as debug memory and 0x1000 as bootrom memory (at least on my copy of BOOM on priv1.9):

debug {
addr 0x0;
size 0x1000;
}
bootrom {
addr 0x1000;
size 0x1000;
}

So you're assessment is correct on the bootstrapping. For Rocket running its riscv-tests in a tethered mode, the booting process will involve loading the test binary using the DTM before it can jump to _start.

> * Compiling a C program (with newlib) has a start address (`_start`) at 0x00010074.
> Assume that the proxy kernel is loaded at 0x8000000 and simply gets the start address from
> the .elf file.

Yup. The memory map spike/rocket uses is designed with processors with VM translation in mind, and you can change it for your bare-metal cores as you see fit. Relatedly, Sodor has recently been updated to the latest priv 1.10 spec and debug spec, and it uses the spike memory map.


-Chris

> --
> You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
> To post to this group, send email to sw-...@groups.riscv.org.
> Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
> To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/0A399C69-5B0A-423D-8300-39D55B0F359C%40jopdesign.com.

[Martin Schoeberl]

Hi Christopher,

thanks for your detailed reply. Maybe it confused me even more ;-)

> On 17 Aug, 2017, at 19:25, Christopher Celio <ce...@eecs.berkeley.edu> wrote:
>
> So I think the challenge here is you are describing tethered executions of RISC-V systems (spike, Sodor, etc.). There needs to be some mechanism for executing things like "print this to screen" and "end test with exit code" between the RISC-V design-under-test (DUT) and the host machine (riscv-fesvr).

I am not sure if it is about tethered or not.

>> * Assembler tests (old, from Sodor?) use x28 with
>> 1 as a notion of pass and 2 as a notion to fail.
>
> The pre-priv1.9 days had no External Debug spec, so the magic connection between the RISC-V DUT and the host machine was the host-target interface (HTIF). HTIF provided special CSRs -- tohost and fromhost -- to manage communications between the tethered DUT and the host (riscv-fesvr). Tests ended with the host received a 1 to signify a pass.

Yes, I’ve read about tohost amd fromhost registers. However, I think I was interested in one level above. How to use ecall (former syscall). I understand that those ecall services might then be implemented with some to/from host magic.

> The assembler test use riscv-tests-envs (a repo on github/riscv) to provide macros to write abstracted tests. Priv v1.7 Rocket/Sodor tests ended with a write to the HTIF tohost CSR.
>
> Now that we have an External Debug spec, the non-standard HTIF has been removed and the Debug spec is used in its place. The "end test" macros in riscv-tests-env now write "tohost" values to a cached memory location, and the Debug Transport Module (DTM) will periodically read the value at the tohost location looking for a non-zero value.

Mmh, if I look into the (assembler) tests, I see following code:

80003084 <fail>:
80003084: 00119513 slli a0,gp,0x1
80003088: 00050063 beqz a0,80003088 <fail+0x4>
8000308c: 00156513 ori a0,a0,1
80003090: 00000073 ecall

80003094 <pass>:
80003094: 00100513 li a0,1
80003098: 00000073 ecall

At the (assembler) test level it is then an ecall and there is no HTIF or debug interface involved.

>> * In a (newlib) compiled C program clib functions (such as file IO) call
>> `__internal_syscall`, which itself (after rotation of registers `a0`-`a7`)
>> executes `ecall`. Does this go to the proxy kernel (pk)?
>
>
> IIRC, the syscalls will get handled by some kernel (including the proxy kernel); than that kernel may kick into proxied routines where it sends the syscall to the host machine (riscv-fesvr) to handle them (printfs for example). In the past (and maybe currently too) this was handled by the tohost register as well. Since proxied syscalls are handled by reading the arguments from an aligned memory address, the lower bit 0 is always 0. Thus, the value tohost[0]==0 means "proxied syscall" and tohost[0]==1 means "end the test" with tohost[N-1:1] holding the exit code.

At the C/newlib level there is some use of ecall. Is there somewhere a document on ecall conventions for newlib? Let’s say if I want to implement those in a simulator or tiny OS kernel.

However, browsing newlib source I found basically the answer. Here for the records:
riscv-gnu-toolchain/riscv-newlib/libgloss/riscv/machine/syscall.h
riscv-gnu-toolchain/riscv-newlib/libgloss/riscv/syscalls.c

>> * What else in ecall is implemented in spike?
>
> Spike is executed as a tethered DUT, with the riscv-fesvr compiled into it. The code, syscalls.c I think, would answer this question better than I could.

Thanks for the hint on syscall.c which pointed me to the newlib riscv code.

Cheers,
Martin