Baremetal program running on C++ Emulator fails when calling printf

354 views
Skip to first unread message

Noureddine AIT SAID

unread,
Apr 11, 2018, 6:15:29 AM4/11/18
to sw-...@groups.riscv.org
Hello,

I'm working on a simple baremetal program destined to run on a multicore RISC-V platform (directly on hardware without pk), after a long search of a supported multicore OS (on real HW).

I got inspired from the riscv-tests + riscv-pk repos to build a simple m-mode software. It works well in Spike, The program tries basically to initiate multiple threads, affects each thread to a separate core, and then joins everything and prints the results, using the printf function defined in syscalls.c.

My questions are:
-  I could build it and get out all the logs and instructions along with executed instructions and cycles. But the problem is that the execution stops at the moment I launch the first printf. Using some dummy instructions I could detect where the problem happened, but don't understand why?

- What is the most convinient way to debug the app using the C++ Emulator? Can anyone explain in a few words the method or just point me to a documentation?

- Is it possible to dump the content of the memory after execution of the program on the C++ Emulator using something else rather than printf?

I'm sorry for the basic questions I got very lost trying to understand the spirit of risc-v, thank you very much.

Cordialement,
​N​
oureddine
​

Michael Chapman

unread,
Apr 11, 2018, 6:18:18 AM4/11/18
to Noureddine AIT SAID, sw-...@groups.riscv.org

Why not adapt a simple RTOS (e.g. FreeRTOS) to run on multicore in a SMP configuration?
It is very simple to do and provides a nice environment for complex application code.
gdb/eclipse will even show the individual processes for you.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
To view this discussion on the web visit https://groups.google.com/a/groups.riscv.org/d/msgid/sw-dev/CALR3b2BXZmxmsyKUDY14gh-NHH4Jpo9fFD2m%3D_22Yf6UwCAGBQ%40mail.gmail.com.

Virus-free. www.avg.com

Noureddine AIT SAID

unread,
Apr 11, 2018, 7:49:28 AM4/11/18
to Michael Chapman, Noureddine AIT SAID, sw-...@groups.riscv.org
​Hello Michael Chapman,

Thanks for the suggestion.

Indeed I wanted to do that in the first time, but I found out the FreeRTOS in [ 1 ] and [ 2 ] doesn't support multicore. I don't think it will be simpler because it demands advanced knowledge on Real-Time systems not only operating systems. So I thought the simplest way is to go with a baremetal. At least I know what's inside.

I guess I'll give it another try later but for now I just want to get the baremetal working to master how the different rocket-chip tools work.

Best regards,

[ 1 ]- https://github.com/illustris/FreeRTOS-RISCV/issues/10
[ 2 ]- https://github.com/RISCV-on-Microsemi-FPGA/FreeRTOS


On Wed, Apr 11, 2018 at 12:19 PM, Michael Chapman <michael.c...@gmail.com> wrote:

Why not adapt a simple RTOS (e.g. FreeRTOS) to run on multicore in a SMP configuration?
It is very simple to do and provides a nice environment for complex application code.
gdb/eclipse will even show the individual processes for you.


On 11-Apr-18 12:14, Noureddine AIT SAID wrote:
Hello,

I'm working on a simple baremetal program destined to run on a multicore RISC-V platform (directly on hardware without pk), after a long search of a supported multicore OS (on real HW).

I got inspired from the riscv-tests + riscv-pk repos to build a simple m-mode software. It works well in Spike, The program tries basically to initiate multiple threads, affects each thread to a separate core, and then joins everything and prints the results, using the printf function defined in syscalls.c.

My questions are:
-  I could build it and get out all the logs and instructions along with executed instructions and cycles. But the problem is that the execution stops at the moment I launch the first printf. Using some dummy instructions I could detect where the problem happened, but don't understand why?

- What is the most convinient way to debug the app using the C++ Emulator? Can anyone explain in a few words the method or just point me to a documentation?

- Is it possible to dump the content of the memory after execution of the program on the C++ Emulator using something else rather than printf?

I'm sorry for the basic questions I got very lost trying to understand the spirit of risc-v, thank you very much.

Cordialement,
​N​
oureddine
​
--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Liviu Ionescu

unread,
Apr 11, 2018, 8:03:24 AM4/11/18
to Noureddine AIT SAID, Michael Chapman, sw-...@groups.riscv.org
On 11 April 2018 at 14:49:29, Noureddine AIT SAID
(noureddin...@gmail.com) wrote:

> ... it demands advanced knowledge on Real-Time systems not only
> operating systems.

The main difference is the way interrupt critical sections must be
implemented, since simply disabling interrupts or raising the priority
threshold is not enough, proper atomics must be used.

Regards,

Liviu

Noureddine AIT SAID

unread,
Apr 12, 2018, 3:04:16 AM4/12/18
to Liviu Ionescu, Noureddine AIT SAID, Michael Chapman, sw-...@groups.riscv.org
Well Thank you for the ideas Mr. Liviu, but my initial questions were rather about debugging on the C++ Emulator and dumping memory ...etc.

I could resolve that printf problem. Now still working on the baremetal multithreaded program, it succeeds on Spike but in the C++ Emulator it doesn't. I need ideas on how to debug?

​Best regards,
Noureddine

Tommy Murphy

unread,
Apr 12, 2018, 3:52:30 AM4/12/18
to RISC-V SW Dev
Possibly dumb question but ... what exactly is this "C++ emulator" that you're referring to (a link to the relevant project would help clarify) and does it have any documentation about running bare metal stdio programs on it?
Are you referring to the RISC-V qemu or something else?

Noureddine AIT SAID

unread,
Apr 12, 2018, 4:04:50 AM4/12/18
to Tommy Murphy, RISC-V SW Dev
I'm talking about the HW emulator generated from CHISEL, In rocket-chip repository it's called the "the high-performance cycle-accurate Verilator".

Sorry for the confusing name (I've seen it in the lowRISC website).

Indeed, it has some doc (a little old) in the lowRISC page. where they explain how to emulate software compiled for linux / pk / baremetal.

I didn't find further documentation besides [1] and [2]. I was confused also by the huge number of signals there, quasi-impossible to decrypt the signals using waveforms!

[1] - http://www.lowrisc.org/docs/tagged-memory-v0.1/emulator/
[2] - https://github.com/freechipsproject/rocket-chip

Best regards,
Noureddine

On Thu, Apr 12, 2018 at 9:52 AM, Tommy Murphy <tommy_...@hotmail.com> wrote:
Possibly dumb question but ... what exactly is this "C++ emulator" that you're referring to (a link to the relevant project would help clarify) and does it have any documentation about running bare metal stdio programs on it?
Are you referring to the RISC-V qemu or something else?
--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Tommy Murphy

unread,
Apr 12, 2018, 4:13:40 AM4/12/18
to RISC-V SW Dev
Any use?

https://groups.google.com/a/groups.riscv.org/forum/m/#!topic/hw-dev/Pv8jUk0DzKI

Noureddine AIT SAID

unread,
Apr 12, 2018, 8:27:53 AM4/12/18
to Tommy Murphy, RISC-V SW Dev
Thanks Mr. Murphy.

Indeed very useful. I saw this in the syscalls.c file code present in the riscv-tests, this way I understand more how the fesvr and my program communicate.

I could resolve the prints problems. And it's working perfectly on spike and the emulator (Mono-core, DefaultConfig).

Now what I don't know is how to debug programs running on the Emulator, has anyone did that before? Because I've got the multithreaded and multicore version working on Spike but it blocks in the Emulator!

Thanks a lot for your help.

Best regards.
Noureddine


On Thu, Apr 12, 2018 at 10:13 AM, Tommy Murphy <tommy_...@hotmail.com> wrote:
Any use?

https://groups.google.com/a/groups.riscv.org/forum/m/#!topic/hw-dev/Pv8jUk0DzKI


--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Tommy Murphy

unread,
Apr 12, 2018, 8:35:53 AM4/12/18
to RISC-V SW Dev, tommy_...@hotmail.com, noureddin...@gmail.com
Glad it helped.

Bare metal programs on a RISC-V target are normally debugged using gdb/openocd/JTAG/DTM..
But I'm not sure that there's any solution for this with verilator?
Unless the testbench section here gives any clues since it already seems to use some DTM functionality?


Linux apps running on Linux on a target would normally be debugged using gdb-server or gdbstub but as far as I know that does not yet exist/work for RISC-V.
I'm not sure that there is a debug solution here other than "printf debugging"?

But others may know more than I do about this.

Noureddine AIT SAID

unread,
Apr 12, 2018, 10:53:04 AM4/12/18
to Tommy Murphy, RISC-V SW Dev, Noureddine AIT SAID
Thank you for the additional information.

Why I didn't understand is why they put that message in the C Emulator about Jtag Remote Btbang if it doesn't support that? I tried to put that +jtag_rbb_enable=1 as an argument but no results. Is this used for another purpose maybe?



As you said there is only the printf mehod which is very painful. I even added some assembly dummy instructions incide the c files to mark suspicious parts where I think it stops, this helps me navigate the log files from the verilator emulation, but these aren't definitely better than a real debugger.
 
Kind regards,
Noureddine

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Megan Wachs

unread,
Apr 12, 2018, 10:58:46 AM4/12/18
to Noureddine AIT SAID, RISC-V SW Dev, Tommy Murphy
Rocket chip regressions run GDB/OpenOCD against the emulator. You can look at the .travis.yml file and regression/Makefile to see how. 

On Thu, Apr 12, 2018 at 07:53 Noureddine AIT SAID <noureddin...@gmail.com> wrote:
Thank you for the additional information.

Why I didn't understand is why they put that message in the C Emulator about Jtag Remote Btbang if it doesn't support that? I tried to put that +jtag_rbb_enable=1 as an argument but no results. Is this used for another purpose maybe?



As you said there is only the printf mehod which is very painful. I even added some assembly dummy instructions incide the c files to mark suspicious parts where I think it stops, this helps me navigate the log files from the verilator emulation, but these aren't definitely better than a real debugger.
 
Kind regards,
Noureddine
On Thu, Apr 12, 2018 at 2:35 PM, Tommy Murphy <tommy_...@hotmail.com> wrote:
Glad it helped.

Bare metal programs on a RISC-V target are normally debugged using gdb/openocd/JTAG/DTM..
But I'm not sure that there's any solution for this with verilator?
Unless the testbench section here gives any clues since it already seems to use some DTM functionality?


Linux apps running on Linux on a target would normally be debugged using gdb-server or gdbstub but as far as I know that does not yet exist/work for RISC-V.
I'm not sure that there is a debug solution here other than "printf debugging"?

But others may know more than I do about this.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+un...@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
--
Megan A. Wachs
Engineer | SiFive, Inc 
1875 South Grant Street
Suite 600
San Mateo, CA 94402

Tommy Murphy

unread,
Apr 12, 2018, 11:01:54 AM4/12/18
to RISC-V SW Dev, tommy_...@hotmail.com, noureddin...@gmail.com
Sorry - I'm not at all familiar with verilator.
But if it does support openocd remote bitbang then it suggest that gdb + openocd could be used to debug it.
You would need a suitable openocd configuration script describing the target and debug interface (remote bitbang).
But if this works then I would expect such a script to exist and I cannot see one in the RISC-V openocd github or the verilator github.
Somebody with more knowledge of verilator would be better placed to advise you than me.

Tommy Murphy

unread,
Apr 12, 2018, 11:02:40 AM4/12/18
to RISC-V SW Dev, tommy_...@hotmail.com, noureddin...@gmail.com
My post crossed with Megan's which is better advice than mine! :-) 

Noureddine AIT SAID

unread,
Apr 12, 2018, 12:27:05 PM4/12/18
to Megan Wachs, Noureddine AIT SAID, RISC-V SW Dev, Tommy Murphy
Thanks a lot M. Wachs and M. Murphy,

The advices helped me a lot.

Following the step M. Wachs indicated here [1], I built the Emulator with rbb enabled. I may need some adjustments like Mr. Murphy said concerning the configuration file and the varying rbb port (i indicated in the comment in the issue), but at least it's being a lot more clearer to me. I'll try to document more about the debugging and regressions mechanism.

Kind regards,
Noureddine.

[1] - https://github.com/freechipsproject/rocket-chip/issues/1339



On Thu, Apr 12, 2018 at 4:58 PM, Megan Wachs <me...@sifive.com> wrote:
Rocket chip regressions run GDB/OpenOCD against the emulator. You can look at the .travis.yml file and regression/Makefile to see how. 

On Thu, Apr 12, 2018 at 07:53 Noureddine AIT SAID <noureddin...@gmail.com> wrote:
Thank you for the additional information.

Why I didn't understand is why they put that message in the C Emulator about Jtag Remote Btbang if it doesn't support that? I tried to put that +jtag_rbb_enable=1 as an argument but no results. Is this used for another purpose maybe?



As you said there is only the printf mehod which is very painful. I even added some assembly dummy instructions incide the c files to mark suspicious parts where I think it stops, this helps me navigate the log files from the verilator emulation, but these aren't definitely better than a real debugger.
 
Kind regards,
Noureddine
On Thu, Apr 12, 2018 at 2:35 PM, Tommy Murphy <tommy_...@hotmail.com> wrote:
Glad it helped.

Bare metal programs on a RISC-V target are normally debugged using gdb/openocd/JTAG/DTM..
But I'm not sure that there's any solution for this with verilator?
Unless the testbench section here gives any clues since it already seems to use some DTM functionality?


Linux apps running on Linux on a target would normally be debugged using gdb-server or gdbstub but as far as I know that does not yet exist/work for RISC-V.
I'm not sure that there is a debug solution here other than "printf debugging"?

But others may know more than I do about this.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.

To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.

Noureddine AIT SAID

unread,
Apr 13, 2018, 10:01:49 AM4/13/18
to Tommy Murphy, RISC-V SW Dev, Noureddine AIT SAID
Hi all!

I tested some other things.
The HW Emulation still doesn't work perfectly.

I added the results of my tests as a comment to the discussion here https://github.com/freechipsproject/rocket-chip/issues/1339#issuecomment-381143707

Thanks for your precious help.

Best regards,
Noureddine,

On Thu, Apr 12, 2018 at 5:02 PM, Tommy Murphy <tommy_...@hotmail.com> wrote:
My post crossed with Megan's which is better advice than mine! :-) 

--
You received this message because you are subscribed to the Google Groups "RISC-V SW Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sw-dev+unsubscribe@groups.riscv.org.
To post to this group, send email to sw-...@groups.riscv.org.
Visit this group at https://groups.google.com/a/groups.riscv.org/group/sw-dev/.
Reply all
Reply to author
Forward
0 new messages