Duktape: Including a large .o causes a fault in TockOS on the Hail

[Christopher Brooks]

Summary:

I'm working on getting the JavaScript Duktape accessor running on a Hail
board.

I have ifdef'd my main down to just printing, which works fine, I can
listen to the board and see the message.

However, if at link time, I include the duktape.o file, then I get a fault.

I'm guessing this is a size issue, but could use some help.

Details:

Here's how to reproduce this.

I'm using these versions:

> bash-3.2$ rustc --version
> rustc 1.12.0-nightly (54c0dcfd6 2016-07-28)
> bash-3.2$ arm-none-eabi-gcc --version
> arm-none-eabi-gcc (GNU Tools for ARM Embedded Processors) 5.4.1
> 20160919 (release) [ARM/embedded-5-branch revision 240496]
> Copyright (C) 2015 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions. There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE.

I'm working in a branch (cxbrooks/ducktape3) on a (fork
https://github.com/cxbrooks/tock). I sync'd my fork to the
helena-project/tock about 60 minutes ago:

> bash-3.2$ git log | head -200
> commit c44eebf8b0aaf6b2df9c5daabc2e69cd9a74bd08
> Author: Christopher Brooks <c...@eecs.berkeley.edu>
> Date: Fri Jan 6 17:46:37 2017 -0800
>
> Duktape Makefile that compiles but faults at run time.
>
> commit c352454bed6e63a5ab5f30329f4a8f7beabebc2e
> Author: Branden Ghena <brg...@umich.edu>
> Date: Fri Jan 6 20:34:41 2017 -0500
>
> Refactor process status printing
>
> * Explain text segment in more depth
> * Refactor `statistics_str` function
> * Remove unnecessary fields from process struct

Here are the commands to run:
> git clone https://github.com/cxbrooks/tock
> cd tock
> git checkout cxbrooks/duktape3
> cd boards/hail
> make PORT=/dev/tty.usbserial-00002014 TOCK_BOARD=hail program
> cd ../../userland/examples/blink
> make PORT=/dev/tty.usbserial-00002014 TOCK_BOARD=hail program
The blink program works fine.

The duktape example program consists of a makefile that has a target
called "update" that checks out the accessors repo and copies .c and .h
files.

> cd ../duktape
> make update
> make PORT=/dev/tty.usbserial-00002014 TOCK_BOARD=hail program
Here are the sizes:
> bash-3.2$ size build/cortex-m4/*
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> text data bss dec hex filename
> 163544 6621 236 170401 299a1 build/cortex-m4/app.elf
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 1562 182 168 1912 778 build/cortex-m4/c_eventloop.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 160 10 0 170 aa build/cortex-m4/duk_stack.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 120092 17206 0 137298 21852 build/cortex-m4/duktape.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 54 15 0 69 45 build/cortex-m4/eduk.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 92 42 0 134 86 build/cortex-m4/ledcontrol.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 384 211 0 595 253 build/cortex-m4/modSearch.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 0 0 0 0 0 build/cortex-m4/nofileio.o
> bash-3.2$
The board has the red-blink-of-death.

(Reminds me of the students that used a red blinking LED as their first
program on a similar board. They thought they got it right the first
time! But it was just the error blink.)

Listening to the port:

> bash-3.2$ tockloader --port /dev/tty.usbserial-00002014 listen
> Kernel panic at /Users/cxh/src/hail/tock/kernel/src/process.rs:278:
> "Process duktape had a fault"
>
> ---| Fault Status |---
> Instruction Access Violation: true
> Forced Hard Fault: true
> Fault Status Register (CFSR): 0x00000001
> Hard Fault Status Register (HFSR): 0x40000000
>
> ---| App Status |---
> App: duktape
> [Fault] - Events Queued: 0 Syscall Count: 0
>
> ╔═══════════╤════════════════════Kernel panic at
> /Users/cxh/src/hail/tock/kernel/src/process.rs:278:
> "Process duktape had a fault"
>
> ---| Fault Status |---
> Instruction Access Violation: true
> Forced Hard Fault: true
> Fault Status Register (CFSR): 0x00000001
> Hard Fault Status Register (HFSR): 0x40000000
>
> ---| App Status |---
> App: duktape
> [Fault] - Events Queued: 0 Syscall Count: 0
> ╔═══════════╤══════════════════════════════════════════╗
> ║ Address │ Region Name Used | Allocated (bytes) ║
> ╚0x20006000═╪══════════════════════════════════════════╝
> │ ▼ Grant 256 | 1024
> 0x20005F00 ┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
> │ Unused
> 0x200052D0 ┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
> │ ▲ Heap 0 | 1024 S
> 0x200052D0 ┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈ R
> │ ▼ Stack 32 | 2048 A
> 0x200052B0 ┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈ M
> │ Unused
> 0x20004AD0 ┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
> │ Data 2768 | 2768
> 0x20004000 ┴───────────────────────────────────────────
> .....
> 0x00070000 ┬───────────────────────────────────────────
> │ Unused
> 0x00058EE0 ┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈ F
> │ Data 2532 L
> 0x000584FC ┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈ A
> │ Text 163544 S
> 0x00030624 ┼┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈ H
> │ Header 1572
> 0x00030000 ┴───────────────────────────────────────────
>
> R0 : 0x20004AD0 R6 : 0x00000000
> R1 : 0x200052D0 R7 : 0x00000000
> R2 : 0x20005F00 R8 : 0x00000000
> R3 : 0x00000000 R10: 0x00000000
> R4 : 0x00000000 R11: 0x00000000
> R5 : 0x00000000 R12: 0x00000000
> R9 : 0x20004000 (Static Base Register)
> PC : 0x0004C0C0 [0x8001BA9C in lst file]
> LR : 0x00030629 [0x80000004 in lst file]
BTW - the output is pretty, but it does not copy and paste that well.
Maybe stick with non-graphical characters?

Ok, so the above failed.

Now remove duktape.c, which is huge, and two files that use functions
defined in duktape.c:
> rm c_eventloop.c ledcontrol.c duktape.c
> make clean
> make PORT=/dev/tty.usbserial-00002014 TOCK_BOARD=hail program
Here are the sizes:
> bash-3.2$ size build/cortex-m4/*
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> text data bss dec hex filename
> 206 4236 20 4462 116e build/cortex-m4/app.elf
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 160 10 0 170 aa build/cortex-m4/duk_stack.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 54 15 0 69 45 build/cortex-m4/eduk.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 384 211 0 595 253 build/cortex-m4/modSearch.o
> /opt/local/libexec/llvm-3.9/bin/llvm-size: error reading file: The
> file was not recognized as a valid object file.
> 0 0 0 0 0 build/cortex-m4/nofileio.o
> bash-3.2$

And here is my message:
> bash-3.2$ tockloader --port /dev/tty.usbserial-00002014 listen
> Hello World!
So, printing works. Below is the part of the main() from eduk.c:

> int main(int argc, char *argv[]) {
> #ifdef HAIL_PRINT
>
> #ifdef __ARM_EABI__
> putnstr_async(hello, sizeof(hello), nop, NULL);
> #else
> printf("%s", hello);
> #endif // __ARM_EABI
> return 0;
>
> #else //HAIL_PRINT


So, what I don't understand is why including duktape.o in the binary is
causing the crash. Any ideas?

Unfortunately, duktape.c is 87k lines of code, so paring this down is
not easy.

I'm using Duktape-1.6.0, which came out in December. Duktape-2.0.0 is
out, but the configuration options have changed.

I have not looked very far in to how to use gdb to debug this. I was
able to use tools/build-arm-gdb to build an arm-specific version of
gdb. I believe I need to use the -tty argument to gdb. I see that
https://github.com/helena-project/tock/blob/master/doc/tutorials/README.md
has plans for a JTAG/GDB tutorial and that
https://github.com/helena-project/tock/blob/master/boards/storm/README.md
has some details about using JLinkGDBServer. My guess is that to use
gdb, I need to get a JTAG box. I saw some notes about which JTAG box is
preferred, but I'm not finding them right now.

So, any ideas here?

_Christopher






--
Christopher Brooks, PMP
Academic Program Manager
iCyPhy/Ptolemy/TerraSwarm
University of California, Berkeley
707.332.0670, c...@berkeley.edu, https://ptolemy.eecs.berkeley.edu/~cxh

[Branden Ghena]

To give a little more information here, the large version of your code is failing at the very first line of `main`. This very much looks like an MPU problem to me and you might be able to get around it for now by simply disabling the MPU.

To determine this:

I checked out and built your code. Then I ran `make debug` which generates an LST file, `build/cortex-m4/app.lst` (Warning: this lst file is 700000 lines and took a couple minutes to generate)

Then I looked up the address where your app is failing from the debug report. 0x8001BA9C in the LST file.

```

...

8001ba9c <main>:

 *

 *  @param argc The number of arguments.

 *  @param argv The arguments.

 */

//main(void)

int main(int argc, char *argv[]) {

8001ba9c:       b507            push    {r0, r1, r2, lr}

#ifdef HAIL_PRINT

#ifdef __ARM_EABI__

  putnstr_async(hello, sizeof(hello), nop, NULL);

8001ba9e:       4b09            ldr     r3, [pc, #36]   ; (8001bac4 <main+0x28>)

...

```

The actual entry point of your program is in `libtock/crt1.c`. Relevant disassembly:

```

...

Disassembly of section .text:

80000000 <_start>:

__attribute__ ((section(".start"), used, naked))

void _start(

    __attribute__((unused))void* mem_start,

    __attribute__((unused))void* app_memory_break,

    __attribute__((unused))void* kernel_memory_break) {

  main();

80000000:       f01b fd4c       bl      8001ba9c <main>

  while(1) { yield(); }

80000004:       f01d ff5e       bl      8001dec4 <yield>

80000008:       e7fc            b.n     80000004 <_start+0x4>

...

```

So, the very first thing your code does is branch to main(), which immediately throws an Instruction Access Violation. It seems that we are probably doing something wrong when setting up the MPU for programs with a lot of flash. For now, you could try turning off the MPU. Comment out the MPU enable in the kernel. (https://github.com/cxbrooks/tock/blob/cxbrooks/duktape3/boards/hail/src/main.rs#L343), then recompile and load the kernel (`make program` from tock/boards/hail/).

Thanks
Branden Ghena
PhD Student - University of Michigan
Computer Science and Engineering

--
You received this message because you are subscribed to the Google Groups "Tock Embedded OS Development Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tock-dev+unsubscribe@googlegroups.com.
To post to this group, send email to tock...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tock-dev/5933efb4-b5a6-2830-39c0-19481d09e3c7%40eecs.berkeley.edu.
For more options, visit https://groups.google.com/d/optout.

[Christopher Brooks]

Thanks!  That worked, I can now load the larger program.

Also, many thanks for the analysis about how you determined the problem.

_Christopher

To unsubscribe from this group and stop receiving emails from it, send an email to tock-dev+u...@googlegroups.com.

To post to this group, send email to tock...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tock-dev/CAN7R6RggoacdJ2ppn_qdd9_SQ0xu50aBnOgdDBmR%2BEHJO_f8SA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.