Instructions for booting Linux on Opsis board

[Tim Ansell]

Hi everyone,

I started trying to put together some instructions for booting Linux on the Opsis board at https://docs.google.com/document/d/1GfWKjrcfA27qbF8F9EtvRP5mzsaBjRzADfZTb5H8irw/edit#

They still need a bunch of modification / improvements before they will just work. I'm hoping to extend the support to the Arty & MimasV2 ASAP.

One major issue at the moment is that the bare metal firmware needs the BIOS to jump to the DDR memory address (normal 0x400000), while to boot Linux we need to modify the BIOS to jump to the DDR location + 0x100 (which is the address of the reset exception vector).

Solving this issue would make it much easier to switch back and forth between Linux and the bare metal firmware while developing.

It's my understanding the vmlinux.bin binary we have should just be zeros between 0x0 and 0x100 and hence should just be all NOPs? However that doesn't seem to be working. FYI This also doesn't work in QEMU, so we should be able to solve it there...

Look forward to getting more things working soon!

Tim 'mithro' Ansell

[joel...@gmail.com]

On 3 Oct. 2017 11:56, "Tim Ansell" <mit...@mithis.com> wrote:

Hi everyone,

I started trying to put together some instructions for booting Linux on the Opsis board at https://docs.google.com/document/d/1GfWKjrcfA27qbF8F9EtvRP5mzsaBjRzADfZTb5H8irw/edit#

Cool. I prefer wikis to Google docs. Could we instead put this on a GitHub wiki?

They still need a bunch of modification / improvements before they will just work. I'm hoping to extend the support to the Arty & MimasV2 ASAP.

One major issue at the moment is that the bare metal firmware needs the BIOS to jump to the DDR memory address (normal 0x400000), while to boot Linux we need to modify the BIOS to jump to the DDR location + 0x100 (which is the address of the reset exception vector).

Solving this issue would make it much easier to switch back and forth between Linux and the bare metal firmware while developing.

It's my understanding the vmlinux.bin binary we have should just be zeros between 0x0 and 0x100 and hence should just be all NOPs? However that doesn't seem to be working. FYI This also doesn't work in QEMU, so we should be able to solve it there...

Why are there noops for those locations?

Is there a reason we can't have the kernel start at the 0x0 location?

I'll take a look at what the kernel is sticking there.

Cheers,

Joel

Look forward to getting more things working soon!

Tim 'mithro' Ansell

--
You received this message because you are subscribed to the Google Groups "Linux for LiteX FPGA SoC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linux-litex+unsubscribe@googlegroups.com.
To post to this group, send email to linux...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-litex/CAHLUNMyRjjOL%3DvW7iHdDQDjtEoC%2Bmd0BT3wCa8oXbtmmiSvYtw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Ansell]

On 3 Oct. 2017 1:23 pm, "joel...@gmail.com" <joel...@gmail.com> wrote:

On 3 Oct. 2017 11:56, "Tim Ansell" <mit...@mithis.com> wrote:

Hi everyone,

I started trying to put together some instructions for booting Linux on the Opsis board at https://docs.google.com/document/d/1GfWKjrcfA27qbF8F9EtvRP5mzsaBjRzADfZTb5H8irw/edit#

Cool. I prefer wikis to Google docs. Could we instead put this on a GitHub wiki?

I generally find it quicker to edit a Google Doc & wiki's don't really let you add comments and such.

I agree at some point we should probably move to the wiki but unless you have a compelling reason right now, let's leave that to another time?

They still need a bunch of modification / improvements before they will just work. I'm hoping to extend the support to the Arty & MimasV2 ASAP.

One major issue at the moment is that the bare metal firmware needs the BIOS to jump to the DDR memory address (normal 0x400000), while to boot Linux we need to modify the BIOS to jump to the DDR location + 0x100 (which is the address of the reset exception vector).

Solving this issue would make it much easier to switch back and forth between Linux and the bare metal firmware while developing.

It's my understanding the vmlinux.bin binary we have should just be zeros between 0x0 and 0x100 and hence should just be all NOPs? However that doesn't seem to be working. FYI This also doesn't work in QEMU, so we should be able to solve it there...

Why are there noops for those locations?

If I understand correctly, this should actually be the code to do setup things for the decompression of the kernel. (When doing zImage / bzImage).

Is there a reason we can't have the kernel start at the 0x0 location?

I assume so, but not sure. I think it has something to do with the exception vectoring. Maybe Stafford knows?

We should be able to also try putting a jump forward 0x100 at 0x0.

I'll take a look at what the kernel is sticking there.

I can't convince objdump to read the .bin file (works fine with the .elf file). Any idea how to make that work?

Tim 'mithro' Ansell

[Sean Cross]

I don't have the LiteX toolchain handy, but here's how I'd do it on a .bin file containing ARM Thumb code:

    arm-none-eabi-objdump.exe -D -marmv5t -bbinary -Mforce-thumb ./palette.bin

Your -m would obviously be different, and you could probably omit the LiteX-equivalent of -Mforce-thumb, but -bbinary, -D, and -m[something] are all what you should need.

Running "objdump --help" will give you a list of acceptable -m options.


Sean

[Tim Ansell]

I tried or1k-elfobjdump -D -b binary -m or1k but it doesn't seem to output things correctly? I assume it has probably something to do with endianess?

See below;

--------------------------------------------------------------------

# or1k-elf-objdump -D vmlinux | less

vmlinux:     file format elf32-or1k

Disassembly of section .text:

c0000000 <_s_kernel_ro>:

        ...

c0000100:       19 e0 c0 2a     l.movhi r15,0xc02a

c0000104:       a9 ef 60 00     l.ori r15,r15,0x6000

c0000108:       19 a0 80 00     l.movhi r13,0x8000

c000010c:       e1 ad 78 00     l.add r13,r13,r15

c0000110:       44 00 68 00     l.jr r13

c0000114:       15 00 00 00     l.nop 0x0

        ...

c0000200 <_dispatch_bus_fault>:

c0000200:       b4 00 00 0b     l.mfspr r0,r0,0xb

c0000204:       d4 00 f0 78     l.sw 120(r0),r30

c0000208:       d4 00 50 7c     l.sw 124(r0),r10

c000020c:       d4 00 08 80     l.sw 128(r0),r1

c0000210:       18 00 00 00     l.movhi r0,0x0

c0000214:       b7 c0 00 40     l.mfspr r30,r0,0x40

c0000218:       a7 de 00 01     l.andi r30,r30,0x1

c000021c:       bc 1e 00 00     l.sfeqi r30,0

c0000220:       0c 00 00 0a     l.bnf c0000248 <_dispatch_bus_fault+0x48>

c0000224:       15 00 00 00     l.nop 0x0

c0000228:       18 20 c0 29     l.movhi r1,0xc029

c000022c:       a8 21 4c 54     l.ori r1,r1,0x4c54

c0000230:       1b c0 80 00     l.movhi r30,0x8000

c0000234:       e3 de 08 00     l.add r30,r30,r1

.....

--------------------------------------------------------------------

# or1k-elf-objdump -D -b binary -m or1k arch/openrisc/boot/vmlinux.bin

arch/openrisc/boot/vmlinux.bin:     file format binary

Disassembly of section .data:

00000000 <.data>:

        ...

     100:       19 e0 c0 2a     *unknown*

     104:       a9 ef 60 00     *unknown*

     108:       19 a0 80 00     *unknown*

     10c:       e1 ad 78 00     *unknown*

     110:       44 00 68 00     *unknown*

     114:       15 00 00 00     *unknown*

        ...

     200:       b4 00 00 0b     *unknown*

     204:       d4 00 f0 78     *unknown*

     208:       d4 00 50 7c     *unknown*

     20c:       d4 00 08 80     *unknown*

     210:       18 00 00 00     *unknown*

     214:       b7 c0 00 40     *unknown*

     218:       a7 de 00 01     *unknown*

     21c:       bc 1e 00 00     *unknown*

     220:       0c 00 00 0a     *unknown*

     224:       15 00 00 00     *unknown*

     228:       18 20 c0 29     *unknown*

     22c:       a8 21 4c 54     *unknown*

     230:       1b c0 80 00     *unknown*

     234:       e3 de 08 00     *unknown*

.....

--------------------------------------------------------------------

--

You received this message because you are subscribed to the Google Groups "Linux for LiteX FPGA SoC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linux-litex+unsubscribe@googlegroups.com.
To post to this group, send email to linux...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-litex/eb20f8bc-6ec5-a5fe-c4a3-a03ca342b6df%40xobs.io.

[Tim Ansell]

Ahh the thing that I needed was a "-EB" on my objdump command.

# or1k-elf-objdump -D -b binary -m or1k -EB arch/openrisc/boot/vmlinux.bin

arch/openrisc/boot/vmlinux.bin:     file format binary

Disassembly of section .data:

00000000 <.data>:

       0:       00 00 00 40     l.j 0x100

       4:       15 00 00 00     l.nop 0x0

        ...

     100:       19 e0 c0 2a     l.movhi r15,0xc02a

     104:       a9 ef 60 00     l.ori r15,r15,0x6000

     108:       19 a0 80 00     l.movhi r13,0x8000

     10c:       e1 ad 78 00     l.add r13,r13,r15

     110:       44 00 68 00     l.jr r13

     114:       15 00 00 00     l.nop 0x0

        ...

     200:       b4 00 00 0b     l.mfspr r0,r0,0xb

     204:       d4 00 f0 78     l.sw 120(r0),r30

[Stafford Horne]

On Tue, Oct 03, 2017 at 12:25:43PM +1100, Tim Ansell wrote:
> Hi everyone,

>

> It's my understanding the vmlinux.bin binary we have should just be zeros
> between 0x0 and 0x100 and hence should just be all NOPs? However that
> doesn't seem to be working. FYI This also doesn't work in QEMU, so we
> should be able to solve it there...

On openrisc l.nop is 0x15000000, that might be an issue? 0x00 opcode is a
jump with 0x0 offset. I think maybe we can put something like this in the
kernel head.S.

.org 0x0
l.j 0x100

-Stafford

[Olof Kindgren]

Den 3 okt. 2017 09:52 skrev "Stafford Horne" <sho...@gmail.com>:

On Tue, Oct 03, 2017 at 12:25:43PM +1100, Tim Ansell wrote:
> Hi everyone,

>

> It's my understanding the vmlinux.bin binary we have should just be zeros
> between 0x0 and 0x100 and hence should just be all NOPs? However that
> doesn't seem to be working. FYI This also doesn't work in QEMU, so we
> should be able to solve it there...

On openrisc l.nop is 0x15000000, that might be an issue?  0x00 opcode is a
jump with 0x0 offset.  I think maybe we can put something like this in the
kernel head.S.

.org 0x0
 l.j 0x100

Might be missing something here, but why not start bare-metal apps from 0x100 as well? That's how all bare-metal apps I use do it, except for my bootloaders. 0x100 is the reset vector defined in the or1k spec. One can argue if it's a good choice (wastes 256 bytes for example), but it is the standard way.

//Olof

-Stafford

> Look forward to getting more things working soon!
>
> Tim 'mithro' Ansell

--
You received this message because you are subscribed to the Google Groups "Linux for LiteX FPGA SoC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linux-litex+unsubscribe@googlegroups.com.

To post to this group, send an email to linux...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-litex/20171003075206.GE2609%40lianli.shorne-pla.net.

[Tim Ansell]

After some fiddling I got this to work -> https://github.com/shenki/linux/compare/litex-minimal...mithro:litex-minimal?expand=1

Tested in both QEMU and on the Opsis board.

This should allow us to merge most of the changes.

Tim 'mithro' Ansell

[Stafford Horne]

On 3 Oct 2017 5:09 p.m., "Olof Kindgren" <olof.k...@gmail.com> wrote:

Den 3 okt. 2017 09:52 skrev "Stafford Horne" <sho...@gmail.com>:

On Tue, Oct 03, 2017 at 12:25:43PM +1100, Tim Ansell wrote:
> Hi everyone,

>

> It's my understanding the vmlinux.bin binary we have should just be zeros
> between 0x0 and 0x100 and hence should just be all NOPs? However that
> doesn't seem to be working. FYI This also doesn't work in QEMU, so we
> should be able to solve it there...

On openrisc l.nop is 0x15000000, that might be an issue?  0x00 opcode is a
jump with 0x0 offset.  I think maybe we can put something like this in the
kernel head.S.

.org 0x0
 l.j 0x100

Might be missing something here, but why not start bare-metal apps from 0x100 as well? That's how all bare-metal apps I use do it, except for my bootloaders. 0x100 is the reset vector defined in the or1k spec. One can argue if it's a good choice (wastes 256 bytes for example), but it is the standard way.

//Olof

That's a good point.

-Stafford

> Look forward to getting more things working soon!
>
> Tim 'mithro' Ansell

--
You received this message because you are subscribed to the Google Groups "Linux for LiteX FPGA SoC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linux-litex+unsubscribe@googlegroups.com.

To post to this group, send an email to linux...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-litex/20171003075206.GE2609%40lianli.shorne-pla.net.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Linux for LiteX FPGA SoC" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linux-litex+unsubscribe@googlegroups.com.

To post to this group, send email to linux...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-litex/CAKaYPCM%2B%3DACsCx-utASyp-PLA_bAMBqs7FEOWJ2fxGBzMTbeZg%40mail.gmail.com.

[Tim Ansell]

That is a good question. I don't know why we always jump to 0x0 -- I assume it is because that is how the lm32 worked (which was the first supported CPU architecture).

You can find the "crt0" we use for bare metal stuff (like the BIOS) at https://github.com/enjoy-digital/litex/tree/master/litex/soc/software/libbase

The BIOS code can be found at https://github.com/enjoy-digital/litex/tree/master/litex/soc/software/bios The code which is loading Linux via tftp or from flash into the DDR memory and then jumping to it can be found here -> https://github.com/enjoy-digital/litex/blob/master/litex/soc/software/bios/boot.c#L227-L294

Tim 'mithro' Ansell

[Tim Ansell]

More information....

soc_core.py does seem to set the default cpu_reset_address=0x00000000 here (but the value is sometimes overridden to another location) -> https://github.com/m-labs/misoc/blob/master/misoc/integration/soc_core.py#L21

This is then passed into the mor1kx module here as OPTION_RESET_PC -> https://github.com/m-labs/misoc/blob/master/misoc/integration/soc_core.py#L74

OPTION_RESET_PC seems to be defined at https://github.com/openrisc/mor1kx/blob/477a7c5a8af8b1f52953338992ef7ccbf903447f/rtl/verilog/mor1kx.v#L67-L68

And as we are using the CAPPUCCINO config, it is stuffed into the PC register on reset here -> https://github.com/openrisc/mor1kx/blob/75b928a2287ca926d320915fa4ff165957bb0eaf/rtl/verilog/mor1kx_fetch_cappuccino.v#L228 

I don't see how would this end up getting vectored to 0x100?

Tim 'mithro' Ansell

[Olof Kindgren]

It's fine to set OPTION_RESET_PC to something else. Most existing or1k SoCs I have seen starts from a small boot ROM[1] (traditionally located at 0xF0000000, but that's just an implementation detail), which does what it needs to load the application to RAM, (like loading an image from SPI Flash or similar), and then jumps to 0x100. I guess this is a lightweight version of your BIOS.

The key here is that all software, both newlib- and Linux-based assume 0x100 to be the program entry point. I'm a bit confused about your bare-metal apps. Are they based on newlib or do you have your own libc implementation? I saw that your RAM base address is 0x40000000. I guess it's just a matter of setting _or1k_board_mem_base to that value, but I haven't tested myself

[1] Some example boot loaders here -> https://github.com/olofk/or1k_bootloaders/

//Olof

To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-litex/CAHLUNMzA%2Bnm9Vb0WHqozUmjZcm5dBWnab8DC2agO8hORHxF1%2BQ%40mail.gmail.com.