Students project: Porting MINIX 3 to the Raspberry Pi 3

[Jean-Baptiste Boric]

Hi all,

As part of my studies I have a year-end project where students in groups work full-time for three weeks on a subject. One of the available subjects is to contribute to an open-source project.

I convinced two fellow students to try and port MINIX 3 with me to the Raspberry Pi and the teachers approved. The project will start next week on Tuesday. Obviously in three weeks we can't make a complete port and can't have it merged upstream, so the goal is to have a working proof-of-concept for the final presentation.

As the project hasn't started officially, there are no further details to share at this point. As far as I'm concerned, the absolute minimum would be to boot a stripped-down system to a ramdisk and have a shell running over the UART to interact with it. Extra style points, like running the slides of the presentation on the board itself, are stretch goals if time permits.

What I do know is that we won't target the original Raspberry Pi. As shown before, its CPU is just way too old for the existing source code to handle. We'll target the Raspberry Pi 3 on hardware since that's what we have on hand, and maybe QEMU's recent support for the Raspberry Pi 2 if it works out for our purposes. Hopefully with this we'll sidestep all issues with toolchains, instruction set compatibility and virtual memory encountered so far.

Also, we'll try to port from 8933525 Fix default partition in clientctl and not master, since that's the last commit known to boot on a physical BeagleBoard, just in case.

More details to follow when the project begins.

[Jean-Baptiste Boric]

So far I've got U-Boot loading and launching the kernel. The kernel hangs after trying to jump to kmain in high virtual addresses. The MMU is somehow not active even though it is enabled in SCR (System Control Register) ; I double-checked every single line of code, register and page entry against ARM reference documentation and dumped them over the UART to check their real values. It doesn't make any sense, especially since the same kernel runs well beyond pre_init() on QEMU's raspi2 emulation.

I decided to stop working on real hardware for the time being and work inside the emulator instead, since at least I can make progress with it.

[Niek Linnenbank]

Hi Jean,

Although I have not looked at your code at all, the issue that you describe matches exactly the same issue which i encountered when porting my own project to the ARM / Raspberry Pi 2. In my case, the problem was that the processor boots initially in hypervisor mode, which has very limited MMU capabilities, which means all virtual addressing does not work in that mode, even if you configured it correctly in page tables. To discover this I had to look in the linux kernel boot code, because this behavior is not clearly documented anywhere. Take a look at this linux kernel source to see the example:

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/compressed/head.S

I spend more than a week debugging MMU code in qemu and hardware before i discovered this. I hope this helps save you time.

Regards,
Niek

On May 18, 2016 16:50, "Jean-Baptiste Boric" <jblbe...@gmail.com> wrote:

So far I've got U-Boot loading and launching the kernel. The kernel hangs after trying to jump to kmain in high virtual addresses. The MMU is somehow not active even though it is enabled in SCR (System Control Register) ; I double-checked every single line of code, register and page entry against ARM reference documentation and dumped them over the UART to check their real values. It doesn't make any sense, especially since the same kernel runs well beyond pre_init() on QEMU's raspi2 emulation.

I decided to stop working on real hardware for the time being and work inside the emulator instead, since at least I can make progress with it.

--
You received this message because you are subscribed to the Google Groups "minix3" group.
To unsubscribe from this group and stop receiving emails from it, send an email to minix3+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jean-Baptiste Boric]

In my case, the problem was that the processor boots initially in hypervisor mode, which has very limited MMU capabilities, which means all virtual addressing does not work in that mode, even if you configured it correctly in page tables.

Thank you, indeed it turns out we're running on hypervisor mode on the real thing but we're running in supervisor mode on QEMU. I haven't managed to switch back to the supervisor mode yet, but it shouldn't take too long now. I'm using the NetBSD source code rather than Linux as a starting point since it achieves it the way I need it to (going back to supervisor mode without setting up a hypervisor first). I also should be able to just use kernel_old=1 and be done with it, but I'd rather not do that.

On the simulator I've got it working up to the idle process, which hangs since the clock interruption doesn't work yet. That means virtual memory and user processes (including VFS startup message) are working out of the box.

[Jean-Baptiste Boric]

Well, that was extremely infuriating, but I finally managed to coerce the Raspberry Pi into supervisor mode without crashing in the process.

It required writing my own bootloader/unpacker from scratch in assembly and losing the better part of my sanity. For now it works only on a Raspberry Pi 2, possibly by pure luck only.

MINIX is now hanging while initializing VM on real hardware (the server, not the MMU). I'll deal with that after I get some sleep.

[Jean-Baptiste Boric]

Interrupts and timer are now ticking, both on QEMU and real hardware. Now I have to find out why I end up with no runnable processes.

[Jean-Baptiste Boric]

TTY is working now. I can use a shell over tty00 inside the boot ramdisk.

MINIX 3 is now running on my Raspberry Pi 2 (and on QEMU) :

MINIX 3 unpacker for Raspberry Pi
Current mode: hypervisor. Switching to supervisor mode...
Current mode: supervisor
Extracting modules...
kernel.bin               size:0x0004CB30    address:0x00200000
mod01_ds.elf             size:0x0002154B    address:0x02000000
mod02_rs.elf             size:0x00028DBE    address:0x02800000
mod03_pm.elf             size:0x00079BFF    address:0x03000000
mod04_sched.elf          size:0x000165A9    address:0x03800000
mod05_vfs.elf            size:0x001143FB    address:0x04000000
mod06_memory.elf         size:0x003A7A4E    address:0x04800000
mod07_tty.elf            size:0x000405DF    address:0x05000000
mod08_mfs.elf            size:0x00027B4F    address:0x05800000
mod09_vm.elf             size:0x000F7235    address:0x06000000
mod10_pfs.elf            size:0x00019A50    address:0x06800000
mod11_init.elf           size:0x0006C897    address:0x07000000
Jumping into kernel, argv[1]="board_name=RPI_2_B console=tty00"
MINIX booting
cstart
intr_init(0)
main()
initializing asyncm... done
initializing idle... done
initializing clock... done
initializing system... done
initializing kernel... done
initializing ds... done
initializing rs... done
initializing pm... done
initializing sched... done
initializing vfs... done
initializing memory... done
initializing tty... done
initializing mfs... done
initializing vm... done
initializing pfs... done
initializing init... done
system_init()... done

MINIX 3.3.0. Copyright 2014, Vrije Universiteit, Amsterdam, The Netherlands
MINIX is open source software, see http://www.minix3.org
cycles_accounting_init()... done
IRQ 121 handler registered by tty / 5
Started VFS: 9 worker thread(s)
End of ramdisk rc script
# ls /
bin     dev     etc     proc    service usr
# sysenv
board_name=RPI_2_B
console=tty00
arch=earm
board=ARM-ARMV7-RPI-RPI_2_B
# echo "IT'S ALIVE!"
IT'S ALIVE!
#

[r0ller]

Wow, that's pretty good news! Not that I'd use raspi but every port of minix making it able to run on various platforms is something we need if we want to see it alive in the future:) Congrats again!

r0ller

[Roc Vallès]

Now I have to find out why I end up with no runnable processes. 

I'm really curious about this... what was the issue? 

[Jean-Baptiste Boric]

Now I have to find out why I end up with no runnable processes. 

I'm really curious about this... what was the issue? 

init panics if it can't open /dev/console (because I hadn't implemented TTY yet, so trying to open /dev/console resulted in ENXIO).

With every other system process waiting for init to do something, no processes were runnable so the micro-kernel just kept scheduling idle.

Thank God for QEMU. Its Raspberry Pi emulation might be far from being complete, but the GDB stub is a life-saver.

[Jean-Baptiste Boric]

Framebuffer is up and running, I have a lovely racoon on my TV screen.

Next goal is to make GPIO work so that I may run my presentation on the board itself and switch slides by short-circuiting pins.

If there's still time left after that, I2C and the PiFace clock (RTC) are next on the list.

[Jean-Baptiste Boric]

GPIO is working. Next step is I2C.

I rebased and cleaned our branch on top of master. It boots on QEMU but hangs on the real hardware. While I can't rule out an error in the port itself, it is consistent with issue #104 which prevents MINIX from booting on real ARM hardware.

We'll keep working on our old branch since it works, but it won't be contributed upstream since it's based on the last known good commit for ARM. I'll keep maintaining the new branch since it will be the one who gets contributed, but as long as issue #104 isn't solved it won't work on the real hardware.

[Jean-Baptiste Boric]

The port is functional enough to allow running a presentation on it.

With a few small modifications to the boot ramdisk, PNG files can be directly decompressed to the framebuffer and the GPIO pins can be used as forward and backwards buttons. A simple shell script launched on boot is used to run the show.

I'll send a pull request tomorrow with the rebased version on master and open an issue to track the port progress on GitHub.

I do not expect this to be merged for quite some time due to a long list of issues (one of the biggest being that this version doesn't boot on real hardware, possibly due to issue 104) and essential drivers missing, but turning this proof-of-concept into a production-grade port won't happen overnight anyway.

[Jean-Baptiste Boric]

The students project has reached its end. We started to write code for I2C but we weren't able to make it work within the constrained timeframe.

A few issues popped out while making the port, the most obvious being QEMU's highly lackluster Raspberry Pi support, which forced us to write our own bootloader and more generally make lots of workarounds to address various issues.

Another problem is the lack of device tree support. Beyond the issues of hard-coding board-specific stuff, it prevents having a flexible memory layout for boot modules, so we couldn't just replace the boot ramdisk with the full-featured one available with x86_ramdisk.sh.

I don't know yet if I'll be able to finish the port (I've run out of easy tasks to do), but I will make sure to merge what's done and at least try to keep it from bit-rotting.

[Asim Abbas]

Hi Jean;
 I have started working on installing the minix on raspberry pi, Can you please give provide me with report or instructions how to port it to raspberry by email @ asim...@hotmail.com

[Jean-Baptiste Boric]

Hi,

There's been a GSoC in between. Check out https://github.com/Stichting-MINIX-Research-Foundation/gsoc/tree/2017-rpi (build with releasetools/arm_sdimage_rpi.sh) and http://wiki.minix3.org/doku.php?id=soc:2017:portrpi.