aarch64 resurrected

[Waldek Kozaczuk]

Happy New Year everyone!

A couple of days ago I sent a patch that fixes an issue with large DTB and thus brings the arm port of OSv back to life. I have tested it on Fedora 29 and was able to boot and run a simple uush.so example on both Fedora and Ubuntu QEMU with aarch64 emulation:

./scripts/build -j4 image=empty fs=ramfs arch=aarch64
qemu-system-aarch64 -s -nographic -machine virt -machine gic-version=2 -kernel ./arm/loader.img -cpu cortex-a57 -m 1024M -append "--verbose --nomount --maxnic=0 /tools/uush.so" -smp 2

I had to manually add uush.so to usr_ramfs.manifest. For whatever reason, the image built on Ubuntu would never boot at all. I wonder if that has to do with some mixed up gcc path on Ubuntu and maybe we end up compiling and/or linking against some wrong artifacts (external/host maybe related).  

I have also been able to run the same image with KVM acceleration enabled on Raspberry PI 4 with Ubuntu 19.10.1 on it.

qemu-system-aarch64 -s -nographic -enable-kvm -cpu host -machine virt -kernel ./loader.img -m 1024M -append "--verbose --nomount --maxnic=0 /tools/uush.so" -smp 1

It works with a single vCPU. Once I run with 2 or more it hangs right after (or in) smp_init():

OSv v0.54.0-68-gabf9cba5

PSCI: version 65536.0 detected.

setup_arm_clock() ENTERED, lr=00000000400c6990

arm_clock(): frequency read as 000000000337f980

interrupt_table::interrupt_table() ENTERED, lr=00000000400a8244

gic_driver::init_cpu() ENTERED, lr=00000000401f1cc0

CPU interface enabled.

gic_driver::init_dist() ENTERED, lr=00000000401f1ccc

number of supported IRQs: 0000000000000120

interrupt table: gic driver created.

 registered IRQ id=0000000000000004

 registered IRQ id=000000000000001b

 registered IRQ id=0000000000000000

Firmware vendor: Unknown

smp_launch ENTERED, lr=00000000400c7c28

The same image works with 1, 2 or more CPUs in emulated setup (see 1 example above).

I have also spent a bit of time trying to run the same example on Firecracker on Raspberry PI 4 (same setup as with qemu/kvm). Unfortunately, I could not even see the 1st boot line (OSv ...) to show up. I think that firecracker does not emulate pl011, but instead some sort of mmio-based console whose configuration is appended to Linux command line like so:

cmdline
	.insert("earlycon", &format!("uart,mmio,0x{:08x}", self.mmio_base))
	.map_err(Error::Cmdline)?;

I am not exactly sure what this device is but figuring it out and implementing it would be the 1st step to make OSv aarch64 run on firecracker arm version.  

Regards,

Waldek

[Rick Payne]

On Wed, 2020-01-01 at 21:17 -0800, Waldek Kozaczuk wrote:
> I had to manually add uush.so to usr_ramfs.manifest. For whatever
> reason, the image built on Ubuntu would never boot at all. I wonder
> if that has to do with some mixed up gcc path on Ubuntu and maybe we
> end up compiling and/or linking against some wrong artifacts
> (external/host maybe related).

When I last looked at this, it was something to do with the
instantiation of the early console. There's a thread where Nadav and I
discussed it. His build hit the dtb issue you have no fixed, whereas I
got what you see.

I have still never got that part working (though I've not put much
effort in, to be fair). I assume its some compiler difference. For any
aarch64 stuff I've been doing, I've resorted to a fedora29 vm :(

Rick

[Nadav Har'El]

On Thu, Jan 2, 2020 at 7:17 AM Waldek Kozaczuk <jwkoz...@gmail.com> wrote:

Happy New Year everyone!

A couple of days ago I sent a patch that fixes an issue with large DTB and thus brings the arm port of OSv back to life.

Thanks, great. I can confirm the "dtb too large?" problem. I'll review and commit your patch.

I have tested it on Fedora 29 and was able to boot and run a simple uush.so example on both Fedora and Ubuntu QEMU with aarch64 emulation:

./scripts/build -j4 image=empty fs=ramfs arch=aarch64
qemu-system-aarch64 -s -nographic -machine virt -machine gic-version=2 -kernel ./arm/loader.img -cpu cortex-a57 -m 1024M -append "--verbose --nomount --maxnic=0 /tools/uush.so" -smp 2

I had to manually add uush.so to usr_ramfs.manifest.

We can easily have a uush module in module/, so one would be able to scripts/build image=uush and get it.

Does this make sense?

One nice thing we could do (but not urgent) is to scripts/run.py, if it sees build/latest is a different architecture, it could call qemu-system-* to run it differently.

 

--
You received this message because you are subscribed to the Google Groups "OSv Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osv-dev+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osv-dev/fa6b2072-6057-493d-a4d6-a10ed4a48f45%40googlegroups.com.