trouble at making syzkaller on aarch64 working

[Robin Meinzer]

Hi, I am trying to run syzkaller on my Raspberry Pi 3B to do some fuzzing.

 

I've followed the guide for setting up syzkaller for an aarch64 device + QEMU, and this is how far I got for now:

 

- properly configured .config file for the linux kernel and compiled the kernel to aarch64 Image

- compiled syzkaller for aarch64

- built rootfs.ext3 with buildroot as shown in the guide. I noticed here that rootfs.ext3 is just a symbolic link to rootfs.ext2 though (size 171MB)

 

Now I fail at booting the VM in QEMU. I dragged the linux kernel image, syzkaller directory and rootfs.ext2 to my Raspberry Pi, and executed this shell script as root:

#!/bin/sh

/usr/bin/qemu-system-aarch64 \
        -machine virt \
        -cpu cortex-a57 \
        -nographic -smp 1 \
        -hda /root/rootfs.ext3 \ 
        -kernel /root/Image \
        -append "console=ttyAMA0 root=/dev/vda oops=panic panic_on_warn=1 panic=-1 f
trace_dump_on_oops=orig_cpu debug earlyprintk=serial slub_debug=UZ" \
        -m 512 \
        -net user,hostfwd=tcp::10023-:22 -net nic
 

I can see how the linux kernel is booting and initializing its components, but then after some time the kernel panics and QEMU reports that it's unable to mount root fs on unknown block 0 0. The result is that I cannot login to the machine. I tried to use rootfs.ext2 instead of the ext3 symbolic link, it didn't work. I also tried using -drive file /root/rootfs.ext3,format=raw instead of -hda, but it doesn't work.
 

Any help is appreciated.

[Dmitry Vyukov]

Hi Robin,

Most likely the problem is either with -hda flag name, e.g. on some
arm's one needs to use "sd":
docs/linux/setup_linux-host_qemu-vm_arm-kernel.md: "image_device": "sd",

or, with filesystem support (e.g. kernel does not have ext4 compiled in).
The error message may contain some hints, e.g. list of available block
devices or something.

[Robin Meinzer]

 

 

 

Gesendet: Mittwoch, 09. September 2020 um 14:25 Uhr
Von: "Dmitry Vyukov" <dvy...@google.com>
An: "Robin Meinzer" <robmei...@gmx.de>
Cc: "syzkaller" <syzk...@googlegroups.com>
Betreff: Re: trouble at making syzkaller on aarch64 working

---

 

I played around with the my booting shell script, and you were right, the error message contains vda to be booted from:

[  217.389138] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[  217.403335] Please append a correct "root=" boot option; here are the available p
artitions:
[  217.433429] fe00         6000000 vda
[  217.439584]  driver: virtio_blk
[  217.460063] 1f00          131072 mtdblock0
[  217.463074]  (driver?)
[  217.485016] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-b
lock(0,0)
 

So I tried root=/dev/vda but also root=vda and root=mtdblock0, still the same message.

 

If I replace the -hda flag with -sd, QEMU won't boot:

qemu-system-aarch64: -sd /root/rootfs.ext2: machine type does not support if=sd,bus=0,unit=0 (in the aarch64 docs you use virt)

 

Note that the docs reference you sent is for 32bit arm, I'm trying to run it on aarch64. I see that for 32bit arm you use machine type vexpress, but I don't know if it's compatible with aarch64 (new built rootfs needed).

[Dmitry Vyukov]

+Mark, Will, maybe you know off the top of your heads what drive flag
needs to be passed to qemu-system-aarch64 if kernel complains about
missing root block device?

Mark, IIRC you also mentioned that the way to do it is virtio rather
than -sda/md, much faster and works more reliably?

[Will Deacon]

On Wed, Sep 09, 2020 at 04:42:59PM +0200, Dmitry Vyukov wrote:
> +Mark, Will, maybe you know off the top of your heads what drive flag
> needs to be passed to qemu-system-aarch64 if kernel complains about
> missing root block device?

Don't recall off the top of my head, but maybe my HOWTO might help? It's for
an x86 host, but it doesn't look like you're using KVM so the same options
should hopefully apply:

https://mirrors.edge.kernel.org/pub/linux/kernel/people/will/docs/qemu/qemu-arm64-howto.html

Will

[Robin Meinzer]

I followed the how-to guide provided by Will and achieved to boot into debian with a standard 4.19 kernel.

 

Loading Linux 4.19.0-10-arm64 ...
Loading initial ramdisk ...
/dev/vda2: recovering journal
/dev/vda2: clean, 30993/987360 files, 321051/3943936 blocks
 

Debian GNU/Linux 10 qemu ttyAMA0
qemu login: (works)

 

I want to use the newest linux kernel (5.9 rc4) with a custom config which is described in the syzkaller docs: https://github.com/google/syzkaller/blob/master/docs/linux/setup_linux-host_qemu-vm_arm64-kernel.md

This is the config file I copied from Will's guide and it works without the -kernel and -append flags (it will boot the 4.19 default kernel)
 

qemu-system-aarch64 -M virt        \
      -machine virtualization=true -machine virt,gic-version=3  \
      -cpu max,sve=off -smp 2 -m 512 \
      -kernel /root/Image \
      -append "earlycon root=/dev/vda2" \
      -drive if=pflash,format=raw,file=efi.img,readonly     \
      -drive if=pflash,format=raw,file=varstore.img         \
      -drive if=virtio,format=qcow2,file=disk.img           \
      -device virtio-scsi-pci,id=scsi0              \
      -object rng-random,filename=/dev/urandom,id=rng0      \
      -device virtio-rng-pci,rng=rng0               \
      -device virtio-net-pci,netdev=net0                \
      -netdev user,id=net0,hostfwd=tcp::8022-:22            \
      -nographic

When I try to use my already compiled kernel (yes, I checked for ext4 support) as shown in the syzkaller docs or in the guide sent by Will, I get the same issue just like before, the kernel panics without having a root device to mount:

 

[   15.226144] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[   15.226825] Please append a correct "root=" boot option; here are the available partitions:
[   15.228423] fe00        16777216 vda
[   15.228575]  driver: virtio_blk
[   15.230558]   fe01          524288 vda1 9af40d4f-b7a0-499b-b89c-5c3d6a209eda
[   15.230655]
[   15.231690]   fe02        15775744 vda2 77c94b83-2ee4-4fb9-bc4c-9b6ed213db51
[   15.231755]
[   15.232793]   fe03          475136 vda3 29a69162-100c-4a91-9a1b-88abdb28c555
[   15.233635]
[   15.235592] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
 

I tried vda1, vda2, vda3, also with prepended /dev/, won't work. So the issue is probably the custom kernel, here's my config file:
https://pastebin.com/raw/HS53tEi2

Dmitry once said it might be lacking ext fs driver, but I see that options CONFIG_EXT2..4_FS are set.