About android emulator on syzkaller.

[Jeremy Huang]

Hi Dmitry,

I'd like to fuzz android kernel via emulator on syzkaller.

So far, the syzkaller only supported arm development board.

So, I tried to modify minimally the vm/adb.go, just like the vm/qemu.go.

But I still can't get the console ? just saw the program 0 and crashes 0...

Could you refer the attachment to help me :)

Jeremy Huang

[Dmitry Vyukov]

Hi Jeremy,

What exactly is "emulator"? Can you make it work without syzkaller? Do
you get console output without syzkaller?
What is the sequence of commands to start it, ssh into it, read console output?

We simply used qemu in emulation mode to test arm64 android kernel on
x86 host. It worked out of the box with "qemu" type.
We also run x86_64 android kernel with debian userspace in qemu-kvm on x86 host.

[Dmitry Vyukov]

On Sun, Jan 22, 2017 at 4:01 AM, Jeremy Huang <jere...@gmail.com> wrote:
> Hi Dmitry,

> I followed you mentioned above and changed my config as following :
> {
> "http": "localhost:5000",
> "workdir": "/home/w4118/syzkaller/workdir",
> "kernel": "/home/w4118/utils/goldfish/arch/x86/boot/bzImage",
> "syzkaller": "/home/w4118/syzkaller",
> "type": "qemu",
> "vmlinux": "-",
> "Image": "9p",
> "initrd":
> "/home/w4118/utils/android-sdk-linux/system-images/android-24/default/x86_64/ramdisk.img",
> "Sshkey": "/home/w4118/syzkaller/tools/ssh/id_rsa",
> "procs": 8,
> "count": 4,
> "cpu": 2,
> "mem": 2048,
> "cover": false,
> "sandbox": "none"
> }
>
> It will appear the root fs error before I added the "initrd" to my config.
> I though that it may have been caused by `root=/dev/root`.
> https://github.com/google/syzkaller/blob/master/vm/qemu/qemu.go#L199
> However, I add the initrd portin to ramdisk.img in my config.
> it showed `binder: 1200:1200 transaction failed 29189, size 0-0'.
> What should I set syzkaller on the goldfish-android-kernel-3.18 ?

Binder is something android-specific, right?
I don't know about android. It is a strange system.


We've opened some docs on using android/arm64 with qemu, maybe you
will find something useful there:
https://github.com/google/syzkaller/wiki/Fuzzing-ARM64-kernels-with-Syzkaller-on-QEMU



> I recall that used to initrd file instead of `root=/dev/sda' on SUSE os.
> Then it does work.
>
> Jeremy
>
> 2017-01-19 17:34 GMT+08:00 Dmitry Vyukov <dvy...@google.com>:
>>
>> On Thu, Jan 19, 2017 at 10:12 AM, Jeremy Huang <jere...@gmail.com>
>> wrote:
>> > Wow! Many Thanks.
>> > Should I modify the `ssh' part of qemu.go instead of `adb' part of
>> > adb.go ?
>>
>> Depending on what image you will choose.
>> If you use a debian userspace, then it does support ssh. So everything
>> should work out-of-the-box. I use this script to build a debian
>> userspace and it does work with x86 android kernel:
>> https://github.com/google/syzkaller/blob/master/tools/create-image.sh
>> Also "9p" option should work with ssh.
>>
>>
>> > I remembered that android kernel doesn't supported ssh defaultly.
>> > What does the 9p mean?
>>
>> 9p option shares part of your _host_ image as guest image (in
>> read-only mode, of course, so your host won't be corrupted as the
>> result of fuzzing). I.e. your host /bin becomes guest's /bin, etc. But
>> it creates private /etc with a simple config for sshd. You can see the
>> init script here:
>> https://github.com/google/syzkaller/blob/master/vm/qemu/qemu.go#L384
>> (search for 9p in that file for other details).
>>
>>
>>
>> > Jeremy Huang
>> >
>> > 2017-01-19 16:54 GMT+08:00 Dmitry Vyukov <dvy...@google.com>:
>> >>
>> >> On Thu, Jan 19, 2017 at 9:35 AM, Jeremy Huang <jere...@gmail.com>

>> >> wrote:
>> >> >
>> >> >
>> >> > 2017-01-19 15:29 GMT+08:00 Dmitry Vyukov <dvy...@google.com>:
>> >> >>
>> >> >> On Thu, Jan 19, 2017 at 7:58 AM, Jeremy Huang <jere...@gmail.com>
>> >> >> wrote:
>> >> >> > Hi Dmitry,
>> >> >> >
>> >> >> > I'd like to fuzz android kernel via emulator on syzkaller.
>> >> >> > So far, the syzkaller only supported arm development board.
>> >> >> > So, I tried to modify minimally the vm/adb.go, just like the
>> >> >> > vm/qemu.go.
>> >> >> > But I still can't get the console ? just saw the program 0 and
>> >> >> > crashes
>> >> >> > 0...
>> >> >> > Could you refer the attachment to help me :)
>> >> >>
>> >> >> Hi Jeremy,
>> >> >>
>> >> >> What exactly is "emulator"?
>> >> >

>> >> > it's android emulator :
>> >> > https://android.googlesource.com/platform/prebuilts/android-emulator

>> >> >
>> >> >> Can you make it work without syzkaller?
>> >> >

>> >> > Yes, I can. just command :
>> >> > `emulator @x86 -kernel arch/x86/boot/bzImage -show-kernel
>> >> > -no-window
>> >> > -qemu --enable-kvm`
>> >> > the kernel is android kernel 3.18 version.

>> >> >
>> >> >> Do you get console output without syzkaller?
>> >> >

>> >> > Yes, I can get it.

>> >> >>
>> >> >> What is the sequence of commands to start it, ssh into it, read
>> >> >> console
>> >> >> output?
>> >> >

>> >> > Firstly, I patched adb.go and executed the command :
>> >> > `emulator @x86 -kernel arch/x86/boot/bzImage -show-kernel
>> >> > -no-window
>> >> > -qemu --enable-kvm`
>> >> > Then, launched syzkaller.

>> >> >
>> >> >>
>> >> >> We simply used qemu in emulation mode to test arm64 android kernel
>> >> >> on
>> >> >> x86 host. It worked out of the box with "qemu" type.
>> >> >> We also run x86_64 android kernel with debian userspace in qemu-kvm
>> >> >> on
>> >> >> x86
>> >> >> host.
>> >> >
>> >> >

>> >> > Great. it sounds that I just want to do.
>> >> > The following config file that I used.
>> >> > {
>> >> > "http": "localhost:5000",
>> >> > "workdir": "/home/w4118/syzkaller/workdir",
>> >> > "vmlinux": "-",
>> >> > "syzkaller": "/home/w4118/syzkaller",
>> >> > "type": "adb",
>> >> > "procs": 1,
>> >> > "cover": false,
>> >> > "sandbox": "none",
>> >> > "devices": ["emulator-5554"]
>> >> > }
>> >> > So, I need to change the type to "qemu" from "adb" ?
>> >> > And somethings else I need to do?
>> >>
>> >>
>> >> Yes, it should be simpler with qemu.
>> >> You also need "kernel" parameter to point to the bzImage file. And
>> >> "image" parameter to point to the image file. It seems that the
>> >> emulator has some prebuilt image file, if you figure out how the
>> >> emulator starts qemu (the command line), then you can extract the
>> >> image file from the command line (it should be in -hda argument).
>> >> Alternatively you can try to specify "9p" in image parameter, then
>> >> syzkaller will build a simple 9p-based image based on your host
>> >> system.
>> >> Initially run syz-manager with -debug flag, then it will print
>> >> commands it is executing and also qemu output.
>> >
>> >
>
>

[seye...@uci.edu]

Hi Jeremy

I am  doing exactly the same thing. I was wondering if you had any luck to get it to work.

Either fuzzing the kernel with adb or ssh?

[Dmitry Vyukov]

qemu VM should work. You don't need android emulator, nor android
userspace to test android kernel.

[Jeremy Huang]

Hi Seyehat,
We has built syzkaller for android emulator kernel(goldfish), so
we don't need really android devices.

Firstly, Need to modify kernel and gcc to support the KASAN.
Then we modify syzkaller that can fuzz goldfish via adb.

goldfish kernel 3.18 with kasan feature.
https://github.com/superguo/android_goldfish3_18_kasan

gcc-kasan-android-prebuilt (compile goldfish with kasan feature. )
https://github.com/superguo/gcc-kasan-android-prebuilt

syzkaller for fuzzing android kernel via adb
https://github.com/superguo/syzkaller/releases

And we found :

keychord BUG: Double free or freeing an invalid pointer
https://github.com/superguo/syzkaller/wiki/Found-bugs-for-Android

> --
> You received this message because you are subscribed to the Google Groups
> "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to syzkaller+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[seye...@uci.edu]

Thanks alot

[dar...@gmail.com]

Hi guys, I have re-built my kernel and have my Android device on a debug build to run adb as root. But on this final step, the kernel is too big to go into the boot image.... I have no idea how to fix this, I've spend the last few hours trying to slim the kernel down to no avail. Having debug symbols in the kernel really blows up the size of the Android kernel. I'm on a Google Pixel XL. I gave up on that for a little bit, and now am trying QEMU with the Android kernel, but I've never been able to get a QEMU VM to boot up an Android kernel (LATER Android kernels... like 2017 which use the Image.gz-dtb format).  Do you guys have any suggestions?

[Jeremy Huang]

2017-09-29 10:58 GMT+08:00 <dar...@gmail.com>:
> Hi guys, I have re-built my kernel and have my Android device on a debug
> build to run adb as root. But on this final step, the kernel is too big to
> go into the boot image.... I have no idea how to fix this, I've spend the
> last few hours trying to slim the kernel down to no avail. Having debug
> symbols in the kernel really blows up the size of the Android kernel. I'm on
> a Google Pixel XL. I gave up on that for a little bit, and now am trying
> QEMU with the Android kernel, but I've never been able to get a QEMU VM to
> boot up an Android kernel

Goldfish is a android kernel from google for android emulator.
We ported the KASAN to Goldfish kernel 3.18, so can fuzz without real devices.

You need to download below source code firstly.

goldfish kernel 3.18 with kasan feature.
https://github.com/superguo/android_goldfish3_18_kasan

gcc-kasan-android-prebuilt (compile goldfish with kasan feature. )
https://github.com/superguo/gcc-kasan-android-prebuilt

Then build and run the kernel,

make -j9 ARCH=arm64
CROSS_COMPILE=~/utils/gcc-kasan-android-prebuilt/aarch64-linux-android-4.9/bin/aarch64-linux-android-
2>&1 |tee kernel_make.out

emulator -avd arm64 -verbose -kernel arch/arm64/boot/Image.gz
-show-kernel -no-window 2>&1 | tee boot_errors

Now you hava a fuzzed kernel with KASAN feature.

Jeremy Huang

[Dmitry Vyukov]

On Fri, Sep 29, 2017 at 4:58 AM, <dar...@gmail.com> wrote:
> Hi guys, I have re-built my kernel and have my Android device on a debug
> build to run adb as root. But on this final step, the kernel is too big to
> go into the boot image.... I have no idea how to fix this, I've spend the
> last few hours trying to slim the kernel down to no avail. Having debug
> symbols in the kernel really blows up the size of the Android kernel. I'm on
> a Google Pixel XL. I gave up on that for a little bit, and now am trying
> QEMU with the Android kernel, but I've never been able to get a QEMU VM to
> boot up an Android kernel (LATER Android kernels... like 2017 which use the
> Image.gz-dtb format). Do you guys have any suggestions?

Please direct KASAN questions to kasan-dev@ mailing list.
In fact, your question was already answered there few days ago:
https://groups.google.com/d/msg/kasan-dev/Ppz85NMGWLg/vHC6DAujAgAJ

[Nickleman]

Isn't "CONFIG_KCOV" required to run syzkaller?

This kernel 3.18 mentioned here does not have KCOV.

[Hamid Tavakoli]

You can port it to the kernel

You received this message because you are subscribed to a topic in the Google Groups "syzkaller" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/syzkaller/cpBR8RzQH-E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to syzkaller+unsubscribe@googlegroups.com.

[Nickleman]

Yes. But I'm wondering if Jeremy is running syzkaller without KCOV.

To unsubscribe from this group and all its topics, send an email to syzkaller+...@googlegroups.com.

[Jeremy Huang]

Hi, Nickleman
I forgot to say that I have ported kcov already on android goldfish 3.18.
It's easier than ported KASAN...

https://github.com/superguo/android_goldfish3_18_kasan/blob/master/kernel/kcov.c

ref :
https://lwn.net/Articles/671640/

[Nickleman]

Hi Jeremy, I've been trying to use your syzkaller with an android emulator but without success.

It seems syzkaller tries to reboot the emulator, but this causes the emulator to freeze with the following message:

reboot: Restarting system with command ''

Reboot failed -- System halted

Could you share your syzkaller configuration?