How to read console log from adb

[xfran...@gmail.com]

Hi All, 

I know syzkaller can support android phone now. But my phone does not have a console or any serial port. 

I want to change syzkaller, so I can read console log from adb ( throuth 'cat /proc/kmsg' or 'dmesg').

Once syzkaller has this capability, lots of people will be very happy.

 

Anyone can teach me how to do this? (Like which go file should I change?)

Thanks,

xFrank.

[Dmitry Vyukov]

Hello xFrank,


The file you need is vm/adb/adb.go.
In particular Run function here:
https://github.com/google/syzkaller/blob/master/vm/adb/adb.go#L328
It streams console output.

Try to revert this commit:
https://github.com/google/syzkaller/commit/a074da17a4055352fea94afbd5a15c53d0946653
Then comment out findConsole call in ctor function.
And then replace in Run function:
- cat := exec.Command("cat", inst.console)
+ cat := exec.Command(inst.cfg.Bin, "-s", inst.cfg.Device, "shell",
"tail -f /var/log/messages")
(maybe you need to change "tail -f /var/log/messages" to something
else that works on android, not sure what it is)
and maybe that will be enough.

[$rik@nth]

Hi xfrank,

Please let me know if it works for you :) i will also move from the
commit which Dmitry suggested to revert and apply new changes.

> --
> You received this message because you are subscribed to the Google Groups "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Thanks & Regards,
M.Srikanth Kumar.

[Dmitry Vyukov]

Just to make it clear, I would like to preserve the current code that
works with console as well. We can have 2 modes: one with console
(better) and one with adb (worse, but does not require console).

Console output is much more reliable.
If a console is writable, we could also use it to reboot the device
(which would be more reliable than the current mechanism).
We could also use console to obtain more info for e.g. these "no
output" reports. If we would send sysrq l, t and d over the console,
it would provide lots of valuable information regarding kernel state.

[xfran...@gmail.com]

Hi Dmitry & Srikanth,

The solution works, and the exact cmd used for read log is "adb shell cat /proc/kmsg" in Run function.

I got  another issues:

1, I found syz-manager keeps copy syz-fuzzer and syz-exector to the device throug adb. Is it really nessaray?

2, syz-fuzzer init failed.

after some work myself, I find the issue point:

in fuzzer.go,  ipc.MakeEnv is called. Then, in ipc.go, function createMapping calls ioutil.TempFile("./", "syzkaller-shm") to create a temp file.

"./" is a root dir. But the root dir in android is readonly. I change the dir to "/data/local/tmp/", works.

please think about this patch.

And, this temp dir is not deleted.

3, syz-exector is not work properly.

log:

panic: executor is not serving:

cover mmap failed (errno 19)

It turns out that mmap to "/sys/kernel/debug/kcov" failed in cover_open function in executor.cc. I don't know if it's a "android device issue", or just something wrong with my Phone Under Test.

Please help me with issue.

Thanks.

[Dmitry Vyukov]

On Thu, Jan 12, 2017 at 3:26 AM, <xfran...@gmail.com> wrote:
> Hi Dmitry & Srikanth,
> The solution works, and the exact cmd used for read log is "adb shell cat
> /proc/kmsg" in Run function.
>
> I got another issues:
>
> 1, I found syz-manager keeps copy syz-fuzzer and syz-exector to the device
> throug adb. Is it really nessaray?

Yes, these binaries need to be on the device.

> 2, syz-fuzzer init failed.
> after some work myself, I find the issue point:
> in fuzzer.go, ipc.MakeEnv is called. Then, in ipc.go, function
> createMapping calls ioutil.TempFile("./", "syzkaller-shm") to create a temp
> file.
> "./" is a root dir. But the root dir in android is readonly. I change the
> dir to "/data/local/tmp/", works.
> please think about this patch.
> And, this temp dir is not deleted.

How cwd ended up being root /?
vm/adb/adb.go:instance.Run starts the process as:
adb := exec.Command(inst.cfg.Bin, "-s", inst.cfg.Device, "shell", "cd
/data; "+command)
so cwd should be /data.

> 3, syz-exector is not work properly.
> log:
> panic: executor is not serving:
> cover mmap failed (errno 19)
>
> It turns out that mmap to "/sys/kernel/debug/kcov" failed in cover_open
> function in executor.cc. I don't know if it's a "android device issue", or
> just something wrong with my Phone Under Test.
> Please help me with issue.

Did you build kernel with CONFIG_KCOV? You need CONFIG_KCOV for syzkaller.

[xfran...@gmail.com]

1,I mean it's not nessrary to copy this two file every loop. 

2,I don't know why, the log said "fail because readonly"

3,I did. check the cover_open function, mmap is behind open and ioctl call:

void cover_open()

{

if (!flag_cover)

return;

for (int i = 0; i < kMaxThreads; i++) {

thread_t* th = &threads[i];

th->cover_fd = open("/sys/kernel/debug/kcov", O_RDWR);

if (th->cover_fd == -1)

fail("open of /sys/kernel/debug/kcov failed");

if (ioctl(th->cover_fd, KCOV_INIT_TRACE, kCoverSize))

fail("cover init write failed");

th->cover_data = (uint64_t*)mmap(NULL, kCoverSize * sizeof(th->cover_data[0]), PROT_READ | PROT_WRITE, MAP_SHARED, th->cover_fd, 0);

if ((void*)th->cover_data == MAP_FAILED)

fail("cover mmap failed");

}

}

在 2017年1月12日星期四 UTC+8下午4:59:26，Dmitry Vyukov写道：

[xfran...@gmail.com]

I add some logging code in the kcov_ioctl_locked and kcov_mmap functions of linux/kcov.c, rebuild the kernel, reflash to the device.

Then do the test.

The  kcov_ioctl_locked's log shows up, but  kcov_mmap's does not !

So, the mmap syscall did not get into kcov!

在 2017年1月12日星期四 UTC+8下午6:25:17，xfran...@gmail.com写道：

[Dmitry Vyukov]

On Thu, Jan 12, 2017 at 12:21 PM, <xfran...@gmail.com> wrote:
> I add some logging code in the kcov_ioctl_locked and kcov_mmap functions of
> linux/kcov.c, rebuild the kernel, reflash to the device.
> Then do the test.
> The kcov_ioctl_locked's log shows up, but kcov_mmap's does not !
> So, the mmap syscall did not get into kcov!

errno 19 is ENODEV
I am not sure in what situation open would succeed but mmap would
return ENODEV...
Maybe debugfs access is restricted to root, and you are running not
under root. Try to add "sandbox": "none" to config file, and try to
chmod -R a+rwx /sys/kernel/debug

[Dmitry Vyukov]

On Thu, Jan 12, 2017 at 11:25 AM, <xfran...@gmail.com> wrote:
> 1,I mean it's not nessrary to copy this two file every loop.

How then we will detect that the ones on the device are up-to-date and
are not broken? Does it matter at all?

> 2,I don't know why, the log said "fail because readonly"

Does it work if you change "/data/local/tmp/" to just "/data"?

[xfran...@gmail.com]

sandbox is "none" already, chmod is done, still the same issue.

[Dmitry Vyukov]

On Thu, Jan 12, 2017 at 1:27 PM, <xfran...@gmail.com> wrote:
> sandbox is "none" already, chmod is done, still the same issue.

Hummm... I can only suggest to add printk's earlier in mmap syscall
and track where/why it bails out with ENODEV.

[Jeremy Huang]

Hi xfrank

Is your syz-executor compiled to arm64 format ?

Cross compiling for android arm64 

https://github.com/google/syzkaller/wiki/Cross-compiling-for-android-arm64

Dmitry Vyukov於 2017年1月12日星期四 UTC+8下午7時34分50秒寫道：

[Jeremy Huang]

hi Dmitry Vyukov

After I revert above commit you said, then still got errors :

2017/01/12 08:23:24 executing adb [shell pwd]

2017/01/12 08:23:24 adb returned

2017/01/12 08:23:24 [adb.go] findConsole()

2017/01/12 08:23:24 loop: instance 0 finished, crash=false

2017/01/12 08:23:24 failed to create instance: no unassociated console devices left

Dmitry Vyukov於 2017年1月12日星期四 UTC+8下午8時38分17秒寫道：

[Dmitry Vyukov]

On Thu, Jan 12, 2017 at 2:27 PM, Jeremy Huang <jere...@gmail.com> wrote:
> hi Dmitry Vyukov
>
> After I revert above commit you said, then still got errors :
>
> 2017/01/12 08:23:24 executing adb [shell pwd]
> 2017/01/12 08:23:24 adb returned
> 2017/01/12 08:23:24 [adb.go] findConsole()
> 2017/01/12 08:23:24 loop: instance 0 finished, crash=false
> 2017/01/12 08:23:24 failed to create instance: no unassociated console
> devices left

That's expected. Also follow the rest of the instructions (in
particular comment out findConsole call).

[xfran...@gmail.com]

Yes.

[xfran...@gmail.com]

Hi Dmitry,

After whole day working, we finally found out the point. 

The latest kernel has removed mmap from debugfs!

The kcov patch should be updated, check this out: https://patchwork.kernel.org/patch/9131859/

>>  static int __init kcov_init(void)
>>  {
>> -       if (!debugfs_create_file("kcov", 0600, NULL, NULL, &kcov_fops)) {
>> +       if (!debugfs_create_file_unsafe("kcov", 0600, NULL, NULL, &kcov_fops)) {

------
It works for me now.

[$rik@nth]

Hi xfrank,

Can you help sharing the steps/Patches which you are trying to enable
KCOV to compile kernel for ARM/ARM64?

> --
> You received this message because you are subscribed to the Google Groups
> "syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to syzkaller+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[xfran...@gmail.com]

Hi srikanth,

This is the steps/patches I used: 

https://github.com/google/syzkaller#linux-kernel

https://github.com/google/syzkaller/wiki/Kernel-configs

And for arm64, the patch I used is coming from Flanker (i...@flanker017.me):

------------------------------------------------------------

 arch/arm64/Kconfig                |  1 +

 arch/arm64/kernel/Makefile     |  2 ++

 arch/arm64/lib/Makefile           |  2 ++

------------------------------------------------------------

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig

index 5fdf947..d625666 100644

--- a/arch/arm64/Kconfig

+++ b/arch/arm64/Kconfig

@@ -7,6 +7,7 @@ config ARM64

  select ARCH_HAS_ALT_SYSCALL

  select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE

  select ARCH_HAS_SG_CHAIN

+        select ARCH_HAS_KCOV

  select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST

  select ARCH_USE_CMPXCHG_LOCKREF

  select ARCH_SUPPORTS_ATOMIC_RMW

diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile

index bedcbec..bbae5f1 100644

--- a/arch/arm64/kernel/Makefile

+++ b/arch/arm64/kernel/Makefile

@@ -7,6 +7,8 @@ AFLAGS_head.o := -DTEXT_OFFSET=$(TEXT_OFFSET)

 CFLAGS_efi-stub.o := -DTEXT_OFFSET=$(TEXT_OFFSET)

 CFLAGS_armv8_deprecated.o := -I$(src)

 

+KCOV_INSTRUMENT_efi-stub.o := n

+KCOV_INSTRUMENT_stacktrace.o := n

 KASAN_SANITIZE_efi-stub.o := n

 KASAN_SANITIZE_stacktrace.o := n

 CFLAGS_REMOVE_ftrace.o = -pg

diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile

index d98d3e3..a1759a7 100644

--- a/arch/arm64/lib/Makefile

+++ b/arch/arm64/lib/Makefile

@@ -1,3 +1,5 @@

+# Produces uninteresting flaky coverage.

+KCOV_INSTRUMENT_delay.o := n

 lib-y := bitops.o clear_user.o delay.o copy_from_user.o \

    copy_to_user.o copy_in_user.o copy_page.o \

    clear_page.o memchr.o memcpy.o memmove.o memset.o \

[$rik@nth]

Thank you. can you confirm on which kernel you are working. I'm on
3.18.* and not able to kcov related code as mentioned in
https://github.com/google/syzkaller#linux-kernel
https://github.com/google/syzkaller/wiki/Kernel-configs