How to fuzz some parts of kernel

[thesis79]

Hello, Dmitry.

I'm a engineer who develops a smartphone using android.

I'm interested in testing kernel using syz-kaller and I believe syz-kaller can help me.

The kernel source I handle is composed of 4 parts.

They are main line kernel, aosp part, soc vendor driver and oem driver.

What I want to do with syz-kaller is testing soc vendor driver and oem driver.

I don't think I can find much more bugs from main line kernel and aosp kernel because it's been already tested enough.

So I want to concentrate on soc vendor driver and oem driver.

My first question is

Can you recoemmend a good way to test only the codes I want?

I already know how to test a specific driver code by adding and enabling extra syscalls seeing some articles and ppts.

Is that the best way to do it?

Second is how I can know if test is done enough.

In my opinion, one of ways to do it is checking coverage percentage.

Of course I know you think coverage percentage is useless

Then, do you have another way to know it?

Anyway I have to be able to know when the test is done enough so that I can stop testing.

please give me advice for that

Thank you

[Dmitry Vyukov]

On Wed, Mar 7, 2018 at 6:22 AM, thesis79 <thes...@gmail.com> wrote:
> Hello, Dmitry.
>
> I'm a engineer who develops a smartphone using android.
> I'm interested in testing kernel using syz-kaller and I believe syz-kaller
> can help me.
>
> The kernel source I handle is composed of 4 parts.
> They are main line kernel, aosp part, soc vendor driver and oem driver.
>
> What I want to do with syz-kaller is testing soc vendor driver and oem
> driver.
> I don't think I can find much more bugs from main line kernel and aosp
> kernel because it's been already tested enough.
> So I want to concentrate on soc vendor driver and oem driver.
>
> My first question is
> Can you recoemmend a good way to test only the codes I want?
> I already know how to test a specific driver code by adding and enabling
> extra syscalls seeing some articles and ppts.
> Is that the best way to do it?

Hi,

Yes, specify the relevant syscalls in "enable_syscalls" config
section. You can specify particular discriminations of syscalls, e.g.
"openat$my_driver" rather than "openat".

> Second is how I can know if test is done enough.
> In my opinion, one of ways to do it is checking coverage percentage.
> Of course I know you think coverage percentage is useless
> Then, do you have another way to know it?
> Anyway I have to be able to know when the test is done enough so that I can
> stop testing.

At the very least you need to look at coverage and ensure that
everything you expect to be covered is covered. E.g. one can misspell
/dev file name in openat syscall, and them fuzz for weeks and even get
some coverage (from other enabled syscalls), but this driver is not
covered at all. Sometimes it's permissions on /dev file that do not
allow to open it under the specified sandbox. Or sometimes layout an
important struct is incorrect in descriptions and it leads to the same
result -- the code is not covered at all.

Once the code is covered, there is no general recipe as to when to
stop. The longer you fuzz the more bugs you find. Probability of
finding new bugs diminishes over time, but it is never zero. At the
very least I think you want to get N days of fuzzing with no crashes
at all (all previous bugs need to be fixed) before stopping fuzzing.

[thesis79]

2018년 3월 7일 수요일 오후 3시 9분 23초 UTC+9, Dmitry Vyukov 님의 말:

I'm curious of detailed way to check coverage.

I mean  when I add a new driver. Can I know how many basic blocks are added to total kernel binary?

As you know, kernel is single binary so I'm not sure if can count only basic blocks made from the new driver source.

Do you know how to do that?

[Dmitry Vyukov]

You can open HTML coverage report from main syz-manager web page ("cover"
link). It will show how many lines are covered in each particular file.

>> /dev file name in openat syscall, and them fuzz for weeks and even get
>> some coverage (from other enabled syscalls), but this driver is not
>> covered at all. Sometimes it's permissions on /dev file that do not
>> allow to open it under the specified sandbox. Or sometimes layout an
>> important struct is incorrect in descriptions and it leads to the same
>> result -- the code is not covered at all.

>> Once the code is covered, there is no general recipe as to when to
>> stop. The longer you fuzz the more bugs you find. Probability of
>> finding new bugs diminishes over time, but it is never zero. At the
>> very least I think you want to get N days of fuzzing with no crashes
>> at all (all previous bugs need to be fixed) before stopping fuzzing.

> --
> You received this message because you are subscribed to the Google Groups
"syzkaller" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to syzkaller+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.