Re: MSan: Uninitialized Memory in do_sys_mknodat

[Evgenii Stepanov]

Could you provide a reproducer? Preprocessed source + compilation flags.

On Sat, Feb 22, 2020, 22:58 Maxime Villard <m...@m00nbsd.net> wrote:

Le 22/02/2020 à 14:15, 'Dmitry Vyukov' via syzkaller-netbsd-bugs a écrit :
> On Fri, Feb 21, 2020 at 7:40 PM Maxime Villard <m...@m00nbsd.net> wrote:
>>
>> Le 21/02/2020 à 19:26, syzbot a écrit :
>>> Hello,
>>>
>>> syzbot found the following crash on:
>>>
>>> HEAD commit:    162c6ffc Do not try to recovery from data page fault in in..
>>> git tree:       netbsd
>>> console output: https://syzkaller.appspot.com/x/log.txt?x=165161dde00000
>>> kernel config:  https://syzkaller.appspot.com/x/.config?x=9544af77043190a5
>>> dashboard link: https://syzkaller.appspot.com/bug?extid=a38dd76a4348973d2e53
>>> compiler:       clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
>>>
>>> Unfortunately, I don't have any reproducer for this crash yet.
>>>
>>> IMPORTANT: if you fix the bug, please add the following tag to the commit:
>>> Reported-by: syzbot+a38dd7...@syzkaller.appspotmail.com
>>>
>>> [  97.5740013] panic: MSan: Uninitialized Variable 'vattr' From do_sys_mknodat()
>>>
>>> [  97.5851426] cpu1: Begin traceback...
>>> [  97.6185217] vpanic() at netbsd:vpanic+0x7c1 sys/kern/subr_prf.c:334
>>> [  97.6964554] panic() at netbsd:panic+0x1ad sys/kern/subr_prf.c:255
>>> [  97.7632636] __msan_warning() at netbsd:__msan_warning+0x127 kmsan_report_inline sys/kern/subr_msan.c:229 [inline]
>>> [  97.7632636] __msan_warning() at netbsd:__msan_warning+0x127 sys/kern/subr_msan.c:612
>>> [  97.8412037] do_sys_mknodat() at netbsd:do_sys_mknodat+0x9e0 sys/kern/vfs_syscalls.c:2290
>>> [  97.9079964] sys___syscall() at netbsd:sys___syscall+0x27e sys/kern/sys_syscall.c:77
>>> [  97.9970667] syscall() at netbsd:syscall+0x35d sy_call sys/sys/syscallvar.h:65 [inline]
>>> [  97.9970667] syscall() at netbsd:syscall+0x35d sy_invoke sys/sys/syscallvar.h:94 [inline]
>>> [  97.9970667] syscall() at netbsd:syscall+0x35d sys/arch/x86/x86/syscall.c:138
>>> [  98.0193352] --- syscall (number 198) ---
>>> [  98.0416020] 77ed1b643b9a:
>>> [  98.0527326] cpu1: End traceback...
>>> [  98.0527326] fatal breakpoint trap in supervisor mode
>>> [  98.0638766] trap type 1 code 0 rip 0xffffffff8022025d cs 0x8 rflags 0x246 cr2 0x20001440 ilevel 0 rsp 0xffffbe0086406930
>>> [  98.0749986] curlwp 0xffffbe00123eba00 pid 45.4 lowest kstack 0xffffbe00863ff2c0
>>> Stopped in pid 45.4 (syz-executor.2) at netbsd:breakpoint+0x5:  leave
>>
>> That's a problem I noticed some time ago when running ATF.
>>
>> There is no actual uninitialized variable here.
>>
>> What happens is that here:
>>
>> 2290    if (error == 0 && optype == VOP_MKNOD_DESCOFFSET
>> 2291        && vattr.va_rdev == VNOVAL)
>> 2292            error = EINVAL;
>>
>> LLVM reorders the checks and accesses 'vattr' before 'error'.
>>
>> The thing is, if error!=0, 'vattr' is actually uninitialized, but
>> because of the reordering KMSAN fires because we still access 'vattr'
>> in that case. Overall this is harmless, but KMSAN will still
>> complain.
>>
>> I'll just inline this block in the parent block.
>
> /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
>
> +Alex, Evgenii for potential msan instrumentation bug
>
> Ooops. NetBSD uses normal clang msan instrumentation, right?

Yes.

[Evgenii Stepanov]

Maxime, if you have the build at hand, could you extract the compiler command line used to produce the "bad" vfs_syscalls.o, and the preprocessed source for it? I can take a look at it from the LLVM side, hopefully some time next week.

On Thu, Feb 27, 2020 at 1:05 PM Maxime Villard <m...@m00nbsd.net> wrote:

Le 27/02/2020 à 11:01, Dmitry Vyukov a écrit :
> I've tried to extract a reproducer, but full netbsd build fails for
> me. Here is what I did:
>
> git checkout 162c6ffc
> cp $SYZKALLER/dashboard/config/netbsd-kmsan.config
> sys/arch/amd64/conf/GENERIC_SYZKALLER
> ./build.sh -m amd64 -U -u -j72 -V MKCTF=no -V MKLLVM=yes -V MKGCC=no
> -V HAVE_LLVM=yes tools
> ./build.sh -m amd64 -U -u -j72 -V MKCTF=no -V MKLLVM=yes -V MKGCC=no
> -V HAVE_LLVM=yes kernel=GENERIC_SYZKALLER

Maybe a problem with Ubuntu, on Fedora it works as-is.

> /netbsd/src/obj/tooldir.Linux-5.2.17-1rodete3-amd64-x86_64/bin/nbgenassym:
> line 197: /usr/local/google/home/dvyukov/src/netbsd/src/../tools/bin/nbawk:
> No such file or directory
> *** Failed target:  assym.h
>
> So I built this single file with some hackery:
>
> ln -s sys/arch/amd64/include machine
> ln -s sys/arch/amd64/include amd64
> ln -s sys/arch/x86/include x86
> clang sys/kern/vfs_syscalls.c -O2 -fsanitize=memory -g -c -Isys -I.
> -Isys/sys -Isys/arch/amd64 -I common/include/ -I
> sys/compat/linux/common/ -I include/ -nostdinc -D_KERNEL
> -D__BSD_VISIBLE=1 -w
>
> But now I am not sure if this still reproduces the problem or not...
>
> Here is preprocessed source:
> https://gist.githubusercontent.com/dvyukov/f086fafffaf19c29010b99274b1dfb09/raw/5de909f6c635c08d1729929bb5582fe7f43598c0/vfs_syscalls.c
>
> and here is disasm of the function:
> https://gist.githubusercontent.com/dvyukov/ef780712c88e0baf9ba1e2e3fc231479/raw/6eac9c324f4253ccefcb058f08ff76927b1cc735/vfs_syscalls.o
>
>
> Maxime, could you assess if the code still contains the bug please? Is
> it the same you saw in your real objdump?

I don't remember, I noticed it four months ago. The test in
        /usr/tests/lib/libc/sys/t_mknod
was triggering the bug 100% reliably.

I've reverted my fix locally and rebuilt. Fails as before. I've extracted
vfs_syscalls.o and disassembled it quickly with ghidra:

        https://m00nbsd.net/dvyukov/

I slightly retyped the output. You can see in the screenshots that the
decompiled C code is:

        if (vp == NULL) {
                vattr_null(&vattr);
                ... 'vattr' is initialized so we don't care ...
        } else {
                check if 'vattr.va_rdev' is initialized
                ...
        }
        ...

Which differs from the semantic of the original code:

        if (vp == NULL) {
                vattr_null(&vattr);
                ... 'vattr' is initialized so we don't care ...
        } else {
                error = EEXIST;
        }
        if (error == 0 && optype == VOP_MKNOD_DESCOFFSET &&
            vattr.va_rdev == VNOVAL)
                error = EINVAL;
        // shouldn't check whether 'vattr.va_rdev' is initialized,
        // because the 'error==0' condition is false

For some reason it looks like the MSan instrumentation decides to check
'vattr.va_rdev' in the else{} even though the read is not supposed to
happen.

Maxime

[Evgenii Stepanov]

Thanks.

I've reduced and filed

https://bugs.llvm.org/show_bug.cgi?id=45220

Not sure what to do with it yet, will take another look later.

On Mon, Mar 16, 2020 at 7:31 AM Alexander Potapenko <gli...@google.com> wrote:

Attached is the LLVM IR file corresponding to this source.

On Sun, Mar 15, 2020 at 2:57 PM Maxime Villard <m...@m00nbsd.net> wrote:
>
> Sorry for the delay. Command:
>         https://m00nbsd.net/dvyukov/cmd.txt
> Preprocessed source:
>         https://m00nbsd.net/dvyukov/preprocessed.txt
>
> Maxime

> > --
> > You received this message because you are subscribed to the Google Groups "syzkaller-netbsd-bugs" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-netbsd...@googlegroups.com <mailto:syzkaller-netbsd...@googlegroups.com>.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-netbsd-bugs/CAFKCwrh-meTNLRSv0iW_iFS8jptz6ATfV0Ktetf8UU5HqosGJg%40mail.gmail.com <https://groups.google.com/d/msgid/syzkaller-netbsd-bugs/CAFKCwrh-meTNLRSv0iW_iFS8jptz6ATfV0Ktetf8UU5HqosGJg%40mail.gmail.com?utm_medium=email&utm_source=footer>.



--
Alexander Potapenko
Software Engineer

Google Germany GmbH
Erika-Mann-Straße, 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg