general protection fault in finish_task_switch

[syzbot]

Hello,

syzkaller hit the following crash on
7dc9f647127d6955ffacaf51cb6a627b31dceec2
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/master
compiler: gcc (GCC) 7.1.1 20170620
.config is attached
Raw console output is attached.
C reproducer is attached
syzkaller reproducer is attached. See https://goo.gl/kgGztJ
for information about syzkaller reproducers


kasan: CONFIG_KASAN_INLINE enabled
kasan: GPF could be caused by NULL-ptr deref or user memory access
general protection fault: 0000 [#1] SMP KASAN
Dumping ftrace buffer:
(ftrace buffer empty)
Modules linked in:
CPU: 0 PID: 4227 Comm: syzkaller244813 Not tainted
4.15.0-rc4-next-20171220+ #77
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
RIP: 0010:__fire_sched_in_preempt_notifiers kernel/sched/core.c:2534
[inline]
RIP: 0010:fire_sched_in_preempt_notifiers kernel/sched/core.c:2540 [inline]
RIP: 0010:finish_task_switch+0x42d/0x740 kernel/sched/core.c:2670
RSP: 0018:ffff8801c284f3b8 EFLAGS: 00010a06
RAX: dffffc0000000000 RBX: ffff8801db22c900 RCX: 1bd5a00000000022
RDX: 0000000000000000 RSI: 1ffff10038509e20 RDI: dead000000000110
RBP: ffff8801c284f468 R08: 1ffff10038661af3 R09: 0000000000000000
R10: ffff8801c284f390 R11: 0000000000000000 R12: ffff8801c2eca100
R13: 1ffff10038509e7c R14: dffffc0000000000 R15: dead000000000100
FS: 00007f7f034c8700(0000) GS:ffff8801db200000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ee8491e78 CR3: 00000001ca6c9003 CR4: 00000000001626f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
context_switch kernel/sched/core.c:2803 [inline]
__schedule+0x8f3/0x2060 kernel/sched/core.c:3376
preempt_schedule_common+0x22/0x60 kernel/sched/core.c:3515
_cond_resched+0x1d/0x30 kernel/sched/core.c:4852
zap_pmd_range mm/memory.c:1443 [inline]
zap_pud_range mm/memory.c:1470 [inline]
zap_p4d_range mm/memory.c:1491 [inline]
unmap_page_range+0x872/0x22e0 mm/memory.c:1512
unmap_single_vma+0x15f/0x2d0 mm/memory.c:1557
zap_page_range+0x35c/0x5f0 mm/memory.c:1611
madvise_dontneed_single_vma mm/madvise.c:510 [inline]
madvise_dontneed_free mm/madvise.c:563 [inline]
madvise_vma mm/madvise.c:688 [inline]
SYSC_madvise mm/madvise.c:863 [inline]
SyS_madvise+0xf9e/0x13d0 mm/madvise.c:791
entry_SYSCALL_64_fastpath+0x1f/0x96
RIP: 0033:0x448fe7
RSP: 002b:00007f7f034c7dd8 EFLAGS: 00000206 ORIG_RAX: 000000000000001c
RAX: ffffffffffffffda RBX: 00007f7f034c6970 RCX: 0000000000448fe7
RDX: 0000000000000004 RSI: 00000000007fb000 RDI: 00007f7f02cc8000
RBP: 000000000000003f R08: 0000000000000012 R09: 0000000000000000
R10: 00008c0000500000 R11: 0000000000000206 R12: 0000000000000000
R13: 00000000ffffffff R14: ffffffffffffffff R15: 00000000206a2000
Code: ff 11 4c 89 f9 48 c1 e9 03 42 80 3c 31 00 0f 85 1b 02 00 00 4d 8b 3f
4d 85 ff 0f 84 c0 fd ff ff 49 8d 7f 10 48 89 f9 48 c1 e9 03 <42> 80 3c 31
00 74 ae e8 a7 cc 5b 00 eb a7 4c 89 fe 4c 89 e7 e8
RIP: __fire_sched_in_preempt_notifiers kernel/sched/core.c:2534 [inline]
RSP: ffff8801c284f3b8
RIP: fire_sched_in_preempt_notifiers kernel/sched/core.c:2540 [inline] RSP:
ffff8801c284f3b8
RIP: finish_task_switch+0x42d/0x740 kernel/sched/core.c:2670 RSP:
ffff8801c284f3b8
---[ end trace 4048fa81110fa823 ]---
Kernel panic - not syncing: Fatal exception
Dumping ftrace buffer:
(ftrace buffer empty)
Kernel Offset: disabled
Rebooting in 86400 seconds..


---
This bug is generated by a dumb bot. It may contain errors.
See https://goo.gl/tpsmEJ for details.
Direct all questions to syzk...@googlegroups.com.
Please credit me with: Reported-by: syzbot <syzk...@googlegroups.com>

syzbot will keep track of this bug report.
Once a fix for this bug is merged into any tree, reply to this email with:
#syz fix: exact-commit-title
If you want to test a patch for this bug, please reply with:
#syz test: git://repo/address.git branch
and provide the patch inline or as an attachment.
To mark this as a duplicate of another syzbot report, please reply with:
#syz dup: exact-subject-of-another-report
If it's a one-off invalid bug report, please reply with:
#syz invalid
Note: if the crash happens again, it will cause creation of a new bug
report.
Note: all commands must start from beginning of the line in the email body.

[Linus Torvalds]

On Wed, Dec 20, 2017 at 8:03 AM, syzbot
<bot+72c44cd8b0e8a1a64b...@syzkaller.appspotmail.com>
wrote:

> Hello,
>
> syzkaller hit the following crash on
> 7dc9f647127d6955ffacaf51cb6a627b31dceec2
> git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/master
>

> kasan: CONFIG_KASAN_INLINE enabled
> kasan: GPF could be caused by NULL-ptr deref or user memory access
> general protection fault: 0000 [#1] SMP KASAN
> Dumping ftrace buffer:
> (ftrace buffer empty)
> Modules linked in:
> CPU: 0 PID: 4227 Comm: syzkaller244813 Not tainted 4.15.0-rc4-next-20171220+
> #77
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
> Google 01/01/2011

> RIP: __fire_sched_in_preempt_notifiers kernel/sched/core.c:2534 [inline]

That line 2534 is the call inside the hlist_for_each_entry() loop:

hlist_for_each_entry(notifier, &curr->preempt_notifiers, link)
notifier->ops->sched_in(notifier, raw_smp_processor_id());

and the Code: line disassembly is

0: ff 11 callq *(%rcx)
2: 4c 89 f9 mov %r15,%rcx
5: 48 c1 e9 03 shr $0x3,%rcx
9: 42 80 3c 31 00 cmpb $0x0,(%rcx,%r14,1)
e: 0f 85 1b 02 00 00 jne 0x22f
14: 4d 8b 3f mov (%r15),%r15
17: 4d 85 ff test %r15,%r15
1a: 0f 84 c0 fd ff ff je 0xfffffffffffffde0
20: 49 8d 7f 10 lea 0x10(%r15),%rdi
24: 48 89 f9 mov %rdi,%rcx
27: 48 c1 e9 03 shr $0x3,%rcx
2b:* 42 80 3c 31 00 cmpb $0x0,(%rcx,%r14,1) <-- trapping instruction
30: 74 ae je 0xffffffffffffffe0
32: e8 a7 cc 5b 00 callq 0x5bccde
37: eb a7 jmp 0xffffffffffffffe0
39: 4c 89 fe mov %r15,%rsi
3c: 4c 89 e7 mov %r12,%rdi

and while the "callq *(%rcx)" might be just the end part of some
previous instruction, I think it may be right (there is indeed an
indirect call in that function - that very "->sched_in()" call).

So I think the oops happens after the indirect call returns.

I think the second "callq" is

call __asan_report_load8_noabort

and the actual trapping instruction is loading the KASAN byte state.

As far as I can tell, the kasan check is trying to check this part of
hlist_for_each_entry():

movq (%r15), %r15 # notifier_110->link.next,

and %r15 is dead000000000100, which is LIST_POISON1.

End result: KASAN actually makes these things harder to debug, because
it's trying to "validate" the list poison values before they are used,
and takes a much more complex and indirect fault in the process,
instead of just getting a page-fault on the LIST_POISON1 that would
have made it more obvious.

Oh well.

There is nothing in this that indicates that it's actually related to
KASAN, and it _should_ oops even without KASAN enabled.

But the reproducer does nothing for me. Of course, I didn't actually
run it on linux-next at all, so it is quite possibly related to
scheduler work (or the TLB/pagetable work) that just hasn't hit
mainstream yet.

None of the scheduler people seem to have been on the report, though.
Adding some in.

Linus

[Peter Zijlstra]

So the only user of that preempt_notifier stuff is KVM, if you don't run
a guest the notifiers are empty and are in fact disabled with a
static_key.

We've not touched this part of the scheduler in a fair while. I'll go
dig out the original report and see if that reproducer does anything for
me.

[Dmitry Vyukov]

I think this is another manifestation of "KASAN: use-after-free Read
in __schedule":
https://groups.google.com/forum/#!msg/syzkaller-bugs/-8JZhr4W8AY/FpPFh8EqAQAJ
+Eric already mailed a fix for it (indeed new bug in kvm code).

Let's tell syzbot:

#syz dup: KASAN: use-after-free Read in __schedule

[Peter Zijlstra]

On Fri, Dec 22, 2017 at 09:26:28AM +0100, Dmitry Vyukov wrote:
> I think this is another manifestation of "KASAN: use-after-free Read
> in __schedule":
> https://groups.google.com/forum/#!msg/syzkaller-bugs/-8JZhr4W8AY/FpPFh8EqAQAJ
> +Eric already mailed a fix for it (indeed new bug in kvm code).

FWIW, these google links keep translating everything to my local
language, is there any way to tell google to not do stupid stuff like
that and give me English like computers ought to speak?

[Dmitry Vyukov]

We added this line before GPF crashes:

kasan: GPF could be caused by NULL-ptr deref or user memory access

But I agree that it's not very useful, it's printed unconditionally.
I asked +Andrey if we can obtain and print the fault address, but as
far as I remember he said that it's very hard.
If we had the fault address, we could revert KASAN shadow calculation
and say something along the lines of:

kasan: GPF could be caused by NULL-ptr deref or user memory access

(original access would have been to 0xdead000000000100).

We could also do KASAN checks after the original access, but that
would lead to another kind of badness. KASAN stores metadata right
around and inside of heap objects, so use-after-free and out-of-bounds
access have high chances of corrupting that data. Then KASAN will
either print nonsense or crash itself...

> But the reproducer does nothing for me. Of course, I didn't actually
> run it on linux-next at all, so it is quite possibly related to
> scheduler work (or the TLB/pagetable work) that just hasn't hit
> mainstream yet.
>
> None of the scheduler people seem to have been on the report, though.
> Adding some in.
>
> Linus
>

> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bug...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/CA%2B55aFyco00CBed1ADAz%2BEGtoP6w%2BnvuR2Y%2BYWH13cvkatOg4w%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Dmitry Vyukov]

The group has "Group's primary language: English" in settings. I guess
that's either your Google account settings (if you are signed in), or
browser settings.
For chrome there is an option in setting for preferred languages,
browsers are supposed to send that in requests. For google account
check https://myaccount.google.com/intro there is "Languages" section.

[Peter Zijlstra]

I do not use (nor want to) a google account to sign in. Chromium has
English set as the preferred language (I typically don't install weird
localisation things and language packs in any case; 7bit ASCII FTW).

I have also done the google.com/ncr thing, which got rid of google.com
defaulting to google.nl, but groups.google.com keeps insisting on
translating the 'app' to Dutch. Seeing both Dutch and English (the
actual messages) at the same time completely screws my brain.

I'd file a bug against groups.google.com for not respecting the /ncr
thing, but I suspect you'd require a google account for that :-(

[Dmitry Vyukov]

On Fri, Dec 22, 2017 at 9:57 AM, Peter Zijlstra <pet...@infradead.org> wrote:
>> >> I think this is another manifestation of "KASAN: use-after-free Read
>> >> in __schedule":
>> >> https://groups.google.com/forum/#!msg/syzkaller-bugs/-8JZhr4W8AY/FpPFh8EqAQAJ
>> >> +Eric already mailed a fix for it (indeed new bug in kvm code).
>> >
>> > FWIW, these google links keep translating everything to my local
>> > language, is there any way to tell google to not do stupid stuff like
>> > that and give me English like computers ought to speak?
>>
>>
>> The group has "Group's primary language: English" in settings. I guess
>> that's either your Google account settings (if you are signed in), or
>> browser settings.
>> For chrome there is an option in setting for preferred languages,
>> browsers are supposed to send that in requests. For google account
>> check https://myaccount.google.com/intro there is "Languages" section.
>
> I do not use (nor want to) a google account to sign in. Chromium has
> English set as the preferred language (I typically don't install weird
> localisation things and language packs in any case; 7bit ASCII FTW).
>
> I have also done the google.com/ncr thing, which got rid of google.com
> defaulting to google.nl, but groups.google.com keeps insisting on
> translating the 'app' to Dutch. Seeing both Dutch and English (the
> actual messages) at the same time completely screws my brain.
>
> I'd file a bug against groups.google.com for not respecting the /ncr
> thing, but I suspect you'd require a google account for that :-(

You mean the messages themselves are translated? That's weird. Seems
that Google Translate somehow kicks in. There is a "Offer to translate
pages that aren't in a language you read" setting in chromium, but I
guess if you have only English in languages it should not matter... I
am out of ideas. Just in case, all syzbot reports are on LKML as well.

[Peter Zijlstra]

On Fri, Dec 22, 2017 at 10:08:00AM +0100, Dmitry Vyukov wrote:

> You mean the messages themselves are translated?

No, just the webapp thing, which is bad enough. The actual messages are
untouched.

[Dmitry Vyukov]

Then try to open dev console in chromium (for me it's shift+ctrl+c),
reload the page, and then on the Network tab of dev console you can
see all request headers your browser sends. For me I see:

accept-language: en-US,en;q=0.9,ru;q=0.8

and the resulting page is in english.

[Peter Zijlstra]

accept-language:en-US,en;q=0.9

> and the resulting page is in english.

But I suspect you are in fact signed in and located in the US (your
email headers suggest you're in PST), right?

I'm sure that if I request the page using an IP that geo-locates to the
US, I'd see the thing in English too.

[Dmitry Vyukov]

On Fri, Dec 22, 2017 at 11:03 AM, Peter Zijlstra <pet...@infradead.org> wrote:
>> >> You mean the messages themselves are translated?
>> >
>> > No, just the webapp thing, which is bad enough. The actual messages are
>> > untouched.
>>
>> Then try to open dev console in chromium (for me it's shift+ctrl+c),
>> reload the page, and then on the Network tab of dev console you can
>> see all request headers your browser sends. For me I see:
>>
>> accept-language: en-US,en;q=0.9,ru;q=0.8
>
> accept-language:en-US,en;q=0.9
>
>> and the resulting page is in english.
>
> But I suspect you are in fact signed in and located in the US (your
> email headers suggest you're in PST), right?
>
> I'm sure that if I request the page using an IP that geo-locates to the
> US, I'd see the thing in English too.

I am in Germany. Also tried to open it unsigned from incognito window,
still english...

[Peter Zijlstra]

On Fri, Dec 22, 2017 at 11:24:45AM +0100, Dmitry Vyukov wrote:

> I am in Germany. Also tried to open it unsigned from incognito window,
> still english...

Not sure what all happened, but I restarted Chrome and now your link at
least displays in English, hooray.

If I go to groups.google.com, still Dutch.

Weird stuff. But I can live with this.

[djvand...@gmail.com]

Maybe some rain here can help me out I've been trying to figure out how to fix my fiance's Google account weave going through all the security questions even tried to do the two-step verification all that it keeps showing up is saying that it could either be corrupted or that it's attached to another phone there is no activity from her phone or the other phone maybe someone here can help me out.