UBSAN: shift-out-of-bounds in kvm_vcpu_after_set_cpuid

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: 5e60366d Merge tag 'fallthrough-fixes-clang-5.11-rc1' of g..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=11c7046b500000
kernel config: https://syzkaller.appspot.com/x/.config?x=db720fe37a6a41d8
dashboard link: https://syzkaller.appspot.com/bug?extid=e87846c48bf72bc85311
compiler: gcc (GCC) 10.1.0-syz 20200507
userspace arch: i386

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+e87846...@syzkaller.appspotmail.com

================================================================================
UBSAN: shift-out-of-bounds in arch/x86/kvm/mmu.h:52:16
shift exponent 64 is too large for 64-bit type 'long long unsigned int'
CPU: 1 PID: 11156 Comm: syz-executor.1 Not tainted 5.10.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:79 [inline]
dump_stack+0x107/0x163 lib/dump_stack.c:120
ubsan_epilogue+0xb/0x5a lib/ubsan.c:148
__ubsan_handle_shift_out_of_bounds.cold+0xb1/0x181 lib/ubsan.c:395
rsvd_bits arch/x86/kvm/mmu.h:52 [inline]
kvm_vcpu_after_set_cpuid.cold+0x35/0x3a arch/x86/kvm/cpuid.c:181
kvm_vcpu_ioctl_set_cpuid+0x28e/0x970 arch/x86/kvm/cpuid.c:273
kvm_arch_vcpu_ioctl+0x1091/0x2d70 arch/x86/kvm/x86.c:4699
kvm_vcpu_ioctl+0x7b9/0xdb0 arch/x86/kvm/../../../virt/kvm/kvm_main.c:3386
kvm_vcpu_compat_ioctl+0x1a2/0x340 arch/x86/kvm/../../../virt/kvm/kvm_main.c:3430
__do_compat_sys_ioctl+0x1d3/0x230 fs/ioctl.c:842
do_syscall_32_irqs_on arch/x86/entry/common.c:78 [inline]
__do_fast_syscall_32+0x56/0x80 arch/x86/entry/common.c:137
do_fast_syscall_32+0x2f/0x70 arch/x86/entry/common.c:160
entry_SYSENTER_compat_after_hwframe+0x4d/0x5c
RIP: 0023:0xf7fe8549
Code: b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10 08 03 74 d8 01 00 00 00 00 00 00 00 00 00 00 00 00 00 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 eb 0d 90 90 90 90 90 90 90 90 90 90 90 90
RSP: 002b:00000000f55e20cc EFLAGS: 00000296 ORIG_RAX: 0000000000000036
RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 000000004008ae8a
RDX: 00000000200000c0 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
================================================================================


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[Jim Mattson]

It looks like userspace can possibly induce this by providing guest
CPUID information with a "physical address width" of 64 in leaf
0x80000008.

Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
CPUID.80000008H:EAX? Better would be to return an error for
out-of-range values, but I understand that the kvm community's stance
is that, in general, guest CPUID information should not be validated
by kvm.

[Sean Christopherson]

On Mon, Jan 11, 2021, Jim Mattson wrote:
> It looks like userspace can possibly induce this by providing guest
> CPUID information with a "physical address width" of 64 in leaf
> 0x80000008.

It was actually the opposite, where userspace provides '0' and caused '63 - 0 + 1'
to overflow. KVM controls the upper bound, and rsvd_bits() explicitly handles
'end < start', so an absurdly large maxpa is handled correctly.

Aleady fixed by Paolo in commit 2f80d502d627 ("KVM: x86: fix shift out of bounds
reported by UBSAN").

> Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
> CPUID.80000008H:EAX? Better would be to return an error for
> out-of-range values, but I understand that the kvm community's stance
> is that, in general, guest CPUID information should not be validated
> by kvm.

And rob Paolo of his crazy^Wbrilliant bit math shenanigans? :-D

[Paolo Bonzini]

On 12/01/21 00:01, Sean Christopherson wrote:
>> Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
>> CPUID.80000008H:EAX?

The low 6 bits I guess---yes, that would make sense and it would have
also fixed the bug.

(Nevertheless it's a good idea to make rsvd_bits more robust as well, as
in the commit that Sean referenced.

Paolo

[Sean Christopherson]

On Tue, Jan 12, 2021, Paolo Bonzini wrote:
> On 12/01/21 00:01, Sean Christopherson wrote:
> > > Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
> > > CPUID.80000008H:EAX?
>
> The low 6 bits I guess---yes, that would make sense and it would have also
> fixed the bug.

No, that wouldn't have fixed this specific bug. In this case, the issue was
CPUID.80000008H:AL == 0; masking off bits 7:6 wouldn't have changed anything.

And, masking bits 7:6 is architecturally wrong. Both the SDM and APM state that
bits 7:0 contain the number of PA bits.

KVM could reject guest.MAXPA > host.MAXPA, but arbitrarily dropping bits would
be wrong.

[Paolo Bonzini]

On 12/01/21 17:53, Sean Christopherson wrote:
> On Tue, Jan 12, 2021, Paolo Bonzini wrote:
>> On 12/01/21 00:01, Sean Christopherson wrote:
>>>> Perhaps cpuid_query_maxphyaddr() should just look at the low 5 bits of
>>>> CPUID.80000008H:EAX?
>>
>> The low 6 bits I guess---yes, that would make sense and it would have also
>> fixed the bug.
>
> No, that wouldn't have fixed this specific bug. In this case, the issue was
> CPUID.80000008H:AL == 0; masking off bits 7:6 wouldn't have changed anything.

Right.

> And, masking bits 7:6 is architecturally wrong. Both the SDM and APM state that
> bits 7:0 contain the number of PA bits.

They cannot be higher than 52, therefore bits 7:6 are (architecturally)
always zero. In other words, I interpret "bit 7:0 contain the number of
PA bits" as "you need not do an '& 63' yourself", which is basically the
opposite of "bit 7:6 might be nonzero". If masking made any difference,
it would be outside the spec already.

In fact another possibility to avoid UB is to do "& 63" of both s and e
in rsvd_bits. This would also be masking bits 7:6 of the CPUID leaf,
just done differently.

Paolo

[Sean Christopherson]

On Wed, Jan 13, 2021, Paolo Bonzini wrote:
> On 12/01/21 17:53, Sean Christopherson wrote:
> > And, masking bits 7:6 is architecturally wrong. Both the SDM and APM state that
> > bits 7:0 contain the number of PA bits.
>
> They cannot be higher than 52,

Drat, I was going to argue that it could be >52 with a new paging mode, but both
the SDM and APM explicitly call out 52 as the max. Spending cycles on the stuff
that really matters here... :-)

> therefore bits 7:6 are (architecturally)
> always zero. In other words, I interpret "bit 7:0 contain the number of PA
> bits" as "you need not do an '& 63' yourself", which is basically the
> opposite of "bit 7:6 might be nonzero". If masking made any difference, it
> would be outside the spec already.
>
> In fact another possibility to avoid UB is to do "& 63" of both s and e in
> rsvd_bits. This would also be masking bits 7:6 of the CPUID leaf, just done
> differently.

Hmm, 'e' is hardcoded in all call sites except kvm_mmu_reset_all_pte_masks(),
and so long as 'e <= 63' holds true, 's &= 63' is unnecessary. What if we add
compile-time asserts on hardcoded values, and mask 'e' for the rare case where
the upper bound isn't hardcoded? That way bogus things like rsvd_bits(63, 65)
will fail the build.

diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h
index 581925e476d6..261be1d2032b 100644
--- a/arch/x86/kvm/mmu.h
+++ b/arch/x86/kvm/mmu.h
@@ -44,8 +44,15 @@
#define PT32_ROOT_LEVEL 2
#define PT32E_ROOT_LEVEL 3

-static inline u64 rsvd_bits(int s, int e)
+static __always_inline u64 rsvd_bits(int s, int e)
{
+ BUILD_BUG_ON(__builtin_constant_p(e) && __builtin_constant_p(s) && e < s);
+
+ if (__builtin_constant_p(e))
+ BUILD_BUG_ON(e > 63);
+ else
+ e &= 63;
+
if (e < s)
return 0;