inconsistent lock state in ima_process_queued_keys

16 views
Skip to first unread message

syzbot

unread,
Jan 14, 2020, 8:56:11 AM1/14/20
to dmitry....@gmail.com, jmo...@namei.org, linux-i...@vger.kernel.org, linux-...@vger.kernel.org, linux-secu...@vger.kernel.org, se...@hallyn.com, syzkall...@googlegroups.com, zo...@linux.ibm.com
Hello,

syzbot found the following crash on:

HEAD commit: 1b851f98 Add linux-next specific files for 20200114
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=12bcbb25e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=3e7d9cf7ebfa08ad
dashboard link: https://syzkaller.appspot.com/bug?extid=a4a503d7f37292ae1664
compiler: gcc (GCC) 9.0.0 20181231 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+a4a503...@syzkaller.appspotmail.com

================================
WARNING: inconsistent lock state
5.5.0-rc6-next-20200114-syzkaller #0 Not tainted
--------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
syz-executor.5/9563 [HC0[0]:SC1[1]:HE1:SE0] takes:
ffffffff8a03aa18 (ima_keys_lock){+.?.}, at: spin_lock
include/linux/spinlock.h:338 [inline]
ffffffff8a03aa18 (ima_keys_lock){+.?.}, at:
ima_process_queued_keys+0x4f/0x320
security/integrity/ima/ima_asymmetric_keys.c:144
{SOFTIRQ-ON-W} state was registered at:
lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4484
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:338 [inline]
ima_queue_key security/integrity/ima/ima_asymmetric_keys.c:111 [inline]
ima_post_key_create_or_update+0x234/0x470
security/integrity/ima/ima_asymmetric_keys.c:194
key_create_or_update+0x6b8/0xcb0 security/keys/key.c:944
load_system_certificate_list+0x1ba/0x25e certs/system_keyring.c:161
do_one_initcall+0x120/0x820 init/main.c:940
do_initcall_level init/main.c:1014 [inline]
do_initcalls init/main.c:1022 [inline]
do_basic_setup init/main.c:1039 [inline]
kernel_init_freeable+0x4ca/0x570 init/main.c:1223
kernel_init+0x12/0x1bf init/main.c:1130
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
irq event stamp: 319180
hardirqs last enabled at (319180): [<ffffffff87ec3333>]
__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:168 [inline]
hardirqs last enabled at (319180): [<ffffffff87ec3333>]
_raw_spin_unlock_irq+0x23/0x80 kernel/locking/spinlock.c:199
hardirqs last disabled at (319179): [<ffffffff87ec351a>]
__raw_spin_lock_irq include/linux/spinlock_api_smp.h:126 [inline]
hardirqs last disabled at (319179): [<ffffffff87ec351a>]
_raw_spin_lock_irq+0x3a/0x80 kernel/locking/spinlock.c:167
softirqs last enabled at (316932): [<ffffffff882006cd>]
__do_softirq+0x6cd/0x98c kernel/softirq.c:319
softirqs last disabled at (319177): [<ffffffff8147800b>] invoke_softirq
kernel/softirq.c:373 [inline]
softirqs last disabled at (319177): [<ffffffff8147800b>]
irq_exit+0x19b/0x1e0 kernel/softirq.c:413

other info that might help us debug this:
Possible unsafe locking scenario:

CPU0
----
lock(ima_keys_lock);
<Interrupt>
lock(ima_keys_lock);

*** DEADLOCK ***

1 lock held by syz-executor.5/9563:
#0: ffffc90000007d50 ((&ima_key_queue_timer)){+.-.}, at: lockdep_copy_map
include/linux/lockdep.h:172 [inline]
#0: ffffc90000007d50 ((&ima_key_queue_timer)){+.-.}, at:
call_timer_fn+0xe0/0x780 kernel/time/timer.c:1394

stack backtrace:
CPU: 0 PID: 9563 Comm: syz-executor.5 Not tainted
5.5.0-rc6-next-20200114-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x197/0x210 lib/dump_stack.c:118
print_usage_bug.cold+0x327/0x378 kernel/locking/lockdep.c:3100
valid_state kernel/locking/lockdep.c:3111 [inline]
mark_lock_irq kernel/locking/lockdep.c:3308 [inline]
mark_lock+0xbb4/0x1220 kernel/locking/lockdep.c:3665
mark_usage kernel/locking/lockdep.c:3565 [inline]
__lock_acquire+0x1e8e/0x4a00 kernel/locking/lockdep.c:3908
lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4484
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:338 [inline]
ima_process_queued_keys+0x4f/0x320
security/integrity/ima/ima_asymmetric_keys.c:144
ima_timer_handler+0x15/0x20 security/integrity/ima/ima_asymmetric_keys.c:46
call_timer_fn+0x1ac/0x780 kernel/time/timer.c:1404
expire_timers kernel/time/timer.c:1449 [inline]
__run_timers kernel/time/timer.c:1773 [inline]
__run_timers kernel/time/timer.c:1740 [inline]
run_timer_softirq+0x6c3/0x1790 kernel/time/timer.c:1786
__do_softirq+0x262/0x98c kernel/softirq.c:292
invoke_softirq kernel/softirq.c:373 [inline]
irq_exit+0x19b/0x1e0 kernel/softirq.c:413
exiting_irq arch/x86/include/asm/apic.h:536 [inline]
smp_apic_timer_interrupt+0x1a3/0x610 arch/x86/kernel/apic/apic.c:1137
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:829
</IRQ>
RIP: 0010:arch_local_irq_restore arch/x86/include/asm/paravirt.h:752
[inline]
RIP: 0010:slab_alloc mm/slab.c:3313 [inline]
RIP: 0010:kmem_cache_alloc+0x283/0x710 mm/slab.c:3484
Code: 7e 0f 85 dc fe ff ff e8 0c 3f 4d ff e9 d2 fe ff ff e8 31 aa c7 ff 48
83 3d 21 84 00 08 00 0f 84 51 03 00 00 48 8b 7d d0 57 9d <0f> 1f 44 00 00
e9 68 fe ff ff 31 d2 be 35 02 00 00 48 c7 c7 2e a1
RSP: 0018:ffffc9000641fd18 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
RAX: 0000000000000007 RBX: 0000000000000cc0 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000006 RDI: 0000000000000286
RBP: ffffc9000641fd80 R08: ffff8880543b22c0 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8880aa5f8a80
R13: ffff8880aa5f8a80 R14: 0000000000000cc0 R15: ffff8880a8916600
getname_flags fs/namei.c:138 [inline]
getname_flags+0xd6/0x5b0 fs/namei.c:128
getname fs/namei.c:209 [inline]
user_path_mountpoint_at+0x29/0x50 fs/namei.c:2820
ksys_umount+0x164/0xef0 fs/namespace.c:1683
__do_sys_umount fs/namespace.c:1709 [inline]
__se_sys_umount fs/namespace.c:1707 [inline]
__x64_sys_umount+0x54/0x80 fs/namespace.c:1707
do_syscall_64+0xfa/0x790 arch/x86/entry/common.c:294
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x45d977
Code: 64 89 04 25 d0 02 00 00 58 5f ff d0 48 89 c7 e8 2f be ff ff 66 2e 0f
1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 a6 00 00 00 0f 05 <48> 3d 01 f0 ff
ff 0f 83 4d 8c fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffeff269038 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6
RAX: ffffffffffffffda RBX: 000000000004d5f3 RCX: 000000000045d977
RDX: 000000000000000c RSI: 0000000000000002 RDI: 00007ffeff26a170
RBP: 000000000000003c R08: 0000000000000001 R09: 0000000001dc6940
R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffeff26a170
R13: 00007ffeff26a160 R14: 0000000000000000 R15: 00007ffeff26a170


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

Dmitry Vyukov

unread,
Jan 14, 2020, 8:58:55 AM1/14/20
to syzbot, Lakshmi Ramasubramanian, Dmitry Kasatkin, James Morris, linux-i...@vger.kernel.org, LKML, linux-security-module, Serge E. Hallyn, syzkaller-bugs, Mimi Zohar
On Tue, Jan 14, 2020 at 2:56 PM syzbot
<syzbot+a4a503...@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit: 1b851f98 Add linux-next specific files for 20200114
> git tree: linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=12bcbb25e00000
> kernel config: https://syzkaller.appspot.com/x/.config?x=3e7d9cf7ebfa08ad
> dashboard link: https://syzkaller.appspot.com/bug?extid=a4a503d7f37292ae1664
> compiler: gcc (GCC) 9.0.0 20181231 (experimental)
>
> Unfortunately, I don't have any reproducer for this crash yet.
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: syzbot+a4a503...@syzkaller.appspotmail.com

+Lakshmi, you seem to have submitted a number of changes to this file recently.

This completely breaks linux-next testing for us, every kernel crashes
a few minutes after boot.

2020/01/14 14:45:00 vm-26: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:00 vm-12: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:01 vm-3: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:02 vm-8: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:03 vm-25: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:04 vm-27: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:04 VMs 0, executed 214279, cover 10140, crashes 5, repro 0
2020/01/14 14:45:05 vm-14: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:05 vm-11: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:06 vm-10: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:07 vm-0: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:08 vm-24: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:09 vm-2: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:10 vm-20: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:11 vm-7: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:12 vm-28: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:12 vm-1: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:13 vm-6: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:14 vm-4: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:14 VMs 0, executed 214279, cover 10140, crashes 17, repro 0
2020/01/14 14:45:15 vm-17: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:16 vm-16: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:16 vm-19: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:17 vm-15: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:18 vm-29: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:19 vm-21: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:20 vm-23: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:20 vm-13: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:21 vm-9: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:22 vm-5: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:23 vm-22: crash: inconsistent lock state in
ima_process_queued_keys
2020/01/14 14:45:24 VMs 0, executed 214279, cover 10140, crashes 28, repro 0
2020/01/14 14:45:24 vm-18: crash: inconsistent lock state in
ima_process_queued_keys
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bug...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/000000000000486474059c19f4d7%40google.com.

Mimi Zohar

unread,
Jan 14, 2020, 9:57:37 AM1/14/20
to Dmitry Vyukov, syzbot, Lakshmi Ramasubramanian, Dmitry Kasatkin, James Morris, linux-i...@vger.kernel.org, LKML, linux-security-module, Serge E. Hallyn, syzkaller-bugs
On Tue, 2020-01-14 at 14:58 +0100, Dmitry Vyukov wrote:
> On Tue, Jan 14, 2020 at 2:56 PM syzbot
> <syzbot+a4a503...@syzkaller.appspotmail.com> wrote:
> >
> > Hello,
> >
> > syzbot found the following crash on:
> >
> > HEAD commit: 1b851f98 Add linux-next specific files for 20200114
> > git tree: linux-next
> > console output: https://syzkaller.appspot.com/x/log.txt?x=12bcbb25e00000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=3e7d9cf7ebfa08ad
> > dashboard link: https://syzkaller.appspot.com/bug?extid=a4a503d7f37292ae1664
> > compiler: gcc (GCC) 9.0.0 20181231 (experimental)
> >
> > Unfortunately, I don't have any reproducer for this crash yet.
> >
> > IMPORTANT: if you fix the bug, please add the following tag to the commit:
> > Reported-by: syzbot+a4a503...@syzkaller.appspotmail.com
>
> +Lakshmi, you seem to have submitted a number of changes to this file recently.
>
> This completely breaks linux-next testing for us, every kernel crashes
> a few minutes after boot.
>
> 2020/01/14 14:45:00 vm-26: crash: inconsistent lock state in
> ima_process_queued_keys

Yikes!  Are you running with an IMA policy?  I assume this is being
caused by commit 8f5d2d06f217 ("IMA: Defined timer to free queued
keys".  Does reverting it prevent this from happening?

Mimi

Dmitry Vyukov

unread,
Jan 14, 2020, 10:21:20 AM1/14/20
to Mimi Zohar, syzbot, Lakshmi Ramasubramanian, Dmitry Kasatkin, James Morris, linux-i...@vger.kernel.org, LKML, linux-security-module, Serge E. Hallyn, syzkaller-bugs
On Tue, Jan 14, 2020 at 3:57 PM Mimi Zohar <zo...@linux.ibm.com> wrote:
>
> On Tue, 2020-01-14 at 14:58 +0100, Dmitry Vyukov wrote:
> > On Tue, Jan 14, 2020 at 2:56 PM syzbot
> > <syzbot+a4a503...@syzkaller.appspotmail.com> wrote:
> > >
> > > Hello,
> > >
> > > syzbot found the following crash on:
> > >
> > > HEAD commit: 1b851f98 Add linux-next specific files for 20200114
> > > git tree: linux-next
> > > console output: https://syzkaller.appspot.com/x/log.txt?x=12bcbb25e00000
> > > kernel config: https://syzkaller.appspot.com/x/.config?x=3e7d9cf7ebfa08ad
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=a4a503d7f37292ae1664
> > > compiler: gcc (GCC) 9.0.0 20181231 (experimental)
> > >
> > > Unfortunately, I don't have any reproducer for this crash yet.
> > >
> > > IMPORTANT: if you fix the bug, please add the following tag to the commit:
> > > Reported-by: syzbot+a4a503...@syzkaller.appspotmail.com
> >
> > +Lakshmi, you seem to have submitted a number of changes to this file recently.
> >
> > This completely breaks linux-next testing for us, every kernel crashes
> > a few minutes after boot.
> >
> > 2020/01/14 14:45:00 vm-26: crash: inconsistent lock state in
> > ima_process_queued_keys
>
> Yikes! Are you running with an IMA policy?

I don't know.

> I assume this is being
> caused by commit 8f5d2d06f217 ("IMA: Defined timer to free queued
> keys". Does reverting it prevent this from happening?

The following seems to help, but don't know if it's the right fix or not.

diff --git a/security/integrity/ima/ima_asymmetric_keys.c
b/security/integrity/ima/ima_asymmetric_keys.c
index 61e478f9e8199..49d559501fe62 100644
--- a/security/integrity/ima/ima_asymmetric_keys.c
+++ b/security/integrity/ima/ima_asymmetric_keys.c
@@ -103,17 +103,18 @@ static bool ima_queue_key(struct key *keyring,
const void *payload,
{
bool queued = false;
struct ima_key_entry *entry;
+ unsigned long flags;

entry = ima_alloc_key_entry(keyring, payload, payload_len);
if (!entry)
return false;

- spin_lock(&ima_keys_lock);
+ spin_lock_irqsave(&ima_keys_lock, flags);
if (!ima_process_keys) {
list_add_tail(&entry->list, &ima_keys);
queued = true;
}
- spin_unlock(&ima_keys_lock);
+ spin_unlock_irqrestore(&ima_keys_lock, flags);

if (!queued)
ima_free_key_entry(entry);

Lakshmi Ramasubramanian

unread,
Jan 14, 2020, 11:54:16 AM1/14/20
to Dmitry Vyukov, Mimi Zohar, syzbot, Dmitry Kasatkin, James Morris, linux-i...@vger.kernel.org, LKML, linux-security-module, Serge E. Hallyn, syzkaller-bugs
I am investigating this.

thanks,
-lakshmi

Lakshmi Ramasubramanian

unread,
Jan 14, 2020, 12:18:49 PM1/14/20
to Dmitry Vyukov, Mimi Zohar, syzbot, Dmitry Kasatkin, James Morris, linux-i...@vger.kernel.org, LKML, linux-security-module, Serge E. Hallyn, syzkaller-bugs
Hi Dmitry,

> --- a/security/integrity/ima/ima_asymmetric_keys.c
> +++ b/security/integrity/ima/ima_asymmetric_keys.c
> @@ -103,17 +103,18 @@ static bool ima_queue_key(struct key *keyring,
> const void *payload,
> {
> bool queued = false;
> struct ima_key_entry *entry;
> + unsigned long flags;
>
> entry = ima_alloc_key_entry(keyring, payload, payload_len);
> if (!entry)
> return false;
>
> - spin_lock(&ima_keys_lock);
> + spin_lock_irqsave(&ima_keys_lock, flags);
> if (!ima_process_keys) {
> list_add_tail(&entry->list, &ima_keys);
> queued = true;
> }
> - spin_unlock(&ima_keys_lock);
> + spin_unlock_irqrestore(&ima_keys_lock, flags);
>
> if (!queued)
> ima_free_key_entry(entry);
>

Using sping_lock_irqsave() and spin_lock_irqrestore() in ima_queue_key()
is the right approach. Found a relevant blog below:

https://stackoverflow.com/questions/50637489/spin-lock-irqsave-in-interrupt-context

I think it would be safe to use the same spinlock functions in
ima_process_queued_keys() as well, but not a must.

Could you please confirm if your change fixed the crash?
I'll post a patch shortly.

thanks,
-lakshmi

syzbot

unread,
Jan 17, 2020, 10:14:10 PM1/17/20
to dmitry....@gmail.com, dvy...@google.com, jmo...@namei.org, linux-i...@vger.kernel.org, linux-...@vger.kernel.org, linux-secu...@vger.kernel.org, nra...@linux.microsoft.com, se...@hallyn.com, syzkall...@googlegroups.com, zo...@linux.ibm.com
syzbot has found a reproducer for the following crash on:

HEAD commit: 2747d5fd Add linux-next specific files for 20200116
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=14ee1cc9e00000
kernel config: https://syzkaller.appspot.com/x/.config?x=22f506e7a3a37fe2
dashboard link: https://syzkaller.appspot.com/bug?extid=a4a503d7f37292ae1664
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=128b4495e00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=12b673b9e00000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+a4a503...@syzkaller.appspotmail.com

================================
WARNING: inconsistent lock state
5.5.0-rc6-next-20200116-syzkaller #0 Not tainted
--------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
kworker/u4:3/125 [HC0[0]:SC1[1]:HE1:SE0] takes:
ffffffff8a03ce58 (ima_keys_lock){+.?.}, at: spin_lock include/linux/spinlock.h:338 [inline]
ffffffff8a03ce58 (ima_keys_lock){+.?.}, at: ima_process_queued_keys+0x4f/0x320 security/integrity/ima/ima_asymmetric_keys.c:144
{SOFTIRQ-ON-W} state was registered at:
lock_acquire+0x190/0x410 kernel/locking/lockdep.c:4484
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:338 [inline]
ima_queue_key security/integrity/ima/ima_asymmetric_keys.c:111 [inline]
ima_post_key_create_or_update+0x234/0x470 security/integrity/ima/ima_asymmetric_keys.c:194
key_create_or_update+0x6b8/0xcb0 security/keys/key.c:944
load_system_certificate_list+0x1ba/0x25e certs/system_keyring.c:161
do_one_initcall+0x120/0x820 init/main.c:1109
do_initcall_level init/main.c:1182 [inline]
do_initcalls init/main.c:1198 [inline]
do_basic_setup init/main.c:1218 [inline]
kernel_init_freeable+0x522/0x5d0 init/main.c:1402
kernel_init+0x12/0x1bf init/main.c:1309
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
irq event stamp: 15703140
hardirqs last enabled at (15703140): [<ffffffff87ee0813>] __raw_spin_unlock_irq include/linux/spinlock_api_smp.h:168 [inline]
hardirqs last enabled at (15703140): [<ffffffff87ee0813>] _raw_spin_unlock_irq+0x23/0x80 kernel/locking/spinlock.c:199
hardirqs last disabled at (15703139): [<ffffffff87ee09fa>] __raw_spin_lock_irq include/linux/spinlock_api_smp.h:126 [inline]
hardirqs last disabled at (15703139): [<ffffffff87ee09fa>] _raw_spin_lock_irq+0x3a/0x80 kernel/locking/spinlock.c:167
softirqs last enabled at (15702640): [<ffffffff87cf9855>] spin_unlock_bh include/linux/spinlock.h:383 [inline]
softirqs last enabled at (15702640): [<ffffffff87cf9855>] batadv_nc_purge_paths+0x265/0x370 net/batman-adv/network-coding.c:470
softirqs last disabled at (15702651): [<ffffffff8147a05b>] invoke_softirq kernel/softirq.c:373 [inline]
softirqs last disabled at (15702651): [<ffffffff8147a05b>] irq_exit+0x19b/0x1e0 kernel/softirq.c:413

other info that might help us debug this:
Possible unsafe locking scenario:

CPU0
----
lock(ima_keys_lock);
<Interrupt>
lock(ima_keys_lock);

*** DEADLOCK ***

4 locks held by kworker/u4:3/125:
#0: ffff88821512a928 ((wq_completion)bat_events){+.+.}, at: __write_once_size include/linux/compiler.h:250 [inline]
#0: ffff88821512a928 ((wq_completion)bat_events){+.+.}, at: arch_atomic64_set arch/x86/include/asm/atomic64_64.h:34 [inline]
#0: ffff88821512a928 ((wq_completion)bat_events){+.+.}, at: atomic64_set include/asm-generic/atomic-instrumented.h:869 [inline]
#0: ffff88821512a928 ((wq_completion)bat_events){+.+.}, at: atomic_long_set include/asm-generic/atomic-long.h:41 [inline]
#0: ffff88821512a928 ((wq_completion)bat_events){+.+.}, at: set_work_data kernel/workqueue.c:615 [inline]
#0: ffff88821512a928 ((wq_completion)bat_events){+.+.}, at: set_work_pool_and_clear_pending kernel/workqueue.c:642 [inline]
#0: ffff88821512a928 ((wq_completion)bat_events){+.+.}, at: process_one_work+0x8dd/0x17a0 kernel/workqueue.c:2235
#1: ffffc90001397dc0 ((work_completion)(&(&bat_priv->nc.work)->work)){+.+.}, at: process_one_work+0x917/0x17a0 kernel/workqueue.c:2239
#2: ffffffff89bb0400 (rcu_read_lock){....}, at: batadv_nc_purge_orig_hash net/batman-adv/network-coding.c:405 [inline]
#2: ffffffff89bb0400 (rcu_read_lock){....}, at: batadv_nc_worker+0xe3/0x760 net/batman-adv/network-coding.c:718
#3: ffffc90000d98d50 ((&ima_key_queue_timer)){+.-.}, at: lockdep_copy_map include/linux/lockdep.h:172 [inline]
#3: ffffc90000d98d50 ((&ima_key_queue_timer)){+.-.}, at: call_timer_fn+0xe0/0x780 kernel/time/timer.c:1394

stack backtrace:
CPU: 1 PID: 125 Comm: kworker/u4:3 Not tainted 5.5.0-rc6-next-20200116-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Workqueue: bat_events batadv_nc_worker
RIP: 0010:lock_acquire+0x20b/0x410 kernel/locking/lockdep.c:4487
Code: 9c 08 00 00 00 00 00 00 48 c1 e8 03 80 3c 10 00 0f 85 d3 01 00 00 48 83 3d 49 7d 58 08 00 0f 84 53 01 00 00 48 8b 7d c8 57 9d <0f> 1f 44 00 00 48 8d 65 d8 5b 41 5c 41 5d 41 5e 41 5f 5d c3 65 8b
RSP: 0018:ffffc90001397c70 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff136774f RBX: ffff8880a922c600 RCX: ffffffff815ae240
RDX: dffffc0000000000 RSI: 0000000000000008 RDI: 0000000000000286
RBP: ffffc90001397cb8 R08: 1ffffffff16a51a4 R09: fffffbfff16a51a5
R10: ffff8880a922cef0 R11: ffff8880a922c600 R12: ffffffff89bb0400
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000002
rcu_lock_acquire include/linux/rcupdate.h:208 [inline]
rcu_read_lock include/linux/rcupdate.h:601 [inline]
batadv_nc_purge_orig_hash net/batman-adv/network-coding.c:407 [inline]
batadv_nc_worker+0x117/0x760 net/batman-adv/network-coding.c:718
process_one_work+0xa05/0x17a0 kernel/workqueue.c:2264
worker_thread+0x98/0xe40 kernel/workqueue.c:2410
kthread+0x361/0x430 kernel/kthread.c:255
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

Lakshmi Ramasubramanian

unread,
Jan 18, 2020, 12:22:10 AM1/18/20
to syzbot, dmitry....@gmail.com, dvy...@google.com, jmo...@namei.org, linux-i...@vger.kernel.org, linux-...@vger.kernel.org, linux-secu...@vger.kernel.org, se...@hallyn.com, syzkall...@googlegroups.com, zo...@linux.ibm.com
On 1/17/2020 7:14 PM, syzbot wrote:

> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: syzbot+a4a503...@syzkaller.appspotmail.com
>
> ================================
> WARNING: inconsistent lock state
> 5.5.0-rc6-next-20200116-syzkaller #0 Not tainted
> --------------------------------
> inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
> kworker/u4:3/125 [HC0[0]:SC1[1]:HE1:SE0] takes:
> ffffffff8a03ce58 (ima_keys_lock){+.?.}, at: spin_lock include/linux/spinlock.h:338 [inline]
> ffffffff8a03ce58 (ima_keys_lock){+.?.}, at: ima_process_queued_keys+0x4f/0x320 security/integrity/ima/ima_asymmetric_keys.c:144
> {SOFTIRQ-ON-W} state was registered at:

The fix for this issue is in next-integrity branch. Should be merged to
linux-next shortly.

-lakshmi

Eric Biggers

unread,
Mar 7, 2020, 3:52:20 PM3/7/20
to syzbot, syzkall...@googlegroups.com
This is marked as fixed by "IMA: inconsistent lock state in
ima_process_queued_keys" which doesn't exist. It looks like the broken code got
replaced with different commits before being merged.

#syz invalid
Reply all
Reply to author
Forward
0 new messages