INFO: task hung in ext4_da_get_block_prep

16 views
Skip to first unread message

syzbot

unread,
Sep 4, 2018, 11:41:05 PM9/4/18
to syzkaller-upst...@googlegroups.com
Hello,

syzbot found the following crash on:

HEAD commit: f2b6e66e9885 Add linux-next specific files for 20180904
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=1735dc92400000
kernel config: https://syzkaller.appspot.com/x/.config?x=15ad48400e39c1b3
dashboard link: https://syzkaller.appspot.com/bug?extid=f0fc7f62e88b1de99af3
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
CC: [adilger...@dilger.ca linux...@vger.kernel.org
linux-...@vger.kernel.org ty...@mit.edu]

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+f0fc7f...@syzkaller.appspotmail.com

[ 7961] 0 7961 17585 8737 131072 0 0
syz-executor3
[ 7971] 0 7971 17585 8738 126976 0 0
syz-executor4
[ 7973] 0 7973 17585 8739 126976 0 0
syz-executor5
[ 7975] 0 7975 17585 8739 126976 0 0
syz-executor7
[ 7976] 0 7976 17585 8739 126976 0 0
syz-executor7
INFO: task syz-executor4:10308 blocked for more than 140 seconds.
[ 7981] 0 7981 17585 8740 126976 0 0
syz-executor0
[ 7984] 0 7984 17585 8739 126976 0 0
syz-executor5
[ 7985] 0 7985 17585 8737 126976 0 0
syz-executor6
Not tainted 4.19.0-rc2-next-20180904+ #55
[ 7988] 0 7988 17618 8738 131072 0 0
syz-executor1
[ 7989] 0 7989 17618 8738 131072 0 0
syz-executor1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 7991] 0 7991 17585 8737 126976 0 0
syz-executor6
[ 7992] 0 7992 17585 8740 126976 0 0
syz-executor0
syz-executor4 D
[ 7994] 0 7994 17585 8737 131072 0 0
syz-executor3
[ 7999] 0 7999 17585 8738 126976 0 0
syz-executor4
[ 8003] 0 8003 17585 8737 131072 0 0
syz-executor3
[ 8014] 0 8014 17585 8740 126976 0 0
syz-executor0
20536 10308 4703 0x00000000
[ 8017] 0 8017 17585 8739 126976 0 0
syz-executor7
Call Trace:
[ 8018] 0 8018 17585 8737 126976 0 0
syz-executor6
[ 8020] 0 8020 17618 8741 126976 0 0
syz-executor5
[ 8021] 0 8021 17585 8740 126976 0 0
syz-executor0
context_switch kernel/sched/core.c:2825 [inline]
__schedule+0x87c/0x1df0 kernel/sched/core.c:3473
[ 8022] 0 8022 17585 8739 126976 0 0
syz-executor7
[ 8023] 0 8023 17585 8737 126976 0 0
syz-executor6
[ 8024] 0 8024 17618 8741 126976 0 0
syz-executor5
[ 8030] 0 8030 17585 8738 126976 0 0
syz-executor4
[ 8034] 0 8034 17585 8737 131072 0 0
syz-executor3
[ 8037] 0 8037 17618 8738 131072 0 0
syz-executor1
[ 8039] 0 8039 17585 8737 131072 0 0
syz-executor3
[ 8040] 0 8040 17618 8738 131072 0 0
syz-executor1
schedule+0xfb/0x450 kernel/sched/core.c:3517
[ 8056] 0 8056 17585 8738 126976 0 0
syz-executor4
[ 8055] 0 8055 17618 8741 126976 0 0
syz-executor5
[ 8060] 0 8060 17585 8740 126976 0 0
syz-executor0
[ 8062] 0 8062 17585 8739 126976 0 0
syz-executor7
[ 8063] 0 8063 17618 8741 126976 0 0
syz-executor5
[ 8066] 0 8066 17585 8740 126976 0 0
syz-executor0
[ 8067] 0 8067 17585 8737 126976 0 0
syz-executor6
[ 8070] 0 8070 17618 8739 131072 0 0
syz-executor3
[ 8073] 0 8073 17618 8738 131072 0 0
syz-executor1
[ 8074] 0 8074 17585 8737 126976 0 0
syz-executor6
__rwsem_down_read_failed_common kernel/locking/rwsem-xadd.c:269 [inline]
rwsem_down_read_failed+0x362/0x610 kernel/locking/rwsem-xadd.c:286
[ 8075] 0 8075 17618 8739 131072 0 0
syz-executor3
[ 8077] 0 8077 17618 8738 131072 0 0
syz-executor1
[ 8079] 0 8079 17585 8739 126976 0 0
syz-executor7
[ 8092] 0 8092 17618 8738 131072 0 0
syz-executor1
[ 8097] 0 8097 17585 8740 126976 0 0
syz-executor0
[ 8100] 0 8100 17585 8739 126976 0 0
syz-executor5
[ 8103] 0 8103 17585 8737 126976 0 0
syz-executor6
[ 8104] 0 8104 17585 8740 126976 0 0
syz-executor0
call_rwsem_down_read_failed+0x18/0x30 arch/x86/lib/rwsem.S:94
[ 8106] 0 8106 17585 8739 126976 0 0
syz-executor5
__down_read arch/x86/include/asm/rwsem.h:83 [inline]
down_read+0xc3/0x1d0 kernel/locking/rwsem.c:26
[ 8107] 0 8107 17618 8738 131072 0 0
syz-executor1
[ 8109] 0 8109 17585 8737 126976 0 0
syz-executor6
[ 8117] 0 8117 17585 8737 131072 0 0
syz-executor3
[ 8119] 0 8119 17618 8741 126976 0 0
syz-executor7
[ 8120] 0 8120 17585 8737 131072 0 0
syz-executor3
[ 8124] 0 8124 17585 8737 126976 0 0
syz-executor6
ext4_da_map_blocks fs/ext4/inode.c:1814 [inline]
ext4_da_get_block_prep+0x10a4/0x1b10 fs/ext4/inode.c:1946
[ 8126] 0 8126 17585 8737 126976 0 0
syz-executor6
[ 8127] 0 8127 17618 8741 126976 0 0
syz-executor7
[ 8135] 0 8135 17585 8740 126976 0 0
syz-executor0
[ 8137] 0 8137 17618 8741 126976 0 0
syz-executor5
[ 8139] 0 8139 17618 8738 131072 0 0
syz-executor1
[ 8140] 0 8140 17585 8740 126976 0 0
syz-executor0
[ 8146] 0 8146 17618 8738 131072 0 0
syz-executor1
ext4_block_write_begin+0x5e2/0x1580 fs/ext4/inode.c:1193
[ 8149] 0 8149 17585 8739 126976 0 0
syz-executor7
[ 8152] 0 8152 17585 8737 131072 0 0
syz-executor3
[ 8155] 0 8155 17585 8739 126976 0 0
syz-executor7
[ 8157] 0 8157 17618 8741 126976 0 0
syz-executor5
[ 8160] 0 8160 17585 8737 126976 0 0
syz-executor6
[ 8161] 0 8161 17585 8737 131072 0 0
syz-executor3
[ 8163] 0 8163 17585 8737 126976 0 0
syz-executor6
[ 8173] 0 8173 17618 8739 131072 0 0
syz-executor3
[ 8175] 0 8175 17618 8739 131072 0 0
syz-executor3
ext4_da_write_begin+0x410/0x11f0 fs/ext4/inode.c:3078
[ 8181] 0 8181 17585 8740 126976 0 0
syz-executor0
[ 8182] 0 8182 17618 8738 131072 0 0
syz-executor1
[ 8183] 0 8183 17618 8738 131072 0 0
syz-executor1
[ 8186] 0 8186 17585 8186 114688 0 0
syz-executor4
[ 8190] 0 8190 17585 8739 126976 0 0
syz-executor7
[ 8191] 0 8191 17585 8740 126976 0 0
syz-executor0
[ 8193] 0 8193 17585 8737 126976 0 0
syz-executor6
[ 8197] 0 8197 17585 8739 126976 0 0
syz-executor7
[ 8198] 0 8198 17585 8737 126976 0 0
syz-executor6
[ 8205] 0 8205 17618 8739 131072 0 0
syz-executor3
[ 8210] 0 8210 17618 8739 131072 0 0
syz-executor3
[ 8211] 0 8211 17585 8739 126976 0 0
syz-executor5
[ 8214] 0 8214 17618 8738 131072 0 0
syz-executor1
[ 8216] 0 8216 17618 8738 131072 0 0
syz-executor1
[ 8218] 0 8218 17585 8739 126976 0 0
syz-executor5
[ 8221] 0 8221 17585 8737 126976 0 0
syz-executor6
[ 8227] 0 8227 17585 8739 126976 0 0
syz-executor7
[ 8229] 0 8229 17585 8740 126976 0 0
syz-executor0
generic_perform_write+0x3ae/0x6c0 mm/filemap.c:3139
[ 8232] 0 8232 17585 8738 126976 0 0
syz-executor4
[ 8233] 0 8233 17585 8737 126976 0 0
syz-executor6
[ 8235] 0 8235 17585 8740 126976 0 0
syz-executor0
[ 8237] 0 8237 17585 8739 126976 0 0
syz-executor7
[ 8241] 0 8241 17618 8738 131072 0 0
syz-executor1
[ 8243] 0 8243 17585 8737 131072 0 0
syz-executor3
[ 8244] 0 8244 17585 8737 131072 0 0
syz-executor3
[ 8245] 0 8245 17618 8738 131072 0 0
syz-executor1
__generic_file_write_iter+0x26e/0x630 mm/filemap.c:3264
[ 8252] 0 8252 17618 8233 126976 0 0
syz-executor5
ext4_file_write_iter+0x390/0x1450 fs/ext4/file.c:266
[ 8259] 0 8259 17618 8233 126976 0 0
syz-executor5
[ 8264] 0 8264 17585 8230 126976 0 0
syz-executor4
[ 8265] 0 8265 17585 8230 131072 0 0
syz-executor3
[ 8268] 0 8268 17585 8230 131072 0 0
syz-executor3
[ 8273] 0 8273 17585 8228 126976 0 0
syz-executor7
[ 8275] 0 8275 17585 8228 126976 0 0
syz-executor7
[ 8279] 0 8279 17618 8231 131072 0 0
syz-executor1
[ 8282] 0 8282 17585 8232 126976 0 0
syz-executor0
[ 8283] 0 8283 17585 8232 126976 0 0
syz-executor0
[ 8285] 0 8285 17585 8230 126976 0 0
syz-executor6
[ 8288] 0 8288 17585 8230 126976 0 0
syz-executor6
[ 8289] 0 8289 17618 8231 131072 0 0
syz-executor1
[ 8294] 0 8294 17585 8231 126976 0 0
syz-executor5
call_write_iter include/linux/fs.h:1826 [inline]
new_sync_write fs/read_write.c:474 [inline]
__vfs_write+0x6af/0x9d0 fs/read_write.c:487
[ 8296] 0 8296 17585 8231 126976 0 0
syz-executor5
[ 8302] 0 8302 17585 8230 126976 0 0
syz-executor4
[ 8304] 0 8304 17585 8230 131072 0 0
syz-executor3
[ 8305] 0 8305 17585 8230 131072 0 0
syz-executor3
[ 8309] 0 8309 17585 8230 126976 0 0
syz-executor6
vfs_write+0x1fc/0x560 fs/read_write.c:549
[ 8310] 0 8310 17585 8230 126976 0 0
syz-executor6
ksys_write+0x101/0x260 fs/read_write.c:598
[ 8316] 0 8316 17618 8230 126976 0 0
syz-executor7
[ 8317] 0 8317 17618 8230 126976 0 0
syz-executor7
[ 8321] 0 8321 17618 8231 131072 0 0
syz-executor1
[ 8328] 0 8328 17618 8231 131072 0 0
syz-executor1
__do_sys_write fs/read_write.c:610 [inline]
__se_sys_write fs/read_write.c:607 [inline]
__x64_sys_write+0x73/0xb0 fs/read_write.c:607
[ 8331] 0 8331 17618 8234 126976 0 0
syz-executor5
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
[ 8333] 0 8333 17585 8232 126976 0 0
syz-executor0
[ 8334] 0 8334 17585 8232 126976 0 0
syz-executor0
[ 8337] 0 8337 17618 8234 126976 0 0
syz-executor5
[ 8341] 0 8341 17585 8231 126976 0 0
syz-executor4
[ 8343] 0 8343 17585 8230 131072 0 0
syz-executor3
[ 8346] 0 8346 17585 8230 131072 0 0
syz-executor3
[ 8350] 0 8350 17618 8231 131072 0 0
syz-executor1
[ 8352] 0 8352 17618 8231 131072 0 0
syz-executor1
entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 8356] 0 8356 17618 8231 126976 0 0
syz-executor6
RIP: 0033:0x457099
[ 8357] 0 8357 17618 8231 126976 0 0
syz-executor6
Code: Bad RIP value.
[ 8361] 0 8361 17585 8232 126976 0 0
syz-executor5
RSP: 002b:00007fb85f6d7c78 EFLAGS: 00000246
[ 8362] 0 8362 17585 8232 126976 0 0
syz-executor5
ORIG_RAX: 0000000000000001
[ 8373] 0 8373 17585 8230 131072 0 0
syz-executor3
RAX: ffffffffffffffda RBX: 00007fb85f6d86d4 RCX: 0000000000457099
[ 8374] 0 8374 17585 8232 126976 0 0
syz-executor0
RDX: 00000000fffffd97 RSI: 0000000020000180 RDI: 0000000000000005
[ 8378] 0 8378 17585 8232 126976 0 0
syz-executor0
RBP: 00000000009300a0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
[ 8379] 0 8379 17585 8230 131072 0 0
syz-executor3
R13: 00000000004d7e78 R14: 00000000004cab08 R15: 0000000000000000
[ 8382] 0 8382 17585 8232 126976 0 0
syz-executor4

Showing all locks held in the system:
[ 8383] 0 8383 17585 8228 126976 0 0
syz-executor7
2 locks held by init/1:
[ 8384] 0 8384 17585 8228 126976 0 0
syz-executor7
6 locks held by kworker/u4:2/54:
[ 8395] 0 8395 17618 8230 131072 0 0
syz-executor1
1 lock held by khungtaskd/792:
[ 8396] 0 8396 17585 8230 126976 0 0
syz-executor6
#0: 000000005727aed3
[ 8397] 0 8397 17618 8230 131072 0 0
syz-executor1
(
[ 8399] 0 8399 17585 8231 126976 0 0
syz-executor4
rcu_read_lock
[ 8401] 0 8401 17585 8230 126976 0 0
syz-executor6
){....}
[ 8407] 0 8407 17585 8232 126976 0 0
syz-executor0
, at: debug_show_all_locks+0xd0/0x428 kernel/locking/lockdep.c:4436
1 lock held by kswapd0/1428:
[ 8408] 0 8408 17585 8230 131072 0 0
syz-executor3
4 locks held by rs:main Q:Reg/4555:
[ 8411] 0 8411 17585 8230 131072 0 0
syz-executor3
2 locks held by rsyslogd/4557:
[ 8412] 0 8412 17585 8232 126976 0 0
syz-executor5
3 locks held by cron/4597:
[ 8413] 0 8413 17585 8232 126976 0 0
syz-executor0
2 locks held by getty/4647:
[ 8419] 0 8419 17585 8231 126976 0 0
syz-executor4
#0:
[ 8422] 0 8422 17618 8230 126976 0 0
syz-executor7
00000000dffad8e1
[ 8423] 0 8423 17585 8232 126976 0 0
syz-executor5
(
[ 8424] 0 8424 17618 8230 126976 0 0
syz-executor7
&tty->ldisc_sem){++++}, at: ldsem_down_read+0x37/0x40
drivers/tty/tty_ldsem.c:353
[ 8436] 0 8436 17618 8231 131072 0 0
syz-executor1
#1:
[ 8437] 0 8437 17618 8231 131072 0 0
syz-executor1
00000000ab93cb6f
[ 8440] 0 8440 17585 8230 131072 0 0
syz-executor3
(
[ 8444] 0 8444 17585 8230 131072 0 0
syz-executor3
&ldata->atomic_read_lock
[ 8446] 0 8446 17585 8230 126976 0 0
syz-executor6
){+.+.}
[ 8447] 0 8447 17585 8231 126976 0 0
syz-executor4
, at: n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
[ 8448] 0 8448 17585 8230 126976 0 0
syz-executor6
2 locks held by getty/4648:
[ 8456] 0 8456 17585 8232 126976 0 0
syz-executor0
#0:
[ 8457] 0 8457 17585 8228 126976 0 0
syz-executor7
00000000eff2032f
[ 8458] 0 8458 17585 8232 126976 0 0
syz-executor0
(&tty->ldisc_sem
[ 8460] 0 8460 17585 8228 126976 0 0
syz-executor7
){++++}
[ 8467] 0 8467 17585 8232 126976 0 0
syz-executor5
, at: ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:353
[ 8468] 0 8468 17585 8232 126976 0 0
syz-executor5
#1: 00000000828f9885
[ 8481] 0 8481 17585 8230 126976 0 0
syz-executor6
(
[ 8482] 0 8482 17618 8233 126976 0 0
syz-executor5
&ldata->atomic_read_lock
[ 8484] 0 8484 17618 8230 131072 0 0
syz-executor1
){+.+.}, at: n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
[ 8486] 0 8486 17585 8231 126976 0 0
syz-executor4
[ 8489] 0 8489 17618 8233 126976 0 0
syz-executor5
2 locks held by getty/4649:
[ 8490] 0 8490 17585 8230 126976 0 0
syz-executor6
#0:
[ 8494] 0 8494 17585 8228 126976 0 0
syz-executor7
000000006720244a
[ 8495] 0 8495 17585 8228 126976 0 0
syz-executor7
(&tty->ldisc_sem
[ 8499] 0 8499 17618 8230 131072 0 0
syz-executor1
){++++}, at: ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:353
[ 8501] 0 8501 17585 8232 126976 0 0
syz-executor0
#1: 000000008f3e0751
[ 8502] 0 8502 17585 8230 131072 0 0
syz-executor3
(
[ 8503] 0 8503 17585 8232 126976 0 0
syz-executor0
&ldata->atomic_read_lock
[ 8508] 0 8508 17585 8230 131072 0 0
syz-executor3
){+.+.}
[ 8512] 0 8512 17585 8228 126976 0 0
syz-executor7
, at: n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
[ 8516] 0 8516 17585 8228 126976 0 0
syz-executor7
2 locks held by getty/4650:
[ 8519] 0 8519 17585 8232 126976 0 0
syz-executor0
#0: 0000000046d99145
[ 8521] 0 8521 17585 8230 126976 0 0
syz-executor6
(&tty->ldisc_sem
[ 8522] 0 8522 17585 8230 126976 0 0
syz-executor6
){++++}, at: ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:353
[ 8523] 0 8523 17585 8232 126976 0 0
syz-executor0
#1: 0000000085fc7bc4 (
[ 8526] 0 8526 17585 8232 126976 0 0
syz-executor4
&ldata->atomic_read_lock
[ 8529] 0 8529 17585 8232 126976 0 0
syz-executor5
){+.+.}
[ 8533] 0 8533 17585 8230 131072 0 0
syz-executor3
, at: n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
2 locks held by getty/4651:
[ 8535] 0 8535 17618 8231 131072 0 0
syz-executor1
#0: 00000000ec8b1337
[ 8537] 0 8537 17618 8231 131072 0 0
syz-executor1
(
[ 8540] 0 8540 17585 8230 131072 0 0
syz-executor3
&tty->ldisc_sem
[ 8541] 0 8541 17585 8232 126976 0 0
syz-executor5
){++++}, at: ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:353
#1:
[ 8548] 0 8548 17585 8230 126976 0 0
syz-executor6
00000000289c58f3
[ 8552] 0 8552 17618 8230 126976 0 0
syz-executor7
(
[ 8555] 0 8555 17585 8231 126976 0 0
syz-executor4
&ldata->atomic_read_lock
[ 8556] 0 8556 17585 8230 126976 0 0
syz-executor6
){+.+.}
[ 8561] 0 8561 17585 8232 126976 0 0
syz-executor0
, at: n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
[ 8562] 0 8562 17585 8232 126976 0 0
syz-executor0
2 locks held by getty/4652:
[ 8565] 0 8565 17618 8230 126976 0 0
syz-executor7
#0:
[ 8572] 0 8572 17585 8230 131072 0 0
syz-executor3
00000000da28219e
[ 8574] 0 8574 17618 8231 131072 0 0
syz-executor1
(
[ 8577] 0 8577 17585 8231 126976 0 0
syz-executor4
&tty->ldisc_sem
[ 8582] 0 8582 17618 8233 126976 0 0
syz-executor5
){++++}
[ 8583] 0 8583 17618 8231 131072 0 0
syz-executor1
, at: ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:353
[ 8585] 0 8585 17585 8230 131072 0 0
syz-executor3
#1:
[ 8589] 0 8589 17585 8232 126976 0 0
syz-executor0
0000000000588cc9
[ 8590] 0 8590 17585 8232 126976 0 0
syz-executor0
(
[ 8591] 0 8591 17618 8233 126976 0 0
syz-executor5
&ldata->atomic_read_lock
[ 8594] 0 8594 17585 8230 126976 0 0
syz-executor6
){+.+.}
[ 8595] 0 8595 17585 8230 126976 0 0
syz-executor6
, at: n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
[ 8604] 0 8604 17585 8232 126976 0 0
syz-executor4
2 locks held by getty/4653:
#0:
[ 8606] 0 8606 17585 8232 126976 0 0
syz-executor0
000000002b36e385
[ 8607] 0 8607 17585 8232 126976 0 0
syz-executor0
(
[ 8609] 0 8609 17585 8228 126976 0 0
syz-executor7
&tty->ldisc_sem){++++}
[ 8610] 0 8610 17585 8228 126976 0 0
syz-executor7
, at: ldsem_down_read+0x37/0x40 drivers/tty/tty_ldsem.c:353
[ 8620] 0 8620 17585 8230 131072 0 0
syz-executor3
#1:
[ 8616] 0 8616 17618 8230 131072 0 0
syz-executor1
0000000079a0803a
[ 8621] 0 8621 17618 8230 131072 0 0
syz-executor1
(
[ 8624] 0 8624 17585 8230 131072 0 0
syz-executor3
&ldata->atomic_read_lock){+.+.}
[ 8626] 0 8626 17585 8230 126976 0 0
syz-executor6
, at: n_tty_read+0x335/0x1ce0 drivers/tty/n_tty.c:2140
[ 8630] 0 8630 17585 8230 126976 0 0
syz-executor6
3 locks held by syz-fuzzer/4671:
[ 8637] 0 8637 17585 8228 126976 0 0
syz-executor4
3 locks held by syz-fuzzer/4673:
[ 8640] 0 8640 17618 8233 126976 0 0
syz-executor5
2 locks held by syz-fuzzer/4675:
2 locks held by syz-fuzzer/4682:
[ 8641] 0 8641 17585 8232 126976 0 0
syz-executor0
3 locks held by syz-fuzzer/4683:
[ 8642] 0 8642 17585 8232 126976 0 0
syz-executor0
1 lock held by syz-executor3/4697:
[ 8643] 0 8643 17618 8233 126976 0 0
syz-executor5
1 lock held by syz-executor7/4699:
[ 8653] 0 8653 17585 8228 126976 0 0
syz-executor7
5 locks held by syz-executor2/4700:
[ 8655] 0 8655 17585 8228 126976 0 0
syz-executor7
3 locks held by syz-executor5/4701:
[ 8656] 0 8656 17585 8230 131072 0 0
syz-executor3
2 locks held by syz-executor4/4703:
[ 8658] 0 8658 17585 8230 131072 0 0
syz-executor3
3 locks held by udevd/5431:
[ 8664] 0 8664 17585 8230 126976 0 0
syz-executor6
1 lock held by syz-executor1/10300:
[ 8667] 0 8667 17585 8232 126976 0 0
syz-executor4
#0:
[ 8669] 0 8669 17585 8230 126976 0 0
syz-executor6
0000000096fcd8ef (
[ 8670] 0 8670 17618 8231 131072 0 0
syz-executor1
pcpu_drain_mutex
[ 8671] 0 8671 17618 8231 131072 0 0
syz-executor1
){+.+.}
[ 8677] 0 8677 17585 8232 126976 0 0
syz-executor0
, at: drain_all_pages+0xa0/0x640 mm/page_alloc.c:2632
[ 8679] 0 8679 17585 8232 126976 0 0
syz-executor0
3 locks held by syz-executor4/10304:
[ 8682] 0 8682 17618 8233 126976 0 0
syz-executor5
5 locks held by syz-executor4/10308:
[ 8685] 0 8685 17585 8230 131072 0 0
syz-executor3
#0:
[ 8686] 0 8686 17618 8233 126976 0 0
syz-executor5
00000000770b8dac
[ 8688] 0 8688 17585 8230 131072 0 0
syz-executor3
(
[ 8697] 0 8697 17585 8228 126976 0 0
syz-executor7
&f->f_pos_lock
[ 8699] 0 8699 17618 8231 131072 0 0
syz-executor1
){+.+.}
[ 8702] 0 8702 17585 8231 126976 0 0
syz-executor4
, at: __fdget_pos+0x1bb/0x200 fs/file.c:766
[ 8703] 0 8703 17618 8231 131072 0 0
syz-executor1
#1:
[ 8704] 0 8704 17585 8228 126976 0 0
syz-executor7
00000000297509bb
[ 8711] 0 8711 17585 8232 126976 0 0
syz-executor0
(
[ 8713] 0 8713 17585 8230 126976 0 0
syz-executor6
sb_writers
[ 8715] 0 8715 17585 8232 126976 0 0
syz-executor0
#5){.+.+}
[ 8717] 0 8717 17585 8230 126976 0 0
syz-executor6
, at: file_start_write include/linux/fs.h:2786 [inline]
, at: vfs_write+0x42a/0x560 fs/read_write.c:548
[ 8724] 0 8724 17585 8228 126976 0 0
syz-executor7
#2:
[ 8727] 0 8727 17618 8231 131072 0 0
syz-executor1
00000000107f7248
[ 8728] 0 8728 17585 8228 126976 0 0
syz-executor7
(
[ 8729] 0 8729 17618 8231 131072 0 0
syz-executor1
&sb->s_type->i_mutex_key
[ 8731] 0 8731 17618 8231 131072 0 0
syz-executor3
#13
[ 8732] 0 8732 17618 8231 131072 0 0
syz-executor3
){++++}
[ 8736] 0 8736 17585 8230 126976 0 0
syz-executor6
, at: inode_trylock include/linux/fs.h:771 [inline]
, at: ext4_file_write_iter+0x2a1/0x1450 fs/ext4/file.c:232
[ 8738] 0 8738 17585 8230 126976 0 0
syz-executor6
#3:
[ 8740] 0 8740 17618 8234 126976 0 0
syz-executor5
00000000f353d280
[ 8743] 0 8743 17585 8232 126976 0 0
syz-executor4
(
[ 8751] 0 8751 17618 8234 126976 0 0
syz-executor5
jbd2_handle){++++}
[ 8754] 0 8754 17585 8232 126976 0 0
syz-executor0
, at: start_this_handle+0x589/0x1260 fs/jbd2/transaction.c:383
#4:
[ 8756] 0 8756 17585 8232 126976 0 0
syz-executor0
00000000a9d45af0
[ 8764] 0 8764 17585 8230 131072 0 0
syz-executor3
(
[ 8767] 0 8767 17618 8231 131072 0 0
syz-executor1
&ei->i_data_sem
[ 8769] 0 8769 17585 8228 126976 0 0
syz-executor7
){++++}
[ 8770] 0 8770 17618 8231 131072 0 0
syz-executor1
, at: ext4_da_map_blocks fs/ext4/inode.c:1814 [inline]
, at: ext4_da_get_block_prep+0x10a4/0x1b10 fs/ext4/inode.c:1946
[ 8773] 0 8773 17585 8230 131072 0 0
syz-executor3
3 locks held by syz-executor5/10305:
[ 8774] 0 8774 17618 8231 126976 0 0
syz-executor6

=============================================

[ 8775] 0 8775 17618 8231 126976 0 0
syz-executor6
NMI backtrace for cpu 1
[ 8777] 0 8777 17585 8228 126976 0 0
syz-executor7
CPU: 1 PID: 792 Comm: khungtaskd Not tainted 4.19.0-rc2-next-20180904+ #55
[ 8783] 0 8783 17585 8232 126976 0 0
syz-executor5
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1c9/0x2b4 lib/dump_stack.c:113
[ 8784] 0 8784 17585 8232 126976 0 0
syz-executor5
[ 8792] 0 8792 17585 8232 126976 0 0
syz-executor0
nmi_cpu_backtrace.cold.3+0x48/0x88 lib/nmi_backtrace.c:101
[ 8793] 0 8793 17585 8232 126976 0 0
syz-executor0
nmi_trigger_cpumask_backtrace+0x151/0x192 lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_all_cpu_backtrace include/linux/nmi.h:144 [inline]
check_hung_uninterruptible_tasks kernel/hung_task.c:204 [inline]
watchdog+0xb39/0x1040 kernel/hung_task.c:265
[ 8794] 0 8794 17585 8232 126976 0 0
syz-executor4
[ 8800] 0 8800 17618 8230 131072 0 0
syz-executor1
[ 8801] 0 8801 17585 8230 131072 0 0
syz-executor3
[ 8803] 0 8803 17618 8232 126976 0 0
syz-executor6
[ 8804] 0 8804 17618 8230 131072 0 0
syz-executor1
kthread+0x35a/0x420 kernel/kthread.c:246
[ 8809] 0 8809 17585 8230 131072 0 0
syz-executor3
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:415
Sending NMI from CPU 1 to CPUs 0:
[ 8813] 0 8813 17585 8228 126976 0 0
syz-executor7
NMI backtrace for cpu 0
CPU: 0 PID: 4700 Comm: syz-executor2 Not tainted 4.19.0-rc2-next-20180904+
#55
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
RIP: 0010:memcpy include/linux/string.h:345 [inline]
RIP: 0010:vsnprintf+0x527/0x1b60 lib/vsprintf.c:2260
Code: 39 e8 7e 08 e8 ca 37 a9 fa 49 63 d5 48 89 95 68 ff ff ff e8 bb 37 a9
fa 48 8b 95 68 ff ff ff 4c 89 e6 48 89 df e8 39 4a e8 fa <e8> a4 37 a9 fa
4c 89 f0 4c 89 f2 48 b9 00 00 00 00 00 fc ff df 48
RSP: 0000:ffff880194f06628 EFLAGS: 00000006
RAX: ffff880194efe580 RBX: 0000000000000006 RCX: ffffffff86d39b60
RDX: 0000000000000000 RSI: ffffffff86d39e6f RDI: 0000000000000001
RBP: ffff880194f066f8 R08: ffff880194efe580 R09: fffffbfff1031440
R10: fffffbfff1031440 R11: ffffffff8818a203 R12: ffffffff872b8865
R13: 0000000000000000 R14: ffffffff872b886e R15: ffff880194f066d0
FS: 0000000001830940(0000) GS:ffff8801db000000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000455176 CR3: 0000000194ef3000 CR4: 00000000001406f0
Call Trace:
snprintf+0xae/0xe0 lib/vsprintf.c:2431
print_time kernel/printk/printk.c:1223 [inline]
print_prefix+0x38e/0x3f0 kernel/printk/printk.c:1248
msg_print_text+0x85/0x1c0 kernel/printk/printk.c:1271
console_unlock+0x71c/0x10d0 kernel/printk/printk.c:2381
vprintk_emit+0x33a/0x910 kernel/printk/printk.c:1926
vprintk_default+0x28/0x30 kernel/printk/printk.c:1967
vprintk_func+0x7a/0x117 kernel/printk/printk_safe.c:398
printk+0xa7/0xcf kernel/printk/printk.c:2000
dump_tasks mm/oom_kill.c:420 [inline]
dump_header+0xf0d/0xf70 mm/oom_kill.c:450
oom_kill_process.cold.28+0x10/0x95a mm/oom_kill.c:953
out_of_memory+0xa88/0x1430 mm/oom_kill.c:1120
__alloc_pages_may_oom mm/page_alloc.c:3529 [inline]
__alloc_pages_slowpath+0x223f/0x2cb0 mm/page_alloc.c:4242
__alloc_pages_nodemask+0xa1b/0xd10 mm/page_alloc.c:4397
__alloc_pages include/linux/gfp.h:473 [inline]
__alloc_pages_node include/linux/gfp.h:486 [inline]
kmem_getpages mm/slab.c:1409 [inline]
cache_grow_begin+0x91/0x710 mm/slab.c:2677
fallback_alloc+0x203/0x2c0 mm/slab.c:3219
____cache_alloc_node+0x1c7/0x1e0 mm/slab.c:3287
__do_cache_alloc mm/slab.c:3356 [inline]
slab_alloc mm/slab.c:3384 [inline]
kmem_cache_alloc+0x1e5/0x710 mm/slab.c:3552
getname_flags+0xd0/0x5a0 fs/namei.c:140
getname+0x19/0x20 fs/namei.c:211
do_sys_open+0x3a2/0x720 fs/open.c:1057
__do_sys_open fs/open.c:1081 [inline]
__se_sys_open fs/open.c:1076 [inline]
__x64_sys_open+0x7e/0xc0 fs/open.c:1076
do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4551a0
Code: Bad RIP value.
RSP: 002b:00007fff55624020 EFLAGS: 00000202 ORIG_RAX: 0000000000000002
RAX: ffffffffffffffda RBX: 000000000183199b RCX: 00000000004551a0
RDX: 000000000000000c RSI: 0000000000090800 RDI: 00007fff556251d0
RBP: 000000000000070b R08: 0000000000000001 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000202 R12: 00007fff556251d0
R13: 00000000000e0d16 R14: 0000000000000000 R15: badc0ffeebadface


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.

Dmitry Vyukov

unread,
Sep 5, 2018, 3:22:36 AM9/5/18
to syzbot, Tetsuo Handa, 'Dmitry Vyukov' via syzkaller-upstream-moderation
On Wed, Sep 5, 2018 at 5:41 AM, syzbot
<syzbot+f0fc7f...@syzkaller.appspotmail.com> wrote:
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit: f2b6e66e9885 Add linux-next specific files for 20180904
> git tree: linux-next
> console output: https://syzkaller.appspot.com/x/log.txt?x=1735dc92400000
> kernel config: https://syzkaller.appspot.com/x/.config?x=15ad48400e39c1b3
> dashboard link: https://syzkaller.appspot.com/bug?extid=f0fc7f62e88b1de99af3
> compiler: gcc (GCC) 8.0.1 20180413 (experimental)
> CC: [adilger...@dilger.ca linux...@vger.kernel.org
> linux-...@vger.kernel.org ty...@mit.edu]
>
> Unfortunately, I don't have any reproducer for this crash yet.
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: syzbot+f0fc7f...@syzkaller.appspotmail.com
>
> [ 7961] 0 7961 17585 8737 131072 0 0
> syz-executor3

Hi Tetsuo,

Maybe you know what are these repeated lines with numbers?
We started getting them on linux-next recently, also:
https://syzkaller.appspot.com/bug?extid=f8fa79b458bcae4d913d
They seem to cause various hangs/stalls.
> --
> You received this message because you are subscribed to the Google Groups
> "syzkaller-upstream-moderation" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to syzkaller-upstream-m...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/syzkaller-upstream-moderation/0000000000004a6b700575178b5a%40google.com.
> For more options, visit https://groups.google.com/d/optout.

Dmitry Vyukov

unread,
Sep 5, 2018, 7:06:53 AM9/5/18
to Tetsuo Handa, syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation, linux-mm, Michal Hocko
On Wed, Sep 5, 2018 at 12:53 PM, Tetsuo Handa
<penguin...@i-love.sakura.ne.jp> wrote:
> On 2018/09/05 16:22, Dmitry Vyukov wrote:
>> On Wed, Sep 5, 2018 at 5:41 AM, syzbot
>> <syzbot+f0fc7f...@syzkaller.appspotmail.com> wrote:
>>> Hello,
>>>
>>> syzbot found the following crash on:
>>>
>>> HEAD commit: f2b6e66e9885 Add linux-next specific files for 20180904
>>> git tree: linux-next
>>> console output: https://syzkaller.appspot.com/x/log.txt?x=1735dc92400000
>>> kernel config: https://syzkaller.appspot.com/x/.config?x=15ad48400e39c1b3
>>> dashboard link: https://syzkaller.appspot.com/bug?extid=f0fc7f62e88b1de99af3
>>> compiler: gcc (GCC) 8.0.1 20180413 (experimental)
>>> CC: [adilger...@dilger.ca linux...@vger.kernel.org
>>> linux-...@vger.kernel.org ty...@mit.edu]
>>>
>>> Unfortunately, I don't have any reproducer for this crash yet.
>>>
>>> IMPORTANT: if you fix the bug, please add the following tag to the commit:
>>> Reported-by: syzbot+f0fc7f...@syzkaller.appspotmail.com
>>>
>>> [ 7961] 0 7961 17585 8737 131072 0 0
>>> syz-executor3
>>
>> Hi Tetsuo,
>>
>> Maybe you know what are these repeated lines with numbers?
>> We started getting them on linux-next recently, also:
>> https://syzkaller.appspot.com/bug?extid=f8fa79b458bcae4d913d
>> They seem to cause various hangs/stalls.
>
> Yes, these lines are from the OOM killer. (Thus, if we can, I want to
> remove ext4 people before upstreaming this report.)

This is not possible at the moment.

> dump_tasks mm/oom_kill.c:420 [inline]
> dump_header+0xf0d/0xf70 mm/oom_kill.c:450
> oom_kill_process.cold.28+0x10/0x95a mm/oom_kill.c:953
> out_of_memory+0xa88/0x1430 mm/oom_kill.c:1120
>
> What is annoying is that one for_each_process() traversal with printk() is
> taking 52 seconds which is too long to do under RCU section. Under such
> situation, invoking the OOM killer for three times will exceed khungtaskd
> threshold 140 seconds. Was syzbot trying to test fork bomb situation?

Hard to tell. I only know what's captured in the console output.

> Anyway, we might need to introduce rcu_lock_break() like
> check_hung_uninterruptible_tasks() does...
>
> [ 999.629589] [ 16497] 0 16497 17585 8739 126976 0 0 syz-executor5
> [ 1026.435955] [ 32764] 0 32764 17585 8739 126976 0 0 syz-executor5
> [ 1026.445027] [ 311] 0 311 17585 8737 131072 0 0 syz-executor3
> [ 1047.914324] [ 10315] 0 10315 17585 8271 126976 0 0 syz-executor0
> [ 1047.923384] Out of memory: Kill process 4670 (syz-fuzzer) score 53 or sacrifice child
> [ 1047.931934] Killed process 5032 (syz-executor1) total-vm:70212kB, anon-rss:60kB, file-rss:0kB, shmem-rss:0kB
> [ 1047.988138] syz-executor2 invoked oom-killer: gfp_mask=0x6040c0(GFP_KERNEL|__GFP_COMP), nodemask=(null), order=1, oom_score_adj=0
> [ 1048.000015] syz-executor2 cpuset=syz2 mems_allowed=0
> [ 1048.005199] CPU: 0 PID: 4700 Comm: syz-executor2 Not tainted 4.19.0-rc2-next-20180904+ #55
> [ 1048.740679] [ 2347] 0 2347 278 186 32768 0 0 none
> [ 1051.319928] [ 16497] 0 16497 17585 8739 126976 0 0 syz-executor5
> [ 1096.740878] [ 8841] 0 8841 17585 8232 126976 0 0 syz-executor5
> [ 1078.140677] [ 32764] 0 32764 17585 8739 126976 0 0 syz-executor5
> [ 1078.149807] [ 311] 0 311 17585 8737 131072 0 0 syz-executor3
> [ 1096.740878] [ 8841] 0 8841 17585 8232 126976 0 0 syz-executor5
>
> Also, another notable thing is that the backtrace for some reason includes
>
> [ 1048.211540] ? oom_killer_disable+0x3a0/0x3a0
>
> line. Was syzbot testing process freezing functionality?

What's the API for this?

Dmitry Vyukov

unread,
Sep 6, 2018, 5:54:24 AM9/6/18
to Tetsuo Handa, syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation, linux-mm, Michal Hocko
On Thu, Sep 6, 2018 at 7:53 AM, Tetsuo Handa
<penguin...@i-love.sakura.ne.jp> wrote:
> Dmitry Vyukov wrote:
>> > Also, another notable thing is that the backtrace for some reason includes
>> >
>> > [ 1048.211540] ? oom_killer_disable+0x3a0/0x3a0
>> >
>> > line. Was syzbot testing process freezing functionality?
>>
>> What's the API for this?
>>
>
> I'm not a user of suspend/hibernation. But it seems that usage of the API
> is to write one of words listed in /sys/power/state into /sys/power/state .
>
> # echo suspend > /sys/power/state

syzkaller should not write to /sys/power/state. The only mention of
"power" is in some selinux contexts.

Dmitry Vyukov

unread,
Sep 6, 2018, 7:07:21 AM9/6/18
to Tetsuo Handa, Michal Hocko, Andrew Morton, David Rientjes, syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation, linux-mm
On Thu, Sep 6, 2018 at 12:58 PM, Tetsuo Handa
<penguin...@i-love.sakura.ne.jp> wrote:
> OK. Then, I have no idea.
> Anyway, I think we can apply this patch.
>
> From 18876f287dd69a7c33f65c91cfcda3564233f55e Mon Sep 17 00:00:00 2001
> From: Tetsuo Handa <penguin...@I-love.SAKURA.ne.jp>
> Date: Thu, 6 Sep 2018 19:53:18 +0900
> Subject: [PATCH] mm, oom: Introduce time limit for dump_tasks duration.
>
> Since printk() is slow, printing one line takes nearly 0.01 second.
> As a result, syzbot is stalling for 52 seconds trying to dump 5600

I wonder why there are so many of them?
We have at most 8 test processes (each having no more than 16 threads
if that matters).
No more than 1 instance of syz-executor1 at a time. But we see output
like the one below. It has lots of instances of syz-executor1 with
different pid's. So does it print all tasks that ever existed (kernel
does not store that info, right)? Or it livelocks picking up new and
new tasks as they are created slower than they are created? Or we have
tons of zombies?

...
[ 8037] 0 8037 17618 8738 131072 0
0 syz-executor1
[ 8039] 0 8039 17585 8737 131072 0
0 syz-executor3
...


> tasks at for_each_process() under RCU. Since such situation is almost
> inflight fork bomb attack (the OOM killer will print similar tasks for
> so many times), it makes little sense to print all candidate tasks.
> Thus, this patch introduces 3 seconds limit for printing.
>
> Signed-off-by: Tetsuo Handa <penguin...@I-love.SAKURA.ne.jp>
> Cc: Dmitry Vyukov <dvy...@google.com>
> ---
> mm/oom_kill.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index f10aa53..48e5bf6 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -399,14 +399,22 @@ static void dump_tasks(struct mem_cgroup *memcg, const nodemask_t *nodemask)
> {
> struct task_struct *p;
> struct task_struct *task;
> + unsigned long start;
> + unsigned int skipped = 0;
>
> pr_info("Tasks state (memory values in pages):\n");
> pr_info("[ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name\n");
> rcu_read_lock();
> + start = jiffies;
> for_each_process(p) {
> if (oom_unkillable_task(p, memcg, nodemask))
> continue;
>
> + if (time_after(jiffies, start + 3 * HZ)) {
> + skipped++;
> + continue;
> + }
> +
> task = find_lock_task_mm(p);
> if (!task) {
> /*
> @@ -426,6 +434,8 @@ static void dump_tasks(struct mem_cgroup *memcg, const nodemask_t *nodemask)
> task_unlock(task);
> }
> rcu_read_unlock();
> + if (skipped)
> + pr_info("Printing %u tasks omitted.\n", skipped);
> }
>
> static void dump_header(struct oom_control *oc, struct task_struct *p)
> --
> 1.8.3.1
>

Dmitry Vyukov

unread,
Sep 6, 2018, 8:09:05 AM9/6/18
to Michal Hocko, Tetsuo Handa, Andrew Morton, David Rientjes, syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation, linux-mm
On Thu, Sep 6, 2018 at 1:53 PM, Michal Hocko <mho...@kernel.org> wrote:
> On Thu 06-09-18 20:40:34, Tetsuo Handa wrote:
>> On 2018/09/06 20:23, Michal Hocko wrote:
>> > On Thu 06-09-18 19:58:25, Tetsuo Handa wrote:
>> > [...]
>> >> >From 18876f287dd69a7c33f65c91cfcda3564233f55e Mon Sep 17 00:00:00 2001
>> >> From: Tetsuo Handa <penguin...@I-love.SAKURA.ne.jp>
>> >> Date: Thu, 6 Sep 2018 19:53:18 +0900
>> >> Subject: [PATCH] mm, oom: Introduce time limit for dump_tasks duration.
>> >>
>> >> Since printk() is slow, printing one line takes nearly 0.01 second.
>> >> As a result, syzbot is stalling for 52 seconds trying to dump 5600
>> >> tasks at for_each_process() under RCU. Since such situation is almost
>> >> inflight fork bomb attack (the OOM killer will print similar tasks for
>> >> so many times), it makes little sense to print all candidate tasks.
>> >> Thus, this patch introduces 3 seconds limit for printing.
>> >>
>> >> Signed-off-by: Tetsuo Handa <penguin...@I-love.SAKURA.ne.jp>
>> >> Cc: Dmitry Vyukov <dvy...@google.com>
>> >
>> > You really love timeout based solutions with randomly chosen timeouts,
>> > don't you. This is just ugly as hell. We already have means to disable
>> > tasks dumping (see /proc/sys/vm/oom_dump_tasks).
>>
>> I know /proc/sys/vm/oom_dump_tasks . Showing some entries while not always
>> printing all entries might be helpful.
>
> Not really. It could be more confusing than helpful. The main purpose of
> the listing is to double check the list to understand the oom victim
> selection. If you have a partial list you simply cannot do that.
>
> If the iteration takes too long and I can imagine it does with zillions
> of tasks then the proper way around it is either release the lock
> periodically after N tasks is processed or outright skip the whole thing
> if there are too many tasks. The first option is obviously tricky to
> prevent from duplicate entries or other artifacts.


So does anybody know if it can live lock picking up new tasks all the
time? That's what it looks like at first glance. I also don't remember
seeing anything similar in the past.
If it's a live lock and we resolve it, then we don't need to solve the
problem of too many tasks here.

Dmitry Vyukov

unread,
Sep 7, 2018, 5:37:17 AM9/7/18
to Michal Hocko, Tetsuo Handa, Andrew Morton, David Rientjes, syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation, linux-mm
On Fri, Sep 7, 2018 at 10:27 AM, Michal Hocko <mho...@kernel.org> wrote:
> On Fri 07-09-18 05:58:06, Tetsuo Handa wrote:
>> On 2018/09/06 23:39, Michal Hocko wrote:
>> >>>> I know /proc/sys/vm/oom_dump_tasks . Showing some entries while not always
>> >>>> printing all entries might be helpful.
>> >>>
>> >>> Not really. It could be more confusing than helpful. The main purpose of
>> >>> the listing is to double check the list to understand the oom victim
>> >>> selection. If you have a partial list you simply cannot do that.
>> >>
>> >> It serves as a safeguard for avoiding RCU stall warnings.
>> >>
>> >>>
>> >>> If the iteration takes too long and I can imagine it does with zillions
>> >>> of tasks then the proper way around it is either release the lock
>> >>> periodically after N tasks is processed or outright skip the whole thing
>> >>> if there are too many tasks. The first option is obviously tricky to
>> >>> prevent from duplicate entries or other artifacts.
>> >>>
>> >>
>> >> Can we add rcu_lock_break() like check_hung_uninterruptible_tasks() does?
>> >
>> > This would be a better variant of your timeout based approach. But it
>> > can still produce an incomplete task list so it still consumes a lot of
>> > resources to print a long list of tasks potentially while that list is not
>> > useful for any evaluation. Maybe that is good enough. I don't know. I
>> > would generally recommend to disable the whole thing with workloads with
>> > many tasks though.
>> >
>>
>> The "safeguard" is useful when there are _unexpectedly_ many tasks (like
>> syzbot in this case). Why not to allow those who want to avoid lockup to
>> avoid lockup rather than forcing them to disable the whole thing?
>
> So you get an rcu lockup splat and what? Unless you have panic_on_rcu_stall
> then this should be recoverable thing (assuming we cannot really
> livelock as described by Dmitry).


Should I add "vm.oom_dump_tasks = 0" to /etc/sysctl.conf on syzbot?
It looks like it will make things faster, not pollute console output,
prevent these stalls and that output does not seem to be too useful
for debugging.

But I am still concerned as to what has changed recently. Potentially
this happens only on linux-next, at least that's where I saw all
existing reports.
New tasks seem to be added to the tail of the tasks list, but this
part does not seem to be changed recently in linux-next..

Dmitry Vyukov

unread,
Sep 8, 2018, 10:00:22 AM9/8/18
to Michal Hocko, Tetsuo Handa, Andrew Morton, David Rientjes, syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation, linux-mm
> I think that oom_dump_tasks has only very limited usefulness for your
> testing.
>
>> But I am still concerned as to what has changed recently. Potentially
>> this happens only on linux-next, at least that's where I saw all
>> existing reports.
>> New tasks seem to be added to the tail of the tasks list, but this
>> part does not seem to be changed recently in linux-next..
>
> Yes, that would be interesting to find out.


Looking at another similar report:
https://syzkaller.appspot.com/bug?extid=0d867757fdc016c0157e
It looks like it can be just syzkaller learning how to do fork bombs
after all (same binary multiplied infinite amount of times). Probably
required some creativity because test programs do not contain loops
per se and clone syscall does not accept start function pc.
I will set vm.oom_dump_tasks = 0 and try to additionally restrict it
with cgroups.

Dmitry Vyukov

unread,
Sep 10, 2018, 10:37:21 AM9/10/18
to Michal Hocko, Tetsuo Handa, Andrew Morton, David Rientjes, syzbot, 'Dmitry Vyukov' via syzkaller-upstream-moderation, linux-mm
FTR, syzkaller now restricts test processes with pids.max=32. This
should prevent any fork bombs.
https://github.com/google/syzkaller/commit/f167cb6b0957d34f95b1067525aa87083f264035

syzbot

unread,
Mar 3, 2019, 6:33:03 AM3/3/19
to ak...@linux-foundation.org, dvy...@google.com, linu...@kvack.org, mho...@kernel.org, ol...@redhat.com, penguin...@i-love.sakura.ne.jp, rien...@google.com, syzkaller-upst...@googlegroups.com
Auto-closing this bug as obsolete.
Crashes did not happen for a while, no reproducer and no activity.
Reply all
Reply to author
Forward
0 new messages