kernel BUG in ext4_free_blocks (2)

19 views
Skip to first unread message

syzbot

unread,
Jul 3, 2022, 10:04:20 PM7/3/22
to syzkaller-a...@googlegroups.com
Hello,

syzbot found the following issue on:

HEAD commit: fa7f6a5f56d9 Merge branch 'android12-5.10' into branch 'an..
git tree: android12-5.10-lts
console+strace: https://syzkaller.appspot.com/x/log.txt?x=1242b968080000
kernel config: https://syzkaller.appspot.com/x/.config?x=30bf844d1a89de15
dashboard link: https://syzkaller.appspot.com/bug?extid=15cd994e273307bf5cfa
compiler: Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1609e668080000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1063cf04080000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+15cd99...@syzkaller.appspotmail.com

EXT4-fs (loop0): ext4_check_descriptors: Checksum for group 0 failed (14603!=0)
EXT4-fs (loop0): orphan cleanup on readonly fs
EXT4-fs error (device loop0): ext4_free_blocks:5434: comm syz-executor392: Freeing blocks in system zone - Block = 16, count = 16
EXT4-fs (loop0): Remounting filesystem read-only
------------[ cut here ]------------
kernel BUG at fs/ext4/ext4.h:3254!
invalid opcode: 0000 [#1] PREEMPT SMP KASAN
CPU: 1 PID: 371 Comm: syz-executor392 Not tainted 5.10.118-syzkaller-00163-gfa7f6a5f56d9 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/29/2022
RIP: 0010:ext4_get_group_info fs/ext4/ext4.h:3254 [inline]
RIP: 0010:ext4_free_blocks+0x29ed/0x2ad0 fs/ext4/mballoc.c:5400
Code: f6 48 0f a3 05 f4 8e bf 04 0f 92 c3 40 0f 92 c6 31 ff e8 c6 62 8d ff 84 db 75 18 e8 ad 5f 8d ff e9 b8 00 00 00 e8 a3 5f 8d ff <0f> 0b e8 9c 5f 8d ff 0f 0b 65 ff 05 2f 1a 23 7e 48 c7 c0 68 bd 9c
RSP: 0018:ffffc900002aef80 EFLAGS: 00010293
RAX: ffffffff81df534d RBX: 0000000000000001 RCX: ffff888106992780
RDX: 0000000000000000 RSI: 00000000ffffffff RDI: 0000000000000001
RBP: ffffc900002af200 R08: ffffffff81df350e R09: ffffc900002af140
R10: fffff52000055e2f R11: 1ffff92000055e28 R12: 0000000000000000
R13: dffffc0000000000 R14: 00000000ffffffff R15: ffff8881065e1000
FS: 00005555557bf300(0000) GS:ffff8881f7100000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffe40c84cd8 CR3: 000000011e1bf000 CR4: 00000000003506a0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
ext4_clear_blocks+0x3b5/0x420 fs/ext4/indirect.c:877
ext4_free_data fs/ext4/indirect.c:950 [inline]
ext4_ind_truncate+0x83f/0x1040 fs/ext4/indirect.c:1141
ext4_truncate+0xae6/0x1270 fs/ext4/inode.c:4347
ext4_evict_inode+0xecf/0x16d0 fs/ext4/inode.c:288
evict+0x2a3/0x6c0 fs/inode.c:578
iput_final fs/inode.c:1656 [inline]
iput+0x61f/0x7d0 fs/inode.c:1682
ext4_quota_enable fs/ext4/super.c:6406 [inline]
ext4_enable_quotas+0x5a5/0x960 fs/ext4/super.c:6429
ext4_orphan_cleanup+0x2ee/0xdb0 fs/ext4/super.c:3058
ext4_fill_super+0x896c/0x9240 fs/ext4/super.c:5089
mount_bdev+0x25f/0x370 fs/super.c:1419
ext4_mount+0x34/0x40 fs/ext4/super.c:6607
legacy_get_tree+0xf0/0x190 fs/fs_context.c:592
vfs_get_tree+0x88/0x290 fs/super.c:1549
do_new_mount+0x289/0xad0 fs/namespace.c:2899
path_mount+0x58d/0xce0 fs/namespace.c:3229
do_mount fs/namespace.c:3242 [inline]
__do_sys_mount fs/namespace.c:3450 [inline]
__se_sys_mount+0x2d2/0x3c0 fs/namespace.c:3427
__x64_sys_mount+0xbf/0xd0 fs/namespace.c:3427
do_syscall_64+0x34/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7fc08f7344da
Code: 83 c4 08 5b 5d c3 66 2e 0f 1f 84 00 00 00 00 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffc5770e458 EFLAGS: 00000202 ORIG_RAX: 00000000000000a5
RAX: ffffffffffffffda RBX: 00007ffc5770e4b0 RCX: 00007fc08f7344da
RDX: 0000000020000000 RSI: 0000000020000040 RDI: 00007ffc5770e470
RBP: 00007ffc5770e470 R08: 00007ffc5770e4b0 R09: 0000000800000015
R10: 0000000000000081 R11: 0000000000000202 R12: 0000000000000004
R13: 0000000000000003 R14: 0000000000000003 R15: 0000000000000010
Modules linked in:
---[ end trace 8d6b13b82edeee5a ]---
RIP: 0010:ext4_get_group_info fs/ext4/ext4.h:3254 [inline]
RIP: 0010:ext4_free_blocks+0x29ed/0x2ad0 fs/ext4/mballoc.c:5400
Code: f6 48 0f a3 05 f4 8e bf 04 0f 92 c3 40 0f 92 c6 31 ff e8 c6 62 8d ff 84 db 75 18 e8 ad 5f 8d ff e9 b8 00 00 00 e8 a3 5f 8d ff <0f> 0b e8 9c 5f 8d ff 0f 0b 65 ff 05 2f 1a 23 7e 48 c7 c0 68 bd 9c
RSP: 0018:ffffc900002aef80 EFLAGS: 00010293
RAX: ffffffff81df534d RBX: 0000000000000001 RCX: ffff888106992780
RDX: 0000000000000000 RSI: 00000000ffffffff RDI: 0000000000000001
RBP: ffffc900002af200 R08: ffffffff81df350e R09: ffffc900002af140
R10: fffff52000055e2f R11: 1ffff92000055e28 R12: 0000000000000000
R13: dffffc0000000000 R14: 00000000ffffffff R15: ffff8881065e1000
FS: 00005555557bf300(0000) GS:ffff8881f7100000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ffe40c84cd8 CR3: 000000011e1bf000 CR4: 00000000003506a0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

syzbot

unread,
Oct 20, 2022, 12:32:32 PM10/20/22
to adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, ty...@mit.edu
This bug is marked as fixed by commit:
ext4: block range must be validated before use in ext4_mb_clear_bb()
But I can't find it in any tested tree for more than 90 days.
Is it a correct commit? Please update it by replying:
#syz fix: exact-commit-title
Until then the bug is still considered open and
new crashes with the same signature are ignored.

syzbot

unread,
Nov 3, 2022, 12:33:36 PM11/3/22
to adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, ty...@mit.edu

syzbot

unread,
Nov 17, 2022, 11:34:27 AM11/17/22
to adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, ty...@mit.edu

syzbot

unread,
Dec 1, 2022, 11:34:32 AM12/1/22
to adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, ty...@mit.edu

syzbot

unread,
Dec 15, 2022, 11:34:36 AM12/15/22
to adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, ty...@mit.edu

Theodore Ts'o

unread,
Dec 15, 2022, 9:11:28 PM12/15/22
to syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
I don't know what is going on with syzkaller's commit detection, but
commit 1e1c2b86ef86 ("ext4: block range must be validated before use
in ext4_mb_clear_bb()") is an exact match for the commit title, and
it's been in the upstream kernel since v6.0.

How do we make syzkaller accept this? I'll try this again, but I
don't hold out much hope.

#syz fix: ext4: block range must be validated before use in ext4_mb_clear_bb()

Syzkaller, go home, you're drunk.

- Ted

Lee Jones

unread,
Dec 16, 2022, 8:01:30 AM12/16/22
to Theodore Ts'o, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
On Thu, 15 Dec 2022, Theodore Ts'o wrote:

> On Thu, Dec 15, 2022 at 08:34:35AM -0800, syzbot wrote:
> > This bug is marked as fixed by commit:
> > ext4: block range must be validated before use in ext4_mb_clear_bb()
> > But I can't find it in any tested tree for more than 90 days.
> > Is it a correct commit? Please update it by replying:
> > #syz fix: exact-commit-title
> > Until then the bug is still considered open and
> > new crashes with the same signature are ignored.
>
> I don't know what is going on with syzkaller's commit detection, but
> commit 1e1c2b86ef86 ("ext4: block range must be validated before use
> in ext4_mb_clear_bb()") is an exact match for the commit title, and
> it's been in the upstream kernel since v6.0.
>
> How do we make syzkaller accept this? I'll try this again, but I
> don't hold out much hope.

I don't see the original bug report (was it posted to a lore
associated list?), so there is no way to tell what branch syzbot was
fuzzing at the time. My assumption is that it was !Mainline.

Although this does appear to be a Stable candidate, I do not see it
in any of the Stable branches yet. So I suspect the answer here is to
wait for the fix to filter down.

In the mean time, I guess we should discuss whether syzbot should
really be posting scans of downstream trees to upstream lists.

> #syz fix: ext4: block range must be validated before use in ext4_mb_clear_bb()
>
> Syzkaller, go home, you're drunk.

=:-)

--
Lee Jones [李琼斯]

Aleksandr Nogikh

unread,
Dec 16, 2022, 9:09:17 AM12/16/22
to Lee Jones, Theodore Ts'o, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
On Fri, Dec 16, 2022 at 2:01 PM Lee Jones <l...@kernel.org> wrote:
>
> On Thu, 15 Dec 2022, Theodore Ts'o wrote:
>
> > On Thu, Dec 15, 2022 at 08:34:35AM -0800, syzbot wrote:
> > > This bug is marked as fixed by commit:
> > > ext4: block range must be validated before use in ext4_mb_clear_bb()
> > > But I can't find it in any tested tree for more than 90 days.
> > > Is it a correct commit? Please update it by replying:
> > > #syz fix: exact-commit-title
> > > Until then the bug is still considered open and
> > > new crashes with the same signature are ignored.
> >
> > I don't know what is going on with syzkaller's commit detection, but
> > commit 1e1c2b86ef86 ("ext4: block range must be validated before use
> > in ext4_mb_clear_bb()") is an exact match for the commit title, and
> > it's been in the upstream kernel since v6.0.
> >
> > How do we make syzkaller accept this? I'll try this again, but I
> > don't hold out much hope.
>
> I don't see the original bug report (was it posted to a lore
> associated list?), so there is no way to tell what branch syzbot was
> fuzzing at the time. My assumption is that it was !Mainline.

Syzbot is actually reacting here to this bug from the Android namespace:

https://syzkaller.appspot.com/bug?id=5266d464285a03cee9dbfda7d2452a72c3c2ae7c

>
> Although this does appear to be a Stable candidate, I do not see it
> in any of the Stable branches yet. So I suspect the answer here is to
> wait for the fix to filter down.
>
> In the mean time, I guess we should discuss whether syzbot should
> really be posting scans of downstream trees to upstream lists.

In this particular case, syzbot has captured all the recipients from
the patch email [1], because that email Cc'd
syzbot+15cd99...@syzkaller.appspotmail.com. To syzbot, all
these people were involved in the original bug discussion, and so it
notified them about the problem.

FWIW I've sent a PR[2] to make the "I can't find it in any tested
tree" message include the link to the syzkaller dashboard. Hopefully
it will help resolve such confusions faster.

[1] https://lore.kernel.org/all/20220713185904.641...@linaro.org/
[2] https://github.com/google/syzkaller/pull/3591

--
Aleksandr


>
> > #syz fix: ext4: block range must be validated before use in ext4_mb_clear_bb()
> >
> > Syzkaller, go home, you're drunk.
>
> =:-)
>
> --
> Lee Jones [李琼斯]
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-android-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-android...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-android-bugs/Y5xsIkpIznpObOJL%40google.com.

Lee Jones

unread,
Dec 16, 2022, 9:12:57 AM12/16/22
to Aleksandr Nogikh, Theodore Ts'o, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
That's helpful, thank you.

--
Lee Jones [李琼斯]

Theodore Ts'o

unread,
Dec 16, 2022, 12:05:01 PM12/16/22
to Aleksandr Nogikh, Lee Jones, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
On Fri, Dec 16, 2022 at 03:09:04PM +0100, Aleksandr Nogikh wrote:
>
> Syzbot is actually reacting here to this bug from the Android namespace:
>
> https://syzkaller.appspot.com/bug?id=5266d464285a03cee9dbfda7d2452a72c3c2ae7c

Thanks for the clarification; stupid question, though -- I see
"upstream" is listed on the dashboard link above. Assuming that
"usptream" is "Linus's tree", why was it still saying, "I can't find
this patch in any of my trees"? What about the upstream tree?

> > Although this does appear to be a Stable candidate, I do not see it
> > in any of the Stable branches yet. So I suspect the answer here is to
> > wait for the fix to filter down.

The reason why it's not hit any of the long-term stable trees is
because the patch doesn't apply cleanly, because there are
pre-requisite commits that were required. Here are the required
commits for 5.15:

https://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git ext4_for_5.15.83

% git log --reverse --oneline v5.15.83..
96d070a12a7c ext4: refactor ext4_free_blocks() to pull out ext4_mb_clear_bb()
[ Upstream commit 8ac3939db99f99667b8eb670cf4baf292896e72d ]
2fa7a1780ecd ext4: add ext4_sb_block_valid() refactored out of ext4_inode_block_valid()
[ Upstream commit 6bc6c2bdf1baca6522b8d9ba976257d722423085 ]
8dc76aa246b1 ext4: add strict range checks while freeing blocks
[ Upstream commit a00b482b82fb098956a5bed22bd7873e56f152f1 ]
deb2e1554497 ext4: block range must be validated before use in ext4_mb_clear_bb()
[ Upstream commit 1e1c2b86ef86a8477fd9b9a4f48a6bfe235606f6 ]

Further backports to LTS kernels for 5.10, 5.4, etc., are left as an
exercise to the reader. :-)

- Ted

P.S. I have not tried to run gce-xfstests regressions yet. so the
only QA done on these backports is "it builds, ship it!" (And it
fixes the syzbot reproducers.) Then again, we're not running this
kind of regression tests on the LTS kernels.

P.P.S. If anyone is willing to volunteer to be an ext4 backports
maintainer, please contact me. The job description is (a) dealing
with the stable backport failures and addressing the patch conflicts,
potentially by dragging in patch prerequisites, and (b) running
"gce-xfstests ltm -c ext4/all -g auto" and making sure there are no
regressions.

- Ted

Aleksandr Nogikh

unread,
Dec 16, 2022, 12:15:03 PM12/16/22
to Theodore Ts'o, Lee Jones, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
On Fri, Dec 16, 2022 at 6:05 PM Theodore Ts'o <ty...@mit.edu> wrote:
>
> On Fri, Dec 16, 2022 at 03:09:04PM +0100, Aleksandr Nogikh wrote:
> >
> > Syzbot is actually reacting here to this bug from the Android namespace:
> >
> > https://syzkaller.appspot.com/bug?id=5266d464285a03cee9dbfda7d2452a72c3c2ae7c
>
> Thanks for the clarification; stupid question, though -- I see
> "upstream" is listed on the dashboard link above. Assuming that
> "usptream" is "Linus's tree", why was it still saying, "I can't find
> this patch in any of my trees"? What about the upstream tree?

Bugs from different namespaces are treated independently, so in this
particular case syzbot was expecting the fixing commit to reach the
Android trees that it fuzzes.

--
Aleksandr

Theodore Ts'o

unread,
Dec 16, 2022, 1:45:55 PM12/16/22
to Aleksandr Nogikh, Lee Jones, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
On Fri, Dec 16, 2022 at 06:14:50PM +0100, Aleksandr Nogikh wrote:
> > Thanks for the clarification; stupid question, though -- I see
> > "upstream" is listed on the dashboard link above. Assuming that
> > "usptream" is "Linus's tree", why was it still saying, "I can't find
> > this patch in any of my trees"? What about the upstream tree?
>
> Bugs from different namespaces are treated independently, so in this
> particular case syzbot was expecting the fixing commit to reach the
> Android trees that it fuzzes.

Is there a way someone can look at the dashboard link to determine
which (a) what namespace a particular syzkaller report is in, and (b)
what trees are included in a particular namespace?

Adding a link to the e-mail to the dashboard page may not help if it's
not obvious why the dashboard mentions "upstream" and yet it's not in
"any of the trees". Maybe the e-mail should explicitly list the trees
that syzkaller will be searching?

And it would seem that it would be a *feature* if looking at a syzbot
dashboard from Android namespace could expose the fact that particular
patch is in any of the LTS trees or Linus's upstream tree, no?

Also, what is the reason for Android for being in a separate
namespace? Is it running on a separate syzbot VM? I can understand
why from a feature perspective, that Fuschia and OpenBSD should be in
separate namespaces; but what are the reasons that there are separate
namespaces for Android versus the upstream kernel? Especially since
the Android dashboard is apparently referencing the upstream kernel?
What's up with that?

Put another way, while I think it's super useful to have a link to
Syzbot dashboard page, in the e-mail, I'm not sure it's going to be a
complete solution to the confusion that was inspired by this case.

That being said, in general I think a link to the Dashboard is useful;
in fact, it might be nice if we could encourage upstream developers
put in the commit trailer:

Link: https://syzkaller.appspot.com/bug?id=5266d464285a03cee9dbfda7d2452a72c3c2ae7c

in addition to, or better yet, instead of:

Reported-by: syzbot+15cd99...@syzkaller.appspotmail.com

... and have Syzbot be able to translate from the Link: tag as being
equivalent to the Reported-by: link. That's becase the Link is going
to be much more useful to humans than the Reported-by --- we've had a
number of cases where as part of the patch review, we really wanted to
get back to the Dashboard page, and it's not easy to get to the
Dashboard from the Reported-by tag.

Thanks,

- Ted

Aleksandr Nogikh

unread,
Dec 19, 2022, 11:12:59 AM12/19/22
to Theodore Ts'o, Lee Jones, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
Hi Ted,

Thanks for the comments!

On Fri, Dec 16, 2022 at 7:45 PM Theodore Ts'o <ty...@mit.edu> wrote:
>
> On Fri, Dec 16, 2022 at 06:14:50PM +0100, Aleksandr Nogikh wrote:
> > > Thanks for the clarification; stupid question, though -- I see
> > > "upstream" is listed on the dashboard link above. Assuming that
> > > "usptream" is "Linus's tree", why was it still saying, "I can't find
> > > this patch in any of my trees"? What about the upstream tree?
> >
> > Bugs from different namespaces are treated independently, so in this
> > particular case syzbot was expecting the fixing commit to reach the
> > Android trees that it fuzzes.
>
> Is there a way someone can look at the dashboard link to determine
> which (a) what namespace a particular syzkaller report is in, and (b)
> what trees are included in a particular namespace?

(a) Once you have opened the bug report page, you can find the
namespace at the top of the page.
(b) One can at least see the list of the tested trees on the main page
of the namespace -- we do share the latest commits for each manager
instance. Also see the comment below.

>
> Adding a link to the e-mail to the dashboard page may not help if it's
> not obvious why the dashboard mentions "upstream" and yet it's not in
> "any of the trees". Maybe the e-mail should explicitly list the trees
> that syzkaller will be searching?

I've sent a PR[1] that makes the bot send the list of the searched
trees. For upstream, we search quite a lot of trees, so the bot will
share some of them in the email and give a link to see the rest.
Otherwise the contents would be totally unintelligible.

[1] https://github.com/google/syzkaller/pull/3593

>
> And it would seem that it would be a *feature* if looking at a syzbot
> dashboard from Android namespace could expose the fact that particular
> patch is in any of the LTS trees or Linus's upstream tree, no?

Yes, that would be definitely nice.
We do have the improvements to the missing commit detection on our
TODO list, but I cannot say at the moment when exactly it will be
done.

>
> Also, what is the reason for Android for being in a separate
> namespace? Is it running on a separate syzbot VM? I can understand
> why from a feature perspective, that Fuschia and OpenBSD should be in
> separate namespaces; but what are the reasons that there are separate
> namespaces for Android versus the upstream kernel? Especially since
> the Android dashboard is apparently referencing the upstream kernel?
> What's up with that?

It's based on Linux, but it's not exactly Linux and can have its own bugs.

>
> Put another way, while I think it's super useful to have a link to
> Syzbot dashboard page, in the e-mail, I'm not sure it's going to be a
> complete solution to the confusion that was inspired by this case.
>
> That being said, in general I think a link to the Dashboard is useful;
> in fact, it might be nice if we could encourage upstream developers
> put in the commit trailer:
>
> Link: https://syzkaller.appspot.com/bug?id=5266d464285a03cee9dbfda7d2452a72c3c2ae7c
>
> in addition to, or better yet, instead of:
>
> Reported-by: syzbot+15cd99...@syzkaller.appspotmail.com
>
> ... and have Syzbot be able to translate from the Link: tag as being
> equivalent to the Reported-by: link. That's becase the Link is going
> to be much more useful to humans than the Reported-by --- we've had a
> number of cases where as part of the patch review, we really wanted to
> get back to the Dashboard page, and it's not easy to get to the
> Dashboard from the Reported-by tag.

FWIW it's quite easy to get the Dashboard link from the Reported-by
tag (although I agree it's not the most intuitive thing imaginable) --
one just needs to substitute the hash code after the + sign to
https://syzkaller.appspot.com/bug?extid=%s

Re. the Link tag.. it's interesting. It doesn't seem to be very
reasonable to include both, as it would look somewhat excessive:

Reported-by: syzbot+abcd...@syzkaller.appspotmail.com
Link: https://syzkaller.appspot.com/bug?extid=abcdef012345678

I'll take a look into the pros and cons of using just the Link tag.

--
Aleksandr

>
> Thanks,
>
> - Ted
>

Theodore Ts'o

unread,
Dec 19, 2022, 1:15:09 PM12/19/22
to Aleksandr Nogikh, Lee Jones, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
On Mon, Dec 19, 2022 at 05:12:47PM +0100, Aleksandr Nogikh wrote:
> (a) Once you have opened the bug report page, you can find the
> namespace at the top of the page.
> (b) One can at least see the list of the tested trees on the main page
> of the namespace -- we do share the latest commits for each manager
> instance. Also see the comment below.

It's not obvious what you mean by the "main page" of the namespace.
I'm guessing, but from the bug report page, there is a horizontal set
of icons, "Open", "Fixed", "Invalid" .... (which all have the same
icons), that the "Open" icon is the one that gets to the main page?

Assuming that this[1] is what was meant by "main page" (which is also
implied by the URL, but it's otherwise **really** not obvious), where
is the list of tested trees?

[1] https://syzkaller.appspot.com/android-5-10

I see the table "Instances", but it looks like the only two instances,
ci2-android-5-10 and ci2-android-5-10-perf, are both apparently
looking at the same commit --- but there's nothing that tells you what
kernel tree those commits are from. I can't see **anything* that
looks like an explicit git repo URL plus branch name. Is it perhaps
one of these?

https://android.googlesource.com/kernel/common/+/refs/heads/android12-5.10
https://android.googlesource.com/kernel/common/+/refs/heads/android13-5.10

It also appears that the android-5-10 "namespace" doesn't track any
other trees other than the Android 5.10 tree. Which means the e-mail
message, "I can't find the commit on any tested tree" is ***super***
misleading. At least for the android-5-10 namespace, why not just
say, "I don't see that commit on the git branch <explicit git repo and
branch name>"?

I did finally find the missing information, but it required a lot of
clicking and searching. From the bug page [2], the status line is:

Status: upstream: reported C repro on 2022/11/27 00:51

Has a link to the e-mail sent to the upstream developers[3]. And in
*that* e-mail, we can find the git tree involved:

git tree: android12-5.10-lts

[3] https://groups.google.com/g/syzkaller-android-bugs/c/LmaUwJpTXkA/m/HjsARFKWCQAJ


Going back to your pull request[4] to add a link to the dashboard in
the e-mail, how about also adding to the e-mail an indication about
the Syzkaller namespace. That way, the upstream developer can quickly
determine that the namespace is "Android-X.Y" and simply hit the 'd'
key as not really relevant to the upstream developer.

[4] https://github.com/google/syzkaller/pull/3591

I assume that there's someone in the Android kernel ecosystem that is
responsible for figuring out how to make sure commits are backported
from upstream into the LTS kernels, and the LTS kernels to the
relevant Android branch.

I do know one thing for certain --- I don't scale to the point where
this can made my problem. So I want to be able to more quickly triage
e-mails that are Not My Problem.

More generally, I think we need some kind of MAINTAINERS-like file
which explicitly lists who does have that responsibility, and which
can be used by Syzkaller so we aren't just blindly spamming all of the
upstream developers in the hopes that one of them will do somebody
else's job just to shut up the nag mail. Otherwise, the natural
reaction will be to resort to a mail filter to /dev/null the nag mail.
:-/

- Ted

Aleksandr Nogikh

unread,
Dec 21, 2022, 6:34:35 AM12/21/22
to Theodore Ts'o, Lee Jones, syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, linux...@vger.kernel.org, linux-...@vger.kernel.org, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org
On Mon, Dec 19, 2022 at 7:15 PM Theodore Ts'o <ty...@mit.edu> wrote:
>
< ... >
> It's not obvious what you mean by the "main page" of the namespace.
> I'm guessing, but from the bug report page, there is a horizontal set
> of icons, "Open", "Fixed", "Invalid" .... (which all have the same
> icons), that the "Open" icon is the one that gets to the main page?

Yes.

>
> Assuming that this[1] is what was meant by "main page" (which is also
> implied by the URL, but it's otherwise **really** not obvious), where
> is the list of tested trees?
>
> [1] https://syzkaller.appspot.com/android-5-10

I've just deployed the changes that add the page with the list of the
tested repositories as well as a link to it from the "main page" (look
above the table with instances). For Android, we indeed don't test
many trees:

https://syzkaller.appspot.com/android-5-10/repos

For Linux upstream, there are more trees:

https://syzkaller.appspot.com/upstream/repos

> At least for the android-5-10 namespace, why not just
> say, "I don't see that commit on the git branch <explicit git repo and
> branch name>"?

Now such email messages will include the kernel name and some of the
tested repositories (with the link to the full list page). E.g.
https://github.com/google/syzkaller/blob/4067838e6b16173af08e062ce434ecfc46d45bda/dashboard/app/notifications_test.go#L106

--
Aleksandr

syzbot

unread,
Jan 4, 2023, 6:34:38 AM1/4/23
to adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, l...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, nog...@google.com, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, ty...@mit.edu
This bug is marked as fixed by commit:
ext4: block range must be validated before use in ext4_mb_clear_bb()

But I can't find it in the tested trees[1] for more than 90 days.
Is it a correct commit? Please update it by replying:

#syz fix: exact-commit-title

Until then the bug is still considered open and new crashes with
the same signature are ignored.

Kernel: Android 5.10
Dashboard link: https://syzkaller.appspot.com/bug?extid=15cd994e273307bf5cfa

---
[1] I expect the commit to be present in:

1. android12-5.10-lts branch of
https://android.googlesource.com/kernel/common

Lee Jones

unread,
Jan 4, 2023, 7:44:39 AM1/4/23
to syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, lcze...@redhat.com, l...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, nog...@google.com, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, ty...@mit.edu, Tudor Ambarus
Tudor (Cc:ed) is now tracking these.

--
You received this message because you are subscribed to the Google Groups "syzkaller-android-bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-android...@googlegroups.com.


--

Google Logo
Lee Jones
Software Engineer
jone...@google.com
+44 (0) 2078814435

syzbot

unread,
Jan 18, 2023, 7:45:31 AM1/18/23
to adilger...@dilger.ca, gre...@linuxfoundation.org, jone...@google.com, lcze...@redhat.com, l...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, nog...@google.com, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, tudor....@linaro.org, ty...@mit.edu

syzbot

unread,
Feb 1, 2023, 7:45:41 AM2/1/23
to adilger...@dilger.ca, gre...@linuxfoundation.org, jone...@google.com, lcze...@redhat.com, l...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, nog...@google.com, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, tudor....@linaro.org, ty...@mit.edu

syzbot

unread,
Feb 15, 2023, 7:45:52 AM2/15/23
to adilger...@dilger.ca, gre...@linuxfoundation.org, jone...@google.com, lcze...@redhat.com, l...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, nog...@google.com, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, tudor....@linaro.org, ty...@mit.edu

syzbot

unread,
Mar 1, 2023, 7:46:41 AM3/1/23
to adilger...@dilger.ca, gre...@linuxfoundation.org, jone...@google.com, lcze...@redhat.com, l...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, nog...@google.com, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, tudor....@linaro.org, ty...@mit.edu

Tudor Ambarus

unread,
Mar 1, 2023, 8:51:48 AM3/1/23
to syzbot, adilger...@dilger.ca, gre...@linuxfoundation.org, jone...@google.com, lcze...@redhat.com, l...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, nog...@google.com, sas...@kernel.org, sta...@vger.kernel.org, syzkaller-a...@googlegroups.com, tadeus...@linaro.org, ty...@mit.edu


On 3/1/23 12:46, syzbot wrote:
> This bug is marked as fixed by commit:
> ext4: block range must be validated before use in ext4_mb_clear_bb()
>
> But I can't find it in the tested trees[1] for more than 90 days.

Indeed it seems this patch was never backported to the stable tree.
I'm handling it.

Cheers,
ta
Reply all
Reply to author
Forward
0 new messages