WARNING in dma_map_page_attrs

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit: c4d6fe73 Merge tag 'xarray-5.9' of git://git.infradead.org..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=14862ff0500000
kernel config: https://syzkaller.appspot.com/x/.config?x=7d790573d3e379c4
dashboard link: https://syzkaller.appspot.com/bug?extid=34dc2fea3478e659af01
compiler: gcc (GCC) 10.1.0-syz 20200507

Unfortunately, I don't have any reproducer for this issue yet.

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+34dc2f...@syzkaller.appspotmail.com

infiniband syz1: set active
infiniband syz1: added vcan0
------------[ cut here ]------------
WARNING: CPU: 1 PID: 9851 at kernel/dma/mapping.c:149 dma_map_page_attrs+0x493/0x700 kernel/dma/mapping.c:149
Modules linked in:
CPU: 1 PID: 9851 Comm: syz-executor.1 Not tainted 5.9.0-syzkaller #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.12.0-59-gc9ba5276e321-prebuilt.qemu.org 04/01/2014
RIP: 0010:dma_map_page_attrs+0x493/0x700 kernel/dma/mapping.c:149
Code: 80 3c 10 00 0f 85 ed 01 00 00 48 8b 1d 36 c3 fa 0c e9 2d fc ff ff 48 89 c3 e9 d1 fd ff ff e8 04 12 12 00 0f 0b e8 fd 11 12 00 <0f> 0b 49 c7 c4 ff ff ff ff e9 d5 fd ff ff e8 ea 11 12 00 48 8d 7b
RSP: 0018:ffffc90001546c68 EFLAGS: 00010246
RAX: 0000000000040000 RBX: ffffffff894d0040 RCX: ffffc9000dbe4000
RDX: 0000000000040000 RSI: ffffffff815d3b03 RDI: ffff88806a988b00
RBP: ffff8880236cc400 R08: 0000000000000002 R09: 0000000000000000
R10: 0000000000000002 R11: 0000000000000000 R12: ffffea00008db300
R13: ffff88806a9886e8 R14: 00000000000004b8 R15: 0000000000000002
FS: 00007f678fae2700(0000) GS:ffff88802ce00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f299a39b190 CR3: 0000000069f31000 CR4: 0000000000350ee0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
dma_map_single_attrs include/linux/dma-mapping.h:279 [inline]
ib_dma_map_single include/rdma/ib_verbs.h:3967 [inline]
ib_mad_post_receive_mads+0x23f/0xd60 drivers/infiniband/core/mad.c:2715
ib_mad_port_start drivers/infiniband/core/mad.c:2862 [inline]
ib_mad_port_open drivers/infiniband/core/mad.c:3016 [inline]
ib_mad_init_device+0x72b/0x1400 drivers/infiniband/core/mad.c:3092
add_client_context+0x405/0x5e0 drivers/infiniband/core/device.c:680
enable_device_and_get+0x1d5/0x3c0 drivers/infiniband/core/device.c:1301
ib_register_device drivers/infiniband/core/device.c:1376 [inline]
ib_register_device+0x7a7/0xa40 drivers/infiniband/core/device.c:1335
rxe_register_device+0x46d/0x570 drivers/infiniband/sw/rxe/rxe_verbs.c:1182
rxe_add+0x12fe/0x16d0 drivers/infiniband/sw/rxe/rxe.c:247
rxe_net_add+0x8c/0xe0 drivers/infiniband/sw/rxe/rxe_net.c:507
rxe_newlink drivers/infiniband/sw/rxe/rxe.c:269 [inline]
rxe_newlink+0xb7/0xe0 drivers/infiniband/sw/rxe/rxe.c:250
nldev_newlink+0x30e/0x540 drivers/infiniband/core/nldev.c:1555
rdma_nl_rcv_msg+0x367/0x690 drivers/infiniband/core/netlink.c:195
rdma_nl_rcv_skb drivers/infiniband/core/netlink.c:239 [inline]
rdma_nl_rcv+0x2f2/0x440 drivers/infiniband/core/netlink.c:259
netlink_unicast_kernel net/netlink/af_netlink.c:1304 [inline]
netlink_unicast+0x533/0x7d0 net/netlink/af_netlink.c:1330
netlink_sendmsg+0x856/0xd90 net/netlink/af_netlink.c:1919
sock_sendmsg_nosec net/socket.c:651 [inline]
sock_sendmsg+0xcf/0x120 net/socket.c:671
____sys_sendmsg+0x6e8/0x810 net/socket.c:2353
___sys_sendmsg+0xf3/0x170 net/socket.c:2407
__sys_sendmsg+0xe5/0x1b0 net/socket.c:2440
do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x45d9f9
Code: bd b1 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 8b b1 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007f678fae1c88 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 000000000071f480 RCX: 000000000045d9f9
RDX: 0000000000000000 RSI: 0000000020000200 RDI: 0000000000000003
RBP: 00000000004aab13 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000075bf00
R13: 00007ffc6f9b8bbf R14: 00007f678fac2000 R15: 0000000000000003


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzk...@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

[Christoph Hellwig]

I don't think the merge commit makes sense here. But what we see here
is that dma_map_page is called on the rxe device, without that device
having a DMA mask. For now this needs a workaround in rxe, but for
5.11 I'll send a patch to remove dma-virt and just handle this case
inside of the rdma core.

---end quoted text---

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit: 3cb12d27 Merge tag 'net-5.10-rc1' of git://git.kernel.org/..
git tree: net
console output: https://syzkaller.appspot.com/x/log.txt?x=13125390500000
kernel config: https://syzkaller.appspot.com/x/.config?x=46c6fea3eb827ae1

dashboard link: https://syzkaller.appspot.com/bug?extid=34dc2fea3478e659af01
compiler: gcc (GCC) 10.1.0-syz 20200507

syz repro: https://syzkaller.appspot.com/x/repro.syz?x=16858664500000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=11f402ef900000

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+34dc2f...@syzkaller.appspotmail.com

netdevsim netdevsim0 netdevsim1: set [1, 0] type 2 family 0 port 6081 - 0
netdevsim netdevsim0 netdevsim2: set [1, 0] type 2 family 0 port 6081 - 0
netdevsim netdevsim0 netdevsim3: set [1, 0] type 2 family 0 port 6081 - 0
infiniband syz2: set active
infiniband syz2: added macvlan0
------------[ cut here ]------------
WARNING: CPU: 1 PID: 8488 at kernel/dma/mapping.c:149 dma_map_page_attrs+0x493/0x700 kernel/dma/mapping.c:149
Modules linked in:
CPU: 1 PID: 8488 Comm: syz-executor144 Not tainted 5.9.0-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011

RIP: 0010:dma_map_page_attrs+0x493/0x700 kernel/dma/mapping.c:149

Code: 80 3c 10 00 0f 85 ed 01 00 00 48 8b 1d 86 38 e9 0c e9 2d fc ff ff 48 89 c3 e9 d1 fd ff ff e8 04 11 12 00 0f 0b e8 fd 10 12 00 <0f> 0b 49 c7 c4 ff ff ff ff e9 d5 fd ff ff e8 ea 10 12 00 48 8d 7b
RSP: 0018:ffffc90000fdec68 EFLAGS: 00010293
RAX: 0000000000000000 RBX: ffffffff894d1060 RCX: ffffffff815df1e3
RDX: ffff8880208c1a40 RSI: ffffffff815df5b3 RDI: ffff8880196f8b00
RBP: ffff88801412d800 R08: 0000000000000002 R09: 0000000000000000
R10: 0000000000000002 R11: 0000000000000000 R12: ffffea0000504b40
R13: ffff8880196f86e8 R14: 00000000000008b8 R15: 0000000000000002
FS: 0000000001b26880(0000) GS:ffff8880b9f00000(0000) knlGS:0000000000000000

CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033

CR2: 00000000200002c0 CR3: 0000000022446000 CR4: 00000000001506e0

RIP: 0033:0x443699
Code: 18 89 d0 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db 0d fc ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007ffc067db418 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 0000000000443699
RDX: 0000000000000000 RSI: 00000000200002c0 RDI: 0000000000000003
RBP: 00007ffc067db420 R08: 0000000001bbbbbb R09: 0000000001bbbbbb
R10: 0000000000000000 R11: 0000000000000246 R12: 00007ffc067db430
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000

[Jakub Kicinski]

CC: rdma, looks like rdma from the stack trace

[Parav Pandit]

Hi Christoph,

> From: Jakub Kicinski <ku...@kernel.org>
> Sent: Saturday, October 24, 2020 11:45 PM
>
> CC: rdma, looks like rdma from the stack trace
>
> On Fri, 23 Oct 2020 20:07:17 -0700 syzbot wrote:
> > syzbot has found a reproducer for the following issue on:
> >
> > HEAD commit: 3cb12d27 Merge tag 'net-5.10-rc1' of git://git.kernel.org/..

In [1] you mentioned that dma_mask should not be set for dma_virt_ops.
So patch [2] removed it.

But check to validate the dma mask for all dma_ops was added in [3].

What is the right way? Did I misunderstood your comment about dma_mask in [1]?

[1] https://www.spinics.net/lists/linux-rdma/msg96374.html
[2] e0477b34d9d ("RDMA: Explicitly pass in the dma_device to ib_register_device")
[3] f959dcd6ddfd ("dma-direct: Fix potential NULL pointer dereference")

> > WARNING: CPU: 1 PID: 8488 at kernel/dma/mapping.c:149
> > dma_map_page_attrs+0x493/0x700 kernel/dma/mapping.c:149 Modules
> linked in:

[h...@lst.de]

On Mon, Oct 26, 2020 at 05:23:48AM +0000, Parav Pandit wrote:
> Hi Christoph,
>
> > From: Jakub Kicinski <ku...@kernel.org>
> > Sent: Saturday, October 24, 2020 11:45 PM
> >
> > CC: rdma, looks like rdma from the stack trace
> >
> > On Fri, 23 Oct 2020 20:07:17 -0700 syzbot wrote:
> > > syzbot has found a reproducer for the following issue on:
> > >
> > > HEAD commit: 3cb12d27 Merge tag 'net-5.10-rc1' of git://git.kernel.org/..
>
> In [1] you mentioned that dma_mask should not be set for dma_virt_ops.
> So patch [2] removed it.
>
> But check to validate the dma mask for all dma_ops was added in [3].
>
> What is the right way? Did I misunderstood your comment about dma_mask in [1]?

No, I did not say we don't need the mask. I said copying over the
various dma-related fields from the parent is bogus.

I think rxe (and ther other drivers/infiniband/sw drivers) need a simple
dma_coerce_mask_and_coherent and nothing else.

[Parav Pandit]

> From: h...@lst.de <h...@lst.de>
> Sent: Tuesday, October 27, 2020 1:41 PM

I see. Does below fix make sense?
Is DMA_MASK_NONE correct?

From cfad78c35788b4ff604abedd96559500c5fd2a72 Mon Sep 17 00:00:00 2001
From: Parav Pandit <pa...@nvidia.com>
Date: Tue, 27 Oct 2020 14:20:07 +0200
Subject: [PATCH] RDMA: Fix software RDMA drivers for dma mapping error

A cited commit in fixes tag avoided setting dma_mask of the ib_device.
Commit [1] made dma_mask as mandetory field to be setup even for
dma_virt_ops based dma devices.

Fix it by setting empty DMA MASK for software based RDMA devices.

[1] commit: f959dcd6ddfd2 ("dma-direct: Fix potential NULL pointer dereference")

Reported-by: syzbot+34dc2f...@syzkaller.appspotmail.com
Fixes: e0477b34d9d1 ("RDMA: Explicitly pass in the dma_device to ib_register_device")
Signed-off-by: Parav Pandit <pa...@nvidia.com>
---
drivers/infiniband/sw/rdmavt/vt.c | 5 +++--
drivers/infiniband/sw/rxe/rxe_verbs.c | 4 +++-
drivers/infiniband/sw/siw/siw_main.c | 5 +++--
3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/vt.c b/drivers/infiniband/sw/rdmavt/vt.c
index 52218684ad4a..1b456f4d4fcf 100644
--- a/drivers/infiniband/sw/rdmavt/vt.c
+++ b/drivers/infiniband/sw/rdmavt/vt.c
@@ -580,8 +580,9 @@ int rvt_register_device(struct rvt_dev_info *rdi)

/* DMA Operations */
rdi->ibdev.dev.dma_parms = rdi->ibdev.dev.parent->dma_parms;
- dma_set_coherent_mask(&rdi->ibdev.dev,
- rdi->ibdev.dev.parent->coherent_dma_mask);
+ ret = dma_coerce_mask_and_coherent(&rdi->ibdev.dev, DMA_MASK_NONE);
+ if (ret)
+ goto bail_wss;

/* Protection Domain */
spin_lock_init(&rdi->n_pds_lock);
diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c
index 1fc022362fbe..357787688293 100644
--- a/drivers/infiniband/sw/rxe/rxe_verbs.c
+++ b/drivers/infiniband/sw/rxe/rxe_verbs.c
@@ -1130,7 +1130,9 @@ int rxe_register_device(struct rxe_dev *rxe, const char *ibdev_name)
rxe->ndev->dev_addr);
dev->dev.dma_parms = &rxe->dma_parms;
dma_set_max_seg_size(&dev->dev, UINT_MAX);
- dma_set_coherent_mask(&dev->dev, dma_get_required_mask(&dev->dev));
+ err = dma_coerce_mask_and_coherent(&dev->dev, DMA_MASK_NONE);
+ if (err)
+ return err;

dev->uverbs_cmd_mask = BIT_ULL(IB_USER_VERBS_CMD_GET_CONTEXT)
| BIT_ULL(IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL)
diff --git a/drivers/infiniband/sw/siw/siw_main.c b/drivers/infiniband/sw/siw/siw_main.c
index ca8bc7296867..d3dc50a42dab 100644
--- a/drivers/infiniband/sw/siw/siw_main.c
+++ b/drivers/infiniband/sw/siw/siw_main.c
@@ -384,8 +384,9 @@ static struct siw_device *siw_device_create(struct net_device *netdev)
base_dev->dev.parent = parent;
base_dev->dev.dma_parms = &sdev->dma_parms;
dma_set_max_seg_size(&base_dev->dev, UINT_MAX);
- dma_set_coherent_mask(&base_dev->dev,
- dma_get_required_mask(&base_dev->dev));
+ if (dma_coerce_mask_and_coherent(&base_dev->dev, DMA_MASK_NONE))
+ goto error;
+
base_dev->num_comp_vectors = num_possible_cpus();

xa_init_flags(&sdev->qp_xa, XA_FLAGS_ALLOC1);
--
2.26.2

[h...@lst.de]

On Tue, Oct 27, 2020 at 12:52:30PM +0000, Parav Pandit wrote:
>
> > From: h...@lst.de <h...@lst.de>
> > Sent: Tuesday, October 27, 2020 1:41 PM
> >
> > On Mon, Oct 26, 2020 at 05:23:48AM +0000, Parav Pandit wrote:
> > > Hi Christoph,
> > >
> > > > From: Jakub Kicinski <ku...@kernel.org>
> > > > Sent: Saturday, October 24, 2020 11:45 PM
> > > >
> > > > CC: rdma, looks like rdma from the stack trace
> > > >
> > > > On Fri, 23 Oct 2020 20:07:17 -0700 syzbot wrote:
> > > > > syzbot has found a reproducer for the following issue on:
> > > > >
> > > > > HEAD commit: 3cb12d27 Merge tag 'net-5.10-rc1' of
> > git://git.kernel.org/..
> > >
> > > In [1] you mentioned that dma_mask should not be set for dma_virt_ops.
> > > So patch [2] removed it.
> > >
> > > But check to validate the dma mask for all dma_ops was added in [3].
> > >
> > > What is the right way? Did I misunderstood your comment about
> > dma_mask in [1]?
> >
> > No, I did not say we don't need the mask. I said copying over the various
> > dma-related fields from the parent is bogus.
> >
> > I think rxe (and ther other drivers/infiniband/sw drivers) need a simple
> > dma_coerce_mask_and_coherent and nothing else.
>
> I see. Does below fix make sense?
> Is DMA_MASK_NONE correct?

DMA_MASK_NONE is gone in 5.10. I think you want DMA_BIT_MASK(64).
That isn't actually correct for 32-bit platforms, but good enough.

[Parav Pandit]

Ok. thanks for the input.
Sending updated fix to set 64-bit mask for 64-bit platform and 32-bit mask otherwise.