Centos 5.4 bnx2i iscsi woes

Tarun Reddy

unread,

Apr 16, 2010, 4:15:49 PM4/16/10

to open-iscsi

I have a bare stock Centos 5.4. (I've also tried updating to the iscsi-
initiator-utils and kernel from 5.5) and I'm trying to get the bnx2i
driver to work using iSCSI offload. Normal iSCSI over TCP works great,
but I'd like to see if I can get better IOPS using the iSCSI offload.

Here are my steps that I've tried and the associated errors. Please
help if possible... I still haven't heard anyone on this list getting
it working.

1) Dell R610 with PCIe Dual port Broadcom card with iSCSI offload
support enabled. eth6 has a hardware address of 00:10:18:48:f2:e8.
Inside /var/lib/iscsi/ifaces, there is a file created called bnx2i.
00:10:18:48:f2:e9. The content of that file is:

# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = bnx2i.00:10:18:48:f2:e9
iface.net_ifacename = eth6
iface.hwaddress = 00:10:18:48:f2:e9
iface.ipaddress = 10.32.13.72
iface.transport_name = bnx2i
# END RECORD

eth6 is set to 10.32.13.70

2) Using a HP Lefthand target at 10.32.13.33, I initiate this command:
iscsiadm -m discovery -t st -p 10.32.13.33 -I bnx2i.00\:10\:18\:
48\:f2\:e9

That returns:
10.32.13.33:3260,1 iqn.
2003-10.com.lefthandnetworks:rallytestbedstorage01:14435:dsc-
loadtest-4
10.32.13.33:3260,1 iqn.
2003-10.com.lefthandnetworks:rallytestbedstorage01:14430:dsc-
loadtest-2
10.32.13.33:3260,1 iqn.
2003-10.com.lefthandnetworks:rallytestbedstorage01:14432:dsc-
loadtest-3
10.32.13.33:3260,1 iqn.
2003-10.com.lefthandnetworks:rallytestbedstorage01:14428:dsc-
loadtest-1

Cool. I am able to discover my volumes on the target portal

3) iscsiadm -m node -l

Logging in to [iface: bnx2i.00:10:18:48:f2:e9, target: iqn.
2003-10.com.lefthandnetworks:rallytestbedstorage01:14435:dsc-
loadtest-4, portal: 10.32.13.33,3260]
Logging in to [iface: bnx2i.00:10:18:48:f2:e9, target: iqn.
2003-10.com.lefthandnetworks:rallytestbedstorage01:14430:dsc-
loadtest-2, portal: 10.32.13.33,3260]
Logging in to [iface: bnx2i.00:10:18:48:f2:e9, target: iqn.
2003-10.com.lefthandnetworks:rallytestbedstorage01:14432:dsc-
loadtest-3, portal: 10.32.13.33,3260]
Logging in to [iface: bnx2i.00:10:18:48:f2:e9, target: iqn.
2003-10.com.lefthandnetworks:rallytestbedstorage01:14428:dsc-
loadtest-1, portal: 10.32.13.33,3260]
iscsiadm: Could not login to [iface: bnx2i.00:10:18:48:f2:e9, target:
iqn.2003-10.com.lefthandnetworks:rallytestbedstorage01:14435:dsc-
loadtest-4, portal: 10.32.13.33,3260]:
iscsiadm: initiator reported error (4 - encountered connection
failure)
iscsiadm: Could not login to [iface: bnx2i.00:10:18:48:f2:e9, target:
iqn.2003-10.com.lefthandnetworks:rallytestbedstorage01:14430:dsc-
loadtest-2, portal: 10.32.13.33,3260]:
iscsiadm: initiator reported error (4 - encountered connection
failure)
iscsiadm: Could not login to [iface: bnx2i.00:10:18:48:f2:e9, target:
iqn.2003-10.com.lefthandnetworks:rallytestbedstorage01:14432:dsc-
loadtest-3, portal: 10.32.13.33,3260]:
iscsiadm: initiator reported error (4 - encountered connection
failure)
iscsiadm: Could not login to [iface: bnx2i.00:10:18:48:f2:e9, target:
iqn.2003-10.com.lefthandnetworks:rallytestbedstorage01:14428:dsc-
loadtest-1, portal: 10.32.13.33,3260]:
iscsiadm: initiator reported error (4 - encountered connection
failure)

Output from /var/log/messages:
Apr 16 14:12:03 dsc-loadtestdb-01 iscsid: Received iferror -1
Apr 16 14:12:03 dsc-loadtestdb-01 iscsid: cannot make a connection to
10.32.13.33:3260 (-1,11)
Apr 16 14:12:05 dsc-loadtestdb-01 iscsid: Received iferror -1
Apr 16 14:12:05 dsc-loadtestdb-01 iscsid: cannot make a connection to
10.32.13.33:3260 (-1,11)
Apr 16 14:12:07 dsc-loadtestdb-01 iscsid: Received iferror -1
Apr 16 14:12:07 dsc-loadtestdb-01 iscsid: cannot make a connection to
10.32.13.33:3260 (-1,11)
Apr 16 14:12:09 dsc-loadtestdb-01 iscsid: Received iferror -1
Apr 16 14:12:09 dsc-loadtestdb-01 iscsid: cannot make a connection to
10.32.13.33:3260 (-1,11)

And I can ping 10.32.13.72 (the iscsi offload nic ip address) for
whatever that is worth. Any ideas? Any help would be much, much
appreciated!!

Tarun

--
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To post to this group, send email to open-...@googlegroups.com.
To unsubscribe from this group, send email to open-iscsi+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/open-iscsi?hl=en.

Mike Christie

unread,

Apr 20, 2010, 1:10:05 AM4/20/10

to open-...@googlegroups.com, Tarun Reddy, Benjamin Li

On 04/16/2010 03:15 PM, Tarun Reddy wrote:
>
> Output from /var/log/messages:
> Apr 16 14:12:03 dsc-loadtestdb-01 iscsid: Received iferror -1
> Apr 16 14:12:03 dsc-loadtestdb-01 iscsid: cannot make a connection to
> 10.32.13.33:3260 (-1,11)
> Apr 16 14:12:05 dsc-loadtestdb-01 iscsid: Received iferror -1
> Apr 16 14:12:05 dsc-loadtestdb-01 iscsid: cannot make a connection to
> 10.32.13.33:3260 (-1,11)
> Apr 16 14:12:07 dsc-loadtestdb-01 iscsid: Received iferror -1
> Apr 16 14:12:07 dsc-loadtestdb-01 iscsid: cannot make a connection to
> 10.32.13.33:3260 (-1,11)
> Apr 16 14:12:09 dsc-loadtestdb-01 iscsid: Received iferror -1
> Apr 16 14:12:09 dsc-loadtestdb-01 iscsid: cannot make a connection to
> 10.32.13.33:3260 (-1,11)
>

I have got the card to work, but I think I am getting lucky. I think you
might be hitting a race. Benjamin from Broadcom is working on a fix. I
ccd him here, so when he is done we can give you some rpms to try out.

Oliver Hookins

unread,

Apr 29, 2010, 5:44:53 AM4/29/10

to open-iscsi

I believe I'm hitting the exact same error... pity.

Any word on the fix?

Mike Christie

unread,

May 5, 2010, 11:42:40 AM5/5/10

to open-...@googlegroups.com, Oliver Hookins

On 04/29/2010 04:44 AM, Oliver Hookins wrote:
> I believe I'm hitting the exact same error... pity.
>
> Any word on the fix?
>

I believe they have a fix. It is being tested now.

Nate Salemme

unread,

May 5, 2010, 3:57:13 PM5/5/10

to open-iscsi

I am trying to do the same thing and am seeing very similar behavior.
I have a brand new HP DL 360 G6 with Broadcom HP nc382i dual-port 1GB
NIC. Software iSCSI works fine but I am cannot get hardware iSCSI to
work. I defined an iface0-HW as seen below:

[root@snd-testg6a ifaces]# cat iface0-HW
# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = iface0-HW
iface.ipaddress = 10.30.40.80
iface.hwaddress = f4:ce:46:B9:49:2d

iface.transport_name = bnx2i
# END RECORD

When I try and login, I see the following:

Setting up iSCSI targets: Logging in to [iface: iface0-HW, target: iqn.
1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.78,3260]
Logging in to [iface: iface0-HW, target: iqn.
1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.79,3260]
Logging in to [iface: iface0-HW, target: iqn.
1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.76,3260]
Logging in to [iface: iface0-HW, target: iqn.
1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.77,3260]
iscsiadm: Could not login to [iface: iface0-HW, target: iqn.
1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.78,3260]:
iscsiadm: initiator reported error (12 - iSCSI driver not found.
Please make sure it is loaded, and retry the operation)
iscsiadm: Could not login to [iface: iface0-HW, target: iqn.
1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.79,3260]:
iscsiadm: initiator reported error (12 - iSCSI driver not found.
Please make sure it is loaded, and retry the operation)
iscsiadm: Could not login to [iface: iface0-HW, target: iqn.
1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.76,3260]:
iscsiadm: initiator reported error (12 - iSCSI driver not found.
Please make sure it is loaded, and retry the operation)
iscsiadm: Could not login to [iface: iface0-HW, target: iqn.
1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.77,3260]:
iscsiadm: initiator reported error (12 - iSCSI driver not found.
Please make sure it is loaded, and retry the operation)
iscsiadm: Could not log into all portals. Err 12.

I've installed all the latest drivers and firmware. lsmod shows the
modules loaded too:

[root@snd-testg6a ifaces]# lsmod |egrep -i "bnx|scsi"
bnx2i 102944 0
cnic 75288 1 bnx2i
iscsi_tcp 50509 4
libiscsi_tcp 53317 2 cxgb3i,iscsi_tcp
libiscsi2 77125 5
ib_iser,bnx2i,cxgb3i,iscsi_tcp,libiscsi_tcp
scsi_transport_iscsi2 74073 7
ib_iser,bnx2i,cxgb3i,iscsi_tcp,libiscsi2
scsi_transport_iscsi 35017 1 scsi_transport_iscsi2
scsi_dh 41665 1 dm_multipath
bnx2 219404 0
scsi_mod 196697 10
ib_iser,bnx2i,cxgb3i,sg,iscsi_tcp,libiscsi2,scsi_transport_iscsi2,scsi_dh,cciss,sd_mod

I am thinking this could be similiar behavior to what you are seeing?

Mike Christie

unread,

May 6, 2010, 12:54:21 PM5/6/10

to open-...@googlegroups.com, Nate Salemme

On 05/05/2010 02:57 PM, Nate Salemme wrote:
> I am trying to do the same thing and am seeing very similar behavior.

Your problem is completely different.

> I have a brand new HP DL 360 G6 with Broadcom HP nc382i dual-port 1GB
> NIC. Software iSCSI works fine but I am cannot get hardware iSCSI to
> work. I defined an iface0-HW as seen below:
>
> [root@snd-testg6a ifaces]# cat iface0-HW
> # BEGIN RECORD 2.0-871
> iface.iscsi_ifacename = iface0-HW
> iface.ipaddress = 10.30.40.80
> iface.hwaddress = f4:ce:46:B9:49:2d
> iface.transport_name = bnx2i
> # END RECORD
>
>
> When I try and login, I see the following:
>
> Setting up iSCSI targets: Logging in to [iface: iface0-HW, target: iqn.
> 1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.78,3260]
> Logging in to [iface: iface0-HW, target: iqn.
> 1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.79,3260]
> Logging in to [iface: iface0-HW, target: iqn.
> 1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.76,3260]
> Logging in to [iface: iface0-HW, target: iqn.
> 1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.77,3260]
> iscsiadm: Could not login to [iface: iface0-HW, target: iqn.
> 1986-03.com.hp:storage.p2000g3.0939da0280, portal: 10.30.40.78,3260]:
> iscsiadm: initiator reported error (12 - iSCSI driver not found.
> Please make sure it is loaded, and retry the operation)

We do not even get to the connection/login phase like the other bug
reporters did. In your case it is complaining that bnx2i is not loaded,
but from the info below it looks like it is.

Are you using RHEL or Centos? If so, how did you start the iscsi system
and what is the iscsi-initiator-utils version?

Did you do

service iscsi start

or was the iscsi service started for you?

Try starting it by hand:

// stop the iscsi service so we can start from scratch
# service iscsi stop

// make sure that the record has the bnx2i driver set
# iscsiadm -m node -I iface0-HW | grep bnx2i

// start iscsi by hand
# modprobe bnx2i
# iscsid
# brcm_iscsi_uio
// login by hand
# iscsiadm -m node -I iface iface0-HW -l

Nate Salemme

unread,

May 6, 2010, 1:38:21 PM5/6/10

to open-iscsi

I agree that I am getting a different error message. The reason I
responded to the thread was that it looked like others were having
trouble getting HW iSCSI working on Centos 5.4 using bnx2i.

> Are you using RHEL or Centos? If so, how did you start the iscsi system
> and what is the iscsi-initiator-utils version?

Using Centos 5.4. iscsi was started as a service (/etc/init.d/iscsi
restart.)
As for the version, I've tried both current upstream yum version and
the version provided on HP's website. The version I am running now is
the HP version.

[root@snd-testg6a ~]# rpm -qa |grep iscsi
iscsi-initiator-utils-6.2.0.871-0.12.el5_4.1

The upstream version is SLIGHTLY older.

iscsi-initiator-utils-6.2.0.871-0.10.el5

> Did you do
>
> service iscsi start
>
> or was the iscsi service started for you?
>
> Try starting it by hand:
>
> // stop the iscsi service so we can start from scratch
> # service iscsi stop
>
> // make sure that the record has the bnx2i driver set
> # iscsiadm -m node -I iface0-HW | grep bnx2i
>
> // start iscsi by hand
> # modprobe bnx2i
> # iscsid
> # brcm_iscsi_uio
> // login by hand
> # iscsiadm -m node -I iface iface0-HW -l
>

Yes I am using service. Running iscsiadm -m node -I iface0-HW | grep
bnx2i confirms that node connections are using the bnx2i transport. I
tried running the commands above to login manually. Still cannot find
the driver.

What versions of the driver are you using and which Broadcom NIC card
do you have?

Since you say your system is working, do you mind pasting the relevant
parts of dmesg when the iSCSI driver is loaded so I can compare my
dmesg? I've included mine below for reference.

Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.8c (Mar 31,
2010)
GSI 20 sharing vector 0x82 and IRQ 20
ACPI: PCI Interrupt 0000:02:00.0[A] -> GSI 31 (level, low) -> IRQ 130
PCI: Setting latency timer of device 0000:02:00.0 to 64
eth0: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found
at mem f8000000, IRQ 130, node addr f4:ce:46:b9:49:2c
GSI 21 sharing vector 0x8A and IRQ 21
ACPI: PCI Interrupt 0000:02:00.1[B] -> GSI 39 (level, low) -> IRQ 138
PCI: Setting latency timer of device 0000:02:00.1 to 64
eth1: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found
at mem f6000000, IRQ 138, node addr f4:ce:46:b9:49:2e
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
Floppy drive(s): fd0 is 1.44M
floppy0: no floppy controllers found
Floppy drive(s): fd0 is 1.44M
floppy0: no floppy controllers found
lp: driver loaded but no devices found
ACPI: Power Button (FF) [PWRF]
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
device-mapper: multipath: version 1.0.5 loaded
EXT3 FS on dm-0, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on cciss/c0d0p1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: initialized (dev cciss/c0d0p1, type ext3), uses xattr
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
Adding 200146936k swap on /dev/VolGroup01/LogVol01. Priority:-1
extents:1 across:200146936k
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses
genfs_contexts
IA-32 Microcode Update Driver: v1.14a <tig...@veritas.com>
Broadcom NetXtreme II CNIC Driver cnic v1.9.13b (Dec 16, 2009)
cnic: Added CNIC device: eth0
cnic: Added CNIC device: eth1
Loading iSCSI transport class v2.0-871.
Broadcom NetXtreme II iSCSI Driver bnx2i v1.8.12f (Jan 19, 2010)
802.1Q VLAN Support v1.8 Ben Greear <gre...@candelatech.com>
All bugs added by David S. Miller <da...@redhat.com>
cxgb3i: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.
iscsi: registered transport (cxgb3i)
iscsi: registered transport (tcp)
iscsi: registered transport (iser)
bnx2: eth0: using MSI
iscsi: registered transport (bcm570x-020000)
bnx2i: netif=eth0, iscsi=bcm570x-020000
bnx2i [02:00.00]: ISCSI_INIT passed
bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive &
transmit flow control ON
bnx2: eth0 NIC Copper Link is Down
bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex, receive &
transmit flow control ON
scsi0 : iSCSI Initiator over TCP/IP
scsi1 : iSCSI Initiator over TCP/IP
scsi2 : iSCSI Initiator over TCP/IP
scsi3 : iSCSI Initiator over TCP/IP
Vendor: HP Model: P200<5> Vendor: HP Model: P2000G3
FC/iSCSI Rev: T100

Thanks.

Mike Christie

unread,

May 6, 2010, 1:56:40 PM5/6/10

to open-...@googlegroups.com, Nate Salemme

On 05/06/2010 12:38 PM, Nate Salemme wrote:
> iscsi: registered transport (cxgb3i)
> iscsi: registered transport (tcp)
> iscsi: registered transport (iser)
> bnx2: eth0: using MSI
> iscsi: registered transport (bcm570x-020000)

For the bnx2i driver that comes with RHEL you should see:

May 6 11:10:11 localhost kernel: iscsi: registered transport (bnx2i)

The registered transport message you got above came from HP's driver.

I am not sure how to configure HP's tools and driver. They are slightly
different because HP's driver is older and based on a non-upstream
driver that registers a transport per pci function and the
RHEL/Centos/upstream driver registers a transport per driver.

So for RHEL you would use bnx2i for the iface.transport_name like you
did below:

# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = iface0-HW
iface.ipaddress = 10.30.40.80
iface.hwaddress = f4:ce:46:B9:49:2d
iface.transport_name = bnx2i
# END RECORD

But for HP's driver you probably want to set the iface.transport_name as
what you see in the registered transport message in your log,
bcm570x-020000. I am not 100% sure what/how to config HP's stuff because
I do not have the source in front of me. I am 99% sure though that just
using the bnx2i name string is wrong for that driver because of how it
is registering transport names.

Mike Christie

unread,

May 6, 2010, 1:59:16 PM5/6/10

to open-...@googlegroups.com, Nate Salemme

With HP's tools and driver I think you want:

iface.transport_name = bcm570x-020000

or you might actually do

node.transport_name = bcm570x-020000

I am not completely sure because I am not sure what code HP is using and
what changes were made to it.

Nate Salemme

unread,

May 7, 2010, 11:15:21 AM5/7/10

to open-iscsi

Mike-

Thanks so much for your help. You were right, needed to specify the
bcm transport for the node transport. I have not tried defining that
in the iface file but nonetheless I got it working. Here is the
command I ran:

iscsiadm --mode node --targetname iqn.
1986-03.com.hp:storage.p2000g3.0939da0280 --portal 10.30.40.76:3260 --
op=update --name=node.transport_name --value=bcm570x-020001

This seems to work great! A quick test shows little to no IO on the
actual ethernet interface and very little CPU usage. I used iozone to
write a 20 GB file and saw much better results than before with
sofware iscsi.

Thanks again. Good luck with your iSCCI testing. Although I might
not be able to be much help, feel free to bounce questions off me.

Tarun Reddy

unread,

May 21, 2010, 8:35:52 PM5/21/10

to mich...@cs.wisc.edu, open-...@googlegroups.com

Mike,

So is there any update to this?

Thank you,

Tarun

Mike Christie

unread,

May 24, 2010, 5:05:08 PM5/24/10

to Tarun Reddy, open-...@googlegroups.com, Benjamin Li

On 05/21/2010 07:35 PM, Tarun Reddy wrote:
> Mike,
>
> So is there any update to this?
>

It seems to fix the problem in RHEL 6 testing.
http://people.redhat.com/mchristi/iscsi/rhel6.0/iscsi-initiator-utils/
(I think the rhel6 init scripts will not work with RHEL5 perfectly).

Ben, did you have a brcm release you wanted me to use for a 5.5.z
release (the 5.5.z should not have new features as you know so I could
not use the brcm release I used in the RHEL 6 realease).

Benjamin Li

unread,

May 24, 2010, 5:16:19 PM5/24/10

to Mike Christie, Tarun Reddy, open-...@googlegroups.com

Hi Mike,

I don't have a version of brcm_iscsiuio which I have readily avaliable
for 5.5.z which has the fix you are looking for without adding new
features. I will need to create a version of brcm_iscsiuio with the
iscsid/brcm_iscsiuio backported for 5.5.z. When does this need to be
accomplished?

Thanks again.

-Ben

Benjamin Li

unread,

Jun 10, 2010, 9:30:27 PM6/10/10

to Tarun Reddy, Mike Christie, open-...@googlegroups.com

Hi Tarun,

I think the bug which most likely matched what you were seeing was

https://bugzilla.redhat.com/show_bug.cgi?id=572596

This is where the iscsid and uIP daemon were not synchronized when the
uIP stack was coming up and iscsid didn't wait for uIP to fullly
initialized.

>From that bugzilla we have generated patches for iscsid and uIP which
resulted in the following bugzilla

https://bugzilla.redhat.com/show_bug.cgi?id=595549

Where the patches will go into the next version of iscsi-initator-utils.

Please let me know if there are any questions or concerns.

Thanks again.

-Ben

On Thu, 2010-06-10 at 15:30 -0700, Tarun Reddy wrote:
> Ben, Mike,
>
>
> Is there a RedHat bug covered by this? I've searched and the closest I
> see is
> https://bugzilla.redhat.com/show_bug.cgi?id=578005
>
>
> but that seems to indicate that a service network restart would be a
> workaround. I'm not seeing that work for me.
>
>
> Thank you,
> Tarun

Tarun Reddy

unread,

Jun 10, 2010, 6:30:34 PM6/10/10

to Benjamin Li, Mike Christie, open-...@googlegroups.com

Ben, Mike,

Is there a RedHat bug covered by this? I've searched and the closest I see is

https://bugzilla.redhat.com/show_bug.cgi?id=578005

but that seems to indicate that a service network restart would be a workaround. I'm not seeing that work for me.

Thank you,

Tarun

On Mon, May 24, 2010 at 3:16 PM, Benjamin Li <be...@broadcom.com> wrote:

Mike Christie

unread,

Jul 11, 2010, 6:32:48 AM7/11/10

to open-...@googlegroups.com, Benjamin Li, Tarun Reddy

On 06/10/2010 08:30 PM, Benjamin Li wrote:
> Hi Tarun,
>
> I think the bug which most likely matched what you were seeing was
>
> https://bugzilla.redhat.com/show_bug.cgi?id=572596
>
> This is where the iscsid and uIP daemon were not synchronized when the
> uIP stack was coming up and iscsid didn't wait for uIP to fullly
> initialized.
>
>> From that bugzilla we have generated patches for iscsid and uIP which
> resulted in the following bugzilla
>
> https://bugzilla.redhat.com/show_bug.cgi?id=595549
>
> Where the patches will go into the next version of iscsi-initator-utils.
>
> Please let me know if there are any questions or concerns.
>

Here are some rpms that we are testing here.
http://people.redhat.com/mchristi/iscsi/rhel5.6/iscsi-initiator-utils/

Tarun Reddy

unread,

Jul 12, 2010, 4:25:30 PM7/12/10

to Mike Christie, open-...@googlegroups.com, Benjamin Li

On Mon, Jul 12, 2010 at 2:25 PM, Tarun Reddy <tar...@gmail.com> wrote:

Sigh... trying again with out much success. Running stock 5.5 system+iscsi-initiator-utils from below (194.3.1 kernel)

So I upgraded my kernel 194.8.1 and I get this message in dmesg when doing a modprobe bnx2i:

cnic: Unknown symbol __ipv6_addr_type
cnic: Unknown symbol ip6_route_output

Reboot back into 194.3.1...

And I still get this error. Any ideas? I've reverted iscsi-initiator as well..

Sorry... should have bottom posted.

Tarun

Tarun Reddy

unread,

Jul 12, 2010, 4:25:07 PM7/12/10

to Mike Christie, open-...@googlegroups.com, Benjamin Li

Sigh... trying again with out much success. Running stock 5.5 system+iscsi-initiator-utils from below (194.3.1 kernel)

So I upgraded my kernel 194.8.1 and I get this message in dmesg when doing a modprobe bnx2i:

cnic: Unknown symbol __ipv6_addr_type

cnic: Unknown symbol ip6_route_output

Reboot back into 194.3.1...

And I still get this error. Any ideas? I've reverted iscsi-initiator as well..

Benjamin Li

unread,

Jul 12, 2010, 4:40:38 PM7/12/10

to Tarun Reddy, Mike Christie, open-...@googlegroups.com

Hi Tarun,

Is the ipv6 kernel module loaded? The missing symbols
('__ipv6_addr_type' and 'ip6_route_output') reported by cnic are
resolved by loading the ipv6.ko module

The path to that module on RHEL 5.5 is

/lib/modules/2.6.18-194.el5/kernel/net/ipv6/ipv6.ko

Is the modules.dep file stale? You can regenerate this via the command
'depmod -a' which should allow the automatic loading of module
dependencies.

Thanks again.

-Ben

Tarun Reddy

unread,

Jul 12, 2010, 6:26:00 PM7/12/10

to Benjamin Li, Mike Christie, open-...@googlegroups.com

Hmm... somehow that got removed. I've rebuilt the system (I'm setting up new hardware fortunately) and am back to trying it out again. Just for background, I'm testing out on a Dell R610 with 4 onboard nics (no iscsi offload enabled) and 4 broadcom nics pci-e cards with iscsi offload enabled.

I think I'm missing a basic step here.. but it feels close.

Installed CentOS 5.5 + updates, w/ iscsi-initiator-utils-6.2.0.871-0.18.el5 on x86_64 machine.
eth0 is set to 10.1.3.20 IP address
eth4 is set to 10.1.4.30 (MAC Address:00:10:18:6A:E4:78)
Lefthand iscsi is setup on 10.1.4.10
I set up /var/lib/iscsi/ifaces/bnx2i.00:10:18:6a:e4:79 with the following content

# BEGIN RECORD 2.0-871

iface.iscsi_ifacename = bnx2i.00:10:18:6a:e4:79

iface.hwaddress = 00:10:18:6a:e4:79

iface.ipaddress = 10.1.4.31

iface.transport_name = bnx2i

# END RECORD

6. iscsiadm -m discovery -t st -I bnx2i.00:10:18:6a:e4:79 -p 10.1.4.10

returns

10.1.4.10:3260,1 iqn.2003-10.com.lefthandnetworks:rally-qd:104:testbnxvol

Cool.. looks like I've found the LUN. I think this uses the eth4 device here, not the bnx2i

Trying to login..

iscsiadm -m node -l -P 1

Logging in to [iface: bnx2i.00:10:18:6a:e4:79, target: iqn.2003-10.com.lefthandnetworks:rally-qd:104:testbnxvol, portal: 10.1.4.10,3260]

iscsiadm: Could not login to [iface: bnx2i.00:10:18:6a:e4:79, target: iqn.2003-10.com.lefthandnetworks:rally-qd:104:testbnxvol, portal: 10.1.4.10,3260]:

iscsiadm: initiator reported error (4 - encountered connection failure)

in /var/log/messages..

Jul 12 16:21:01 qd-db-01 iscsid: Received iferror -1

Jul 12 16:21:01 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

And I can't ping 10.1.4.31, but I'm not sure if that is expected or not.

I assume that I should not be setting anything up in the Broadcom setup for iSCSI. I guess I'm assuming that is for iSCSI boot which I don't want or need for now. But maybe I'm wrong? It's happened on plenty of occasions.

Note: Using TCP for iscsi works. I.e. the following commands work great:

iscsiadm -m discovery -t st -p 10.1.4.10

iscsiadm -m node -l

Any ideas?

Tarun

P.S. Relevant lspci:

01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

02:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

04:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

04:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

05:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

05:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

Mike Christie

unread,

Jul 12, 2010, 11:45:43 PM7/12/10

to Tarun Reddy, Benjamin Li, open-...@googlegroups.com

On 07/12/2010 05:26 PM, Tarun Reddy wrote:
> Hmm... somehow that got removed. I've rebuilt the system (I'm setting up new
> hardware fortunately) and am back to trying it out again. Just for
> background, I'm testing out on a Dell R610 with 4 onboard nics (no iscsi
> offload enabled) and 4 broadcom nics pci-e cards with iscsi offload enabled.
>
> I think I'm missing a basic step here.. but it feels close.
>

> 1. Installed CentOS 5.5 + updates, w/
> iscsi-initiator-utils-6.2.0.871-0.18.el5 on x86_64 machine.
> 2. eth0 is set to 10.1.3.20 IP address
> 3. eth4 is set to 10.1.4.30 (MAC Address:00:10:18:6A:E4:78)
> 4. Lefthand iscsi is setup on 10.1.4.10
> 5. I set up /var/lib/iscsi/ifaces/bnx2i.00:10:18:6a:e4:79 with the

> following content
>
> # BEGIN RECORD 2.0-871
> iface.iscsi_ifacename = bnx2i.00:10:18:6a:e4:79
> iface.hwaddress = 00:10:18:6a:e4:79
> iface.ipaddress = 10.1.4.31
> iface.transport_name = bnx2i
> # END RECORD
>
> 6. iscsiadm -m discovery -t st -I bnx2i.00:10:18:6a:e4:79 -p 10.1.4.10
> returns
> 10.1.4.10:3260,1 iqn.2003-10.com.lefthandnetworks:rally-qd:104:testbnxvol
>
> Cool.. looks like I've found the LUN. I think this uses the eth4 device
> here, not the bnx2i

Yes. That is expected. We will use the offload engine for discovery in
the next version.

> Trying to login..
>
> iscsiadm -m node -l -P 1
> Logging in to [iface: bnx2i.00:10:18:6a:e4:79, target:
> iqn.2003-10.com.lefthandnetworks:rally-qd:104:testbnxvol, portal:
> 10.1.4.10,3260]
> iscsiadm: Could not login to [iface: bnx2i.00:10:18:6a:e4:79, target:
> iqn.2003-10.com.lefthandnetworks:rally-qd:104:testbnxvol, portal:
> 10.1.4.10,3260]:
> iscsiadm: initiator reported error (4 - encountered connection failure)
>
> in /var/log/messages..
> Jul 12 16:21:01 qd-db-01 iscsid: Received iferror -1
> Jul 12 16:21:01 qd-db-01 iscsid: cannot make a connection to
> 10.1.4.10:3260(-1,11)
>

Are there some bnx2i messages in /var/log/messages and could you send
them too?

Also what is the Centos 5.5 kernel version?

Kai Meyer

unread,

Jul 15, 2010, 12:57:52 PM7/15/10

to open-iscsi

We are experiencing nearly exactly the same issues as the original
poster. We are fully updated RHEL 5.5, including the iscsi-initiator-
utils-6.2.0.871-0.18.el5 posted earlier in the list (http://
people.redhat.com/mchristi/iscsi/rhel5.6/iscsi-initiator-utils/)

We are on a Dell Blade M610 with a Quad 1GB NIC + iscsi offloader
mezzanine card. We're using the physical interfaces eth2 and eth3, and
would like to add Multipath into the mix once iSCSI is worked out. We
can use the 1GB connections on eth2 and eth3 to connect, login, and do
anything we want, so we thought it would be worthwhile to switch to
the iscsi offloader. During boot, we see these messages a lot:
ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
bnx2i: iSCSI not supported, dev=eth3

Then much later, when iscsi starts, we get:
Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)
iscsi: registered transport (bnx2i)
scsi6 : Broadcom Offload iSCSI Initiator
scsi7 : Broadcom Offload iSCSI Initiator
scsi8 : Broadcom Offload iSCSI Initiator
scsi9 : Broadcom Offload iSCSI Initiator
scsi10 : Broadcom Offload iSCSI Initiator
scsi11 : Broadcom Offload iSCSI Initiator

iscsi: registered transport (tcp)
iscsi: registered transport (iser)

iscsi: registered transport (be2iscsi)

We are basing our selection of iscsi offloading nics by their HW
address:
eth0 Link encap:Ethernet HWaddr 00:26:B9:FA:10:38
inet addr:172.20.1.1 Bcast:172.20.15.255 Mask:
255.255.240.0
eth1 Link encap:Ethernet HWaddr 00:26:B9:FA:10:3A
eth2 Link encap:Ethernet HWaddr 00:23:AE:F9:38:74
inet addr:172.20.17.1 Bcast:172.20.31.255 Mask:
255.255.240.0
eth3 Link encap:Ethernet HWaddr 00:23:AE:F9:38:76
inet addr:172.20.18.1 Bcast:172.20.31.255 Mask:
255.255.240.0
eth4 Link encap:Ethernet HWaddr 00:23:AE:F9:38:78
eth5 Link encap:Ethernet HWaddr 00:23:AE:F9:38:7A

# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>
bnx2i.00:23:ae:f9:38:79 bnx2i,
00:23:ae:f9:38:79,<empty>,<empty>,<empty>
bnx2i.00:26:b9:fa:10:3b bnx2i,00:26:b9:fa:
10:3b,<empty>,<empty>,<empty>
bnx2i.00:23:ae:f9:38:77 bnx2i,
00:23:ae:f9:38:77,172.20.20.1,<empty>,<empty>
bnx2i.00:26:b9:fa:10:39 bnx2i,00:26:b9:fa:
10:39,<empty>,<empty>,<empty>
bnx2i.00:23:ae:f9:38:75 bnx2i,
00:23:ae:f9:38:75,172.20.19.1,<empty>,<empty>
bnx2i.00:23:ae:f9:38:7b bnx2i,
00:23:ae:f9:38:7b,<empty>,<empty>,<empty>

We are guessing (bnx2i.00:23:ae:f9:38:75 == eth2) and (bnx2i.
00:23:ae:f9:38:77 == eth3), which is only important because of the
VLan configuration.

iscsiadm -m iface -I bnx2i.00:23:ae:f9:38:75
# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = bnx2i.00:23:ae:f9:38:75
iface.net_ifacename = <empty>
iface.ipaddress = 172.20.19.1
iface.hwaddress = 00:23:ae:f9:38:75
iface.transport_name = bnx2i
iface.initiatorname = <empty>
# END RECORD

iscsiadm -m iface -I bnx2i.00:23:ae:f9:38:77
# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = bnx2i.00:23:ae:f9:38:77
iface.net_ifacename = <empty>
iface.ipaddress = 172.20.20.1
iface.hwaddress = 00:23:ae:f9:38:77
iface.transport_name = bnx2i
iface.initiatorname = <empty>
# END RECORD

Send Targets works just fine, here's the result of one of the targets
I got back:
iscsiadm -m node -I bnx2i.00:23:ae:f9:38:75 -T iqn.
2001-05.com.equallogic:0-8a0906-db234b706-9e70001687b4c378-008513 -p
172.20.16.14
# BEGIN RECORD 2.0-871
node.name = iqn.2001-05.com.equallogic:0-8a0906-
db234b706-9e70001687b4c378-008513
node.tpgt = 1
node.startup = automatic
iface.hwaddress = 00:23:ae:f9:38:75
iface.ipaddress = 172.20.19.1
iface.iscsi_ifacename = bnx2i.00:23:ae:f9:38:75
iface.net_ifacename = <empty>
iface.transport_name = bnx2i
iface.initiatorname = <empty>
node.discovery_address = 172.20.16.14
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 1024
node.session.queue_depth = 128
node.session.auth.authmethod = CHAP
node.session.auth.username = chap1
node.session.auth.password = ********
node.session.auth.username_in = <empty>
node.session.auth.password_in = <empty>
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 20
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 172.20.16.14
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxRecvDataSegmentLength = 131072
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD

iscsiadm -m node -I bnx2i.00:23:ae:f9:38:77 -T iqn.
2001-05.com.equallogic:0-8a0906-db234b706-9e70001687b4c378-008513 -p
172.20.16.14
# BEGIN RECORD 2.0-871
node.name = iqn.2001-05.com.equallogic:0-8a0906-
db234b706-9e70001687b4c378-008513
node.tpgt = 1
node.startup = automatic
iface.hwaddress = 00:23:ae:f9:38:77
iface.ipaddress = 172.20.20.1
iface.iscsi_ifacename = bnx2i.00:23:ae:f9:38:77
iface.net_ifacename = <empty>
iface.transport_name = bnx2i
iface.initiatorname = <empty>
node.discovery_address = 172.20.16.14
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 1024
node.session.queue_depth = 128
node.session.auth.authmethod = CHAP
node.session.auth.username = chap1
node.session.auth.password = ********
node.session.auth.username_in = <empty>
node.session.auth.password_in = <empty>
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 20
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 172.20.16.14
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxRecvDataSegmentLength = 131072
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD

Logins fail though:
iscsiadm -m node -I bnx2i.00:23:ae:f9:38:75 -T iqn.
2001-05.com.equallogic:0-8a0906-db234b706-9e70001687b4c378-008513 -p
172.20.16.14 -l
Logging in to [iface: bnx2i.00:23:ae:f9:38:75, target: iqn.
2001-05.com.equallogic:0-8a0906-db234b706-9e70001687b4c378-008513,
portal: 172.20.16.14,3260]
iscsiadm: Could not login to [iface: bnx2i.00:23:ae:f9:38:75, target:
iqn.2001-05.com.equallogic:0-8a0906-db234b706-9e70001687b4c378-008513,
portal: 172.20.16.14,3260]:
iscsiadm: initiator reported error (8 - connection timed out)

Syslog gives:
Jul 15 10:42:47 kvmt1 iscsid: Received iferror -1
Jul 15 10:42:47 kvmt1 iscsid: cannot make a connection to
172.20.16.14:3260 (-1,11)
Jul 15 10:42:53 kvmt1 iscsid: Received iferror -1
Jul 15 10:42:53 kvmt1 iscsid: cannot make a connection to
172.20.16.14:3260 (-1,11)
Jul 15 10:42:58 kvmt1 iscsid: Received iferror -1
Jul 15 10:42:58 kvmt1 iscsid: cannot make a connection to
172.20.16.14:3260 (-1,11)
Jul 15 10:43:03 kvmt1 iscsid: Received iferror -1
Jul 15 10:43:03 kvmt1 iscsid: cannot make a connection to
172.20.16.14:3260 (-1,11)
Jul 15 10:43:08 kvmt1 iscsid: Received iferror -1
Jul 15 10:43:08 kvmt1 iscsid: cannot make a connection to
172.20.16.14:3260 (-1,11)

We're ok going back to tcp transport for the time being, but then we
have to play games with ARP, and routing tables to get the data to go
down the right wires when using multipath. Any help would be
appreciated.

Mike Christie

unread,

Jul 15, 2010, 5:05:25 PM7/15/10

to open-...@googlegroups.com, Kai Meyer

On 07/15/2010 11:57 AM, Kai Meyer wrote:
> We are experiencing nearly exactly the same issues as the original
> poster. We are fully updated RHEL 5.5, including the iscsi-initiator-

Just to make sure, you are using something like kernel version
2.6.18-194.8.1?

> utils-6.2.0.871-0.18.el5 posted earlier in the list (http://
> people.redhat.com/mchristi/iscsi/rhel5.6/iscsi-initiator-utils/)
>
> We are on a Dell Blade M610 with a Quad 1GB NIC + iscsi offloader
> mezzanine card. We're using the physical interfaces eth2 and eth3, and
> would like to add Multipath into the mix once iSCSI is worked out. We
> can use the 1GB connections on eth2 and eth3 to connect, login, and do
> anything we want, so we thought it would be worthwhile to switch to
> the iscsi offloader. During boot, we see these messages a lot:
> ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
> bnx2i: iSCSI not supported, dev=eth3

Are you getting this no supported error on the other ethXs?

When using iscsi_tcp are you using iface binding and with that do you
still have to do tricks with the routing tables and ARP? Are doing
binding by iscsi iface hw address/MAC or by the vlan (iscsi iface
iface.net_ifacename = ethX.Y setting)?

Tarun Reddy

unread,

Jul 16, 2010, 3:39:03 PM7/16/10

to Mike Christie, Benjamin Li, open-...@googlegroups.com

Sorry it took me so long to get back to you..

Here are all of the bnx2i messages in /var/log/messages while doing a /etc/init.d/iscsi restart

Jul 16 13:36:00 qd-db-01 iscsid: iscsid shutting down.

Jul 16 13:36:00 qd-db-01 kernel: cnic: Removed CNIC device: eth7

Jul 16 13:36:00 qd-db-01 kernel: cnic: Removed CNIC device: eth6

Jul 16 13:36:00 qd-db-01 kernel: cnic: Removed CNIC device: eth5

Jul 16 13:36:00 qd-db-01 kernel: cnic: Removed CNIC device: eth4

Jul 16 13:36:00 qd-db-01 kernel: cnic: Removed CNIC device: eth3

Jul 16 13:36:00 qd-db-01 kernel: cnic: Removed CNIC device: eth2

Jul 16 13:36:00 qd-db-01 kernel: cnic: Removed CNIC device: eth1

Jul 16 13:36:00 qd-db-01 kernel: cnic: Removed CNIC device: eth0

Jul 16 13:36:00 qd-db-01 kernel: Loading iSCSI transport class v2.0-871.

Jul 16 13:36:00 qd-db-01 kernel: cxgb3i: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.

Jul 16 13:36:00 qd-db-01 kernel: iscsi: registered transport (cxgb3i)

Jul 16 13:36:00 qd-db-01 kernel: Broadcom NetXtreme II CNIC Driver cnic v2.1.0 (Oct 10, 2009)

Jul 16 13:36:00 qd-db-01 kernel: cnic: Added CNIC device: eth0

Jul 16 13:36:00 qd-db-01 kernel: cnic: Added CNIC device: eth1

Jul 16 13:36:00 qd-db-01 kernel: cnic: Added CNIC device: eth2

Jul 16 13:36:00 qd-db-01 kernel: cnic: Added CNIC device: eth3

Jul 16 13:36:00 qd-db-01 kernel: cnic: Added CNIC device: eth4

Jul 16 13:36:00 qd-db-01 kernel: cnic: Added CNIC device: eth5

Jul 16 13:36:00 qd-db-01 kernel: cnic: Added CNIC device: eth6

Jul 16 13:36:00 qd-db-01 kernel: cnic: Added CNIC device: eth7

Jul 16 13:36:00 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 16 13:36:00 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 16 13:36:00 qd-db-01 kernel: scsi11 : Broadcom Offload iSCSI Initiator

Jul 16 13:36:00 qd-db-01 kernel: scsi12 : Broadcom Offload iSCSI Initiator

Jul 16 13:36:00 qd-db-01 kernel: scsi13 : Broadcom Offload iSCSI Initiator

Jul 16 13:36:00 qd-db-01 kernel: scsi14 : Broadcom Offload iSCSI Initiator

Jul 16 13:36:00 qd-db-01 kernel: scsi15 : Broadcom Offload iSCSI Initiator

Jul 16 13:36:00 qd-db-01 kernel: scsi16 : Broadcom Offload iSCSI Initiator

Jul 16 13:36:00 qd-db-01 kernel: scsi17 : Broadcom Offload iSCSI Initiator

Jul 16 13:36:00 qd-db-01 kernel: scsi18 : Broadcom Offload iSCSI Initiator

Jul 16 13:36:02 qd-db-01 kernel: iscsi: registered transport (tcp)

Jul 16 13:36:02 qd-db-01 kernel: iscsi: registered transport (iser)

Jul 16 13:36:02 qd-db-01 kernel: iscsi: registered transport (be2iscsi)

Jul 16 13:36:02 qd-db-01 iscsid: iSCSI logger with pid=8137 started!

Jul 16 13:36:03 qd-db-01 iscsid: transport class version 2.0-871. iscsid version 2.0-871

Jul 16 13:36:03 qd-db-01 iscsid: iSCSI daemon with pid=8138 started!

And the kernel is 194.8.1.

Thank you,

Tarun

(Actually here is grep bnx /var/log/messages since this is a newly built machine and it is small)

Jul 12 14:46:36 qd-db-01 kernel: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.2 (Aug 21, 2009)

Jul 12 14:46:38 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 12 14:46:38 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 12 14:46:38 qd-db-01 kernel: bnx2: eth0: using MSIX

Jul 12 14:46:38 qd-db-01 kernel: bnx2i: iSCSI not supported, dev=eth0

Jul 12 14:46:38 qd-db-01 kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex

Jul 12 14:54:41 qd-db-01 kernel: bnx2: eth4: using MSIX

Jul 12 14:54:41 qd-db-01 kernel: bnx2i [04:00.00]: ISCSI_INIT passed

Jul 12 14:54:44 qd-db-01 kernel: bnx2: eth4 NIC Copper Link is Up, 1000 Mbps full duplex

Jul 12 14:55:05 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 12 14:55:05 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 12 14:55:53 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 12 14:55:53 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 12 15:00:08 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 12 15:00:08 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 12 16:29:14 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 12 16:29:14 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 12 16:29:22 qd-db-01 kernel: bnx2i [04:00.00]: ISCSI_INIT passed

Jul 12 16:29:34 qd-db-01 kernel: bnx2i [04:00.00]: ISCSI_INIT passed

Jul 12 17:07:20 qd-db-01 kernel: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.2 (Aug 21, 2009)

Jul 12 17:07:23 qd-db-01 kernel: bnx2: eth0: using MSIX

Jul 12 17:07:23 qd-db-01 kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex

Jul 16 13:35:38 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 16 13:35:38 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 16 13:36:00 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 16 13:36:00 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Tarun

Benjamin Li

unread,

Jul 16, 2010, 3:51:25 PM7/16/10

to Tarun Reddy, Mike Christie, open-...@googlegroups.com

Hi Tarun,

It looks like from the logs that eth0 is not iSCSI licensed but eth4 is
=)

In addition, I was also wondering if you could also post the following:

1. The routing table of the machine. Currently CNIC uses the L2
routine table to determine which port on the NIC to send requests to.

2. If you could also provide the /var/log/brcm_iscsi.log file, we can
also take a look at the Broadcom uIP user space daemon to see if
anything is wrong. To enable deeper debugging for the brcm_iscsiuio
daemon, please modify the /etc/init.d/iscsid init script and append '-d
100' to the line which start 'brcm_iscsiuio'. This should output all
the debug messages in the daemon.

Thanks again.

-Ben

Tarun Reddy

unread,

Jul 16, 2010, 4:44:16 PM7/16/10

to Benjamin Li, Mike Christie, open-...@googlegroups.com

Hi Benjamin,

I believe that I am trying to use eth4's bnx2i. I had setup eth4 for traffic but apparently I didn't have it up onboot=yes.

I rebooted the machine and actually got the iscsi device attached to /dev/sdb! Of course a /etc/init.d/iscsi restart failed it again with the following messages in /var/log/messages:

Jul 16 14:40:27 qd-db-01 kernel: iscsi: registered transport (cxgb3i)

Jul 16 14:40:27 qd-db-01 kernel: Broadcom NetXtreme II CNIC Driver cnic v2.1.0 (Oct 10, 2009)

Jul 16 14:40:27 qd-db-01 kernel: cnic: Added CNIC device: eth0

Jul 16 14:40:27 qd-db-01 kernel: cnic: Added CNIC device: eth1

Jul 16 14:40:27 qd-db-01 kernel: cnic: Added CNIC device: eth2

Jul 16 14:40:27 qd-db-01 kernel: cnic: Added CNIC device: eth3

Jul 16 14:40:27 qd-db-01 kernel: cnic: Added CNIC device: eth4

Jul 16 14:40:27 qd-db-01 kernel: cnic: Added CNIC device: eth5

Jul 16 14:40:27 qd-db-01 kernel: cnic: Added CNIC device: eth6

Jul 16 14:40:27 qd-db-01 kernel: cnic: Added CNIC device: eth7

Jul 16 14:40:27 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 16 14:40:27 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 16 14:40:27 qd-db-01 kernel: scsi19 : Broadcom Offload iSCSI Initiator

Jul 16 14:40:27 qd-db-01 kernel: scsi20 : Broadcom Offload iSCSI Initiator

Jul 16 14:40:27 qd-db-01 kernel: scsi21 : Broadcom Offload iSCSI Initiator

Jul 16 14:40:27 qd-db-01 kernel: scsi22 : Broadcom Offload iSCSI Initiator

Jul 16 14:40:29 qd-db-01 kernel: scsi23 : Broadcom Offload iSCSI Initiator

Jul 16 14:40:29 qd-db-01 kernel: scsi24 : Broadcom Offload iSCSI Initiator

Jul 16 14:40:29 qd-db-01 kernel: scsi25 : Broadcom Offload iSCSI Initiator

Jul 16 14:40:29 qd-db-01 kernel: scsi26 : Broadcom Offload iSCSI Initiator

Jul 16 14:40:31 qd-db-01 kernel: iscsi: registered transport (tcp)

Jul 16 14:40:31 qd-db-01 kernel: iscsi: registered transport (iser)

Jul 16 14:40:31 qd-db-01 kernel: iscsi: registered transport (be2iscsi)

Jul 16 14:40:31 qd-db-01 iscsid: iSCSI logger with pid=8248 started!

Jul 16 14:40:32 qd-db-01 iscsid: transport class version 2.0-871. iscsid version 2.0-871

Jul 16 14:40:32 qd-db-01 iscsid: iSCSI daemon with pid=8249 started!

Jul 16 14:40:32 qd-db-01 iscsid: Received iferror -1

Jul 16 14:40:32 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:40:38 qd-db-01 iscsid: Received iferror -1

Jul 16 14:40:38 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:40:43 qd-db-01 iscsid: Received iferror -1

Jul 16 14:40:43 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:40:48 qd-db-01 iscsid: Received iferror -1

....

Routing table:

Kernel IP routing table

Destination Gateway Genmask Flags MSS Window irtt Iface

10.1.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth4

10.1.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth4

0.0.0.0 10.1.3.1 0.0.0.0 UG 0 0 0 eth0

And here is brcm_iscsi.log

INFO [Fri Jul 16 14:40:31 2010]Initialize logger using log file: /var/log/brcm-iscsi.log

INFO [Fri Jul 16 14:40:31 2010]Started BRCM iSCSI stack: Ver 0.5.15

INFO [Fri Jul 16 14:40:31 2010]Build date: Sun Jul 11 06:17:31 EDT 2010

INFO [Fri Jul 16 14:40:31 2010]uevent Listening for uevents

INFO [Fri Jul 16 14:40:31 2010]nic_utils Found host[19]: host19

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/iscsi_host/host19/netdev

INFO [Fri Jul 16 14:40:31 2010]nic_utils NIC not found creating an instance for host_no: 19

INFO [Fri Jul 16 14:40:31 2010]nic_utils Found host[20]: host20

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/iscsi_host/host20/netdev

INFO [Fri Jul 16 14:40:31 2010]nic_utils NIC not found creating an instance for host_no: 20

INFO [Fri Jul 16 14:40:31 2010]nic_utils Found host[21]: host21

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/iscsi_host/host21/netdev

INFO [Fri Jul 16 14:40:31 2010]nic_utils NIC not found creating an instance for host_no: 21

INFO [Fri Jul 16 14:40:31 2010]nic_utils Found host[22]: host22

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/iscsi_host/host22/netdev

INFO [Fri Jul 16 14:40:31 2010]nic_utils NIC not found creating an instance for host_no: 22

INFO [Fri Jul 16 14:40:31 2010]nic_utils Found host[23]: host23

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/iscsi_host/host23/netdev

INFO [Fri Jul 16 14:40:31 2010]nic_utils NIC not found creating an instance for host_no: 23

INFO [Fri Jul 16 14:40:31 2010]nic_utils Found host[24]: host24

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/iscsi_host/host24/netdev

INFO [Fri Jul 16 14:40:31 2010]nic_utils NIC not found creating an instance for host_no: 24

INFO [Fri Jul 16 14:40:31 2010]nic_utils Found host[25]: host25

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/iscsi_host/host25/netdev

INFO [Fri Jul 16 14:40:31 2010]nic_utils NIC not found creating an instance for host_no: 25

INFO [Fri Jul 16 14:40:31 2010]nic_utils Found host[26]: host26

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/iscsi_host/host26/netdev

INFO [Fri Jul 16 14:40:31 2010]nic_utils NIC not found creating an instance for host_no: 26

INFO [Fri Jul 16 14:40:31 2010]NIC_NL Netlink to CNIC on pid 8242 is ready

INFO [Fri Jul 16 14:40:31 2010]signal handling thread ready

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc Received request for 'eth4' to set IP address: '10.1.4.31' VLAN: ''

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc Found interface: eth4, using existing NIC

INFO [Fri Jul 16 14:40:31 2010]nic_utils looking for uio device for eth4

INFO [Fri Jul 16 14:40:31 2010]nic_utils eth4 associated with uio1

INFO [Fri Jul 16 14:40:31 2010]nic_id eth4: is found at 04:00.00

INFO [Fri Jul 16 14:40:31 2010]eth4: found NIC 'Broadcom NetXtreme II BCM5709 1000Base-T'

INFO [Fri Jul 16 14:40:31 2010]nic_utils eth4: spinning up thread for nic

INFO [Fri Jul 16 14:40:31 2010]Created nic thread: eth4

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc eth4 library set using transport_name bnx2i

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc eth4 couldn't find VLAN 0 interface creating it

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc eth4: create network interface

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc eth4: requesting configuration using static IP address

WARN [Fri Jul 16 14:40:31 2010]nic_utils eth4: device already disabled:

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc eth4: configuring using static IP

IPv4 address :10.1.4.31

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc netmask :255.0.0.0

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc ISCSID_UIP_IPC_GET_IFACE: command: 1 name: bnx2i.00:10:18:6a:e4:79, netdev: eth4 ipaddr: 10.1.4.31 vlan: 0 transport_name:bnx2i

INFO [Fri Jul 16 14:40:31 2010]iscsi_ipc eth4: started NIC enable thread state: 0x1

ERR [Fri Jul 16 14:40:31 2010]bnx2 /dev/uio1: uio device has been brought up via pid: 8242 on fd: 10

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/uio/uio1/name

INFO [Fri Jul 16 14:40:31 2010]bnx2 eth4: Verified is a cnic_uio device

INFO [Fri Jul 16 14:40:31 2010]Done capturing /sys/class/uio/uio1/event

INFO [Fri Jul 16 14:40:31 2010]bnx2 eth4: detected using MSI-X vector: 8

INFO [Fri Jul 16 14:40:31 2010]bnx2 eth4: tx_bidx_io: 0x12788 tx_bseq_io: 0x12790

INFO [Fri Jul 16 14:40:31 2010]bnx2 eth4: Using mac address: 0:10:18:6a:e4:79

INFO [Fri Jul 16 14:40:31 2010]nic_utils eth4: Added address 33:33:ff:6a:e4:79 to multicast list for eth4

INFO [Fri Jul 16 14:40:31 2010]eth4: CNIC uio initialized

INFO [Fri Jul 16 14:40:31 2010]nic eth4: Initialized ip stack: VLAN: 0

INFO [Fri Jul 16 14:40:31 2010]nic eth4: mac: 00:10:18:6a:e4:79

INFO [Fri Jul 16 14:40:31 2010]nic eth4: Using IP address: 10.1.4.31

INFO [Fri Jul 16 14:40:31 2010]nic eth4: Using netmask: 255.0.0.0

INFO [Fri Jul 16 14:40:31 2010]uip ustack->uip_hostaddr6 IP: ::

INFO [Fri Jul 16 14:40:31 2010]uip ustack->link_local_addr IP: fe80::10:18ff:fe6a:e479

INFO [Fri Jul 16 14:40:31 2010]nic eth4: is now enabled done

INFO [Fri Jul 16 14:40:32 2010]iscsi_ipc Received request for 'eth4' to set IP address: '10.1.4.31' VLAN: ''

INFO [Fri Jul 16 14:40:32 2010]iscsi_ipc Found interface: eth4, using existing NIC

INFO [Fri Jul 16 14:40:32 2010]nic_utils looking for uio device for eth4

INFO [Fri Jul 16 14:40:32 2010]nic_utils eth4 associated with uio1

INFO [Fri Jul 16 14:40:32 2010]nic_id eth4: is found at 04:00.00

INFO [Fri Jul 16 14:40:32 2010]eth4: found NIC 'Broadcom NetXtreme II BCM5709 1000Base-T'

INFO [Fri Jul 16 14:40:32 2010]iscsi_ipc eth4 library set using transport_name bnx2i

INFO [Fri Jul 16 14:40:32 2010]iscsi_ipc eth4: using existing network interface

INFO [Fri Jul 16 14:40:32 2010]iscsi_ipc eth4: requesting configuration using static IP address

INFO [Fri Jul 16 14:40:32 2010]iscsi_ipc eth4: IP configuration didn't change using 0x2

INFO [Fri Jul 16 14:40:32 2010]iscsi_ipc eth4: NIC already enabled flags: 0x1026 state: 0x2

INFO [Fri Jul 16 14:40:32 2010]iscsi_ipc ISCSID_UIP_IPC_GET_IFACE: command: 1 name: bnx2i.00:10:18:6a:e4:79, netdev: eth4 ipaddr: 10.1.4.31 vlan: 0 transport_name:bnx2i

INFO [Fri Jul 16 14:40:32 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:32 2010]uevent Unknown: remove@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:38 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:38 2010]uevent Unknown: remove@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:43 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:43 2010]uevent Unknown: remove@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:48 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:48 2010]uevent Unknown: remove@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:53 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:53 2010]uevent Unknown: remove@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:58 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:40:58 2010]uevent Unknown: remove@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:41:03 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:41:03 2010]uevent Unknown: remove@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:41:08 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:41:08 2010]uevent Unknown: remove@/class/iscsi_endpoint/ep-1

INFO [Fri Jul 16 14:41:13 2010]uevent Unknown: add@/class/iscsi_endpoint/ep-1

(Rebooting server again to see if it comes up on boot)

Tarun

Tarun Reddy

unread,

Jul 16, 2010, 4:52:06 PM7/16/10

to Benjamin Li, Mike Christie, open-...@googlegroups.com

The first time I login to the bnx2i interface after reboot, it works. Subsequent times fails. Here is my /var/log/messages for first and second time

Jul 16 14:47:13 qd-db-01 kernel: ip_conntrack version 2.4 (8192 buckets, 65536 max) - 304 bytes per conntrack

Jul 16 14:47:13 qd-db-01 kernel: bnx2: eth0: using MSIX

Jul 16 14:47:13 qd-db-01 kernel: bnx2: eth0 NIC Copper Link is Up, 1000 Mbps full duplex

Jul 16 14:47:13 qd-db-01 kernel: bnx2: eth4: using MSIX

Jul 16 14:47:13 qd-db-01 kernel: bnx2: eth4 NIC Copper Link is Up, 1000 Mbps full duplex

Jul 16 14:47:13 qd-db-01 kernel: ipmi message handler version 39.1

Jul 16 14:47:19 qd-db-01 puppetd[5289]: Finished catalog run in 7.48 seconds

Jul 16 14:48:31 qd-db-01 kernel: NET: Registered protocol family 10

Jul 16 14:48:31 qd-db-01 kernel: lo: Disabled Privacy Extensions

Jul 16 14:48:31 qd-db-01 kernel: IPv6 over IPv4 tunneling driver

Jul 16 14:49:02 qd-db-01 iscsid: iscsid shutting down.

Jul 16 14:49:02 qd-db-01 kernel: Loading iSCSI transport class v2.0-871.

Jul 16 14:49:02 qd-db-01 kernel: cxgb3i: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.

Jul 16 14:49:02 qd-db-01 kernel: iscsi: registered transport (cxgb3i)

Jul 16 14:49:02 qd-db-01 kernel: Broadcom NetXtreme II CNIC Driver cnic v2.1.0 (Oct 10, 2009)

Jul 16 14:49:02 qd-db-01 kernel: cnic: Added CNIC device: eth0

Jul 16 14:49:02 qd-db-01 kernel: cnic: Added CNIC device: eth1

Jul 16 14:49:02 qd-db-01 kernel: cnic: Added CNIC device: eth2

Jul 16 14:49:02 qd-db-01 kernel: cnic: Added CNIC device: eth3

Jul 16 14:49:02 qd-db-01 kernel: cnic: Added CNIC device: eth4

Jul 16 14:49:02 qd-db-01 kernel: cnic: Added CNIC device: eth5

Jul 16 14:49:02 qd-db-01 kernel: cnic: Added CNIC device: eth6

Jul 16 14:49:02 qd-db-01 kernel: cnic: Added CNIC device: eth7

Jul 16 14:49:02 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 16 14:49:02 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 16 14:49:02 qd-db-01 kernel: scsi3 : Broadcom Offload iSCSI Initiator

Jul 16 14:49:02 qd-db-01 kernel: scsi4 : Broadcom Offload iSCSI Initiator

Jul 16 14:49:02 qd-db-01 kernel: scsi5 : Broadcom Offload iSCSI Initiator

Jul 16 14:49:02 qd-db-01 kernel: scsi6 : Broadcom Offload iSCSI Initiator

Jul 16 14:49:04 qd-db-01 kernel: scsi7 : Broadcom Offload iSCSI Initiator

Jul 16 14:49:04 qd-db-01 kernel: scsi8 : Broadcom Offload iSCSI Initiator

Jul 16 14:49:04 qd-db-01 kernel: scsi9 : Broadcom Offload iSCSI Initiator

Jul 16 14:49:04 qd-db-01 kernel: scsi10 : Broadcom Offload iSCSI Initiator

Jul 16 14:49:06 qd-db-01 kernel: iscsi: registered transport (tcp)

Jul 16 14:49:06 qd-db-01 kernel: iscsi: registered transport (iser)

Jul 16 14:49:06 qd-db-01 kernel: iscsi: registered transport (be2iscsi)

Jul 16 14:49:06 qd-db-01 iscsid: iSCSI logger with pid=6631 started!

Jul 16 14:49:07 qd-db-01 iscsid: transport class version 2.0-871. iscsid version 2.0-871

Jul 16 14:49:07 qd-db-01 iscsid: iSCSI daemon with pid=6632 started!

Jul 16 14:49:07 qd-db-01 iscsid: Could not broadcast to uIP after 3 tries

Jul 16 14:49:11 qd-db-01 last message repeated 4 times

Jul 16 14:49:12 qd-db-01 iscsid: Received iferror -1

Jul 16 14:49:12 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:49:15 qd-db-01 kernel: bnx2i [04:00.00]: ISCSI_INIT passed

Jul 16 14:49:15 qd-db-01 kernel: connection1:0: detected conn error (1011)

Jul 16 14:49:16 qd-db-01 kernel: bnx2i [04:00.00]: ISCSI_INIT passed

Jul 16 14:49:21 qd-db-01 kernel: bnx2i [04:00.00]: ISCSI_INIT passed

Jul 16 14:49:22 qd-db-01 kernel: connection1:0: detected conn error (1011)

Jul 16 14:49:22 qd-db-01 kernel: bnx2i [04:00.00]: ISCSI_INIT passed

Jul 16 14:49:23 qd-db-01 kernel: connection1:0: bnx2i: conn update - MBL 0x100000 FBL 0x40000MRDSL_I 0x40000 MRDSL_T 0x40000

Jul 16 14:49:23 qd-db-01 kernel: connection1:0: bnx2i: iscsi_warning - invalid TTT

Jul 16 14:49:23 qd-db-01 kernel: Vendor: LEFTHAND Model: iSCSIDisk Rev: 8500

Jul 16 14:49:23 qd-db-01 kernel: Type: Direct-Access ANSI SCSI revision: 05

Jul 16 14:49:23 qd-db-01 kernel: SCSI device sdb: 20971520 512-byte hdwr sectors (10737 MB)

Jul 16 14:49:23 qd-db-01 kernel: sdb: Write Protect is off

Jul 16 14:49:23 qd-db-01 kernel: SCSI device sdb: drive cache: none

Jul 16 14:49:23 qd-db-01 kernel: SCSI device sdb: 20971520 512-byte hdwr sectors (10737 MB)

Jul 16 14:49:23 qd-db-01 kernel: sdb: Write Protect is off

Jul 16 14:49:23 qd-db-01 kernel: SCSI device sdb: drive cache: none

Jul 16 14:49:23 qd-db-01 kernel: sdb: unknown partition table

Jul 16 14:49:23 qd-db-01 kernel: sd 6:0:0:0: Attached scsi disk sdb

Jul 16 14:49:23 qd-db-01 kernel: sd 6:0:0:0: Attached scsi generic sg4 type 0

Jul 16 14:49:23 qd-db-01 iscsid: connection1:0 is operational now

Jul 16 14:49:23 qd-db-01 iscsid: Could not write to /sys/bus/scsi/devices/6:0:0:0/queue_depth. Invalid permissions.

Jul 16 14:49:23 qd-db-01 iscsid: Could not queue depth for LUN 0 err 13.

Jul 16 14:49:42 qd-db-01 kernel: connection1:0: received itt 0 expected session age (1)

Jul 16 14:49:44 qd-db-01 puppetd[5289]: Caught TERM; calling stop

Jul 16 14:50:01 qd-db-01 kernel: connection1:0: detected conn error (1011)

Jul 16 14:50:02 qd-db-01 iscsid: iscsid shutting down.

Jul 16 14:50:02 qd-db-01 kernel: cnic: Removed CNIC device: eth7

Jul 16 14:50:02 qd-db-01 kernel: cnic: Removed CNIC device: eth6

Jul 16 14:50:02 qd-db-01 kernel: cnic: Removed CNIC device: eth5

Jul 16 14:50:02 qd-db-01 kernel: cnic: Removed CNIC device: eth4

Jul 16 14:50:02 qd-db-01 kernel: cnic: Removed CNIC device: eth3

Jul 16 14:50:02 qd-db-01 kernel: cnic: Removed CNIC device: eth2

Jul 16 14:50:02 qd-db-01 kernel: cnic: Removed CNIC device: eth1

Jul 16 14:50:02 qd-db-01 kernel: cnic: Removed CNIC device: eth0

Jul 16 14:50:02 qd-db-01 kernel: Loading iSCSI transport class v2.0-871.

Jul 16 14:50:02 qd-db-01 kernel: cxgb3i: tag itt 0x1fff, 13 bits, age 0xf, 4 bits.

Jul 16 14:50:02 qd-db-01 kernel: iscsi: registered transport (cxgb3i)

Jul 16 14:50:02 qd-db-01 kernel: Broadcom NetXtreme II CNIC Driver cnic v2.1.0 (Oct 10, 2009)

Jul 16 14:50:02 qd-db-01 kernel: cnic: Added CNIC device: eth0

Jul 16 14:50:02 qd-db-01 kernel: cnic: Added CNIC device: eth1

Jul 16 14:50:02 qd-db-01 kernel: cnic: Added CNIC device: eth2

Jul 16 14:50:02 qd-db-01 kernel: cnic: Added CNIC device: eth3

Jul 16 14:50:02 qd-db-01 kernel: cnic: Added CNIC device: eth4

Jul 16 14:50:02 qd-db-01 kernel: cnic: Added CNIC device: eth5

Jul 16 14:50:02 qd-db-01 kernel: cnic: Added CNIC device: eth6

Jul 16 14:50:02 qd-db-01 kernel: cnic: Added CNIC device: eth7

Jul 16 14:50:02 qd-db-01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v2.1.0 (Dec 06, 2009)

Jul 16 14:50:02 qd-db-01 kernel: iscsi: registered transport (bnx2i)

Jul 16 14:50:02 qd-db-01 kernel: scsi11 : Broadcom Offload iSCSI Initiator

Jul 16 14:50:02 qd-db-01 kernel: scsi12 : Broadcom Offload iSCSI Initiator

Jul 16 14:50:02 qd-db-01 kernel: scsi13 : Broadcom Offload iSCSI Initiator

Jul 16 14:50:02 qd-db-01 kernel: scsi14 : Broadcom Offload iSCSI Initiator

Jul 16 14:50:04 qd-db-01 kernel: scsi15 : Broadcom Offload iSCSI Initiator

Jul 16 14:50:04 qd-db-01 kernel: scsi16 : Broadcom Offload iSCSI Initiator

Jul 16 14:50:04 qd-db-01 kernel: scsi17 : Broadcom Offload iSCSI Initiator

Jul 16 14:50:04 qd-db-01 kernel: scsi18 : Broadcom Offload iSCSI Initiator

Jul 16 14:50:06 qd-db-01 kernel: iscsi: registered transport (tcp)

Jul 16 14:50:06 qd-db-01 kernel: iscsi: registered transport (iser)

Jul 16 14:50:06 qd-db-01 kernel: iscsi: registered transport (be2iscsi)

Jul 16 14:50:06 qd-db-01 iscsid: iSCSI logger with pid=7855 started!

Jul 16 14:50:07 qd-db-01 iscsid: transport class version 2.0-871. iscsid version 2.0-871

Jul 16 14:50:07 qd-db-01 iscsid: iSCSI daemon with pid=7856 started!

Jul 16 14:50:07 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:07 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:13 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:13 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:18 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:18 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:23 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:23 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:28 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:28 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:33 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:33 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:38 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:38 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:43 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:43 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:48 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:48 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:53 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:53 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Jul 16 14:50:58 qd-db-01 iscsid: Received iferror -1

Jul 16 14:50:58 qd-db-01 iscsid: cannot make a connection to 10.1.4.10:3260 (-1,11)

Thank you,

Tarun

Eddie Wai

unread,

Jul 15, 2010, 5:49:43 PM7/15/10

to open-...@googlegroups.com, Kai Meyer

Hello Kai,

FYI, the "bnx2i: iSCSI not supported" message indicates that the port associated with eth3 is not licensed to do iSCSI offload. Echoing Mike's question, are you getting this message on all the ethXs that you intend to do iSCSI offload?

Eddie

Gianluca

unread,

Jul 16, 2010, 10:17:35 AM7/16/10

to open-iscsi

Hello,
I have the same problem.
HW is M610 with RH EL 5.5 x86_64 updated (kernel 2.6.18-194.8.1).
I'm connected with an EQL PS6010XV through 10Gbs adapters on the blade
sw iSCSI works with and without multipath.

Same problems with iSCSI offload.
Tried also installing version 6.2.0.871-0.18.el5 without success.
Latest steps:
- disable iscsi/iscsid at startup
- stop iscsid
- update iscsi-initiator-utils
- shutdown -r
- clean /var/lib/iscsi from previous configs:
cd /var/lib/iscsi
rm -rf nodes/*
rm -rf send_targets/*
# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:10:18:58:E8:F8
inet addr:10.10.100.178 Bcast:10.10.100.255 Mask:
255.255.255.0
inet6 addr: fe80::210:18ff:fe58:e8f8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:1025 errors:0 dropped:0 overruns:0 frame:0
TX packets:50 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:104130 (101.6 KiB) TX bytes:4700 (4.5 KiB)
Interrupt:82 Memory:dc800000-dcffffff

# ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:10:18:58:E8:FA
inet addr:10.10.100.179 Bcast:10.10.100.255 Mask:
255.255.255.0
inet6 addr: fe80::210:18ff:fe58:e8fa/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:1014 errors:0 dropped:0 overruns:0 frame:0
TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:83638 (81.6 KiB) TX bytes:630 (630.0 b)
Interrupt:90 Memory:dd800000-ddffffff

# cat ifaces/bnx2i.00:10:18:58:e8:f9
# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = bnx2i.00:10:18:58:e8:f9
iface.ipaddress = 10.10.100.198
iface.hwaddress = 00:10:18:58:e8:f9

iface.transport_name = bnx2i
# END RECORD

# cat ifaces/bnx2i.00:10:18:58:e8:fb
# BEGIN RECORD 2.0-871
iface.iscsi_ifacename = bnx2i.00:10:18:58:e8:fb
iface.ipaddress = 10.10.100.199
iface.hwaddress = 00:10:18:58:e8:fb

iface.transport_name = bnx2i
# END RECORD

# service iscsid start
Starting iSCSI daemon: [ OK ]
[ OK ]
# iscsiadm -m discovery -t st -p 10.10.100.30:3260 -I bnx2i.
00:10:18:58:e8:fb -I bnx2i.00:10:18:58:e8:f9 -P 1
Target: iqn.2001-05.com.equallogic:0-8a0906-97d4b5e06-596000000264bc83-
blg9-vol3
Portal: 10.10.100.30:3260,1
Iface Name: bnx2i.00:10:18:58:e8:f9
Iface Name: bnx2i.00:10:18:58:e8:fb
Target: iqn.2001-05.com.equallogic:0-8a0906-8904b5e06-b66000000204bc83-
blg9-vol1
Portal: 10.10.100.30:3260,1
Iface Name: bnx2i.00:10:18:58:e8:f9
Iface Name: bnx2i.00:10:18:58:e8:fb
Target: iqn.2001-05.com.equallogic:0-8a0906-94c4b5e06-df9000000234bc83-
blg9-vol2
Portal: 10.10.100.30:3260,1
Iface Name: bnx2i.00:10:18:58:e8:f9
Iface Name: bnx2i.00:10:18:58:e8:fb

# iscsiadm -m node --loginall=all

the command show suddenly the first "successful" line, then stays
there for about three minutes and at the end shows the remaining
successfull one and the failed ones.
Complete output of the command:

Logging in to [iface: bnx2i.00:10:18:58:e8:f9, target: iqn.
2001-05.com.equallogic:0-8a0906-97d4b5e06-596000000264bc83-blg9-vol3,
portal: 10.10.100.30,3260]
Logging in to [iface: bnx2i.00:10:18:58:e8:fb, target: iqn.
2001-05.com.equallogic:0-8a0906-97d4b5e06-596000000264bc83-blg9-vol3,
portal: 10.10.100.30,3260]
Logging in to [iface: bnx2i.00:10:18:58:e8:f9, target: iqn.
2001-05.com.equallogic:0-8a0906-8904b5e06-b66000000204bc83-blg9-vol1,
portal: 10.10.100.30,3260]
Logging in to [iface: bnx2i.00:10:18:58:e8:fb, target: iqn.
2001-05.com.equallogic:0-8a0906-8904b5e06-b66000000204bc83-blg9-vol1,
portal: 10.10.100.30,3260]
Logging in to [iface: bnx2i.00:10:18:58:e8:f9, target: iqn.
2001-05.com.equallogic:0-8a0906-94c4b5e06-df9000000234bc83-blg9-vol2,
portal: 10.10.100.30,3260]
Logging in to [iface: bnx2i.00:10:18:58:e8:fb, target: iqn.
2001-05.com.equallogic:0-8a0906-94c4b5e06-df9000000234bc83-blg9-vol2,
portal: 10.10.100.30,3260]
Login to [iface: bnx2i.00:10:18:58:e8:f9, target: iqn.
2001-05.com.equallogic:0-8a0906-97d4b5e06-596000000264bc83-blg9-vol3,
portal: 10.10.100.30,3260]: successful
iscsiadm: Could not login to [iface: bnx2i.00:10:18:58:e8:fb, target:
iqn.2001-05.com.equallogic:0-8a0906-97d4b5e06-596000000264bc83-blg9-
vol3, portal: 10.10.100.30,3260]:

iscsiadm: initiator reported error (8 - connection timed out)

iscsiadm: Could not login to [iface: bnx2i.00:10:18:58:e8:f9, target:
iqn.2001-05.com.equallogic:0-8a0906-8904b5e06-b66000000204bc83-blg9-
vol1, portal: 10.10.100.30,3260]:

iscsiadm: initiator reported error (4 - encountered connection
failure)

iscsiadm: Could not login to [iface: bnx2i.00:10:18:58:e8:fb, target:
iqn.2001-05.com.equallogic:0-8a0906-8904b5e06-b66000000204bc83-blg9-
vol1, portal: 10.10.100.30,3260]:

iscsiadm: initiator reported error (4 - encountered connection
failure)

Login to [iface: bnx2i.00:10:18:58:e8:f9, target: iqn.
2001-05.com.equallogic:0-8a0906-94c4b5e06-df9000000234bc83-blg9-vol2,
portal: 10.10.100.30,3260]: successful
iscsiadm: Could not login to [iface: bnx2i.00:10:18:58:e8:fb, target:
iqn.2001-05.com.equallogic:0-8a0906-94c4b5e06-df9000000234bc83-blg9-
vol2, portal: 10.10.100.30,3260]:

iscsiadm: initiator reported error (8 - connection timed out)

iscsiadm: Could not log into all portals. Err 8.

# iscsiadm -m session
bnx2i: [1] 10.10.100.30:3260,1 iqn.2001-05.com.equallogic:
0-8a0906-94c4b5e06-df9000000234bc83-blg9-vol2
bnx2i: [2] 10.10.100.30:3260,1 iqn.2001-05.com.equallogic:
0-8a0906-97d4b5e06-596000000264bc83-blg9-vol3

During the three-minutes interval in messages:
Jul 16 15:16:21 orasvi2 iscsid: Received iferror -1
Jul 16 15:16:21 orasvi2 iscsid: cannot make a connection to
10.10.100.30:3260 (-1,11)
Jul 16 15:16:21 orasvi2 kernel: bnx2i [05:00.01]: ISCSI_INIT passed
Jul 16 15:16:21 orasvi2 kernel: bnx2i [05:00.00]: ISCSI_INIT passed
Jul 16 15:16:22 orasvi2 iscsid: Received iferror -101
Jul 16 15:16:22 orasvi2 iscsid: cannot make a connection to
10.10.100.30:3260 (-101,11)
Jul 16 15:16:23 orasvi2 kernel: bnx2i [05:00.01]: ISCSI_INIT passed
Jul 16 15:16:23 orasvi2 kernel: connection1:0: detected conn error
(1011)
Jul 16 15:16:23 orasvi2 iscsid: Could not broadcast to uIP after 3
tries
Jul 16 15:16:23 orasvi2 iscsid: Received iferror -101
Jul 16 15:16:23 orasvi2 iscsid: cannot make a connection to
10.10.100.30:3260 (-101,11)
Jul 16 15:16:23 orasvi2 kernel: bnx2i [05:00.01]: ISCSI_INIT passed
Jul 16 15:16:23 orasvi2 kernel: connection2:0: detected conn error
(1011)
Jul 16 15:16:24 orasvi2 iscsid: Received iferror -101
Jul 16 15:16:24 orasvi2 iscsid: cannot make a connection to
10.10.100.30:3260 (-101,11)
Jul 16 15:16:24 orasvi2 iscsid: Login authentication failed with
target iqn.2001-05.com.equallogic:0-8a0906-94c4b5e06-df9000000234bc83-
blg9-vol2
Jul 16 15:16:26 orasvi2 iscsid: Login authentication failed with
target iqn.2001-05.com.equallogic:0-8a0906-97d4b5e06-596000000264bc83-
blg9-vol3
Jul 16 15:16:29 orasvi2 kernel: connection1:0: bnx2i: conn update -
MBL 0x40000 FBL 0x10000MRDSL_I 0x40000 MRDSL_T 0x10000
Jul 16 15:16:29 orasvi2 kernel: Vendor: EQLOGIC Model:
100E-00 Rev: 4.3
Jul 16 15:16:29 orasvi2 kernel: Type: Direct-

Access ANSI SCSI revision: 05

Jul 16 15:16:29 orasvi2 kernel: SCSI device sdb: 419450880 512-byte
hdwr sectors (214759 MB)
Jul 16 15:16:29 orasvi2 kernel: sdb: Write Protect is off
Jul 16 15:16:29 orasvi2 kernel: SCSI device sdb: drive cache: write
through
Jul 16 15:16:29 orasvi2 kernel: SCSI device sdb: 419450880 512-byte
hdwr sectors (214759 MB)
Jul 16 15:16:29 orasvi2 kernel: sdb: Write Protect is off
Jul 16 15:16:29 orasvi2 kernel: SCSI device sdb: drive cache: write
through
Jul 16 15:16:29 orasvi2 multipathd: sdb: add path (uevent)
Jul 16 15:16:29 orasvi2 kernel: sdb: unknown partition table
Jul 16 15:16:29 orasvi2 kernel: sd 9:0:0:0: Attached scsi disk sdb
Jul 16 15:16:29 orasvi2 kernel: sd 9:0:0:0: Attached scsi generic sg3
type 0
Jul 16 15:16:30 orasvi2 kernel: connection2:0: bnx2i: conn update -
MBL 0x40000 FBL 0x10000MRDSL_I 0x40000 MRDSL_T 0x10000
Jul 16 15:16:30 orasvi2 kernel: Vendor: EQLOGIC Model:
100E-00 Rev: 4.3
Jul 16 15:16:30 orasvi2 kernel: Type: Direct-

Access ANSI SCSI revision: 05

Jul 16 15:16:30 orasvi2 kernel: SCSI device sdc: 419450880 512-byte
hdwr sectors (214759 MB)
Jul 16 15:16:30 orasvi2 kernel: sdc: Write Protect is off
Jul 16 15:16:30 orasvi2 kernel: SCSI device sdc: drive cache: write
through
Jul 16 15:16:30 orasvi2 kernel: SCSI device sdc: 419450880 512-byte
hdwr sectors (214759 MB)
Jul 16 15:16:30 orasvi2 kernel: sdc: Write Protect is off
Jul 16 15:16:30 orasvi2 kernel: SCSI device sdc: drive cache: write
through
Jul 16 15:16:30 orasvi2 kernel: sdc: unknown partition table
Jul 16 15:16:30 orasvi2 kernel: sd 9:0:1:0: Attached scsi disk sdc
Jul 16 15:16:30 orasvi2 kernel: sd 9:0:1:0: Attached scsi generic sg4
type 0
Jul 16 15:16:30 orasvi2 kernel: device-mapper: multipath round-robin:
version 1.0.0 loaded
Jul 16 15:16:30 orasvi2 multipathd: vol2: load table [0 419450880
multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:16 512]
Jul 16 15:16:30 orasvi2 multipathd: vol2: event checker started
Jul 16 15:16:30 orasvi2 multipathd: sdc: add path (uevent)
Jul 16 15:16:30 orasvi2 iscsid: connection1:0 is operational now
Jul 16 15:16:30 orasvi2 iscsid: Could not write to /sys/bus/scsi/
devices/9:0:0:0/queue_depth. Invalid permissions.
Jul 16 15:16:30 orasvi2 iscsid: Could not queue depth for LUN 0 err
13.
Jul 16 15:16:30 orasvi2 iscsid: connection2:0 is operational now
Jul 16 15:16:30 orasvi2 iscsid: Could not write to /sys/bus/scsi/
devices/9:0:1:0/queue_depth. Invalid permissions.
Jul 16 15:16:30 orasvi2 iscsid: Could not queue depth for LUN 0 err
13.
Jul 16 15:16:30 orasvi2 multipathd: vol3: load table [0 419450880
multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:32 512]
Jul 16 15:16:30 orasvi2 multipathd: vol3: event checker started
Jul 16 15:16:30 orasvi2 multipathd: dm-3: add map (uevent)
Jul 16 15:16:30 orasvi2 multipathd: dm-3: devmap already registered
Jul 16 15:16:30 orasvi2 multipathd: dm-4: add map (uevent)
Jul 16 15:16:30 orasvi2 multipathd: dm-4: devmap already registered
Jul 16 15:16:31 orasvi2 kernel: bnx2i [05:00.01]: ISCSI_INIT passed
Jul 16 15:16:31 orasvi2 kernel: bnx2i [05:00.01]: ISCSI_INIT passed
Jul 16 15:16:31 orasvi2 iscsid: Received iferror -101
Jul 16 15:16:31 orasvi2 iscsid: cannot make a connection to
10.10.100.30:3260 (-101,0)

and then many of the latest 3 lines above for the interval.

Also, when using offload, the file /var/log/brcm-iscsi.log exploded in
size logging many of these lines:
WARN [Fri Jul 16 16:08:01 2010]uip ip: packet shorter than reported
in IP header: tcp_ipv4_hdr->len: 328 ustack->uip_len:60.
WARN [Fri Jul 16 16:08:01 2010]uip ip: packet shorter than reported
in IP header: tcp_ipv4_hdr->len: 328 ustack->uip_len:60.
WARN [Fri Jul 16 16:08:22 2010]uip ip: packet shorter than reported
in IP header: tcp_ipv4_hdr->len: 328 ustack->uip_len:60.
WARN [Fri Jul 16 16:08:22 2010]uip ip: packet shorter than reported
in IP header: tcp_ipv4_hdr->len: 328 ustack->uip_len:60.
ERR [Fri Jul 16 16:08:53 2010]nic eth0: Missed interrupt! on 855 not
853
WARN [Fri Jul 16 16:10:05 2010]uip ip: packet shorter than reported
in IP header: tcp_ipv4_hdr->len: 328 ustack->uip_len:60.
WARN [Fri Jul 16 16:10:05 2010]uip ip: packet shorter than reported
in IP header: tcp_ipv4_hdr->len: 328 ustack->uip_len:60.
WARN [Fri Jul 16 16:10:17 2010]uip ip: packet shorter than reported
in IP header: tcp_ipv4_hdr->len: 328 ustack->uip_len:60.
WARN [Fri Jul 16 16:10:17 2010]uip ip: packet shorter than reported
in IP header: tcp_ipv4_hdr->len: 328 ustack->uip_len:60.

After the rpm update it seems that the login problem remains but I
don't get the errors in brcm-iscsi.log

Gianluca

unread,

Jul 16, 2010, 10:21:37 AM7/16/10

to open-iscsi

Same error with M610 and rh el 5.5 x86_64 connected with EQL PS6010XV.

Kai Meyer

unread,

Jul 30, 2010, 4:32:27 PM7/30/10

to open-...@googlegroups.com

Sorry for the duplicate email Eddie. I'm not used to gmail's interface just yet.

On Thu, Jul 15, 2010 at 3:49 PM, Eddie Wai <wa...@broadcom.com> wrote:

Hello Kai,

FYI, the "bnx2i: iSCSI not supported" message indicates that the port associated with eth3 is not licensed to do iSCSI offload. Echoing Mike's question, are you getting this message on all the ethXs that you intend to do iSCSI offload?

I get the same message on all of the ethXs, even the ones that we do not intend to do iSCSI offloading. I was not aware there was a licensing issue to deal with here. Should I be addressing this with Dell then?

Eddie

-----Original Message-----
From: open-...@googlegroups.com [mailto:open-...@googlegroups.com] On Behalf Of Mike Christie
Sent: Thursday, July 15, 2010 2:05 PM
To: open-...@googlegroups.com
Cc: Kai Meyer
Subject: Re: Centos 5.4 bnx2i iscsi woes

On 07/15/2010 11:57 AM, Kai Meyer wrote:

Just to make sure, you are using something like kernel version
2.6.18-194.8.1?

Yes, 2.6.18-194.8.1.el5

Are you getting this no supported error on the other ethXs?

Yes, see above response to Eddie Wai

When using iscsi_tcp are you using iface binding and with that do you
still have to do tricks with the routing tables and ARP? Are doing
binding by iscsi iface hw address/MAC or by the vlan (iscsi iface
iface.net_ifacename = ethX.Y setting)?

When using iscsi_tcp, we are basically leaving all iscsi settings to default (whatever the iscsi and iscsid services default to.) The routing tricks I was talking about are implemented in /etc/rc.local, like so:
---
. /etc/sysconfig/network-

scripts/ifcfg-eth2
eval `ipcalc -n -p $IPADDR $NETMASK`
ip route add $NETWORK/$PREFIX dev $DEVICE tab 2
ip rule add from $IPADDR/32 tab 2 priority 200

. /etc/sysconfig/network-scripts/ifcfg-eth3
eval `ipcalc -n -p $IPADDR $NETMASK`
ip route add $NETWORK/$PREFIX dev $DEVICE tab 3
ip rule add from $IPADDR/32 tab 3 priority 300

ip route flush cache
---

and in /etc/sysctl.conf, we added the following two lines:
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

These changes are intended to ensure that inbound traffic to a specific ethX device is responded to out the same interface it was received on, instead of the first route in the routing table, as well as to ensure that the ARP requests are only responded to on the interface that the physical mac address is on (with out the sysctl rules, eth3 will respond to arps for eth2's mac).

From what it sounds like, I should be asking Dell if we are licensed to use the iSCSI offloader?

-Kai Meyer

Reply all

Reply to author

Forward