Re: need help before I kill myself

[Mike Christie]

On 04/04/2013 01:46 PM, gee...@gmail.com wrote:
> Hello,
> I have been trying to setup an iSCSI storage array for several weeks
> now. I have some things working but others are not. I am running out of
> time and am willing to to hire someone with the skill and experience to
> help me out. If this kind of request is not accepted on this list I
> apologize and will stop posting at the first notice I receive. If it's
> ok then please respond.
>
> Here is what I have and need help with. I have purchased a Synology
> DS1812+, running the latest DSM management software and configured with
> 14 TB of storage. I want to use the array for my /home and /backup
> partitions on my web hosting servers. There will be about 3-4 servers
> using it for this purpose. The individiual servers will be running
> CentOS, various versions, and open-iscsi as the initiator. I have
> purchaes a few ToE cards (the broadcom 5709) for the physical
> connections. I am currently working on a new server (centos 6.4) and
> testing the concept out. I have been able to connect using the server's
> onboard NIC but not the ToE card. I have played with the iface config
> but it refuses to connect.
>

For Centos 6.4 you should be able to use the default bnx2i ifaces.


> I am not going to post the details as I am hoping to have someone help
> me out with this.

I am not sure what that means. We need some details :)

Could you post the error message you get when you try to login with
iscsiadm -m node -l

and also post the /var/log/messages for when you try to login.

[Sebastian Riemer]

On 08.04.2013 08:00, Mike Christie wrote:
> On 04/04/2013 01:46 PM, gee...@gmail.com wrote:
>> Hello,
>> I have been trying to setup an iSCSI storage array for several weeks
>> now. I have some things working but others are not. I am running out of
>> time and am willing to to hire someone with the skill and experience to
>> help me out. If this kind of request is not accepted on this list I
>> apologize and will stop posting at the first notice I receive. If it's
>> ok then please respond.
>>
>> Here is what I have and need help with. I have purchased a Synology
>> DS1812+, running the latest DSM management software and configured with
>> 14 TB of storage. I want to use the array for my /home and /backup
>> partitions on my web hosting servers. There will be about 3-4 servers
>> using it for this purpose. The individiual servers will be running
>> CentOS, various versions, and open-iscsi as the initiator. I have
>> purchaes a few ToE cards (the broadcom 5709) for the physical
>> connections. I am currently working on a new server (centos 6.4) and
>> testing the concept out. I have been able to connect using the server's
>> onboard NIC but not the ToE card. I have played with the iface config
>> but it refuses to connect.
>>
>
> For Centos 6.4 you should be able to use the default bnx2i ifaces.
>

These also require special free and nonfree firmware. Check your kernel
config! Check your firmware-nonfree package!

Look at:
CONFIG_BNX2
CONFIG_SCSI_BNX2_ISCSI

Kernel:
$ ls firmware/bnx2
bnx2-mips-06-6.2.1.fw bnx2-rv2p-06-6.0.15.fw.ihex
bnx2-mips-06-6.2.1.fw.ihex bnx2-rv2p-09-6.0.17.fw
bnx2-mips-09-6.2.1a.fw bnx2-rv2p-09-6.0.17.fw.ihex
bnx2-mips-09-6.2.1a.fw.ihex bnx2-rv2p-09ax-6.0.17.fw
bnx2-rv2p-06-6.0.15.fw bnx2-rv2p-09ax-6.0.17.fw.ihex

If compiled manually:
$ make install modules_install firmware_install

firmware-nonfree:
$ ls bnx2/bnx2
bnx2-mips-06-5.0.0.j3.fw bnx2-rv2p-06-5.0.0.j3.fw
bnx2-mips-06-6.2.1.fw bnx2-rv2p-06-6.0.15.fw
bnx2-mips-06-6.2.3.fw bnx2-rv2p-09-5.0.0.j3.fw
bnx2-mips-09-5.0.0.j3.fw bnx2-rv2p-09-6.0.17.fw
bnx2-mips-09-6.2.1a.fw bnx2-rv2p-09ax-5.0.0.j3.fw
bnx2-mips-09-6.2.1b.fw bnx2-rv2p-09ax-6.0.17.fw

Cheers,
Sebastian

[Cip]

On 04/08/2013 11:16 AM, Sebastian Riemer wrote:
>
>> For Centos 6.4 you should be able to use the default bnx2i ifaces.
>>
> These also require special free and nonfree firmware. Check your kernel
> config! Check your firmware-nonfree package!
>
> Look at:
> CONFIG_BNX2
> CONFIG_SCSI_BNX2_ISCSI
>

Maybe not your exactly your situation but keep in mind that that at
least the "on-board" Broadcoms in Dell servers need a special "license
key" in order for the firmware to function with TOE! Without that
everything looks OK, you see TOE all over the place (BIOS screens,
driver loading etc) but it just won't work; In Dell boxes the license
key looks like a really small USB token to be inserted directly into the
motherboard. Can only hope that your interfaces came with the said
license embedded or that they don't need it.

[lian...@gmail.com]

I finally got this to work but I have some questions about the card and its setup. Some specifics:

 

Server with 4 onboard NICS (eth0, eth1, eth4, eth5), eth0 connected to array via 174.x.x.x (this port will be closed once setup complete, used for testing remotely atm)

BC 5709 dual port (eth2, eth3), eth2 connected to array via 192.x.x.x

Created and iface config with the following:

 

# BEGIN RECORD 6.2.0-873.2.el6
ace.iscsi_ifacename = eth2
iface.transport_name = bnx2i
iface.hwaddress = 00:10:18:F3:43:00
iface.ipaddress = 192.x.x.x
iface.bootproto = static
iface.subnet_mask = 255.255.255.0
iface.vlan_id = 0
iface.vlan_priority = 0
iface.iface_num = 0
iface.mtu = 9000
iface.port = 0
# END RECORD

 

I read that ToE cards and other accelerator cards have a separate MAC for the accelerator engines. Is this true? So my dual port BC 5709 would have 4 mac addresses? If so I take the proper Mac to place in the iface config would be that of the accelerator?

 

# iscsiadm -m session -o show
tcp: [1] 192.x.x.x:3260,0 iqn.2000-01.com.synology:san.quincy

Is the 'tcp' telling me it is not properly setup? should it be bnx2i at the beginning of the above result?

 

I was also under the impression that when I created an iface config file it would act as the default and only connection. I say this because the connection with the server's nic is still appearing when I reboot. How do I stop this from happening and force the 5709 to be the only interface? Right now I just firewalled the 174.x.x.x network from the array but that causes a long delay as it times out at boot time, not a very elegant fix. I did delete all traces of earlier discovered targets with the 174.x.x.x IP.

 

On Thursday, April 4, 2013 2:46:12 PM UTC-4, gee...@gmail.com wrote:

Hello,
I have been trying to setup an iSCSI storage array for several weeks now. I have some things working but others are not. I am running out of time and am willing to to hire someone with the skill and experience to help me out. If this kind of request is not accepted on this list I apologize and will stop posting at the first notice I receive. If it's ok then please respond.

Here is what I have and need help with. I have purchased a Synology DS1812+, running the latest DSM management software and configured with 14 TB of storage. I want to use the array for my  /home and /backup partitions on my web hosting servers. There will be about 3-4 servers using it for this purpose. The individiual servers will be running CentOS, various versions, and open-iscsi as the initiator. I have purchaes a few ToE cards (the broadcom 5709) for the physical connections. I am currently working on a new server (centos 6.4) and testing the concept out. I have been able to connect using the server's onboard NIC but not the ToE card. I have played with the iface config but it refuses to connect.

I am not going to post the details as I am hoping to have someone help me out with this. I am not a very experienced Linux administrator but I can manage with proper instruction.

Thanks for reading
jservices

[Sebastian Riemer]

On 08.04.2013 13:25, lian...@gmail.com wrote:
> Created and iface config with the following:
>
> # BEGIN RECORD 6.2.0-873.2.el6
> ace.iscsi_ifacename = eth2
> iface.transport_name = bnx2i
> iface.hwaddress = 00:10:18:F3:43:00
> iface.ipaddress = 192.x.x.x
> iface.bootproto = static
> iface.subnet_mask = 255.255.255.0
> iface.vlan_id = 0
> iface.vlan_priority = 0
> iface.iface_num = 0
> iface.mtu = 9000
> iface.port = 0
> # END RECORD
>

[...]

>
> # iscsiadm -m session -o show
> tcp: [1] 192.x.x.x:3260,0 iqn.2000-01.com.synology:san.quincy
> Is the 'tcp' telling me it is not properly setup? should it be bnx2i at
> the beginning of the above result?
>

Yes, that's wrong. It has been discovered with the default transport
name. So you have to tell open-iscsi which interface to use (e.g. by
editing the discovered data base entry in /etc/iscsi/nodes). You can
also use iscsiadm.

Change this line:
iface.transport_name = tcp

Into:
iface.transport_name = bnx2i

We already had this situation with iSER. There it is the same.

Cheers,
Sebastian

[Eddie Wai]

This is true. You should have 2 MACs and 2 SAN MACs.
The SAN MACs are the ones you need to specify in the iface config file
if you choose to specify the iface.hwaddress explicity.

You should be able to find the corresponding SAN MACs
under /sys/class/iscsi_host.

> # iscsiadm -m session -o show
> tcp: [1] 192.x.x.x:3260,0 iqn.2000-01.com.synology:san.quincy
>
> Is the 'tcp' telling me it is not properly setup? should it be bnx2i
> at the beginning of the above result?

Yes, the bnx2i transport name would be indicated upon a successful
offload connection.

>
> I was also under the impression that when I created an iface config
> file it would act as the default and only connection. I say this
> because the connection with the server's nic is still appearing when I
> reboot. How do I stop this from happening and force the 5709 to be the
> only interface? Right now I just firewalled the 174.x.x.x network from
> the array but that causes a long delay as it times out at boot time,
> not a very elegant fix. I did delete all traces of earlier discovered
> targets with the 174.x.x.x IP.

If the node created already contains the correct transport_name, then
perhaps the MAC address specified was not the SAN MAC. This would lead
to a fallback to use software iSCSI for your connection.

> --
> You received this message because you are subscribed to the Google
> Groups "open-iscsi" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to open-iscsi+...@googlegroups.com.
> To post to this group, send email to open-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/open-iscsi?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[lian...@gmail.com]

I have looked in this location but found an empty folder. Can someone provide a way to obtain the actual SAN mac address form a broadcom 5709 ToE card? Without it I can't seem to move forward.

Thanks!