HA with DRBD over 10-gig ethernet - how to set up ALUA ?

[Bill Lambert]

I'm trying to set up 2 nodes to replicate as per Marc's walkthrough , except I'm using 10-gig ethernet instead of FC as the point-to-point replication interface (Intel X540 adaptors).  I'm a little puzzled at the point where ALUA is configured, specifically this series of commands:

scstadmin -add_dgrp esos
scstadmin -add_tgrp local -dev_group esos
scstadmin -set_tgrp_attr local -dev_group esos -attributes group_id=1
scstadmin -add_tgrp_tgt 50:01:43:80:21:df:9b:4c -dev_group esos -tgt_group local
scstadmin -set_ttgt_attr 50:01:43:80:21:df:9b:4c -dev_group esos -tgt_group local \
-attributes rel_tgt_id=1
scstadmin -add_tgrp remote -dev_group esos
scstadmin -set_tgrp_attr remote -dev_group esos -attributes group_id=2
scstadmin -add_tgrp_tgt 50:01:43:80:21:df:c7:f4 -dev_group esos -tgt_group remote
scstadmin -set_ttgt_attr 50:01:43:80:21:df:c7:f4 -dev_group esos -tgt_group remote \
-attributes rel_tgt_id=2

(emphasis mine)

My experience with FC is admittedly nil, but I recognize those are WWNs.  That's the only place in the tutorial where such WWNs are used.  Should I be replacing them with my usual "iqn.2013-11.esos.hostname:abcde -driver iscsi" target descriptors ?

Cheers
-Bill

[Marc Smith]

Hi Bill,

Yes, they should be the target names (so "iqn.2013-11.esos.hostname:abcde" in your string). If I remember correctly, the target names on the opposite host (eg, the remote host from one machine) doesn't actually matter, as it doesn't have to be the real remote target name, just something that isn't a local target name... I made them the real remote target names just to keep my sanity as looking at that is quite confusing. =)

And just to clarify, you're using 10 GbE iSCSI as your front-side target interface (that initiators will connect to), not Fibre Channel. You're still using IP (10 GbE or whatever) for the DRBD replication link. Right?

Let me know if you have any other questions.

--Marc

--
You received this message because you are subscribed to the Google Groups "esos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to esos-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Bill Lambert]

Yep, it's a dual port 10gbe card, one for iSCSI to a few small ESXi boxes, and a crossover cable for the DRBD link.  We're actually using gigabit on the initiators for the time being, simply because I have yet to upgrade the NICs and switch.

Thanks for the speedy reply!
-Bill

[Bill Lambert]

Interesting little quirk I found.

Upon running scstadmin -set_ttgt_attr iqn.2013-11.esos.hy-san-02:c76d7 -dev_group esos -tgt_group local -attributes rel_tgt_id=1

scstadmin barks:     WARNING: Target 'esos/local/iqn.2013-11.esos.hy-san-02:c76d7' lacks the settable attribute 'rel_tgt_id', ignoring.

But if I do it through sysfs, it accepts the rel_tgt_id just fine.  I simply used echo 2 > /sys/kernel/scst_tgt/targets/iscsi/$IQN/rel_tgt_id

This only happens when setting the local attribute.  The remote one goes through just fine via scstadmin.

-Bill


On Tuesday, November 12, 2013 12:27:44 PM UTC-5, Marc Smith wrote:

[Marc Smith]

Interesting... I'll look upstream and see if there were any changes/fixes for the scstadmin tool. What revision of ESOS are you using?

--Marc

[Bill Lambert]

I'm on r548

[Marc Smith]

Hey Bill,

I do recall having the same issue and I just verified it on one of my test systems, and get the same result for Fibre Channel too. In my haste to finish that HA project / article, I didn't document the issue (or report it upstream)... my bad.

I looked at the scstadmin tool and discovered why it behaves that way and posted upstream: http://sourceforge.net/mailarchive/forum.php?thread_name=5283CA70.1000103%40acm.org&forum_name=scst-devel

So, its expected behavior, and the proper way to set the attribute for a local type group would be like this:

scstadmin -set_tgt_attr iqn.2013-11.esos.hy-san-02:c76d7 -driver iscsi -attributes rel_tgt_id=1

(A local target group target is the same as a real local target name -- its a sym. link to it.)

I'll update the blog article. Thanks for the reminder. =)

--Marc

[Bill Lambert]

Very cool, thanks for following up.  Looks like I owe you another beer :)

Thanks
-Bill

[Marc Smith]

Actually, I just looked at the SCST list again, and saw this from Bart later that day:

--snip--
Hello Marc,

It turned out to be easy to make -set_ttgt_attr work for local targets without modifying the scstadmin behavior for saving and restoring configuration files. This functionality has been added in r5106. As usual, further feedback is welcome.

Bart.

--snip--

So, he made it so the scstadmin command will work as we expected and its in r5106... which will be in ESOS eventually. =)

--Marc