[Rocks-Discuss] How to add CONNECTED_MODE=yes to ifcfg-ib0 in the database
Go Yoshimura
How can we add CONNECTED_MODE=yes to ifcfg-ib0 in the database?
- We want to use IPoIB in connected mode.
- For it, we need to add CONNECTED_MODE=yes to /etc/sysconfig/network-scripts/ifcfg-ib0
- Now we add CONNECTED_MODE=yes to /etc/sysconfig/network-scripts/ifcfg-ib0 by hand.
But "rocks sync network" and so on will clear CONNECTED_MODE=yes.
- We believe we should do something so that we get desired output with "rocks report host interface compute-0-0 iface=ib0"
- We can add MTU=9000 with "rocks set network mtu"
We are happy if we can add CONNECTED_MODE=yes.
- We are interested in rocks5.4 with servicepack2.
thank you
go
-----
((version))
[root@rocks54g4 rocksadd]# cat /etc/rocks-release
Rocks release 5.4 (Maverick)
((rocks report host interface:without CONNECTED_MODE=yes))
[root@rocks54g4 rocksadd]# rocks report host interface compute-0-0 iface=ib0
DEVICE=ib0
IPADDR=10.1.2.254
NETMASK=255.255.255.0
BOOTPROTO=static
ONBOOT=yes
MTU=9000
[root@rocks54g4 rocksadd]# rocks list host interface compute-0-0
SUBNET IFACE MAC IP NETMASK MODULE NAME VLAN OPTIONS CHANNEL
private eth0 00:30:48:c9:a2:6a 10.1.3.254 255.255.255.0 ------ compute-0-0 ---- ------- -------
------- eth1 00:30:48:c9:a2:6b ---------- ------------- ------ -------------- ---- ------- -------
ib ib0 ----------------- 10.1.2.254 255.255.255.0 ------ compute-0-0-ib ---- ------- -------
[root@rocks54g4 rocksadd]# rocks list network
NETWORK SUBNET NETMASK MTU DNSZONE SERVEDNS
10g: 10.1.1.0 255.255.255.0 9000 10g.local False
ib: 10.1.2.0 255.255.255.0 9000 ib.local False
private: 10.1.3.0 255.255.255.0 1500 local True
public: 192.168.33.0 255.255.255.0 1500 test.local False
----
Go Yoshimura <go-yos...@sstc.co.jp>
Scalable Systems Co., Ltd. <http://www.sstc.co.jp/>
Osaka Office HONMACHI-COLLABO Bldg. 4F, 4-4-2 Kita-kyuhoji-machi, Chuo-ku, Osaka 541-0057 Japan
Tel: 81-6-6224-4115
Tokyo Kojimachi Office BUREX Kojimachi 11F, 3-5-2 Kojimachi, Chiyoda-ku, Tokyo 102-0083 Japan
Tel: 81-3-5875-4718 Fax: 81-3-3237-7612
"Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D."
hi
it seems that this will be a new features request,
by looking at codes of mysql table, one need to new item for CONNECTED_MODE
regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: laotsao.vcf
Type: text/x-vcard
Size: 653 bytes
Desc: not available
Url : https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20110616/656d0767/laotsao.vcf
Anoop Rajendra
Add the line
CONNECTED_MODE=yes
to the file.
Restart your networking.
# service network restart
Let me know if that works for you.
By the way, this really isn't a documented feature, and I found this
out by experimenting.
So YMMV.
To make this change permanent over re-installs, you can add this to
your local node xml files (eg. extend-compute.xml)
-a
Go Yoshimura
Thank you for your helps.
With your advices, we can do what we want.
- As for ifcfg-ib0-local,
"/etc/sysconfig/network-scripts/ifcfg-ib0-local" works fine!
We are not sure how this works exactly
but source_config () in /etc/sysconfig/network-scripts/network-functions seems to play an important part
to enable that ifup ifcfg-ib0-local override ib0 configuration.
- As for permanence,
we have desided that ifcfg-ib0-local should be created by IPoIB.sh scripts rather than roll itself.
Because
(1)we will get error like "Missing config file ifcfg-ib0" when we have ifcfg-ib0-local but we don't have ifcfg-ib0
(2)ifcfg-ib0-local is only required when mtu > 2044 (IPoIB.sh sets mtu=9000)
((ifcfg-ib0/ifcfg-ib0-local))
[root@compute-0-0 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ib0-local
CONNECTED_MODE=yes
[root@compute-0-0 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ib0
DEVICE=ib0
IPADDR=10.1.2.254
NETMASK=255.255.255.0
BOOTPROTO=static
ONBOOT=yes
MTU=9000
[root@compute-0-0 ~]# ifconfig ib0
ib0 Link encap:InfiniBand HWaddr 80:00:00:48:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00
inet addr:10.1.2.254 Bcast:10.1.2.255 Mask:255.255.255.0
inet6 addr: fe80::230:48c9:7de8:1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:16 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:5 overruns:0 carrier:0
collisions:0 txqueuelen:256
RX bytes:1092 (1.0 KiB) TX bytes:616 (616.0 b)
((IPoIB.sh))
#### subnet name
rocks add network ib 10.1.2.0 255.255.255.0 dnszone=ib.local mtu=9000
rocks open firewall network=ib service=all protocol=all
#### node loop
for nodes in `rocks list host | grep Compute | cut -f 1 -d :` ; do
###### IP of IB
IBip=`rocks list host interface $nodes | grep private | awk '{print $4}'| sed {s/10\.1\.1/10\.1\.2/g}`
###### assign device ib0
rocks add host interface ${nodes} ib0
###### assign IP of IB
rocks set host interface ip ${nodes} iface=ib0 ip=${IBip}
###### assign subnet of IB
rocks set host interface subnet ${nodes} iface=ib0 ib
###### ifcfg-ib0-localassign for connectmode
rocks run host ${nodes} command="cp /var/sstctool/ifcfg-ib0-local /etc/sysconfig/network-scripts/ifcfg-ib0-local"
done
((Missing config file ifcfg-ib0))
[root@rocks54g4 rocksadd]# service network restart
Shutting down interface eth1: [ OK ]
Shutting down interface eth2: [ OK ]
Shutting down loopback interface: [ OK ]
Disabling IPv4 packet forwarding: net.ipv4.ip_forward = 0
[ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth1: [ OK ]
Bringing up interface eth2: [ OK ]
Bringing up interface ib0-local: Missing config file ifcfg-ib0.
[FAILED]
thank you
go
---