[Rocks-Discuss] IPMI configuration
Joerg Hohwieler
I followed the slides [0] about the IPMI configuration.
# rocks list network
NETWORK SUBNET NETMASK MTU
private: 10.1.0.0 255.255.0.0 1500
public: 192.168.3.0 255.255.255.0 1500
ipmi: 10.3.0.0 255.255.0.0 1500
I added the IPMI interface like:
# rocks add host interface compute-0-0 iface=ipmi ip=10.3.255.254 subnet=ipmi gateway=1 name=ipmi-0-0
Since I'm using SuperMicro board gateway=1 is correct, but the gateway
parameter doesn't show up in the list below:
# rocks list host interface compute-0-0
SUBNET IFACE MAC IP NETMASK MODULE NAME VLAN
private eth0 00:30:48:c8:7b:b6 10.1.255.254 255.255.0.0 igb compute-0-0 ----
------- eth1 00:30:48:c8:7b:b7 ------------ ----------- igb ----------- ----
ipmi ipmi ----------------- 10.3.255.254 255.255.0.0 ------ ipmi-0-0 ----
After reinstalling this node, the following file is exists on that host:
[root@compute-0-0 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ipmi
DEVICE=ipmi
IPADDR=10.3.255.254
NETMASK=255.255.0.0
BOOTPROTO=static
ONBOOT=yes
MTU=1500
Following the slides, there should now be file named like ipmi-1 that contains
lines with ipmitool settings.
What's going wrong there?
Best regards
Joerg
[0] - http://www.rocksclusters.org/rocksapalooza/2009/customizing.pdf
Philip Papadopoulos
this is where reality of newer code hits the (now) incorrect text of older
slides.
instead of gateway, set the module.
eg.
rocks set host interface module compute-0-0 iface=ipmi module=1
if you do a
rocks report host interface compute-0-0, you should then see a section that
looks like
<file name="/etc/sysconfig/ipmi-settings">
ipmitool lan set 1 ipaddr 192.168.20.61
ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 arp respond on
ipmitool user set password 1 admin
ipmitool lan set 1 access on
ipmitool lan set 1 user
ipmitool lan set 1 auth ADMIN PASSWORD
</file>
These are the ipmitool commands to set up the BMC.
If you then did a
# rocks sync host interface compute-0-0
# ssh compute-0-0 "sh /etc/sysconfig/ipmi-settings"
you should be set up.
You -may- have to run the following on the node (Compute-0-0)
ipmitool enable user 1
to make remote access possible.
-P
--
Philip Papadopoulos, PhD
University of California, San Diego
858-822-3628 (Ofc)
619-331-2990 (Fax)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20100113/6a4a89bf/attachment.html
Joerg Hohwieler
thanks for your help.
I've run the following sequence:
# rocks remove host interface compute-0-0 iface=ipmi
# rocks report host interface compute-0-0 > interface.old
# rocks add host interface compute-0-0 iface=ipmi ip=10.3.255.254 subnet=ipmi module=1 name=ipmi-0-0
# rocks report host interface compute-0-0 > interface.new
# diff -uNp interface.old interface.new
--- interface.old 2010-01-14 08:20:02.000000000 +0100
+++ interface.new 2010-01-14 08:20:12.000000000 +0100
@@ -34,3 +34,26 @@ BOOTPROTO=static
ONBOOT=yes
MTU=1500
</file>
+<file name="/etc/sysconfig/ipmi-settings">
+ipmitool lan set 1 ipaddr 10.3.255.254
+ipmitool lan set 1 netmask 255.255.0.0
+ipmitool lan set 1 arp respond on
+ipmitool user set password 1 admin
+ipmitool lan set 1 access on
+ipmitool lan set 1 user
+ipmitool lan set 1 auth ADMIN PASSWORD
+</file>
+<![CDATA[
+grep -v "\<ipmi\>" /etc/modprobe.conf > /tmp/modprobe.conf
+echo 'alias ipmi 1' >> /tmp/modprobe.conf
+mv /tmp/modprobe.conf /etc/modprobe.conf
+chmod 444 /etc/modprobe.conf
+]]>
+<file name="/etc/sysconfig/network-scripts/ifcfg-ipmi">
+DEVICE=ipmi
+IPADDR=10.3.255.254
+NETMASK=255.255.0.0
+BOOTPROTO=static
+ONBOOT=yes
+MTU=1500
+</file>
As you can see, there is also an ifcfg-ipmi file.
There are some error messages at the first boot saying that "get
channel info command" failed. Maybe this is the reason why the
ipmi-settings file will be renamed to:
# file /etc/sysconfig/ipmi-settings.bak
/etc/sysconfig/ipmi-settings.bak: ASCII text
On the second boot it just complains about an non existing ipmi
interface, caused by the existance of ifcfg-ipmi file.
Is it possible to prevent ROCKS from creating this file?
Furthermore in my case it would be sufficient to just run:
ipmitool lan set 1 ipaddr 10.3.255.254
ipmitool lan set 1 netmask 255.255.0.0
How can I get ROCKS to just create these two lines for the ipmi-settings
file?
Best regards
Joerg
Philip Papadopoulos
what is the status of the ipmi service? Is it actually on?
May have missed
>
> # file /etc/sysconfig/ipmi-settings.bak
> /etc/sysconfig/ipmi-settings.bak: ASCII text
>
> On the second boot it just complains about an non existing ipmi
> interface, caused by the existance of ifcfg-ipmi file.
>
Safe to ignore this warning, but we'll address it for the next release.
>
> Is it possible to prevent ROCKS from creating this file?
>
Yes -- but not with the current code. Requires a modification to the
command that creates these files.
>
> Furthermore in my case it would be sufficient to just run:
> ipmitool lan set 1 ipaddr 10.3.255.254
> ipmitool lan set 1 netmask 255.255.0.0
>
> How can I get ROCKS to just create these two lines for the ipmi-settings
> file?
>
You would have to edit the command (python code on the frontend) that
creates the various network files. This is our first foray into integrating
IPMI into the main code, we're trying to treat it like all of our other
network interfaces, but still
need to treat it in a special manner.
Can't really "fix" it without writing new code. The the reality of IPMI is
that the information only needs to be set
once (theoretically) in the lifetime of the machine.
Let me talk with Mason -- he did the IPMI integration.
-P
Joerg Hohwieler
On Thu, 14 Jan 2010 - 17:52, Philip Papadopoulos wrote:
> what is the status of the ipmi service? Is it actually on?
> May have missed
yes, this service will be up.
> > Is it possible to prevent ROCKS from creating this file?
> >
> Yes -- but not with the current code. Requires a modification to the
> command that creates these files.
>
>
> >
> > Furthermore in my case it would be sufficient to just run:
> > ipmitool lan set 1 ipaddr 10.3.255.254
> > ipmitool lan set 1 netmask 255.255.0.0
> >
> > How can I get ROCKS to just create these two lines for the ipmi-settings
> > file?
> >
> You would have to edit the command (python code on the frontend) that
> creates the various network files. This is our first foray into integrating
> IPMI into the main code, we're trying to treat it like all of our other
> network interfaces, but still
> need to treat it in a special manner.
>
> Can't really "fix" it without writing new code. The the reality of IPMI is
> that the information only needs to be set
> once (theoretically) in the lifetime of the machine.
>
> Let me talk with Mason -- he did the IPMI integration.
> -P
I just edited
/opt/rocks/lib/python2.4/site-packages/rocks/commands/report/host/interface/__init__.py
to reflect the changes I need for IPMI and I also removed the creation
if the ifcfg-ipmi file and insertion of ipmi in modprobe.conf. See attached patch.
Here the settings:
# rocks list host interface compute-0-0
SUBNET IFACE MAC IP NETMASK MODULE NAME VLAN
private eth0 00:30:48:c8:7b:b6 10.1.255.254 255.255.0.0 igb compute-0-0 ----
------- eth1 00:30:48:c8:7b:b7 ------------ ----------- igb ----------- ----
ipmi ipmi ----------------- 10.3.255.254 255.255.0.0 1 ipmi-0-0 ----
[root@frontend ~]# rocks report host interface compute-0-0
<![CDATA[
grep -v "\<eth0\>" /etc/modprobe.conf > /tmp/modprobe.conf
echo 'alias eth0 igb' >> /tmp/modprobe.conf
mv /tmp/modprobe.conf /etc/modprobe.conf
chmod 444 /etc/modprobe.conf
]]>
<file name="/etc/sysconfig/network-scripts/ifcfg-eth0">
DEVICE=eth0
HWADDR=00:30:48:c8:7b:b6
IPADDR=10.1.255.254
NETMASK=255.255.0.0
BOOTPROTO=static
ONBOOT=yes
MTU=1500
</file>
<![CDATA[
grep -v "\<eth1\>" /etc/modprobe.conf > /tmp/modprobe.conf
echo 'alias eth1 igb' >> /tmp/modprobe.conf
mv /tmp/modprobe.conf /etc/modprobe.conf
chmod 444 /etc/modprobe.conf
]]>
<file name="/etc/sysconfig/network-scripts/ifcfg-eth1">
DEVICE=eth1
HWADDR=00:30:48:c8:7b:b7
BOOTPROTO=none
ONBOOT=no
MTU=1500
</file>
<file name="/etc/sysconfig/ipmi-settings">
ipmitool lan set 1 ipaddr 10.3.255.254
ipmitool lan set 1 netmask 255.255.0.0
</file>
Would this patch be convenient or would it cause some bad sideeffects I
cannot know of ?
Best regards
Joerg
-------------- next part --------------
--- /opt/rocks/lib/python2.4/site-packages/rocks/commands/report/host/interface/__init__.py.orig 2010-01-17 12:02:15.000000000 +0100
+++ /opt/rocks/lib/python2.4/site-packages/rocks/commands/report/host/interface/__init__.py 2010-01-17 12:07:03.000000000 +0100
@@ -161,15 +161,15 @@ class Command(rocks.commands.HostArgumen
% (channel, ip))
self.addOutput(host, 'ipmitool lan set %s netmask %s'
% (channel, netmask))
- self.addOutput(host, 'ipmitool lan set %s arp respond on'
- % (channel))
- self.addOutput(host, 'ipmitool user set password 1 admin')
- self.addOutput(host, 'ipmitool lan set %s access on'
- % (channel))
- self.addOutput(host, 'ipmitool lan set %s user'
- % (channel))
- self.addOutput(host, 'ipmitool lan set %s auth ADMIN PASSWORD'
- % (channel))
+# self.addOutput(host, 'ipmitool lan set %s arp respond on'
+# % (channel))
+# self.addOutput(host, 'ipmitool user set password 1 admin')
+# self.addOutput(host, 'ipmitool lan set %s access on'
+# % (channel))
+# self.addOutput(host, 'ipmitool lan set %s user'
+# % (channel))
+# self.addOutput(host, 'ipmitool lan set %s auth ADMIN PASSWORD'
+# % (channel))
self.addOutput(host, '</file>')
def writeConfig(self, host, mac, ip, device, netmask, vlanid, mtu):
@@ -201,7 +201,7 @@ class Command(rocks.commands.HostArgumen
def writeModprobe(self, host, device, module):
- if not module:
+ if not module or device == 'ipmi':
return
self.addOutput(host, '<![CDATA[')
@@ -285,6 +285,9 @@ class Command(rocks.commands.HostArgumen
if dev != device:
device = '%s.%d' % (dev, vlanid)
+ if device == 'ipmi':
+ return
+
if self.iface:
if self.iface == device:
self.writeConfig(host, mac, ip, device,