diskless / remote boot with open-iscsi?
Tomasz Chmielewski
I know that in theory it should look like below:
1) load a kernel via tftp
2) load initrd via tftp
3) initrd (containing iscsid etc.) would mount the node, and start the
system from a iscsi target
In reality, I'm having problems creating such a initrd.
There were also some discussions here on the list, but also, mostly
theoretical or concerning other software than open-iscsi.
Does anyone have any good experiences setting up diskless / remote boot
with open-iscsi?
--
Tomasz Chmielewski
http://wpkg.org
Mike Christie
The distros like SUSE and Red Hat are working on it. I think SLES 10
supports it. FC6 sort of supports it - there are lots of bugs still. FC6
uses iscsistart btw.
Tomasz Chmielewski
(...)
>> Does anyone have any good experiences setting up diskless / remote boot
>> with open-iscsi?
>>
>
> The distros like SUSE and Red Hat are working on it. I think SLES 10
> supports it. FC6 sort of supports it - there are lots of bugs still. FC6
> uses iscsistart btw.
Is there any documentation to "iscsistart" other than "iscsistart.c"?
Is "iscsistart" an equivalent to "iscsid", but only capable to connect
one session etc. (making it a good tool for initrd).
I noticed it's statically linked, so perhaps my guesses are right?
# file usr/iscsistart
usr/iscsistart: ELF 32-bit LSB executable, Intel 80386, version 1
(SYSV), for GNU/Linux 2.6.9, statically linked, for GNU/Linux 2.6.9, not
stripped
Tomasz Chmielewski
> Mike Christie wrote:
>
> (...)
>
>>> Does anyone have any good experiences setting up diskless / remote boot
>>> with open-iscsi?
>>>
>> The distros like SUSE and Red Hat are working on it. I think SLES 10
>> supports it. FC6 sort of supports it - there are lots of bugs still. FC6
>> uses iscsistart btw.
>
> Is there any documentation to "iscsistart" other than "iscsistart.c"?
All right, the syntax is quite simple:
iscsistart \
-i "iqn.2007-01.net.syneticon:server.how-we-name-ourselves" \
-t "iqn.2006-10.net.syneticon:storage.name-on-a-target-we-connect-to" \
-g 1 \
-a ISCSI_TARGET_IP
What is the meaning of "-g" parameter?
-g, --tgpt=N set target portal group tag to N (Required)
BTW, there is a typo in the explanation of "-a" ("addres" instead of
"addresS"):
-a, --address=A.B.C.D set IP addres to A.B.C.D (Required)
Tomasz Chmielewski
So I tried to do it via initrd, without much success.
When I start "iscsistart" from initrd, it complains:
# iscsistart \
-i "iqn.2007-01.net.syneticon:server.how-we-name-ourselves" \
-t "iqn.2006-10.net.syneticon:storage.name-on-a-target-we-connect-to" \
-g 1 \
-a ISCSI_TARGET_IP
iscsistart: transport class version 2.0-754. iscsid version 2.0-754
iscsistart: version 2.0-754
scsi2 : iSCSI Initiator over TCP/IP
iscsi : couldn't create a new connection for session 0
iscsistart: received iferror -12
iscsistart: can't create connection (115)
iscsistart: initiator reported error (9 - internal error)
The same command line started from a "full" system works as expected.
The network is working fine (I can ping the iscsi target), I have nodes
created in /dev.
Any ideas what's wrong?
Tomasz Chmielewski
(...)
> iscsistart: transport class version 2.0-754. iscsid version 2.0-754
> iscsistart: version 2.0-754
> scsi2 : iSCSI Initiator over TCP/IP
> iscsi : couldn't create a new connection for session 0
> iscsistart: received iferror -12
> iscsistart: can't create connection (115)
> iscsistart: initiator reported error (9 - internal error)
>
>
> The same command line started from a "full" system works as expected.
>
> The network is working fine (I can ping the iscsi target), I have nodes
> created in /dev.
>
> Any ideas what's wrong?
I was missing libcrc32c and crc32c modules loaded.
Right now, I'm able to access a remote iscsi session, from initrd, great!
Still it needs some polishing to get a iscsi filesystem mounted as a
local root automatically.
Mike Christie
> Mike Christie wrote:
>
> (...)
>
>>> Does anyone have any good experiences setting up diskless / remote boot
>>> with open-iscsi?
>>>
>> The distros like SUSE and Red Hat are working on it. I think SLES 10
>> supports it. FC6 sort of supports it - there are lots of bugs still. FC6
>> uses iscsistart btw.
>
> Is there any documentation to "iscsistart" other than "iscsistart.c"?
>
oops, the man page Robin made never got svn added. I will do that in a
minute here.
Mike Christie
> Mike Christie wrote:
>
> (...)
>
>>> Does anyone have any good experiences setting up diskless / remote boot
>>> with open-iscsi?
>>>
>> The distros like SUSE and Red Hat are working on it. I think SLES 10
>> supports it. FC6 sort of supports it - there are lots of bugs still. FC6
>> uses iscsistart btw.
>
> Is there any documentation to "iscsistart" other than "iscsistart.c"?
>
> Is "iscsistart" an equivalent to "iscsid", but only capable to connect
> one session etc. (making it a good tool for initrd).
> I noticed it's statically linked, so perhaps my guesses are right?
>
Yeah, it just makes a single session then exits. It is only suitable for
temporary use like the pivot root and system startup because the
userspace part of open-iscsi does the error handling so you do not want
to be without for very long.
Mike Christie
iscsistart does or the mkinitrd and init scripts? iscsistart is just a
tool that distros can use for their initramfs tools. If you run FC's
mkinitrd it would, search for a root iscsi disk and if found, throw in
the networking tools and modules and iscsi tools and modules (FC's tools
need a update though). If you want to write a common script for all
distros go ahead and send it.
The distros are also modifiying their installers so you can install
directly to the iscsi disk, so there is no need for local installs and
hand modifying files. FC6 is broken right now though. Maybe SUSE's is
working.
Tomasz Chmielewski
>> Right now, I'm able to access a remote iscsi session, from initrd, great!
>>
>> Still it needs some polishing to get a iscsi filesystem mounted as a
>> local root automatically.
>>
>
> iscsistart does or the mkinitrd and init scripts? iscsistart is just a
> tool that distros can use for their initramfs tools. If you run FC's
> mkinitrd it would, search for a root iscsi disk and if found, throw in
> the networking tools and modules and iscsi tools and modules (FC's tools
> need a update though). If you want to write a common script for all
> distros go ahead and send it.
All right. I have it up and running.
In a couple of days I'll write some theory on how to do a diskless boot
with open-iscsi.
This will include running Xen hosts (dom0), they need some slight
different handling if we want to start them via pxe+open-iscsi, because
of how Xen operates (it changes network settings, takes interfaces down
and up during start, which is not what iscsi likes).
matteve...@googlemail.com
if you are looking for ready-made, open-source solution i would
like to recommend to check the openQRM project (http://
www.openqrm.org/).
It supports deploying Xen-hosts via Iscsi-boot and also features to
attach and
manage the Xen-partitions via Iscsi.
all the best,
Matt
>
> --
> Tomasz Chmielewskihttp://wpkg.org
rhl...@tfc.cz
On Jan 30, 6:32 pm, Mike Christie <micha...@cs.wisc.edu> wrote:
> > Is "iscsistart" an equivalent to "iscsid", but only capable to connect
> > one session etc. (making it a good tool for initrd).
> > I noticed it's statically linked, so perhaps my guesses are right?
>
> Yeah, it just makes a single session then exits. It is only suitable for
> temporary use like the pivot root and system startup because the
> userspace part of open-iscsi does the error handling so you do not want
> to be without for very long.
I'm very concerned about this part.
I'm setting up FC6 diskless booting and everything works fine now.
Initrd starts network, iscsistart connects the target and FC6 boots.
I would like to know whether running iscsi daemon (service iscsi
start) is enough to upgrade the connection made by iscsistart to "full
and complete" state. What about initiator name, should it be specified
in /etc/iscsi/initiatorname ? Should it be the same or different from
initiator name used by iscsistart? Must be the node and/or target in
the iscsi daemon database? Do I have to call something like --relogin?
And just to make this post a little useful to others, here's my way of
creating initrd. I've slightly modified FC6's mkinitrd script
(hopefully I will be able to post some patch later) to add iscsitarget
binary and call it with proper parameters. I've just hardcoded them
into the script. The rest part could be done with options --net-
dev=eth0 and a lot of --with (iscsi,network card driver,...). Creating
extra fstab is also good idea but --root-dev may be enough.
And for others who may encounter the same problem: See bug 225479 on
bugzilla.redhat.com if your network is not starting from initrd, I've
had to replace nash network command with busybox's ifconfig.
Prasanna Mumbai
I am getting the following error.
Kernel panic – not syncing: VFS: Unable to mount root fs on unknown-block(0, 0) .
I looked into the /boot partation and didn't find the initrd.img file so reinstalled the system & before rebooting thought i would create the initrd file manually and used the following command to do that:
/sbin/mkinitrd -f /boot/initrd-2.6.18-1.2798.fc6.img initrd-2.6.18-1.2798.fc6
iscsiadm : exiting due to configuration error
Unable to find iscsi record for /sys/devices/platform/host2/session0/target2:0:0/2:0:0:0
Is that the way I am trying to create the initrd file right? if not please let me know the proper way.
If there is any README or HOWTO for diskless install please give me pointer to that.
Albert Pauw
> am getting the following error:
> iscsiadm : can not connect to iscsi daemon!
> iscsiadm : exiting due to configuration error
Looks like the iscsid daemon is not running.
Try "service open-iscsi start" before doing the mkinitrd.
> Unable to find iscsi record for
> /sys/devices/platform/host2/session0/target2:0:0/2:0:0:0
>
> Is that the way I am trying to create the initrd file right? if not please
> let me know the proper way.
>
> If there is any README or HOWTO for diskless install please give me pointer
> to that.
I second that. It would be nice to have a step-by-step procedure how
to do it.
This could then be added into a README or something.
Mind you, you have to do mkinitrd again when the kernel gets updated
(I think).
Albert
Tomasz Chmielewski
Hi,
For your convenience, I created such a HOWTO:
http://wpkg.org/index.php/Diskless_/_remote_boot_with_Open-iSCSI
I believe it's a step-by-step procedure :) - if not, let me know, or
just correct the things yourself (it's a wiki).
Additionally, it contains example downloads (if you're just curious if
it works), and some info on how to start a Xen server via open-iscsi / PXE.
Prasanna Mumbai
Thanks for the deatiled mail and WIKI page.
I creating the initrd file by following the wiki page, but after that also my system is giving kernel panic error.
the logs seem like this:
Booting 'Fedora Core(2.6.18-1.2798.fc6)'
root(hd0, 0)
Filesystem type is ext2fs, partition type 0x83
kernel /vmlinuz-2.6.18-1.2798.fc6 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
[Linux-bzImage, setup=0x1e00, size=ox1b1b3c]
Uncompressing Linux... OK, booting the kernel.
PCI: BIOS Bug : MCFG area at f0000000 is not E820-reserved
PCI: Not using MMCONFIG.
ACPI: Getting cpuindex for acpiid 0x3.
ACPI: Getting cpuindex for acpiid 0x4.
Kernel panic - not syncing: VFS: unable to mount root fs on unknown-block(0,0)
Please let me know what might be the problem.
~Prasanna
Tomasz Chmielewski
> Hi Tomas,
> Thanks for the deatiled mail and WIKI page.
> I creating the initrd file by following the wiki page, but after that
> also my system is giving kernel panic error.
>
> the logs seem like this:
>
> Booting 'Fedora Core(2.6.18-1.2798.fc6)'
>
> root(hd0, 0)
> Filesystem type is ext2fs, partition type 0x83
> kernel /vmlinuz-2.6.18-1.2798.fc6 ro root=/dev/VolGroup00/LogVol00 rhgb
> quiet
> [Linux-bzImage, setup=0x1e00, size=ox1b1b3c]
This looks like grub, not like starting from tftp... Are you sure your
system indeed starts from PXE/tftp?
> Uncompressing Linux... OK, booting the kernel.
> PCI: BIOS Bug : MCFG area at f0000000 is not E820-reserved
> PCI: Not using MMCONFIG.
> ACPI: Getting cpuindex for acpiid 0x3.
> ACPI: Getting cpuindex for acpiid 0x4.
> Kernel panic - not syncing: VFS: unable to mount root fs on
> unknown-block(0,0)
>
> Please let me know what might be the problem.
Did you try using the images I provided? You might want to start
experimenting with them, first.
Prasanna Mumbai
Yes i tried with the images provide in the Link, it did work.
Prasanna Mumbai
please ignore my earlier mail, i tried with the images provided in the link, but no luck, it didn't work.
~Cheers
Prasanna
On 2/13/07, Tomasz Chmielewski < man...@wpkg.org> wrote:
Prasanna Mumbai schrieb:
> Hi Tomas,
> Thanks for the deatiled mail and WIKI page.
> I creating the initrd file by following the wiki page, but after that
> also my system is giving kernel panic error.
>
> the logs seem like this:
>
> Booting 'Fedora Core(2.6.18-1.2798.fc6)'
>
> root(hd0, 0)
> Filesystem type is ext2fs, partition type 0x83
> kernel /vmlinuz- 2.6.18-1.2798.fc6 ro root=/dev/VolGroup00/LogVol00 rhgb
> quiet
> [Linux-bzImage, setup=0x1e00, size=ox1b1b3c]
This looks like grub, not like starting from tftp... Are you sure your
system indeed starts from PXE/tftp?
> Uncompressing Linux... OK, booting the kernel.
> PCI: BIOS Bug : MCFG area at f0000000 is not E820-reserved
> PCI: Not using MMCONFIG.
> ACPI: Getting cpuindex for acpiid 0x3.
> ACPI: Getting cpuindex for acpiid 0x4.
> Kernel panic - not syncing: VFS: unable to mount root fs on
> unknown-block(0,0)
>
> Please let me know what might be the problem.
Did you try using the images I provided? You might want to start
experimenting with them, first.
Tomasz Chmielewski
> Hey Thomas,
>
> please ignore my earlier mail, i tried with the images provided in the
> link, but no luck, it didn't work.
And where the error was?
Because the log you shown earlier was clearly from a machine starting
using grub, and not from a machine starting using PXE.
Prasanna Mumbai
Prasanna Mumbai schrieb:
> Hey Thomas,
>
> please ignore my earlier mail, i tried with the images provided in the
> link, but no luck, it didn't work.
And where the error was?
Because the log you shown earlier was clearly from a machine starting
using grub, and not from a machine starting using PXE.
Ya, i amn't using PXE booting, i am using GRUB, Is the solution provided only for PXE booting, has anyone tested for GRUB?
Do i need to change something in the initrd file(created using the wiki page) if i am using GRUB?
Tomasz Chmielewski
>
>
> On 2/13/07, *Tomasz Chmielewski* <man...@wpkg.org
>
> Prasanna Mumbai schrieb:
> > Hey Thomas,
> >
> > please ignore my earlier mail, i tried with the images provided
> in the
> > link, but no luck, it didn't work.
>
> And where the error was?
>
> Because the log you shown earlier was clearly from a machine starting
> using grub, and not from a machine starting using PXE.
>
>
> Ya, i amn't using PXE booting, i am using GRUB, Is the solution
> provided only for PXE booting, has anyone tested for GRUB?
> Do i need to change something in the initrd file(created using the wiki
> page) if i am using GRUB?
You'd need to pass the correct options to the kernel, as are passed by
tftp server, for example:
initrd=remote/initrd.img iscsi_i=iqn.2007-01.com.example:server.remote1
iscsi_i_ip=10.1.1.165/255.255.255.192
iscsi_t=iqn.2007-01.com.example:storage.remote1 iscsi_a=10.1.1.180
Prasanna Mumbai
Prasanna Mumbai schrieb:
>
>
> On 2/13/07, *Tomasz Chmielewski* <man...@wpkg.org
> <mailto:man...@wpkg.org>> wrote:
>
> Prasanna Mumbai schrieb:
> > Hey Thomas,
> >
> > please ignore my earlier mail, i tried with the images provided
> in the
> > link, but no luck, it didn't work.
>
> And where the error was?
>
> Because the log you shown earlier was clearly from a machine starting
> using grub, and not from a machine starting using PXE.
>
>
> Ya, i amn't using PXE booting, i am using GRUB, Is the solution
> provided only for PXE booting, has anyone tested for GRUB?
> Do i need to change something in the initrd file(created using the wiki
> page) if i am using GRUB?
You'd need to pass the correct options to the kernel, as are passed by
tftp server, for example:
initrd=remote/initrd.img iscsi_i=iqn.2007-01.com.example:server.remote1
iscsi_i_ip=10.1.1.165/255.255.255.192
iscsi_t=iqn.2007-01.com.example:storage.remote1 iscsi_a=10.1.1.180
ya, I have passed these options properly.
Charles Chou
Guenther
Thank you for the manual, I was searching for about 2 months for a
documentation like this.
One question: How do you get your linux-installation on the iscsi-
volume? Do you use imaging, copy, ...? SLES 10 can be installed with
the parameter withiscsi=1 to an iscsi-target, but after the first
boot, when yast configures the installation, my system hanged.
thank you
guenther
Prasanna Mumbai
Hi Tomasz,
Thank you for the manual, I was searching for about 2 months for a
documentation like this.
One question: How do you get your linux-installation on the iscsi-
volume? Do you use imaging, copy, ...? SLES 10 can be installed with
the parameter withiscsi=1 to an iscsi-target, but after the first
boot, when yast configures the installation, my system hanged.
I had the similar problem of system hanging, It was because i was using static IP. If you are also using some static IP then you can try out automatic network configuration using DHCP.