open-iscsi with built-in scsi_transport_iscsi

87 views
Skip to first unread message

alge...@gmail.com

unread,
Aug 3, 2007, 9:31:31 AM8/3/07
to open-iscsi
Hi,
I'm trying to get open-iscsi running on a 2.6.19 kernel without module
support. With built-in scsi_transport_iscsi, /sys/modules/
scsi_transport_iscsi/version is never created, so iscsid refuses to
start. It works fine after I hacked get_iscsi_kernel_version to always
return "2.0-865".

Is it a design decision that the iscsi modules should never be built
into the kernel? If not, there ought to be a better way to check for a
compatible iscsi module.

--
Helge

Mike Christie

unread,
Aug 5, 2007, 11:36:18 PM8/5/07
to open-...@googlegroups.com, alge...@gmail.com

It is a bug. I have been working on different fixes but am not sure
which is best at the moment. The version sysfs attr will actually show
up even if you compile scsi_transport_iscsi into the kernel, but there
is a bug. Well, I am not sure if it is a bug or if it is by design,
because it only works if scsi_transport_iscsi were to define other
module params.

So you could hack scsi_transport_iscsi to define a dummy module_parm and
then the version sysfs attr will also get created even if you compiled
it into the kernel instead of making it a module.

Andreas Balg

unread,
Aug 6, 2007, 4:53:32 PM8/6/07
to open-...@googlegroups.com
One more small issue:

if I use an attached iSCSI-target in a Xen Guest-Domain (=initiator) -
the connection to the target drops during live migration of the Guest
domain. At least the target is mounted read-only afterwards ... Is this
already a known issue?

Cheers
Andreas

Tomasz Chmielewski

unread,
Aug 7, 2007, 1:54:19 PM8/7/07
to open-...@googlegroups.com
Andreas Balg schrieb:

Works fine here.

How long does the migration take?

What's your "node.session.timeo.replacement_timeout" setting? By
default, it's 120 seconds, perhaps you should increase it (although,
with live migration the "disconnection" should usually take just few
seconds).


--
Tomasz Chmielewski
http://wpkg.org

Mike Christie

unread,
Aug 7, 2007, 1:58:00 PM8/7/07
to open-...@googlegroups.com, a.b...@xincs.eu

I have no idea what you said above :)

I am not sure what this is "(=initiator)". Are you running a iscsi
target in one guest and the initiator in another quest, then migrating
the target to a new guest? How long does the migration take? By default
the initiator will send "iscsi pings" which check if the target is
around (default timeout for the iscsi ping is 10 seconds and they are
sent every 15), and if FS/disk IO is running through the iscsi session
during the migration then the each scsi command has a timer (which is 30
seconds by default). So if I understand you right, during the live
migration of the target, if it cannot respond to the iscsi pings or if a
scsi command timesout then we will probably hit the case you are asking
about.

For the iscsi ping issues, you can turn them off by setting the noop
settings to 0. If you are not using dm or md multipath you probably want
to do this anyways (or you might want to set the noop values higher).

For the scsi command timeout issue, you can set that timeout higher by doing

echo X > /sys/block/sdX/device/timeout

Let me know if that is what you were asking about and if it helps.

Mike Christie

unread,
Aug 7, 2007, 2:04:10 PM8/7/07
to open-...@googlegroups.com

Ah yeah, that is another good suggestion.

Let me try to write all this up this time and put it in the README.

Mike Anderson

unread,
Aug 9, 2007, 12:56:59 PM8/9/07
to open-...@googlegroups.com, a.b...@xincs.eu
Mike Christie <mich...@cs.wisc.edu> wrote:
> For the scsi command timeout issue, you can set that timeout higher by doing
>
> echo X > /sys/block/sdX/device/timeout
>
> Let me know if that is what you were asking about and if it helps.
>

Just an FYI. If your distro contains a udev rule for setting timeouts you
may want to update the rule also.
(e.g.,)
# grep timeout 50-udev.rules
RUN+="/bin/sh -c 'echo 60 > /sys$$DEVPATH/timeout'
...

-andmike
--
Michael Anderson
and...@us.ibm.com

mari...@gmail.com

unread,
Aug 29, 2007, 3:51:59 AM8/29/07
to open-iscsi
On Aug 5, 8:36 pm, Mike Christie <micha...@cs.wisc.edu> wrote:

The dummy module_param hack doesn't work either. As far as I know the
MODULE_VERSION macro only adds the version info to a special elf
header in the loadable module. So that info is not available at all
when the module is built in. I have been trying to figure out how
change that, my first thought was to check if the module was built in
and if so manually add a 'version' attribute to the module's kobject
but it appears that there isn't any sane way of getting to the kobject
when the module is built in. (available in the THIS_MODULE structure
otherwise but when it is built in THIS_MODULE is just a null pointer)
My other thoughts are to tack it in somewhere under /sys/class by
creating a new kobject but that seems likely to be frowned upon or I
could try to change the MODULE_VERSION macro so it tucks the version
info away somewhere else so the attribute can be created up for any
module that is built in.

But why is this check needed in the first place? Does the 2.0 version
number indicate an api change from 1.0, and earlier? If that is the
case maybe it would be reasonable to check the kernel version to be
newer than the first mainline release with a 2.0-something module if
the version file is not in sysfs?

My whole goal with all this is so I can A) use the iscsi modules
provided in the mainline kernel because external modules complicate
things and B) use a single kernel image on my systems because modules
in general complicate things.

Mike Christie

unread,
Aug 31, 2007, 1:50:56 PM8/31/07
to open-...@googlegroups.com

Yeah, you could easily add it into iscsi_transport, but we are not
allowed to add our own version info in a new sysfs file. The upstream
scsi requirement is to use the MODULE_VERSION macro so we would need to
fix that.

>
> But why is this check needed in the first place? Does the 2.0 version
> number indicate an api change from 1.0, and earlier? If that is the

Yes.

> case maybe it would be reasonable to check the kernel version to be
> newer than the first mainline release with a 2.0-something module if
> the version file is not in sysfs?

The problem with that is we allow users to use the module distributed on
open-iscsi.org. Those modules are basically what is upstream, but with
some compat code so they can work in older kernels.

The iser guys need something for their 2.6.9 port where the
MODULE_VERSION macro does not even exist. I think the best solution
would have been to say forget sysfs and just embedd it into our netlink
api. We could start that now but I am not sure if that will be
acceptable. It looks like this is done for fc so it should be ok with
someone else is doing defense :)

So we would do:
1. Send iscsi netlink request to get scsi_transport_iscsi version.
2. If we get it, then just check it. If the new iscsi netlink version
request is not supported, then we are dealing with a old kernel and a
older kernel's modules and we have to settle for just being able to
check the module version from sysfs. That last case will remain broken
unless user are willing to patch their kernels.

Christopher Probst

unread,
Aug 31, 2007, 2:12:37 PM8/31/07
to open-...@googlegroups.com
Hello all,

Regarding Bug 253989: iSCSI Hangs on reboot in the Red hat5 where
"Rebooting a node with a mounting iSCSI LUN hangs"

Is there anything I can do on my RH5 system to prevent this from happening?

Thanks,
Christo

_________________________________________________________________
Get Cultured With Arts & Culture Festivals On Live Maps
http://local.live.com/?mkt=en-ca&v=2&cid=A6D6BDB4586E357F!2010&encType=1&style=h&FORM=SERNEP

Christopher Probst

unread,
Aug 31, 2007, 2:14:30 PM8/31/07
to open-...@googlegroups.com
Hello all,

Regarding Bug 253989: iSCSI Hangs on reboot in the Red hat5 where
"Rebooting a node with a mounting iSCSI LUN hangs"

Is there anything I can do on my RH5 system to prevent this from happening?

Thanks,
Christo

_________________________________________________________________
Enter to win a night a VIP night out at TIFF
http://redcarpet.sympatico.msn.ca/

Ritesh Raj Sarraf

unread,
Aug 31, 2007, 2:56:38 PM8/31/07
to open-...@googlegroups.com
On Friday 31 August 2007, Christopher Probst wrote:
> Regarding Bug 253989: iSCSI Hangs on reboot in the Red hat5 where
> "Rebooting a node with a mounting iSCSI LUN hangs"
>
> Is there anything I can do on my RH5 system to prevent this from happening?

It is already mentioned in the comments.

You can use the "_netdev" option to avoid the network going down and iscsid
being killed early.

Add it as:
/dev/vg/lv /mnt/log_vol gfs2 defaults,_netdev 0 0

Ritesh
--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."

signature.asc

Christopher Probst

unread,
Aug 31, 2007, 3:07:25 PM8/31/07
to open-...@googlegroups.com
Thanks Ritesh,

For your answer. Actually, it will still happen if you don't have your
partition or logical volume mounted. So what do you do if your partition on
your iscsi drive is not mounted and not in your fstab

Christopher


><< signature.asc >>

_________________________________________________________________
Show Your Messenger Buddies How You Really Feel
http://www.freemessengeremoticons.ca/?icid=EMENCA122

Mike Christie

unread,
Aug 31, 2007, 4:49:17 PM8/31/07
to open-...@googlegroups.com
Christopher Probst wrote:
> Thanks Ritesh,
>
> For your answer. Actually, it will still happen if you don't have your
> partition or logical volume mounted. So what do you do if your partition on
> your iscsi drive is not mounted and not in your fstab
>

That might not be your bug then. What target are you using, what error
messages are you getting and when, and could you send the
/var/log/messages output when you login and find disks? It may be that
your disk need a cache sync, and kexec was shutting down the interfaces
too soon. That will be fixed in 5.1. If that is your problem I can send
you the bugzilla which has a one line patch to fix it.

If you just do

shutdown -h 0

Does it shutdown cleanly. Shutdown does not use kexec like reboot so it
avoids the problem. I am suggesting this as a fix, but just so we can
make sure we are talking about the same bug.

Christopher Probst

unread,
Sep 4, 2007, 5:39:18 PM9/4/07
to open-...@googlegroups.com
Hi Mike,

Thank you for your response. This is what I do

1) iscsiadm -m discovery --type sendtargets --portal _IPorHos
2) iscsiadm -m node -T _IQN -p _IP -l
3) reboot

It hangs with the following
connection0:0;iscsi:detected conn error (1011)

while it does the killall.

When you do shutdown -h 0. It works fine.

It's hard for me to sort out what is in the /var/log/messages file. I put it
in attachment.
Thanks again for your help.

Christo


>From: Mike Christie <mich...@cs.wisc.edu>
>Reply-To: open-...@googlegroups.com
>To: open-...@googlegroups.com
>Subject: Re: iSCSI Hangs on reboot

_________________________________________________________________
Windows Live Hotmail. Even hotter than before. Get a better look now.
www.newhotmail.ca?icid=WLHMENCA148

messages

Mike Christie

unread,
Sep 5, 2007, 2:37:50 PM9/5/07
to open-...@googlegroups.com
Christopher Probst wrote:
> Hi Mike,
>
> Thank you for your response. This is what I do
>
> 1) iscsiadm -m discovery --type sendtargets --portal _IPorHos
> 2) iscsiadm -m node -T _IQN -p _IP -l
> 3) reboot
>
> It hangs with the following
> connection0:0;iscsi:detected conn error (1011)
>
> while it does the killall.
>
> When you do shutdown -h 0. It works fine.

The patch in this bz, which will be in RHEL 5.1, should work for you then
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=223932

Christopher Probst

unread,
Sep 5, 2007, 2:54:47 PM9/5/07
to open-...@googlegroups.com

Mike,

Thank you very much for your help.

>From: Mike Christie <mich...@cs.wisc.edu>
>Reply-To: open-...@googlegroups.com
>To: open-...@googlegroups.com
>Subject: Re: iSCSI Hangs on reboot

_________________________________________________________________
Put Your Face In Your Space with Windows Live Spaces
http://spaces.live.com/?mkt=en-ca

Reply all
Reply to author
Forward
0 new messages