lio target crashes when windows initiator logs in

95 views
Skip to first unread message

ablock

unread,
Dec 8, 2009, 1:15:32 PM12/8/09
to Linux-iSCSI.org Target Development

Hi,
I have problems with the lio-target software. I tried lio-core-2.6.31
and lio-core-2.6.
I compiled it together with lio-utils under ubuntu 9.10 and debian
5.0.
Ubuntu and debian was installed in a virtual machine. I used virtual
box 3.0.12.
I tried it also on bare metal with the same problems.


I can get it working when i use a block device like /dev/sdb.
It crashes completely when i use a block device like /dev/sdb1 (The
Partition exists!!!)
It also crashes completely when i use a logical volume or a md-device.

The crash happens whenever a Windows Initiator logs in. I tried
Windows Vista and Windows Server 2008.

When I start the target module I get the following output:

Loading target_core_mod/ConfigFS core: [OK]
Calling ConfigFS script /etc/target/tcm_start.sh for
target_core_mod: [OK]
Calling ConfigFS script /etc/target/lio_start.sh for
iscsi_target_mod: [OK]


In /var/log/messages I get:

Dec 8 18:50:51 debian kernel: [ 106.480865] TARGET_CORE[0]: Loading
Generic Kernel Storage Engine: v3.1.0 on Linux/x86_64 on 2.6.31.4v3.1
Dec 8 18:50:51 debian kernel: [ 106.481007] TARGET_CORE[0]:
Initialized ConfigFS Fabric Infrastructure: v2.0.0 on Linux/x86_64 on
2.6.31.4v3.1
Dec 8 18:50:51 debian kernel: [ 106.481036] SE_PC[0] - Registered
Plugin Class: TRANSPORT
Dec 8 18:50:51 debian kernel: [ 106.481061] PLUGIN_TRANSPORT[1] -
pscsi registered
Dec 8 18:50:51 debian kernel: [ 106.481084] PLUGIN_TRANSPORT[2] -
stgt registered
Dec 8 18:50:51 debian kernel: [ 106.481212] CORE_STGT[0]: Bus
Initalization complete
Dec 8 18:50:51 debian kernel: [ 106.481232] PLUGIN_TRANSPORT[4] -
iblock registered
Dec 8 18:50:51 debian kernel: [ 106.481250] PLUGIN_TRANSPORT[5] -
rd_dr registered
Dec 8 18:50:51 debian kernel: [ 106.481268] PLUGIN_TRANSPORT[6] -
rd_mcp registered
Dec 8 18:50:51 debian kernel: [ 106.481285] PLUGIN_TRANSPORT[7] -
fileio registered
Dec 8 18:50:51 debian kernel: [ 106.481307] SE_PC[1] - Registered
Plugin Class: OBJ
Dec 8 18:50:51 debian kernel: [ 106.481326] PLUGIN_OBJ[1] - dev
registered


I then initialize the iscsi target with the following commands

tcm_node --block iblock_0/my_dev2 /dev/vg1/lv1
lio_node --addlun iqn.2009-11.local.schule.target.i686:sn.123456789 1
0 my_dev_port iblock_0/my_dev2
lio_node --disableauth iqn.2009-11.local.schule.target.i686:sn.
123456789 1
lio_node --addnp iqn.2009-11.local.schule.target.i686:sn.123456789 1
192.168.56.101:3260
lio_node --addlunacl iqn.2009-11.local.schule.target.i686:sn.123456789
1 iqn.1991-05.com.microsoft:andreas-pc 0 0
lio_node --enabletpg iqn.2009-11.local.schule.target.i686:sn.123456789
1

They produce the following output:
Output tcm_node:

Status: DEACTIVATED Execute/Left/Max Queue Depth: 0/32/32
SectorSize: 512 MaxSectors: 255
iBlock device: dm-0
Major: 253 Minor: 0 CLAIMED: IBLOCK
ConfigFS HBA: iblock_0
Successfully added TCM/ConfigFS HBA: iblock_0
ConfigFS Device Alias: my_dev2
Device Params ['/dev/vg1/lv1']
Set T10 WWN Unit Serial for iblock_0/my_dev2 to: 57f6b040-3159-49df-
a5bd-2acdb948ef6f
Successfully created TCM/ConfigFS storage object: /sys/kernel/config/
target/core/iblock_0/my_dev2

Output lio_node --addlun:
Successfully created iSCSI Target Logical Unit

Output lio_node --disableauth:
Successfully disabled iSCSI Authentication on iSCSI Target Portal
Group: iqn.2009-11.local.schule.target.i686:sn.123456789 1

Output lio_node --addnp:
Successfully created network portal: 192.168.56.101:3260 created iqn.
2009-11.local.schule.target.i686:sn.123456789 TPGT: 1

Output von lio_node --addlunacl:
Successfully added iSCSI Initiator Mapped LUN: 0 ACL iqn.
1991-05.com.microsoft:andreas-pc for iSCSI Target Portal Group: iqn.
2009-11.local.schule.target.i686:sn.123456789 1

Output von lio_node --enabletpg:
Successfully enabled iSCSI Target Portal Group: iqn.
2009-11.local.schule.target.i686:sn.123456789 1


In /var/log/messages the initialization leads to the following:

Dec 8 18:53:11 debian kernel: [ 246.679996] Target_Core_ConfigFS:
Located se_plugin: ffff88000dd630e0 plugin_name: iblock hba_type: 4
plugin_dep_id: 0
Dec 8 18:53:11 debian kernel: [ 246.680398] CORE_HBA[0] - Linux-
iSCSI.org iBlock HBA Driver 3.1 on Generic Target Core Stack v3.1.0
Dec 8 18:53:11 debian kernel: [ 246.680425] CORE_HBA[0] - Attached
iBlock HBA: 0 to Generic Target Core TCQ Depth: 512
Dec 8 18:53:11 debian kernel: [ 246.680452] CORE_HBA[0] - Attached
HBA to Generic Target Core
Dec 8 18:53:11 debian kernel: [ 246.680852] IBLOCK: Allocated ib_dev
for my_dev2
Dec 8 18:53:11 debian kernel: [ 246.680879] Target_Core_ConfigFS:
Allocated se_subsystem_dev_t: ffff88000d86b000 se_dev_su_ptr:
ffff88000ec07800
Dec 8 18:53:11 debian kernel: [ 246.720958] Target_Core_ConfigFS:
iblock_0/my_dev2 set udev_path: /dev/vg1/lv1
Dec 8 18:53:11 debian kernel: [ 246.735619] IBLOCK: Claiming struct
block_device: ffff88000f2d8200
Dec 8 18:53:11 debian kernel: [ 246.735714] bio: create slab <bio-1>
at 1
Dec 8 18:53:11 debian kernel: [ 246.735736] IBLOCK: Created bio_set
() for major/minor: 253:0
Dec 8 18:53:11 debian kernel: [ 246.735743] iblock: Using
SPC3_PERSISTENT_RESERVATIONS emulation
Dec 8 18:53:11 debian kernel: [ 246.735746] iblock: Enabling ALUA
Emulation for SPC-3 device
Dec 8 18:53:11 debian kernel: [ 246.735760] iblock: Adding to
default ALUA LU Group: core/alua/lu_gps/default_lu_gp
Dec 8 18:53:11 debian kernel: [ 246.735764] CORE_iBLOCK[0] -
Activating Device with TCQ: 0 at Major: 253 Minor 0
Dec 8 18:53:11 debian kernel: [ 246.735870] Vendor: LIO-ORG
Model: IBLOCK Revision: 3.1
Dec 8 18:53:11 debian kernel: [ 246.735879] Type: Direct-
Access ANSI SCSI revision: 05
Dec 8 18:53:11 debian kernel: [ 246.735907] T10 VPD Unit Serial
Number: 1234567890:0_253_0
Dec 8 18:53:11 debian kernel: [ 246.735924] T10 VPD Page Length: 38
Dec 8 18:53:11 debian kernel: [ 246.735927] T10 VPD Identifer
Length: 34
Dec 8 18:53:11 debian kernel: [ 246.735930] T10 VPD Identifier
Association: addressed logical unit
Dec 8 18:53:11 debian kernel: [ 246.735937] T10 VPD Identifier Type:
T10 Vendor ID based
Dec 8 18:53:11 debian kernel: [ 246.735940] T10 VPD ASCII Device
Identifier: LIO-ORG
Dec 8 18:53:11 debian kernel: [ 246.735958] Target_Core_ConfigFS:
Registered iblock se_dev->se_dev_ptr: ffff88000ec00c00 from fd
Dec 8 18:53:11 debian kernel: [ 246.790576] Target_Core_ConfigFS:
Set emulated VPD Unit Serial: 57f6b040-3159-49df-a5bd-2acdb948ef6f
Dec 8 18:53:11 debian kernel: [ 246.791401] T10 VPD Page Length: 76
Dec 8 18:53:11 debian kernel: [ 246.791405] T10 VPD Identifer
Length: 16
Dec 8 18:53:11 debian kernel: [ 246.791411] T10 VPD Identifier
Association: addressed logical unit
Dec 8 18:53:11 debian kernel: [ 246.791414] T10 VPD Identifier Type:
NAA
Dec 8 18:53:11 debian kernel: [ 246.791418] T10 VPD Binary Device
Identifier: 3600140557f6b040d3159d49dfda5bdd2
Dec 8 18:53:11 debian kernel: [ 246.791422] T10 VPD Identifer
Length: 52
Dec 8 18:53:11 debian kernel: [ 246.791428] T10 VPD Identifier
Association: addressed logical unit
Dec 8 18:53:11 debian kernel: [ 246.791431] T10 VPD Identifier Type:
T10 Vendor ID based
Dec 8 18:53:11 debian kernel: [ 246.791434] T10 VPD ASCII Device
Identifier: LIO-ORG
Dec 8 18:56:13 debian kernel: [ 428.046434] Target_Core_ConfigFS:
REGISTER -> group: ffffffffa0368600 name: iscsi
Dec 8 18:56:13 debian kernel: [ 428.070311] Linux-iSCSI.org iSCSI
Target Core Stack v3.1.0 on Linux/x86_64 on 2.6.31.4v3.1
Dec 8 18:56:13 debian kernel: [ 428.070357] <<<<<<<<<<<<<<<<<<<<<<
BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
Dec 8 18:56:13 debian kernel: [ 428.070362] Initialized struct
target_fabric_configfs: ffff88000d8b8400 for iscsi
Dec 8 18:56:13 debian kernel: [ 428.070370] <<<<<<<<<<<<<<<<<<<<<<
END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
Dec 8 18:56:13 debian kernel: [ 428.070373] LIO_TARGET[0] - Set
fabric -> lio_target_fabric_configfs
Dec 8 18:56:13 debian kernel: [ 428.076401]
iscsi_allocate_thread_sets:195: ***OPS*** Spawned 4 thread set(s) (8
total threads).
Dec 8 18:56:13 debian kernel: [ 428.077016] TARGET_CORE[iSCSI]:
Allocated Discovery se_portal_group_t for endpoint: None, Portal Tag:
1
Dec 8 18:56:13 debian kernel: [ 428.077239] CORE[0] - Allocated
Discovery TPG
Dec 8 18:56:13 debian kernel: [ 428.077242] Loading Complete.
Dec 8 18:56:13 debian kernel: [ 428.078426] Target_Core_ConfigFS:
REGISTER -> Located fabric: iscsi
Dec 8 18:56:13 debian kernel: [ 428.078430] Target_Core_ConfigFS:
REGISTER -> ffffffffa03a7d90
Dec 8 18:56:13 debian kernel: [ 428.078434] Target_Core_ConfigFS:
REGISTER -> Allocated Fabric: iscsi
Dec 8 18:56:13 debian kernel: [ 428.078437] Target_Core_ConfigFS:
REGISTER -> Set tf->tf_fabric for iscsi
Dec 8 18:56:13 debian kernel: [ 428.078483]
lio_target_call_coreaddtiqn(): name: iqn.
2009-11.local.schule.target.i686:sn.123456789
Dec 8 18:56:13 debian kernel: [ 428.078587] CORE[0] - Added iSCSI
Target IQN: iqn.2009-11.local.schule.target.i686:sn.123456789
Dec 8 18:56:13 debian kernel: [ 428.078591] LIO_Target_ConfigFS:
REGISTER -> iqn.2009-11.local.schule.target.i686:sn.123456789
Dec 8 18:56:13 debian kernel: [ 428.078595] LIO_Target_ConfigFS:
REGISTER -> Allocated Node: iqn.2009-11.local.schule.target.i686:sn.
123456789
Dec 8 18:56:13 debian kernel: [ 428.078609] lio_target_tiqn_addtpg()
parent name: iqn.2009-11.local.schule.target.i686:sn.123456789
Dec 8 18:56:13 debian kernel: [ 428.078705] TARGET_CORE[iSCSI]:
Allocated Normal se_portal_group_t for endpoint: iqn.
2009-11.local.schule.target.i686:sn.123456789, Portal Tag: 1
Dec 8 18:56:13 debian kernel: [ 428.078919] CORE[iqn.
2009-11.local.schule.target.i686:sn.123456789]_TPG[1] - Added iSCSI
Target Portal Group
Dec 8 18:56:13 debian kernel: [ 428.078923] LIO_Target_ConfigFS:
REGISTER -> iqn.2009-11.local.schule.target.i686:sn.123456789
Dec 8 18:56:13 debian kernel: [ 428.078927] LIO_Target_ConfigFS:
REGISTER -> Allocated TPG: tpgt_1
Dec 8 18:56:13 debian kernel: [ 428.079157] LIO_Target_ConfigFS:
REGISTER -> iqn.2009-11.local.schule.target.i686:sn.123456789 TPGT: 1
LUN: 0
Dec 8 18:56:13 debian kernel: [ 428.098153] iblock/iSCSI: Adding to
default ALUA Target Port Group: alua/default_tg_pt_gp
Dec 8 18:56:13 debian kernel: [ 428.098180] iSCSI_TPG[1]_LUN[0] -
Activated iSCSI Logical Unit from CORE HBA: 0
Dec 8 18:56:13 debian kernel: [ 428.098203] LIO_Target_ConfigFS:
Created Port Symlink my_dev2 -> lun_0
Dec 8 19:01:42 debian kernel: [ 757.891574] Disabling iSCSI
Authentication Methods for TPG: 1.
Dec 8 19:02:59 debian kernel: [ 834.659367] LIO_Target_ConfigFS:
REGISTER -> iqn.2009-11.local.schule.target.i686:sn.123456789 TPGT: 1
PORTAL: 192.168.56.101:3260
Dec 8 19:02:59 debian kernel: [ 834.659508] CORE[0] - Added Network
Portal: 192.168.56.101:3260 on TCP on network device: None
Dec 8 19:02:59 debian kernel: [ 834.659516] CORE[iqn.
2009-11.local.schule.target.i686:sn.123456789] - Added Network Portal:
192.168.56.101:3260,1 on TCP on network device: None
Dec 8 19:02:59 debian kernel: [ 834.659522] CORE[iqn.
2009-11.local.schule.target.i686:sn.123456789]_TPG[1] - Incremented
np_exports to 1
Dec 8 19:02:59 debian kernel: [ 834.659533] LIO_Target_ConfigFS:
addnptotpg done!
Dec 8 19:05:03 debian kernel: [ 958.394369] iSCSI_TPG[1] - Added ACL
with TCQ Depth: 16 for iSCSI Initiator Node: iqn.
1991-05.com.microsoft:andreas-pc
Dec 8 19:05:03 debian kernel: [ 958.394416] LIO_Target_ConfigFS:
REGISTER -> iqn.2009-11.local.schule.target.i686:sn.123456789 TPGT: 1
Initiator: iqn.1991-05.com.microsoft:andreas-pc CmdSN Depth: 16
Dec 8 19:05:03 debian kernel: [ 958.394705] LIO_Target_ConfigFS:
Initialized Initiator LUN ACL: iqn.1991-05.com.microsoft:andreas-pc
Mapped LUN: lun_0
Dec 8 19:05:03 debian kernel: [ 958.411504] iSCSI_TPG[1]_LUN[0->0] -
Added RW ACL for InitiatorNode: iqn.1991-05.com.microsoft:andreas-pc
Dec 8 19:05:03 debian kernel: [ 958.411512] LIO_Target_ConfigFS:
Created Initiator LUN ACL Symlink: iqn.1991-05.com.microsoft:andreas-
pc TPG LUN: lun_0 Mapped LUN: lun_0 Write Protect: OFF
Dec 8 19:06:16 debian kernel: [ 1031.263995] iSCSI_TPG[1] - Enabled
iSCSI Target Portal Group


The command /etc/init.d/target status produces:

[---------------------------] TCM/ConfigFS Status
[----------------------------]
\------> iblock_0
HBA Index: 0 plugin: iblock version: v2.0.0
\-------> my_dev2
Status: ACTIVATED Execute/Left/Max Queue Depth: 0/32/32
SectorSize: 512 MaxSectors: 255
iBlock device: dm-0
Major: 253 Minor: 0 CLAIMED: IBLOCK
udev_path: /dev/vg1/lv1

[---------------------------] LIO-Target Status
[----------------------------]
\------> iqn.2009-11.local.schule.target.i686:sn.123456789
\-------> tpgt_1 TargetAlias: LIO Target
TPG Status: ENABLED
TPG Network Portals:
\-------> 192.168.56.101:3260
TPG Logical Units:
\-------> lun_0/my_dev_port -> target/core/iblock_0/
my_dev2

Target Engine Core ConfigFS Infrastructure v2.0.0 on Linux/x86_64 on
2.6.31.4v3.1
Linux-iSCSI.org Target v3.1.0 on Linux/x86_64 on 2.6.31.4v3.1


In the iscsi initiator dialog from Microsoft I can add the target
portal, that means the lio-target. I then select the Target tab and
then the lio-target from the list.
When i click the log on Button 2 things can happen:
The first log on is ok. I then can open in Windows the computer
managemant and there the disk management.
The iscsi device is displayed as a new disc drive and i can partition
it. But once i format the partition the lio-target
crashes completely. The linux os reacts no more.

When i restart the linux machine it craches from now on as soon as
Windows log on to lio target.

In the Windows Event log the following error is displayed:
the initiator could not send a iscsi pdu...

Can anybody help me?
Thanks
Andreas Block

ablock

unread,
Dec 10, 2009, 12:45:37 PM12/10/09
to Linux-iSCSI.org Target Development
Hello,
here comes some updated Information:
If I compile lio at a 32-Bit-System, I can successfully connect to a
target on top of devices like /dev/sdb, /dev/sdbX, /dev/mdX. The
target software crashes, when I use a lvm-device.
But only the target-software crashes, not the operating system.
The software crash reports something like a Kernel Bug in
scatterlist.h :65

A. Block

justincky

unread,
Jan 21, 2010, 10:47:32 PM1/21/10
to Linux-iSCSI.org Target Development
I am having the same exact problem. I am trying to use a software
raid volume at /dev/md0 that is 7TB. Note that it does not freeze up
the system it just freezes up the kernel module.

I have the kernel dump info...

[ 721.764907] ------------[ cut here ]------------
[ 721.764950] kernel BUG at include/linux/scatterlist.h:63!
[ 721.764988] invalid opcode: 0000 [#1] SMP
[ 721.765027] last sysfs file: /sys/module/target_core_mod/initstate
[ 721.765071] Modules linked in: iscsi_target_mod target_core_mod
configfs fbcon tileblit font bitblit softcursor iptable_filter i915
ip_tables x_tables drm_kms_helper drm i2c_algo_bit lm85 hwmon_vid
i2c_i801 video output ppdev lp intel_agp parport_pc tpm_infineon tpm
tpm_bios agpgart psmouse serio_raw parport raid10 raid456
async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy
async_tx raid1 raid0 multipath linear sky2
[ 721.765502]
[ 721.765518] Pid: 1396, comm: LIO_iblock Not tainted (2.6.32 #1)
[ 721.765564] EIP: 0060:[<e11eb62a>] EFLAGS: 00010202 CPU: 0
[ 721.765620] EIP is at transport_map_mem_to_sg+0x11a/0x140
[target_core_mod]
[ 721.765668] EAX: 0000dc00 EBX: de519ea0 ECX: de20d718 EDX: de5437dc
[ 721.765711] ESI: 00000021 EDI: 00000001 EBP: de519dc8 ESP: de519da8
[ 721.765754] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 721.765792] Process LIO_iblock (pid: 1396, ti=de518000
task=dd4fb2c0 task.ti=de518000)
[ 721.765845] Stack:
[ 721.766985] de519dc8 00000002 df033540 00000e00 00006c61 de519e38
de5437e8 df033540
[ 721.767065] <0> de519dec e11e7017 de1956a0 de519e38 de519e34
de519ea0 e1209540 de519e38
[ 721.768298] <0> de1956a0 de519e4c e11e9d06 00000000 de1956a0
de519e38 de519e34 de519ea0
[ 721.768728] Call Trace:
[ 721.768728] [<e11e7017>] ? dev_obj_do_se_mem_map+0xa7/0xb0
[target_core_mod]
[ 721.768728] [<e11e9d06>] ? transport_generic_get_cdb_count
+0x1e6/0x3c0 [target_core_mod]
[ 721.768728] [<e11e6e5f>] ? dev_obj_get_cdb_count+0x4f/0x60
[target_core_mod]
[ 721.768728] [<e11e9967>] ? transport_new_cmd_obj+0xd7/0x110
[target_core_mod]
[ 721.768728] [<e11ed5dc>] ? transport_generic_new_cmd+0x9c/0x270
[target_core_mod]
[ 721.768728] [<c057ea20>] ? schedule+0x450/0xaa0
[ 721.768728] [<c0127508>] ? default_spin_lock_flags+0x8/0x10
[ 721.768728] [<e11f068c>] ? transport_processing_thread+0x1dc/0x730
[target_core_mod]
[ 721.768728] [<c01622d0>] ? autoremove_wake_function+0x0/0x40
[ 721.768728] [<e11f04b0>] ? transport_processing_thread+0x0/0x730
[target_core_mod]
[ 721.768728] [<c016206c>] ? kthread+0x6c/0x80
[ 721.768728] [<c0162000>] ? kthread+0x0/0x80
[ 721.768728] [<c0103e87>] ? kernel_thread_helper+0x7/0x10
[ 721.768728] Code: ea 0c 74 3c 8b 7d 10 83 07 01 8b 3b 89 7d ec c7
03 00 00 00 00 eb 82 66 90 89 41 08 03 45 ec 89 03 eb b9 31 f6 eb b8
0f 0b eb fe <0f> 0b eb fe c7 04 24 58 26 20 e1 e8 0c 2e 39 df 83 c8 ff
eb ac
[ 721.768728] EIP: [<e11eb62a>] transport_map_mem_to_sg+0x11a/0x140
[target_core_mod] SS:ESP 0068:de519da8
[ 721.795666] ---[ end trace 47011adb8b82b1e0 ]---
[

> Target ...
>
> read more »

ablock

unread,
Jan 26, 2010, 5:57:46 AM1/26/10
to Linux-iSCSI.org Target Development
Hello,
sorry I still have no solution. I think lio-target is experimental
code. On a bare-metal-machine I can get it working only on devices
like /dev/sda. I wonder if anybody has successfully used lio-target
with other block-devices and a windows initiator.
Any hints are appreciated.

Nicholas A. Bellinger

unread,
Jan 28, 2010, 4:03:52 AM1/28/10
to linux-iscsi...@googlegroups.com, Andreas Block, Justin Chambers

Greetings Andreas and Justin,

My apologies for the delayed response.. Here are my comments wrt to
using an MD RAID0 struct block_device with TCM/IBLOCK subsystem plugin
export:

I have previously ran into an issue where the submission of struct bios
via submit_bio() in TCM/IBLOCK had caused problems with the bio mapping
logic to the underlying MD RAID0 element struct block_devices in
drivers/md/raid0.c:raid0_make_request(). What I had previously seen on
v2.6.18 was not exactly similar to what you have reported with LIO 3.x
on v2.6.3x with BUG_ON() firing from include/linux/scatterlist.h, but I
believe the root issue is the same.

Also, I had found that this issue is ONLY specific to TCM/IBLOCK with
software MD RAID0. Using TCM/IBLOCK on top of any other MD RAID
algoritim (and optionally in conjuction with Linux/LVM on top) or
hardware RAID0 will work as expected. Obviously this should not trigger
an OOPs, but unfortuately the TCM/IBLOCK code is not able to easily tell
the difference between different MD RAID algoritims..

The current workaround for allowing export of MD RAID0 struct
block_devices (including Linux/LVM on top of MD RAID0 struct
block_devices) is to use TCM/FILEIO export (eg: tcm_node --fileio) with
O_SYNC. Using TCM/FILEIO w/ O_SYNC will disable the VFS level buffer
cache to ensure that ACKS are not sent back to the initiator side before
they actually make it down to MD RAID0 array members (same logic to
bypass VFS buffer cache with TCM/IBLOCK).

Also Andreas, there is a seperate bug wrt to exporting individual
partitions (eg: /dev/sdc1) with both TCM/IBLOCK and TCM/FILEIO subsystem
plugins. Both of these plugins currently assume export of the entire
struct block_device, and claim the underlying struct block_device even
when a partition is specified. Resolving this issue is on my short-term
TODO, and will be fixed shortly..

Until then, please let me know if you run into any other problems or
have any questions..

Thank you for reporting these issues!

--nab


ablock

unread,
Jan 31, 2010, 3:26:28 PM1/31/10
to Linux-iSCSI.org Target Development
Hello,
thank you for the answer. Can you tell me, how exactly I can choose
TCM/FILIO with O_SYNC? I don't find the option in the man pages. Does
O_Sync work correctly in kernel 2.6.32? (See the comment in
http://lwn.net/Articles/350219/)
I want lio on top of a drbd-device. It is important, that there is no
caching between lio and the drbd-device.

Andreas

On 28 Jan., 10:03, "Nicholas A. Bellinger" <n...@linux-iscsi.org>
wrote:

Nicholas A. Bellinger

unread,
Feb 2, 2010, 4:38:49 AM2/2/10
to linux-iscsi...@googlegroups.com
On Sun, 2010-01-31 at 12:26 -0800, ablock wrote:
> Hello,

Greetings Andreas!

> thank you for the answer. Can you tell me, how exactly I can choose
> TCM/FILIO with O_SYNC?

So, lio-core-2.6.git/drivers/target/target_core_file.c uses O_SYNC by
default when calling filp_open() for all kernel level struct file
access.

> I don't find the option in the man pages. Does
> O_Sync work correctly in kernel 2.6.32? (See the comment in
> http://lwn.net/Articles/350219/)

Yes, O_SYNC should be working as expected for kernel space struct file
operations wrt to TCM/FILEIO in v2.6.32. I had asked hch about this
last year when this came up, and he says using O_SYNC for
target_core_file.c is indeed the correct approach.


> I want lio on top of a drbd-device. It is important, that there is no
> caching between lio and the drbd-device.

If you are using DRBD then you will want to be using TCM/IBLOCK export
for asynchronous TCQ > 1 logic into DRBD's virtual struct block_device.
By default DRBD will present max_sectors=32 for the struct
block_device's underlying request_queue_t, and target_core_mod will
generate multiple local CDB tasks to complete the single received SCSI
CDB containing a sector count exceeding the TCM/IBLOCK backstore's
max_sectors value.

Also, TCM/IBLOCK + LIO-Target v3.1 has been tested as is known to be
stable with DRBD on SLES 11 (v2.6.27). The TCM/IBLOCK code and upstream
struct bio intrastructure API has not changed much from the target's
prespective between v2.6.27 and v2.6.32, so if you run into any issues
with TCM/IBLOCK + LIO-Target v3.2 with DRBD exports, please go ahead and
post your logs here..

Thanks!

--nab

Reply all
Reply to author
Forward
0 new messages