mkvg on iscsi disk won't work (6.1)
mr.aart
Recently I'm taking my first steps in AIX, initially with no real
issues. Until now, off course...
I hooked up a couple of iSCSI disks to the system, with a RHEL5
machine running tgt on the other side. When running cfgmgr after
initial configuration, I got an error:
----------------------------------------------------------------------------------------------------
# cfgmgr -l iscsi0
cfgmgr: 0514-621 WARNING: The following device packages are required
for
device support but are not currently installed.
devices.iscsi.array
----------------------------------------------------------------------------------------------------
I googled my ass off, but couldn't find anything, other than some
chinese site, wich I find very difficult to understand since chinese
is one of very few major languages they don't teach in dutch
schools...
Nonetheless, I was able to see the devices that I offered through tgt,
and I was able to create PV's on them:
----------------------------------------------------------------------------------------------------
# lspv
hdisk0 00c9a1fbe1fd35d0 rootvg
active
hdisk1 00c9a1fbe5bf8aaf rootvg
active
hdisk2 00c9a1fb88919112 None
hdisk3 00c9a1fb6f64e590 None
hdisk4 00c9a1fb6f652404 None
----------------------------------------------------------------------------------------------------
However, I'm not able to make VG's out of any of these PV's:
----------------------------------------------------------------------------------------------------
# mkvg -y foovg hdisk2
0516-008 /usr/sbin/mkvg: LVM system call returned an unknown
error code (-267).
0516-1184 /usr/sbin/mkvg: IO failure on hdisk2.
0516-862 /usr/sbin/mkvg: Unable to create volume group.
----------------------------------------------------------------------------------------------------
Though I am able to read off the disk with dd:
----------------------------------------------------------------------------------------------------
# dd if=/dev/rhdisk2 of=/dev/null bs=1024 count=1000
1000+0 records in.
1000+0 records out.
----------------------------------------------------------------------------------------------------
Also lsdev doesn't seem to have a problem with the disk:
----------------------------------------------------------------------------------------------------
# lsdev -l hdisk2
hdisk2 Available Other iSCSI Disk Drive
----------------------------------------------------------------------------------------------------
With errpt I see the following error:
----------------------------------------------------------------------------------------------------
#errpt -a | head -46 #-46 was exactly the lenth of the error,
#I made sure it covered the _whole_ error
LABEL: SC_DISK_ERR2
IDENTIFIER: B6267342
Date/Time: Tue May 11 13:15:34 CDT 2010
Sequence Number: 29
Machine Id: 00C9A1FB4C00
Node Id: watson
Class: H
Type: PERM
WPAR: Global
Resource Name: hdisk2
Resource Class: disk
Resource Type: osdisk
Location:
VPD:
Manufacturer................Areca
Machine Type and Model......AIX/System
ROS Level and ID............52303031
Serial Number...............
Device Specific.(Z0)........000005423D000002
Device Specific.(Z1)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
0
SENSE DATA
0A00 2E00 0000 0008 0000 0100 0000 0000 0000 0000 0000 0000 0102 0000
7000 0500
0000 000A 0000 0000 2000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
0000 0000 0000 0000 0000 0000 1C00 0000 0000 0000 0000 0000 0000 0000
0083 0000
0000 0029 0017
----------------------------------------------------------------------------------------------------
To make things more complicated (or clearer to some), I was able to
create a filesystem on the disk:
----------------------------------------------------------------------------------------------------
# mkfs -Vjfs2 /dev/hdisk2
mkfs: destroy /dev/hdisk2 (yes)?
File system created successfully.
97652620 kilobytes total disk space.
Device /dev/hdisk2:
Standard empty filesystem
Size: 195305240 512-byte (DEVBLKSIZE) blocks
----------------------------------------------------------------------------------------------------
Maybe a detailed output for lsattr enlightens my situation to some:
----------------------------------------------------------------------------------------------------
# lsattr -l hdisk2 -E
clr_q no Device
CLEARS its Queue on error True
host_addr 172.16.100.15 Hostname
or IP Address False
location Location
Label True
lun_id 0x1000000000000 Logical
Unit Number ID False
max_transfer 0x40000 Maximum
TRANSFER Size True
port_num 0xcbc PORT
Number False
pvid 00c9a1fb889191120000000000000000 Physical
volume identifier False
q_err yes Use QERR
bit True
q_type simple Queuing
TYPE True
queue_depth 1 Queue
DEPTH True
reassign_to 120 REASSIGN
time out value True
rw_timeout 30 READ/
WRITE time out value True
start_timeout 60 START
unit time out value True
target_name iqn.2010-01.storage.example:foohost:areca.raidset3
Target NAME False
----------------------------------------------------------------------------------------------------
Where [foohost.example.storage] has IP [172.16.100.15] and is said
RHEL5 host running tgtd. This host also serves iSCSI disks to a HP-UX
11.31 machine, where it works just fine.
I'm afraid that this is where I find myself lacking the aix experience
to handle this, maybe you have some pointers for me to fix this?
Tony
># cfgmgr -l iscsi0
>cfgmgr: 0514-621 WARNING: The following device packages are required
>for
> device support but are not currently installed.
>devices.iscsi.array
You need your base AIX CD's and you need to install the devices.iscsi.array
fileset from them, and then patch to the latest level. (Or install from
your NIM server, or LPP Resource, or whatever other option, you need the
base fileset and any pre-requisites it in turn needs).
--
Tony Evans
Saving trees and wasting electrons since 1993
blog -> http://perceptionistruth.com/
books -> http://www.bookthing.co.uk/
[ anything below this line wasn't written by me ]
mke
mkvg sends scsi "write and verify" commands
alas tgt,stgt .... do not support these commands
ibm docs state that "write and verify" support is mandatory.
i could not find any way to bypass this
mke
"mr.aart" <mr....@gmail.com> a �crit dans le message de
news:25d9a429-e32a-4498...@a34g2000yqn.googlegroups.com...
mr.aart
> i had the same problem.
>
> mkvg sends scsi "write and verify" commands
>
> alas tgt,stgt .... do not support these commands
>
> ibm docs state that "write and verify" support is mandatory.
>
> i could not find any way to bypass this
>
> mke
This is an explanation that is as good as it is painfull. When
searching around it appears that AIX is the only OS around that
actually makes mandatrory use of opcode 0x2E(write and verify), and it
seems that this operation is indeed not implemented in stgt. Als could
I find no actual proof that ietd does implement this feature, though
there are some reports of functioning setups trough openfiler.
How did you initially find out that stgt lacked support for 0x2E? Did
you find another solution to offer iSCSI on linux to AIX?
Thanks a lot for your reply!
Maarten van der Aart
mr.aart
> You need your base AIX CD's and you need to install the devices.iscsi.array
> fileset from them, and then patch to the latest level. (Or install from
> your NIM server, or LPP Resource, or whatever other option, you need the
> base fileset and any pre-requisites it in turn needs).
> --
> Tony Evans
While that certainly makes sense, I have not been able to locate this
'devices.iscsi.array' fileset on the original media (I'll check again,
maybe I've overlooked), or on the ibm website. Is there another place
I could possibly obtain this fileset?
Thanks for your reply!
Maarten van der Aart
Tony
>On May 12, 8:10 pm, Tony <t...@darkstorm.invalid> wrote:
>> You need your base AIX CD's and you need to install the devices.iscsi.array
>> fileset from them, and then patch to the latest level. (Or install from
>> your NIM server, or LPP Resource, or whatever other option, you need the
>> base fileset and any pre-requisites it in turn needs).
>While that certainly makes sense, I have not been able to locate this
>'devices.iscsi.array' fileset on the original media (I'll check again,
>maybe I've overlooked), or on the ibm website. Is there another place
>I could possibly obtain this fileset?
How about devices.iscsi_sw.rte ? Got that anywhere?
mke
initiator
we brought a brunch of sas disks and offered them to our AIX lpars via the
VIO.
May be it is just an illusion but vio exported hdisks seem to have a lower
performance then directly attached hdisks ( is there any way to tune this
? )
PS: a wireshark trace showed the "write and verify problem"
mkvg seems to be the only command that uses this obscure scsi feature
Mke
"mr.aart" <mr....@gmail.com> a �crit dans le message de
news:7e7f61e8-c2ff-460e...@h39g2000yqn.googlegroups.com...
Hank M. Higher
...
> we brought a brunch of sas disks and offered them to our AIX lpars via
> the VIO.
>
> May be it is just an illusion but vio exported hdisks seem to have a
> lower performance then directly attached hdisks ( is there any way to
> tune this ? )
to the contrary. my experience is that disks mapped to vscsi as a whole
perform almost identically as FC attached disks. that's likely to be
different for mapping LVs to vscsi. And of course assuming that you do
not intensively overbook your vio FC bandwidth with numerous vscsi clients.
did you measure anything with a benchmark?
Uwe Auer
Hi,
you are right with FC disks on SVC or DSXk, but he ordered SAS disks on the VIO
and that makes in fact his poor local attached performance :-), since there is
obviously no caching mechanism on VIOs like it is on the other SAN devices.
Regards,
Uwe
mr.aart
Sorry for the delay, but here is what I have installed, concerning
iSCSI, so far:
--------------------------------------------------------------------------------
# lslpp -l|grep -i iscsi
devices.common.IBM.iscsi.rte
6.1.5.0 COMMITTED Common iSCSI Files
devices.iscsi.disk.rte 6.1.5.0 COMMITTED iSCSI Disk Software
devices.iscsi.tape.rte 6.1.0.0 COMMITTED iSCSI Tape Software
devices.iscsi_sw.rte 6.1.5.1 COMMITTED iSCSI Software Device
Driver
devices.pci.14102203.diag 6.1.0.0 COMMITTED IBM 1 Gigabit-TX
iSCSI TOE
devices.pci.14102203.rte 6.1.0.0 COMMITTED IBM 1 Gigabit-TX
iSCSI TOE
devices.pci.1410cf02.diag 6.1.0.0 COMMITTED 1000 Base-SX PCI-X
iSCSI TOE
devices.pci.1410cf02.rte 6.1.0.0 COMMITTED 1000 Base-SX PCI-X
iSCSI TOE
devices.pci.1410d002.com 6.1.5.0 COMMITTED Common PCI iSCSI TOE
Adapter
devices.pci.1410d002.diag 6.1.5.0 COMMITTED 1000 Base-TX PCI-X
iSCSI TOE
devices.pci.1410d002.rte 6.1.0.0 COMMITTED 1000 Base-TX PCI-X
iSCSI TOE
devices.pci.1410e202.diag 6.1.0.0 COMMITTED IBM 1 Gigabit-SX
iSCSI TOE
devices.pci.1410e202.rte 6.1.0.0 COMMITTED IBM 1 Gigabit-SX
iSCSI TOE
devices.pci.77102e01.diag 6.1.0.0 COMMITTED 1000 Base-TX PCI-X
iSCSI TOE
devices.pci.77102e01.rte 6.1.0.0 COMMITTED PCI-X 1000 Base-TX
iSCSI TOE
devices.common.IBM.iscsi.rte
6.1.5.0 COMMITTED Common iSCSI Files
devices.iscsi_sw.rte 6.1.0.0 COMMITTED iSCSI Software Device
Driver
devices.pci.1410d002.com 6.1.0.0 COMMITTED Common PCI iSCSI TOE
Adapter
devices.pci.1410d002.rte 6.1.0.0 COMMITTED 1000 Base-TX PCI-X
iSCSI TOE
--------------------------------------------------------------------------------
It appears to me that all the files are there, unfortunately... I
still don't really get that 'devices.iscsi.array' thing, though. Where
am I supposed to find that? I'm quite certain it's not on one of the
original install discs, and searching for it was to no avail either,
so far.
However, thanks a lot for taking the time to answer my questions :-)
Maarten van der Aart
mr.aart
> Sorry i could not find any way to implement a linux iscsi target for an aix
> initiator
>
> we brought a brunch of sas disks and offered them to our AIX lpars via the
> VIO.
>
> May be it is just an illusion but vio exported hdisks seem to have a lower
> performance then directly attached hdisks ( is there any way to tune this
> ? )
>
> PS: a wireshark trace showed the "write and verify problem"
>
> mkvg seems to be the only command that uses this obscure scsi feature
>
> Mke
I'll certainly keep looking for a way to solve it, and will get back
on this when I succeed, but thanks a lot for toning down my
frustration a little :-)
Maarten van der Aart
mr.aart
At the risk of looking spammy, here my final remark on this subject:
I've successfully ran 'mkvg' on an iSCSI disk, exported from an RHEL5
machine running IET (http://iscsitarget.sourceforge.net/).
--------------------------------------------------------------------------------
# chdev -l hdisk2 -a pv=clear
hdisk2 changed
# lspv
hdisk0 00c9a1fbe1fd35d0 rootvg
active
hdisk1 00c9a1fbe5bf8aaf rootvg
active
hdisk2 none None
hdisk3 00c9a1fb6f64e590 None
hdisk4 00c9a1fb6f652404 None
# chdev -l hdisk2 -a pv=yes
hdisk2 changed
# lspv
hdisk0 00c9a1fbe1fd35d0 rootvg
active
hdisk1 00c9a1fbe5bf8aaf rootvg
active
hdisk2 00c9a1fbad59e652 None
hdisk3 00c9a1fb6f64e590 None
hdisk4 00c9a1fb6f652404 None
# mkvg -y foovg hdisk2
foovg
# lspv
hdisk0 00c9a1fbe1fd35d0 rootvg
active
hdisk1 00c9a1fbe5bf8aaf rootvg
active
hdisk2 00c9a1fbad59e652 foovg
active
hdisk3 00c9a1fb6f64e590 None
hdisk4 00c9a1fb6f652404 None
--------------------------------------------------------------------------------
Thanks to everyone that helped me fix this really _really_ annoying
issue!
Maarten van der Aart
ilomae...@gmail.com
Please where can i get IET for AIX.
And do i set it up. I am having similar issue with AIX 7.1
Thanks.
vinod...@actifio.com
scst config file
HANDLER vdisk_fileio {
.
.
.
DEVICE i-14becaa26b6 {
filename /act/pg/omd/gvdisk_name/123456..........
}
}