SAN & VIO Server / LPARs

[bomahony]

Im currently looking at implementing VIO Server and LPARs in our local
environment. At the moment, I have three sucessful LPARS & VIOS server
running on a p505with 16GB of RAM.

We are only using one of the NICs on the system. I have three iSCSI
LUNs created on a windows target (our NAS box atm). I can successfully
connect and read/write from another AIX box.

Q1:
Basically I am wondering for the LPARs,, should i be connecting the
VIO server to the LUNs and then assigning them through the HMC as
vSCSI devices, or should I be assigning the LUNs directly to the
LPARs. I would like the ability to move the LUNs between the LPARs but
for now its not required.

Q2:
When i implement this further I am planning on putting the second
ethernet port on the p505 on a seperate subnet, which will have a SAN
on it. I would also prefer here if the LUNs were swapable between
various virtual servers. Is there any docs out there on how to set a
SAN up on the second NIC on a different subnet, for LPARs?

Thanks

B

[Polani]

If your Client Lpars want to access a SAN disk , you have to assign
SAN disks to VIO server ( having FC adapter), put all necessary
drivers on VIO server and then create vscsi disks on client Lpars to
allow SAN disks through VIO.

However in case of ISCSI drives , things might change. I mean in case
of ISCSI , luns can be directly attached to client lpars..

Regards

Polani

Want to have unique solutions for AIX, Linux, Storage
Have a look at my personal blogs at http://reliablesolutions.blogspot.com

[Hajo Ehlers]

...

> Q1:
> Basically I am wondering for the LPARs,, should i be connecting the
> VIO server to the LUNs and then assigning them through the HMC as
> vSCSI devices, or should I be assigning the LUNs directly to the
> LPARs. I would like the ability to move the LUNs between the LPARs but
> for now its not required.

You start talking about iscsi and switching to lun . I assume you are
talking about accessing a LUN via iSCSI from a mpar ( lpar)
If this is the case then there is no need to configure anything on the
VIO server. You configure the iSCSI Initiator on the mpar and thats
it. Since its best practise to have a dedicated LAN for iscsi just use
your free network port on your server for the iscsi traffic. This
network must of course configured on the VIO server and mpar.

hth
hajo

[bomahony]

Thanks for both your prompt responses.

Im mucking about with this atm, on two other p505s. I have accessto a
Lun from a machine, created a filesystem, wrote a few files.
However, I cant seem to remove the iscsi LUN from this machine. The
disk is hdisk2 and the iscsi device is iscsi0
I ran rmdev -dl iscsi0 after unmounting the volume. However, i cannot
remove the device. I get a
0514-062 Cannot perform the requested function because the specified
device is busy.

I want to muck about with this so i know how AIX deals with the iSCSI
targets. I am thinking for ease of use, when I go to implement this,
assigning the LUNs as vscsi disks via VIO server would probably be the
easiest, as that will allow me move the disks around pretty easy (at
least i assume it will be easy, like moving vscsi disks already
there). However i do want to understand and get to grips with how AIX
does this from the OS itself.

So how do i remove the device in AIX. Id rather not have to reboot
(although i can) as that may not be an option if i ever use this.

Thanks

B

[Pete's]

rmdev -d -l <device name>. In the case of an hdisk, you might
consider an "exportvg" which would remove information out of the odm
for you, that's assuming that the LUN is the only disk in the volume
group. You then can allocate the disk over to the partition of choice
via VIO. No reboot should be necessary.

HTH,
Pete's

[bomahony]

Pete

I am currently working on servers that DONT have VIO, so as to
familiarise myself with moving about the LUNs. The exportvg command
threw up a vary off error, so after running varyoffvg, then exportvg,
and lastly rmdev -dl iscsi0 -R it removed the hdisk2 and the iscsi0
devices.

Now on another machine (the machine where the filesystem wasnt
created) I have the iscsi disk visible as hdisk3. When i use smit, i
can see it as a Physical Volume. however when i go to "List contents
of a physical volume" i dont see disk 3 (only disk0 and disk1).

I was then able to import it into the new machine using importvg

Here is my problem now. If i delete the disk before moving it to a new
machine, when i move it back anything written on this machine is gone.
If i dont delete the disk, i can move it around between machines. When
it comes back to this machine, anything created is visible. however i
cannot see anything created on other machines. how do i see this?

[Hajo Ehlers]

On Sep 23, 1:30 pm, bomahony <bomah...@gmail.com> wrote:
> Thanks for both your prompt responses.
>
> Im mucking about with this atm, on two other p505s. I have accessto a
> Lun from a machine, created a filesystem, wrote a few files.
> However, I cant seem to remove the iscsi LUN from this machine. The
> disk is hdisk2 and the iscsi device is iscsi0
> I ran rmdev -dl iscsi0 after unmounting the volume. However, i cannot
> remove the device. I get a
> 0514-062 Cannot perform the requested function because the specified
> device is busy.

A device is busy because it is in use. Umounting a fs closes the
underlying LV but the VG should be active thus its hdisk.
To be honest i do not whether a varied on VG will keep a hdisk open or
not.
Thus i would try the following:
- umount all FS from a VG using a given iscsi based hdisk
- undefine the iscsi hdisk with rmdev -l ( setting it to defined )
- In case it fails varyof the VG and redo the previous step
- remove the iscsi device with smitty or rmdev -l ( setting it to
defined ) - depends how you configured you iscsi ( ODM or flat file
based )

To see valid parent for your hdisk2 issue
$ lsparent -C -l hdisk2
You can take the output from this command to get the parent device
a.s.o ( bottom up )

In case you know the top device you can use the lsdev command.

Example:
$ lsdev -p fcs0 -S A | awk ' { print $1 '} | xargs -n1 -I{} lsdev -p
{} -S A # should be a script to get all level recursive script

This way to get a device tree iscsi0 , .... , hdisk2 . There is maybe
a better solution .

cheers
Hajo

[bomahony]

Was a little bit dazed after lunch. Seems I wasnt mounting the device
correctly, hence the issues. I can now seem to move it around without
deleting the iscsi0 device.

[bomahony]

Now i have taken this LUN, and assigned it to VIO_server. It is set up
as vhost8. I assigned this to a mLPAR, pvm_server1 using dynamic
partitioning. I ran cfgmgr, and the disk appeared as hdisk1. However:
# lspv
hdisk0 00018afa85a9dfef rootvg
active
hdisk1 none None

# importvg -y tinyiscsilv01 hdisk1
0516-066 lqueryvg: Physical volume is not a volume group member.
Check the physical volume name specified.
0516-066 lqueryvg: Physical volume is not a volume group member.
Check the physical volume name specified.
0516-1140 importvg: Unable to read the volume group descriptor area
on specified physical volume.

# lqueryvg -p hdisk1 -At
0516-304 lqueryvg: Unable to find device id hdisk1 in the Device
Configuration Database.
0516-066 lqueryvg: Physical volume is not a volume group member.
Check the physical volume name specified.

In smit, it appears as a phyical disk, but doesnt appear under any
other options.

On the VIO_server, where it is assigned as a iSCSI disk:
$ lspv
NAME PVID VG
STATUS
hdisk0 00018afa8221fa12 rootvg
active
hdisk1 00018afa70e83fa0 rootvg_clients
active
hdisk2 0001f26adcaba497 tinyiscsivg
active

So my virtual server can see the disk, but i have no idea why it cant
find PVID etc...

From the HMC
VIO_Server1(2)
99
Active
vhost8
tinyiscsilv01
server1(3)
49

Im guessing from the importvg errors, that it cant see the VG on it?
(this LUN still has the created VG and LV from before i started
messign with LPARS - ie when i was deqaling with physical machines)
hdisk1

[leszekwd]

You can try:
chdev -a pv=yes hdisk1 on virtual server and then importvg.

Best Regards,
Leszek Duda