Dual shared SCSI device
Roberto Zini
I'd like to setup a single shared SCSI disk between 2 SCSI machines
configured as follows:
CPU PIII-700Mhz + 256MB of RAM
Adaptec 39160 dual channel SCSI controller (see below)
2 Intel Pro10/100 NICs
My aim is to configure either OS5, OU8 or Linux on the above machines
as to share a filesystem built on the shared disk.
According to the docs I have, the SCSI equipment (HAs, disks and so on)
must be able to function in a multi-initiator and multi-target way; as
far as I know, the Adaptec 39160 __should__ be able to support this
feature although I was unable to find any evidence of that in the
manuals.
Since the 39160 is a dual channel SCSI controller, I configured the HA
on the above boxes as follows:
HostA + HostB
SCSI Channel A:
SCSI ID=7
boot disk SCSI ID=0
HostA SCSI Channel B:
SCSI ID=14
HostB SCSI Channel B:
SCSI ID=15
Shared disk on SCSI Channel B:
SCSI ID=13
I make sure I placed a shunt over jumper J13 as suggested on the Adaptec's
User's Manual os to ensure that the SCSI channel B got "terminated" when one
PC suffers from a power down condition.
Also, via the SCSISetup (CTRL-A) BIOS utility I selected:
. reset SCSI BUS at IC initialization = DISABLED
. under "SCSI Device Configuration" -> device ID=13 (shared disk):
Enable Write Back Cache = NO
SCSI channel B has 3 different devices (2 HAs + 1 SCSI disk) so there
shouldn't be any conflict among them.
On the above boxes I installed Caldera's OpenLinux 3.1 and both boxes
are able to mount the shared disk at the same time (presently, I'm not
worried about data sinchronization issues since I'm only testing the
HW configuration).
On the same disk (but on a different partition) I installed OpenUNIX 8.0.0
but, during boot, the first node which gets powered on is able to recognize
the external disk whereas the second one gives a series of
WARNING: adst70: Device busy [<number] (ha/targetID/lun 1/13/0)
messages before reaching the multi-user mode. Of course, the second box
is unable to "detect" the shared disk.
Before going on installing OpenServer5 and since I'm not a SCSI expert, I'd
like to know the opinion of the group about the above SCSI configuration and
other suggestions/hints/experiences you'll be able to provide me with.
Thanks,
Roberto
--
---------------------------------------------------------------------
Roberto Zini email : r.z...@strhold.it
Technical Support Manager -- Strhold Evolution Division R.E. (ITALY)
---------------------------------------------------------------------
"Has anybody around here seen an aircraft carrier?"
(Pete "Maverick" Mitchell - Top Gun)
John Hughes
3C9A0139...@strhold.it...
> Howdy.
Hi Roberto!
> I'd like to setup a single shared SCSI disk between 2 SCSI machines
> configured as follows:
>
> CPU PIII-700Mhz + 256MB of RAM
> Adaptec 39160 dual channel SCSI controller (see below)
> 2 Intel Pro10/100 NICs
What's the disk? In my tests (NSC don't ya know) I found
some disks couldn't do it. (Old quantum ATLAS IV's, so I
doubt this is your problem).
> My aim is to configure either OS5, OU8 or Linux on the above machines
> as to share a filesystem built on the shared disk.
What filesystem? If it's not readonly you'll need to port GFS :-)
> On the above boxes I installed Caldera's OpenLinux 3.1 and both boxes
> are able to mount the shared disk at the same time (presently, I'm not
> worried about data sinchronization issues since I'm only testing the
> HW configuration).
so, no problem with OpenLinux?
> On the same disk (but on a different partition) I installed OpenUNIX 8.0.0
> but, during boot, the first node which gets powered on is able to
> recognize the external disk whereas the second one gives a series of
>
> WARNING: adst70: Device busy [<number] (ha/targetID/lun 1/13/0)
>
> messages before reaching the multi-user mode. Of course, the second box
> is unable to "detect" the shared disk.
I saw this in my NSC configs *BEFORE* I configured the disks
in vxvm. My guess was that UW (sorry, I don't think I'll ever
get into the habit of calling it "OU") is doing a RESERVE operation
on the disk.
Roberto Zini
>
> "Roberto Zini" <r.z...@strhold.it> a écrit dans le message de news:
> 3C9A0139...@strhold.it...
> > Howdy.
>
> Hi Roberto!
Hi John !
Nice to read from you.
>
> > I'd like to setup a single shared SCSI disk between 2 SCSI machines
> > configured as follows:
> >
> > CPU PIII-700Mhz + 256MB of RAM
> > Adaptec 39160 dual channel SCSI controller (see below)
> > 2 Intel Pro10/100 NICs
>
> What's the disk? In my tests (NSC don't ya know) I found
> some disks couldn't do it. (Old quantum ATLAS IV's, so I
> doubt this is your problem).
The disc is a Quantum ATLAS 10K2-TY092L, firmware version DA40, 8.7GB.
>
> > My aim is to configure either OS5, OU8 or Linux on the above machines
> > as to share a filesystem built on the shared disk.
>
> What filesystem? If it's not readonly you'll need to port GFS :-)
Well, I was assigned the task to find an affordable, low-cost solution
for a high-availability system so (ala Reliant-HA one) so, at a given moment,
only a single machine will have access to the filesystem. Should it crash,
the failover one will take over by mounting the shared disk and restarting
the service.
I know about GPF for Linux but I've never tried it; would you like to share
your experience with it (perhaps privately since this is not a Linux group) ?
>
> > On the above boxes I installed Caldera's OpenLinux 3.1 and both boxes
> > are able to mount the shared disk at the same time (presently, I'm not
> > worried about data sinchronization issues since I'm only testing the
> > HW configuration).
>
> so, no problem with OpenLinux?
As I said, not at all: both boxes are able to mount the shared Linux ext2
filesystem without problems (again, data sync is not an issue at the
moment) so the base HW __should__ be OK.
>
> > On the same disk (but on a different partition) I installed OpenUNIX 8.0.0
> > but, during boot, the first node which gets powered on is able to
> > recognize the external disk whereas the second one gives a series of
> >
> > WARNING: adst70: Device busy [<number] (ha/targetID/lun 1/13/0)
> >
> > messages before reaching the multi-user mode. Of course, the second box
> > is unable to "detect" the shared disk.
>
> I saw this in my NSC configs *BEFORE* I configured the disks
> in vxvm. My guess was that UW (sorry, I don't think I'll ever
> get into the habit of calling it "OU") is doing a RESERVE operation
> on the disk.
Uhm ... that sounds like a good one; I mean, given that under Linux
everything seems to work find (ie, both boxes are able to access the
shared disk), I suppose the HW config is fine. In the next few days I'll
have a look at the /etc/conf/pack.d/asdt70/space.c file (from memory
so the path couldn't be 100% right) since it seems that you can fiddle
with some driver parameters.
Also, the vxvm "factor" rings a bell ...
steve overy
> John Hughes wrote:
> >
> > "Roberto Zini" <r.z...@strhold.it> a écrit dans le message de news:
> > 3C9A0139...@strhold.it...
etc...
In general it should work, certainly we have sites doing stuff like
this (ie disks shared between systems).
Question, are you using a scsi "Y" cable or individual cables from
each of the systems to a dual ported disk unit?
This sounds a bit like scsi termination type stuff. We sometimes use
independent "active" scsi terminators to fix the "powering down a
system" problem.
Question, what sort of scsi cable lengths are you using?
You have "SCSI reset" disabled - I have rx'd comment that says modern
drivers should be able to cope with (the remote node starting up &
sending a reset). However which modern drivers implement this is more
of a question. Disabled is probably safe enuff!
Not certain about the "register with vxvm" comment - you dont need to
put the disks under ODM control, certainly there are excelent reasons
for so doing, but you dont "need" to. I have reliant sites that do not
use ODM.
I wonder if the "sdighost" command would reveal anything... I suspect
not since the fault is very low level (hardware not recognised), but
might be worth a whirl.
I assume that if you turn off either system the disks are recognised
by the other system?
steve
Roberto Zini
Steve and John,
thank you for your hints !
I eventually tracked down the problem thanks to the invaluable support
I got from Caldera (before starting a flame war I'd like to point out that we
DO PAY a lot of bucks for their support so don't start complaining about the
lack of responses you might have experienced with them) ; in fact, although I
created a valid Unix partition on the shared disk, I forgot to create a valid
__filesystem__ on it :-(
Thus making, the first booted OpenUNIX8 did search for a valid VTOC scheme on the
external storage and it locked out other SCSI requests from the second
machine (and that was a known issue they made me aware of).
As soon as I created (by using "diskadd") a valid filesystem slice and
rebooted (just to be sure) both boxes, they were able to "see" the
shared device without the "device busy" message.
I also tried to mount the filesystem on a box (which succeeded) but when
I tried to do the same on the second one, I got an "inode corruption"
message and the operation failed (but that was expected since OU8 does
not have a kind of "global filesystem" support).
So the key point is to creare a valid __FILESYSTE__ (an Unix partition
will __NOT__ suffice) before even trying to access the shared disk
(even with fdisk).
Hope this helps !
Best,
Roberto Zini
[My ISP's news server has been playing up lately so dunno if the following
actually got out so here it comes again]
Steve and John,
thank you for your hints !
I eventually tracked down the problem thanks to the invaluable support
I got from Caldera (before starting a flame war I'd like to point out that we
DO PAY a lot of bucks for their support so don't start complaining about the
lack of responses you might have experienced with them) ; in fact, although I
created a valid Unix partition on the shared disk, I forgot to create a valid
__filesystem__ on it :-(
Thus making, the first booted OpenUNIX8 did search for a valid VTOC scheme on the
external storage and it locked out other SCSI requests from the second
machine (and that was a known issue they made me aware of).
As soon as I created (by using "diskadd") a valid filesystem slice and
rebooted (just to be sure) both boxes, they were able to "see" the
shared device without the "device busy" message.
I also tried to mount the filesystem on a box (which succeeded) but when
I tried to do the same on the second one, I got an "inode corruption"
message and the operation failed (but that was expected since OU8 does
not have a kind of "global filesystem" support).
So the key point is to creare a valid __FILESYSTEM__ (an Unix partition
will __NOT__ suffice) before even trying to access the shared disk
(even with fdisk).
Hope this helps !
Best,
steve overy
> steve overy wrote:
> >
> > Roberto Zini <r.z...@strhold.it> wrote in message news:<3C9AE6FD...@strhold.it>...
> > > John Hughes wrote:
> > > >
> > > > "Roberto Zini" <r.z...@strhold.it> a écrit dans le message de news:
> > > > 3C9A0139...@strhold.it...
> >
> > etc...
> >
severe pruning (it is spring-time)
>
> I also tried to mount the filesystem on a box (which succeeded) but when
> I tried to do the same on the second one, I got an "inode corruption"
> message and the operation failed (but that was expected since OU8 does
> not have a kind of "global filesystem" support).
>
> So the key point is to creare a valid __FILESYSTEM__ (an Unix partition
> will __NOT__ suffice) before even trying to access the shared disk
> (even with fdisk).
>
> Hope this helps !
>
> Best,
> Roberto
Roberto,
I'm interested in your "inode corruption" report - I've seen stuff
like that in syslogs (a vxfs msgcnt report) from reliant sites ie
shared disks. I've no proof both nodes were mounting but I'd be
interested to know more details. Please mail direct to
steve...@unisys.com if theres lots of material - no hurry, I'm off
on fly&fix for a week.
steve
Roberto Zini
Steve,
perhaps I didn't make myself clear enough in my above statements.
By "inode corruption" I mean that you get this error when, outside of
Reliant-HA control, you try to mount the external storage on (eg) NodeB
when it's already mounted by (eg) NodeA.
The above was the only test I've been able to perform since the HW
installation; without a valid VTOC stamped on the external storage,
only NodeA was able to mount it whereas NodeB would simply fail. I've
only performed the "simultaneous mount" as to demonstrate that both
nodes were able to access (mount) the external disk.
The problem now is to configure ReliatHA on top of the above HW config
but time is the key issue - in the sense that I failed to allocate a time
slot in the past 2 weeks :-(
I'll keep you informed as the situation unfolds.
steve overy
Roberto,
there shouldnt be a problem configuring a disk mount under reliant (so
reliant controls which system has the disk) - however there are
occassions where reliant loses the heartbeat & the switchover is not
able to co-ordinate closedown/startup properly - normally due to
system overload. some of the more recent ptfs have fixes to some of
the problems here. Putting the disks under ODM control under reliant
makes this safer due to the need to import/export.
but I'm really interested to see if the format of your inode
corruptions is similar to the examples i've seen (where it is not
definate both systems had the disks online). Therefore if you've seen
vxfs msgcnt mesg 038 (I've also seen 016/017) it might imply my case
is more definately "split brain" (as veritas call it).
steve