Sybase and very large RAID devices
Paul Burry
Recently we have begun using Sybase on a couple of hosts (both Sun and SGI)
with large Clariion Raid devices. Both of these OSes limit the number of
partitions that can be made on a logical device and Sybase itself limits
the size of a database device.
So, the question is..
How do people with large Raid/disk devices configure them for Sybase?
Is it possible to make huge (>2GB) partitions and use DISK INIT's VSTART
option to refer to 2GB slices within a N*2GB partition?
ie.
DISK INIT name="syb001",physname="/dev/rdsk/dks7d0l0s2",
vdevno=2,size=1048576,vstart=0
GO
DISK INIT name="syb002",physname="/dev/rdsk/dks7d0l0s2",
vdevno=3,size=1048576,vstart=1048576
GO
DISK INIT name="syb003",physname="/dev/rdsk/dks7d0l0s2",
vdevno=4,size=1048576,vstart=2097152
GO
DISK INIT name="syb004",physname="/dev/rdsk/dks7d0l0s2",
vdevno=5,size=1048576,vstart=3145728
GO
DISK INIT name="syb005",physname="/dev/rdsk/dks7d0l0s2",
vdevno=6,size=1048576,vstart=4194304
GO
DISK INIT name="syb006",physname="/dev/rdsk/dks7d0l0s2",
vdevno=7,size=1048576,vstart=5242880
GO
DISK INIT name="syb007",physname="/dev/rdsk/dks7d0l0s2",
vdevno=8,size=1048576,vstart=6291456
GO
DISK INIT name="syb008",physname="/dev/rdsk/dks7d0l0s2",
vdevno=9,size=1048576,vstart=7340032
GO
Does this work?
What do others do?
..Paul
--
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Paul Burry
Voice: (613)-991-7329 Internet: pbu...@cse.dnd.ca
Fax: (613)-991-8436
Pablo Sanchez
In article <5jgjvf$9...@manitou1.cse.dnd.ca>, pbu...@cse.dnd.ca writes:
> Recently we have begun using Sybase on a couple of hosts (both Sun and SGI)
> with large Clariion Raid devices. Both of these OSes limit the number of
> partitions that can be made on a logical device and Sybase itself limits
> the size of a database device.
>
> So, the question is..
>
> How do people with large Raid/disk devices configure them for Sybase?
>
How big is your device and what RAID level are you using? We have a
RAID device with 20 disks... but then we're using part of the disks
with RAID 0+1... kinda eats 'em up.
--
Pablo Sanchez | wk: 415.933.3812| pg: 800.930.5635 -or- pab...@pager.sgi.com
--------------+-----------------+--------------------------------------------
pa...@sgi.com ... when mailing me, place "not spam" in the Subject
Mike Maas
It won't work because you can only have one device with the same
physical name. Otherwise vstart would work and you might even be able
to work around it by addressing the disk through several different
links in the os. I doubt that you would find it worth the
administrative overhead though.
The 2 gig limit will go away soon in release 11.
Another thing you might want to consider though it sounds like it
might be a bit late at this point is that many disk errors are not
trapped by the os or disk os. When this is the case in devices that
are directly addressable like scsi or logical volume managers the
physical disk with the problem can be tracked down because all the
addressing is sequential meaning that if we know where the vstart is
on a device we can tell on which device the problem is happening.
Unfortunately this is not possible in raid because their is no direct
relationship between any place on the device and where it might be
located on a physical disk. As a result when you have similar
problems on raid your only way to find the bad disk is to start
removing the disks one by one until you find the problem.
This is only the case when the os or raid os fails to detect the
error. When the os or raid os detects the error it will hopefully
lead you to the faulty disk. Unfortunately a significant number of
hardware/os caused corruption problems go undetected by their error
detection software.
pbu...@cse.dnd.ca (Paul Burry) wrote:
>Recently we have begun using Sybase on a couple of hosts (both Sun and SGI)
>with large Clariion Raid devices. Both of these OSes limit the number of
>partitions that can be made on a logical device and Sybase itself limits
>the size of a database device.
>
>So, the question is..
>
> How do people with large Raid/disk devices configure them for Sybase?
>
Mike Maas (510) 658-0487
Masimo Consulting 2701 Woolsey
specializing in Sybase DBA Issues and Data Recovery Berkeley, CA 94705-2606
Glenn
In article <335c490f...@10.0.2.1>, mas...@iname.com (Mike Maas) wrote:
>
> Another thing you might want to consider though it sounds like it
> might be a bit late at this point is that many disk errors are not
> trapped by the os or disk os. When this is the case in devices that
> are directly addressable like scsi or logical volume managers the
> physical disk with the problem can be tracked down because all the
> addressing is sequential meaning that if we know where the vstart is
> on a device we can tell on which device the problem is happening.
>
> Unfortunately this is not possible in raid because their is no direct
> relationship between any place on the device and where it might be
> located on a physical disk. As a result when you have similar
> problems on raid your only way to find the bad disk is to start
> removing the disks one by one until you find the problem.
>
> This is only the case when the os or raid os fails to detect the
> error. When the os or raid os detects the error it will hopefully
> lead you to the faulty disk. Unfortunately a significant number of
> hardware/os caused corruption problems go undetected by their error
> detection software.
> >
>=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> >Paul Burry
What you describe is valid for RAID devices which do not verify the stripe
ECC during access. That means all RAID5 capable systems. We have seen
605 errors with several RAID systems.
If you want to know about a RAID system that verifies the RAID stripe ECC
before it passes data to the host, take a look at http://www.baydel.com
Glenn B
Jung S. Lee
Mike Maas wrote:
>
> The 2 gig limit will go away soon in release 11.
I am looking at System Adm 11 manual, and it says the maximum size of
database device is 2 GB.
> >So, the question is..
> >
> > How do people with large Raid/disk devices configure them for Sybase?
> >
My case is different from yours but hope this helps.
We have T 500 with Disk Array model 20(nikey) box(80GB) which can be
configure to any RAID. We set that up as raid 0/1 and using Logical
Volume Manager we created 20 2 GB logical volumes later used as Sybase
devices to create 40 GB database.
> >..Paul
> >--
> >=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> >Paul Burry
> >Voice: (613)-991-7329 Internet: pbu...@cse.dnd.ca
> >Fax: (613)-991-8436
>
> Mike Maas (510) 658-0487
> Masimo Consulting 2701 Woolsey
> specializing in Sybase DBA Issues and Data Recovery Berkeley, CA 94705-2606
--
Jay Lee AT&T Production Processing Efficiency Group
Lake Mary, FL 32746
Al Bledsoe
Jung S. Lee (orb...@worldnet.att.net) wrote:
: Mike Maas wrote:
: >
: > The 2 gig limit will go away soon in release 11.
: I am looking at System Adm 11 manual, and it says the maximum size of
: database device is 2 GB.
Probably it is the Adpative Server v11.5 that is being referred.
But, in the meantime, if a server is limited to 255 devices and
disk init is limited to 2 GB, how do you get bigger than 510 GB?
I thought there were multi-terabyte Sybase Installs out there!
Al Bledsoe
Independent Consultant | Compressed Graphics: .
Mike Maas
Gl...@gbrack.demon.co.uk (Glenn) wrote:
I'm interested in knowing more about this. However the problem does
not appear to be carelessness at least in the case of similar problems
on straight scsi drives but unforeseen ways of the device failing or
software bugs (well, maybe that could be considered careless <g>). I
know of at least two scsi driver bugs on one platform that were
responsible for 605 and 2503 errors in specific high traffic
situations. On another platform there was a fastwide scsi bus
problems that the vendor refused to even consider to be their problem
for more than 18 months that caused 821 and 605 errors. In both of
these cases the problem was obscure enough that it only occurred
occasionally and since the problem occurred in Sybase the clients and
the hardware vendors both assumed that the problem was a sybase
problem. In most disk corruption cases Sybase is simply the
victim/messenger of the problem and not the cause.
My point is even if a raid vendor is assiduous in their coding,
verification and manufacturing there are still things that they will
miss just like the two vendors above did. When that happens on
straight scsi the physical device is easily detected because you can
see where the corruption is happening. On raid, unfortunately that is
not the case and again the only way to find the problem when that
occurs is the arduous technique described earlier of pulling each
drive out until you find the bad drive. In a raid pack with 20 drives
you might be lucky and hit it on the first drive pulled, then again it
might be the 20th. I don't know if I would want to open myself up to
such a large window of vulnerability.
Mike
Jung S. Lee
I don't know if Navigation server was meant for this or not. But if I am
right, you can have multiple SQL servers access through Navigation
server. Again, I cannot be sure so if anyone out there knows more about
this, please shed some light on this.
Thanks.
Dick Brooks
I've noticed that for some platforms the release bulletins for 11.0.1 and
11.0.2 correct the statement in the Admin Guide about 2 gig being the
limit; look under Known Problems or Documentation Updates. It seems the
limit is now platform dependent but up to 32 gig.
By the way, I haven't yet tested larger devices, just read that they're
supposed to work.
Paul Burry
In article <5jh82c$3...@mew.corp.sgi.com>, Pablo Sanchez <pa...@sgi.com> wrote:
|In article <5jgjvf$9...@manitou1.cse.dnd.ca>, pbu...@cse.dnd.ca writes:
|> with large Clariion Raid devices. Both of these OSes limit the number of
|> partitions that can be made on a logical device and Sybase itself limits
|> the size of a database device.
|>
|>
|> How do people with large Raid/disk devices configure them for Sybase?
|
|RAID device with 20 disks... but then we're using part of the disks
|with RAID 0+1... kinda eats 'em up.
I have at my disposal up to four Clariion devices with 20 8.8 GB
drives in each. We have not yet decided on the RAID levels to use,
but it'll likely be a combination of RAID 5 and RAID 0+1. Any suggestions
about what level(s) of RAID to use for Sybase would certainly be welcome.
As far as the disk usage goes, this is how I understand the limitations
of Sybase, Irix and the Clariions:
Irix only supports ~13 user defined partitiions and Sybase
only supports a 2GB partition. If I configure a brick as either
RAID 3 or 5, the LUN will have ~32-35 GB of useable space. Irix can
create 14 usable partitions (16 less the volume header and whole disk).
So, Sybase will be able to use 14x2GB at most and the rest of the
space will be unusable. Correct?
Pablo Sanchez
In article <5jis3u$k...@manitou1.cse.dnd.ca>, pbu...@cse.dnd.ca writes:
> In article <5jh82c$3...@mew.corp.sgi.com>, Pablo Sanchez <pa...@sgi.com> wrote:
> |How big is your device and what RAID level are you using? We have a
> |RAID device with 20 disks... but then we're using part of the disks
> |with RAID 0+1... kinda eats 'em up.
>
> I have at my disposal up to four Clariion devices with 20 8.8 GB
> drives in each. We have not yet decided on the RAID levels to use,
> but it'll likely be a combination of RAID 5 and RAID 0+1. Any suggestions
> about what level(s) of RAID to use for Sybase would certainly be welcome.
If your goal is for performance (read/write) go with RAID 0+1.
Dedicating four disks (2 + 2) gives you about 200 IOPS (I/O per sec)
without saturating the bus. I believe if you try (3 + 3) that does
not scale and you get about 240 - 260 IOPS (this is because the bus
becomes saturated). Use a stripe size (initially) of 64K... I'll get
you a revised value from a benchmark someone did with Sybase on SGI
and the Clariion...
Back to a LUN of four disks, that's 8.8 * 2 so that's 17.6g. With 14
partitions, you should be able to partition that sucker up into 2 gig
chunks.
Hope this helps you...
Glenn
> Dedicating four disks (2 + 2) gives you about 200 IOPS (I/O per sec)
> without saturating the bus. I believe if you try (3 + 3) that does
> not scale and you get about 240 - 260 IOPS (this is because the bus
> becomes saturated). Use a stripe size (initially) of 64K... I'll get
> you a revised value from a benchmark someone did with Sybase on SGI
> and the Clariion...
>
No way does 260 IOPs saturate a bus (unless its a DEC Unibus).
We can get over 5000+ IOPs per sec, so this saturation you are talking
about must be either application or specific subsystem related.
Glenn B
Paul Burry
Glenn
In article <5jostn$n...@manitou1.cse.dnd.ca>, pbu...@cse.dnd.ca (Paul
Burry) wrote:
> In article <Glenn-24049...@gbrack.demon.co.uk>,
> Glenn <Gl...@gbrack.demon.co.uk> wrote:
> |In article <5jl7rv$d...@mew.corp.sgi.com>, pa...@sgi.com wrote:
> |
> |> Dedicating four disks (2 + 2) gives you about 200 IOPS (I/O per sec)
> |> without saturating the bus. I believe if you try (3 + 3) that does
> |> not scale and you get about 240 - 260 IOPS (this is because the bus
> |> becomes saturated). Use a stripe size (initially) of 64K... I'll get
> |> you a revised value from a benchmark someone did with Sybase on SGI
> |> and the Clariion...
> |>
> |
> |No way does 260 IOPs saturate a bus (unless its a DEC Unibus).
> |
> |We can get over 5000+ IOPs per sec, so this saturation you are talking
> |about must be either application or specific subsystem related.
> |
> |Glenn B
>
> I believe he meant the SCSI bus.
>
> ..Paul
> ..Paul
If he meant the SCSI bus then he is wrong (which is why I replied to the
comment)
260 IOP's per sec equates to 520 KB/s with the typical 2K Sybase transfer.
This is just over 5% of a 10 MB/s SCSI bus, 2.5% of a 20 MB/s SCSI bus and
1.25% of ULTRA etc.
It doesn't take a rocket scientist to see that SCSI bus saturation is not
going to be an issue.
By the way, a 64 K stripe size might have an effect on write performance
and might also introduce stripe alignment issues. I have a paper on the
topic if you are interested.
Glenn B
Pablo Sanchez
In article <Glenn-25049...@gbrack.demon.co.uk>, Gl...@gbrack.demon.co.uk writes:
>
> If he meant the SCSI bus then he is wrong (which is why I replied to the
> comment)
>
> 260 IOP's per sec equates to 520 KB/s with the typical 2K Sybase transfer.
>
> This is just over 5% of a 10 MB/s SCSI bus, 2.5% of a 20 MB/s SCSI bus and
> 1.25% of ULTRA etc.
>
> It doesn't take a rocket scientist to see that SCSI bus saturation is not
> going to be an issue.
Well that's the problem, I'm not even close to a rocket scientist...
hey what do you want for free... :-)... anyway, on your 5000+ IOPS,
was that scaling linearly? How many disks? Perspiring minds would
like to know.
> By the way, a 64 K stripe size might have an effect on write performance
> and might also introduce stripe alignment issues. I have a paper on the
> topic if you are interested.
Does it apply to Clariion?
Robert David
In <5jtvff$c...@mew.corp.sgi.com> news_...@mew.corp.sgi.com (Pablo
Glenn should put a disclaimer in his emails that he works for RAID
manufacturer Baydel (when he's promoting white papers, performance
results, etc.)...
>> 260 IOP's per sec equates to 520 KB/s with the typical 2K Sybase
transfer.
>> It doesn't take a rocket scientist to see that SCSI bus saturation
is not going to be an issue.
>
>Well that's the problem, I'm not even close to a rocket scientist...
>hey what do you want for free... :-)... anyway, on your 5000+ IOPS,
>was that scaling linearly? How many disks? Perspiring minds would
>like to know.
>
>> By the way, a 64 K stripe size might have an effect on write
performance and might also introduce stripe alignment issues. I have a
paper on the topic if you are interested.
>
>Does it apply to Clariion?
The paper focuses on his Baydel box. I'm sure that Glenn will send you
a copy.
Pablo (and others), check out RAID vendor MTI's (www.mti.com) web site
(disclaimer: my former employer and current customer). I think they
might have an online white paper on something similar to what Glenn is
talking about.
Later,
Robert David
(working for RAID-helping solid-state disk SSD supplier, Imperial
Technology)
John McVicker
Al,
The limits are based on Operating System API limits for lseek and
various Async I/O operations too. Right now, you can use 4GB
devices for Digital Unix and I believe NT. NT had a problem a while
back which allowed 4GB devices for the dataserver but did have
some problems with 4GB through the Backup Server. So, I would only
suggest that you go beyond 2GB is you have Digital Unix 3.x or 4.x
only.
Sybase will support > 2GB devices when Operating Systems bring
out their APIs with 64bit addressing. Such as Solaris 2.6, etc.
Sure, there are multi-terabyte installations out there - but they
use *more than one* SQL Server to do it.
Al Bledsoe wrote:
>
> Jung S. Lee (orb...@worldnet.att.net) wrote:
> : Mike Maas wrote:
> : >
> : > The 2 gig limit will go away soon in release 11.
>
> : I am looking at System Adm 11 manual, and it says the maximum size of
> : database device is 2 GB.
>
> Probably it is the Adpative Server v11.5 that is being referred.
>
> But, in the meantime, if a server is limited to 255 devices and
> disk init is limited to 2 GB, how do you get bigger than 510 GB?
> I thought there were multi-terabyte Sybase Installs out there!
>
> Al Bledsoe
> Independent Consultant | Compressed Graphics: .
--
John McVicker
Principal Consultant, District Lead Architect
Sybase Professional Services
Pennsylvania/New Jersey District
301-896-1765
mcvi...@sybase.com, mcvi...@bellatlantic.net,
JohnMc...@compuserve.com
Al Bledsoe
I understand the 2 gb problem, but I don't understand the 255 devices
per server constaint.
John McVicker (mcvi...@bellatlantic.net) wrote:
: Al,
Kevin Sherlock
In article <abb2uE9...@netcom.com> Al Bledsoe, ab...@netcom.com
writes:
>I understand the 2 gb problem, but I don't understand the 255 devices
>per server constaint.
Sybase uses a "virtual page" numbering schema which includes the
device number in the high-order byte of the virtual page number.
One byte equals 256 devices (not 255). With this schema, and 64 bit
OS's (or more accurately, a 64 bit POSIX api), you would still only be
able to address an 8Gb device.
-----------------------------------------
Kevin Sherlock
USWEST Dex
Omaha, NE
anti-spam: ksherlo-at-uswest-dot-com
-----------------------------------------
John McVicker
Kevin,
With 64-bit file management (Posix) and SQL Server's ability to
have virtual pages on each device, let's look at how it works.
The sysdevices table in master stores a "low" and "high" value.
This is a two-valued column. First 8-bits (1 byte) is the vdevno.
The next 24-bits are the page number on that device, starting at 0.
So, the limit is 16777215 pages per device. This, multiple by 2048,
the SQL Server page size is 34359736320 bytes. Divided out, it
comes to just under 32.767GB or 32767.998 MB (so real usable space
would be 32767 MB.
I don't know the Posix spec, but if this is truly usable space, one
device could possibly address 32+ GB of pages on one device. Does
the Posix spec pertain to the 8GB limit you mentioned? I am happy
to just get 4GB per device - that takes SQL Server up to a limit of
1 Terabyte (with 4GB * 256 devices).
--