Recently we have begun using Sybase on a couple of hosts (both Sun and SGI) with large Clariion Raid devices. Both of these OSes limit the number of partitions that can be made on a logical device and Sybase itself limits the size of a database device.
So, the question is..
How do people with large Raid/disk devices configure them for Sybase?
Is it possible to make huge (>2GB) partitions and use DISK INIT's VSTART option to refer to 2GB slices within a N*2GB partition?
ie. DISK INIT name="syb001",physname="/dev/rdsk/dks7d0l0s2", vdevno=2,size=1048576,vstart=0 GO DISK INIT name="syb002",physname="/dev/rdsk/dks7d0l0s2", vdevno=3,size=1048576,vstart=1048576 GO DISK INIT name="syb003",physname="/dev/rdsk/dks7d0l0s2", vdevno=4,size=1048576,vstart=2097152 GO DISK INIT name="syb004",physname="/dev/rdsk/dks7d0l0s2", vdevno=5,size=1048576,vstart=3145728 GO DISK INIT name="syb005",physname="/dev/rdsk/dks7d0l0s2", vdevno=6,size=1048576,vstart=4194304 GO DISK INIT name="syb006",physname="/dev/rdsk/dks7d0l0s2", vdevno=7,size=1048576,vstart=5242880 GO DISK INIT name="syb007",physname="/dev/rdsk/dks7d0l0s2", vdevno=8,size=1048576,vstart=6291456 GO DISK INIT name="syb008",physname="/dev/rdsk/dks7d0l0s2", vdevno=9,size=1048576,vstart=7340032 GO
In article <5jgjvf$...@manitou1.cse.dnd.ca>, pbu...@cse.dnd.ca writes: > Recently we have begun using Sybase on a couple of hosts (both Sun and SGI) > with large Clariion Raid devices. Both of these OSes limit the number of > partitions that can be made on a logical device and Sybase itself limits > the size of a database device.
> So, the question is..
> How do people with large Raid/disk devices configure them for Sybase?
How big is your device and what RAID level are you using? We have a RAID device with 20 disks... but then we're using part of the disks with RAID 0+1... kinda eats 'em up. -- Pablo Sanchez | wk: 415.933.3812| pg: 800.930.5635 -or- pabl...@pager.sgi.com --------------+-----------------+------------------------------------------ -- pa...@sgi.com ... when mailing me, place "not spam" in the Subject
It won't work because you can only have one device with the same physical name. Otherwise vstart would work and you might even be able to work around it by addressing the disk through several different links in the os. I doubt that you would find it worth the administrative overhead though.
The 2 gig limit will go away soon in release 11.
Another thing you might want to consider though it sounds like it might be a bit late at this point is that many disk errors are not trapped by the os or disk os. When this is the case in devices that are directly addressable like scsi or logical volume managers the physical disk with the problem can be tracked down because all the addressing is sequential meaning that if we know where the vstart is on a device we can tell on which device the problem is happening.
Unfortunately this is not possible in raid because their is no direct relationship between any place on the device and where it might be located on a physical disk. As a result when you have similar problems on raid your only way to find the bad disk is to start removing the disks one by one until you find the problem.
This is only the case when the os or raid os fails to detect the error. When the os or raid os detects the error it will hopefully lead you to the faulty disk. Unfortunately a significant number of hardware/os caused corruption problems go undetected by their error detection software.
pbu...@cse.dnd.ca (Paul Burry) wrote: >Recently we have begun using Sybase on a couple of hosts (both Sun and SGI) >with large Clariion Raid devices. Both of these OSes limit the number of >partitions that can be made on a logical device and Sybase itself limits >the size of a database device.
>So, the question is..
> How do people with large Raid/disk devices configure them for Sybase?
>Is it possible to make huge (>2GB) partitions and use DISK INIT's VSTART >option to refer to 2GB slices within a N*2GB partition?
In article <335c490f.11942...@10.0.2.1>, mas...@iname.com (Mike Maas) wrote:
> Another thing you might want to consider though it sounds like it > might be a bit late at this point is that many disk errors are not > trapped by the os or disk os. When this is the case in devices that > are directly addressable like scsi or logical volume managers the > physical disk with the problem can be tracked down because all the > addressing is sequential meaning that if we know where the vstart is > on a device we can tell on which device the problem is happening.
> Unfortunately this is not possible in raid because their is no direct > relationship between any place on the device and where it might be > located on a physical disk. As a result when you have similar > problems on raid your only way to find the bad disk is to start > removing the disks one by one until you find the problem.
> This is only the case when the os or raid os fails to detect the > error. When the os or raid os detects the error it will hopefully > lead you to the faulty disk. Unfortunately a significant number of > hardware/os caused corruption problems go undetected by their error > detection software.
What you describe is valid for RAID devices which do not verify the stripe ECC during access. That means all RAID5 capable systems. We have seen 605 errors with several RAID systems.
If you want to know about a RAID system that verifies the RAID stripe ECC before it passes data to the host, take a look at http://www.baydel.com
> The 2 gig limit will go away soon in release 11.
I am looking at System Adm 11 manual, and it says the maximum size of database device is 2 GB.
> >So, the question is..
> > How do people with large Raid/disk devices configure them for Sybase?
My case is different from yours but hope this helps. We have T 500 with Disk Array model 20(nikey) box(80GB) which can be configure to any RAID. We set that up as raid 0/1 and using Logical Volume Manager we created 20 2 GB logical volumes later used as Sybase devices to create 40 GB database.
: > : > The 2 gig limit will go away soon in release 11.
: I am looking at System Adm 11 manual, and it says the maximum size of : database device is 2 GB.
Probably it is the Adpative Server v11.5 that is being referred.
But, in the meantime, if a server is limited to 255 devices and disk init is limited to 2 GB, how do you get bigger than 510 GB? I thought there were multi-terabyte Sybase Installs out there!
Al Bledsoe Independent Consultant | Compressed Graphics: .
>> Another thing you might want to consider though it sounds like it >> might be a bit late at this point is that many disk errors are not >> trapped by the os or disk os. When this is the case in devices that >> are directly addressable like scsi or logical volume managers the >> physical disk with the problem can be tracked down because all the >> addressing is sequential meaning that if we know where the vstart is >> on a device we can tell on which device the problem is happening.
>> Unfortunately this is not possible in raid because their is no direct >> relationship between any place on the device and where it might be >> located on a physical disk. As a result when you have similar >> problems on raid your only way to find the bad disk is to start >> removing the disks one by one until you find the problem.
>> This is only the case when the os or raid os fails to detect the >> error. When the os or raid os detects the error it will hopefully >> lead you to the faulty disk. Unfortunately a significant number of >> hardware/os caused corruption problems go undetected by their error >> detection software.
>What you describe is valid for RAID devices which do not verify the stripe >ECC during access. That means all RAID5 capable systems. We have seen >605 errors with several RAID systems.
>If you want to know about a RAID system that verifies the RAID stripe ECC >before it passes data to the host, take a look at http://www.baydel.com
>Glenn B
I'm interested in knowing more about this. However the problem does not appear to be carelessness at least in the case of similar problems on straight scsi drives but unforeseen ways of the device failing or software bugs (well, maybe that could be considered careless <g>). I know of at least two scsi driver bugs on one platform that were responsible for 605 and 2503 errors in specific high traffic situations. On another platform there was a fastwide scsi bus problems that the vendor refused to even consider to be their problem for more than 18 months that caused 821 and 605 errors. In both of these cases the problem was obscure enough that it only occurred occasionally and since the problem occurred in Sybase the clients and the hardware vendors both assumed that the problem was a sybase problem. In most disk corruption cases Sybase is simply the victim/messenger of the problem and not the cause.
My point is even if a raid vendor is assiduous in their coding, verification and manufacturing there are still things that they will miss just like the two vendors above did. When that happens on straight scsi the physical device is easily detected because you can see where the corruption is happening. On raid, unfortunately that is not the case and again the only way to find the problem when that occurs is the arduous technique described earlier of pulling each drive out until you find the bad drive. In a raid pack with 20 drives you might be lucky and hit it on the first drive pulled, then again it might be the 20th. I don't know if I would want to open myself up to such a large window of vulnerability.
Mike Mike Maas (510) 658-0487 Masimo Consulting 2701 Woolsey specializing in Sybase DBA Issues and Data Recovery Berkeley, CA 94705-2606
> Jung S. Lee (orbi...@worldnet.att.net) wrote: > : Mike Maas wrote: > : > > : > The 2 gig limit will go away soon in release 11.
> : I am looking at System Adm 11 manual, and it says the maximum size of > : database device is 2 GB.
> Probably it is the Adpative Server v11.5 that is being referred.
> But, in the meantime, if a server is limited to 255 devices and > disk init is limited to 2 GB, how do you get bigger than 510 GB? > I thought there were multi-terabyte Sybase Installs out there!
> Al Bledsoe > Independent Consultant | Compressed Graphics: .
I don't know if Navigation server was meant for this or not. But if I am right, you can have multiple SQL servers access through Navigation server. Again, I cannot be sure so if anyone out there knows more about this, please shed some light on this.
Thanks. -- Jay Lee AT&T Production Processing Efficiency Group Lake Mary, FL 32746
I've noticed that for some platforms the release bulletins for 11.0.1 and 11.0.2 correct the statement in the Admin Guide about 2 gig being the limit; look under Known Problems or Documentation Updates. It seems the limit is now platform dependent but up to 32 gig. By the way, I haven't yet tested larger devices, just read that they're supposed to work.
In article <5jh82c$...@mew.corp.sgi.com>, Pablo Sanchez <pa...@sgi.com> wrote:
|In article <5jgjvf$...@manitou1.cse.dnd.ca>, pbu...@cse.dnd.ca writes: |> Recently we have begun using Sybase on a couple of hosts (both Sun and SGI) |> with large Clariion Raid devices. Both of these OSes limit the number of |> partitions that can be made on a logical device and Sybase itself limits |> the size of a database device. |> |> So, the question is.. |> |> How do people with large Raid/disk devices configure them for Sybase? | |How big is your device and what RAID level are you using? We have a |RAID device with 20 disks... but then we're using part of the disks |with RAID 0+1... kinda eats 'em up.
I have at my disposal up to four Clariion devices with 20 8.8 GB drives in each. We have not yet decided on the RAID levels to use, but it'll likely be a combination of RAID 5 and RAID 0+1. Any suggestions about what level(s) of RAID to use for Sybase would certainly be welcome.
As far as the disk usage goes, this is how I understand the limitations of Sybase, Irix and the Clariions:
Irix only supports ~13 user defined partitiions and Sybase only supports a 2GB partition. If I configure a brick as either RAID 3 or 5, the LUN will have ~32-35 GB of useable space. Irix can create 14 usable partitions (16 less the volume header and whole disk). So, Sybase will be able to use 14x2GB at most and the rest of the space will be unusable. Correct?
In article <5jis3u$...@manitou1.cse.dnd.ca>, pbu...@cse.dnd.ca writes: > In article <5jh82c$...@mew.corp.sgi.com>, Pablo Sanchez <pa...@sgi.com> wrote: > |How big is your device and what RAID level are you using? We have a > |RAID device with 20 disks... but then we're using part of the disks > |with RAID 0+1... kinda eats 'em up.
> I have at my disposal up to four Clariion devices with 20 8.8 GB > drives in each. We have not yet decided on the RAID levels to use, > but it'll likely be a combination of RAID 5 and RAID 0+1. Any suggestions > about what level(s) of RAID to use for Sybase would certainly be welcome.
If your goal is for performance (read/write) go with RAID 0+1.
Dedicating four disks (2 + 2) gives you about 200 IOPS (I/O per sec) without saturating the bus. I believe if you try (3 + 3) that does not scale and you get about 240 - 260 IOPS (this is because the bus becomes saturated). Use a stripe size (initially) of 64K... I'll get you a revised value from a benchmark someone did with Sybase on SGI and the Clariion...
Back to a LUN of four disks, that's 8.8 * 2 so that's 17.6g. With 14 partitions, you should be able to partition that sucker up into 2 gig chunks.
Hope this helps you... -- Pablo Sanchez | wk: 415.933.3812| pg: 800.930.5635 -or- pabl...@pager.sgi.com --------------+-----------------+------------------------------------------ -- pa...@sgi.com ... when mailing me, place "not spam" in the Subject
In article <5jl7rv$...@mew.corp.sgi.com>, pa...@sgi.com wrote: > Dedicating four disks (2 + 2) gives you about 200 IOPS (I/O per sec) > without saturating the bus. I believe if you try (3 + 3) that does > not scale and you get about 240 - 260 IOPS (this is because the bus > becomes saturated). Use a stripe size (initially) of 64K... I'll get > you a revised value from a benchmark someone did with Sybase on SGI > and the Clariion...
No way does 260 IOPs saturate a bus (unless its a DEC Unibus).
We can get over 5000+ IOPs per sec, so this saturation you are talking about must be either application or specific subsystem related.
|In article <5jl7rv$...@mew.corp.sgi.com>, pa...@sgi.com wrote: | |> Dedicating four disks (2 + 2) gives you about 200 IOPS (I/O per sec) |> without saturating the bus. I believe if you try (3 + 3) that does |> not scale and you get about 240 - 260 IOPS (this is because the bus |> becomes saturated). Use a stripe size (initially) of 64K... I'll get |> you a revised value from a benchmark someone did with Sybase on SGI |> and the Clariion... |> | |No way does 260 IOPs saturate a bus (unless its a DEC Unibus). | |We can get over 5000+ IOPs per sec, so this saturation you are talking |about must be either application or specific subsystem related. | |Glenn B
Burry) wrote: > In article <Glenn-2404970941100...@gbrack.demon.co.uk>, > Glenn <Gl...@gbrack.demon.co.uk> wrote: > |In article <5jl7rv$...@mew.corp.sgi.com>, pa...@sgi.com wrote: > | > |> Dedicating four disks (2 + 2) gives you about 200 IOPS (I/O per sec) > |> without saturating the bus. I believe if you try (3 + 3) that does > |> not scale and you get about 240 - 260 IOPS (this is because the bus > |> becomes saturated). Use a stripe size (initially) of 64K... I'll get > |> you a revised value from a benchmark someone did with Sybase on SGI > |> and the Clariion... > |> > | > |No way does 260 IOPs saturate a bus (unless its a DEC Unibus). > | > |We can get over 5000+ IOPs per sec, so this saturation you are talking > |about must be either application or specific subsystem related. > | > |Glenn B
> I believe he meant the SCSI bus.
> ..Paul > ..Paul
If he meant the SCSI bus then he is wrong (which is why I replied to the comment)
260 IOP's per sec equates to 520 KB/s with the typical 2K Sybase transfer.
This is just over 5% of a 10 MB/s SCSI bus, 2.5% of a 20 MB/s SCSI bus and 1.25% of ULTRA etc.
It doesn't take a rocket scientist to see that SCSI bus saturation is not going to be an issue.
By the way, a 64 K stripe size might have an effect on write performance and might also introduce stripe alignment issues. I have a paper on the topic if you are interested.
In article <Glenn-2504971120050...@gbrack.demon.co.uk>, Gl...@gbrack.demon.co.uk writes:
> If he meant the SCSI bus then he is wrong (which is why I replied to the > comment)
> 260 IOP's per sec equates to 520 KB/s with the typical 2K Sybase transfer.
> This is just over 5% of a 10 MB/s SCSI bus, 2.5% of a 20 MB/s SCSI bus and > 1.25% of ULTRA etc.
> It doesn't take a rocket scientist to see that SCSI bus saturation is not > going to be an issue.
Well that's the problem, I'm not even close to a rocket scientist... hey what do you want for free... :-)... anyway, on your 5000+ IOPS, was that scaling linearly? How many disks? Perspiring minds would like to know.
> By the way, a 64 K stripe size might have an effect on write performance > and might also introduce stripe alignment issues. I have a paper on the > topic if you are interested.
Does it apply to Clariion? -- Pablo Sanchez | wk: 415.933.3812| pg: 800.930.5635 -or- pabl...@pager.sgi.com --------------+-----------------+------------------------------------------ -- pa...@sgi.com ... when mailing me, place "not spam" in the Subject
Glenn should put a disclaimer in his emails that he works for RAID manufacturer Baydel (when he's promoting white papers, performance results, etc.)...
>> 260 IOP's per sec equates to 520 KB/s with the typical 2K Sybase transfer. >> It doesn't take a rocket scientist to see that SCSI bus saturation
is not going to be an issue.
>Well that's the problem, I'm not even close to a rocket scientist... >hey what do you want for free... :-)... anyway, on your 5000+ IOPS, >was that scaling linearly? How many disks? Perspiring minds would >like to know.
>> By the way, a 64 K stripe size might have an effect on write
performance and might also introduce stripe alignment issues. I have a paper on the topic if you are interested.
>Does it apply to Clariion?
The paper focuses on his Baydel box. I'm sure that Glenn will send you a copy.
Pablo (and others), check out RAID vendor MTI's (www.mti.com) web site (disclaimer: my former employer and current customer). I think they might have an online white paper on something similar to what Glenn is talking about.
Later, Robert David (working for RAID-helping solid-state disk SSD supplier, Imperial Technology)
Al, The limits are based on Operating System API limits for lseek and various Async I/O operations too. Right now, you can use 4GB devices for Digital Unix and I believe NT. NT had a problem a while back which allowed 4GB devices for the dataserver but did have some problems with 4GB through the Backup Server. So, I would only suggest that you go beyond 2GB is you have Digital Unix 3.x or 4.x only.
Sybase will support > 2GB devices when Operating Systems bring out their APIs with 64bit addressing. Such as Solaris 2.6, etc.
Sure, there are multi-terabyte installations out there - but they use *more than one* SQL Server to do it.
> Jung S. Lee (orbi...@worldnet.att.net) wrote: > : Mike Maas wrote: > : > > : > The 2 gig limit will go away soon in release 11.
> : I am looking at System Adm 11 manual, and it says the maximum size of > : database device is 2 GB.
> Probably it is the Adpative Server v11.5 that is being referred.
> But, in the meantime, if a server is limited to 255 devices and > disk init is limited to 2 GB, how do you get bigger than 510 GB? > I thought there were multi-terabyte Sybase Installs out there!
> Al Bledsoe > Independent Consultant | Compressed Graphics: .
-- John McVicker Principal Consultant, District Lead Architect Sybase Professional Services Pennsylvania/New Jersey District 301-896-1765 mcvic...@sybase.com, mcvic...@bellatlantic.net, JohnMcVic...@compuserve.com
: Al, : The limits are based on Operating System API limits for lseek and : various Async I/O operations too. Right now, you can use 4GB : devices for Digital Unix and I believe NT. NT had a problem a while : back which allowed 4GB devices for the dataserver but did have : some problems with 4GB through the Backup Server. So, I would only : suggest that you go beyond 2GB is you have Digital Unix 3.x or 4.x : only.
: Sybase will support > 2GB devices when Operating Systems bring : out their APIs with 64bit addressing. Such as Solaris 2.6, etc.
: Sure, there are multi-terabyte installations out there - but they : use *more than one* SQL Server to do it.
: Al Bledsoe wrote:
: > : > Jung S. Lee (orbi...@worldnet.att.net) wrote: : > : Mike Maas wrote: : > : > : > : > The 2 gig limit will go away soon in release 11. : > : > : I am looking at System Adm 11 manual, and it says the maximum size of : > : database device is 2 GB. : > : > Probably it is the Adpative Server v11.5 that is being referred. : > : > But, in the meantime, if a server is limited to 255 devices and : > disk init is limited to 2 GB, how do you get bigger than 510 GB? : > I thought there were multi-terabyte Sybase Installs out there! : > : > Al Bledsoe : > Independent Consultant | Compressed Graphics: .
: -- : John McVicker : Principal Consultant, District Lead Architect : Sybase Professional Services : Pennsylvania/New Jersey District : 301-896-1765 : mcvic...@sybase.com, mcvic...@bellatlantic.net, : JohnMcVic...@compuserve.com
In article <abb2uE9pt0v....@netcom.com> Al Bledsoe, ab...@netcom.com writes:
>I understand the 2 gb problem, but I don't understand the 255 devices >per server constaint.
Sybase uses a "virtual page" numbering schema which includes the device number in the high-order byte of the virtual page number.
One byte equals 256 devices (not 255). With this schema, and 64 bit OS's (or more accurately, a 64 bit POSIX api), you would still only be able to address an 8Gb device.
----------------------------------------- Kevin Sherlock USWEST Dex Omaha, NE anti-spam: ksherlo-at-uswest-dot-com -----------------------------------------
Kevin, With 64-bit file management (Posix) and SQL Server's ability to have virtual pages on each device, let's look at how it works.
The sysdevices table in master stores a "low" and "high" value. This is a two-valued column. First 8-bits (1 byte) is the vdevno. The next 24-bits are the page number on that device, starting at 0. So, the limit is 16777215 pages per device. This, multiple by 2048, the SQL Server page size is 34359736320 bytes. Divided out, it comes to just under 32.767GB or 32767.998 MB (so real usable space would be 32767 MB.
I don't know the Posix spec, but if this is truly usable space, one device could possibly address 32+ GB of pages on one device. Does the Posix spec pertain to the 8GB limit you mentioned? I am happy to just get 4GB per device - that takes SQL Server up to a limit of 1 Terabyte (with 4GB * 256 devices).
> In article <abb2uE9pt0v....@netcom.com> Al Bledsoe, ab...@netcom.com > writes: > >I understand the 2 gb problem, but I don't understand the 255 devices > >per server constaint.
> Sybase uses a "virtual page" numbering schema which includes the > device number in the high-order byte of the virtual page number.
> One byte equals 256 devices (not 255). With this schema, and 64 bit > OS's (or more accurately, a 64 bit POSIX api), you would still only be > able to address an 8Gb device.
> ----------------------------------------- > Kevin Sherlock > USWEST Dex > Omaha, NE > anti-spam: ksherlo-at-uswest-dot-com > -----------------------------------------
-- John McVicker Principal Consultant, District Lead Architect Sybase Professional Services Pennsylvania/New Jersey District 301-896-1765 mcvic...@sybase.com, mcvic...@bellatlantic.net, JohnMcVic...@compuserve.com
