ASM and RAID
Michael Austin
twist to the debate. What is your experience either in the lab or real
sites where you have the virtualization at the array level (pick your
RAID level) in addition to the virtualization provided by ASM in an
Oracle eBusiness environment.
Let's say you have Tier1 storage (high end whatever...) you have 50
spindles in a RAID0+1 providing ~12.5TB of raw storage that we carve out
100 50GB LUNS and present them to our ASM environment. How, in your
testing and research, does performance either improve or suffer with the
addition of ASM. The database is scattered across 6 separate disk
groups with logfiles etc in their own disk group.
joel garry
I have no experience with that. However, I have weekly (formerly
daily) experience with virtualized XP locking up and making a pain in
the ass for me to have to fix the data of stupid apps that half-tried
to implement multiple levels of transaction, a coming feature never
added to SQL that winds up having the effect of serial transactions
when there should be a single transaction.
So this has made me sceptical of _any_ type of virtualization.
I'm also sceptical about most places ability to test properly.
http://mikerault.blogspot.com/ (for the record, I like Mike and his
work going way back, though I've disagreed with some methods, and
think he's been unfairly tarred in the past. Whether he is currently
biased remains to be seen IMNSHO).
jg
--
@home.com is bogus. At the end of the day, the situation is fairly
unique. With all due respect, at this moment in time, I personally do
not want to jump on the bandwagon. Don't get me wrong, I absolutely
want to take a moment to think outside the box and pick all the low
hanging fruit. We are neck deep in this nightmare and need to be on
top of this 24/7.
AlexB
wrote:
While not being a very large database shop, all I can say about ASM on top of
RAID is that it uses extra CPU, but makes DBA life a bit easier (no need to
specify datafile names and easier to create new instances, for example). We are
using IBM DS4000 series SAN with RAID10 LUNs presented (through VIO server) to
AIX/Linux LPARs on a pSeries 570. AIX LPARs are using direct access to raw
devices by Oracle 9iR2 processes, while Oracle 10gR2 on Linux uses ASM. If we
look at 'sar' output - we can hardly notice the overhead (our CPUs are 1.65GHz
POWER5), while looking at nmon/topas screens during data loads or queries we can
see ASM processes taking some 7-10% of CPU at peak times. Did not encounter any
reliability matters so far.
We are about to install a new p570-MMA and I will be definitely going with ASM
on AIX LPARs as well.
On the RAID5 matter - I do not see anything terminally wrong with RAID5 even
after reading the whole thread, but we only use RAID5 LUNs for disk backups.
Alex
Robert Klemme
> While not being a very large database shop, all I can say about ASM on top of
> RAID is that it uses extra CPU, but makes DBA life a bit easier (no need to
> specify datafile names and easier to create new instances, for example). We are
> using IBM DS4000 series SAN with RAID10 LUNs presented (through VIO server) to
> AIX/Linux LPARs on a pSeries 570. AIX LPARs are using direct access to raw
> devices by Oracle 9iR2 processes, while Oracle 10gR2 on Linux uses ASM. If we
> look at 'sar' output - we can hardly notice the overhead (our CPUs are 1.65GHz
> POWER5), while looking at nmon/topas screens during data loads or queries we can
> see ASM processes taking some 7-10% of CPU at peak times. Did not encounter any
> reliability matters so far.
This is in line with an article I recently stumbled across:
http://www.linuxjournal.com/article/8539#N0xa50890.0xb3d4b8
Cheers
robert
robert
> How, in your
> testing and research, does performance either improve or suffer with the
> addition of ASM. The database is scattered across 6 separate disk
> groups with logfiles etc in their own disk group.
I've no numbers to contribute (though I'm also very interested in the
collective experience of this group on this).
But I do have a question: how much would we reasonably *expect*
performance to be impacted one way or the other by ASM, when we are on a
high performance SAN?
DA Morgan
On a high end SAN you are reading and writing to a cache not to disk so
the number of spindles becomes less important. This is part of what
makes NetApp's RAID4 different from RAID4 from other vendors.
ASM seems to prove either a neutral or slightly positive impact (a
few percent) on performance if compared to raw disk with no LVM. If
comparing with other LVMs what we see is a substantial drop in CPU
utilization. Most LVMs are cpu pigs: ASM is not.
--
Daniel A. Morgan
Oracle Ace Director & Instructor
University of Washington
damo...@x.washington.edu (replace x with u to respond)
Puget Sound Oracle Users Group
www.psoug.org
Michael Austin
> robert wrote:
>> Michael Austin wrote:
>>> How, in your
>>> testing and research, does performance either improve or suffer with
>>> the addition of ASM. The database is scattered across 6 separate
>>> disk groups with logfiles etc in their own disk group.
>>
>> I've no numbers to contribute (though I'm also very interested in the
>> collective experience of this group on this).
>> But I do have a question: how much would we reasonably *expect*
>> performance to be impacted one way or the other by ASM, when we are on
>> a high performance SAN?
>
> On a high end SAN you are reading and writing to a cache not to disk so
> the number of spindles becomes less important. This is part of what
> makes NetApp's RAID4 different from RAID4 from other vendors.
Not sure, but doesn't this statement make my point from earlier RAID5
and SAN discussions :)
>
> ASM seems to prove either a neutral or slightly positive impact (a
> few percent) on performance if compared to raw disk with no LVM. If
> comparing with other LVMs what we see is a substantial drop in CPU
> utilization. Most LVMs are cpu pigs: ASM is not.
ASM also does not do the actual reading/writing - the RDBMS does. In
other words - ASM does not proxy the I/O for the RDBMS - RDBMS writes
directly to the data files. ASM just tells the RDBMS what the extent
map is and only at file open time... which is why you need additional
shared_pool (1MB for every 100GB of file space).
There is a book called Automatic Storage Management - practical
under-the-hood ??? that is very good at the mechanisms within ASM and
RDBMS...
robert
> [...] ASM just tells the RDBMS what the extent
> map is and only at file open time... which is why you need additional
> shared_pool (1MB for every 100GB of file space).
>
> There is a book called Automatic Storage Management - practical
> under-the-hood ??? that is very good at the mechanisms within ASM and
> RDBMS...
Thank you for the recommendation. I'll get that one.
Especially if it goes into detail about additional requirements like
shared pool.
Robert Klemme
> ASM also does not do the actual reading/writing - the RDBMS does. In
> other words - ASM does not proxy the I/O for the RDBMS - RDBMS writes
> directly to the data files. ASM just tells the RDBMS what the extent
> map is and only at file open time... which is why you need additional
> shared_pool (1MB for every 100GB of file space).
That's an interesting bit of information. How is the ASM able to
replace a clustered file system? Does it provide only the meta data
layer which controls concurrent access to files?
> There is a book called Automatic Storage Management - practical
> under-the-hood ??? that is very good at the mechanisms within ASM and
> RDBMS...
I am assuming you mean this one: http://www.amazon.com/dp/0071496076
Kind regards
robert
Michael Austin
> On 18.12.2008 17:32, Michael Austin wrote:
>> ASM also does not do the actual reading/writing - the RDBMS does. In
>> other words - ASM does not proxy the I/O for the RDBMS - RDBMS writes
>> directly to the data files. ASM just tells the RDBMS what the extent
>> map is and only at file open time... which is why you need additional
>> shared_pool (1MB for every 100GB of file space).
>
> That's an interesting bit of information. How is the ASM able to
> replace a clustered file system? Does it provide only the meta data
> layer which controls concurrent access to files?
Sorry taking so long to get back to this thread...
In order to have a "clustered file system" one of the prerequisites is
to have some method of I/O fencing - a mechanism to prevent both servers
from writing the same data block at the same time. This is what Cache
Fusion which includes the Distributed Lock Manager (DLM) is for - to
allow concurrent writes to raw devices from multiple servers in a RAC
cluster.
If you ever worked in an OpenVMS environment - this is how they made it
work... The DLM is the most important piece of this whole thing. Without
it, you would have chaos.
>
>> There is a book called Automatic Storage Management - practical
>> under-the-hood ??? that is very good at the mechanisms within ASM and
>> RDBMS...
>
> I am assuming you mean this one: http://www.amazon.com/dp/0071496076
Yes.
>
> Kind regards
>
> robert
Robert Klemme
> Robert Klemme wrote:
>> On 18.12.2008 17:32, Michael Austin wrote:
>>> ASM also does not do the actual reading/writing - the RDBMS does. In
>>> other words - ASM does not proxy the I/O for the RDBMS - RDBMS writes
>>> directly to the data files. ASM just tells the RDBMS what the extent
>>> map is and only at file open time... which is why you need additional
>>> shared_pool (1MB for every 100GB of file space).
>>
>> That's an interesting bit of information. How is the ASM able to
>> replace a clustered file system? Does it provide only the meta data
>> layer which controls concurrent access to files?
>
> Sorry taking so long to get back to this thread...
Thank you for the update nevertheless!
> In order to have a "clustered file system" one of the prerequisites is
> to have some method of I/O fencing - a mechanism to prevent both servers
> from writing the same data block at the same time. This is what Cache
> Fusion which includes the Distributed Lock Manager (DLM) is for - to
> allow concurrent writes to raw devices from multiple servers in a RAC
> cluster.
Ok, that's what I meant. ASM manages the meta data (i.e. which process
is writing which block) and the instances query this before writing or
reading.
> If you ever worked in an OpenVMS environment - this is how they made it
> work... The DLM is the most important piece of this whole thing. Without
> it, you would have chaos.
Which can be nice at times - but not in an Oracle tablespace. :-)
>>> There is a book called Automatic Storage Management - practical
>>> under-the-hood ??? that is very good at the mechanisms within ASM and
>>> RDBMS...
>>
>> I am assuming you mean this one: http://www.amazon.com/dp/0071496076
>
> Yes.
Ok.
Cheers
robert
--
remember.guy do |as, often| as.you_can - without end