For the new hardware, we're going to 64bit os, and 64 bit Oracle.
We're expecting 50 or so concurrent connections to the database (all
coming from an application pool). Given the nature of the application
though, its very unusual for the acutal concurrency to get much over
15 or so active sessions.
The question I have is, is there any downside to me buying, say, a 32G
box and setting the SGA size at 20G? Will I actually end up harming my
performance with an over-large SGA (assuming I have enough physical
memory to keep the box out of swap)?
> Working set is
> hard to estimate, but in analagous 100G systems I see a buffer hit
> ratio or around 70% with a 2.3 G SGA (about as big as I can get it on
> a 32 bit OS). Based on this and a fair amount of spitball analysis I'd
> guess the working set on the new box to be around 8-10G.
Choose any hit ratio
http://www.oracledba.co.uk/tips/choose.htm
Using BCHR for anything other than pure wishful dreaming,
is like using Tarot cards to select your spouse.
There is NO, NO, NO relationship between raw database size (200GB)
and database performance as measured by ANY metric!
Good Luck on your search for the Holy Grail!
What I want to do is, as much as possible, serve data out of cache. On
analogous 32 bit boxes with a 2.3G SGA, I'm seeing an awful lot of
physical IOs, enough that many of my queries are spending > 50% of
their time in IO wait.
The classic solution to this is:
add more memory
What i want to know is if there's a potential downside to throwing
memory at the problem. I have the hardware budget to buy an aweful lot
of memory, but I don't want to spend it there if there if it'll be
counterproductive.
When your only tool is a hammer, all problems are viewed as nails.
>
> The classic solution to this is:
> add more memory
What you are attempting to do is covert Physical I/O to Logical I/O.
A smarter solution is to add an index to reduce I/O by orders of magnitude.
The problem here isn't excessive table scans or an absence of indexes.
The working set of indexes simply don't fit in cache all that well.
I've got mutltiple indexes > 1 G in size and a half dozen or so >
500M.
So, while I appreciate the tutorial on the importance of indexes as a
component to an efficient data retreival strategy, I find it a bit odd
that you're acting as though cache memory isn't an analagous
component.
This is the database back end for an enterprise application, it's not
a data warehouse application. It tends to aggressively chew over the
same working set (the aforementioned 10-12G of memory) querying it in
all sorts of unpredictable, end-user defined, ways. If I knew a set of
additional indexes I could add that would reduce my working set, I'd
have already added them. At this point, the only solution I can see
here is to bump up the SGA so that my (existing) index and data blocks
fit in memory.
You have all the answers.
BCHR is a measure of performance.
DB size relates to perfromance.
More memory results in better response time.
Have A Nice Day!
> physical IOs, enough that many of my queries are spending > 50% of their
> time in IO wait.
>
> The classic solution to this is:
>
> add more memory
The proper solution would be to tune the #$%&! queries.
--
Mladen Gogala
http://mgogala.freehostia.com
But if you have too much $ , buy more. I would be clad to show how utilize
it!
(By order)
"Pat" <pat....@service-now.com> wrote in message
news:c3b1b601-abd9-4d2c...@d77g2000hsb.googlegroups.com...
Possibly. It's also possible that this is a highly transactional
configuration, effecting large volumes of data changes. An index
won't help there.
> The problem here isn't excessive table scans or an absence of indexes.
> The working set of indexes simply don't fit in cache all that well.
> I've got mutltiple indexes > 1 G in size and a half dozen or so >
> 500M.
>
> So, while I appreciate the tutorial on the importance of indexes as a
> component to an efficient data retreival strategy, I find it a bit odd
> that you're acting as though cache memory isn't an analagous
> component.
>
It isn't, really, since if you're running a system which modifies
large volumes of data the cached data blocks may be invalidated by the
insert/update/delete activity, requiring them to be refreshed to
ensure reliable and accurate result sets are returned. Increasing the
cache size won't help when blocks are marked as modified and thus
refreshed due to transactional activity.
> This is the database back end for an enterprise application, it's not
> a data warehouse application. It tends to aggressively chew over the
> same working set (the aforementioned 10-12G of memory) querying it in
> all sorts of unpredictable, end-user defined, ways.
No, it 'chews over' the application data which, in turn, 'churns' the
existing cache because the data blocks have been modified since the
last query used them. No amount of memory will stop that behaviour.
> If I knew a set of
> additional indexes I could add that would reduce my working set, I'd
> have already added them.
I don't believe it's the size of your 'working set' that is the
'problem'; the issue is querying constantly changing data which is
brought into the cache because the blocks, essentially, undergo
continuous change during the business day.
> At this point, the only solution I can see
> here is to bump up the SGA so that my (existing) index and data blocks
> fit in memory.
I can't believe that will do much good except to give the vendor of
your memory a better bottom line. You should install the PLUSTRACE
role then use autotrace on some of these queries during the day to
report some useful query statistics, such as redo generated. Yes, a
query can generate redo, and undo, due to delayed block cleanout, and
that phenomenon will increase your physical reads, and, yes,
increasing the SGA, and the resulting buffer cache, can help some but
Oracle restricts block cleanout for a transaction of any size to 10%
of the total buffer cache blocks; any transaction modifying a block
count in excess of the 10% threshold will relegate the cleanout of the
remaining blocks affected to the next operation which touches those
blocks, even if that operation is a (relatively) simple select
statement. To eliminate this entirely you would need to allocate a
number of blocks in the buffer cache equal to 10 times the size of the
largest transaction you could possibly execute in your database, and I
doubt you have the budget, or a machine, that can provide that much
memory. Throwing memory at this 'problem' is, in my opinion, not the
solution.
You need to find the source of this buffer cache block churning, and I
expect it's due to high transactional activity. Yes, you can install
the maximum amount of RAM your machine can support, and you can
allocate 80% of that to your database, that will only do so much to
keep data in memory. Once that data changes the cached values are no
longer valid and require a refresh, which involves physical I/O.
Unless you stop all transactional activity you can't guarantee that
the data you loaded into cache at 9:15 this morning will still be
there at 9:37 that same morning.
In this case bigger isn't always better.
David Fitzjarrell
Arne Ortlinghaus
ACS Data Systems
"Pat" <pat....@service-now.com> schrieb im Newsbeitrag
news:e71181dd-9753-4709...@a70g2000hsh.googlegroups.com...
> Unfortunately the Windows 64bit Operating System does seem not to make
> always the best usage of the additional memory: We see many page faults
> in the processes. But nevertheless I would say: after having a
> multiprocessor CPU the most important part is the quantity of main
> memory.
This is the standard fallacy that has resulted in many heavily messed
up data centers. It's the I/O capacity and architecture that counts the
most when it comes to databases. Machine with weaker CPU, smaller memory
and good I/O capacity will usually beat overclocked monsters with 32GB
or more of main memory hands down. IBM p5 560 will run circles around
Dell machines with more raw CPU power and memory then the modest little
560. And it isn't that expensive, it starts around 50k. Stability and
efficiency of AIX 5.3 is something that both Linux and Windows can only
dream about.
I've seen 560 seamlessly performing 30,000 I/O requests per second and
I've seen Dell380's choking up with 10,000 I/O requests per second.
Databases are all about I/O. You need a machine to find generate reports
from your database, not a machine to answer the question of life, universe
and everything. That has been answered already.
Since everyone is busy telling you how to tune instead of answering
your question, you might have to infer that the answer is "No, there
is no downside to adding memory." That's my takeaway from no
negatives pointed out anyway...
As said in another post the most important part is NOT memory or
number of CPUs available, it's I/O architecture. Granted, Windows is
not the ideal 'operating system' (and I do use that term loosely) when
it comes to memory 'management', but that isn't your main problem; the
issue is the bottleneck created by the limited capacity of your I/O
subsystem and it's associated architecture. What I said earlier still
applies; bulking up the cache in hopes you'll get more hits when your
data is constantly changing is akin to 'tilting at windmills'. You're
not addressing the underlying problem, you're attempting to mask the
symptoms, which rarely, if ever, works.
Of course, when it's all said and done it's your (meaning the
company's) money and time and effort; spend it or waste it as you see
fit. And, personally, I think goosing your RAM allotment to expand
your buffer cache is wasting money which will provide no real return.
David Fitzjarrell
Then you're not reading the entire thread, as I posted that installing
all of the physical memory a server can accept, then allocating 80% of
that to the database would be wasteful, to say the least. Knowing
that this is a Windows operating sytem, which requires 2 gig for the
operating system alone, may make that 80% allocation 'impossible' thus
creating a scenario of constant paging/swapping to/from disk. Of
course even a successful allocation of that much memory to the SGA
would create a paging/swapping situation as PGA components may require
more free memory than is available. Which, in turn, sends performance
into the proverbial dumpster.
Even if he's lucky and no paging occurs it's highly likely his memory
allocations will be unused as constantly changing data causes the
cache to be refreshed from disk, thus killing the 'benefit' of having
all of those lovely data blocks in cache. And bloating the SGA to
starve the O/S is ... not the wisest of moves.
The negatives of this situation are known by most of those who have
posted to this thread. Siimply because you can't see them in print is
no indication they don't exist.
David Fitzjarrell
To me "wasteful" does not translate to "harming performance"
> Knowing that this is a Windows operating sytem, which requires 2 gig for the
> operating system alone, may make that 80% allocation 'impossible' thus
> creating a scenario of constant paging/swapping to/from disk. Of
> course even a successful allocation of that much memory to the SGA
> would create a paging/swapping situation as PGA components may require
> more free memory than is available. Which, in turn, sends performance
> into the proverbial dumpster.
>
> Even if he's lucky and no paging occurs it's highly likely his memory
> allocations will be unused as constantly changing data causes the
> cache to be refreshed from disk, thus killing the 'benefit' of having
> all of those lovely data blocks in cache. And bloating the SGA to
> starve the O/S is ... not the wisest of moves.
Pat suggested "a 32G box and setting the SGA size at 20G." Are you
saying that 12G for the OS would be starving it?
> The negatives of this situation are known by most of those who have
> posted to this thread. Siimply because you can't see them in print is
> no indication they don't exist.
>
> David Fitzjarrell
I understand that 99% of the posters here are more knowledgable than
me. I am not trying to be argumentative, I simply haven't seen any
negatives listed - perhaps I need to learn to read between the lines
better? All I see is people saying it is wasteful and that there are
better ways to spend time and money, but I haven't seen anyone say
"The reason you DON'T want a 20G SGA allocated out of 32G is because
this will happen and this will happen, you would be better off with a
10G SGA allocated out of 32G."
And no one can honestly say that without actually seeing the system in
question.
David Fitzjarrell
In some cases you're correct, in others you may be wrong. Since you
can't witness the affected system 'in action' you can't comment either
way.
> > Knowing that this is a Windows operating sytem, which requires 2 gig for the
> > operating system alone, may make that 80% allocation 'impossible' thus
> > creating a scenario of constant paging/swapping to/from disk. Of
> > course even a successful allocation of that much memory to the SGA
> > would create a paging/swapping situation as PGA components may require
> > more free memory than is available. Which, in turn, sends performance
> > into the proverbial dumpster.
>
> > Even if he's lucky and no paging occurs it's highly likely his memory
> > allocations will be unused as constantly changing data causes the
> > cache to be refreshed from disk, thus killing the 'benefit' of having
> > all of those lovely data blocks in cache. And bloating the SGA to
> > starve the O/S is ... not the wisest of moves.
>
> Pat suggested "a 32G box and setting the SGA size at 20G." Are you
> saying that 12G for the OS would be starving it?
>
Certainly not, and if you read it that way you need to look at the
context again. That comment was made with reference to the previous
paragraph where, on a smaller 'box', the maximum RAM installable may
be 8 gig, and 80% of that would leave less than 2 gig for the O/S,
which COULD starve it for resources.
> > The negatives of this situation are known by most of those who have
> > posted to this thread. Siimply because you can't see them in print is
> > no indication they don't exist.
>
> > David Fitzjarrell
>
> I understand that 99% of the posters here are more knowledgable than
> me. I am not trying to be argumentative, I simply haven't seen any
> negatives listed - perhaps I need to learn to read between the lines
> better? All I see is people saying it is wasteful and that there are
> better ways to spend time and money, but I haven't seen anyone say
> "The reason you DON'T want a 20G SGA allocated out of 32G is because
> this will happen and this will happen, you would be better off with a
> 10G SGA allocated out of 32G."
As I said before since you, nor anyone else responding to this thread
outside of the O/P, can actually witness this configuration in action
it would be foolish to make such a claim.
David Fitzjarrell
How did you reach the conclusion the OP is running Windows? I saw nothing in
their posts which indicated which OS they were using.
Mike
> The question I have is, is there any downside to me buying, say, a 32G
> box and setting the SGA size at 20G? Will I actually end up harming my
> performance with an over-large SGA (assuming I have enough physical
> memory to keep the box out of swap)?
I am in agreement with bhonaker. I do not see any issues with doing this.
However, as others have indicated, this may not solve your problem. But I
don't see it causing harm. Is it possible you could comment on the
configuration of your current system as well as the configuration of the
replacement system?
Mike
Current system is:
4 X Intel(R) Xeon(TM) CPU 2.80GHz (8 cores)
8 G RAM
1.1 TB of storage (SCSI RAID 10 array, direct attached)
Red Hat AS4, 32 bit
Oracle 10.2.0.3 (32 bit)
2.3 G SGA
Replacement system is:
2 X Intel(R) Xeon(R) CPU E5310 @ 1.60GHz (8 cores)
16 G RAM
1.6 TB of storage (SCSI RAID !0 array, direct attached)
Red Hat
Oracle 10.2.0.3 (64 bit)
12 G SGA
I've got the hardware budget left to bump up the memory on the new
system a bit if that'll help and I'm tempted to do it (hence this
thread).
One totally off topic question I do have though is, does anybody have
experience with the new 4 core line of intel chips? My operations guys
are procuring these now instead of the old dual core Xeons, but they
make me nervous since the new ones report lower bogomips than the old
ones. I'm aware that the "bogo" in bogomips stands for bogus, so I'm
totally wiling to be convinced the new chips are, in fact, faster, but
I'd love to hear any real world experience here.
I pretty much agree with the others in this thread, though I have a
quibble with a statement in David's first post:
"> > A smarter solution is to add an index to reduce I/O by orders of
magnitude.
Possibly. It's also possible that this is a highly transactional
configuration, effecting large volumes of data changes. An index
won't help there"
What I've seen in most transactional configurations is that indices
help a lot. What often winds up happening is certain objects thrash
the SGA, so Oracle spends a lot of time doing housekeeping on the
buffers there, as well as all the other things David mentioned. What
I've seen on my systems is that there are usually just a few objects
responsible for most of the thrash (which makes sense, since most
activity is the mission critical app), and separating them into their
own cache (particularly a recycle cache) helps enormously. See the
performance tuning manual for the basics about looking at V$BH, and
google about for other commentary about it. But definitely don't
listen to anybody who advocates using tablespaces with different block
sizes for performance.
If you look at what is happening on your current configuration, I'd
expect you'll see there's not all that much size involved in the
candidates for recycling. Then when you look at the total PGA usage
you'll see how much your users really need. Check out your advisor
views.
The more modern Oracle handle large SGA better than in the olden days,
so using an oversize SGA isn't really the big deal it was in the past,
as long as you aren't way stupid in something.
>
> I've got the hardware budget left to bump up the memory on the new
> system a bit if that'll help and I'm tempted to do it (hence this
> thread).
My experience indicates it is best to grab what you can when you can,
YMMV.
>
> One totally off topic question I do have though is, does anybody have
> experience with the new 4 core line of intel chips? My operations guys
> are procuring these now instead of the old dual core Xeons, but they
> make me nervous since the new ones report lower bogomips than the old
> ones. I'm aware that the "bogo" in bogomips stands for bogus, so I'm
> totally wiling to be convinced the new chips are, in fact, faster, but
> I'd love to hear any real world experience here.
Tuning becomes complicated because it involves twiddling many
different bottlenecks, so it is entirely possible to make things worse
in an unexpected way. For example, making the SGA larger could take
the bottleneck away from I/O and put it onto cpu, suddenly making
bogomips important, and making everything slower in both
configurations, even more so in the "faster" one. That's when it is
important to already understand methodologies like Cary Milsap
publishes. I'm not disagreeing with Mladen, in fact I agree with what
he posted, but I'm pointing out that lots of factors are involved that
can make a specific situation "interesting."
jg
--
@home.com is bogus.
"...the Internet is the best way to disseminate accurate
information..." - Mark Cogan, attorney for alleged baby killers.
http://www.worthingtondefense.info
This seems as an uninformed attempt to solve a problem. If it doesn't
harm - it might work.
Resist your temptation to this "checklist approach" if you seek
insight in the problem. Adopt a methodology as Cary Milsap provides.
Analyse before implementing a solution.
But the temptation to buy more hardware is strong. I remember a
situation where a new and critical application was performing very
badly. The CEO personally ordered a new 8 CPU machine with more RAM to
push performance in order to buy time for the programmers to tune the
application.
The old machine was a 1 CPU machine. There was an increase of
performance between 0 to 2 % (if i remember well) , instead of the
800% the CEO expected.
So if you want to try the RAM scenario, do as you please. But please
measure your performance in response times, and not some hitratio. And
then have a look at the difference in performance.
H.
> The old machine was a 1 CPU machine. There was an increase of
> performance between 0 to 2 % (if i remember well) , instead of the
> 800% the CEO expected.
>
> So if you want to try the RAM scenario, do as you please. But please
> measure your performance in response times, and not some hitratio. And
> then have a look at the difference in performance.
>
> H.
:D
And there's a bug in 9.2 (patch 6, iirc, and it was HP specific)
that will send your response time skyhigh, because the optimizer
keeps looping the huge buffer - it simply does not understand
searching memory for 6 hours is *not* faster than giving up
after 1 second, and fetching from disk - although it may
be "cheaper".
FvB