Confusion of result from "topas" and "vmstat"
Toppan
===========
Tue Oct 22 12:29:14 2002 Interval: 2 Cswitch 12915 Readch 709186248
Syscall 2528 Writech 3156
Kernel 2.1 |# | Reads 8806 Rawin 0
User 0.0 | | Writes 35 Ttyout 329
Wait 0.0 | | Forks 0 Igets 0
Idle 97.8 |########################### | Execs 0 Namei 79
Runqueue 9.0 Dirblk 2
Interf KBPS I-Pack O-Pack KB-In KB-Out Waitqueue 2.0
lo0 8.0 47.4 47.4 4.0 4.0
en0 0.6 3.9 0.4 0.3 0.3 PAGING MEMORY
Faults 1 Real,MB 4095
Disk Busy% KBPS TPS KB-Read KB-Writ Steals 0 % Comp 38.6
hdisk0 0.9 5.9 1.4 0.0 5.9 PgspIn 0 % Noncomp 44.5
hdisk2 0.4 1.9 0.4 0.0 1.9 PgspOut 0 % Client 0.5
hdisk1 0.0 0.0 0.0 0.0 0.0 PageIn 0
PageOut 1 PAGING SPACE
aioserver(21414) 7.9% PgSp: 0.0mb oracle Sios 1 Size,MB 1568
aioserver(20902) 7.9% PgSp: 0.0mb oracle % Used 68.7
oracle (40320) 6.8% PgSp: 1.6mb oracle % Free 31.2
aioserver(4208) 6.8% PgSp: 0.0mb root
oracle (45064) 6.8% PgSp: 1.6mb oracle
aioserver(21672) 6.8% PgSp: 0.0mb oracle Press "h" for help screen.
aioserver(20642) 6.8% PgSp: 0.0mb oracle Press "q" to quit program.
oracle (76324) 6.0% PgSp: 1.6mb oracle
aioserver(21160) 5.1% PgSp: 0.0mb oracle
sqloImpor(18430) 4.3% PgSp:18.2mb eric
aioserver(21930) 2.7% PgSp: 0.0mb oracle
oracle (113450 1.7% PgSp: 1.6mb oracle
gil (1290) 0.9% PgSp: 0.0mb root
topas (87412) 0.9% PgSp: 0.5mb root
aioserver(22188) 0.7% PgSp: 0.0mb oracle
oracle (96840) 0.0% PgSp: 1.6mb oracle
=======
----- ----------- ------------------------ ------------ -----------
r b avm fre re pi po fr sr cy in sy cs us sy id wa
4 2 392642 187068 0 0 0 3 12 0 451 607 269 32 25 42 1
0 2 392642 187065 0 0 0 0 0 0 192 4219 10739 7 38 55 0
0 2 392642 187065 0 0 0 0 0 0 204 2416 10812 14 35 51 0
0 2 392642 187067 0 0 0 0 0 0 196 941 213 1 2 96 1
0 2 392642 187067 0 0 0 0 0 0 202 1581 201 7 1 92 0
Steve Johnson
Topas doesn't say that you are using 100% of memory. It says that you
have
4GB of real memory, of the memory that is being used 38.6% of that is
computational memory and 44.5% is noncomputational. Also vmstat
reports
the number of pages on the free list, this does not necessarily mean
that
those pages are not occupied in memory. These pages may have been used
for
noncomp file pages and never modified and are no longer needed. Aix
doesn't
flush pages unless the have been modified.
Toppan
(Client)0.5? Then how can I know that how many memory free in AIX?
Besides, I quite confusion about "Noncomp", I discover that it never drop.
How can I release the memory in Noncomp? I think no process will use so many
memory. I guess some programs dominate the memory and cannot release even I
stop it.
Thx a lot~
"Steve Johnson" <steve....@bull.com> ?????
news:b629965a.02102...@posting.google.com...
John R. Campbell
> You mean that the used memory is not equal to (Comp)38.6 + (Noncomp)44.5 +
> (Client)0.5? Then how can I know that how many memory free in AIX?
38.6+44.5 is 83.1% of the 4GB of memory in the beastie. That
leaves a fair chunk.
Client is usually chunks of an executable that came in via NFS;
I *think* there are other reasons for something to be seen as
client but I'm a wee bit senile just now.
> Besides, I quite confusion about "Noncomp", I discover that it never drop.
> How can I release the memory in Noncomp? I think no process will use so many
> memory. I guess some programs dominate the memory and cannot release even I
> stop it.
Non-Computational pages includes what is known as "persistent
storage".
AIX is weird. I mean it. All disk I/O is managed through the
VMM (Virtual Memory Manager) since files are "mapped" as if
they're paging spaces, so that a read tries to access a page in
a file's memory segment and a page fault bring the page in. A
write is even more entertaining.
Computational pages are usually program code segments and the
associated data segments that are usually manipulated quite
directly, hence the term "computational". Mind you, the code
segments are consided "persistent storage" as well, but they
aren't considered part of the disk buffer cache- which is
what the non-computational pages act as.
Like I said, AIX is weird- but it gets the job done. I'm quite
comfortable with it, too. (I've got certifications for both
System Administration and System Support for AIX v4.3, so I'm
a certifiable masochist.)
Mind you, there are lots of things I like more about Linux,
but each system has it's own strengths- and work well together.
--
John R. Campbell Speaker to Machines so...@jtan.com
- As a SysAdmin, yes, I CAN read your e-mail, but I DON'T get that bored!
"It is impossible for ANY man to learn about impotence the hard way." - me
"ZIF is not a desirable trait when selecting a spouse." - me
Toppan
Why do I want to release some memory from Noncomp because after real memory
is exhausted, then something continuously will consume our paging space
until out of memory.
In case of “Out of Memory”, command cannot be typed, so we need to
physically restart our AIX machine. It does injure our Oracle Database which
maybe be crash without normally shutdown.
Therefore, I would like to know how can I release the un-used memory and how
to see the allocation of memory in each processes.
"John R. Campbell" <so...@penrij.uucp.jtan.com> 撰寫於郵件
news:slrnarbuh...@soup2nets.net.dhis.org...
mark taylor
svmon -P PID
Mark
John R. Campbell
> Why do I want to release some memory from Noncomp because after real memory
> is exhausted, then something continuously will consume our paging space
> until out of memory.
Non-computational (disk file buffering) usage will be the first
to shrink when faced with growth in computational pages- since
they're mapped to a disk file, there's no problem "paging them
out" to their backing store (the disk file itself).
Do you have an application suffering from a memory leak? That'd
crowd things real fast by consuming the paging space(s) with "dead"
(used once) pages.
You may want to have a cron task doing an lsps -a every 5 minutes
(or so) and when paging space has been consumed to a particular
point force a shutdown.
I've got a heartbeat script I run every 5 minutes that collects
a wide variety of information which gets delivered to another
system via NFS; If a heardbeat files isn't updated w/i 12 minutes
I consider the machine to have crashed and I get paged. The
heartbeat file contains a wide variety of useful material-
who, w, fps (a "fast" ps that doesn't look up uid#s), netstat and
even errpt. This allows me to have a snapshot taken from a point
in time 5 minutes (or less) before a system outage- which acts
like a flight data recorder. You may want to construct your own
so that you can see whether a process has lost it's mind- and is
taking over all of RAM.
BTW, "ps alx" does report on memory sizes if you can't get fps
from the Bull freeware site.
I hope this helps.
Toppan
Pid Command Inuse Pin Pgsp Virtual 64-bit Mthrd
80038 zzzsLawPro 7549 1500 1233 7131 N N
408d0 2 work process private 3486 1 0 3485 0..3526 :
65304..65535
0 0 work kernel seg 2038 1499 1213 3549 0..21919 :
65474..65535
54015 d work shared library text 1320 0 20 34 0..65535
20668 1 pers code,/dev/hd1:132711 609 0 - - 0..944
3088c f work shared library data 88 0 0 63 0..549
446f1 - pers /dev/hd1:520303 8 0 - - 0..54
Toppan
consumed to a particular dangerous point, what will you do? I will try to
restart some program and let it release some memory. However, it seems that
this methord can only help a little. Something still dominates the memory
and cannot release.
In our AIX, there are some out-sourced program installed, so if I can
monitor which process is dominating the memory, I can notify them to modify
the program. I tried "ps alx", but I don't know how to read it, do you know
which column is refering to real memory and paging space? And what is the
util of the result?
Thank you very much for your valuable opinion!
"John R. Campbell" <so...@penrij.uucp.jtan.com> 撰寫於郵件
news:slrnardbe...@soup2nets.net.dhis.org...
John R. Campbell
> Actaully, if you discover that real memory and paging space has been
> consumed to a particular dangerous point, what will you do? I will try to
> restart some program and let it release some memory. However, it seems that
> this methord can only help a little. Something still dominates the memory
> and cannot release.
>
> In our AIX, there are some out-sourced program installed, so if I can
> monitor which process is dominating the memory, I can notify them to modify
> the program. I tried "ps alx", but I don't know how to read it, do you know
> which column is refering to real memory and paging space? And what is the
> util of the result?
>
> Thank you very much for your valuable opinion!
I don't have an AIX machine handy right now, but (from memory)
there are two columns of interest: the RSS and MEM. I'm a
little cloudy on whether these are in K or pages (4K granules);
vmstat, for instance, uses the 4K granularity for AVM and FRE,
for instance.
The RSS is the "Resident Storage Set" (my acronym may be a little
off) and is supposed to report the real memory consumed by the
process while the other (MEM) reports on the total size (both
in-core and in paging space(s)).
If I had half a brain right now I could give you the necessary
inputs for sort to process the output of ps alx (usually of
the form sort -n -r +5 or some such) so that the largest
processes will be presented first.
I'm sorry my recall for specifics like that is feeble right now;
I don't have any AIX boxes at home (the 60+ development/test
boxes at work being a little overwhelming at times) and my
linux boxes won't provide all of the goodies I need.
Nicholas Dronen
> The RSS is the "Resident Storage Set" (my acronym may be a little
> off) and is supposed to report the real memory consumed by the
> process while the other (MEM) reports on the total size (both
> in-core and in paging space(s)).
RSS usually means "resident set size." (I too do not recall
whether it's reported in k or pages.)
Regards,
Nicholas
Todd Tavasci
From the man page for "ps" on AIX 4.3.3.......
RSS
(v flag) The real-memory (resident set) size of the process (in 1KB
units).
regards,
Todd T
Toppan
"John R. Campbell" <so...@penrij.uucp.jtan.com> 撰寫於郵件
news:slrnaremr...@soup2nets.net.dhis.org...
John R. Campbell
> Can I sort with RSS by desc order?
Yes, that'd be the -r on the sort command; The -n forces it to
be treated as a numeric field.
Some additional items:
1) If paging space is filling up then a process has a
serious memory leak; I've seen plenty of Java Runtimes
(AIX, Linux, Windoze, etc) that have that kind of problem
but I haven't spent much time chasing it down. It seems
to vary depending upon the Java App.
2) Until the paging space is getting full the rest of the
system won't be hurting too badly despite any memory
leaks since the pages get pushed out but seldom referenced
to bring 'em back. This tends to buy you time.
Once you go over the edge, though, there's not much left to
bring you back. Using a DBMS and raw logical volumes (rather
than using "files" in a jfs) does make you more sensitive to
runaways like this, so you'll need to closely monitor your use
of RAM and paging space.
Actually, your DBMS will likely be a memory abuser as well
since the use of raw LVs will require it to use it's own memory
bufferring scheme; This may work out to a dbms tuning issue.
I remember from a DB2 admin workshop (for v5.2!) that using
"DMS" (Database Managed Storage) requires a lot more attention
to sizing of the dbms itself; I would expect that the buffers
would be pinned into memory and should be treated as computational
pages (assuming the character LV device rather than the block LV
device).
Assuming a memory leak in an application since this isn't a
problem until the system has been running for a while is a
reasonable place to start. You may want to ensure that the
"sar" tools are installed so you can collect performance
information; At the same time run ps alx and accumulate many
samples. Sar can give you system-wide trends (with some disk
activity information) but won't provide any process-related
information- which you need to find your memory consumer.
At this point it's a matter of detective work- And, as Yogi
Berra said, "you can learn a lot by watching". You may need
to construct your own instruments to both watch and collect
(so you can play back what happened while you weren't looking)
but that's not a difficult task.