Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

memory usage per process

1,831 views
Skip to first unread message

Sierp

unread,
Feb 11, 2009, 3:27:48 PM2/11/09
to
Hello,

I'm thinking about some very simple thing - I was assuming it would be
simple. But it's not that simple.
I'm talking about memory usage, real memory usage for PID and for all
machines PIDs.

Let's say I'm looking at nmon output, and see something like this:

Physical PageSpace
MB Used 15156 2256
MB Free 1740 3887
Total(MB) 16896 6144

Is there any command I could use to check, what exactly is consuming
this 17412MB of RAM?
I know, that some part of memory is used for filesystemcache - in this
case, 10%.
I was thinking about svmon -Pu and taking "Inuse" field, for example I have

svmon -Pu

Pid Command Inuse Pin Pgsp Virtual .....
11111 oracle 755797 65584 303066 746923

but when I add all Inuse/4/1024=something in MB, I recive something more
than 22GB. As I assumed from manual for svmon, this might be becouse of
shared memory between processes, which is every time counted separatly.

In the other hand, when I try to do something like this:

for PID in `ps -ef | awk '{ print $2}'`; do
ps v $PID | grep -v COMMAND | awk '{ print $7}';
done

which means I have all RSS outputs for all PIDs on this machine - and
when I sum all this RSS outputs - I receive value of 4835808, which
means about 4722MB used. About 1700MB is used for filesystem cache.
So this is all I know - I know what is happening with 6,5GB of my RAM,
but what about 10GB of it?
What is utilizing it?

I would be gratefull for good answer, I was googling for it and to be
honest, I couldn't find the answer except what I wrote above.

regards,

Sierp

Andreas Schulze

unread,
Feb 11, 2009, 5:14:06 PM2/11/09
to
On 11 Feb., 21:27, Sierp <si...@nolspamatsierp.net> wrote:
>[...] I'm looking at nmon output, and see something like this:

>
>                 Physical        PageSpace
> MB Used         15156           2256
> MB Free         1740            3887
> Total(MB)       16896           6144
>
> Is there any command I could use to check, what exactly is consuming
> this 17412MB of RAM?
> I know, that some part of memory is used for filesystemcache - in this
> case, 10%.
> I was thinking about svmon -Pu and taking "Inuse" field,[...]

You are on the right track by using svmon to identify memory usage.
I'd like to suggest to you that you use the system wide view with
svmon -S instead of the process view with svmon -P. With the system
wiede view every memory segment will show up once and you can also see
memory segments that don't belong to a process.
If you want to know which process uses a certain memory segment use
svmon -P <PID> -l and/or svmon -Pns. Look out for same virtual segment
IDs vsid and shmat/mmap description.
A good starting point for reading IMHO would be the AIX documentation
at the Information Center:
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.prftungd/doc/prftungd/mem_usage.htm

Henry

unread,
Feb 11, 2009, 5:16:59 PM2/11/09
to

I'll be interested because I too have done a lot of work with svmon,
ps (and awk).

I wrote this to translate the svmon -P output to MB and have tried all
sorts with running totals of all procs

cat /opt/local/bin/svmon-P.awk
#!/usr/bin/awk
BEGIN{ printf("%-10s%20s%10s%10s%10s%10s\n",
"Pid","Command","Inuse","Pin","Pgsp","Virtual")}
{
if ( /Pid Command/ )
{
#print
getline
printf("%-10s%20s%10.0f%10.0f%10.0f%10.0f\n", $1, $2, ($3*4/1024),
($4*4/1024), ($5*4/1024), ($6*4/1024))
sum_Inuse+=($3*4/1024)
sum_Pin+=($4*4/1024)
sum_Pgsp+=($5*4/1024)
sum_Virtual+=($6*4/1024)
}
}
END{
printf("%-30s%10.0f%10.0f%10.0f%10.0f\n", "Totals in MB", sum_Inuse,
sum_Pin, sum_Pgsp, sum_Virtual)


so I run this to see if DB2 can be calculated - I don't believe its
correct

#!/bin/ksh
# $Id$

PATH=/usr/bin

function svmon_pid {
# check svmon memory mapping

# see if there's any DB2 processes running
# the ps "comm" and "args" produce different output for the "db2sysc"
command
list_of_db2_pids=$( ps -e -o user,pid,ppid,etime,time,pcpu,args | awk
'$7 ~ "db2sysc|db2agent|db2ckpwd|db2fmp|db2acd" && $5 !=
"00:00:00" {print $2}')
if [[ -n $list_of_db2_pids ]] ; then

# need root for svmon
if [[ $(whoami) != "root" ]]; then
print have to be root user, exiting
exit
fi

echo "svmon stats"
for db2_pid in $list_of_db2_pids ; do
# some output modifications
svmon -P $db2_pid | awk -f /opt/local/bin/svmon-P.awk
done | awk '{print}/Totals in MB/ {sum_Totals+=$4;+
+num_db2_hits}END {print "Totals of all DB2 " sum_Totals " MB - ",
num_db2_hits " PIDS\n"}'
else
echo no DB2 running
exit 0
fi
}

function db2_by_user {
# look through process table for DB2
echo "only show processes using TIME and CPU"
ps -e -o user,pid,ppid,etime,time,pcpu,args| awk '
BEGIN { printf("\n%-10s%-10s%-10s%-15s%-15s%-10s\n", "UID", "PID",
"PPID", "ETIME", "TIME", "CPU%") }
$7 ~ "db2sysc|db2agent|db2ckpwd|db2fmp|db2acd" && $5 != "00:00:00" &&
$6 != "0.0" {
UID=$1
PID=$2
PPID=$3
ETIME=$4
TIME=$5
PCPU=$6
printf("%-10s%-10s%-10s%-15s%-15s%-10s\n" , UID, PID , PPID, ETIME,
TIME, PCPU)
}'

}
# entry point
svmon_pid
db2_by_user

and running it

sudo show_db2_size.sh
svmon stats


Pid Command Inuse Pin Pgsp Virtual

262182 db2sysc 135 31 2 144
Totals in MB 135 31 2 144


Pid Command Inuse Pin Pgsp Virtual

266340 db2sysc 136 31 2 144
Totals in MB 136 31 2 144


Pid Command Inuse Pin Pgsp Virtual

323726 db2sysc 136 31 2 145
Totals in MB 136 31 2 145


Pid Command Inuse Pin Pgsp Virtual

372760 db2sysc 232 31 3 241
Totals in MB 232 31 3 241


Pid Command Inuse Pin Pgsp Virtual

377068 db2fmp 125 31 2 133
Totals in MB 125 31 2 133


Pid Command Inuse Pin Pgsp Virtual

434226 db2sysc 135 31 2 144
Totals in MB 135 31 2 144


Pid Command Inuse Pin Pgsp Virtual

462916 db2sysc 136 31 2 145
Totals in MB 136 31 2 145


Pid Command Inuse Pin Pgsp Virtual

475318 db2sysc 136 31 2 145
Totals in MB 136 31 2 145


Pid Command Inuse Pin Pgsp Virtual

479232 db2fmp 181 31 5 192
Totals in MB 181 31 5 192


Pid Command Inuse Pin Pgsp Virtual

528400 db2sysc 162 31 2 171
Totals in MB 162 31 2 171


Pid Command Inuse Pin Pgsp Virtual

561222 db2sysc 321 31 3 329
Totals in MB 321 31 3 329


Pid Command Inuse Pin Pgsp Virtual

618522 db2fmp 125 31 2 133
Totals in MB 125 31 2 133


Pid Command Inuse Pin Pgsp Virtual

630910 db2fmp 131 31 2 139
Totals in MB 131 31 2 139


Pid Command Inuse Pin Pgsp Virtual

721076 db2sysc 162 31 2 171
Totals in MB 162 31 2 171


Pid Command Inuse Pin Pgsp Virtual

802824 db2sysc 162 31 2 171
Totals in MB 162 31 2 171


Pid Command Inuse Pin Pgsp Virtual

843922 db2fmp 125 31 2 133
Totals in MB 125 31 2 133


Pid Command Inuse Pin Pgsp Virtual

929960 db2fmp 133 31 2 141
Totals in MB 133 31 2 141


Pid Command Inuse Pin Pgsp Virtual

970770 db2sysc 733 31 3 722
Totals in MB 733 31 3 722
Totals of all DB2 3406 MB - 18 PIDS

only show processes using TIME and CPU

UID PID PPID ETIME TIME CPU%
toltt1in 970770 938066 57-15:12:03 2-23:53:08 1.3

Sierp

unread,
Feb 12, 2009, 3:12:07 AM2/12/09
to
Andreas Schulze pisze:

> You are on the right track by using svmon to identify memory usage.
> I'd like to suggest to you that you use the system wide view with
> svmon -S instead of the process view with svmon -P. With the system
> wiede view every memory segment will show up once and you can also see
> memory segments that don't belong to a process.
> If you want to know which process uses a certain memory segment use
> svmon -P <PID> -l and/or svmon -Pns. Look out for same virtual segment
> IDs vsid and shmat/mmap description.

ok.. is there any little bit more simple way to do it? :-)
svmon -P PID -l is.. well.. long ;-) and very hard to script it

> A good starting point for reading IMHO would be the AIX documentation
> at the Information Center:
> http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.prftungd/doc/prftungd/mem_usage.htm

yes, I know, I've read it already and this is why I'm asking this question :)

regards,

Sierp

Mark Taylor

unread,
Feb 12, 2009, 7:30:17 AM2/12/09
to
You probably just was RSS don't you ? i.e. SZ from ps -leaf .. or "ps -
efF vsz,comm,pid | sort -nr" which excludes all of the other crap
like shared libs, persistient blah ..

HTH
Mark Taylor

Sierp

unread,
Feb 12, 2009, 3:14:05 PM2/12/09
to
Mark Taylor pisze:

> You probably just was RSS don't you ? i.e. SZ from ps -leaf .. or "ps -
> efF vsz,comm,pid | sort -nr" which excludes all of the other crap
> like shared libs, persistient blah ..

but if you sum SZ from all processes you will not receive all your
memory consumption, am I right?

Sierp

Henry

unread,
Feb 12, 2009, 10:25:51 PM2/12/09
to

my numbers still don't add up from the svmon -G and svmon -S
sudo ksh svmon-G.sh

mem_size mem_inuse mem_free mem_pin mem_virtual pg_space_size
pg_space_inuse pin_work pin_pers pin_clnt in_use_work in_use_pers
in_use_clnt
9216 8831 385 3903 7680
2048 1121 3534 0 0 7376
0 1454

sudo ksh sum_svon-S.sh

total clnt 1175.1
total work 1067.91
total work kernel heap 1180.05
total work kernel segment 31.918
total work mbuf pool 2328.38
total work other kernel segments 129.297
total work mmap source 2208.07
Grand total 8120.72

Andreas Schulze

unread,
Feb 13, 2009, 10:48:18 AM2/13/09
to
On 13 Feb., 04:25, Henry <snogfest_hosebe...@yahoo.com> wrote:
> On Feb 13, 9:14 am, Sierp <si...@nolspamatsierp.net> wrote:
>
> > Mark Taylor pisze:
>
> > > You probably just was RSS don't you ? i.e. SZ from ps -leaf .. or "ps -
> > > efF vsz,comm,pid | sort -nr"  which excludes all of the other crap
> > > like shared libs, persistient blah ..
>
> > but if you sum SZ from all processes you will not receive all your
> > memory consumption, am I right?

Yes.

>
> > Sierp
>
> Henry wrote:
>[...]


> my numbers still don't add up from the svmon -G and svmon -S
> sudo ksh svmon-G.sh

> [...]

svmon -G and svmon -S should return an almost identical number for
working storage AVM. It is not identical because of some differences
in handling pinned memory. The output of these commands is useful for
a global view at AIX memory usage.

With the svmon -P command you get a lot of duplicate entries for inuse
usually. E.g. a work kernel and a work shared library text segment. At
the same time some memory used will not show up at al with the process
view, e.g. files that were opened before (by a process that stopped/
ended already) and are kept in cache for possible reuse later.

Sierp

unread,
Feb 16, 2009, 12:08:53 PM2/16/09
to
Andreas Schulze pisze:

> With the svmon -P command you get a lot of duplicate entries for inuse
> usually. E.g. a work kernel and a work shared library text segment. At
> the same time some memory used will not show up at al with the process
> view, e.g. files that were opened before (by a process that stopped/
> ended already) and are kept in cache for possible reuse later.

in my case there was some additional issue
I had a lot of memory reserved in shared memory segment, which could be
seen by ipcs -ap
The problem is (I think) that the PID which reserved this memory wasn't
alive (there was no such PID in OS at all). I could remove this shared
memory segment with ipcrm, but to be honest, I don't know what are the
implications of this issue for system (when there is no PID)

I wonder, why this memory was still reserved (there were oracle user's),
and did I broke anything ;-)

regards,

Sierp

Thomas Braunbeck

unread,
Feb 16, 2009, 1:12:12 PM2/16/09
to
Sierp schrieb:
Why 'reserved"? This is shared memory, one process creates it, others
can use it (the idea behind shared memory). And why should the system
remove this shared memory in case the creator terminaes? There can be
still users (other processes) needing it.

Sierp

unread,
Feb 16, 2009, 2:02:45 PM2/16/09
to
Thomas Braunbeck pisze:

> Why 'reserved"? This is shared memory, one process creates it, others
> can use it (the idea behind shared memory).

ok, the problem is, all oracle processes were stopped - this memory was
still reserved, I don't think something was using it

> And why should the system
> remove this shared memory in case the creator terminaes? There can be
> still users (other processes) needing it.

I'm not writing that system should remove this shared memory, I just
wonder why oracle didn't do it

regards

Sierp

Thomas Braunbeck

unread,
Feb 16, 2009, 10:27:58 PM2/16/09
to
Sierp schrieb:

>
> I'm not writing that system should remove this shared memory, I just
> wonder why oracle didn't do it
>

Some type of failure (memory fault - core dump) or a kill -9 of
the process supposed to cleanup. Check the oracle logs and the
errpt -a.

0 new messages