hp-ux, top, ps, glance and memory size
mat...@my-deja.com
all give different sizes of resident memory and virtual memory.
1. Top: I have tried three different versions of top. The one that
comes with hp-ux 11, the 3.5beta9 compiled from the author's source
code and 3.5beta12 (??) downloaded from hpux.cae.wisc.edu. Each gives a
different size for "SIZE" and "RES". Anyone know how this is calculated
by top?
2. ps -el: ps -el will give the size of a process in number of pages.
If you multiply this by 4096 then you should get the memory size, right?
What would this number correspond to in top, "SIZE" or "RES"?
3. glance/gpm shows yet another figure for the memory size of a process.
Anyone know how it is generated?
I'm looking for the best method to size the memory on a system that will
have thousands of the same process running at the same time and I'm
worried that 4GB ram will not be enough. Does anyone have a better
method to calculate this?
Matt Butler
Agilent Technologies
Sent via Deja.com
http://www.deja.com/
Jim Sadler
I don't know enough about top to help. With ps starting with 11.x
it is possible to have variable page sizes. I think you can use
chatr on the binary and find out what page size it is using.
With glance if you are looking at the process list RSS column, be aware
that it is a educated guess. To get a definitive answer double click on
the process that you are interested in and then go to reports-->memory
there you will find all of the memory regions defined. Also shared memory
and memory mapped regions make it trickier to define "How much memory does
a process use?"
Jim Sadler
HP Consulting
b...@atl.hp.com
> I'm trying to find the size of a process in memory. Top, ps and glance
> all give different sizes of resident memory and virtual memory.
Not surprising at all.
> I'm looking for the best method to size the memory on a system that will
> have thousands of the same process running at the same time and I'm
> worried that 4GB ram will not be enough. Does anyone have a better
> method to calculate this?
First, don't worry at all about how much you can put in RAM. HP-UX
is a virtual memory system. I run a server with 256 megs of RAM and
about 2,000 megs of processes (typically about 750 processes at the
same time). The only thing that will prevent a process from fitting
into your virtual memory is swap space.
Think of RAM as a special condition of swap space. If RAM is too small
for everything at the same time, idle processes are deactivated and
paged out to the swap area. This is no problem at all if most of your
processes are interactive (as they are on the previously mentioned
server). If all the processes are truly busy, then swapping will
be excessive and overhead will be enormous, but the processes will
still run.
Also, RAM is occupied with several other items which are related to
processes but are shared. The kernel has a small portion (a few
megs) followed by many megs of table space for I/O, process table,
file table and so on. So take away 30-100 megs for the kernel area
(glance -m to see how much is used). Now there are variable areas
used for shared libraries. These are shared by most of the processes
currently running but the amount of space will vary depending on
just what each process is using. But it's not a lot of space
relative to gigabytes of RAM.
Then processes may share memory areas for cooperative activities like
a database engine. In this case, you could probably assign all the
shared memory area to one process. Now comes the buffer cache, an
area that is used to read-ahead and buffer writes for the filesystems.
The default is 5-50% of RAM...hummm, quite a range. 50% is just
slightly too much and even though it will back off, I think a more
reasonable value for 4Gb is 2-20% but it depends on how much file I/O
is being done.
Now for individual processes, it still isn't simple. Each process
has 3 areas that are mapped into memory, text, stack and data. The
text area is all fixed instructions (no data) but multiple copies of
the same program will share one text area.
So the amount of space occupied by a program is quite fuzzy to
calculate. All you need is to make sure you have plenty of swap
space as HP-UX is a virtual memory operating system. Swap space
defines the amount of memory you can have for processes. RAM
is a special case of virtual memory--and swapping isn't a bad
thing as long as it isn't excessive. Monitor paging activity
with vmstat or Glance. Only page-outs are meaningful (page-in
refers to process starts as well as returns from swap).
So to see all processes sorted by size:
# UNIX95= ps -e -o ruser,vsz,pid,args
And use swapinfo -tm to see if swap space is getting low.
--
?
/'^'\
( o o )
*====oOOO===(_)===OOOo======*=====================================*
| Bill Hassell | Hewlett-Packard Response Center |
| SysAdmin Oooo. | b...@atl.hp.com / Atlanta, GA. |
*======.oooO===( )========*=====================================*
( ) ) / "There are two types of computer users in the
\ ( (_/ world...those that have lost data, and those
\_) that are going to." (blh, circa 1972)
Juan Paradinas Saenz de Viteri;n00990
> have thousands of the same process running at the same time and I'm
> worried that 4GB ram will not be enough. Does anyone have a better
> method to calculate this?
Compile this program with a C++ compiler. It's very useful.
To run it : progName -pid N
where N is the pid of the process you want to watch.
There a several kinds of memory that processes use ( map files, shared
segments, stack, text ) so is difficult to say the actual amount of memory
that a process uses.
#include <SC/Args.h>
#include <sys/param.h>
#include <sys/pstat.h>
#include <sys/unistd.h>
#include <malloc.h>
#include <stdio.h>
#include <nlist.h>
#include <a.out.h>
#include <iostream.h>
#include <stdlib.h>
#undef isset
const char* keywords[] = { "pid:", 0};
int main(int argc, const char* argv [] )
{
Args args(argc, argv, "", 0, keywords);
// default values for command line arguments
int target;
if (args.isset("pid"))
{
int i;
if (sscanf(args.value("pid"), "%d", &i) == 1)
target = i;
}
cout << "System Memory overview" << endl;
struct pst_static pst;
struct pst_dynamic dyn;
long page;
long maxdsiz;
long malloc_free_buffer;
if (pstat_getstatic(&pst, sizeof(pst), (size_t)1, 0) == -1) {
perror("pstat_getstatic");
} else {
page = pst.page_size;
cout<< "page size is " << page << "bytes" << endl;
cout<< "physical memory size is " << pst.physical_memory << "
pages, " << pst.physical_memory * pst.page_size << " bytes" << endl;
}
if (pstat_getdynamic(&dyn, sizeof(dyn), 1, 0) == -1) {
perror("pstat_getdynamic");
} else {
cout<<"total virtual memory allocated is " << dyn.psd_vm << "
pages, " << dyn.psd_vm * page << " bytes" << endl;
cout<<"active virtual memory is " << dyn.psd_avm <<" pages, " <<
dyn.psd_avm * page << " bytes" << endl;
cout<<"total real memory is " << dyn.psd_rm << " pages, " <<
dyn.psd_rm * page << " bytes" << endl;
cout<<"active real memory is " << dyn.psd_arm << " pages, " <<
dyn.psd_arm * page << " bytes" << endl;
cout<<"free memory is " << dyn.psd_free << " pages, " <<
dyn.psd_free * page << " bytes" << endl;
}
{
struct mallinfo mi;
free(malloc(8000)); /* just to have some free area to report */
mi = mallinfo();
malloc_free_buffer = mi.fordblks;
cout<<"free malloc ordinary buffer memory is " << mi.fordblks << "
bytes" << endl;
cout<<"free malloc small block memory is " << mi.fsmblks << "
bytes" << endl;
}
{
struct nlist nl[2];
FILE *kernel;
struct header head;
struct som_exec_auxhdr auxhead;
long offset;
char *kernel_path;
if (access("/hp-ux", R_OK) == 0) {
kernel_path = "/hp-ux";
} else if (access("/stand/vmunix", R_OK) == 0) {
kernel_path = "/stand/vmunix";
} else {
cout<<"couldn't read kernel" << endl;
exit(1);
}
nl[0].n_name = "maxdsiz";
nl[1].n_name = "";
if (nlist(kernel_path, nl) == -1) {
perror("nlist");
} else {
kernel = fopen(kernel_path, "r");
if (kernel == NULL) {
perror("fopen");
} else {
fread(&head, sizeof(head), 1, kernel);
fseek(kernel, head.aux_header_location, SEEK_SET);
fread(&auxhead, sizeof(auxhead), 1, kernel);
offset =
nl[0].n_value-auxhead.exec_dmem+auxhead.exec_dfile;
fseek(kernel, offset, SEEK_SET);
fread(&maxdsiz, sizeof(maxdsiz), 1, kernel);
cout<<"maximum process data size is " << maxdsiz * page <<
" bytes" << endl;
}
}
}
{
struct pst_static pstatic;
struct pst_status pst;
long page;
if (pstat_getstatic(&pstatic, sizeof(pstatic), (size_t)1, 0) == -1) {
perror("pstat_getstatic");
} else {
page = pstatic.page_size;
}
if (pstat_getproc(&pst, sizeof(pst), (size_t)0, target) == -1)
perror("pstat_getproc");
else {
cout<< endl << "This process is using " << page * (
pst.pst_dsize+ /* # real pages used for data */
pst.pst_tsize+ /* # real pages used for text */
pst.pst_ssize+ /* # real pages used for stack */
pst.pst_shmsize+ /* # real pages used for shared memory */
pst.pst_mmsize+ /* # real pages used for memory mapped
files */
pst.pst_usize+ /* # real pages used for U-Area & K-Stack */
pst.pst_iosize) /* # real pages used for I/O device mapping
*/ << " bytes of RAM" <<endl;
cout<< "real bytes used for data " << pst.pst_dsize * page << endl;
cout<< "real bytes used for text " << pst.pst_tsize * page << endl;
cout<< "real bytes used for stack " << pst.pst_ssize * page << endl;
cout<< "real bytes used for shared memory " << pst.pst_shmsize * page <<
endl;
cout<< "real bytes used for memory mapped files " << pst.pst_mmsize *
page << endl;
cout<< "real bytes used for U-Area & K-Stack " << pst.pst_usize * page
<< endl;
cout<< "real bytes used for I/O device mapping " << pst.pst_iosize *
page << endl;
cout<< "This process is addressing " << page * pst.pst_vtsize
<< " bytes of virtual text" << endl;
cout<< "This process is addressing " << page * pst.pst_vdsize
<< " bytes of virtual data" << endl;
cout<< "This process is addressing " << page * pst.pst_vssize
<< " bytes of virtual stack" << endl;
cout<< "This process is addressing " << page * pst.pst_vshmsize
<< " bytes of virtual shared memory" << endl;
cout<< "This process is addressing " << page * pst.pst_vmmsize
<< " bytes of virtual memory map file" << endl;
cout<< "This process is addressing " << page * pst.pst_vusize
<< " bytes of virtual U-Area & K-Stack" << endl;
cout<< "This process is addressing " << page * pst.pst_viosize
<< " bytes of virtual I/O dev mapping" << endl;
cout<< "User time spent executing (in seconds): " << pst.pst_utime <<
endl;
cout<< "System time spent executing (in seconds): " << pst.pst_stime <<
endl;
cout<< "%cpu for this process during p_time: " << 100*pst.pst_pctcpu <<
"%" <<endl;
}
}
return 0;