program memory usage [cross-post]

[Alessandro Basili]

I have written a program in C language, but I'm facing a memory problem,
since from time to time the memory usage (shown with ps and/or top
utilities) increases.

I used the getrusage function from the GNU C library, but
unfortunately the kernel we are using (linux 2.6) is not supporting the
fields for memory usage in the rusage structure.

After some search I started to use the /proc file system and the statm
file related to the pid of the running program.
Now I have inserted throughout the program several calls to a function
that prints the statistics from the statm file in order to isolate the
increase of memory usage.

Unfortunately the result points to a part of the program that does not
have any hidden ambiguities related to memory usage, which is also
confirmed by isolating it in a test program.

Now my questions:

- is there any replacement to the getrusage function?
- is there any alternative method to figure out the memory usage of the
program within the program itself?
- is there any where explained how the GNU/Linux system updates the
statm file and how much it is reliable at spotting increasing memory usage?

Thanks for any suggestion.

Al

p.s.: my apologies for the cross-posting, I also posted a similar
article to the comp.lang.c but I was advised to search somewhere else,
since none of those questions where C related.

--
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

[Chris Davies]

Alessandro Basili <alessand...@cern.ch> wrote:
> - is there any alternative method to figure out the memory usage of the
> program within the program itself?

One of the debugging/profiling malloc libraries may help.

Chris

[Tim Watts]

Valgrind for testing for memory leaks?

--
Tim Watts

[Henrik Carlqvist]

Tim Watts <tw+u...@dionic.net> wrote:
> Valgrind for testing for memory leaks?

Valgrind is really great for checking memory leaks and use of
uninitialized variables. With --tool=callgrind it is also a great
profiling tool together with kcachegrind.

I haven't tried it myself, but there is also the massif heap profiler for
valgrind:

http://valgrind.org/docs/manual/ms-manual.html

regards Henrik
--
The address in the header is only to prevent spam. My real address is:
hc123(at)poolhem.se Examples of addresses which go to spammers:
root@localhost postmaster@localhost

[Alessandro Basili]

thanks a lot for pointing that out. I didn't know the mtrace() function
and, even though I struggled a bit to find out glibc-utils was not
installed on my system, is a nice to have.

Unfortunately the outcome is either unreadable (even using two
parameters) or not very useful at least so far.
It would be useful to "print" on the stdout the evolution of the memory
usage at the time it is increased, in order to trace it while running.

> Chris

[Alessandro Basili]

On 9/24/2011 9:26 AM, Tim Watts wrote:
[...]

>
> Valgrind for testing for memory leaks?
>

I guess I need some deep study before understanding the output of this
profiler (that's why I don't normally user). Nothing even close to what
I can currently understand, but I will give it a try and see if
something comes out.

FYI my function that checks the statm file is logging much more when
running with valgrind, as if valgrind is using more and more memory (or
induce my program to use more and more memory???).

[Henrik Carlqvist]

Alessandro Basili <alessand...@cern.ch> wrote:
>> Valgrind for testing for memory leaks?

> I guess I need some deep study before understanding the output of this
> profiler

The output from the default memcheck tool in valgrind is rather
straightforward to read. However, to make some sense it really helps if
your program was compiled with the -g flag. Then the output will point to
line numbers in your code instead of hexadecimal addresses.

[Peter 'Shaggy' Haywood]

Groovy hepcat Alessandro Basili was jivin' in comp.os.linux.misc on Fri,
23 Sep 2011 7:11 pm. It's a cool scene! Dig it.

[Snip.]

> - is there any alternative method to figure out the memory usage of
> the program within the program itself?

Of course. Every time you allocate memory, keep track of it. Maybe use
malloc()/calloc()/realloc()/free() wrappers that keep a running total
of all allocated memory. Of course, that may not always be an accurate
indication of how much of the system memory is actually assigned to
your program, but it could give you some clue about where memory
problems occur.

--
Dig the sig!

----------- Peter 'Shaggy' Haywood ------------
Ain't I'm a dawg!!

[Colin S. Miller]

On 25/09/11 19:20, Henrik Carlqvist wrote:
> Alessandro Basili<alessand...@cern.ch> wrote:
>>> Valgrind for testing for memory leaks?
>
>> I guess I need some deep study before understanding the output of this
>> profiler
>
> The output from the default memcheck tool in valgrind is rather
> straightforward to read. However, to make some sense it really helps if
> your program was compiled with the -g flag. Then the output will point to
> line numbers in your code instead of hexadecimal addresses.
>
> regards Henrik

Remember to link with -g as well.
You probably want to disable optimization ( -O0 minus oh zero ),
and perhaps install the debug versions of any shared libraries you are using

Debian derived Linuxs call these libfoo-dbg,
Mandrake derived call them libfoo-debug.

With valgrind you can expect a 5-fold (at least) slow down of your program
(if cpu-bound).
Valgrind also is highly tied to the CPU-OS combination you are running on;
check its website if you aren't using linux-i386 (32 bit).

http://valgrind.org/info/platforms.html for the list

HTH,
Colin S. Miller



--
Replace the obvious in my email address with the first three letters of the hostname to reply.