Can we detect mem leak detection using pmap?

[Jaydeep Chovatia]

Hi,

I am working on a C++ Linux project. In this project I see some memory leaks under heavy load run only, if i execute functionality manually(w/o load) then i do not see any leakages.

It looks like under load it executes some different code path which is not covered under functional testing (without load). I cannot run under valgrind as this problem occurs under load only.

So I have collected couple of "pmap" under load and found that there are lots of
"[anon]" memory of same size. I have address of blocks which have leaked probably but I am unable to map those memory address to code.

If somebody has idea about how to debug leak using "pmap" then please share it with me.

-jaydeep

[Ian Collins]

Jaydeep Chovatia wrote:

{please clean up the mess google makes of your posts!}

> Hi,
>
> I am working on a C++ Linux project. In this project I see some
> memory leaks under heavy load run only, if i execute functionality
> manually(w/o load) then i do not see any leakages.
>
> It looks like under load it executes some different code path which
> is not covered under functional testing (without load). I cannot run
> under valgrind as this problem occurs under load only.

Have you tried an allocator with debugging support such as libumem?

--
Ian Collins

[Jorgen Grahn]

On Tue, 2013-07-30, Ian Collins wrote:
> Jaydeep Chovatia wrote:
>
> {please clean up the mess google makes of your posts!}

(Replying via Ian's posting so I don't have to clean up, too.)

>> I am working on a C++ Linux project. In this project I see some
>> memory leaks under heavy load run only, if i execute functionality
>> manually(w/o load) then i do not see any leakages.
>>
>> It looks like under load it executes some different code path which
>> is not covered under functional testing (without load). I cannot run
>> under valgrind as this problem occurs under load only.
>
> Have you tried an allocator with debugging support such as libumem?

Since it's C++, also see if you can find the problem by code review.
In well-written C++ it is hard to create memory leaks, and the
potential places are rather few and easy to find and review.

(In badly written C++ it is even easier than in C, since you have
exceptions working against you rather than /for/ you.)

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

[Richard Kettlewell]

Jaydeep Chovatia <chovatia...@gmail.com> writes:
> I am working on a C++ Linux project. In this project I see some memory
> leaks under heavy load run only, if i execute functionality
> manually(w/o load) then i do not see any leakages.
>
> It looks like under load it executes some different code path which is
> not covered under functional testing (without load). I cannot run
> under valgrind as this problem occurs under load only.

I’m not convinced that last statement necessarily follows...

> So I have collected couple of "pmap" under load and found that there
> are lots of "[anon]" memory of same size. I have address of blocks
> which have leaked probably but I am unable to map those memory address
> to code.
>
> If somebody has idea about how to debug leak using "pmap" then please
> share it with me.

Linux’s pmap just reports /proc/PID/maps, AFAICS. I don’t think you’ll
find it very helpful.

The Boehm GC has a leak detection mode, that might help if you really
can’t use valgrind.

--
http://www.greenend.org.uk/rjk/

[Jorgen Grahn]

On Wed, 2013-07-31, Richard Kettlewell wrote:
> Jaydeep Chovatia <chovatia...@gmail.com> writes:
>> I am working on a C++ Linux project. In this project I see some memory
>> leaks under heavy load run only, if i execute functionality
>> manually(w/o load) then i do not see any leakages.
>>
>> It looks like under load it executes some different code path which is
>> not covered under functional testing (without load). I cannot run
>> under valgrind as this problem occurs under load only.
>
> I'm not convinced that last statement necessarily follows...

True: in a sense it becomes easier to trigger some "high load"
problems when you have valgrind slowing everything down.

/Jorgen

PS. This posting from Jaydeep seems to be very similar to his "when
V-pointer becomes NULL" in comp.lang.c++ two weeks ago:
<34619b1d-b3be-42cc...@googlegroups.com>.
It would be interesting to know if this is the same question,
only formulated differently ...

[Barry Margolin]

In article <slrnkvi0dl.2...@frailea.sa.invalid>,

Jorgen Grahn <grahn...@snipabacken.se> wrote:

> On Wed, 2013-07-31, Richard Kettlewell wrote:
> > Jaydeep Chovatia <chovatia...@gmail.com> writes:
> >> I am working on a C++ Linux project. In this project I see some memory
> >> leaks under heavy load run only, if i execute functionality
> >> manually(w/o load) then i do not see any leakages.
> >>
> >> It looks like under load it executes some different code path which is
> >> not covered under functional testing (without load). I cannot run
> >> under valgrind as this problem occurs under load only.
> >
> > I'm not convinced that last statement necessarily follows...
>
> True: in a sense it becomes easier to trigger some "high load"
> problems when you have valgrind slowing everything down.

But if the problem only occurs after it has performed N transactions
(for large N), and the slowdown means that will take days, weeks, or
months.... There could be a feedback effect: if the users avoid using
the service because it's slowed down, that will also increase the time
to reach N. But it could also oscillate: when enough users avoid using
it, it will speed up for the rest, who tell them "try it again, they've
fixed the performance problem".

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Jorgen Grahn]

Yes, thus my "some" above, and I guess RK's "[not] necessarily" too.

/Jorgen

[Richard Kettlewell]

Barry Margolin <bar...@alum.mit.edu> writes:
> Jorgen Grahn <grahn...@snipabacken.se> wrote:
>> On Wed, 2013-07-31, Richard Kettlewell wrote:
>>> Jaydeep Chovatia <chovatia...@gmail.com> writes:

>>>> It looks like under load it executes some different code path which is
>>>> not covered under functional testing (without load). I cannot run
>>>> under valgrind as this problem occurs under load only.
>>>
>>> I'm not convinced that last statement necessarily follows...
>>
>> True: in a sense it becomes easier to trigger some "high load"
>> problems when you have valgrind slowing everything down.
>
> But if the problem only occurs after it has performed N transactions
> (for large N), and the slowdown means that will take days, weeks, or
> months....

Indeed, sometimes one may have no alternative to patience.

> There could be a feedback effect: if the users avoid using the service
> because it's slowed down, that will also increase the time to reach
> N. But it could also oscillate: when enough users avoid using it, it
> will speed up for the rest, who tell them "try it again, they've fixed
> the performance problem".

Normally an early debugging step would be figuring out how to reproduce
the problem in a test environment, rather than continuing to rely on
real users to provoke the bug.

--
http://www.greenend.org.uk/rjk/

[Barry Margolin]

In article <87siyuz...@araminta.anjou.terraraq.org.uk>,

Yeah. It sounds like he needs to write a load generator, so he can try
to simulate the production load.

[Jorgen Grahn]

On Wed, 2013-07-31, Barry Margolin wrote:

Yes, although not necessarily write one /from scratch/.

If my hunch is right and this is some kind of server, perhaps
networked:

In my experience you can catch a few obvious problems by manual
testing (e.g. running a brower to test a web server) or by slow,
scripted one-client testing. Or the popular unit tests.

But that is just the beginning. The really difficult problems show up
under load, and/or under "dirty" conditions like a less-than perfect
networking environment. You can test a server with one client, but
that doesn't give you confidence that it will work under /real/
conditions.

/Jorgen

[Jaydeep Chovatia]

Thanks a lot for your help, finally I have fixed this issue by running static analysis too "cppcheck" on my code. This was my mistake,

char *ptr = new char[...];

.....
ptr = realloc(...)

....

delete[] ptr;

I was using realloc+delete combination which causes un-define behavior and for me un-define behavior was memory leak :)

[Ian Collins]

Jaydeep Chovatia wrote:
> Thanks a lot for your help, finally I have fixed this issue by
> running static analysis too "cppcheck" on my code. This was my
> mistake,
>
> char *ptr = new char[...];
>

> ...... ptr = realloc(...)
>
> .....

>
> delete[] ptr;
>
> I was using realloc+delete combination which causes un-define
> behavior and for me un-define behavior was memory leak :)

Which goes to show why a common recommendation with C++ is to avoid
messing with raw pointers! It sounds like you would be better off using
std::vector.

--
Ian Collins

[Jorgen Grahn]

On Fri, 2013-08-02, Ian Collins wrote:
> Jaydeep Chovatia wrote:
>> Thanks a lot for your help, finally I have fixed this issue by
>> running static analysis too "cppcheck" on my code. This was my
>> mistake,
>>
>> char *ptr = new char[...];
>>
>> ...... ptr = realloc(...)
>>
>> .....
>>
>> delete[] ptr;
>>
>> I was using realloc+delete combination which causes un-define
>> behavior and for me un-define behavior was memory leak :)
>
> Which goes to show why a common recommendation with C++ is to avoid
> messing with raw pointers!

Or at the very least to avoid messing with new.

I'd also like to repeat what I wrote elsewhere in the thread:

In well-written C++ it is hard to create memory leaks, and the
potential places are rather few and easy to find and review.

(In badly written C++ it is even easier than in C, since you have
exceptions working against you rather than /for/ you.)

The code above is what I'd call "badly written C++". It looks like a
kind you sometimes see: C translated to C++ using a few simplistic
rules like "replace malloc with new[]".

> It sounds like you would be better off using
> std::vector.

Yes, using the standard container data structures is what makes
the difference.

Followup-To set to comp.lang.c++.

[Rainer Weikusat]

The last time I looked at this (which must have been about 16 years
ago) 'the generic C++ string library' (it's not that you can't get
more of these for C than anyone could possibly want handle) was a little
useles in this respect because it didn't guarantee that vectors were
continously allocated. Did someone change this?

[Ian Collins]

The C++ standard!

--
Ian Collins

[Jorgen Grahn]

On Fri, 2013-08-02, Ian Collins wrote:

And it wasn't as if most uses of std::string/vector suffered from it.
I mostly saw it when mixing C++ with good old Unix I/O, e.g. socket I/O.

As far as I can tell, most people saw things like RW does, and it
/had/ to be standardized eventually, because everyone already assumed
you could get away with

std::vector<char> v = ...;
write(&v[0], v.size());

[Rainer Weikusat]

That's actually a nice example of common sense prevailing despite some
ivory tower eggheads really DISAPPROVE (!!!) of it ...