Finding memory leaks

[Jan Gerbecks]

Hello everybody,

I made some changes to OverSim to simulate an overlay network that incorporates trust information in its routing process.

Unfortunately, I am quite new to C++ as an programming language and have apparently created one (or more) memory leaks.

Can anybody point me in the general direction where it would be most likely to that these occur. I found some help in the OMNeT++/OMNEST Manual (Section 8.11.2) and applied the fixes mentioned there. 

Unfortunately, there must be more :(

The problem especially shows itself when I use many repetitions and my RAM slowly but steadily fills up with each run.

Any help is appreciated,

Jan

[Dilum Bandara]

Hi Jan,

How many nodes are in your simulation? It's natural for OverSim to consume 2GB+ memory even for a network with few 100 nodes.

If no of actives & waiting events in the event engine continue to increase (without becoming steady) you are accumulating messages. Make sure to delete messages if you are not using RPC messages (RPC messages are deleted automatically).

There will be a separate time for each key stored in the DHT this consume lots of memory. Remove TTL timers for DHT entries, if you don't need keys to expire.

Best Regards,
Dilum

--
You received this message because you are subscribed to the Google Groups "oversim" group.
To post to this group, send email to ove...@googlegroups.com.
To unsubscribe from this group, send email to oversim+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/oversim?hl=en.

[Stephan Krause]

Am Mittwoch, 12. Januar 2011, um 17:19:32 schrieb Jan Gerbecks:

> Unfortunately, I am quite new to C++ as an programming language and have
> apparently created one (or more) memory leaks.
>
> Can anybody point me in the general direction where it would be most likely
> to that these occur. I found some help in the OMNeT++/OMNEST Manual
> (Section 8.11.2) and applied the fixes mentioned there. Unfortunately,
> there must be more :(
>
> The problem especially shows itself when I use many repetitions and my RAM
> slowly but steadily fills up with each run.

Use valgrind. This tool has a memory checker that will tell you where memory
is allocated that is never freed.

Regards,

Stephan

[Jan Gerbecks]

Thanks for all your suggestions.

I actually used http://alleyoop.sourceforge.net/usage.html which uses valgrind but provides a GUI as well.

Anyway, I found out that the problem was the XML plugin.

I had disabled line 307-311 in SimpleUnderlayConfigurator.cc.

#if OMNETPP_VERSION>=0x0401

    // free memory used for xml document

    ev.forgetXMLDocument(nodeCoordinateSource);

    malloc_trim(0);

#endif

If you have this enabled, OverSim won't compile under OS X 10.6 and Windows, because malloc_trim is not found.

Normally, malloc.h is included, and there even is a special case for OSX (line 24 )

#if !defined(__APPLE__)

#include <malloc.h>

#endif

But unfortunately, this doesn't work.

I found a few articles, where malloc.h is actually located on OSX (http://macosx.forked.net/comp.html /usr/include/sys/malloc.h)

Can anybody tell me where OverSim looks for the necessary header files?

Take care,

Jan

[Stephan Krause]

Am Donnerstag, 13. Januar 2011, um 10:31:08 schrieb Jan Gerbecks:

> Anyway, I found out that the problem was the XML plugin.
> I had disabled line 307-311 in SimpleUnderlayConfigurator.cc.
> #if OMNETPP_VERSION>=0x0401
> // free memory used for xml document
> ev.forgetXMLDocument(nodeCoordinateSource);
> malloc_trim(0);
> #endif

Please try to only disable the malloc_trim line, and keep the rest. (I've no
access to a mac, so that's untested, but it should work)

Regards,
Stephan

[Jan Gerbecks]

Maybe I have to elaborate a bit ;)

I had the line disabled and everything works on OS X. I programm on my Mac and then uploaded the source code to our Ubuntu based simulation server.

Obviously, I will always forget to reenable the malloc_trim(0) line ;)

As a workaround, I now changed line 307 to

#if OMNETPP_VERSION>=0x0401 && !defined(__APPLE__)

This is fine for me, as I will not run long and complex simulations on my laptop ;)

If I have more time, I might look into this more closely.

Regards,

Jan

[Stephan Krause]

Am Donnerstag, 13. Januar 2011, um 10:57:18 schrieb Jan Gerbecks:
> Maybe I have to elaborate a bit ;)
>
> I had the line disabled and everything works on OS X. I programm on my Mac
> and then uploaded the source code to our Ubuntu based simulation server.
> Obviously, I will always forget to reenable the malloc_trim(0) line ;)
>
> As a workaround, I now changed line 307 to
> #if OMNETPP_VERSION>=0x0401 && !defined(__APPLE__)

OK, so I'll elaborate a bit, too. If you disable the whole block, the memory
for the coordfile, which is parsed on the begin of every run, is never freed,
i.e. your memory consumption increases if you run multiple runs.

If you only disable the malloc_trim call, the memory will be marked free, but
not immediately released. However, it will be re-used eventually, so there is
no real memory leak. Basically, if you change the block to

#if OMNETPP_VERSION>=0x0401
// free memory used for xml document
ev.forgetXMLDocument(nodeCoordinateSource);

#if !defined(__APPLE__) && !defined(_WIN32)
malloc_trim(0);
#endif
#endif

you will 1. have everything as usual on Linux and 2. have no real memory leak
on OS X and Windows, too.

Regards,
Stephan

[Jan Gerbecks]

Thank you Stephan, in that context your solution is much more elegant.

[Hill YU]

I put the file /usr/include/sys/malloc.h into the omnet's include
directory, and it seems passing the compilation.
but i don't know how to test if it is actually working.

On Jan 13, 5:31 pm, Jan Gerbecks <jan.gerbe...@stud.uni-due.de> wrote:
> Thanks for all your suggestions.
>

> I actually usedhttp://alleyoop.sourceforge.net/usage.htmlwhich uses valgrind but provides a GUI as well.