Memory leak detected in protobuf
Jean-Sebastien Stoezel
I am observing a memory leak after including the protobufs in my
project. I would like to investigate whether this leak is due to this
library or the use I make of it.
I am using the protobufs (latest version) with VC++, running 3
threads.
2 server threads are pending, waiting for connections, another thread
acts as client, connecting to the servers and sending messages to
them.
I tested this code without the protobufs and I did not notice an
increase in memory usage over time. After adding the protobufs, I see
a constant increase in memory usage.
I instrumented the code with VC++ crtdbg library, the tool used to
detect memory leaks. The tool is reporting memory leaks which I
believe point to protobuf. I read the crtdbg documentation and if
_CRTDBG_MAP_ALLOC is defined, the tool should report the name of the
file where the leak happened. However no file name is provided by the
tool, it could be because I statically link to the protobufs.
Anyhow, from the server thread's context, protobufs are stored in a
vector. A packet delimiter parses data from a socket. Each packet is
parsed for a single protobuf message.
Pseudo code:
m_IncomingMessages is defined as
std::vector<LFLComProtocol::LFLComProtocolMsg> m_IncomingMessages>,
where LFLComProtocol::LFLComProtocolMsg is a protobuf message.
// adding a protobuf to the protobuf queue, after a packet of data was
received
// Add an element to the vector
m_IncomingMessages.resize(m_IncomingMessages.size() + 1);
// Parse the data from the new message
m_IncomingMessages[m_IncomingMessages.size() -
1].ParseFromArray(p_Data, dataSize);
// consuming the protobufs, FIFO style
m_IncomingMessages.erase(m_IncomingMessages.begin());
I have checked that the messages are properly queued and dequeued, the
size of the vector does not increase and stay constant (usually 0
since the message is consumed right away)
At this point I am thinking that this way of erasing may be causing
the leak. From the STL documentation I gather that the vector::erase
method is supposed to call the destructor for each element... Is this
good enough to free all the memory allocated for the message?
Any other idea why this memory leak is happening?
Jean-Sebastien
Henner Zeller
Can you create a compilable self contained C++ file including
proto-file (stripped down to the essentials) with the essence of your
code which would make it simpler (for you and people on the list) to
verify if there is a problem ?
Also, I would probably not use protocol buffers as values within
vectors (esp. if they're big - this advice is true for any bigger
object) because you implicitly might call copy constructors if you
resize() the vector and it has to realloc. Rather allocate protos on
the heap (new/delete). This allows for further improvements later,
like having a pool of pre-allocated protobuffers.
-h
>
>
> I have checked that the messages are properly queued and dequeued, the
> size of the vector does not increase and stay constant (usually 0
> since the message is consumed right away)
>
> At this point I am thinking that this way of erasing may be causing
> the leak. From the STL documentation I gather that the vector::erase
> method is supposed to call the destructor for each element... Is this
> good enough to free all the memory allocated for the message?
>
> Any other idea why this memory leak is happening?
>
> Jean-Sebastien
>
>
> --
> You received this message because you are subscribed to the Google Groups "Protocol Buffers" group.
> To post to this group, send email to prot...@googlegroups.com.
> To unsubscribe from this group, send email to protobuf+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
>
>
