proto messages allocated even if not anymore referenced

[Belisko Marek]

Hi,

I'm using protobuf in golang programming language.Currently I'm hunting for some memory issue and used pprof tool to check heap usage of my program.

Proto message is created in function and returned then in main neverending loop is sent to channel to sent data.

I create heap snapshot using pprof tool and looks like even memory to message is not referenced buffer of message size is still occupied (I used message payload like ~10M to be more visible).

code snippet:

func createMessage([]byte buff) []byte {

   m := &protocol.Message{

Application: "1",

Payload:       buff,

}

  s, err := proto.Marshal(m)

  return m

}

func main() {

   ....

   for {

     select {

     case buff := receivedData:

         p := createMessage(buff)

         dataTosend <- p

     }

   }

}

Am I doing something wrong here? Program work fine but when using payload like 100M (on embedded device) I'm getting after few attempts out of memory issue.

Thanks.

BR,

marek

[Joe Tsai]

Hi Belisko,

The code snippets seems insufficient to diagnose any form of memory leak. For example, what happens to the s variable in createMessage? Is the dataTosend variable a buffered channel? What's consuming the other end of the channel and is it keeping up with the transmission rate? There's a lot of factors (most of which have little to nothing to do with protocol buffers) that can affect a lot of memory being held on the heap.

Memory leaks can be notoriously hard to diagnose. It might help to see how others tackled the problem: https://blog.detectify.com/2019/09/05/how-we-tracked-down-a-memory-leak-in-one-of-our-go-microservices/

JT

[Belisko Marek]

Hi JT,

more info:

there is type in method createMessage variabel s is returned and no m. dataToSend is nont buffered channel. Other side is basically usb driver which consume buffer and send it over wire to other party. I know pprof can be false posituve also but what puzzles me that is shows that 100MB buffer is still allocated even if buffer was sent to other device without issue. Also thanks for link.

BR,

marek

Dňa nedeľa, 31. mája 2020 1:18:00 UTC+2 Joe Tsai napísal(a):