cgo memory fragmetation?

[miha.v...@gmail.com]

Hi guys,

We are having a very weird problem, where the process memory keeps rising, but both the pprof and valgrind report no memory leaks. But the RSS just keeps rising.The C part is in external so library

To be more exact, the pprof when the process becomes idle reports a few megabytes used and the Valgrind reports 88bytes lost(this are global objects) But the RSS for example is ~1G and virtual 4G+

I've run also run the app with the GODEBUG=madvdontneed=1,gctrace=1 but madvdontneed doesn't seem to help.

If I describe the MO of communicating with the library's API (all memory is allocated on the library side).

for {

  * initialize the processor

  * say to the library to create the blob of size x on it's side

  * convert the returned pointer to a fake slice (code below)

  * copy data to the fake slice

  * instruct processor to do it's thing (e.g make calculations/compress the data,...)

  * process the data on Go side (depending on the processor this either makes a copy of the data or creates a fake slice from it)

  * destroy the processor (this frees the memory on the library's side)

  * upload the data somewhere

}

func cVoidPtrToByteSlice(data unsafe.Pointer, size int, bytes *[]byte) {
    header := (*reflect.SliceHeader)(unsafe.Pointer(bytes))
    header.Data = uintptr(data)
    header.Len = size
    header.Cap = size
}

BR,

Miha

[Jamil Djadala]

Hi, can you try to run your program with GOGC=10 environment ?

--
Jamil Djadala

[miha.v...@gmail.com]

This doesn't seem to help...

gc 589 @171.013s 0%: 0.012+0.69+0.17 ms clock, 0.41+0.079/2.9/1.6+5.5 ms cpu, 6->6->5 MB, 7 MB goal, 32 P
scvg: 0 MB released
scvg: inuse: 7, idle: 54, sys: 61, released: 54, consumed: 7 (MB)
scvg: inuse: 7, idle: 54, sys: 61, released: 54, consumed: 7 (MB)
scvg: inuse: 7, idle: 54, sys: 61, released: 54, consumed: 7 (MB)
scvg: inuse: 7, idle: 54, sys: 61, released: 54, consumed: 7 (MB)
scvg: inuse: 7, idle: 54, sys: 61, released: 54, consumed: 7 (MB)
scvg: inuse: 7, idle: 54, sys: 61, released: 54, consumed: 7 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 52, consumed: 8 (MB)
gc 590 @171.018s 0%: 0.25+0.89+0.020 ms clock, 8.2+0.28/3.8/2.6+0.65 ms cpu, 7->7->6 MB, 8 MB goal, 32 P
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)
scvg: 0 MB released
scvg: inuse: 8, idle: 53, sys: 61, released: 53, consumed: 8 (MB)

The ps shows 3674660 826844 (VSZ RSS)

BR,

Miha

[Ian Lance Taylor]

On Mon, Oct 7, 2019 at 12:54 AM <miha.v...@gmail.com> wrote:
>
> We are having a very weird problem, where the process memory keeps rising, but both the pprof and valgrind report no memory leaks. But the RSS just keeps rising.The C part is in external so library

In the absence of other memory pressure on the system as a whole,
increasing RSS just means that your program is using different
addresses for allocated memory. This isn't necessarily unexpected.
If overall memory usage is not increasing, there may not be a problem
here.

Ian

[miha.v...@gmail.com]

It's not that. The system had 4G of memory initially as it is supposed to be enough for 18 processors (They should max out at around 250M but have the typical of about 100M) and it started swapping heavily. (I then raised the limits to 16G) so things can continue to run.

BR,

Miha