It's possible to replace cgo memory allocator?

1,259 views
Skip to first unread message

Diego Bernardes

unread,
Jun 10, 2021, 3:19:15 PM6/10/21
to golang-nuts
I'm using cgo and struggling to find a memory leak which can be easily detected if I can replace the cgo memory allocator to the one the C library I'm using have because they did a leak detector implementation there.

It's possible?


Howard C. Shaw III

unread,
Jun 10, 2021, 3:53:50 PM6/10/21
to golang-nuts
The DGraph.io team has a blog post about swapping to using jemalloc for CGo (they are going further and feeding their allocations into Go to avoid garbage collection, but that is a separate issue). 

Ian Lance Taylor

unread,
Jun 11, 2021, 5:02:30 PM6/11/21
to Diego Bernardes, golang-nuts
What do you mean by "the cgo memory allocator?" Calls to, for
example, C.malloc should already be using the same allocator that your
C library uses.

Ian

Tamás Gulácsi

unread,
Jun 12, 2021, 5:40:22 AM6/12/21
to golang-nuts
jemalloc or tcmalloc, and debug with them.

Diego Bernardes

unread,
Jun 18, 2021, 3:52:39 PM6/18/21
to golang-nuts
Sorry for the late reply. The problem was solved, thanks to everyone for the help.

Just to give a little more context. The lib that I'm using is MuPDF and I thought that I had a memory leak but in the end, it was just glibc not releasing the memory fast enough to the OS. I solved the problem by changing the allocator at MuPDF to jemalloc. At my first analysis, I did not know where was the problem because the code looked correct, as it was, but I started to think that maybe I was leaking the memory in C from the Go code calling things like C.CString. Then I thought how can I allocate in C from Go utility functions like C.CString using an allocator that has leak detection called memento that the MuPDF team built? Well, I still don't know. I did it once using unsafe, but not by using the actual C.CString call. And regards LD_PRELOAD I did not found a way of making everything work together, I don't have much experience with C.
Reply all
Reply to author
Forward
0 new messages