Proper memory management in CFFI

446 views
Skip to first unread message

Israel Brewster

unread,
Dec 22, 2014, 2:24:59 PM12/22/14
to pytho...@googlegroups.com
In general, Python and CFFI are pretty good about memory management. Python itself, of course, is good about cleaning up after itself, and as stated in the CFFI docs, "the returned pointer object has ownership on the allocated memory: when this exact object is garbage-collected, then the memory is freed" However, I am running into a situation where this is not the case, probably due to the object not being "this exact object".

The c code I am using has a function that takes as one of its parameters a char **. During function execution, the deference of this char ** gets malloc'd. It's then up to the programer to free the value when they are done with it. So in straight C code, it would look something like this:


char *buffer;
function_call(...,&buffer,...) //mallocs buffer and fills it with data
....
//do whatever we want with buffer
free(buffer)

So translating this to CFFI, I get a construct like the following:

buffer=ffi.new("char*[1]")
my_c_lib.function_call(...,buffer,...)
# Use buffer[0]
# Free memory malloce'd in function_call at buffer[0]

But now I have a problem: CFFI does not automatically free buffer[0], perhaps because it is now a different object than what was created by the ffi.new call. And trying to call del(buffer[0]) just gives me an error. So how do I properly do this such that I can free the memory malloc'd by the C function?

Just as an aside, I do definitely need to free this memory. The call is in a loop that could potentially run hundreds of thousands of times, and in my testing when this loop is running I can watch the memory usage of the python process growing up to over a GB used. So I definitely have a leak here.
-------
Israel Brewster
Ravn Alaska

Armin Rigo

unread,
Dec 23, 2014, 3:14:04 AM12/23/14
to pytho...@googlegroups.com
Hi Israel,

On 22 December 2014 at 20:24, Israel Brewster <macav...@gmail.com> wrote:
> But now I have a problem: CFFI does not automatically free buffer[0],
> perhaps because it is now a different object than what was created by the
> ffi.new call. And trying to call del(buffer[0]) just gives me an error.

If in C you call "free(buffer)", then you need to add the ``void
free(void *);`` declaration to the cdef, and from Python you call
"my_c_lib.free(buffer[0])".


A bientôt,

Armin.

Israel Brewster

unread,
Dec 23, 2014, 1:56:37 PM12/23/14
to pytho...@googlegroups.com, ar...@tunes.org
Ah, of course - just expose whatever C functions I need to use to CFFI. I'm not limited to just functions in the specific library I am working with. Duh. Thanks!
-------
Israel
Reply all
Reply to author
Forward
0 new messages