Memory owning and pointers/arrays

[Floris Bruynooghe]

Hi,

When allocating memory using ffi.new the cdata object returned owns the memory:

procent_list = ffi.new('struct procentry64[1]')

Now procent_list is a cdata object which owns the allocated memory.
However when accessing the first item the new cdata object does not
keep the memory alive:

procent = procent_list[0]
del procent_list # dealocates memory
procent.pi_pid # random stuff

This seems all fine and logical, the thing which tripped me up however
was the different behaviour when the initial ffi.new call used a
pointer instead:

procent_list = ffi.new('struct procentry64*')
procent = procent_list[0]
del procent_list
procent.pi_pid # all fine

Given how cffi kind of treats the 1-item array and pointer the same on
the surface of it (both allocate the space for one item which is not
the same as C does and both need indexing with [0] to get the item)
this sudden change in how memory is owned seems rather surprising.
And while the documentation does mention this it is not the most
obvious to find.

Is there any reason why the cdata items of an array do not keep the
whole array alive? Would that not make the API more consistent?


Regards,
Floris



--
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org

[Armin Rigo]

Hi Floris,

On 31 March 2014 12:37, Floris Bruynooghe <fl...@devork.be> wrote:
> Is there any reason why the cdata items of an array do not keep the
> whole array alive? Would that not make the API more consistent?

No, the inconsistency here is the behavior of the other case. If we
go down the road of keeping more and more things alive, we end up very
quickly with a ctypes-like result: it's nice and keeps things alive
for you in 90% of the cases, and you crash unexpectedly in the
remaining 10%. Better to never go there imho.

Originally the behavior was not to keep the memory alive either in any
of the cases you describe, and I'm thinking about returning to the old
behavior. The obscure reason is for callbacks that return a structure
(not a pointer to a structure); that's the only case where we want to
be able to return "p[0]" after p goes out of scope. If we can find
another way to do that naturally, then we might remove this special
case...


A bientôt,

Armin.

[Floris Bruynooghe]

Fair enough, seems like some good reasons. Yes, avoiding the
exception all together would probably be more desirable in that case
but I have no suggestion on how to solve the callback issue. So I'll
just stop relying on this exception then.

Thanks,
Floris