Lists of userdata in C code possible?

[bil til]

... just in design of a new lib - I use the liolib.c with the FILE
objects as blueprint for writing my C code.

So I design an object similar to the FILE object in C. The Lua user
will be able to create such objects with an open function. My C code
from time to time would like to iterate over all created objects... .

The obvious / most simple way in C would be to add a pNext pointer
into the userdata of object. (e.g. for FILE, this would mean to add
pNext Pointer to the FILE struct in C...).

Is this valid? I am a bit frightened about garbage collection / gc ...
not that gc at some time would decide to reorganize my objects in
memory, so then all the pNext pointers would get invalid? (or is gc
instructed not touch userdata objects during lifetime?).

Or is there some other for sure safe way, to list all my userdata
objects? (e. g. is there some way for C code to run through all
created FILE objects?).

[Sainan]

You could use a "shared pointer" to keep your own refcounting and just have the Lua instance constitute one such reference. You can use the __gc metamethod to handle refcount decrementing / cleanup as needed.

-- Sainan

[bil til]

thanks for fast answer, but maybe misunderstanding?:

I do not need the refcounting. My C code definitely knows, when my
object has been closed / has become invalid.

Just I want to be sure, that the allocated memory buffer for the
opened object (mem pointer returned by lua_newuserdatauv) will not
change during lifetime of my object. Then all fine.

... if this is NOT guaranteed, then I would need some way to get the
mem pointers of all currently active objects... .

[Sainan]

I see, I think I misunderstood your concern, but no, I don't think the GC would do anything 'insane' to your userdata, only to free it which would be preceeded by a __gc call.

-- Sainan

[bil til]

Thanks, this sounds great.

Perhaps I am too frightened of buerocracy :)... .

Maybe Lua should spend a short introduction in the GC chapter of the
ref man, that GC is a nice and fast fox, just taking away garbage and
not entering the houses and enforcing compression :).

[Gé Weijers]

On Fri, Oct 31, 2025 at 9:33 PM bil til <bilt...@gmail.com> wrote:

.[...]

The obvious / most simple way in C would be to add a pNext pointer
into the userdata of object. (e.g. for FILE, this would mean to add
pNext Pointer to the FILE struct in C...).

This won't work reliably, because pNext could point to a 'file' that's not reachable otherwise, and the garbage collector would be unaware of the reference and collect the object while you are iterating through the list. The other problem is that when you iterate you'll have C pointers to objects, when what you need is a Lua userdata value to return to the Lua side of your code.

The best method I can think of is to create a table with weak keys when the C module is initialized. When a 'file' object is created you add it to the table, the value does not matter ('true' would work) This weak reference will not keep the object from being collected, but you can iterate over the table at any time, using 'pairs' or 'next' or your own iterator.

When you close the 'file' explicitly you should remove it from the table immediately, which will stop the iterator from returning closed files. It's legal to remove entries from a table while you are iterating over it, you just should not add entries (i.e. open 'files') while doing that. 

If you don't explicitly close the 'file' then when it becomes inaccessible except through the table the garbage collector will remove the entry from the table before it calls __gc, so the iterator will not return zombie objects.

--

Gé

[bil til]

>pNext could point to a 'file' that's not reachable otherwise, and the garbage collector would be unaware of the reference and collect the object while you are iterating through the list

... the FILE objects were only meant as example.

My own objects will use the _gc meta function and will automatically
be removed from the list if _gc is invoked. (removing objects from a
list is a very fast operation, you just have to modify the pNext
pointer pointing to the object in deletion - here must be exactly ONE
such pNext, otherwise somethng wrong in the list...).

So I hope with this list removal in _gc then all will work fine for
sure? Or do you still see any issue?

[Gé Weijers]

On Sat, Nov 1, 2025 at 9:25 AM bil til <bilt...@gmail.com> wrote:

So I hope with this list removal in _gc then all will work fine for
sure? Or do you still see any issue?

I don't see any issue with removing an element from a list, but I don't think the pNext pointer is useful for iteration. You can convert a Lua userdata reference on the stack into a C pointer to the data, but the reverse is not possible, so when you traverse the list and find an object that meets your criteria you can't pass it to your Lua code. 

C pointers to userdata objects are not generally useful unless you have some way to look up the Lua object reference, which typically is a Lua table from the pointer/light userdata to the full userdata. Once you have that you might as well use that table for iteration.

--

Gé

[bil til]

>when you traverse the list and find an object that meets your criteria you can't pass it to your Lua code

... this is clear to me - I do not need this "object-touring" for Lua
interactions.

In my system the Lua metaclasses are c++ class contructs.

I need such "object -touring" to check the state of my objects, e. g.
if a new object is opened by the Lua code, I have to check for any
conflicts with exisiting objects... .