null pointer exception in luaC_checkfinalizer

[Edward Gynt]

Hi all,

I am new here but an experienced lua user (both the C side and the lua syntax). I am using the cffi-lua library (https://github.com/q66/cffi-lua) in lua 5.4.6

Sometimes, but not always, the cffi.new() call (which calls luaL_setmetatable to set a userdata metatable with a __gc method as the meta table), triggers a null pointer exception in luaC_checkfinalizer (which is called by lua_setmetatable). Also see the call stack at the bottom of this message.

It happens on the line (see snippet below for the whole function body).

    for (p = &g->allgc; *p != o; p = &(*p)->next) { /* empty */ }

Which as I understand it, implies no reference to o was found inside allgc. What went wrong here?

Best of wishes,

Gynt

/*
** if object 'o' has a finalizer, remove it from 'allgc' list (must
** search the list to find it) and link it in 'finobj' list.
*/
void luaC_checkfinalizer (lua_State *L, GCObject *o, Table *mt) {
  global_State *g = G(L);
  if (tofinalize(o) ||                 /* obj. is already marked... */
      gfasttm(g, mt, TM_GC) == NULL ||    /* or has no finalizer... */
      (g->gcstp & GCSTPCLS))                   /* or closing state? */
    return;  /* nothing to be done */
  else {  /* move 'o' to 'finobj' list */
    GCObject **p;
    if (issweepphase(g)) {
      makewhite(g, o);  /* "sweep" object 'o' */
      if (g->sweepgc == &o->next)  /* should not remove 'sweepgc' object */
        g->sweepgc = sweeptolive(L, g->sweepgc);  /* change 'sweepgc' */
    }
    else
      correctpointers(g, o);
    /* search for pointer pointing to 'o' */
    for (p = &g->allgc; *p != o; p = &(*p)->next) { /* empty */ }
    *p = o->next;  /* remove 'o' from 'allgc' list */
    o->next = g->finobj;  /* link it in 'finobj' list */
    g->finobj = o;
    l_setbit(o->marked, FINALIZEDBIT);  /* mark it as such */
  }
}

Call stack:

  lua.dll!luaC_checkfinalizer(lua_State * L, GCObject * o, Table * mt) Line 1033 C
  lua.dll!lua_setmetatable(lua_State * L, int objindex) Line 958 C
  lua.dll!luaL_setmetatable(lua_State * L, const char * tname) Line 326 C
  cffi.dll!lua::mark_cdata(lua_State * L) Line 123 C++
  cffi.dll!ffi::newcdata(lua_State * L, const ast::c_type & tp, unsigned int vals) Line 234 C++
  cffi.dll!ffi::make_cdata(lua_State * L, const ast::c_type & decl, int rule, int idx) Line 1614 C++
> cffi.dll!ffi_module::new_f(lua_State * L) Line 1097 C++

[bil til]

Am So., 14. Sept. 2025 um 19:14 Uhr schrieb Edward Gynt <edwar...@gmail.com>:
>
> It happens on the line (see snippet below for the whole function body).
> for (p = &g->allgc; *p != o; p = &(*p)->next) { /* empty */ }
> Which as I understand it, implies no reference to o was found inside allgc. What went wrong here?

... this looks like typical C pointer problem - if p comes to 0 at
some piont of the loop "p=&(*p)->next", then on start of next loop *p
will give null pointer assignemnt. This is a clear C programming
error, which in no way should happen... . Change this line best to

for (p = &g->allgc; p && *p != o; p = &(*p)->next) { /* empty */ }

Then you should not get null pointer error any more... .

[bil til]

Can you specify the File, where this code is in? Is it in the original
Lua code (which version of Lua?), or is it in the github code of your
lib?

Am So., 14. Sept. 2025 um 19:14 Uhr schrieb Edward Gynt <edwar...@gmail.com>:
>

> --
> You received this message because you are subscribed to the Google Groups "lua-l" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/lua-l/df779650-e440-4cd5-a4dd-146a95727a1dn%40googlegroups.com.

[Edward Gynt]

The file is lgc.c and is lua 5.4.6. Anyway the lua 5.4.8 function looks identical to me.

https://github.com/lua/lua/blob/9ea06e61f20ae34974226074fc6123dbb54a07c2/lgc.c#L1068

What I am wondering though, maybe it is written like this because the null pointer should never happen, but to decide whether that is true my lua gc internals knowledge is lacking.

Op ma 15 sep 2025, 08:13 schreef bil til <bilt...@gmail.com>:

You received this message because you are subscribed to a topic in the Google Groups "lua-l" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lua-l/fbf9INyhSAQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lua-l+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/lua-l/CAOnDXmFGKjPNw%2BBD%2BshHEBVQbVxWa7QqMPfGgkvVJ-sK3mWeFA%40mail.gmail.com.

[Sainan]

> for (p = &g->allgc; p && *p != o; p = &(*p)->next) { /* empty */ }

This alone wouldn't fix the issue because it would just end the for loop and go right to writing to it (*p = o->next;), also causing a segfault.

I think there may be some information missing here as the assumption is that the object should still be linked in 'allgc' due to lack of FINALIZEDBIT but we reached the end of the linked list before finding it.

We'd have to know what exactly caused this assumption to no longer hold, which could just as well be due to an out-of-bounds write in one of the libraries you are interacting with via FFI.

-- Sainan

[Edward Gynt]

Dear Sainan,

I am also having an Access Violation issue with the cffi library (logged here: https://github.com/q66/cffi-lua/issues/62). And a heap corruption issue in lua that only occurs sporadically (I am using Page Heap to find that bug https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/example-12---using-page-heap-verification-to-find-a-bug?source=recommendations).

What makes you say it could be an out-of-bounds write and how I would I track down whether that is the case?

Best of wishes,

Gynt

Op ma 15 sep 2025, 09:14 schreef 'Sainan' via lua-l <lu...@googlegroups.com>:

--
You received this message because you are subscribed to a topic in the Google Groups "lua-l" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lua-l/fbf9INyhSAQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lua-l+un...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/lua-l/jARIsWbMmWCa-HlEyiKBRCUzLNNll2mpEUHZBsaViYBsJnm0uzwLChU0E9qTDTm0wHXMs5rMHnnBr7OsPvIm8IJ-OLd6b1B0oxl46sbeXlU%3D%40calamity.inc.

[Sainan]

> What makes you say it could be an out-of-bounds write and how I would I track down whether that is the case?

Well, given the way userdata is allocated, there's a bit of header data from Lua itself, and then there's your actual 'userdata memory', so an out-of-bounds that goes a bit before your memory start would overwrite Lua's headers. And also if you have multiple userdata of the same size, it's not unlikely the OS allocator puts them after the other, so going beyond the end of the allocated memory, could overwrite another userdata's header.

Tracking it down may be a bit harder. Valgrind could help, but might also not have coverage for the first case I described where you are technically in-bounds of the OS memory allocation, but not in-bounds for Lua's sake. Otherwise, just write some code that periodically checks the linked list for sanity and call this sanity check in a lot of places to narrow it down.

-- Sainan

[Sainan]

> checks the linked list for sanity

I should clarify that in this case I mean the 'finobj' linked list. All GCObjects within it should have the FINALIZEDBIT flag based on my understanding of Lua's GC.

-- Sainan