An assertion is triggered in lgc.c:freeobj()

[Sergey Bronnikov]

Hello!

The commit a4762b6ffe74f5878882ef238d37bfa92d90e418 ("'objsize' returns 'l_mem'")  has introduced regression - an assertion is triggered in a function lgc.c:freeobj().

How to reproduce:

Enable compilation flag LUAI_ASSERT:

--- a/makefile
+++ b/makefile
@@ -71,7 +71,7 @@ LOCAL = $(TESTS) $(CWARNS)
 # To enable Linux goodies, -DLUA_USE_LINUX
 # For C89, "-std=c89 -DLUA_USE_C89"
 # Note that Linux/Posix options are not compatible with C89
-MYCFLAGS= $(LOCAL) -std=c99 -DLUA_USE_LINUX
+MYCFLAGS= -DLUAI_ASSERT $(LOCAL) -std=c99 -DLUA_USE_LINUX
 MYLDFLAGS= $(LOCAL) -Wl,-E
 MYLIBS= -ldl
 

Build Lua by executing make and run:

$ ./lua -e "
> local function func() func() end
> coroutine.resume(coroutine.create(func))"
lua: lgc.c:861: void freeobj(lua_State *, GCObject *): Assertion `(((L->l_G))->GCtotalbytes - ((L->l_G))->GCdebt) == newmem' failed.
Aborted (core dumped)

Sergey

[Roberto Ierusalimschy]

> The commit a4762b6ffe74f5878882ef238d37bfa92d90e418 ("'objsize' returns
> 'l_mem'") has introduced regression - an assertion is triggered in a
> function lgc.c:freeobj().
>

> [...]

Indeed. Many thanks for the feedback.

-- Roberto

[Roberto Ierusalimschy]

The problem is that the counter L->nci, which is an unsigned short, can
overflow, as Lua can have way more than 65,000 nested calls. Actually,
this "bug" is present since Lua 5.4.2, but those versions don't use nci
for anything, so the bug has no consequences there. (Is that still a bug?)

-- Roberto