`Assertion `0' failed.` in lua_getmetatable

[jy l]

Hi! We found an assertion failure crash when testing with the fuzzing driver luaL_dostring_test

PoC:

```

weak={}
setmetatable(weak,{})
for i=0,1000 do
    weak[i]=0
end

local si0e=#weak
do e=0
end
for i=0,si0e do
    setmetatable(weak,{__mode="kv"})
    do weak[i+si0e]=0
    end
end

```

Backtrace:

```

luaL_dostring_test: lapi.c:817: int lua_getmetatable(lua_State *, int): Assertion `0' failed.
==2740207== ERROR: libFuzzer: deadly signal
...
    #11 0x7f35a49f5e95 in __assert_fail assert/assert.c:103:3
    #12 0x4e7e94 in lua_getmetatable /src/testdir/build/lua-master/source/lapi.c:817:5
    #13 0x588416 in luaL_getmetafield /src/testdir/build/lua-master/source/lauxlib.c:885:8
    #14 0x5b075d in luaB_setmetatable /src/testdir/build/lua-master/source/lbaselib.c:143:7
    #15 0x503020 in precallC /src/testdir/build/lua-master/source/ldo.c:612:7
    #16 0x504219 in luaD_precall /src/testdir/build/lua-master/source/ldo.c
    #17 0x5736f6 in luaV_execute /src/testdir/build/lua-master/source/lvm.c:1696:22
    #18 0x5049e2 in ccall /src/testdir/build/lua-master/source/ldo.c:723:5
    #19 0x5049e2 in luaD_callnoyield /src/testdir/build/lua-master/source/ldo.c:741:3
    #20 0x4fea08 in luaD_rawrunprotected /src/testdir/build/lua-master/source/ldo.c:148:3
    #21 0x506b77 in luaD_pcall /src/testdir/build/lua-master/source/ldo.c:1045:12
    #22 0x4eda7d in lua_pcallk /src/testdir/build/lua-master/source/lapi.c:1091:14
    #23 0x4dbcfe in LLVMFuzzerTestOneInput /src/testdir/tests/capi/luaL_dostring_test.c:37:2

```

Version: 0508724910b252010ae5f6bcaef0e54cf198cf5b 2025-06-02 11:54:30+0300

[Roberto Ierusalimschy]

> Hi! We found an assertion failure crash when testing with the fuzzing
> driver luaL_dostring_test
>
> PoC:
> ```
> weak={}
> setmetatable(weak,{})
> for i=0,1000 do
> weak[i]=0
> end
>
> local si0e=#weak
> do e=0
> end
> for i=0,si0e do
> setmetatable(weak,{__mode="kv"})
> do weak[i+si0e]=0
> end
> end
> ```
>
> Backtrace:
> ```
> luaL_dostring_test: lapi.c:817: int lua_getmetatable(lua_State *, int):
> Assertion `0' failed.
> ==2740207== ERROR: libFuzzer: deadly signal

> [...]

>
> Version: 0508724910b252010ae5f6bcaef0e54cf198cf5b 2025-06-02 11:54:30+0300

Many thanks for the feedback. This bug has been lurking in Lua for at
least 10 years...

(Please, try to refer to the Lua repo when reporting errors I almost
dismissed that message, since Lua neither have anything called
luaL_dostring_test nor a commit 0508724910.)


-- Roberto

[Roberto Ierusalimschy]

> > Hi! We found an assertion failure crash when testing with the fuzzing
> > driver luaL_dostring_test
> >

> [...]

>
> Many thanks for the feedback. This bug has been lurking in Lua for at
> least 10 years...

The bug is in the garbage collector. After visiting an all-weak table,
the collector dismisses the table, as it has neither keys nor values to be
traversed. But it has a metatable, which can change (although it seldom
does).

-- Roberto