Use after free in luaV_finishset

[Xmilia Hermit]

The luaV_finishset function does not pin the h table prior to calling
luaH_finishset(L, h, key, val, hres).
In the case where luaH_finishset might do a rehash which triggers an
emergency GC this table could be collected when it is only weakly
referenced resulting in a use after free.

The following code will trigger this issue with the help of the tests
allocfailnext function to trigger the emergency GC which collects the
index table.

local index = {}
local mt = setmetatable({__newindex=index}, {__mode='v'})
local t = setmetatable({}, mt)
index = nil -- Allow collection of the index table

T.allocfailnext()

for i=1, 10000 do
t[i] = 1
end

Regards,
Xmilia

[Roberto Ierusalimschy]

> The luaV_finishset function does not pin the h table prior to calling
> luaH_finishset(L, h, key, val, hres).
> In the case where luaH_finishset might do a rehash which triggers an
> emergency GC this table could be collected when it is only weakly
> referenced resulting in a use after free.

Independently of this particular bug per se, it raises an interesting
point on whether weak tables should be cleared during an emergency
GC. By the same idea that an emergency GC does not call finalizers, it
probably should not clear weak tables, too.

-- Roberto