BUG report: SEGV in `equalkey` function in `lua/ltable.c`

[Ye Liangchen]

Here is a SEGV in `equalkey` function in `lua/ltable.c`. The following shows the reproduction steps.

# Build Lua with AddressSanitizer

```diff

diff --git a/makefile b/makefile

index 7cfcbfe1..f13897f7 100644

--- a/makefile

+++ b/makefile

@@ -70,8 +70,8 @@ MYLDFLAGS= $(LOCAL) -Wl,-E

MYLIBS= -ldl -lreadline

-CC= gcc

-CFLAGS= -Wall -O2 $(MYCFLAGS) -fno-stack-protector -fno-common -march=native

+CC ?= gcc

+CFLAGS += -Wall $(MYCFLAGS) -fno-stack-protector -fno-common -march=native

AR= ar rc

RANLIB= ranlib

RM= rm -f

@@ -81,7 +81,7 @@ RM= rm -f

# == END OF USER SETTINGS. NO NEED TO CHANGE ANYTHING BELOW THIS LINE =========

-LIBS = -lm

+LIBS += -lm

CORE_T= liblua.a

CORE_O= lapi.o lcode.o lctype.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o \

@@ -111,7 +111,7 @@ $(CORE_T): $(CORE_O) $(AUX_O) $(LIB_O)

$(RANLIB) $@

$(LUA_T): $(LUA_O) $(CORE_T)

- $(CC) -o $@ $(MYLDFLAGS) $(LUA_O) $(CORE_T) $(LIBS) $(MYLIBS) $(DL)

+ $(CC) -o $@ $(LDFLAGS) $(MYLDFLAGS) $(LUA_O) $(CORE_T) $(LIBS) $(MYLIBS) $(DL)

clean:

```

```sh

CC=clang CXX=clang++ CFLAGS="-fsanitize=address -g" CXXFLAGS="-fsanitize=address -g" LDFLAGS="-fsanitize=address" make -j liblua.a

CC=clang CXX=clang++ CFLAGS="-fsanitize=address -g" CXXFLAGS="-fsanitize=address -g" LDFLAGS="-fsanitize=address" make -j lua

```

# PoC

```lua

local M = {}

local setmetatable, stderr, collectgarbage, debug =

setmetatable, io.stderr, collectgarbage, debug

_ENV = nil

local active = false

-- Set a line hook to modify L->hookmask

debug.sethook(function() end, "l")

-- Ensure the hook key table is present in the registry

debug.getregistry()["_HOOKKEY"] =true

local mt = {}

function mt.__gc (o)

stderr:write'.' -- mark progress

if active then

setmetatable(o, mt) -- remark object for finalization

end

end

function M.start ()

if not active then

active = true

setmetatable({}, mt) -- create initial object

end

end

function M.stop ()

if active then

active = false

collectgarbage() -- call finalizer for the last time

end

end

return 5

```

Run the PoC with the following command:

```sh

./lua poc.lua

```

ASAN report:

```

AddressSanitizer:DEADLYSIGNAL

=================================================================

==3888353==ERROR: AddressSanitizer: SEGV on unknown address 0x1001d6560480 (pc 0x000000527e3d bp 0x7fffffff8360 sp 0x7fffffff8260 T0)

==3888353==The signal is caused by a READ memory access.

#0 0x527e3d in equalkey lua/ltable.c:254:21

#1 0x523fe3 in getgeneric lua/ltable.c:284:9

#2 0x523f25 in luaH_get lua/ltable.c:1028:14

#3 0x4d7609 in lua_rawget lua/lapi.c:759:9

#4 0x5736fe in hookf lua/ldblib.c:331:7

#5 0x4e9f58 in luaD_hook lua/ldo.c:410:5

#6 0x4e4a99 in luaG_traceexec lua/ldebug.c:951:7

#7 0x53b1db in luaV_execute lua/lvm.c:1362:9

#8 0x4ecd28 in ccall lua/ldo.c:716:5

#9 0x4ecdc7 in luaD_callnoyield lua/ldo.c:734:3

#10 0x4dbbe0 in f_call lua/lapi.c:1065:3

#11 0x4e8097 in luaD_rawrunprotected lua/ldo.c:158:3

#12 0x4ee904 in luaD_pcall lua/ldo.c:1037:12

#13 0x4db3d3 in lua_pcallk lua/lapi.c:1091:14

#14 0x4d0220 in docall lua/lua.c:162:12

#15 0x4cff76 in handle_script lua/lua.c:266:14

#16 0x4ceb97 in pmain lua/lua.c:722:9

#17 0x4ebd89 in precallC lua/ldo.c:605:7

#18 0x4ec293 in luaD_precall lua/ldo.c:674:7

#19 0x4eccb2 in ccall lua/ldo.c:714:13

#20 0x4ecdc7 in luaD_callnoyield lua/ldo.c:734:3

#21 0x4dbbe0 in f_call lua/lapi.c:1065:3

#22 0x4e8097 in luaD_rawrunprotected lua/ldo.c:158:3

#23 0x4ee904 in luaD_pcall lua/ldo.c:1037:12

#24 0x4db3d3 in lua_pcallk lua/lapi.c:1091:14

#25 0x4ce53d in main lua/lua.c:750:12

#26 0x7ffff7ca7d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16

#27 0x7ffff7ca7e3f in __libc_start_main csu/../csu/libc-start.c:392:3

#28 0x422554 in _start (lua/lua+0x422554)

AddressSanitizer can not provide additional info.

SUMMARY: AddressSanitizer: SEGV lua/ltable.c:254:21 in equalkey

==3888353==ABORTING

```

[Roberto Ierusalimschy]

> Here is a SEGV in `equalkey` function in `lua/ltable.c`. The following
> shows the reproduction steps.
>

> [...]

> -- Ensure the hook key table is present in the registry
> debug.getregistry()["_HOOKKEY"] =true

Again, the bug is in your code; no bug in Lua.

Again, read the reference manual:

"You should exert care when using [the debug] library. Several
of its functions violate basic assumptions about Lua code (e.g., that
variables local to a function cannot be accessed from outside; that
userdata metatables cannot be changed by Lua code; that Lua programs do
not crash) and therefore can compromise otherwise secure code."

-- Roberto

[Ye Liangchen]

Hi. Thanks for the prompt response. 

> There is no bug here.

I disagree. Even though the test input is malformed, the program SHOULD NOT crash or cause security issues, which may incur spiteful attacks. The program SHOULD handle abnormal scenarios without incurring security issues, especially since it is a language interpreter.

[Sainan]

If you do not want the program to crash, disable the debug library. That's the first step you wanna do in sandboxing, and it's also what the manual recommends, as has been pointed out to you several times now.

-- Sainan

[Foster]

If you are using debug in a production environment you have far more problems going on.  

There seems to be a recent spate of reports all using the debug code.  As my favorite blogger Raymond Chen often says "It involved being on the other side of this airtight hatchway"

https://devblogs.microsoft.com/oldnewthing/20060508-22/?p=31283

I read the bug reports and I see "debug" I go "Huh, another of the thousands of ways I can crash Lua using the debugger" and hit delete.  

[Yao Zi]

On Sun, Feb 02, 2025 at 04:56:16AM -0800, Ye Liangchen wrote:
> Hi. Thanks for the prompt response.
>
> > There is no bug here.
> I disagree. Even though the test input is malformed, the program SHOULD NOT
> crash or cause security issues, which may incur spiteful attacks.

If you don't want some untrusted code to crash the interpreter,
disable the debug library. It's straight and simple in Lua.

> The program SHOULD handle abnormal scenarios without incurring
> security issues, especially since it is a language interpreter.

You could always crash a well-formed executable with a debugger
attached to it, should the case be handled well as well?

Please, stop fuzzing the debug library whose unsafety is
well-documented. There's no bug and no security concerns, crashing is
expected.

Thanks,
Yao Zi

[Roberto Ierusalimschy]

> You could always crash a well-formed executable with a debugger
> attached to it, should the case be handled well as well?

Brilliant :-)

-- Roberto

[rolf.kal...@kalbermatter.nl]

On Sunday, 2 February 2025 at 14:54:25 +01:00, 'Yao Zi' via lua-l <lu...@googlegroups.com> wrote:

You could always crash a well-formed executable with a debugger

attached to it, should the case be handled well as well?

One could argue that this can be attempted to be prevented by trying to detect an attached debugger and refuse starting up or simply shutting down! Some copy protection and license manager software tries to do that. But it is an unwinnable catch up game.

For every measure there will be eventually a countermeasure to get around it. The only truly safe computer system is one locked in a safe and dropped into the Mariana trench. Of course it's not very usable either but there will be nobody able to hack it for sure.

So yes, if your code uses the debug library, you can crash it and there is no way to prevent that, safe from disabling the debug library.

Rolf Kalbermatter

[Viacheslav Usov]

On Sat, Feb 1, 2025 at 9:04 PM Roberto Ierusalimschy
<rob...@inf.puc-rio.br> wrote:

> "You should exert care when using [the debug] library. Several
> of its functions violate basic assumptions about Lua code (e.g., that
> variables local to a function cannot be accessed from outside; that
> userdata metatables cannot be changed by Lua code; that Lua programs do
> not crash) and therefore can compromise otherwise secure code."

Like most other programmers, I have spent more time debugging than
"programming" in the strict sense of the word. And all this experience
tells me that the most useful kind of debugging is not destructive:
you usually just need to have a better idea of the current state
rather than modify the state. To that end, if nothing else is
available, printf-debugging usually is, but, curiously, it is very
destructive sort of debugging because the program has to be modified
and restarted to use it.

So it seems somewhat desirable to have limited but safe debugging
means. Naively, I would think that debug.get*() functions should be
safe because they only "get" some information. But this very thread
exists because somebody used debug.getregistry() and thereafter,
apparently, modified the obtained registry table. I suppose all the
other get*() functions should present the same problem. Which brings
me to this question: suppose the debug.get*() functions have been
modified so that they return only immutable data. Can they then be
considered safe? If not, why?

Cheers,
V.