c++ compatibility issue

[eugeny gladkih]

Hi,

please make Lua more C++ compatible, catch only your own exceptions in LUAI_TRY macro not all. the smallest patch for 5.4.6 is follow

--- ldo.c.orig 2024-01-01 16:41:57
+++ ldo.c 2024-01-01 16:44:29
@@ -55,7 +55,7 @@
#if defined(__cplusplus) && !defined(LUA_USE_LONGJMP) /* { */

/* C++ exceptions */
-#define LUAI_THROW(L,c) throw(c)
+#define LUAI_THROW(L,c) throw((const lua_longjmp*)c)
#define LUAI_TRY(L,c,a) \
- try { a } catch(...) { if ((c)->status == 0) (c)->status = -1; }
+ try { a } catch(const lua_longjmp*) { if ((c)->status == 0) (c)->status = -1; }
#define luai_jmpbuf int /* dummy variable */

#elif defined(LUA_USE_POSIX) /* }{ */
@@ -1020,5 +1020,3 @@
decnny(L);
return status;
}

and, we’ll have another advantage making struct lua_longjmp publicly visible in lua.h or somewhere else

--
Yours sincerely, Eugeny.

[Viacheslav Usov]

> On Jan 1, 2024, at 15:13, 'eugeny gladkih' via lua-l <lu...@googlegroups.com> wrote:
> please make Lua more C++ compatible, catch only your own exceptions in LUAI_TRY

LUAI_TRY is a barrier that Lua expects will prevent further unwinding of its callstack. Your proposal trivially violates the expectation.

C++ should not throw any exceptions at Lua.

Unfortunately, the Lua error mechanism, while powerful, is mutually dangerous with third-party C or C++ code. Even though the manual does disclose the error mechanism, from which the danger may be inferred, it does not explain the kind of defensive coding required to play it safe.

Cheers,
V.

[Luiz Henrique de Figueiredo]

Viacheslav Usov> Unfortunately, the Lua error mechanism, while

powerful, is mutually dangerous with third-party C or C++ code. Even
though the manual does disclose the error mechanism, from which the
danger may be inferred, it does not explain the kind of defensive
coding required to play it safe.

Perhaps you could expand and explain this. Thanks.

[eugeny gladkih]

Lua code is very clean for ages, so it does not violate anything. It’s safe and stable.

--
Yours sincerely, Eugeny.

[Viacheslav Usov]

This is what the manual says:

Internally, Lua uses the C longjmp facility to handle errors. (Lua
will use exceptions if you compile it as C++; search for LUAI_THROW in
the source code for details.)

It says even less in another place:

Internally, Lua uses the C longjmp facility to yield a coroutine.

While in fact exactly the same mechanism is used in both cases (at
least in recent versions), with the same implication when compiling as
C++.

1. If Lua is compiled as plain C and used with a C++ library or host,
raising a Lua error (or yielding, also in what follows) might result
in undefined behavior (if it unwinds any stack frame with a C++ object
that has a non-trivial destructor). It is tempting to insist that the
manual ought to say "whenever Lua is likely to be used with C++
libraries or hosts, it should be compiled as C++ as a rule", but we
have #3 below.

2. If Lua is compiled as C++ and used with (solely) C++ libs and
hosts, then, generally speaking, the code that calls any error-raising
Lua API must be exception safe (in the C++ sense). While that might
seem obvious, it is also worth noting in the manual. Even though
writing exception-safe code in C++ is not too difficult, this does
have a cost, both in terms of dev effort and runtime performance.

3. If Lua is compiled as C++ and used with plain C libs or hosts,
calling error-raising Lua API is at best implementation-defined
behavior when it involves propagating a C++ exception through (and
out) plain C code. If the libs or host or Lua are compiled
independently, this is a problem without a solution (except by a lucky
chance). This is especially nasty when both C and C++ libs need to be
used.

4. If Lua is compiled as plain C and used with (solely) plain C libs
and hosts, the code that calls any error-raising Lua API must be
"exception safe", except that the language provides no means for that.
I suppose the Lua stack could be used for that purpose, but even then
it is not too easy. Consider this:

char *p = malloc(length);
/* construct some string at p */
lua_pushstring(L, p);
free(p);

This code is unsafe, because lua_pushstring(p) may raise an error (out
of memory), which leaks the allocated buffer. One way to solve this
is:

char *p = lua_newuserdatauv(L, length, 0);
/* construct some string at p */
lua_pushstring(L, p);
lua_remove(L, -2);

I won't say anything about the elegance of that, but we now have a
complication, if we originally wanted to save p and use it internally.
We might use malloc() again to create a clone, but things get more
complicated when one needs to allocate userdata with nested pointers
to some other memory allocations.

Probably a more robust alternative to the above, which could deal with
those complications, is this:

/* m is a metatable with __gc that calls free(), all omitted for brevity */
char *p, **q = lua_newuserdatauv(L, sizeof p, 0);
lua_geti(L, LUA_REGISTRYINDEX, m);
lua_setmetatable(L, -2);
*q = p = malloc(length);
/* construct some string at p */

Now we can do *q = 0 and detach the lifetime of the allocated memory
from the user datum, or we could keep it and, in case we have some
master user datum that references this memory, make it its user value.
But note the amount of code required for all that, and that is after
half the code has been omitted.

I'd think the manual should say something about these complications
and offer a way to deal with them.

Then we might have a C/C++ library or host calling longjmp() or
throwing an exception at Lua. The manual is completely silent about
that. In my previous message I suggested that this should never be
done, which is the safest option, but having some clarity about that
would not hurt. In principle, the same four cases might be considered
in this regard. I’d think the implementers of Lua should be in a
better position to do this.

Cheers,
V.

[ellie]

On 1/3/24 2:41 PM, Viacheslav Usov wrote:
> I'd think the manual should say something about these complications
> and offer a way to deal with them.

For what it's worth, I've tried to manually write safe alternatives for
all the lua_push* functions at some point, helped in part with lua
mailing list advice, that don't ever trigger setjmp() errors. Due to
memory leaks problem. Sadly, I couldn't figure it out in the end.

I think it would be a great addition if official lua_trypush* functions
that safely return an error instead of setjmp() would be added as
alternative for all the push functions, to address this. They would be a
way better fit inside a lot of C/C++ code, in my personal opinion.

[Roberto Ierusalimschy]

> I think it would be a great addition if official lua_trypush* functions
> that safely return an error instead of setjmp() would be added as
> alternative for all the push functions, to address this. They would be a
> way better fit inside a lot of C/C++ code, in my personal opinion.

For what it's worth, the following should do that for strings:
-----------------------------------------------
typedef struct {
const char *s;
size_t len;
} trypushlstring_t;

static int trypushlstring_aux (lua_State *L) {
trypushlstring_t *d = (trypushlstring_t*)lua_touserdata(L, 1);
lua_pushlstring(L, d->s, d->len);
return 1;
}

int trypushlstring (lua_State *L, const char *s, size_t len) {
trypushlstring_t d;
// either it assumes two empty slots or it does this check:
// if (!lua_checkstack(L, 2)) return 0; /* no stack space */
d.s = s; d.len = len;
lua_pushcfunction(L, trypushlstring_aux);
lua_pushlightuserdata(L, (void*)&d);
return lua_pcall(L, 1, 1, 0);
}
-----------------------------------------------
It's not particularly efficient, but it should be safe.

-- Roberto

[actboy168]

throw will unwinding callstack, it will be much slower than longjmp. Unfortunately, LUAI_THROW is not only used to raise errors, but also used to yield coroutines. The performance of using throw is not acceptable for code that heavily uses coroutines.

[David Sicilia]

Unfortunately, the Lua error mechanism, while
powerful, is mutually dangerous with third-party C or C++ code. Even
though the manual does disclose the error mechanism, from which the
danger may be inferred, it does not explain the kind of defensive
coding required to play it safe.

I think it would be useful to explore this a bit further and to document what

is needed for safe interop between Lua and C++.

Being C code, I suspect that Lua is ill-equipped to handle uncaught exceptions

that pass through the Lua layer.

If this is the case then perhaps it should be documented explicitly and/or

any additional nuances should be explained.  Are there any safety mechanism

that we can enable (or add) that would help to detect violations of this, even

just in a debug build?

--
You received this message because you are subscribed to the Google Groups "lua-l" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lua-l/CAD55k3ou_ZsXwUirRK6drzc0%2B-gRJjsX9edkF-naFvkK3G3h0A%40mail.gmail.com.

[eugeny gladkih]

> On Jan 4, 2024, at 08:10, actboy168 <actb...@gmail.com> wrote:
>
> throw will unwinding callstack, it will be much slower than longjmp.

guess you cannot measure the slowness :) it would or would not, depends on.

> Unfortunately, LUAI_THROW is not only used to raise errors, but also used to yield coroutines. The performance of using throw is not acceptable for code that heavily uses coroutines.
>

yep, that’s right but it’s out of context at all.
Lua uses c++ exceptions, that’s the fact. so let it to use them the right way.

> Hi,
>
> please make Lua more C++ compatible, catch only your own exceptions in LUAI_TRY macro not all. the smallest patch for 5.4.6 is follow
>
> --- ldo.c.orig 2024-01-01 16:41:57
> +++ ldo.c 2024-01-01 16:44:29
> @@ -55,7 +55,7 @@
> #if defined(__cplusplus) && !defined(LUA_USE_LONGJMP) /* { */
>
> /* C++ exceptions */
> -#define LUAI_THROW(L,c) throw(c)
> +#define LUAI_THROW(L,c) throw((const lua_longjmp*)c)
> #define LUAI_TRY(L,c,a) \
> - try { a } catch(...) { if ((c)->status == 0) (c)->status = -1; }
> + try { a } catch(const lua_longjmp*) { if ((c)->status == 0) (c)->status = -1; }
> #define luai_jmpbuf int /* dummy variable */
>
> #elif defined(LUA_USE_POSIX) /* }{ */
> @@ -1020,5 +1020,3 @@
> decnny(L);
> return status;
> }
>
> and, we’ll have another advantage making struct lua_longjmp publicly visible in lua.h or somewhere else
>
> --
> Yours sincerely, Eugeny.
>

> --
> You received this message because you are subscribed to the Google Groups "lua-l" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/lua-l/7fc82a8d-c9b1-4adb-9b91-e140812305c6n%40googlegroups.com.



--
Yours sincerely, Eugeny.