Unit Testing w/ Custom C API Extensions

[Michael Bonnet]

I'm considering using Lua as an embedded scripting language for a much wider C project. I intend to expose a set of project-specific functions to Lua, and then run Lua scripts that use those functions to hook into the project.

Something like this:

```
#include <stdio.h>
#include <stdlib.h>
#include "lua5.4/lua.h"
#include "lua5.4/lauxlib.h"
#include "lua5.4/lualib.h"

/// @brief Polls most recent value(s) for telemetry channel(s) on the message bus.
/// Lua argument 1: mnemonic of the command to be sent.
/// Lua argument 2: table of command parameter mnemonics and their values.
/// Lua return value 1: void
/// @param lua_State* Lua state
/// @return status
static int sendCmd(lua_State* L)
{
    // Check we have both arguments
    int nArgs = lua_gettop(L);
    if(nArgs != 2) {
        return luaL_error(L, "sendCmd expects exactly 2 arguments!");
    }

    if(!lua_isnumber(L, 1)) {
        return luaL_error(L, "sendCmd expects a string (command mnemonic) for argument 1!");
    }

    if(!lua_istable(L, 2)) {
        return luaL_error(L, "sendCmd expects a table of command parameters for argument 2!");
    }

    // (Build and send command)

    return 0;
}

/// @brief Polls most recent value(s) for telemetry channel(s) on the message bus.
/// Lua argument 1: array of telemetry channels to get recent values from.
/// Lua return value 1: table of telemetry channel names mapped to their most recent values.
/// @param lua_State* Lua state
/// @return status
static int pollTlm(lua_State* L)
{
    // Check we have the argument
    int nArgs = lua_gettop(L);
    if(nArgs != 1) {
        return luaL_error(L, "pollTlm expects exactly 1 argument!");
    }

    if(!lua_istable(L, 1)) {
        return luaL_error(L, "pollTlm expects a table of telemetry channel mnemonics for argument 1!");
    }

    // (Poll telemetry values)

    // Push result onto the Lua stack
    lua_pushtable(L, result);

    return 0;
}

int main(int argc, char** argv) {
    // 1) Create Lua state
    lua_State* L = luaL_newstate();

    // 2) Open standard libraries
    luaL_openlibs(L);

    // 3) Register functions so Lua scripts can call them
    lua_register(L, "sendCmd", sendCmd);
    lua_register(L, "pollTlm", pollTlm);

    // Check if the user passed a script name
    if(argc < 2) {
        printf("Usage: %s <Lua script>\n", argv[0]);
        lua_close(L);
        return 1;
    }

    // 4) Load and run the Lua script
    if(luaL_loadfile(L, argv[1]) || lua_pcall(L, 0, 0, 0)) {
        fprintf(stderr, "Error: %s\n", lua_tostring(L, -1));
        lua_pop(L, 1); // remove error message
        lua_close(L);
        return 1;
    }

    // 5) Clean up and close Lua
    lua_close(L);
    return 0;
}

```

My question is, how might I go about unit testing the Lua scripts (that are using the custom C backend) themselves? Can something like busted or luaunit do it? If so, how would I configure the testing tool to use these custom C functions?

[Sainan]

Well, you could just put a few `asserts` and that may suffice. Alternatively, you could move your new C API functions into a C module that can be `require`d by Lua itself and then use `assert` or another testing harness with that module.

-- Sainan

[Frank Kastenholz]

I am doing a similar project with a couple

Of very large C libraries. The basic idea for testing C/Lua API functions is to make sure that arguments are properly marshaled from Lua to C and return values properly handled. (The icky case is when the C function takes a pointer to something and returns a value there (or, worse, a ** and returns a pointer to some allocated memory, etc, etc…)

My basic plan was to use the underlying C code’s unit tests and create lua code to exercise them … if you get the same “answer” and then code test then you win.

YouDO NOT want to get into testing the underlying code. Take their unit tests as “gospel truth” and don’t ask questions…

Good luck

Frank kastenholz

On Apr 14, 2025, at 5:51 PM, Michael Bonnet <maab...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "lua-l" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/lua-l/e1feb37f-c0c6-42b2-8558-2294f6ef3135n%40googlegroups.com.

[Thijs Schreijer]

On Mon, 14 Apr 2025, at 23:49, Michael Bonnet wrote:
> I'm considering using Lua as an embedded scripting language for a much wider C project. I intend to expose a set of project-specific functions to Lua, and then run Lua scripts that use those functions to hook into the project.
>
> Something like this:

> <snip>

>
> My question is, how might I go about unit testing the Lua scripts (that are using the custom C backend) themselves? Can something like busted or luaunit do it? If so, how would I configure the testing tool to use these custom C functions?

It depends to what you exactly want to test, and which combinations of Lua and C code. Considering you mentioned testing the Lua scripts;

- mock the C implementation (function level), in your mocked functions verify that the arguments passed in by the Lua scripts are the ones you expected
- create a Lua version of your C code, still a mock, but now a functional unit of all the functions, mimicking the C code. This will allow you to test multiple function calls in succession and verify overall logic/flow.

Both of these approaches are decoupled from the real C code. So whenever that code changes, your Lua code still passes, unless you also update the mocks. This is easy to forget and then your code will fail in a later stage. To mitigate this you'll have to set up an actual environment in which your actual C code is used (integration testing, docker is your friend in this case).

At Kong we use Busted as the test framework. It has several features that allow you to mock or stub functions. Our test library (Kong specific) contains a lot of functionality to start instances, or even clusters, mock backends, etc. to make it as easy as possible to write powerful tests.
For this we also have custom assertions (for Luassert/Busted) like this one: https://github.com/Kong/kong/blob/1c4b859be73f716b8e6796868cc35a9eee7413ca/spec/internal/asserts.lua#L344-L383
which allow us to write test assertions like this:

local res = http_client:get(whatever...)
local length = assert.response(res).has.header("Content-Length")
assert.are.equal(512, length)

This ensure that 1; the tests are very readable, also by non-domain experts. And 2; failure messages of those assertions will provide way more context than just the failure (in the example link, the failure will also show all headers that were found), which makes debugging tests far easier.

hth
Thijs

[Francisco Olarte]

Hi Michael...

On Mon, 14 Apr 2025 at 23:51, Michael Bonnet <maab...@gmail.com> wrote:
....

> My question is, how might I go about unit testing the Lua scripts (that are using the custom C backend) themselves? Can something like busted or luaunit do it? If so, how would I configure the testing tool to use these custom C functions?

I use testy for my tests, in a similar case, a lua interpreter embeded
in a module for the yate telephony server. What I do is just invoke it
by collecting the neede test file names in the global "arg" and
pcalling...


local oldarg=arg
arg = {}
-- unisteresting matching and popening of ls to get filenames in arg zapped...
local os_exit=os.exit -- Testy calls this on fails!

os.exit=function(code, close)
io.stderr:write(string.format("Testy exit(%s,%s)\n",code,close))
end

io.stderr:write("-----TESTY START--------\n")
io.stderr:write(string.format("Calling testy with %s\n", table.concat(arg)));

local ok,err = xpcall(dofile,debug.traceback,"testy.lua")

arg=oldarg
os.exit=os_exit

if (not ok) then
io.stderr:write(string.format("Testy errored: %s\n", err))
end
io.stderr:write("-----TESTY END----------\n")

I invoke this by either a telnet command handler built in lua or by a
special command in my init code. The test results end up on screen or
in the server log, where stderr is redirected when daemonized. Having
a mocked global arg and a mocked os.exit is enough to make testy
happy.

I just have two test dirs, one with the lua-only modules, other with
the server-specific. This way I can run testy fast on the command line
for the lua-only and run both on the server ( lua-ones are rechecked
because I have a stub module for the server for mockable things so I
can test logic, but some objects like http client/servers, thread
pools, posix extensions and other things are only available on the
server interpreter, and I have a lot of mocked things which need to be
rechecked on the server )

Francisco Olarte.