Building multi-module C package

[Stefano Cossu]

Hello there,
I am new to Lua and I'm building some bindings for a C library.

The original library is spread out across several .c and .h files, and I
want to keep a similar separation in the Lua-C modules. So far, I have
been writing one C files per module, compiling it into a separate .so
file. E.g.

mylib/lua
|
`- src
| |
| `- mod1.c
| `- mod2.c
| `- mylib.h
|
`- lib
|
`- mod1.so
`- mod2.so

Each of the *.so files contains a separate luaopen_* symbol. So that in
Lua I can call

mod1 = require "mod1"

etc.

So far so good. Now, however, mod1.c and mod2.c need to access common
functions defined in the other module. I thought about adding the shared
function declarations in mylib.h, which is included by both module
files, but the symbols won't be found when the libraries are loaded
because `require` loads them separately.

I also thought about compiling all the object files into a single
mylib.so file:

gcc $(CFLAGS) -o lib/mylib.so src/*.c

This way, if I set one of the C files and the `luaopen_*` function to
the main library name, the library loads without problems and inspecting
it with `nm` all the `luaopen_*` symbols are visible, but Lua only sees
the members defined in the `luaopen_mylib` function.

What's a recommended practice for dealing with this situation?

Thanks,
Stefano

[Sainan]

Lua itself is spread across over several .c and .h files, that doesn't mean that e.g. the lexer and parser are two separate libraries; they're still just a part of Lua. Of course, it's possible that this does make sense in your use-case, e.g. if it's sort of like a monorepo.

As you mention needing to access common functions, you could do so using a static library, but that would mean having that common code duplicated into each module.

Otherwise, iirc, Lua only calls the relevant 'luaopen' function, so you can export multiple of it and only the one the user elected to `require` will be used.

-- Sainan

[Sean Conner]

It was thus said that the Great 'Stefano Cossu' via lua-l once stated:

>
> I also thought about compiling all the object files into a single
> mylib.so file:
>
> gcc $(CFLAGS) -o lib/mylib.so src/*.c
>
> This way, if I set one of the C files and the `luaopen_*` function to
> the main library name, the library loads without problems and inspecting
> it with `nm` all the `luaopen_*` symbols are visible, but Lua only sees
> the members defined in the `luaopen_mylib` function.
>
> What's a recommended practice for dealing with this situation?

From the manual (Lua 5.4) about package.searchers:

The fourth searcher tries an all-in-one loader. It searches the C
path for a library for the root name of the given module. For
instance, when requiring a.b.c, it will search for a C library for
a. If found, it looks into it for an open function for the
submodule; in our example, that would be luaopen_a_b_c. With this
facility, a package can pack several C submodules into one single
library, with each submodule keeping its original open function.

That might be the way to go.

-spc

[mjmouse9999]

On Friday, June 6, 2025 at 7:29:35 AM UTC+10 Sean Conner wrote:

... 

The fourth searcher tries an all-in-one loader. It searches the C
path for a library for the root name of the given module. For
instance, when requiring a.b.c, it will search for a C library for
a. If found, it looks into it for an open function for the
submodule; in our example, that would be luaopen_a_b_c. With this
facility, a package can pack several C submodules into one single
library, with each submodule keeping its original open function.

That might be the way to go.

-spc

The luasec[1] package does this with its core modules:

- ssl.x509, ssl.config, ssl.context, and ssl.core are all in the same `ssl.dll` once compiled

[1]: https://github.com/lunarmodules/luasec

[Stefano Cossu]

OK, I rewired the modules to use luaopen_mylib_mod*, and compiled into a single shared object, and now I can require "mylib.mod1" which is also a more appropriate syntax. Thanks for all your suggestions. 

[Denis Dos Santos Silva]

luarocks maybe a good start to build your module win32/linux

https://github.com/luarocks/luarocks/blob/main/docs/creating_a_rock.md

[Stefano Cossu]

Yes, eventually I'll build a rock because it's a library, but for now I
wanted to leave that additional complexity aside if possible (I'm
already learning Lua and its C API).

Thanks,
Stefano

> [1]: https://github.com/lunarmodules/luasec <https://github.com/
> lunarmodules/luasec>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "lua-l" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/
> lua-l/Kya2JhqdlYs/unsubscribe <https://groups.google.com/d/topic/lua-l/
> Kya2JhqdlYs/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to lua-
> l+unsu...@googlegroups.com <mailto:lua-l+un...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/lua-
> l/6c26e79d-3117-450f-b3f6-9d8964db36c9n%40googlegroups.com <https://
> groups.google.com/d/msgid/lua-l/6c26e79d-3117-450f-
> b3f6-9d8964db36c9n%40googlegroups.com?utm_medium=email&utm_source=footer>.