how clone a lua state ?

[Denis Dos Santos Silva]

#include <stdio.h>

#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"

int main(void) {
  lua_State *base = NULL;
  lua_State *b1 = NULL;

  base = luaL_newstate();
  luaL_openlibs(base);
  luaL_mylib(base);

  /* ... */
  /* how clone 'base' to b1 ? */
  /* ... */

  lua_close(base);
  lua_close(b1);

  return 0;
}

[Sainan]

There is no way to clone Lua states other than asking your OS to fork the process, which would clone everything in your process memory.

As for features in Lua itself, it has coroutines which are somewhat similar in concept except that it's concurrency (cooperative multitasking) as opposed to parallelism (preemptive multitasking).

If you do want to achieve the latter so that you do need multiple independent states, I suggest you just initialise them independently. I can hardly imagine it being much more complex than copying a whole state.

-- Sainan

[Vinícius dos Santos Oliveira]

Em dom., 20 de abr. de 2025 às 12:27, 'Sainan' via lua-l
<lu...@googlegroups.com> escreveu:

> As for features in Lua itself, it has coroutines which are somewhat similar in concept except that it's concurrency (cooperative multitasking) as opposed to parallelism (preemptive multitasking).

You can have preemptive multitasking without parallelism. Preemption
is a property of interrupting a task "forcefully". It has nothing to
do with the concept of parallelism. Cooperative/preemptive
multitasking is not the same taxonomy as concurrency/parallelism.

To achieve parallelism in Lua, one uses separate Lua states (as you
mentioned) to drive different threads. To achieve concurrency in Lua,
one may use coroutines. Lua coroutines only support cooperative
multitasking. Therefore if you want to use preemptive multitasking in
Lua, you'll have to resort to OS facilities to interrupt independent
Lua states (e.g. UNIX signals, POSIX threads).

[Sainan]

Ah yeah, that's fair, tho it always felt a bit hacky to me to try and force a yield. You can set a VM hook/trap, but then you're still kinda at the mercy of what functions the VM is calling and how long they take to return.

-- Sainan

[Francisco Olarte]

Denis:

On Sun, 20 Apr 2025 at 16:27, Denis Dos Santos Silva <de...@roo.com.br> wrote:
> int main(void) {
> lua_State *base = NULL;
> lua_State *b1 = NULL;

...>

> base = luaL_newstate();
> luaL_openlibs(base);
> luaL_mylib(base);
>
> /* ... */
> /* how clone 'base' to b1 ? */
> /* ... */

Do you do anything else before cloning? or are you in a thight loop
and need to squeeze cycles? I have a somehow similar problem and just
use a builder function? I have more or less...
lua_State build() {
lua_State *base = luaL_newstate();
luaL_openlibs(base);
luaL_mylib(base);
/* some hundred lines of init code */
return base;
}

Because in your example case ( and in my real one ) ( no work done in
base which cannot be done by building b1 the same ) a hypothetical
clone would leave you with two equal shared-nothing states, and (
certainly my ) states build quite fast.

Now, if you execute some non-reproducible, or slow, code in one state
( either lua or api calls, I use both of them while building, but they
are reproducible and fast ) you would need cloning to build b1.

Francisco Olarte.

[Augusto Rodrigues]

Denis and others, 

In one of the Lua projects I participated in, I saw the need to duplicate the Lua state for performance reasons, without using multithreading. In this project, a Lua state is 

initialized with a record of Lua C API functions, previously loaded Lua modules, and global data with very complex data structures, etc.

The address of this Lua state would then be assigned to a global variable to be used at another point in the application, where the previously configured Lua state would be duplicated to a new Lua state. This duplicated state would be used to execute certain application processes. After these processes were executed, the duplicated state would be discarded and another Lua state would be created from the previously configured Lua state.

In other words, the objective of duplicating a previously configured Lua state would be to avoid executing all the initialization procedures repetitively.

Unfortunately, the project was interrupted, but one of the ways I saw to do this duplication would be to change the allocation function used by Lua to record the addresses and sizes of memory space used during the creation of the Lua state. After a certain point, another function in C would use the previously known addresses and sizes and duplicate the content stored in these destinations and specify a new Lua state.

Would this approach have been successful, considering that this application would not require the use of multithreading?

Regards

--
You received this message because you are subscribed to the Google Groups "lua-l" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/lua-l/CA%2BbJJbxNdsiB_dK4WZY32fdA1j3JWfXkU4P7f0SApRKTqeCsTA%40mail.gmail.com.

[Sainan]

The question is why would you want to introduce such immense complexity into your application when simply abstracting away your API initialisation into one or more functions would suffice? I can hardly imagine the trade-off to be worth it.

And if the initialisation process indeed takes a long time in your system, maybe that's a problem worth looking at first.

-- Sainan

[Augusto Rodrigues]

Sainan,

Well, in fact, the project to use Lua has been stopped, so I didn't develop the idea of duplicating 

the Lua state. In practice, I developed something similar to your suggestion: abstracting the API

initialization into two or more functions.

So… why the idea of duplicating the Lua state? Well, I like exploring all possibilities to get maximum 

performance when running Lua scripts. That’s the reason for my question about duplicating the Lua state.

Thanks for your reply

--
You received this message because you are subscribed to the Google Groups "lua-l" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/lua-l/qZiPCqMmao7f0fUtyjPyom-6B5gbdfAfu1B6IrirdxG7B5xlOp1VUbCYIjYYCmZnezHVpsekjMS9bKhEJf87G1ehFw0qGvrdt8HQ7lfjGhM%3D%40calamity.inc.

[Andre Leiradella]

You could have a cache of initialized Lua states.

Andre

[Augusto Rodrigues]

I will search about this idea of cache of initialized Lua states.

Thanks. 

To view this discussion visit https://groups.google.com/d/msgid/lua-l/db49d783-1871-4b5f-b2fc-1c7c1ddda40en%40googlegroups.com.

[Denis Dos Santos Silva]

thx all for ideas. more context:

I'm developing a web server with Lua scripting support for an embedded environment. I've been considering a few design alternatives for handling Lua execution:

a) Create a new Lua VM instance for each incoming request.   (using a pool, like php-fpm)
b) Set up a pre-configured base Lua VM and clone it for each request.
c) Implement some form of caching mechanism to improve performance.

d) Open to other suggestions — any recommendations?

note: the project is working using model 'A'

[Sainan]

In my personal deployments of web apps using Lua for the backend, I have had 2 approaches:
1. Invoking Lua via PHP-FFI and running my scripts like so.
2. Giving Lua access to TCP APIs and having it speak HTTP by itself.

Can't say I like either approach, but the first one works fine enough and seems to not be a huge issue even with many requests per minute, tho realistically I should've written more of my logic in PHP.

-- Sainan

[bil til]

Am Mo., 21. Apr. 2025 um 16:48 Uhr schrieb Denis Dos Santos Silva
<de...@roo.com.br>:

> I'm developing a web server with Lua scripting support for an embedded environment. I've been considering a few design alternatives for handling Lua execution:
>
> a) Create a new Lua VM instance for each incoming request. (using a pool, like php-fpm)

Can you give some info about your target "embedded environment"?
- multi-core or single core?
- external RAM, or single-chip controller with internal RAM (typically
very restricted size? typically 32bit controller?)

for embedded single-chip controller with internal RAM your approach to
me sounds like wasting lots or precious RAM... .

Generally before looking to excited at such "speed gains" and
"application-isolation advantages", think well whether you need ANY
interaction / data exchange between such "parallel Lua states" - this
can get very tedious, dangerous and time-consuming, easily eating up
all "speed and isolation glamour" of pre-emptive multitasking
easily... .

[Denis Dos Santos Silva]

Thanks for all the help!

Core question: I initially asked if it was possible to clone or duplicate a Lua state to speed up the application. The short answer is: no.

Project overview:

The project is currently up and running, built to run on proprietary hardware with the following specs:

• CPU: ARM 400 MHz, single-core

• RAM: 64 MB

• Compiler: armcc (C99 with partial pthreads support)

The system is inspired by the Node.js architecture, but built using Lua.

Its primary usage is as a personal vault, since the device is a PED (PIN Entry Device).

It's up and running.

Weekend project:

I've been working on a side project called Moonbird. You can check it out here: 

Code partial repository: https://gitlab.com/denisdemais/moonbird
Screenshot/preview: https://gitlab.com/denisdemais/moonbird/-/blob/625892fa61d49a54c5bce3751aa4dab796962d31/images/000.jpeg

NOTE: I will study the base code and other projects.

https://github.com/fnuecke/eris

http://lua-users.org/wiki/PlutoLibrary

https://stackoverflow.com/questions/2497482/clone-a-lua-state

https://stackoverflow.com/questions/1383768/cloning-lua-state