Hello, threading question.

[Francisco Olarte]

Hello everybody.

I'm researching scripting languages for some programs I have, an stumbled upon wren. Had a look at the documentations, blogs, groups and I think I can use it and is a nice language.

That being said, I have some reenting / multithreading doubts which I'd like to consult, as I have been unable to find docs ( if there are I'll be happy to read them if pointed to, I've just been unable to find them and thought this may have been answered before ).

First the simple one, C code needs to call Wren methods Wa, which in turns call a C implemented method Ca, which calls into the engine and ends up calling wrench methods Wb.

If I have understodd correctly, Ca preserves any thing it needs from the slots using handles before calling the engine, so it can restore what it need when it returns from the engine, is this correct?

Then my threading problems. The (C++) programs are heavily multithreaded. I DO NOT need multithreading inside the VM.

I've found in the doc I can create VMs in different threads, but I may need to share them among threads on a basic setup, i.e. I associate a mutex with every VM I create and just lock/use/unlock, is this possible?

And third question, If that is possible, I may need to reenter VM from a different thread on some corner cases. The scenario is like the first one but the callback from the engine would come from a different thread, so I would like to know if this is possible:

Thread 1: C code needs to call Wren methods Wa,

    --- I LOCK, prepare all slots, call into the VM

Thread 1: ... which in turns call a C implemented method Ca, which calls into the engine

   --- I grab handles ( private, no one else is going to know about them ) for everything, UNLOCK and call into the engine.

 

Thread 2: ... some other thread then ends up calling wrench methods Wb.

   -- so it LOCKS, calls the vm, grabs everything it needs on return and UNLOCKS and goes on

Thread 1: engine returns, LOCKS the VM, repopuplates slots as needed, returns from Ca

Thread 1: gets return from Wa, grabs what it needs, UNLOCKS, goes along.

Basically the first single thread scenario but unlocking in some moments where I'm not touching the engine and switching threads on the calls. Is this possible ?

Francisco Olarte.

[Michel Hermier]

From the little I understood, what you want to do is some kind of reentrancy with multithread. So this is a no no unfortunately.

[Brian Slesinsky]

It is explained on this page:

"While your foreign method is executing, the VM is completely suspended. No other fibers run until your foreign method returns. You should not try to resume the VM from within a foreign method by calling wrenCall() or wrenInterpret().The VM is not re-entrant."

http://wren.io/embedding/calling-c-from-wren.html

[Francisco Olarte]

Michel:

On Tuesday, June 4, 2019 at 2:21:34 PM UTC+2, Michel Hermier wrote:

From the little I understood, what you want to do is some kind of reentrancy with multithread. So this is a no no unfortunately.

I want three things, confirm I've got it right on how to reenter a state, all on the same thread, know if I can reuse it from different threads ( one at a time, full lock between ), and third is some reentrancy, but very limited ( switching threads only when control is fully outside of the vm ).

On a single thread, I assume I'm allowed to reenter, as normally any useful VM needs to allow it.

OTOH VMs without global data, like docs state wren is, can be used from different threads but only one at a time with locking (i.e., lock before entering, unlock after exiting ) as there is not  a way in C to distinguish what thread is calling ( in ANSI C, without calling the treading libraries ). I haven't tested, but I suppose this can be done.

And the third thing I ask can normally be done if the first one is true unless the code for the VM does things to prohibit it. After all the VM is just a big chunk of C code with callbacks, unless it does some non standard trick to notice it has been switched ( like detecting and testing stack growth direction, or calling the platform threading code ) it cannot detect if I switch threads.

Thanks.

   Francisco Olarte.

I'll try to do a real code example when/if I learn a little bit more, the third one tends to be misunderstood by nearly everyone I ask (i.e., if I can reenter a VM from a single thread ( C calls VM, VM calls C, C calls VM again ) you normally can do it from two differnent threads unless the VM code is doing something to specifically prohibit it.

[Francisco Olarte]

Brian:

Uh, oh. I read that page and did not notice it, thank you for pointing it. I guess I'll have to search for another language, threading I can deal without, reentrancy is a must for any of my moderately complex code. I never imagined that would be the case.

Thanks all for your time, I'll be back periodically to check, wren seems like a really interesting thing.

Francisco Olarte.

 

[QuentinC]

Hello,

Like you, I were quite interested by Wren, but finally decided not to
use it. One of my reasons were also the impossibility to be re-entrant,
i.e. C++ calls Wren, Wren calls C++, but then C++ can't call Wren again.
It's really a killer even for simple things. For example, impossible to
make a sort method for the List class that would be implemented in C++
and that would call an < method on the list items. You are forced to
write a sort method in Wren, which is of course a lost of time and
performance with all the chance to introduce bugs in your algorithm.
With the hope that this issue will be considered and fixed, but for the
moment it doesn't seem to be the case.

You may have a look at other scripting languages listed at
https://github.com/r-lyeh-archived/scriptorium
More than 50 are listed.

If you find lua too simple or too unusual, you may try embedding python
or ruby. AngelScript is also quite good.

Finally, I have decided to start my own scripting language project, for
several reasons. I'm still following Wren because it helped me a lot for
that other language, and I must again say thank you to them.
I won't post a link to my own project without permission though, and
also because it's quite still experimental.

[Francisco Olarte]

Quentin:

On Tue, Jun 4, 2019 at 6:12 PM QuentinC <webm...@quentinc.net> wrote:
> Like you, I were quite interested by Wren, but finally decided not to
> use it. One of my reasons were also the impossibility to be re-entrant,

...
And not only simple stuff easily done in wren, like the sort methods.
I want to put it in control of a call system, calls generates events,
I transform them to linear flows with coroutines, and I call into C to
make the calls do stuff. But sometimes C needs something to be done in
the high level scripted , fast, simple things.

> You may have a look at other scripting languages listed at
> https://github.com/r-lyeh-archived/scriptorium
> More than 50 are listed.

Already done.

> If you find lua too simple or too unusual,

That's the one I'm going away from.

> you may try embedding python

That's too heavy.

> or ruby. AngelScript is also quite good.

Ruby, same as python, same reasons. AngelScript is too huge on
dependencies. I considered many of them, I want something in the
complexity range of lua, Squirrel ( nice, but awful docs ) ,wren (
problem already stated ), duktape ( unusable coroutines, but I may
have a solution for that ). Lua is nice but too minimalistic in many
places and too weird in others.

> Finally, I have decided to start my own scripting language project, for
> several reasons. I'm still following Wren because it helped me a lot for
> that other language, and I must again say thank you to them.
> I won't post a link to my own project without permission though, and
> also because it's quite still experimental.

Send me a direct mail if you want, and I'll take a look, I may even be
able to read on it if it is in french ( your domain points to a french
site )( but I warn you my writing style is considered a bit agressive
by someone ).

Francisco Olarte.