A few questions from a newcomer

[webm...@quentinc.net]

Hello,

I have just found wren and quite like it. I'm trying to embed it for a game.

I first embedded lua: the language is too limited, and the syntax is unusual (in particular the way to do pseudo-OO)
I tried to embed python 3: the problem is that it's extremely hard to proevent malicious scripters from doing bad things in their code (i.e. difficult to properly sandbox)
I tried AngelScript: it's quite a good language, but their handle thing is still quite bizarre: sometimes you have to put @, sometimes not; and it's quite hard to register everything right (especially with templates).
Ultimately I wanted to embed V8, because JavaScript is cool and sound for me that getting near to the perfect language ; but I quickly gave up. Their examples in C++ look very cool, but it's way way too difficult to get up and running with something.
So now I found Wren, and it seems to be quite a good fit.

The only real missing feature I have found so far is the ability to save bytecode into a file, because game scripters may don't want to publish their code.

Of course as a newcomer, I have several questions. Here are already a few below.
Thank you for your answers !

1. In the implementation of a foreign method, how can I return an object of another foreign type ?

Wren code:
foreign class Foo { }
foreign class Bar {
foreign foo
}

C code:
struct Foo { };
struct Bar {
Foo* foo;
};
void barGetFoo (WrenVM* vm) {
Bar* bar = (Bar*)wrenGetSlotForeign(vm, 0));
wrenSetSlot???(vm, 0, bar->foo);
}

2. Why can't I write this ?
Is there an alternative ?
Wren code:
foreign class A {}
foreign class B is A {}

3. What about foreign class having multiple constructors ?
Which one should I register in WrenForeignClassMethods.allocate ?
Wren code:
foreign class A {
construct new () {}
construct new (x) {}
}

I have found this workaround, but well... I doubt that it will always be possible to provide simple default values:
Wren code:
foreign class A {
static new () { new(someDefaultValue) }
construct new (x) {}
}

By the way, small language inconsistancy imo:
why the foreign constructor declaration need {}, when all other foreign methods don't ?
Why do we write: construct now () {}
instead of: foreign construct new ()
?

4. In the language, I saw that string conversion wasn't explicit:
Wren code:
var a = "Hello"
var b = 123
var c = a+b // right operant must be a string
OF course I can use b.toString here.
However, is there a way to make string conversion implicit with operator + ? How should I implement it ?

5. More a philosophical than technical question now.
In wren_core.h, I can read this comment:
// With Wren, we try to do as much of it in C as possible. Primitive methods
// are always faster than code written in Wren, and it minimizes startup time
// since we don't have to parse, compile, and execute Wren code.

I totally agreen with you until I see wren_core.wren, with a bunch of core methods implemented in wren.
Why are these methods implemented in wren and not in C ?
Probably that methods like String.join, String.split, etc. could be better implemented in C rather than in Wren.
Additionally, when I see this wren code:
    for (element in this) {
      if (!first) result = result + sep
      first = false
      result = result + element.toString
    }
I immediately think about the string buffer problem: the fact that the method is going to be slow (complexity O(n^2)).

[Michel Hermier]

Le mer. 25 juil. 2018 à 07:05, <webm...@quentinc.net> a écrit :

Hello,

I have just found wren and quite like it. I'm trying to embed it for a game.

I first embedded lua: the language is too limited, and the syntax is unusual (in particular the way to do pseudo-OO)
I tried to embed python 3: the problem is that it's extremely hard to proevent malicious scripters from doing bad things in their code (i.e. difficult to properly sandbox)
I tried AngelScript: it's quite a good language, but their handle thing is still quite bizarre: sometimes you have to put @, sometimes not; and it's quite hard to register everything right (especially with templates).
Ultimately I wanted to embed V8, because JavaScript is cool and sound for me that getting near to the perfect language ; but I quickly gave up. Their examples in C++ look very cool, but it's way way too difficult to get up and running with something.
So now I found Wren, and it seems to be quite a good fit.

The only real missing feature I have found so far is the ability to save bytecode into a file, because game scripters may don't want to publish their code.

An alternative solution would be to encrypt your script, unless Bob change its mind so we get binary format someday.

Of course as a newcomer, I have several questions. Here are already a few below.
Thank you for your answers !

1. In the implementation of a foreign method, how can I return an object of another foreign type ?

Wren code:
foreign class Foo { }
foreign class Bar {
foreign foo
}

C code:
struct Foo { };
struct Bar {
Foo* foo;
};
void barGetFoo (WrenVM* vm) {
Bar* bar = (Bar*)wrenGetSlotForeign(vm, 0));
wrenSetSlot???(vm, 0, bar->foo);
}

Because wren don't have control of the lifetime of bar->foo. You have to wrap foo in an handle to do that.

2. Why can't I write this ?
Is there an alternative ?
Wren code:
foreign class A {}
foreign class B is A {}

This is currently forbidden, but might be available in the future. I think the reason was that it add quite a complexity, and Bob didn't wanted to go in the rabbit hole without users.

The only generic solution I see is to use delegation. Something like:

class A {}

foreign class A_impl {}

class B is A {}

foreign class B_impl{}

It add quite a boiler plate but for now it is the only solution I see, unless you can show code.

3. What about foreign class having multiple constructors ?

They have.

Which one should I register in WrenForeignClassMethods.allocate ?

As the name suggest it is an allocator, not a constructor.

Wren code:
foreign class A {
construct new () {}
construct new (x) {}
}

This is valid.

I have found this workaround, but well... I doubt that it will always be possible to provide simple default values:
Wren code:
foreign class A {
static new () { new(someDefaultValue) }
construct new (x) {}
}

Not the same code as earlier. Maybe you are disappointed by the initialisation. Hard to tell without real code 

By the way, small language inconsistancy imo:
why the foreign constructor declaration need {}, when all other foreign methods don't ?
Why do we write: construct now () {}
instead of: foreign construct new ()
?

Because they are 2 different things. First one is a normal constructor. Second one don't really mean anything.

The way wren constructor works is by automagicaly creating a method to the class, and adding the body of the class.

The way you make a foreign constructor is by making a static foreign method.

By the way the name of the method is not important. new is the normal convention, but nothing prevent you from naming you want.

4. In the language, I saw that string conversion wasn't explicit:
Wren code:
var a = "Hello"
var b = 123
var c = a+b // right operant must be a string
OF course I can use b.toString here.
However, is there a way to make string conversion implicit with operator + ? How should I implement it ?

AFAIK it is not possible. There is no such thing as a global operator +. Otherwise you have to hack every + methods on the correct language.

Else you can do explicit string formating like:

var c = "${a}${b}"

(Might be wrong syntax doing it from memory)

5. More a philosophical than technical question now.
In wren_core.h, I can read this comment:
// With Wren, we try to do as much of it in C as possible. Primitive methods
// are always faster than code written in Wren, and it minimizes startup time
// since we don't have to parse, compile, and execute Wren code.

I totally agreen with you until I see wren_core.wren, with a bunch of core methods implemented in wren.
Why are these methods implemented in wren and not in C ?

I would say lack of time/making it in C is either more complicated or does not contribute to speed.

We need a better benchmarking/tracing tool.

Probably that methods like String.join, String.split, etc. could be better implemented in C rather than in Wren.
Additionally, when I see this wren code:
    for (element in this) {
      if (!first) result = result + sep
      first = false
      result = result + element.toString
    }
I immediately think about the string buffer problem: the fact that the method is going to be slow (complexity O(n^2)).

Sure, wren still need to hit the world to improve...

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/738b1607-59e2-4106-91bd-886d89bddd4c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[QuentinC]

Hello,

Thank you for your quite quick answers.

Q0:

> An alternative solution would be to encrypt your script, unless Bob
change its mind so we get binary format someday.

OK. Anyway, scripts are already bundled and encrypted in akind of zip
file that may also contain resources (images, sounds, translation
messages, etc.).
Storing bytecode probably takes less space and is faster to load than
source code though.

Q1:

> Because wren don't have control of the lifetime of bar->foo. You have
to wrap foo in an handle to do that.

Can you tell me more on how to do this ?
Can I use wrenSetSlotNewForeign outside of a foreign class allocator ?
If yes: how do I retriev the class object of class named X in C code ?

Q2:

> This is currently forbidden, but might be available in the future. I
think the reason was that it add quite a complexity, and Bob didn't
wanted to go in the rabbit hole without users.

OK. As an alternative I saw that I can overload the is operator. So I
can probably go around the limitation by being smart when registering
methods and by overloading the is operator of the subclass to return
true when called with the superclass.
Or, as you say, by using sort of interfaces.

Q3 (about multiple constructors of a foreign class) :
With your explanations I better understand, thank you. The brackets are
there because in fact that's a normal constructor, I can write any wren
construction code as I wish.
The thing that confused me was that, in the allocator method, I was able
to fetch the parameter passed to the constructor in slot 1. So I though
that the brackets were totally useless and that the actual construction
was the call to my C allocator function (hance my question which
constructor if there are several).

Thank you.

[Michel Hermier]

Le mer. 25 juil. 2018 à 18:20, QuentinC <webm...@quentinc.net> a écrit :

Hello,

Thank you for your quite quick answers.

Q0:
 > An alternative solution would be to encrypt your script, unless Bob
change its mind so we get binary format someday.

OK. Anyway, scripts are already bundled and encrypted in akind of zip
file that may also contain resources (images, sounds, translation
messages, etc.).
Storing bytecode probably takes less space and is faster to load than
source code though.

Last statement is totally wrong, due to language design, and current implementation, and manly because of error handling source must be bundled with the bytecode. So it is smaller and probably faster to have scripts instead.

Q1:
 > Because wren don't have control of the lifetime of bar->foo. You have
to wrap foo in an handle to do that.

Can you tell me more on how to do this ?
Can I use wrenSetSlotNewForeign outside of a foreign class allocator ?

Yes you can.

If yes: how do I retriev the class object of class named X in C code ?

Should be wrenGetVariable.

Q2:
 > This is currently forbidden, but might be available in the future. I
think the reason was that it add quite a complexity, and Bob didn't
wanted to go in the rabbit hole without users.

OK. As an alternative I saw that I can overload the is operator. So I
can probably go around the limitation by being smart when registering
methods and by overloading the is operator of the subclass to return
true when called with the superclass.
Or, as you say, by using sort of interfaces.

Q3 (about multiple constructors of a foreign class) :
With your explanations I better understand, thank you. The brackets are
there because in fact that's a normal constructor, I can write any wren
construction code as I wish.
The thing that confused me was that, in the allocator method, I was able
to fetch the parameter passed to the constructor in slot 1. So I though
that the brackets were totally useless and that the actual construction
was the call to my C allocator function (hance my question which
constructor if there are several).

Well it is useful. There are class of foreign that might need to cherry some argument for the allocator (an array implementation might use it).

But all this is the result of a (crude) optimisation and implementation of a generic constructor (without templates)

Thank you.

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B58A349.3020609%40quentinc.net.

[QuentinC]

Hello,

Thank you, I managed to return an object of another foreign type with
your explanations.
Effectively, I need to get first the class slot using wrenGetVariable,
and then use wrenSetSlotNewForeign.

I have now other questions:

I'm now working on basic input type checking, and noticed that
wrenAbortFiber was not working when called inside a foreign method.
Why ? CAn I do something about it ?


I wrote this kind of utility function:
void wrenTypeError (WrenVM* vm, int idx, const char* expectedType) {
char buf[256] = {0};
snprintf(buf, 255, "expected parameter %d to be %s", idx, expectedType);
wrenEnsureSlots(vm, 16);
wrenSetSlotString(vm, 15, buf);
wrenAbortFiber(vm, 15);
}

bool wrenSlotCheckDouble (WrenVM* vm, int slot) {
if (WREN_TYPE_NUM!=wrenGetSlotType(vm, slot)) {
wrenTypeError(vm, slot, "Num");
return false;
}
return true;
}

Used in the following way:

void someForeignMethod (WrenVM* vm) {
if (!wrenSlotCheckDouble(vm, 1)) return;
// do some stuff
}

I also have the problem that, with the base API, I'm not able to check
if a foreign object is really of the expected type, i.e. given foreign
class X {} and foreign class Y {}, there don't seem to have a way to
check in a foreign method if I'm getting an object of type X and not Y.

Isn't there another way than doubling every method like this ?
foreign class X {
foreign someMethod_(x)
someMethod(x){
if (!(x is X)) Fiber.abort("Plouf!")
return someMethod_(x)
}
}

One more question: it is said in the documentation that I shouldn't try
to call wrenCall from inside a foreign method because the VM isn't
reentrant.

What's wrong with it if I save the slots first, like this ?

WrenHandle* callHandle = wrenMakeCallHandle("someMethod()");
int nHandles = wrenGetSlotCount(vm);
WrenHandle* handles[nHandles];
for (int i=0; i<nHandles; i++) handles[i] = wrenGetSlotHandle(vm, i);
wrenSetSlotXXX(vm, 0, methodCallReceiver);
wrenCall(vm, callHandle);
for (int i=0; i<nHandles; i++) {
wrenSetSlotHandle(vm, i, handles[i]);
wrenReleaseHandle(handles[i]);
}


That's enough for now.
Thank you for your answers.

[Michel Hermier]

Le ven. 27 juil. 2018 à 09:07, QuentinC <webm...@quentinc.net> a écrit :

Hello,

Thank you, I managed to return an object of another foreign type with
your explanations.
Effectively, I need to get first the class slot using wrenGetVariable,
and then use wrenSetSlotNewForeign.

I have now other questions:

I'm now working on basic input type checking, and noticed that
wrenAbortFiber was not working when called inside a foreign method.
Why ? CAn I do something about it ?

It should be working. Maybe you are hitting another bug...

I wrote this kind of utility function:
void wrenTypeError (WrenVM* vm, int idx, const char* expectedType) {
char buf[256] = {0};
snprintf(buf, 255, "expected parameter %d to be %s", idx, expectedType);
wrenEnsureSlots(vm, 16);
wrenSetSlotString(vm, 15, buf);
wrenAbortFiber(vm, 15);
}

wrenEnsureSlot has a potential bug that can lead to corruption (unless it was fixed)

bool wrenSlotCheckDouble (WrenVM* vm, int slot) {
if (WREN_TYPE_NUM!=wrenGetSlotType(vm, slot)) {
wrenTypeError(vm, slot, "Num");
return false;
}
return true;
}

Used in the following way:

void someForeignMethod (WrenVM* vm) {
if (!wrenSlotCheckDouble(vm, 1)) return;
// do some stuff
}

I also have the problem that, with the base API, I'm not able to check
if a foreign object is really of the expected type, i.e. given foreign
class X {} and foreign class Y {}, there don't seem to have a way to
check in a foreign method if I'm getting an object of type X and not Y.

Isn't there another way than doubling every method like this ?
foreign class X {
foreign someMethod_(x)
someMethod(x){
if (!(x is X)) Fiber.abort("Plouf!")
return someMethod_(x)
}
}

Can be a missing API to return an object class to a slot. Maybe propose a patch or fill a bug with use case.

One more question: it is said in the documentation that I shouldn't try
to call wrenCall from inside a foreign method because the VM isn't
reentrant.

What's wrong with it if I save the slots first, like this ?

WrenHandle* callHandle = wrenMakeCallHandle("someMethod()");
int nHandles = wrenGetSlotCount(vm);
WrenHandle* handles[nHandles];
for (int i=0; i<nHandles; i++) handles[i] =  wrenGetSlotHandle(vm, i);
wrenSetSlotXXX(vm, 0, methodCallReceiver);
wrenCall(vm, callHandle);
for (int i=0; i<nHandles; i++) {
wrenSetSlotHandle(vm, i, handles[i]);
wrenReleaseHandle(handles[i]);
}

The problem is not that the API forbid you to do it. I made the VM reentrant quite a lots of time. The problem is in the stack management and optimisation.

I had a lot of discussions with Bob about it, but he don't want to change it for now.

I have lots of pros like architecture cleaning, some simplification in the UI/libuv library, but Bob expose the biggest cons with possible speed impact(, and maybe usefulness?).

That's enough for now.
Thank you for your answers.

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B5AC491.8060305%40quentinc.net.

[QuentinC]

Hello,

> It should be working. Maybe you are hitting another bug...

Yes, it was another bug. It found it in the meantime and it works now.

< The problem is not that the API forbid you to do it. I made the VM
reentrant quite a lots of time. The problem is in the stack management
and optimisation.
> I had a lot of discussions with Bob about it, but he don't want to
change it for now.
I have lots of pros like architecture cleaning, some simplification in
the UI/libuv library, but Bob expose the biggest cons with possible
speed impact(, and maybe usefulness?).

Is there a patch to add reentrancy, or how to proceed to make it ?

There are certainly a lot of usecases:
1. List.sort is missing. Sorting a list is probably something that
should be present in the standard library of any decent language,
shouldn't it?
However, currently, if I want to do it, I would be obliged to write it
in wren and it's probably going to be slow, without mentionning that I'm
going to reinvent the wheel (when I could just call qsort in C).
It's impossible to write it in C because from C, I'm unable to call any
wren comparison operator method (possibly overloaded).
2. by the same reasonning, it would allow writing methods like List.join
in C, correctly calling .toString for each element and probably gaining
some speed in the process; again idem for Listremove(item) (calling ==),
and I'm sure, a lot of others.

In fact it's quite a big stopper.
By the way it answers to one of the question I had previously: why some
methods are implemented in wren and not in C.

> Can be a missing API to return an object class to a slot. Maybe
propose a patch or fill a bug with use case.

Ahaha so it's effectively missing, thank you. I have done it for myself,
code below.
This kind of thing should effectively probably be available in standard:

void wrenGetSlotClass (WrenVM* vm, int srcSlot, int dstSlot) {
ObjClass *classObj = wrenGetClass(vm, vm->apiStack[srcSlot]);
vm->apiStack[dstSlot] = OBJ_VAL(classObj);
}

bool wrenGetSlotIsClass (WrenVM* vm, int slot, WrenHandle* classHandle) {
ObjClass *classObj = wrenGetClass(vm, vm->apiStack[slot]);
ObjClass *baseClassObj = AS_CLASS(classHandle->value);
do {
if (baseClassObj == classObj) return true;
classObj = classObj->superclass;
} while (classObj != NULL);
return false;
}

Almost copied from wren_core.c, object_is.


Thank you.

[Michel Hermier]

Le 27/07/2018 à 16:27, QuentinC a écrit :
> Hello,
>
> > It should be working. Maybe you are hitting another bug...
>
> Yes, it was another bug. It found it in the meantime and it works now.
>
> < The problem is not that the API forbid you to do it. I made the VM
> reentrant quite a lots of time. The problem is in the stack management
> and optimisation.
> > I had a lot of discussions with Bob about it, but he don't want to
> change it for now.
> I have lots of pros like architecture cleaning, some simplification in
> the UI/libuv library, but Bob expose the biggest cons with possible
> speed impact(, and maybe usefulness?).
>
> Is there a patch to add reentrancy, or how to proceed to make it ?

I have a patch series, in a branch, but it has to be rebased to master.

>
> There are certainly a lot of usecases:
> 1. List.sort is missing.  Sorting a list is probably something that
> should be present in the standard library of any decent language,
> shouldn't it?
> However, currently, if I want to do it, I would be obliged to write it
> in wren and it's probably going to be slow, without mentionning that
> I'm going to reinvent the wheel (when I could just call qsort in C).
> It's impossible to write it in C because from C, I'm unable to call
> any wren comparison operator method (possibly overloaded).
> 2. by the same reasonning, it would allow writing methods like
> List.join in C, correctly calling .toString for each element and
> probably gaining some speed in the process; again idem for
> Listremove(item) (calling ==), and I'm sure, a lot of others.

Well for me the biggest pro, is that consifering the new emergence of
functional programing, callbacks are now everywhere, and for quite good
I think. So being able to have a huge chain of callbacks with native in
the middle is an evidence.

>
> In fact it's quite a big stopper.
> By the way it answers to one of the question I had previously: why
> some methods are implemented in wren and not in C.

Me being me, I'm not a big fan of having all/most in C. I would prefer
to have all/most in wren, and enable C version of some methods *on demand*.
Rationale is to have a reference implementation, that is portable,
optimisable (JIT or what ever) and less subject to host
implementation/optimisation errors.
That way one can also compare C implementation against reference code.

>
>
> > Can be a missing API to return an object class to a slot. Maybe
> propose a patch or fill a bug with use case.
>
> Ahaha so it's effectively missing, thank you. I have done it for
> myself, code below.
> This kind of thing should effectively probably be available in standard:
>
> void wrenGetSlotClass (WrenVM* vm, int srcSlot, int dstSlot) {
>   ObjClass *classObj = wrenGetClass(vm, vm->apiStack[srcSlot]);
> vm->apiStack[dstSlot] = OBJ_VAL(classObj);
> }
>
> bool wrenGetSlotIsClass (WrenVM* vm, int slot, WrenHandle* classHandle) {
>   ObjClass *classObj = wrenGetClass(vm, vm->apiStack[slot]);
>   ObjClass *baseClassObj = AS_CLASS(classHandle->value);
>   do  {
>     if (baseClassObj == classObj) return true;
>     classObj = classObj->superclass;
>   }   while (classObj != NULL);
> return false;
> }

Instead I would go: (, and for the sake using the stack and not relying
on Handles):

void wrenGetSlotClass (WrenVM* vm, int dstSlot, int srcSlot) // for
mirroing slot order with other functions
bool wrenGetSlotIsClass (WrenVM* vm, int slot, int classSlot) // less
efficient in most case since you have to push Handle to stack, but I
think the spread of WrenHandle in methods arguments is a good think.

But else nice, please propose a patch/merge request for this. (also
don't forget to check slot values against stack size)

[QuentinC]

Hello,

I have found that you have proposed a pull request for reentrancy.

I have checked it out and trying to merge it with the current master,
but didn't manage to handle conflicts correctly.

Reentrancy works well, but now I have another strange problem with the
merged code:

var f = Fiber.new{
var x = 1
var y = 2
var z = 3
System.print("x=%(x), y=%(y), z=%(z)")
}
System.print("Before")
f.call()

It should obviously print x=1, y=2, z=3
But prints x=System, y=1, z=2

All the best.

[QuentinC]

Hello,

> But else nice, please propose a patch/merge request for this. (also
don't forget to check slot values against stack size)

thank you; I could try. This would be my first pull request to someone
else's project, so I need first to check how to proceed exactly.


Best

[Michel Hermier]

Looks like an of by one. Did you ran the tests first, and does it also fail on trunk? Just to be sure it is the change set that cause the problem.

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B5C1513.5030606%40quentinc.net.

[QuentinC]

Hello,

> Looks like an of by one. Did you ran the tests first, and does it
also fail on trunk? Just to be sure it is the change set that cause the
problem.

As far as I know, It worked fine with the master branch, so obviously I
introduced the bug while trying to merge your reentrancy branch. I will
try again unless you fix it before.

I have problems to run tests, so I can't really tell if it works. I'm on
windows and using MinGW. All tests that do something with strings and
UTF-8 fail, and at some point test.py crashes with an unicode decode
error. So all tests that follow are never run. My default python is 3.6.
I don't have python 2.x. Not sure how to fix test.py.

Before that I had to tweak a bit the makefile and my environment so that
I can run make.
I commented out lines 93 to 101 of util/wren.mk. First because at line
96-98 "else if" doesn't seem to be allowed (it says excess text after
'else' or something like that), and secondly because it adds a -fPIC
option that don't seem to be needed (I don't know what it does).
Additionally I had to add a cc.bat file redirecting to gcc, otherwise
make says file not found:
@echo off
gcc %*

[Michał Kozakiewicz]

This can help you:

https://github.com/munificent/wren/issues/578

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B5C95E5.9070102%40quentinc.net.

[Michał Kozakiewicz]

For MinGw see @genio comments:

https://github.com/libuv/libuv/issues/1924

(I don't test his(genio) script) 

See this list of scripting languages:

https://github.com/dbohdan/embedded-scripting-languages

[Michel Hermier]

Do not know when I'll have time to give it a try. Maybe tomorrow or or the day after.

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B5C95E5.9070102%40quentinc.net.

[QuentinC]

Hello,

> This can help you:
> https://github.com/munificent/wren/issues/578

> For MinGw see @genio comments:
> https://github.com/libuv/libuv/issues/1924

Thank you. However, I won't put effor in making libuv stuff working. I
won't use it in my real project anyway.

For the moment I haven't embedded Wren into my real project yet. I'm
just in the experimentation phase and will decide whether I'm really
going to use it or no. So I just wrote a basic script runner like this:

int main (int argc, char** argv) {
// Won't go in the details of my Wren class and template+macro
registration stuff, ~400 lines; perhaps later if it works well. I
already had a similar thing for lua when I used it.
Wren wren;
wren .module("main")
.beginClass<Test>("Test")
CONSTRUCTOR(Test, double, double)
DESTRUCTOR(Test)
METHOD("someMethod(_)", Test, someMethod)
METHOD("otherMethod(_,_)", Test, otherMethod)
PROPERTY("someProperty", Test, someProperty)
.endClass();

FILE* fp = fopen(argv[1], "rb");
if (!fp) { printf("Couldn't open %s\n", argv[1]); return 1; }
fseek(fp, 0, SEEK_END);
int sourcelen = ftell(fp);
fseek(fp, 0, SEEK_SET);
char* source = malloc(sourcelen+1);
int nread = fread(source, sourcelen, 1, fp);
if (1!=nread) { printf("Failed to read %s: result=%d, length=%d\n",
argv[1], nread, sourcelen); return 1; }
source[sourcelen]=0;
fclose(fp);
int result = wren.run("main", source);
return result;
}

[Brian Slesinsky]

On Friday, July 27, 2018 at 11:50:36 PM UTC-7, Michel Hermier wrote:

Le 27/07/2018 à 16:27, QuentinC a écrit :
> Hello,
>
> > It should be working. Maybe you are hitting another bug...
>
> Yes, it was another bug. It found it in the meantime and it works now.
>
> < The problem is not that the API forbid you to do it. I made the VM
> reentrant quite a lots of time. The problem is in the stack management
> and optimisation.
> > I had a lot of discussions with Bob about it, but he don't want to
> change it for now.
> I have lots of pros like architecture cleaning, some simplification in
> the UI/libuv library, but Bob expose the biggest cons with possible
> speed impact(, and maybe usefulness?).
>
> Is there a patch to add reentrancy, or how to proceed to make it ?
I have a patch series, in a branch, but it has to be rebased to master.

For anyone else following along, the pull request seems to be https://github.com/munificent/wren/pull/491

This is not for anything important, but I'm curious how you handle fibers. What happens when switching to a different fiber in a callback from C code? Suppose two fibers switch back and forth and both want to have a suspended C function on the stack?

[QuentinC]

> For anyone else following along, the pull request seems to be
https://github.com/munificent/wren/pull/491

YEs, exactly, it's this one.

> This is not for anything important, but I'm curious how you handle
fibers. What happens when switching to a different fiber in a callback
from C code?
> Suppose two fibers switch back and forth and both want to have a
suspended C function on the stack?

Michel will certainly better answer, but as far as I have understood
wren so far, there isn't any problem. Stop me if I'm totally wrong:

Scenario 1:
1. C calls wrenCall or wrenInterpret
2. Wren calls Fiber.yield
3. ON C side, the return result of wrenCall or wrenInterpret should be
suspended instead of success
4. C may wrenCall again to resume the fiber, or why not call another one.

Scenario 2:
1. C calls wrenCall or wrenInterpret
2. Wren calls fiber.transfer; execution goes to another fiber
3. As far as I understand, fiber.transfer make the new executing fiber
don't retain where it was called from. So at this point, calling
Fiber.yield is whether an error, or returns directly to C with a
WrenInterpretResult=suspended, too
4. The C fonction stay in progress on the stack as long as the fiber
that called it don't get control back. It's up to you to react in case
wrenCall/wrenInterpret returned an error state in C. Probably you should
return immediately.

Reentrant or not, it doesn't change anyhting. In both cases you can add
a step 0 where Wren is calling C, and as many steps 0, 1, 0, 1... as you
wish.

Now imagine this one:
1. f1: C calls Wren
2. f1: Wren calls C
3. f1: C calls Wren
4. f1: Wren calls f2
If at this point f2.yields: we go back to f1, and if f1 yields we go
back to C as in scenario 1.
If f2 transfers to anything other than f1: it works as in scenario 2
IF f2 calls f1.transfer: do we have to short-circuit the c function
called between steps 2 and 3 ? in fact no, it cannot happen and the
check against it is already there. Consider this code:

var f1
var f2

f1 = Fiber.new{
System.print("Fiber 1 start")
f2.call()
System.print("Fiber 1 end")
}

f2 = Fiber.new{
System.print("Fiber 2 start")
f1.transfer() // error: fiber has already been called
System.print("Fiber 2 end")
}
f2.call()

So I don't see any special case where we could be in trouble. Whether
you use call and yield, or you use transfer, but not a mix. Even with 3
or more fibers we have no problem.

[Michel Hermier]

In short, there should be only one problematic scenario I can think of: if a fiber try to yield, with a least one C call in the stack. This should be promoted to an error, otherwise it should be ok.

The scenario 3 is *broken*, yield from upper later to C is only garantee to success if the C code is finished. And in this case C should transform the yield to an error.

I don't think it is frequent thought, I doubt a generator function, would yield outside of its body, and I doubt any native function will (be able to) yield.

Transfer should be safe since it does garantee the order of execution. So any pushed C call should be unstacked in order.

But maybe I'm mistaken, this is why it is interesting to confront it to the wild.

My patch series don't contain any sort of protection on *purpose*, I would like to see how it interact with real world usage first.

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B5CF009.1060901%40quentinc.net.

[Brian Slesinsky]

I don't need to do this, but here is another scenario to think about:

Suppose List.sort is implemented in C (using qsort) with a callback to Wren for each comparison. You wouldn't normally expect the callback to do I/O, but it's possible. Suppose someone wants to sort records in a table that's larger than memory? And now, suppose two different fibers call List.sort() to sort two different tables concurrently, and both callbacks do I/O? We effectively have to switch back and forth between two C-implemented qsort calls.

One way I can think of to allow this is to use OS-level threads and a global lock around the Wren interpreter. But this seems quite different from how the uv library does things?

To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+unsubscribe@googlegroups.com.

To post to this group, send email to wren...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B5CF009.1060901%40quentinc.net.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Wren" group.

To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+unsubscribe@googlegroups.com.

To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/CAAZ5spBTt%3Daa9FoSZqJV0rD192o%3Dk7RS8sn0OsFmiJOnofVU%3Dg%40mail.gmail.com.

[QuentinC]

Hello,

I have found a slight syntax inconsistancy imo.

This works:
if (true) {
} else {
}

But this don't:
if (true) {
}
else {
}

It says: error after 'else': expected expression
Note the slight difference: the fact that there is or not a newline
between } and else. This is a bit weird.

IN the same chapter, if we think on functional style programming, it's a
bit sad that chaining like this doesn't work either:

object
.method1()
.method2()
.method3()

while it works with all other usual languages.

The cause is probably the rule saying that the satement/expression
always ends at newline except if it's really impossible. It should
perhaps be the opposite, as a replacement or perhaps as addition: if a
statement/expression cannot begin with something, that's because it's
the continuation of the previous one.

This would make the syntax more friendly and less surprising, for
scripters who are used to other common programming languages. The 'else'
thing above is really weird.


While I'm talking about syntax, it may also be nice to add the
following, though it's of course much less important imo (probably in
order of difficulty):
- Allow optional ;
- Allow to declare multiple variables on a single line, e.g. var a=1,
b=2, c=3
- Strings can use " ' or `, e.g. System.print('Hello!')
- Allow any character >=128 to be part of identifier, e.g. var
jeDéveloppeEnFrançais="même si encourager le franglais est un peu dommage"

[Michel Hermier]

Not really an inconsistency, but razer a language (compiler) design choice. In wren code format is important. It is a little like python, but at a different level.

While I'm talking about syntax, it may also be nice to add the
following, though it's of course much less important imo (probably in
order of difficulty):
- Allow optional ;

Expend, optional might meet multiple things.

- Allow to declare multiple variables on a single line, e.g. var a=1,
b=2, c=3

This one should be pretty trivial.

- Strings can use " ' or `, e.g. System.print('Hello!')

It doesn't because it doesn't need to (yet). There is no motivation for alternate quotes for now.

- Allow any character >=128 to be part of identifier, e.g. var
jeDéveloppeEnFrançais="même si encourager le franglais est un peu dommage"

Quite complex, requires you to rely on a proper utf implementation, and they are huge external dependencies. And handling some languages where glyph are word might need extra care...

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B5D477C.3060502%40quentinc.net.

[Michel Hermier]

I updated the patch set. It is not perfect but should be usable for most
of your use cases. I don't have time to investigate more right now, and
my system as issue with gdb/valgrind and I don't want to go to the
printf debugging for now.

[QuentinC]

Hello,

Again another question, this one is a little stupid.
How to print a '%' ?

It seems that I'm obliged to write "%("%")".
A bit of a pain. As it was written in the documentation, if '%' isn't
followed by '(', it seemed to be normal, i.e. "%" would print the '%'.
In fact I'm getting an error saying that '%' must be followed by '('.

In my real project I was using %1, %2, etc. to do stuff with
translations, I will have to change it because of that.

[QuentinC]

Hello,

Thank you very much for updating the reentrancy patch.

I checked it out right now and it seems to work.

I also ran a very quick test script and it finally appears to be 10-12%
slower. I hope that it will still be accepted.

Thank you again.

[Michel Hermier]

Le lun. 30 juil. 2018 à 05:33, QuentinC <webm...@quentinc.net> a écrit :

Hello,

Again another question, this one is a little stupid.
How to print a '%' ?

Did you tried "\%" ?

It seems that I'm obliged to write "%("%")".
A bit of a pain. As it was written in the documentation, if '%' isn't
followed by '(', it seemed to be normal, i.e. "%" would print the '%'.
In fact I'm getting an error saying that '%' must be followed by '('.

In my real project I was using %1, %2, etc. to do stuff with
translations, I will have to change it because of that.

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/5B5E86ED.3060607%40quentinc.net.

[Michel Hermier]

It is slower, because of the API choices.

There is only one central object in the API: WrenVM. As such it is altering code to gain speed.

If you look at the patch series, it does basically 2 things:

- make C function registerable to wren fiber frames stack.

- make WrenVM::apiStack what it is really: FiberObj::stackStart.

This implies that every call, with current API design, have to do extra indirections to access the fiber data. And to fix that, FiberObj should be a more central piece in the public API. If you look at how the code become, and you see that accessing WrenVM is become more marginal (since it only contains the global States and memory management, which should be slower path).

Cheers.

[QuentinC]

Hello,

> Did you tried "\%" ?

Now I can feel stupid. \% works. I should ahve thought about it.
But it's written nowhere in the documentation

> It is slower, because of the API choices.
> There is only one central object in the API: WrenVM. As such it is
altering code to gain speed.
> If you look at the patch series, it does basically 2 things:
> - make C function registerable to wren fiber frames stack.
> - make WrenVM::apiStack what it is really: FiberObj::stackStart.
> This implies that every call, with current API design, have to do
extra indirections to access the fiber data. And to fix that, FiberObj
should be a more central piece in the public API. If you look at how the
code become, and you see that accessing WrenVM is become more marginal
(since it only contains the global States and memory management, which
should be slower path).

I had more or less understood the second item of what your patch does,
but not really the first one.

About speed: do you mean that, in fact, we should generally operate on a
fiber, rather than on the VM object, i.e. the first argument of most of
the API calls should be a fiber and not the VM ?
Or I haven't understood your explanation at all ?

[Michel Hermier]

From my point of view yes.While it add a little of complexity to the API, its more logical. Foreign are computational operation, they need to operate on the computational unit, aka fiber, aka virtual CPU. They need to know how to access the whole system resources it's true, but its less frequent then they need to access the stack data.

[Thorbjørn Lindeijer]

I think Lua API also operates on the "coroutine", which is similar to Wren's Fiber. At least the Lua state pointer is different when functions get called from different coroutines.

Cheers,
Bjørn

[Michel Hermier]

Looked quickly at Lua, and it seems, coroutines needs to be extracted from the state, which involves stack manipulation. But anyway even if they benefit from subclassing to implement coroutines, I think it is a confusing solution. At some point one API will refuse some pointers because they are not of the valid subtype.

But yes, I agree that every callback should be invoked with a different context object per fiber at minimum. As such FiberObj is the obvious candidate.

It is also more future proof of we have multithread support someday... But we can already do a lot with the callbacks in single thread, so it is long long long term idea.

Cheers,
Bjørn

--
You received this message because you are subscribed to the Google Groups "Wren" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wren-lang+...@googlegroups.com.
To post to this group, send email to wren...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/wren-lang/1532969834.812724.1457635984.0A35989E%40webmail.messagingengine.com.

[QuentinC]

Hello,

I have just submitted a pull request, adding new functions in the C API
regarding type checking, as we ahve discussed a few days ago on this list.

This is my first pull request, I never participated in any open source
project before, so I hope I did it correctly. Of course it is open for
comments, whether here or on the pull request page.

https://github.com/munificent/wren/pull/591

[QuentinC]

Hello,

After having almost succeeded in implementing compound operators += *=
etc. including ||= and &&=, I noticed that there was no continue keyword
in Wren. In comparison to those it should be pretty easy to add.

I'm trying to implement it and I don't understand what I could have
missed. It works well with while loops but not for loops.

My quick test wren code:
for (i in 1..5) {
System.print("Before: %(i)")
if (i==3) continue
System.print("After: %(i)")
}

Should obviously print this:
Before: 1
After: 1
Before: 2
After: 2
Before: 3
Before: 4
After: 4
Before: 5
After: 5

But it prints this:
Before: 1
After: 1
Before: 2
After: 2
Before: 3
Before: 3
Before: 3

Once the continue ahs been executed, the after print isn't executed any
longer, and more strange, i stays stuck at 3 although the loop is
correcty ran 5 times.

In trying to understand the problem, I changed the wren code to this:

var generator = Fiber.new{
for (i in 1..5) {
System.print("Yielding %(i)")
Fiber.yield(i)
}
}

for (i in generator) {
System.print("Before: %(i)")
if (i==3) continue
System.print("After: %(i)")
}

The output become really weird:
Yielding 1
Before: 1
After: 1
Yielding 2
Before: 2
After: 2
Yielding 3
Before: 3
Yielding 4
Before: 3
Yielding 5
Before: 3

Do someone has an idea of what happens ?
Do you know why once the continue instruction has been executed, the
loop variable doesn't seem to be updated any longer ?


Note: I have made Fiber extending Sequence and added appropriate iterate
and iteratorValue methods so that I can write for(i in generator). This
works well and was easy to add in wren_core.wren.


Here's the interesting part of wren_compiler.c:

void statement(Compiler* compiler)
{
...
else if (match(compiler, TOKEN_CONTINUE))
{
if (compiler->loop == NULL)
{
error(compiler, CSTR("Cannot use 'continue' outside of a loop."));
return;
}
// Code taken from function endLoop. This should jump back to the
beginning of the loop.
int loopOffset = compiler->fn->code.count - compiler->loop->start + 2;
emitShortArg(compiler, CODE_LOOP, loopOffset);
}

Thank you !

[Michel Hermier]

I suspect you have stack corruption happening. This is why it was not done already, and maybe also for other reasons.

[QuentinC]

Hello,

> I suspect you have stack corruption happening. This is why it was not
done already, and maybe also for other reasons.

This was effectively correct. Thank you for putting me on the right
track, I was unaware of this. In my particular case I had to pop one
element in excess before jumping, and now it works as expected.

For the general solution, the evolution of the stack is already tracked
in compiler->numSlots, so in fact most of the job is already done. When
the loop begins, I store the current value of compiler->numSlots in the
loop. When encountering a continue, before emitting the jump back to the
beginning of the loop, I pop as many elements as the difference between
the current value of compiler->numSlots and its value when the loop
began. To be completely sure I conversely push null values in case the
difference is negative, but in fact I don't think it can happen.

[Michel Hermier]

Put an assert instead, it should really never happen unless there is a compiler error.