Javascript. What is better 'global action' or hundred small 'actions'?

[Vik]

Hi,
If I have in scene hundred of similar objects (for example door.
001,...door.099), i can attach to each object javascript for actions
(say "OpenDoor"). Or I can attach one script to scene and in this
script distinguish door by name at run time.
Which solution is better. I mean 'better" from architectural point
of view of Tundra in terms of performance and memory consumption.
Thanks
Vik.

[Toni Alatalo]

On Thu, 2012-03-29 at 09:41 -0700, Vik wrote:
> 001,...door.099), i can attach to each object javascript for actions
> (say "OpenDoor"). Or I can attach one script to scene and in this
> script distinguish door by name at run time.
> Which solution is better. I mean 'better" from architectural point
> of view of Tundra in terms of performance and memory consumption.

I tend to make with one master JS and one EC_Script for the whole
application. I like normal programming, and also that used to be way to
make it efficiently (minimal mem use as there's only one ScriptEngine).
Have that in an invisible entity called MyGame or so.

The system is however made so that using a Script component in each
scripted entity can be nice. For example for a non-programming level
designer to put a script to an object. And nowadays there's the
ScriptApplication system, which basically allows you to make several
script components to be executed with a single engine, so you get the
same optimally minimal memory usage. I think it's well suited for
instanciating several entities that use the same script, basically
declaring a class in JS and then marking entities in the scene as
instances of that. Similar to the RexScript IronPython system in old
realXtend with Linden based viewer and Opensim which is there in ModRex.

One cool upside of using several EC_Scripts can perhaps be that the live
reloading may work less instrusively as only one script engine is
reseted and not the whole app. For example if you make AI for a predator
and it's prey, can modify either code, just save the file so it gets
reloaded in Tundra, and at least the other animal keeps working cleanly
from the old state as it was untouched. It's possible to make a main app
support restart too, quite easy perhaps even if keep the data in EC
attributes (script's can create their own so-called DynamicComponents
for custom data). The attribute values are ofc untouched when the script
engine is reseted, so that state can always stay (position of the
objects etc., but if you put e.g. some AI mode or tracking target in own
DynamicComponent, that stays too.)

> Vik.

~Toni

[Vik]

Thank you, Tony
I will go with single EC_Script ( normal programming :) )
Even if ScriptEngine is performant, load hundreds of EC_Script may be
painful ( as Jukka said in issue 439 https://github.com/realXtend/naali/issues/439
)
And I don't think in this case live reloading will be useful...
Vik.

[Jonne Nauha]

Single EC_Script absolutely, much more efficient speed and memory wise. Our script engines are not particularly cheap.

Another option is to use the app name and app class in EC_Script, this way you can have the component in all of the doors but they share the same script engine. With this tech "me" in the script refers to the entity that has the actual scriptRef set. But every instance will get a conatructor call that passes the real "me" entity. This might sound complicated but it's really not, check out scenes/Avatar script files. At least simpleavatar.js uses this tech.

Best Regards,
Jonne Nauha
Adminotech developer

> --
> http://groups.google.com/group/realxtend
> http://www.realxtend.org

[Vik]

Ok, I will look how simpleavatar.js work...

Thank Jonne,

Vik

On Mar 29, 3:00 pm, Jonne Nauha <jo...@adminotech.com> wrote:
> Single EC_Script absolutely, much more efficient speed and memory wise. Our script engines are not particularly cheap.
>
> Another option is to use the app name and app class in EC_Script, this way you can have the component in all of the doors but they share the same script engine. With this tech "me" in the script refers to the entity that has the actual scriptRef set. But every instance will get a conatructor call that passes the real "me" entity. This might sound complicated but it's really not, check out scenes/Avatar script files. At least simpleavatar.js uses this tech.
>
> Best Regards,
> Jonne Nauha
> Adminotech developer
>

[Toni Alatalo]

On Thu, 2012-03-29 at 11:57 -0700, Vik wrote:
> I will go with single EC_Script ( normal programming :) )
> Even if ScriptEngine is performant, load hundreds of EC_Script may be
> painful ( as Jukka said in issue 439 https://github.com/realXtend/naali/issues/439

No the point was that using the Script *Application* mechanism with
multiple EC_Scripts is performant -- then you again only get a single
engine. The same thing that Jonne mentioned and indeed the reference
Avatar app uses.

But yep I think a single EC_Script can still be nice.

> And I don't think in this case live reloading will be useful...

I think it's always useful when developing :)

And you do get that with a single Script component too, it just resets
the whole app then.

> Vik.

~Toni

[Jonne Nauha]

Yes you will want live realoding. You will go crazy if you need to restart client/server all the time when you save file. Script live reloading is the best feature Tundra has for devs :)

This other approach that simpleavatar.js uses gives you the same kind of "this.me" == entity the script is attached to feeling (you need to assign this.me as youll see in simpleavatar.js ctor). Without running huge number of script engines. So really, its how you like to program. I like to make more these separate apps that run in their own entities and keep their data component (EC_DynamicComponent) in the script entity. Then internal logic modifies/creates/removes entities/component how it needs. So I'm used to going "me.something" only for my data component, and for that I always make getter inside the app this.DataComponent(); :P

Another neat thing if you want "real object" in your scripts is class.js that ships with Tundra. I basically use this in all my scripts. I haven't used it in the example scenes I've made as they should be kept simple. But the browser ui js I made shows this quite well. This also has some advanced UI stuff if you want to learn that :)

Defining class https://github.com/realXtend/naali/blob/tundra2/bin/jsmodules/browserui/LoginBrowser.js#L26. You define init: function() that gets called when the class is new'ed, its little bit like python __init__. Main intance owns other objects https://github.com/realXtend/naali/blob/tundra2/bin/jsmodules/browserui/LoginBrowser.js#L36. Booting up the app https://github.com/realXtend/naali/blob/tundra2/bin/jsmodules/browserui/LoginBrowser.js#L1956

Best regards,
Jonne Nauha
Adminotech developer

[Vik]

Thanks for all your comments,
I made some test for compare single vs. multiple.

This test add some hundreds of fishes to Avatar's scene, using two
techniques discussed here.
add_dyn.js ( http://pastebin.com/knpit5H3 ), Fish.js (
http://pastebin.com/8AnFdB9p ), and GlobalFish.js ( http://pastebin.com/RhNB9jLe
)
You can change technique in line 3 of add_dyn.js. Change How many
fiches add by line 2 (n*n fish will be added) . Using scene.txml from
Avatar examlple, some fiche will fall (need to make bigger floor).
Added second avatar for demo purposes.
Start Tundra in command line with param --run add_dyn.js. Or use
console...

Some observation:
Adding 10x10 fish is ok by two method. But adding 20x20 fish with
Fish.js (many instance of javascript) will fail with error "Could not
allocate register file N" where N is different 2, 33.
I would like to know what this mean?
Using GlobalFish.js - we add single js, but instantiate many instance
by EC_SCRIPT, as Jonne explained. This method permit add as many as
50x50 fish without problems...

I have also question about how to know when scene is ready for
manipulation. I used some sort of tentative every 1 sec... It is
correct or i can use more clean approach?

Thanks.
Vik
Vik

On Mar 30, 2:18 am, Jonne Nauha <jo...@adminotech.com> wrote:
> Yes you will want live realoding. You will go crazy if you need to restart
> client/server all the time when you save file. Script live reloading is the
> best feature Tundra has for devs :)
>
> This other approach that simpleavatar.js uses gives you the same kind of "
> this.me" == entity the script is attached to feeling (you need to assign
> this.me as youll see in simpleavatar.js ctor). Without running huge number
> of script engines. So really, its how you like to program. I like to make
> more these separate apps that run in their own entities and keep their data
> component (EC_DynamicComponent) in the script entity. Then internal logic
> modifies/creates/removes entities/component how it needs. So I'm used to
> going "me.something" only for my data component, and for that I always make
> getter inside the app this.DataComponent(); :P
>
> Another neat thing if you want "real object" in your scripts is class.js
> that ships with Tundra. I basically use this in all my scripts. I haven't
> used it in the example scenes I've made as they should be kept simple. But
> the browser ui js I made shows this quite well. This also has some advanced
> UI stuff if you want to learn that :)
>

> Defining classhttps://github.com/realXtend/naali/blob/tundra2/bin/jsmodules/browser....

> You define init: function() that gets called when the class is new'ed, its

> little bit like python __init__. Main intance owns other objectshttps://github.com/realXtend/naali/blob/tundra2/bin/jsmodules/browser....
> Booting up the apphttps://github.com/realXtend/naali/blob/tundra2/bin/jsmodules/browser...

[Jonne Nauha]

Hi, good looking scripts you have there. I can give you some tips, i'll do it via pastebin with code comments, this might be most clear way of doing it. But first:

Adding 10x10 fish is ok by two method. But adding 20x20 fish with
Fish.js (many instance of javascript) will fail with error "Could not
allocate register file N" where N is different 2, 33.
I would like to know what this mean?

I have seen this couple of times myself. I am 99% sure its due to out of memory. 20x20 script engines = 400, lets say one takes up 2 megs of memory (I don't know the actual figures, might even take more) would be 800 megs of mem. I usually see "catastrofic failure" on Tundra.exe at around 1,3 gigs of memory usage for the process, and out of memory exceptions start to pop from Tundra or from Ogre. Basically you will dump. Did you btw get a dump dialog or did it continue to run happily?

I think Jukka is referring to the same engine register file issue here https://github.com/realXtend/naali/issues/444 "often inside QScriptEngine register file allocation code", this would also indicate out of memory situation. So as we knew and see here clearly this amount of script engines is a very bad approach for your problem and the single engine is excellent. The less engines you have the better. But id say in a moderately heavy scene you can still have 10-15 engines doing their own things, but 400 (or 50x50=2500) is clearly too much.

I have also question about how to know when scene is ready for
manipulation.  I used some sort of tentative every 1 sec... It is
correct or i can use more clean approach? 

I will go over this and some other stuff that popped to me while reading you scripts. I prefer paste.pocoo.org for code pastes, easier to read than pastebin because its wider.

• add_dyn.js http://paste.pocoo.org/show/573825/ (see the original AddRigidBodyToEntities() implementation, i added some useful info)
• The other scripts look fine to me. You got the idea how to do it with both ways :) Looks good to me. You are correctly only doing the rotation on the server which is important so client and server wont do it. And if you would manipulate rigidbody like avatar app does, it has to be done on the server as only it has the authority to do it.

One this is that you might want to move away from --run parameter. Its fine for you to test out things of couse if you prefer it that way. But for you app here, I would make a txml that has only add_dyn.js as a server side script + avatar application ready to go, here is a example of it (you can copy this to your data/assets or where you'd like as a txml).

Scene http://paste.pocoo.org/show/573829/ Save this as /bin/data/assets/fishes.txml

add_dyn.js for this kind of usage http://paste.pocoo.org/show/573834/ this is short and cleaned version of you script, same outcome. Save this as /bin/data/assets/add_dyn.js

No open cmd prompt to and run these:

Tundra.exe --server --headless --console --file data/assets/fishes.txml

Tundra.exe --config viewer.xml --login tundra://localhost/?username=Vik

Ask if you have any questions. I don't know if I went a bit overboard here :P

Best regards,
Jonne Nauha
Adminotech developer

[Jonne Nauha]

Had some typos there, I'm sure the point goes across anyways.

One correction/addition. If you don't want to copy paste all files from scenes/Avatar to where your txml is (so local://simpleavatar.js etc will be found, you can do this. This will add and additional asset storage so the assets will be found from there when the scene is loaded. I actually think you dont even need it to the client configure because Tundra server always replicates its storage information to the client during login procedures (you cant test to leave it out if you want).

Tundra.exe --server --headless --console --storage scenes/Avatar --file data/assets/fishes.txml

Tundra.exe --config viewer.xml --storage scenes/Avatar --login tundra://localhost/?username=Vik

[Vik]

Thank you Jonne for your comments - they are really helpful.
For example, I never thought to put sturtup() at the end of the file
- for me it was more clearly put in front :) Also lot of unnecessary
GetComponent("EC_Name")...
About parameter --run and AddRigidBody. This is due to the fact that I
have exported scene from some works in a blender. This export do not
add RigidBody, so I decided to add it for some element at startup in
JavaScript. Another reason - this script will change the scene on the
server, and thus, the client should not know about this JS. If I added
it to the scene, as you say, I have to give it to the client as well.
But I want to put some validation for UserConnection (check username
and password). I did this already in Python (as well using command
line option --python filename.py). But in the latest version, Python
has been temporarily removed, and I thought about validating users in
the same JS script through qt.webkit as Tony pointed me in another
thread.
About --storage options - I use it, and I have a question about this.
When I start Tundra server in the remote real server machine, I run
the client with option --storage src=http://someserver.com/webdav. In
this case, I need to replace all assets references 'local://somefile'
with simple 'somefile', if not, this 'somefile' refuse to the auto-
download from server ... I have other solutions for the local://
thing?

Regards,
Vik.

On Mar 30, 6:33 pm, Jonne Nauha <jo...@adminotech.com> wrote:
> Hi, good looking scripts you have there. I can give you some tips, i'll do
> it via pastebin with code comments, this might be most clear way of doing
> it. But first:
>
> Adding 10x10 fish is ok by two method. But adding 20x20 fish with
>
> > Fish.js (many instance of javascript) will fail with error "Could not
> > allocate register file N" where N is different 2, 33.
> > I would like to know what this mean?
>
> I have seen this couple of times myself. I am 99% sure its due to out of
> memory. 20x20 script engines = 400, lets say one takes up 2 megs of memory
> (I don't know the actual figures, might even take more) would be 800 megs
> of mem. I usually see "catastrofic failure" on Tundra.exe at around 1,3
> gigs of memory usage for the process, and out of memory exceptions start to
> pop from Tundra or from Ogre. Basically you will dump. Did you btw get a
> dump dialog or did it continue to run happily?
>

> I think Jukka is referring to the same engine register file issue herehttps://github.com/realXtend/naali/issues/444"often inside QScriptEngine

> register file allocation code", this would also indicate out of memory
> situation. So as we knew and see here clearly this amount of script engines
> is a very bad approach for your problem and the single engine is excellent.
> The less engines you have the better. But id say in a moderately heavy
> scene you can still have 10-15 engines doing their own things, but 400 (or
> 50x50=2500) is clearly too much.
>
> I have also question about how to know when scene is ready for
>
> > manipulation.  I used some sort of tentative every 1 sec... It is
> > correct or i can use more clean approach?
>
> I will go over this and some other stuff that popped to me while reading
> you scripts. I prefer paste.pocoo.org for code pastes, easier to read than
> pastebin because its wider.
>

>    - *add_dyn.js*http://paste.pocoo.org/show/573825/(see the original

>    AddRigidBodyToEntities() implementation, i added some useful info)

>    - The other scripts look fine to me. You got the idea how to do it with

>    both ways :) Looks good to me. You are correctly only doing the rotation on
>    the server which is important so client and server wont do it. And if you
>    would manipulate rigidbody like avatar app does, it has to be done on the
>    server as only it has the authority to do it.
>
> One this is that you might want to move away from --run parameter. Its fine
> for you to test out things of couse if you prefer it that way. But for you
> app here, I would make a txml that has only add_dyn.js as a server side
> script + avatar application ready to go, here is a example of it (you can
> copy this to your data/assets or where you'd like as a txml).
>

> Scenehttp://paste.pocoo.org/show/573829/*Save this as
> /bin/data/assets/fishes.txml*

> add_dyn.js for this kind of usagehttp://paste.pocoo.org/show/573834/this

> is short and cleaned version of you script, same outcome. *Save this as
> /bin/data/assets/add_dyn.js*

>
> No open cmd prompt to and run these:
>
> Tundra.exe --server --headless --console --file data/assets/fishes.txml
> Tundra.exe --config viewer.xml --login tundra://localhost/?username=Vik
>
> Ask if you have any questions. I don't know if I went a bit overboard here
> :P
>
> Best regards,
> Jonne Nauha
> Adminotech developer
>
>
>
>
>
>
>
> On Fri, Mar 30, 2012 at 11:00 PM, Vik <vyetr...@gmail.com> wrote:
> > Thanks for all your comments,
> > I made some test for compare single vs. multiple.
>
> > This test add some hundreds of fishes to Avatar's scene, using two
> > techniques discussed here.
> >  add_dyn.js   (http://pastebin.com/knpit5H3), Fish.js (

> >http://pastebin.com/8AnFdB9p), and GlobalFish.js (

> ...
>
> read more »

[Jonne Nauha]

About parameter --run and AddRigidBody. This is due to the fact that I
have exported scene from some works in a blender. This export do not
add RigidBody, so I decided to add  it for some element at startup in
JavaScript. Another reason - this script will change the scene on the
server, and thus, the client should not know about this JS. If I added
it to the scene, as you say, I have to give it to the client as well. 

You will get the same effect inside the txml by setting the scripts entity as Local entity (and its components). Run the headless server, run console commans 'scenestruct', right click on the script entity -> Convert To Local. Then right click somewhere in the editor -> Save scene as... and overwrite the fishes.txml or just makea copy. Close the server and run this new txml, connect a client and you should see that entity never gets replicated to the client. This is what you wanted right?

About --storage options - I use it, and I have a question about this.
When I start Tundra server in the remote real server machine, I run
the client with option  --storage src=http://someserver.com/webdav. In
this case, I need to replace all assets references 'local://somefile'
with simple 'somefile', if not, this 'somefile' refuse  to the auto-
download from server ... I have other solutions for the local://
thing?

Yes, but you need to give a bit longer version of --storage. It goes something like this --storage name=ViksWebStorage;src=http://someserver.com/webdav/;type=HttpAssetStorage;trusted;default;

This should give you the desired effect so yes, whenever you have a relative asset ref (mesh, script, material, whatever) as "myfish.mesh" without local:// or http:// it will go to the default storage (marked with default; in the --storage param) it will try to fetch it from http://someserver.com/webdav/myfish.mesh. And you should only need to set this --storage of course on the server and it will replicate the storage to clients. Someone who knows could answer what happens if you have "scripts/add_finshes.js" as a ref, will it go to http://someserver.com/webdav/scripts/add_finshes.js. I havent used the default storage + relative ref combo a lot myself. I would assume that is what happens and its quite easy for you to test. If not then everything needs to be kept in the same web directory.

But I want to put some validation for UserConnection (check username
and password). I did this already in Python (as well using command
line option  --python filename.py). But in the latest version, Python
has been temporarily removed, and I thought about validating users in
the same JS script  through qt.webkit as Tony pointed me in another
thread.

Yes Python is disabled for now on windows releases until the dependencies build are automated by the module authors. But python is kind of a heavy hitter, it depends on how you going to authenticate the users really. If you have some database backend that is easy to access with some nice python lib then its probably the way.

If you have simple needs like a global password that every user mush know to get in it is simply ~10 lines of javascript, and you can have similar Local entity on server that never goes to clients (so they cant snoop the file and read the password. Or you can have user "database" in the file if you dont have many users.

One thing JS does very well is HTTP requests, so if you have a backend where you can query "is this guys password correct" you simply do something like 'var transfer = asset.RequestAsset("http://someserver.com/auth?username=<username>&password=<pwd>");' Make that transfer not cache and once completes read the HTTP response body in JS.

So it depends hugely what is your intent and backend for authentication. If you can provide some more info for me I can help you make the auth JS. Hope I can get account too so I can check you fishworld :)

Best regards,
Jonne Nauha
Adminotech developer

[Vik]

O, thanks. This is very good tips - 'Convert to Local'. It really
what I need!

> Yes, but you need to give a bit longer version of --storage. It goes

> something like this *--storage name=ViksWebStorage;src=http://someserver.com/webdav/;type=HttpAssetStorage;trusted;default;*

> This should give you the desired effect so yes, whenever you have a
> relative asset ref (mesh, script, material, whatever) as "myfish.mesh"
> without local:// or http:// it will go to the default storage (marked with

> default; in the --storage param) it will try to fetch it fromhttp://someserver.com/webdav/myfish.mesh. And you should only need to set

> this --storage of course on the server and it will replicate the storage to
> clients.

About --storage - I don't get it.
When I start server i load .txml from some folder, where all the
assets resides. ./Tundra --server --file /var/www/web/myserver/
mytxml.txml. And this will be default storage for server. If
additionally i specify *--storage name=ViksWebStorage;src=http://
someserver.com/webdav/;type=HttpAssetStorage;trusted;default;* for
this server it will start download for my server from http://someserver.com/webdav/
? I think this need to be done only on client side. Client can be
started like: * Tundra --connect http://someserver.com/webdav;2345;
udp;vik;vik --storage name=ViksWebStorage;src=http://someserver.com/
webdav/;type=HttpAssetStorage;trusted;default;*

I tried to set dropbox folder as storage, but i can't get it work
without specify storage on client. If I specify on client side this
public dropbox storage it go well...

As for user authentication, I simply used MySQLDb with Python on the
server.

> One thing JS does very well is HTTP requests, so if you have a backend
> where you can query "is this guys password correct" you simply do something
> like 'var transfer = asset.RequestAsset("http://someserver.com/auth?username=<username>&password=<pwd>");' Make that
> transfer not cache and once completes read the HTTP response body in JS.

I have not thought about asset.RequestAsset. In place, with JS I used
QNetworkAccessManager
* var netmanager= new QNetworkAccessManager(null);
netmanager.get( new QNetworkRequest(new QUrl(urldb+"/
getUsersInfo.php")) *
May be your solution with RequestAsset will be better...

> Hope I can get account too so I can check you fishworld :)

Sorry, but it's too early for fishing. Fish should grow, as I just
started to deep into realXtend :)
Vik.