node loader in V2.0

[Datenbank archINFORM]

Hi,
playing around with the new sceneJS version. Now I'm trying to load
nodes (texture+geometries) asynchronously on demand. Tried the Node
Loader service, but there You have to create an instance node for
requesting the stuff. And instance nodes are not a part of SceneJS 2
anymore :(
Is the Node Loader service deprecated, too?

Sincerely, Sascha

[Lindsay Kay]

Yes instance nodes were removed in V2.0 and replaced by shareable node cores:

https://github.com/xeolabs/scenejs/wiki/Node-Cores

The problem with instance nodes was that they created multiple parent
paths in the scene graph, which complicated all sorts of things and
stood in the way of performance optimisations.

Although they were really convenient, but I found that the sort of
reuse they provided could be actually managed by application code
outside of SceneJS, ie. application-managed pools of scene JSON
fragments.

If you really need scene content pulled in on demand though, let me
know what your use case is..

cheers,
LK

[Datenbank archINFORM]

Hi Lindsay,

thanks for the immediate answer.
Is it maybe possible to user the Node Loader service with the new Node-
Cores (or in another way)?
I need on demand load of textures and geometries for a 3D map viewer
(spatial building models mixed with flat mapfaces). If the user moves
on the map more mapfaces & models should be loaded.

Sascha

On Nov 18, 3:52 pm, Lindsay Kay <lindsay.stanley....@gmail.com> wrote:
> Yes instance nodes were removed in V2.0 and replaced by shareable node cores:
>
> https://github.com/xeolabs/scenejs/wiki/Node-Cores
>
> The problem with instance nodes was that they created multiple parent
> paths in the scene graph, which complicated all sorts of things and
> stood in the way of performance optimisations.
>
> Although they were really convenient, but I found that the sort of
> reuse they provided could be actually managed by application code
> outside of SceneJS, ie. application-managed pools of scene JSON
> fragments.
>
> If you really need scene content pulled in on demand though, let me
> know what your use case is..
>
> cheers,
> LK
>

[Lindsay Kay]

On-demand loading of geometries can be done with a "GeoLoaderService":

http://scenejs.org/dist/v2.0.0/extr/examples/geo-load-service/index.html

Nothing like that for textures yet..

SceneJS encourages staging content "in-core", ie. staging content in
the scene graph, setting "enabled" flags false to cull it from the
view until you want to render it, Eg:

{
type: "flags",
id: "my-teapot-content",
flags: { enabled: false },
nodes: [
{ type: "teapot" }
]
}

myScene.findNode("my-teapot-content").set("flags", { enabled: true });

If your scene is too big to stage everything in the scene graph, then
swapping in and out must be done by creating and destroying nodes:

scene.findNode("some-node").add("node", { type: "teapot", id: "my-teapot" });

scene.findNode("my-teapot").destroy();

Bear in mind that this will cause scene-to-draw list recompilation (an
internal optimisation) each time, so these add/destroys are best done
in batches, and will cause a moment's delay when they happen.

(I'm going to blog this since it's a good point of discussion!)

[Datenbank archINFORM]

> On-demand loading of geometries can be done with a "GeoLoaderService":
> http://scenejs.org/dist/v2.0.0/extr/examples/geo-load-service/index.html
> Nothing like that for textures yet..

I'm looking for a Load service for whole subnodes (with texture,
geometry, name, etc). Such a service, independent of the node type,
would be great.

> SceneJS encourages staging content "in-core", ie. staging content
in
> the scene graph, setting "enabled" flags false to cull it from the
> view until you want to render it, Eg:

Can't preload the whole world ;)

> If your scene is too big to stage everything in the scene graph, then
> swapping in and out must be done by creating and destroying nodes:
> scene.findNode("some-node").add("node", { type: "teapot", id: "my-teapot" });
> scene.findNode("my-teapot").destroy();
> Bear in mind that this will cause scene-to-draw list recompilation (an
> internal optimisation) each time, so these add/destroys are best done
> in batches, and will cause a moment's delay when they happen.

OK, so I have to load the data (as JSON) with own AJAX functions and
include it that way. Btw, does the GEOLOADER service works without
draw list recompilation?

Sascha

[Datenbank archINFORM]

another question to your comment:

> > If your scene is too big to stage everything in the scene graph, then
> > swapping in and out must be done by creating and destroying nodes:
> > scene.findNode("some-node").add("node", { type: "teapot", id: "my-teapot" });
> > scene.findNode("my-teapot").destroy();
> > Bear in mind that this will cause scene-to-draw list recompilation (an
> > internal optimisation) each time, so these add/destroys are best done
> > in batches, and will cause a moment's delay when they happen.

If I have to do several node inserts/updates/deletes, can I "stop" the
rendering engine (to avoid multiple list recompilations) and after
node manipulation switch it back on?

Sascha

[Lindsay Kay]

The GeoLoader wont recompile the scene when geometry arrives - just
triggers a draw list redraw, so that's efficient.

Stopping the rendering engine while adding nodes can be a good idea.
Not sure, but the rendering interval might execute in the middle of
your node destroys otherwise.

Possibly also good to stop rendering during destroys.

Looks like you'll need to make some application code to swap in/out
"out-of-core" nodes..

[Lindsay Kay]

EDIT: I mean "rendering interval might execute in the middle of your
node *additions* otherwise"

[Lindsay Kay]

I wrote a blog article on the general subject: 

http://blog.xeolabs.com/80736697

[Datenbank archINFORM]

Hi Lindsay,
wow, what kind of feedback: a whole blog entry in response of a small
newsgroup question. Thanks!
Now I have implemented my own node loader and adding nodes to the
scene works pretty good (even without scene.stop & scene.start). But
if I try to delete a subnode with node.remove("node","subnoteId") the
sceneJS engine fails. If I use scene.stop() before doing that, the
following error is send to console:

Uncaught Error: TYPE_MISMATCH_ERR: DOM Exception 17
scene.js Line 7058 (line content:
requestAnimFrame(window[fnName]); )

If I don't stop scene rendering before the remove, there it fails
without error message.

Sascha

[Lindsay Kay]

Hi,

sounds dire..can you reproduce the error in a simple example and
publish it in a gist or a jsFiddle?
Then I'll debug and fix.
cheers,LK

[Datenbank archINFORM]

Sorry I misunderstood .insert method. Had a closer look on
the docs and understand now, that the subnode will be inserted ABOVE
the mainnode.

Doc: https://github.com/xeolabs/scenejs/wiki/Node-Accessor-Methods
".insert: Insert a node above child nodes"

Sascha

On Nov 19, 2:34 pm, Lindsay Kay <lindsay.stanley....@gmail.com> wrote:

> Hi,
>
> sounds dire..can you reproduce the error in a simple example and
> publish it in a gist or a jsFiddle?
> Then I'll debug and fix.
> cheers,LK
>

[Lindsay Kay]

Hey no problem - everything working OK now?

[Datenbank archINFORM]

Hi Lindsay,

some problems solved with this, but have some more. Have You got the
links with the examples?
Updated the ..._work.htm example to show the issues:

1.) If You move a while in a certain direction new tiles are the gonna
loaded. On displaying the tiles You will see that they are flickering.
The initial state of the faces is not the plain material color or
default texture. It seems, that they grab the last loaded texture at
the moment. After all textures are loaded everything looks fine.

2.) I interpolate normals between neighboor terrain faces. This way
every vertex normal of a triangle have it's own direction. Rendering
looks fine in Chrome (Firefox won't display these faces), but there is
a weird effect. Some areas loose texture and shading and displayed in
pure basecolor. This depends from viewpoint. If You move in the
example you will see this "snow" on the mountains. Seems like an shade
problem/bug.

3.) If You set for the scene look:{x:0;y:0:z:0} and
eye:{x:0;y:100:z:0} with up:{x:0;y:1:z:0} nothing will be displayed.
With a minimal offset for example eye:{x:0.01;y:100:z:0.01} it works
perfect.

Many thanks for your help,
Sascha

2011/11/24 Lindsay Kay <lindsay.s...@gmail.com>:

--
www.archINFORM.net
Postfach 540103
D-10042 Berlin
Telefon: +49 (30) 44308505
Telefax: +49 (30) 44308178
arc...@archINFORM.de

[Datenbank archINFORM]

Hi again,

after further investigating, there are some news:

> 1.) If You move a while in a certain direction new tiles are the gonna
> loaded. On displaying the tiles You will see that they are flickering.
> The initial state of the faces is not the plain material color or
> default texture. It seems, that they grab the last loaded texture at
> the moment. After all textures are loaded everything looks fine.

Problem still exists, no idea what causing this behaviour. Help
needed.

> 2.) I interpolate normals between neighboor terrain faces. This way
> every vertex normal of a triangle have it's own direction. Rendering
> looks fine in Chrome (Firefox won't display these faces),

Firefox is working now. Problem was a prototype command
(object.clone). Not related to SceneJS. Sorry.

> but there is
> a weird effect. Some areas loose texture and shading and displayed in
> pure basecolor. This depends from viewpoint. If You move in the
> example you will see this "snow" on the mountains. Seems like an shade
> problem/bug.

This problem only occurs, if I use the predefined teapot as example
object. If I use another object (cube, sphere, self defined geometry)
no "snow" shows up. Maybe there is a bug in the teapot geometry
causing the weird shader behaviour. Btw I noticed that the teapot
frontfaces are cw instead of default ccw.

> 3.) If You set for the scene look:{x:0;y:0:z:0} and
> eye:{x:0;y:100:z:0} with up:{x:0;y:1:z:0} nothing will be displayed.
> With a minimal offset for example eye:{x:0.01;y:100:z:0.01} it works
> perfect.

Haven't found the bug in the SceneJS source, but no problem to use a
small offset as workaround

Sascha

[Datenbank archINFORM]

Another update on the update:

the reason for the shader problem seems to be missing uv data. I used
the teapot as a subnode of a texture node and the teapot obviously
contains no uv information.
If I use a geometry with uv data (sphere, cube, selfdefined) shading
works. If use the teapot without a parent texture node, shading works,
too.

Sascha

[Lindsay Kay]

Hi Sascha
I'll just knock off (3) because I think that's an easy one:

>> 3.) If You set for the scene look:{x:0;y:0:z:0} and>> eye:{x:0;y:100:z:0} with up:{x:0;y:1:z:0} nothing will be displayed.>> With a minimal offset for example eye:{x:0.01;y:100:z:0.01} it works>> perfect.> Haven't found the bug in the SceneJS source, but no problem to use a> small offset as workaround

Yes that's to be expected I think - the parameters for a "lookat"
transform will always require the UP vector to be orthogonal to the
EYE->LOOK vector. You'll get weird results if the vectors point the
same direction as in that case, where UP points back at EYE.

> Another update on the update:>> the reason for the shader problem seems to be missing uv data. I used> the teapot as a subnode of a texture node and the teapot obviously> contains no uv information.> If I use a geometry with uv data (sphere, cube, selfdefined) shading> works. If use the teapot without a parent texture node, shading works,> too.

Aha - that's a problem with SceneJS automatic shader generation, which
is actually top of the list for improvement. Good info, I can debug
from that.
Could you please send me those URLs again? Sorry I can't find them in
my mail, I might have lost them..

[Datenbank archINFORM]

Hi Lindsay,

here is an updated version:

http://deu.archinform.net/service/scjs3.htm

backface culling is disabled, you will see inside out teapot
teapot has a texture parent, which causes shading porblems (snow on
the mountains)
weird initial textures onload (if you move new tiles will be loaded
and some existing ones will be replaced with higher resolution)

some instructions at page end

Thanks for looking,
Sascha

PS: Seems like I haven't understand the up vector right now. Why it is
not possible to look on a scene exactly from above (eye-look vector =
(negative) up vector)?

2011/12/8 Lindsay Kay <lindsay.s...@gmail.com>:

--

[Lindsay Kay]

> PS: Seems like I haven't understand the up vector right now. Why it is
> not possible to look on a scene exactly from above (eye-look vector =
> (negative) up vector)?

You can, but you need to set it up something like this:

{
type: "lookAt",
eye: { x: 0, y: -100, z: 0 },
look: { x: 0, y: 0, z: 0 },
up: { x: 0, y: 0, z: -100 }
}

See how the UP points along the Z-axis, while we are looking along the
Y axis. So in this case the UP and EYE-LOOK are nicely perpendicular.

Thanks for the example set-up. Let me see if I understand the issue -
when running with the predefined cube as example object, the problems
are:

1. an undesirable swapping of textures, where some of the white
patches switch to green and back
2. weird initial textures while drawing geometry while the textures
are still loading

This looks a bit like state not being properly switched in the draw
list (ie. textures sticking between objects as they are drawn) when it
is being modified as things load, so I have some idea of where to look
for this.

When running with the teapot it does appear as if SceneJS is not
generating the lighting aspects of the shader correctly, I have an
idea of what that is too.

PS. In an earlier SceneJS I had an option to suppress geometry
visibility while textures load, so it looks like it would be useful to
have that back.

I'm going to work on this issue on the weekend, hope that's not
holding things up.

cheers,
LK

[Datenbank archINFORM]

Hi Lindsay,

thanks for looking at my project.

> 1. an undesirable swapping of textures, where some of the white
> patches switch to green and back
> 2. weird initial textures while drawing geometry while the textures
> are still loading

Yup, that's the problem

> This looks a bit like state not being properly switched in the draw
> list (ie. textures sticking between objects as they are drawn) when it
> is being modified as things load, so I have some idea of where to look
> for this.
>
> When running with the teapot it does appear as if SceneJS is not
> generating the lighting aspects of the shader correctly, I have an
> idea of what that is too.

Cool, hopefully your ideas bringing you on the right path.

> PS. In an earlier SceneJS I had an option to suppress geometry
> visibility while textures load, so it looks like it would be useful to
> have that back.

You mean the waitForLoad flag? I noticed this flag and checked it out.
In v2.0 source code You have it already implemented (but just as a
property without any effects), but even without this flag the faces
should display the proper parent material/texturee values until the
own texture is loaded.

Have a nice & successful weekend,
Sascha

[Datenbank archINFORM]

Hi Lindsay,

have some spare time to work on my WebGL project. Just noticed, that
the the issue (#153) with the flickering textures is still there. Did
You have a look on this problem? Any solution in sight?

Another thing: I outsourced textures (map tile server) to an external
domain. Due to cross domain policy there a some problems. But with
CORS (setting some headers on the texture server) things gonna get
working. Only one extra line of code is needed in the SceneJS source
(Pull request issue #163).

And third: Got a Android phone from Sony Ericcsson for developing
purposes. The device browser supports WebGL! I will test SceneJS for
compatibility in the next time (and add touch support to the SceneJS
examples, if You don't mind).

Have a nice weekend,
Sascha

2011/12/8 Datenbank archINFORM <arc...@archinform.de>:

--

[Lindsay Kay]

Hi Sascha,

sorry for late reply, had a flu.

I think I can fix the flickering textures by delaying the rendering of the geometries until the textures are loaded - I'll give that a go.

Thanks for the CORS fix, that's real useful. 

I just bought an android phone myself (Galaxy N7000) and I got SceneJS working with it simply by adding this to the SceneJS shaders:

precision mediump float;

That would be fantastic if you added touch support to the examples, please go ahead!

cheers,
Lindsay

[Datenbank archINFORM]

Hi Lindsay,

updated now most of the examples with touch support and sent several
pull requests therefore. Would be great if You could review them soon.

Are there any news on issue (#153) (flickering textures)?

Sascha

2012/2/6 Lindsay Kay <lindsays...@gmail.com>:

--