Merge class updated

[Fabrice3D]

Hi all,
Many requested an update for the Merge class. As it was done in early days of Broomstick existence, and lots has changed, it was kinda broken
at first, in few cases, lately it was totally useless.

Now after a serious massage, so far I can test on my machine, it's now doing the expected:

- (new) respects of smoothing surface. (you can merge flat and smooth shaded meshes)
- supports single and (new) multiple materials
- its even bits faster than previous version while doing more.
- (new) its filling submeshes up to their noses to prevent to submeshes generation unless necessary for shared materials cases.
- supports auto generation of extra submeshes in case of bufferlimit hit.

Thinks to know: where the previous one was adding submeshes as found in source mesh, if you would want to keep track of the merged geometry,
it will be necessary for you to keep track of the submeshes/vertices indices to id shapes where previously a simple submesh offset count was required.
Reason is simply smaller memory footprint and less uploads.

That having said, even if I've tested many combos of different meshes and I'm pretty confident it should be robust.
Its still possible something might go wrong. So please if you have a case where it fails, deliver me the code+sources so I can debug/enhance it.

In meanwhile, I'll be updating/cleaning here and there few details and will write a little demo code that I'll upload asap.

The baby is in svn.

Fabrice

[rjgtav]

Awesome Fabrice! Cant wait to get home to test it :-D. Great Job!

[Tiago Ling]

Nice work Fabrice. Seems that i won't need to to manually add each face on the grid. The multiple material support was just what i needed. Thanks a lot, and keep the good work!

2011/5/11 Fabrice3D <fabr...@gmail.com>

[John Brookes]

Not sure if this is related to merge or the loader3d thing.

Have an obj file with a few meshes and materials

levelloader = new Loader3D;
levelloader.load(new URLRequest(LEVEL_URL), new OBJParser(LEVELASSET_URL));
levelloader.addEventListener(LoaderEvent.RESOURCE_COMPLETE, onLoadComplete);

private function onLoadComplete(e : LoaderEvent) : void
{
if (e.url ==LEVEL_URL)
{
    levelloader.removeEventListener(LoaderEvent.RESOURCE_COMPLETE,onLoadComplete);
   
    var merge:Merge = new Merge(false, true, true);
    var city:Mesh = merge.applyToContainer(levelloader)
    scene.addChild(city);

Works great, single mesh and multiple materials :)

But later I can still do
for  ( i= 0; i < levelloader.numChildren; i++)
{
    trace(levelloader.getChildAt(i).name); //obj1...obj8
)

tried levelloader.dispose(false) both on the children and leveloader
levelloader.dispose(true) throws errors.

So erm how do you delete any reference to it?

[rjgtav]

Well, was checking your example, and i have a question. Why do you apply the merge so many times? Wasnt it supposed to merge only one time? Or do you need to merge again each time you move/rotate an already merged object?

[Fabrice3D]

yes, as replied in another thread

http://www.closier.nl/broomstick/mergetest.html

so as you see, you should have no problems with your cubes. Your challenge will be to offset the uv's, that's probably the only "big" thing regarding your Merge generation

this code is in svn (holding one small glitch that I try to fix but svn is down for me atm for some dark reason. Will fix later on)

Fabrice

[Fabrice3D]

that's the beauty of it: you choose! ;)

in this example, I was trying a similar case as was discussed with cubes
so in this case, it would be stupid to build first loads of cubes and then merge them
in this particular case, you would, if you would use my demo code, simply alter the uv's of the source cube during the loop
and offset its position. See this test more as a stress test rather than what you would usually do.

John for instance, he makes a city add to a container --> pass the whole micmac to the merge class. Thats the most common use of it.

if you would use PathDuplicator, and pass a tree, and a container, you could pass this container to the class and name it MyForest...

Fabrice

[Fabrice3D]

if you set myMerge.disposeSources = true;
it does atempt to clear.

Just runned a test where I try after a merge with dispose on, to reuse the same source, and got null error. So Obvioulsy the problem lays else where.

> Works great, single mesh and multiple materials :)

really happy about that, this method was a serious pain :)

I have just noticed a little problem on merges mega unsanely huge meshes with multiple materials, no errors, but not propper display.
Will fix later on, it should probably not occur in most "normal" cases.

Fabrice

[rjgtav]

Thanks. Will check it later on pc, as using mobile is a little hard to understand code :-)

[rjgtav]

Well, started trying the Merge class... First, with 200 cubes, it took 200ms to merge them (in each loop i clone the cube and then merge it into the mergedMesh)... Then tried with the full 500... and flash took 58seconds! to merge them all with a result of... 2135566 polys! and 500 mb of ram... So i'm not sure if this is my fault (probably is). I'll continue testing...

[rjgtav]

New test: with no cloning, just adding the cubes, this time it took 5s less and uses a little less ram. This time i was able to control my character, and for my surprise, it looks like it filled all the empty space! I'll try to explain my map:

My map is made of 5 layers:

The 1st one is a 50x50 cubes, which represent the floor.

Then the rest have only cubes around the 50x50 square created in the 1st layer.

For example (the map is seen from above):

Layer 1:                                      The rest of the layers:

[1, 1, 1, 1, 1]                                    [2, 2, 2, 2, 2]

[1, 1, 1, 1, 1]                                    [2, 0, 0, 0, 2]

[1, 1, 1, 1, 1]                                    [2, 0, 0, 0, 2]

[1, 1, 1, 1, 1]                                    [2, 0, 0, 0, 2]

[1, 1, 1, 1, 1]                                    [2, 2, 2, 2, 2]

Legend:

0 - Empty tile

1 - Floor

2 - Wall

And when i test, the wall cubes fill the empty area and also seem to go much farther than the actual map size. Important Note: The ground and wall tiles have different materials and i'm applying them to the same mesh, and they show the correct materials.

UPDATE:

Just what i thought. Now i tried apllying the floor cubes to a floorMesh and the wallCubes to a wallMesh and all works smoothly (even better fps - 25 - over the max!, just 9mb of ram and the same 9000 polys).

Conclusion: Is this a bug or do i really need to create a mesh for each type of cube?

[Tiago Ling]

I'm testing it in the simplest way possible to start. I'll try with multiple materials later, but for now, it's working as expected:

        private function createMap(size:uint):void
        {
            var planeA:Mesh = new Plane(_material, 100, 100, 1, 1, false);
            var planeB:Mesh = new Plane(_material, 100, 100, 1, 1, false);
           
            var _merge:Merge = new Merge(false, false, false);
           
            for (var i:uint = 0; i < size; ++i)
            {
                for (var j:uint = 0; j < size; ++j)
                {
                    planeB.moveTo(100 * i, 0, 100 * j);
                    _merge.apply(planeA, planeB);
                }
            }
           
            _map = planeA;
            _view.scene.addChild(_map);
        }

2011/5/11 rjgtav <rjg...@gmail.com>

[rjgtav]

Well, now im getting an "Error #1502: A script has executed for longer than the default timeout period of 15 seconds." error at line 274 of the Merge class. I've already tried with both merger.apply(cubeContainer, cube) and with merger.applyToContainer(cubeContainer), and this just gives this error. This only happens if i try to add 2 types of cubes to the same mesh and i think its because the cubeContainer keeps getting so bigger, with so many vertices, that it takes an year to loop through them...

So, (finally), i've created an example to show what im saying.

I have uploaded 3 files:

MergeTester.fla - the fla of the test :-D

MergeTester.as - the file where the code is. If you want to switch from 1 single mesh to 2 meshes (one for the floor cubes and other for the wall cubes), please follow the instructions in the code comments.

Input3D.as - this is a class i created, based on Flare3D's one, that is used for handling keyboard input. It is still uncompleted, but it works. I've added this file because in the project i have a camera which you can move with your keyboard.

CAMERA INSTRUCTIONS:

W - Increase Y

S - Decrease Y

A - Increase X

D - Decrease X

UP arrow - Increase Z

DOWN arrow - Decrease Z

I've also added a WireframeAxesGrid, to help orientating :-).

NOTES:

If you only use 1 mesh, it gives the time-out error, but if you use the 2 meshes, it works like a charm.

The question remains: is this the expected behavior? If yes, so in that case i need to create a Mesh, that will represent a map, with various submeshes, each one for 1 type of cube, right?

Thanks,

rjgtav

[rjgtav]

P.S. Please add your away3d src to the fla before publishing it :-)

[John Brookes]

Works if you just use an objectcontainer3D and add wall,floor cubes to that and then after the for loop merge that objectcontainer and add to the scene.

eg

var _merger:Merge= new Merge(false, true, true);
var obj:ObjectContainer3D = new ObjectContainer3D();
           
for (var py:uint = 0; py < _map.length; py++) {
for (var pz:uint = 0; pz < _map[py].length; pz++ ) {
for (var px:uint = 0; px < _map[py][pz].length; px++ ) {
switch(_map[py][pz][px]) {
case 1://FLOOR
var floorCube:Cube = new Cube(floorMaterial, CUBESIZE, CUBESIZE, CUBESIZE);
floorCube.moveTo(CUBESIZE * px, CUBESIZE * py, CUBESIZE * pz);
obj.addChild(floorCube);

break;
case 2://WALL
var wallCube:Cube = new Cube(wallMaterial, CUBESIZE, CUBESIZE, CUBESIZE);
wallCube.moveTo(CUBESIZE * px, CUBESIZE * py, CUBESIZE * pz);
obj.addChild(wallCube);
break;
}
}
}
}
_view.scene.addChild(_merger.applyToContainer(obj));

[rjgtav]

hi. I think i.ve said i.ve tried it it gave me a time.out error. Does it work on your side?

[John Brookes]

yes.

[rjgtav]

hum ok then. Will try that tommorow. Does anyone else have an idea?

[John Brookes]

In fact if you put the two cubes in the for loop, it works with _merger.apply(_mergedMesh, wallCube);

[rjgtav]

hum.. What do you mean? Does it work if you use the original code, with no changes?

[Özgür]

why marge ?
What to do with marge?

2011/5/12 rjgtav <rjg...@gmail.com>

hum.. What do you mean? Does it work if you use the original code, with no changes?

--
Özgür Çimen
MM Developer & CG Arts
0090 535 766 33 14
http://www.cimenozgur.com/

[Fabrice3D]

just uploaded an update that should speed up containers feeds.

where it was doing

loop
parse
merge

it now does

loop
parse & collect data

merge

It does not speed up a single merging of two objects.
It prevents a parsing*n loops for reciever object and all the vectors/var generation
which can be pretty significant boost.

looking now at the massive meshes issue.

@rjgtav
Even if the above update will probably speed up your case, I think generating new cubes
each iteration is not the way to go...

Fabrice

[rjgtav]

@Fabrice
Hi. Did you check my example? I only create 1 cube of each type before
the loop and then, in each iteraction, i only move it to the new
position and merge to the container. Or that also creates a new cube?

[John Brookes]

:o

[Fabrice3D]

My mistake. I was looking at the snippet that was posted, indeed your class doesn't make new ones. good ;)

Fabrice

[John Brookes]

Doesn't work if you dont have the cubes in the for loop.
Just been doing similar and have to have new cube() inside the loop.

9708 cubes 1 image

http://www.shrewballooba.co.uk/marge/
mouse wheel zoom in


for (var x:uint = 0; x < bmd.width; x++)
{
    for (var y:uint = 0; y < bmd.height; y++)
    {
        var color:uint = bmd.getPixel32(x, y);
       
        if (color > 0xffffff )
        {
        var c:Cube = new Cube(mat, 4, 4, 4,1,1,1,false);
        c.moveTo(x*6, y*6, 0);
        scaleUV(c, new Point((bmd.width),  (bmd.height)));
        incrementUV(c, new Point(x /  (bmd.width ),  y  / (bmd.height )));
        c.rotationZ = 180
        imgHolder.addChild(c);
...
scene.addChild(merge.applyToContainer(imgHolder))

[Fabrice3D]

Marge merged. Love it :)

runs 30/30.

what would be speed with single cubes btw?

Fabrice

[John Brookes]

erm 0

Error: Error #3691: Resource limit for this resource type exceeded.
    at flash.display3D::Context3D/createVertexBuffer()

:))

[Fabrice3D]

ah I see, so looks like this class will be used more often...

As I'm gonna move on to other classes now, if you think of something that should ease or help make it better,
let me know .

oh btw, a little detail on your snippet

scaleUV(c, new Point((bmd.width), (bmd.height)));
incrementUV(c, new Point(x / (bmd.width ), y / (bmd.height )));

you could spare "new Point()" 9706 times. the first one could be declared once outside.
and the second could be a simple holder point that just need new x,y values each iteration...
the scaleUV seams pretty constant as well, may be could be done just once before loop...

Fabrice

[John Brookes]

good point ;)

[Fabrice3D]

You're welcome ;)

just added a little thingy. For the merge with containers, as the class picks the first mesh it encounters to use it as reciever mesh.
It can be in some cases not that obvious to know who became the donkey, to eventually loop over the others to destroy them for instance if the remove was not left to the class.

so now you have an optional mesh name param for this. To reuse your snippet
it could become.

scene.addChild(merge.applyToContainer(imgHolder, "marge"));

same result but thought I could become handy if you merge from unknown loaded content or work without id's.

Fabrice

On May 12, 2011, at 4:06 PM, John Brookes wrote:

> good point ;)
>

[John Brookes]

I see how you do it.
Was expecting 9708 subgeometries basically one for each cube. But its 17 subG, each subG being filled to a max of 64998
Which makes moving 1 cube after merge much more fun.

whcih brings me to...

Confused on vertexdata length

What I understand :0

Basically if keepMaterial is true a new subgeometry is created for each cube.

eg
var merge:Merge = new Merge(false, false, true);
var cont:ObjectContainer3D = new ObjectContainer3D();
for (var i:int = 0; i < 100; i++)
{
var c1:Cube = new Cube(new ColorMaterial(0xff0000),5,5,5);
c1.x = i*5
cont.addChild(c1)

var newmesh:Mesh = merge.applyToContainer(cont, "cubes")
scene.addChild(newmesh);

trace(newmesh.subMeshes.length); //traces 100

So later its easy to do something like

newmesh.subMeshes[15].vertexData //the vertices of the 15th cube



Now the tricky bit (single material)

var merge:Merge = new Merge(false, true, true);
var cont:ObjectContainer3D = new ObjectContainer3D();
var mat:ColorMaterial = new ColorMaterial(0xff0000);
for (var i:int = 0; i < 100; i++)
{
var c1:Cube = new Cube(mat,5,5,5);
c1.x = i*5
cont.addChild(c1)
}

var newmesh:Mesh = merge.applyToContainer(cont, "twocubes")
scene.addChild(newmesh);

trace(newmesh.subMeshes.length); //traces 1

var vNum:Number = newmesh.geometry.subGeometries[0].vertexData.length
// a single cubes vertexdata.length is 72
trace(vNum); // traces 10764

Cant work out how 10764 applies to 100 cubes?

[John Brookes]

Got it :)

The first cubes vertexData is 0 -> 72
Every cubes vertexData.length after that is 108

[rjgtav]

No problem. Well, tried the updated Merge class, but it still gives time-out events... Any clue?

Btw, managed again to make it create the map by adding both cubes to the same mesh, and, it just messes the map all around. The floor is correct, but the wall is just messed up.

Can you please try my example and test both ways? I only want to know if it is possible to add to the same mesh as im doing or not.

Thanks

[rjgtav]

More Info:

With _merger.applyToContainer() at the end of the loop, it takes 53 seconds and uses 500mb of ram.(Also creates a huge amount of polys and doesn't show the map correctly (the cubes are too far from eachother)).

With _merger.apply() to the same mesh at each iteration, it takes 52 seconds and uses nearly the same mb of ram. (also creates the same amount of polys and distorts the map (as explained in the previous post)).

With _merger.apply() to a mesh per type of cube (or per material if you prefer) and also at each iteration, it takes around 250ms and uses only around 15-20 mb of ram. (and just creates 9000 polys and the map shows perfectly).

So, as you can see, it looks like there is a problem with merging meshes of different materials. I'll keep investigating. If you need more information, just ask, and i'll be glad to help to solve this.

Note: I've used the same code i've posted a few posts above.

Thanks

[Choons]

Merged Marge! teehee. Nice demo, John ; )

[Fabrice3D]

dam, now you know the secret! :)
17 sg fulled up to neck instead of 9708 is indeed why it runs that well.

the trick is simply to offset the vertice count by the amount of cube vertices.

in speudo you will have first if you want to later vertices of a cube representation
to
1 isolate the sb, a floor %64998
2 diif to count from geo target
3 alter the vertices
4 update all the sg's

Fabrice

[Fabrice3D]

I'm sorry, but I have not the time to test your project, as I would basically do your work. At some point I need draw a line ;)

however if you have a snippet using just the class that can eventually reproduce the case, with no dependencies

then it would certainly help.

Fabrice

[rjgtav]

Ok... (thought it was already a simple class xD - as my game is much bigger). Tomorrow will create an even more simple class with just the map loading and with the 3 ways explained above.

[John Brookes]

rjgtav have you even tried anything Ive posted.

Even worse
wonder what happens if  you alter 1 line of your posted code
var _merger:Merge= new Merge(false, true, false);
to
var _merger:Merge= new Merge(false, false, false);

ooh er compiles in a couple of seconds.

But. boo hoo now its all the same colour.

Now how could that possibly be done. Multiple cubes with a single material.
Nope cant be done.

could be some sarcasm above.
could also be answers in above posts.

[rjgtav]

Hi, yes i have already tried everything i can remember, including
merger properties. today will post a class with all the tests.

[rjgtav]

Ok, what I suspected is correct. When adding cubes of different materials to the same mesh, with keepMaterial set to true, the length of vertices increases so much that it takes an year to loop.

Important Note:

In the main mesh (where i add all the cubes to it), if i set its material to null, the loop never ends and the length of vertices keeps increasing till its max (21678)

But, if i set a material to it, the loop only takes a minute and the length of vertices at the beginning stay at 36, but only at the end they increase till their max.

(will post the class later)

[Fabrice3D]

There is a little nuance to your comment:

here's the logic:

when the mesh to be merged must not keep own material, it's data is added to the last subgeometry of the mesh and inherits its material.

when keepmaterial is true

the code doesn't add a subgeometry (here comes the nuance) it first look if it can find a subgeometry shared the same material

so if and ONLY if the material is not to be found in gteh reciever mesh, it will add a new one, even if there is room in other subgeometries

simply because each upload can have only one material reference.

If you read back older mails, you will notice I have said to you:  offset the uv's and scale them down or apply any kind of logic to them

so that a larger amount of cube can share the same material, yet look like if they are all unique. It would be, even from a classic cpu flash perpective

very optimistic to expect flash to run ok with hundreds of maps on screen. Not even talking about the loading nightmares, the bandwidth that would bring with it etc...

basically if you do not change the uv's, you're looking at the worst case scenario for this class.

To answer John's question on the diff in vertices length from 72 to 108, that's because I do respect the smoothing of the surfaces.

A smoothed mesh holds less vertices/uv's than indices, while a flat shaded one holds exactly same amount. If the mesh would have been manipulated so that its

using both flat and smoothed parts then the count lays in between the two values.

Because I do not want be lynched, I have written the class so that it respects this finishes without adding the cost of a massive compair to find doubles

in case of shared vertices during generation. Therefor, new normals are either rotated, recalculated or kept as is as often as possible, but indices even pointing at same

ref value are becomming unique, no matter if they were shared or not. In short, unless I did wrong, the data is most of the time reorganized in an optimized way

to allow massive generations and still run fast while retaining the sources appearance. Yet doing all this as fast as possible. That explains the 17 subgeom compaired to your expected 9000+

which is a substantial improvement I think ;)

if i set its material to null, the loop never ends

This is something that I must check. I'll look into it asap.

Fabrice

[John Brookes]

On the removing  leftovers from doing merge.applyToContainer.

That merge demo (marge) uses 107M becuase it has 9708 cubes still hanging around.

Have to set
var merge:Merge = new Merge(false, true,false);

then loop through the cubes
do
 merge.applyToContainer(imgHolder, "marge");

Now have to delete each cube
for each (var cc:Cube in cubesArr)
{
    cc.dispose(true);
    Mesh(cc).geometry.subGeometries[0].dispose();
    cc.material = null;
    cc = null
    //System.gc();
}
cubesArr.length = 0;

Problem is the material is shared with the merged mesh, so have to reapply the material to each of the 17 subG

And finally add the merged mesh to the scene.

http://www.shrewballooba.co.uk/marge2/
107M down to 44M

Tried various other methods bit this is the only one that worked.
Oh and that with the fix to the Mesh.as dispose missing super()

[rjgtav]

@John Hi. It looks like thats the problem of my app. It just uses creates lots of cubes, subgeoms, etc, and the damm garbage collector doesn't collect them... will try your method to even improve the tests im doing :-).

Just a question: Why did you disable the disposeSources in the merger? Did you disable it because it wasn't definitely disposing the cubes?

[John Brookes]

That and the fact I use a single material.
if I do
var merge:Merge = new Merge(false, true,true);
or even
var merge:Merge = new Merge(false, false,false);
it will throw an error
TypeError: Error #1009: Cannot access a property or method of a null object reference.
    at away3d.core.managers::Texture3DProxy/getTextureForContext()
    at away3d.materials.methods::BasicDiffuseMethod/activate()
    at away3d.materials.passes::DefaultScreenPass/activate()
    at away3d.materials::MaterialBase/activatePass()
    at away3d.materials::BitmapMaterial/activatePass()
    at away3d.core.render::DefaultRenderer/drawRenderables()
    at away3d.core.render::DefaultRenderer/draw()
    at away3d.core.render::RendererBase/executeRender()
    at away3d.core.render::RendererBase/render()
    at away3d.core.render::DefaultRenderer/render()
    at away3d.containers::View3D/render()
    at MergeTest/handleEnterFrame()

Then again if I dont use applyTocontainer and use just apply with (var merge:Merge = new Merge(false, false,false);) then I dont get the cubes hanging around.

Would like to explain better but been through lots of combinations and its hard to know the exact way.


Also, fabrice, understand why it fails but maybe catch/warn of the error.

Using
var merge:Merge = new Merge(false, false,true);
on your demo from the examples

[rjgtav]

Hi Guys. Here is the simplified and improved version of my class.

In this class i have 4 tests:

Test 1: In this test the cubes are added to one single mesh (with a null material) in each iteration, by using Merge.apply() method and keepMaterial set to true (I've created this to show how it handles a null material in the "parent" mesh);

Test 2: This test is the same as Test 1, but the "parent" mesh has a material attached to it;

Test 3: In this test it uses 2 meshes (one for each type of cube, with the correct material added) and it adds each cube to the correct mesh in each iteration, by using Merge.apply() method and keepMaterial set to false;

Test 4: Finally, in this test, the cubes are added to one single container in each iteration, and at the end it uses Merge.applyToContainer() and keepMaterial is set to true (this test is like John Brookes example)

This time i've organized the tests in functions, so to try one, simply uncomment it :-).

Camera Controls:

W: Increases Y

S: Decreases Y

A: Increases X

D: Decreases X

E: Increases Z

Q: Decreases Z

Here are my results:

Test1:

     Memory: in Windows Task Manager shows 600mb and fires a time-out error (i've set the Script time limit to 120 seconds, so, 2 minutes)

Test2:

     Memory: around 500 - 510 mb

     FPS Average: 25

     Poly count: 2134714, that explains why the map gets distorted

     Loop time: around 51 seconds

     Total time: the same as Loop time

Test3:

     Memory: 8.4 and stable

     FPS Average: 25

     Poly count: 8868

     Loop time: around 230-240 ms

     Total time: the same as Loop time

Test4:

     Memory: 12.8 and stable

     FPS Average: 25

     Poly count: 8844

     Loop time: around 60 ms

     Total time: around 300 ms

So, as you can see, Test 3 is the fastest and who has the best performance. But it implies that i have to create a Mesh for each cube type, but i would prefer having all in the same mesh, as it saves more code.

So my question is:

Do i have to use multiple meshes to get the max performance?

@Fabrice Sorry, i know you've already told me lots of times about offSetting uvs... But i dont understand what you mean with that... Do i have to move the uvs? Sorry

[rjgtav]

@Fabrice Google is the best! So you want me to change the way the material is show to the cube (for example tile, scale, use just a selected part of the texture, etc). The problem is that the game i'm developing will have all the texture applied to each face of the cube (square textures), so i think i dont need uv mapping.

Im making my game by taking as inspiration the game Kula World, for ps1, and there the cubes faces are identical. Or is there a way to increasing performance and obtaining the same effect?

thanks

P.S. If you want to see Kula World, check here: http://www.youtube.com/watch?v=Ol1usmZs9r8

[rjgtav]

Hi Fabrice. I.ve decided to use the Test3 approach, but would like to
hear what do you think or if there is any update :-) Thanks