Batched 3D Model format issues

[Lucio Piccoli]

hi all

i am attempting to write out a b3dm for 3D tileset.

i read the various specs 

https://github.com/AnalyticalGraphicsInc/3d-tiles/blob/master/TileFormats/Batched3DModel/README.md

https://github.com/KhronosGroup/glTF/tree/master/extensions/Khronos/KHR_binary_glTF

steps i used

1 . i started by creating a binary gltf using the  colladaToBglTFConverter tool.

2. i wrote the 20 byte header for the b3dm ( 0 for the batch table) 

3. then appended the file created from #1.

my files looks nothing like the sample b3dm from the tileset directory

Q1, what exactly is meant by the statement

Binary glTF immediately follows the batch table.

 does it mean the entire binary gitf file including header or without the header follows the batch table?

 

Q2. ar the Binary gITF output from colladaToBglTFConverter tool compatible with cesium....my are generating runtime errors.

Q3. more broadly how are others generating the batched 3d models?

any help appreciated.

-lp

[Lucio Piccoli]

these are the difference between the sample b3dm files and the once i created.

this is the supplied sample that works in sandcastle.

Notice the header.

this generated according to the specs. Notice the magic number of 'b3dm' in header.

am i reading the wrong spec or what else have i missed?

thanks

-lp

[Manuel Massing]

Hi Lucio,

> steps i used
>
> 1 . i started by creating a binary gltf using the
> colladaToBglTFConverter tool.
>
> 2. i wrote the 20 byte header for the b3dm ( 0 for the batch table)
>
> 3. then appended the file created from #1.

yes that is the correct approach - your test.b3dm hexdump looks correct
as well.

The sample differs because is is gzipped and then served over
nodejs with the appropriate "Content-encoding" flag added (i.e.,
it is not compressed on the fly). So you should either gzip the b3dm
before serving, or edit server.js to not add the "Content- Encoding:
gzip" header for b3dm-files.

Try gunzipping the sample if you want to analyze it.

cheers,

Manuel

[Manuel Massing]

Hi Lucio,

> Q1, what exactly is meant by the statement
>
> Binary glTF immediately follows the batch table.
>
>
> does it mean the entire binary gitf file including header or without
> the header follows the batch table?

yes, you should append the entire file including header.

> Q2. ar the Binary gITF output from colladaToBglTFConverter tool
> compatible with cesium....my are generating runtime errors.

what errors are you seeing?

Manuel

[Lucio Piccoli]

hello Manual

you have been very helpful, Once i  gunzip the sample b3dm files the look similar 

it seems the gzip of b3dm has caught me again.

just to confirm.

1.the sample b3dm files are already gzipped on disk.

2.the node.js has content encoding of 'gzip for  .b3dm files.

3.when i removed the content encoding of 'gzip' from the server.js the sample tileset were NOT displayed.

Q1. so are the files on disk are gzip and then gzip again over the http transfer?

thanks

-lucio

--

regards

-Lucio Piccoli

Director

------------------------------

www.asteriski.com

[Lucio Piccoli]

hi all

now i have gzipped my b3dm file i getting a proper error now

Cesium.js:67089 [Cesium WebGL] Vertex shader compile log: ERROR: 0:25: 'a_batchId' : undeclared identifier 

createAndLinkProgram @ Cesium.js:67089

Cesium.js:67095 [Cesium WebGL] Vertex shader translation failed.createAndLinkProgram @ Cesium.js:67095

Cesium.js:176141 An error occurred while rendering.  Rendering has stopped.

undefined

RuntimeError: Vertex shader failed to compile.  Compile log: ERROR: 0:25: 'a_batchId' : undeclared identifier 

Error

    at new RuntimeError (http://localhost:8080/Cesium/Cesium.js:9279:19)

    at createAndLinkProgram (http://localhost:8080/Cesium/Cesium.js:67100:23)

    at initialize (http://localhost:8080/Cesium/Cesium.js:67305:23)

    at ShaderProgram.defineProperties.vertexAttributes.get (http://localhost:8080/Cesium/Cesium.js:66967:17)

    at getAttributeLocations (http://localhost:8080/Cesium/Cesium.js:109097:93)

    at createVertexArrays (http://localhost:8080/Cesium/Cesium.js:109324:46)

    at createResources (http://localhost:8080/Cesium/Cesium.js:110169:13)

    at Model.update (http://localhost:8080/Cesium/Cesium.js:110584:13)

    at Batched3DModel3DTileContent.update (http://localhost:8080/Cesium/Cesium.js:134124:21)

    at Cesium3DTile.process (http://localhost:8080/Cesium/Cesium.js:142844:23)CesiumWidget.showErrorPanel @ Cesium.js:176141

Q1. is the 'a_batchid' referring to the batch table?

i have set my batch table length to 0.

any help appreciated.

-- 

regards

-Lucio Piccoli

[Patrick Cozzi]

Hi Lucio,

Here's an example test dataset without a batch table:

https://github.com/AnalyticalGraphicsInc/cesium/tree/3d-tiles/Specs/Data/Cesium3DTiles/Batched/BatchedWithoutBatchTable

For the related Cesium code, see:

https://github.com/AnalyticalGraphicsInc/cesium/blob/3d-tiles/Source/Scene/Cesium3DTileBatchTableResources.js#L331-L368

Let me know if you think there is an issue in Cesium here.  Anything you can do to narrow it down is appreciated.

Thanks,

Patrick

[Manuel Massing]

Hi Lucio,
Hi Patick,

I think the issue here is that you need to set the b3dm headers'
"batchLength" to zero if you don't declare the "a_batchId" attribute
in your glb mesh.

> Let me know if you think there is an issue in Cesium here. Anything you
> can do to narrow it down is appreciated.

I'd say cesium is working as expected in this case, although the
documentation for "batchLength" could maybe be made more explicit
about requiring the "a_batchId" attribute if batchLength is non-zero.

Manuel

[Lucio Piccoli]

hi manual and patrick

i see my problem, i didnt have the batchid set in glb as i assumed i didnt need it as i only had a single item in batch.

so from the spec i have to add the following :

1. add the batchid to the technique attribute and parameters

ie

from

techniques":{"technique0":{"attributes":{"a_normal":"normal","a_position":"position"},

"parameters":{"ambient":{"type":35666},"diffuse"...

to

techniques":{"technique0":{"attributes":{"a_batchId":"batchId","a_normal":"normal","a_position":"position"},

"parameters":{"ambient":{"type":35666},"batchId":{"semantic":"BATCHID","type":5126},"diffuse"...

2. add the declaration to the shader 

attribute float a_batchId;

 

does that look correct?

thanks

-lucio

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/Rt9M3ogYhFY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Manuel Massing]

Hi Lucio,

> i see my problem, i didnt have the batchid set in glb as i assumed i
> didnt need it as i only had a single item in batch.

If you do not need the picking functionality, you can do without the
a_batchId attribute by setting batchLength to zero (i.e.,
both batchTableLength + batchLength should be zero in the b3dm header).

cheers,

Manuel

[Lucio Piccoli]

hi manuel

ok i misunderstood the batchlength value.

so i set both batchlength and batchtable to 0

the b3dm now loads without error but nothing is displayed.

all that my scene contains is a red box. 

i got nothing left to try out

any suggestions/advice or help is appreciated.

-lucio

Manuel

[Patrick Cozzi]

I update the b3dm spec to clarify when a_batchId is required, see #78.

Lucio - perhaps compare your file with the sample data used in the unit tests, see

https://github.com/AnalyticalGraphicsInc/cesium/tree/3d-tiles/Specs/Data/Cesium3DTiles/Batched

Patrick

--

twitter.com/pjcozzi

[Manuel Massing]

Hi Lucio,

What coordinate system is your mesh expressed in?

Are you using the CESIUM_RTC extension to define a relative
origin?

cheers,

Manuel

[Lucio Piccoli]

thanks patrick.

 

-lp

On 3 March 2016 at 01:47, Patrick Cozzi <pjc...@gmail.com> wrote:

I update the b3dm spec to clarify when a_batchId is required, see #78.

Lucio - perhaps compare your file with the sample data used in the unit tests, see

https://github.com/AnalyticalGraphicsInc/cesium/tree/3d-tiles/Specs/Data/Cesium3DTiles/Batched

Patrick

-- 

regards

-Lucio Piccoli

[Lucio Piccoli]

hi manuel

i am NOT the CESIUM_RTC extension.

i guess my model is using the UCS of the modelling tool.

i am starting to feel i dont understand very much about 3d data.

i guess i need to specify the models location relative to the 3d tile.

-lucio

cheers,

Manuel

[Manuel Massing]

Hi Lucio,

you will need to specify the model in ECEF (Earth-Centered Earth-Fixed)
cartesian coordinates, with the additional twist that Model.js
transforms from y-Up to z-Up:

https://github.com/AnalyticalGraphicsInc/cesium/blob/3d-tiles/Source
/Scene/Model.js#L132

so you will have to rotate the ECEF coordinates by the inverse transform
(i.e., rotate -90° around the x-axis). I am unfamiliar with
collada2gltf, but it might already take care of that step?

To transform from lat/lon/height to cartesian using e.g. GDAL/proj4:

cs2cs +proj=latlon +ellps=WGS84 +datum=WGS84 +to +proj=geocent
+ellps=WGS84 +datum=WGS84

cheers,

Manuel

[Lucio Piccoli]

hi manuel

i got no progress with the ECEF coordinates in the model, so i changed direction and looked at using the CESIUM_RTC extension.

https://github.com/KhronosGroup/glTF/tree/master/extensions/Vendor/CESIUM_RTC

i added the following to my model

 "extensions" : {

"CESIUM_RTC" : {

"center" : [1215019.3092979142, -4736312.191468832, 4081596.540251589]

}

},

"extensionsUsed" : ["CESIUM_RTC"]

and now i am getting a bounding volume drawn in my tileset (without batchtable)....but NOT the actual model!

i assume that my model is now using a coordinate based at value defined in CESIUM_RTC. is that correct?

in the collada model my mesh is defined simply

<float_array id="Geometry-mesh002-positions-array" count="24">-50 -50 -50 -50 50 -50 50 50 -50 50 -50 -50 -50 -50 50 -50 50 50 50 50 50 50 50 50</float_array>

i am thoroughly confused as to why the tileset is not drawn.

cheers,

Manuel

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/Rt9M3ogYhFY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Benjamin]

Hi Lucio,

reading trough this thread was like reading a journey of my last three days. I had to figure this all out, while the answers were all here..

Did you have any luck with using the CESIUM_RTC extension?

You may have to add this to your b3dm as well:

"techniques": {
    "technique0": {
        "parameters": {
            "modelViewMatrix": {
                "semantic": "CESIUM_RTC_MODELVIEW",
                "type": 35676
             },
             // ...
         }
     }
 }

I will implement the extension tomorrow, and hope that the model will appear.

Also if a cesium dev is reading this:

In the b3dm Readme.md the link to CESIUM_RTC Readme.md extension is 404.

In the CESIUM_RTC Readme.md the link to the cesium implementation is 404.

[Lucio Piccoli]

Hello Benjamin

adding the CESIUM_RTC_MODELVIEW to the model worked! 

the 3d tile is now visible!

i thought i had already tried that before but who knows.

thanks for you help.

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/Rt9M3ogYhFY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Benjamin]

I'm glad I could help!

When I use the CESIUM_RTC extension, the orientation of my model messed up. Is your orientation correct?

Am Donnerstag, 17. März 2016 01:59:09 UTC+1 schrieb Lucio Piccoli:

Hello Benjamin

adding the CESIUM_RTC_MODELVIEW to the model worked! 

the 3d tile is now visible!

i thought i had already tried that before but who knows.

thanks for you help.

On 17 March 2016 at 04:08, Benjamin <benn...@gmail.com> wrote:

Hi Lucio,

reading trough this thread was like reading a journey of my last three days. I had to figure this all out, while the answers were all here..

Did you have any luck with using the CESIUM_RTC extension?

You may have to add this to your b3dm as well:

"techniques": {
    "technique0": {
        "parameters": {
            "modelViewMatrix": {
                "semantic": "CESIUM_RTC_MODELVIEW",
                "type": 35676
             },
             // ...
         }
     }
 }

I will implement the extension tomorrow, and hope that the model will appear.

Also if a cesium dev is reading this:

In the b3dm Readme.md the link to CESIUM_RTC Readme.md extension is 404.

In the CESIUM_RTC Readme.md the link to the cesium implementation is 404.

--

[Lucio Piccoli]

hi Benjamin

my orientation is also strange.

i think i recall reading about some transformation in axis that occurs.

it should be  easy to trace the cause now i have something displaying.

regards

lucio

--

[Lucio Piccoli]

hi Benjamin

the orientation seems to rotated around the x axis 45 degrees.

which is strange.

not sure what i missed.

[Benjamin]

Hi Lucio,

rotating it around the x-axis 45 degrees didn't fix the problem for me. I think it depends on where the model is on the planet, though I didn't try it out.

I removed the rotation for making the Y axis the Up-Axis (90 deg rotation because of what Manuel already mentioned). Then my model was kind of 45 deg wrong.

I rotated the model -45 deg on x with this matrix in the node:

"matrix":[1.0,0.0,0.0,0.0,0.0,0.7071067811865476,0.7071067811865476,0.0,0.0,-0.7071067811865476,0.7071067811865476,0.0,0.0,0.0,0.0,1.0]

But the models orientation is still a bit off. 

[Manuel Massing]

Hi Lucio,
Hi Benjamin,

glad you made some progress :-)

as previously mentioned, the model should be specified
in global ECEF cartesian coordinates. In a fixed frame like ECEF, the
up / north / east vectors depend on your position on earths' surface.

See Source/Core/Transforms.js to see how to transform from a local frame
(e.g. ENU) to fixed frame.

Manuel

[Lucio Piccoli]

hi manuel

just to clarify

1. i have a model in defined in user defined coordinate system.

2. the model is edited as needed in the modelling software

3. the model is exported to gltf

4. each model will have each mesh vertex  transformed to ECEF

5. upload the tileset

is that the expected process?

thanks

-lucio

 

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/Rt9M3ogYhFY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

regards

-lucio

[Manuel Massing]

Hi Lucio,

> 1. i have a model in defined in user defined coordinate system.
> 2. the model is edited as needed in the modelling software
> 3. the model is exported to gltf

4. each model will have each mesh vertex transformed to ECEF

I don't have experience with the collada2gltf converter,
but I think it does not implement a transform to ECEF. You will have to
take care of the transformation to ECEF in your modelling software or
in a custom converter.

Manuel

[Benjamin]

Converting the vertices to ECEF works, but you should still use the CesiumRTC extension, or every vertex of your model will jitter extremely.