How do I convert my data to 3D Tiles?

[Patrick Cozzi]

Hi all,

One of the most common questions we receive is "how do I convert my data to 3D Tiles?"  Given that Cesium's 3D Tiles renderer will be ready in the next few months, I expect this question to become even more popular.

In the short term, the answer is to please email me, pco...@agi.com, and ask.  We likely already have in-house tools that can convert your data to 3D Tiles, and we're happy to run them with your data.

We have tilers for several types of data (CAD, photogrammetry, 3D buildings, point clouds, etc.) and have successfully worked with many folks, and have written a few blog posts with the results, for example: SiteSee Photogrammetry and 3D Tiles.

Slightly longer term, these tools will be part of Cesium Composer, a commercial web service and OnPrem platform we are developing to further expand and fund open-source Cesium development.  You can sign up for the beta here.

Thanks,

Patrick
--

twitter.com/pjcozzi

[2801...@qq.com]

在 2017年5月11日星期四 UTC+8下午8:26:23，Patrick Cozzi写道：

在 2017年5月11日星期四 UTC+8下午8:26:23，Patrick Cozzi写道：

Hi，I'm Jack,
I got some trouble recently,
when I open the city 3d model  on cesium ,It is   above  the virtual  global,they can't laminated together well ,I don't know  why this happen ,and how  to solve  this problem .can you give some suggestions?

[Rachel Hwang]

Hi Jack,

We'd be happy to help with your issue, but we'll need more details. Can you post your code and / or a screenshot demonstrating the issue?

Best,

- Rachel

[Jack Lee]

Hi Rachel

I have tried the way that Sean Lilly told on the forum ,but there is something wrong when I cloned the Cesium. It can't appear the 3d model after cloning. I don't know why this happen and how to solve this issue, I 'm waiting for your suggestions.And please forgive my poor English_(:з」∠)_

thanks,

Jack

---Original---

From: "Rachel Hwang"<elkw...@gmail.com>

Date: 2017/6/13 02:47:43

To: "cesium-dev"<cesiu...@googlegroups.com>;

Cc: "280147597"<2801...@qq.com>;

Subject: [cesium-dev] Re: How do I convert my data to 3D Tiles?

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/xLkYIuku9hA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jack Lee]

---原始邮件---

发件人: "Jack Lee"<2801...@qq.com>

发送时间: 2017年6月13日 22:43:30

收件人: "Rachel Hwang"<elkw...@gmail.com>;

主题: 回复： How do I convert my data to 3D Tiles?

Hi Rachel,

it is very nice that you can help me, here is the code:

// Get your own Bing Maps API key at https://www.bingmapsportal.com, prior to publishing your Cesium application:
Cesium.BingMapsApi.defaultKey = 'put your API key here';

// Construct the default list of terrain sources.
var terrainModels = Cesium.createDefaultTerrainProviderViewModels();

// Construct the viewer with just what we need for this base application
var viewer = new Cesium.Viewer('cesiumContainer', {  //前一个字符串对应ID
 timeline:false,//不会创建timeline小窗口
 animation:false,//不创建动画小部件
 vrButton:true,
 sceneModePicker:false,
 infoBox:true,
 baseLayerPicker:false,//不创建选取地图小部件
 scene3DOnly:true,
 terrainProviderViewModels: terrainModels,
 selectedTerrainProviderViewModel: terrainModels[1], // Select STK high-res terrain
  //添加图像层，此时baseLayerPicker设为false
  imageryProvider : new Cesium.ArcGisMapServerImageryProvider({
        url : 'https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer'
    })
 
});

// No depth testing against the terrain to avoid z-fighting
viewer.scene.globe.depthTestAgainstTerrain = false;

// Add credit to Bentley

// Bounding sphere
var boundingSphere = new Cesium.BoundingSphere(Cesium.Cartesian3.fromDegrees(6.681247027, 46.55189101, 871.5000123), 493.3407689);

// Override behavior of home button
viewer.homeButton.viewModel.command.beforeExecute.addEventListener(function(commandInfo) {
 // Fly to custom position
 viewer.camera.flyToBoundingSphere(boundingSphere);

 // Tell the home button not to do anything
 commandInfo.cancel = true;
});

// Set custom initial position
viewer.camera.flyToBoundingSphere(boundingSphere, {duration: 0});

/*---------------------------------------------------------------------------------**//**
* @bsimethod
+---------------+---------------+---------------+---------------+---------------+------*/
// Functions to adapt screen space error and memory use to the device
var isMobile = {
 Android: function() {
  return navigator.userAgent.match(/Android/i);
 },
 BlackBerry: function() {
  return navigator.userAgent.match(/BlackBerry/i);
 },
 iOS: function() {
  return navigator.userAgent.match(/iPhone|iPad|iPod/i);
 },
 Opera: function() {
  return navigator.userAgent.match(/Opera Mini/i);
 },
 Windows: function() {
  return navigator.userAgent.match(/IEMobile/i);
 },
 any: function() {
  return (isMobile.Android() || isMobile.BlackBerry() || isMobile.iOS() || isMobile.Opera() || isMobile.Windows());
 }
};

// Add tileset. Do not forget to reduce the default screen space error to 1
var tileset = viewer.scene.primitives.add(new Cesium.Cesium3DTileset({
 url: '../Scene/Production_4.json',
 maximumScreenSpaceError : isMobile.any() ? 8 : 1, // Temporary workaround for low memory mobile devices - Increase maximum error to 8.
 maximumNumberOfLoadedTiles : isMobile.any() ? 10 : 1000 // Temporary workaround for low memory mobile devices - Decrease (disable) tile cache.
 
}));

 // Create buttons
Sandcastle.addToolbarButton('显示三维地图', function() {
 tileset.show = !tileset.show;
});

Sandcastle.addToolbarButton('显示标注', function() {
 var len;
 for (var index = 0, len = POIs.length; index < len; ++index)
 {
  POIs[index].show = !(POIs[index].show);
 }
});

Sandcastle.addToolbarButton('显示区域', function() {
 districts.show = !districts.show;
});

Sandcastle.addToolbarButton('显示自行车道', function() {
 bikeLanes.show = !bikeLanes.show;
});

thank you very much.

------------------ 原始邮件 ------------------

发件人: "Rachel Hwang";<elkw...@gmail.com>;

发送时间: 2017年6月13日(星期二) 凌晨2:47

收件人: "cesium-dev"<cesiu...@googlegroups.com>;

抄送: "Jack Lee"<2801...@qq.com>;

主题: Re: How do I convert my data to 3D Tiles?

[Jack Lee]

Hi Rachel

I have tried the way that Sean Lilly told on the forum ,but there is something wrong when I cloned the Cesium. It can't appear the 3d model after cloning. I don't know why this happen and how to solve this issue, I 'm waiting for your suggestions.And please forgive my poor English.

thanks,

Jack

---Original---

From: "Rachel Hwang"<elkw...@gmail.com>

Date: 2017/6/13 02:47:43

To: "cesium-dev"<cesiu...@googlegroups.com>;

Cc: "280147597"<2801...@qq.com>;

Subject: [cesium-dev] Re: How do I convert my data to 3D Tiles?

--

[Rachel Hwang]

Hi Jack,

Try positioning your 3d tileset. Check out the code from this post: https://groups.google.com/forum/#!searchin/cesium-dev/height$203d$20tiles%7Csort:relevance/cesium-dev/CqYeu6L6bhU/P0WywsBhBQAJ

Let's reserve this thread for discussion of 3d tile conversion -- feel free to ask more questions on another thread if necessary.

Hope that helps,

- Rachel

On Thursday, June 15, 2017 at 8:23:02 PM UTC-4, Jack Lee wrote:

Hi Rachel

I have tried the way that Sean Lilly told on the forum ,but there is something wrong when I cloned the Cesium. It can't appear the 3d model after cloning. I don't know why this happen and how to solve this issue, I 'm waiting for your suggestions.And please forgive my poor English.

thanks,

Jack

---Original---

From: "Rachel Hwang"<elkw...@gmail.com>

Date: 2017/6/13 02:47:43

To: "cesium-dev"<cesium-dev@googlegroups.com>;

To unsubscribe from this group and all its topics, send an email to cesium-dev+unsubscribe@googlegroups.com.

[ranvir singh]

Hello there, 

I recently viewed your work on your website and I really seem to like it a lot. I am a student from India and I want to create the 3-D views for the buildings in my locality. I have already installed OSM on my local. I am new to this so please guide me in the things that I require to make and do rendering tasks. 

Keep up the nice work.

--
Ranvir Singh
https://ranvirsinghprojects.wordpress.com
https://github.com/singh1114
----

 

[dmitry.v...@gmail.com]

Hi I'm working on OSM Buildings to Cesium 3d tiles conversion.

It's quite close to be done. But I still have some issues and questions.

1. What's the actual tag or branch for Cesium with 3d Tilesets support?

2. How big the overhead for tileset of tilesets? What if I'll use 4-5 levels of tilesets nesting. Are there any flaws of that?

3. IS that possible to add Tileset to scene via CZML?

[Rachel Hwang]

Hi there,

1. 3D Tiles will be in the next release of Cesium (July 5). Until then, you can get it from our github repo (3D Tiles has been recently merged with master). https://github.com/AnalyticalGraphicsInc/cesium

2. Our tileset traversal is as optimized as we've been able to make it. (Check out our blog for more on that.) 4-5 levels of nesting should not be a problem at all.

3. Not possible right now, unfortunately.

hope that helps,

- Rachel

[zxue]

I am trying to convert a large obj file to gltf using the obj2gltf command line tool. The obj file is about 800MB. It has been taking 4 hours for the conversion, and it is still working on it. I am just wondering if the library would work for such large obj file. I did the following steps to get the obj file:

1. Extract pbf data from metro extract

2. Use osm2world to generate the obj file

Now I am using obj2gltf to generate the gltf file. It seems the obj file is too large to be attached here...

[Дмитрий Киселев]

Hi Zxue,

Metro extracts are too big for that, even if you convert the whole piece into gltf, it will kill you'r browser.

I'm making that in steps: 

1. I split the NY extract into z12 tiles with my own tool https://github.com/kiselev-dv/gazetteer/releases/tag/Gazetteer-1.9rc4

2. I've done tiled output for OSM2World (https://github.com/tordanik/OSM2World/pull/122) which creates tileset.json for each z12 tile and split them into z14 tiles.
3. Using some scripts, I combine all z12 tilesets into one big tileset. https://github.com/kiselev-dv/osm-cesium-3d-tiles/blob/master/scripts/convert.sh (Sorry no appropriate docker image yet)

Here is the demo with what I've got so far http://osm.me/cesium_3d_tiles/Apps/CesiumViewer/
I still have some issues with geometricError calculation. So for instance some parts of Manhattan appears only when you fly very close to them.

So I'll apriciate some examples of geometricError calculation or the tutorial how to twick it.

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/xLkYIuku9hA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thank you for your time. Best regards.
Dmitry.

[zxue]

Hi Dmitry,

Thanks for sharing this method. Do you have any sample code for this? Or do you have any documentation including the commands I need to run for this process? In other words, do you have a step-by-step guide. For the last step how do you convert the gltf files to b3dm files? Or do you convert obj to b3dm and generate the titleset.json file at step 2?

To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[zxue]

As the issue I had in my previous post, I am trying to work on small dataset. I am working on the Harvard Square data from Metro Extra. Attached is the result I got. Here is the code I am using for showing this gltf tile:

 

 var viewer = new Cesium.Viewer('cesiumContainer');
 var position = Cesium.Cartesian3.fromDegrees(-71.1190, 42.3736, 0);


 var entity = viewer.entities.add({
        name : 'Apple Hill',
        position : position,
        model : {
            uri : 'http://localhost:8003/tilesets/Harvard/harvard.gltf',
            minimumPixelSize : 128,
            maximumScale : 20000
        }
    });
  viewer.trackedEntity = entity;

It seems when I tried to convert the pbf to obj via OSM2World, it created a "base" for the area. Is there any way that I can only extract buildings from this dataset? Attached is my pbf file.

Any idea?

Thanks! 

[Дмитрий Киселев]

1. you can filter highways and other objects with osmfilter

2. to skip terrain generation use --config here is my for instance https://github.com/kiselev-dv/osm-cesium-3d-tiles/blob/master/scripts/prop.properties

--

You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/xLkYIuku9hA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cesium-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Дмитрий Киселев]

I've updated my docker setup here https://github.com/kiselev-dv/osm-cesium-3d-tiles

1. build and run that docker, map some folder on the host to /opt/data

2. download data and save it as /opt/data/map.osm.bz2

3. run convert.sh

What could be wrong:

convert.sh will run gazetteer.jar to filter buildings and split the large dataset into smaller tiles.

you can do it manually, it require two steps
1. java -jar /opt/gazetteer/gazetteer.jar --data-dir /opt/data split /path/to/osm.xml.bz2 or you can use pbf with osmconvert

    osmconvert /path/to/pbf | java -jar /opt/gazetteer/gazetteer.jar --data-dir /opt/data split -

2. java -jar /opt/gazetteer/gazetteer.jar tile-buildings --out-dir /opt/osm-tiles --level 12

Next convert osm into obj and split it into z14 tiles

It's done by convert_parallel.py

Manually it's:

cd to /opt/OSM2World

run ./osm2world.sh -i /path/to/osm -o path/to/obj --config /scripts/prop.properties

Than, obj2gltf and gltf2b3dm

Than you can use generated by OSM2World tileset.json

[Дмитрий Киселев]

I didn't test the whole process from within docker container, but successfully converted NY metroarea using convert.sh, with some paths overwritten.

2017-07-07 14:55 GMT-03:00 zxue <hell...@gmail.com>:

Thanks so much! It helps a lot!

Thanks a million!

To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thank you for your time. Best regards.
Dmitry.

[zxue]

Thanks for your help! It works great for my case! Thanks so much! Just one more question. How to do color the buildings after you convert them from obj to gltf?

Thanks a million! 

On Friday, July 7, 2017 at 1:14:41 PM UTC-4, Дмитрий Киселев wrote:

To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[zxue]

Thanks for such detailed guide! I am going to try it!

Thanks!

To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thank you for your time. Best regards.
Dmitry.

[Дмитрий Киселев]

By default, OSM2world will use coloring from OSM or apply default color/material, and obj2gltf and gltf to b3dm will keep materials from obj.

Or if you are talking about coloring on hover, like in 3d tiles demo for NY - that's tricky part, I still don't dig gltf deep. There is gltf lib in java, i think it's possible to attach ids to groups of primitives using that lib, but I din't work on using it together with OSM2World

[zxue]

When I use obj2gltf, it just gives me the "raw" buildings 3d models without color...
Should I just set the following option to be true?

material_BUILDING_DEFAULT_texture_colorable = true

[Дмитрий Киселев]

To be honest, I don't know. 

I've tested obj2gltf on some obj models and got models with colors and textures.

OSM2World also worked with textured models out of the box.

[zxue]

No problem. I will try to figure it out. Thanks for help!

[Дмитрий Киселев]

No worries, keep me posted.

[zxue]

I just noticed when I converted from obj to gltf, I did not include the .mtl file. I believe the color information is stored in this material file.

[Дмитрий Киселев]

Yes, but usually it's referenced from .obj and obj2gltf links it properly, if it's not moved. 

2017-07-07 16:07 GMT-03:00 zxue <hell...@gmail.com>:

I just noticed when I converted from obj to gltf, I did not include the .mtl file. I believe the color information is stored in this material file.

[dams....@gmail.com]

Hi,

I've followed your method and managed to generate the tiles for Finland. However I'm seeing some very poor performance rendering the tiles compared to your New York example linked higher up in this thread. Frame rate drops significantly from the get go, and tiles take a long time to load. Any ideas what might cause that? I've already ruled out bandwidth issues.

[Дмитрий Киселев]

Hi Johan,

1. Check the subtiles. The whole structure should look like this:

tileset.json

• z12-tile-tileset.json
• z14 tile
• subtile level 0
• subtile level 1
• z14 tile
• subtiles...

The actual data (3d models) split in portions. So not the whole content of z14 tile loaded at once. 

First subtile should have just 5-10 biggest buildings. Next level subtile should have next 20 and so on.

Subtiles generation should be in the last few commits to my fork of OSM2World in tiled_out branch.

2017-07-10 13:30 GMT-03:00 <dams....@gmail.com>:

Hi,

I've followed your method and managed to generate the tiles for Finland. However I'm seeing some very poor performance rendering the tiles compared to your New York example linked higher up in this thread. Frame rate drops significantly from the get go, and tiles take a long time to load. Any ideas what might cause that? I've already ruled out bandwidth issues.

[Patrick Cozzi]

Dmitry,

Thanks for sharing osm-cesium-3d-tiles.  I added it to the list of 3D Tiles projects at https://github.com/AnalyticalGraphicsInc/3d-tiles

I also took at quick look at

https://github.com/tordanik/OSM2World/pull/122/files

It looks like the code in WGS84Util was ported from Cesium.  Do you know if this is the case?  If it is, can you please include the Cesium LICENSE.md file?

Thanks,

Patrick

[Дмитрий Киселев]

Hi Patrick, thanks for adding a link onto my project.

Yes, I've used Cesium code and port it to java. 

I'll ask the owner of OSM2World about adding Cesium license to the list of used code and it's licenses. Since it's Apache license, it shouldn't be a problem.

Or if adding Cesium license is an issue for OSM2World I'll remove meta data generation from OSM2World and add it as separate project with appropriate license.

[Patrick Cozzi]

Sounds good, thanks Dmitry!

Patrick

--

twitter.com/pjcozzi

[dams....@gmail.com]

Privet Dmitry,

Thanks for that pointer - spasibo. I think I'm making progress.
On a relate note, do you happen to know how I can make 3D tiles clamp to terrain, i.e., how do I get buildings to sit on top of the terrain?

Best regards,
Johan

Hi

[Дмитрий Киселев]

Hi Johan,

as I understand, OSM2World doesn't support elevated terrain for now. 

I'll ask the owner is that possible or easy to add.

I can suppose some workaround, but it will require some manual data preprocessing.

1. Download OSM area, open it in QGIS (or ArcGIS but I worked only with QGIS)

2. Get elevation data (from stamen for instance)

3. Find minimal elevation of building and write it as ele argument, export back to osm, write ele as ele osm tag.

4. OSM2World should support ele tag from osm data

[Дмитрий Киселев]

Oh sorry

2. Get elevation data (from stamen for instance) 

from mapzen not from stamen https://mapzen.com/documentation/terrain-tiles/

[rait...@gmail.com]

I did some tests filling the ele tags in osm, but without luck. Seems that using them is turned off in latest OSM2World. Was working fine in 0.1.9, but that version doesn't produce tileset metadata. Maybe someone was successful with this?

[feng lin]

Hi Patrick,

       Recently I want to cover the imagery wmts tile to photogrammetry 3d tiles.

        The imagery layer and photogrammetry 3d tiles  have the same WGS84 geographic coordinates.  

        I have three steps to  implement this function. First step, I convert 3D tiles model coordinates into logitude and latitude coordinates. 

Second step,according to the model latitude and longitude coordinates and image tile geographical coordinates, I can compute the image 

UV values for each 3d tile model vertex. Third step, render the model like GlobeFS.glsl. but the result is incorrect.

      

        Can you give me some suggestion?

 

[James Cuff]

Hi Patrick, 

I would like to talk to you about converting floor level polygon data for large cities into 3D tiles. I've been digging around and have had little success finding a way to do this conversion. 

Any assistance would be welcomed. 

thanks, 

James. 

On Thursday, May 11, 2017 at 6:26:23 AM UTC-6, Patrick Cozzi wrote:

Hi all,

One of the most common questions we receive is "how do I convert my data to 3D Tiles?"  Given that Cesium's 3D Tiles renderer will be ready in the next few months, I expect this question to become even more popular.

In the short term, the answer is to please email me, pco...@agi.com, and ask.  We likely already have in-house tools that can convert your data to 3D Tiles, and we're happy to run them with your data.

We have tilers for several types of data (CAD, photogrammetry, 3D buildings, point clouds, etc.) and have successfully worked with many folks, and have written a few blog posts with the results, for example: SiteSee Photogrammetry and 3D Tiles.

Slightly longer term, these tools will be part of Cesium Composer, a commercial web service and OnPrem platform we are developing to further expand and fund open-source Cesium development.  You can sign up for the beta here.

Thanks,

Patrick
--

twitter.com/pjcozzi

[Patrick Cozzi]

James - reach me at pco...@agi.com

Patrick

--

You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/xLkYIuku9hA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cesium-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

twitter.com/pjcozzi

[James Cuff]

Hi Patrick I just sent you an email at pco...@agi.com but it did not resolve. Is that the correct email? 

Thanks,

James. 

To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

twitter.com/pjcozzi

[Patrick Cozzi]

I don't know why it was truncated.  It is pcozzi at agi dot com.

To unsubscribe from this group and all its topics, send an email to cesium-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

twitter.com/pjcozzi

[pradeep kumar]

Hi all,

I've been using cesium for 3d viewing.

For Converting .LAS to 3D point tileset for Cesium, I've google it and got the reference link as 

https://github.com/mattshax/cesium_pnt_generator

Setup all the settings in my Ubuntu Server 16 LTS but I do have only .LAS file only  but when running the conversion script

./process_lidar.sh doll.las image.tif 15 viewer/data -20 2994

I've got errors because i don't know what the image.tif parameter was actually referring to ?

Kindly help me 

[Gabby Getz]

Hi Pradeep,

We don't have any official connection with the cesium_pnt_generator project, you'll need to ask the developer directly.

Thanks!

Gabby

--
You received this message because you are subscribed to the Google Groups "cesium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cesium-dev+unsubscribe@googlegroups.com.