Native GPX and Shapefile support

[Matthew Amato]

I have the pleasure of once again mentoring André Nunes as part of this year's Google Summer of Code.  Long time mailing list readers may remember André from the initial KML support he worked on as part of in GSoC 2013. This time around he will be implementing native Cesium support for both GPS Exchange Format (GPX) and Shapefiles. While some users have had success converting these formats to ones Cesium already supports, like GeoJSON or CZML, native support will be more complete and also make it easy for Cesium to ingest the millions of files that already exist.

If you're interested in either of these formats, consider this the official place to discuss them and ask any questions you may have. André will also be posting status updates and other info here as he makes progress.

Welcome back André!

[bobac...@gmail.com]

Hello,
do you intend to use this repository ( https://github.com/wavded/js-shapefile-to-geojson) to parse shapefile?

[André Nunes]

Thanks Matt.

Hi everyone, it's a pleasure to work with Cesium once again. 

If you want to follow my progress, here are the links for my branches:

Shapefile: https://github.com/andre-nunes/cesium/tree/shapefile
GPX: https://github.com/andre-nunes/cesium/tree/gpx)

GPX is smaller and easier than shapefiles so I will begin with that format. Feel free to discuss and ask any questions you may have.

[André Nunes]

Hi,

Writing a parser was a possibility but I think using an existing one will be more practical, we just need to find the best parser for us... do you recommend that one?

In my initial research for Shapefile parsers I also found:

https://github.com/RandomEtc/shapefile-js/

https://github.com/mbostock/shapefile

Other suggestions are welcome.

[Matthew Amato]

We will definitely investigate third-party data readers for Shapefiles, but we will not be converting to another format, like GeoJSON, behind the scenes. Converting to an intermediate format will not only be slower, but also be incomplete, since GeoJSON can not represent all shapefiles. For example, as far as I know there is no GeoJSON equivalent to shapefile's MultiPatch geometry.  If I'm mistaken,let me know.

André, shapfile-js has not been updated since 2011, and mbostock/shapefile is a node.js library and not fit for running in a browser.  We want this to be a complete client-side implementation.  If there isn't a suitable OSS client-side shapefile reader out there, we will have to either roll our own or contribute back to an existing one to get it up to snuff.  I'm pretty sure I've come across competent ones in the past though, so I'll see what I can dig up.

--
You received this message because you are subscribed to the Google Groups "cesium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cesium-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Patrick Cozzi]

Welcome back André!  I'm looking forward to your progress.  Have you posted the roadmap?

Patrick

To unsubscribe from this group and stop receiving emails from it, send an email to cesium-dev+unsubscribe@googlegroups.com.

[André Nunes]

Thanks Patrick!

I've posted the initial draft for each format's roadmap as a GitHub issue:

Shapefile: https://github.com/AnalyticalGraphicsInc/cesium/issues/2750
GPX: https://github.com/AnalyticalGraphicsInc/cesium/issues/2751

Please add the label when you can, feel free to edit and/or give feedback.

To unsubscribe from this group and stop receiving emails from it, send an email to cesium-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/Ov014I88lMA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.

[Patrick Cozzi]

Thanks André, the roadmaps look good.  Any thoughts on Shapefile metadata and using it for styling?

Patrick

--

twitter.com/pjcozzi

[Berwyn McKilligan]

Another suggestion for javascript shapefile processing

https://github.com/calvinmetcalf/shapefile-js

the author of this was also working on 

https://github.com/proj4js/proj4js

this appears to the official javascript port of proj4, a major component of the OSGeo/gdal stack

https://github.com/OSGeo/proj.4/wiki

the coordinate transformations are going to be a big part of the shapefile processing

[Matthew Amato]

Thanks for the links.  We're well aware of proj4js and have even considered depending on it in Cesium proper in the past, but haven't had a good reason to yet.  The shapefile-js looks promising, but if it only handles things that can be converted to geojson (as it's project description leads me to believe), it won't be good enough for our needs (see my original comments above).  It's worth looking into, and perhaps we can even contribute something back to it in order to improve it enough for Cesium's use cases.

[Patrick Cozzi]

André - do you have an update on your work?

Thanks,

Patrick

To unsubscribe from this group and stop receiving emails from it, send an email to cesium-dev+unsubscribe@googlegroups.com.

[André Nunes]

Hi,

I'm sorry for the lack of progress but as I mentioned in my application, the initial part of GSoC clashes with my school's exam season. The last couple of weeks have been very busy and I couldn't get anything done. The rest of the week should be calmer and then I have another exam on June 16th.

My focus for the rest of the week is extending CesiumViewer to load .gpx documents and implement a GpxDataSource to process these documents generating the respective geometric types to be rendered by the viewer. I want to start experimenting with simple GPX's Waypoints until they show up on CesiumViewer. Then I'll move on to the other features (Tracks and Routes).

I'm also trying to draw some parallel between KML's geometry types and GPX's:

- GPX's Waypoints are very similar to KML's Placemarks. Contrary to KML there's no style, just WGS 84(GPS) coordinates of a point and possibly other descriptive information (elevation, name, comment, etc).

- GPX's Tracks are very similar to KML's gx:Track. They are an ordered list of routepoints (waypoints) leading to a destination.

- GPX's Routes basically Tracks without timestamps. The points should be connected to form a line like KML's LineString.

Thanks,

André Nunes

To unsubscribe from this group and stop receiving emails from it, send an email to cesium-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/Ov014I88lMA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.

[Patrick Cozzi]

Thanks for the update.  Good luck with your finals.

Once you are in full swing, please make sure to post demos as you have them.

Patrick

--

twitter.com/pjcozzi

[bmcki...@conetec.com]

came across this article today
http://www.directionsmag.com/pressreleases/4dmapper-launches-geospatial-industry-game-changer/448056

quick look at source, looks like they are using cesium leaflet ... not sure if there is any issues no credits being displayed

they have proj4js in there

no support for .shp files

[Kevin Ring]

Looks like that application is using TerriaJS, which we originally developed for Australia's National Map.  Plus some unique server-side bits.

Looks like the imagery is from MapBox and looks really nice.  I hadn't realized MapBox's imagery had gotten that good.

I'm not sure about the 4DMapper setup, but TerriaJS itself does support shapefiles via conversion to GeoJSON on the server.  Once Cesium has native support for shapefiles (thanks André!) we'll switch to using that.

--
You received this message because you are subscribed to the Google Groups "cesium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cesium-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

www.kotachrome.com/kevin

[André Nunes]

Hello everyone!

I want to apologize for the lack of progress, I was really busy with my final exams and couldn't focus on both things at the same time.

I'm done now and I'll be able to give my full attention to this project, more updates soon.

Thanks for your patience.

[Denver Pierce]

Good luck Andre.

Have you looked at zipped shapefile support at all? Zip.js is already being loaded for KMZ support, and it'd be great to have the additional functionality.

[André Nunes]

Right now I'm working on GPX support but I see no reasons why zipped shapefiles shouldn't be supported in the future.

Quick question related to GPX, the format specifies symbols/icons of Waypoints by name, no href and there's no list of "standard" icons that I could find, it depends on vendor. I found some icons supported by Garmin which appears to be the most famous vendor using GPX:

Garmin symbols

More Garmin

Big icon library with all gps symbols supported by Garmin

GPSBabel icon list

What icons should be supported by Cesium? I'd like to read input from the community and the main developers.

Thanks.

[Matthew Amato]

Am I correct in that the spec doesn't specify a set list but just allow for string descriptions?  If so I think the best approach would be to allow users to specify a default mapping of names to images.  By default we could just use a simple pin like we do in GeoJson.

[André Nunes]

You are correct, but how will the user specify the default mapping? How will that work in practice?

[André Nunes]

FYI, I'm almost done with GPX support, all the geometric features are working and just need to be made "pretty". There's some issues with time dynamic Tracks but nothing I can't fix in a couple of days. I'm improving test coverage right now and implementing some missing (non visual) features.

I'd appreciate any example .gpx files you can provide for me to test this further, also feel free to fork my repository and check out the gpx branch to test it yourself. 

Patrick Cozzi, you mentioned Demos somewhere, could you please be more specific on what kind of Demos would be appreciated? Would I need to host a cesium app to showcase these Demos?

Thanks,

André Nunes

[Patrick Cozzi]

Hi André,

I think just a Sandcastle example for GPX would be fine.

Patrick

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/Ov014I88lMA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cesium-dev+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

twitter.com/pjcozzi

[yoh...@gmail.com]

Sorry for reviving an old thread...

Was this ever merged into the main project (GPX loading)? I can't find references or sandcastle example.

Thanks,

[Hannah Pinkos]

Hello,

No, this was never merged.  See this pull request: https://github.com/AnalyticalGraphicsInc/cesium/pull/2939

Best,

Hannah

[Left Gully]

So I'm stuck writing my own gpx datasource code for cesium. Is this trivial, so you're leaving it to users to develop? I've got something to work, but really, your development efforts haven't helped me much in the last 3 years since you positioned cesium as a platform to replace the GE plugin.
Inability to clamp geoJSON polygons (or polylines) to terrain has left my live project in 2D google maps, while I waste valuable hours trying to make cesium work as a 3D GIS data viewer.

[Tom Fili]

Hi
Geojson has had a clampToGround option since 1.23. With this set to true points, polygons and polylines are drawn on terrain. One caveat is that polylines are drawn as corridors, so the width is set in meters and not pixels.

--Tom

[Jonathan Hall]

Thanks Tom I will look at that

--
You received this message because you are subscribed to a topic in the Google Groups "cesium-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cesium-dev/Ov014I88lMA/unsubscribe.

To unsubscribe from this group and all its topics, send an email to cesium-dev+unsubscribe@googlegroups.com.

[Umar Sharif]

Hello,

I am trying to run gpx file in cesium. I found your github , downloaded cesium source, but that does not run. Can you please help me with that. I believe you already have gpx cesium support.

Thanks

[Rachel Hwang]

Hi Umar,

We currently have a pull request open for GPX support: https://github.com/AnalyticalGraphicsInc/cesium/pull/2939

I don't think it's complete, but perhaps it can help you. For some help downloading and building the codebase locally, see our build guide: https://github.com/AnalyticalGraphicsInc/cesium/tree/master/Documentation/Contributors/BuildGuide

Hope that helps!

- Rachel