Leaflet GeoJson Performance

[Fabian Zeindl]

Hi,

 I have a leaflet map with thousands of GeoJSON features and seek to improve performance.

My questions:

* I'm loading features based on bounds. Is it faster to remove the entire GeoJson-layer and add a new one, or is it ok to clear it and add the new data like this;

geoJsonLayer.clearLayers();

geoJsonLayer.addData(data);

* Can I add features to the layer with duplicates automatically thrown out?

* I have to swap lat/lng bounds since PostGis does them the other way rounds. Is there a way to tell leaflet to use reversed couples?

Regards and thank you

Fabian Zeindl

[Nicklas Aven]

Is the bottleneck rendering or the bandwidth?

If it is bandwidth you might be successfull turning on gzip compression if you are using php. With websockets I have not tried any library supporting compression on the fly.

You might also be interested in testing to export the data in twkb format instead of geoJSON. Twkb is a very new format that I have constructed and implemented in PostGIS. It is now in PostGIS trunk.

It is far from optimized yet but still a lot smaller than geoJSON.

Take a look at the demo-sites
1) comparing twkb with geoJSON:
http://178.79.156.122/twkb_test
2) just showing twkb on a leaflet map. Data is sent by a websocket:
http://178.79.156.122/twkb_node

There is also links to blog posts about twkb.

Best Regards

[Nelson Minar]

On Tuesday, July 30, 2013 7:19:22 AM UTC-7, Fabian Zeindl wrote:

 I have a leaflet map with thousands of GeoJSON features and seek to improve

There's no complete solution here, but you may find this previous discussion useful:

https://groups.google.com/d/topic/leaflet-js/_WJquNpdmH0/discussion

Long story short, I've got a tiled layer that uses GeoJSON to draw lots of Polylines. Doing it with Leaflet's GeoJSON was very slow. Norbert Renner did some work removing parts of Leaflet's GeoJSON system until he got something that was faster. I ended up using a plugin for Leaflet that uses D3 to render GeoJSON.

https://github.com/NelsonMinar/vector-river-map/blob/master/clients/lib/TileLayer.d3_geoJSON.js

[Owen Densmore]

Interesting response from a former SFIer, and great guy to follow on Twitter:

  -- Owen

--
 
---
You received this message because you are subscribed to the Google Groups "Leaflet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leaflet-js+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Owen Densmore]

Oops meant to forward .. sorry!

[Fabian Zeindl]

On Aug 7, 2013, at 23:05 , Nicklas Aven <nickla...@jordogskog.no> wrote:
> Is the bottleneck rendering or the bandwidth?

Both, but rendering is the bigger problem. I will try the d3-geojson plugin.

> You might also be interested in testing to export the data in twkb format instead of geoJSON. Twkb is a very new format that I have constructed and implemented in PostGIS. It is now in PostGIS trunk.

What's the benefit? The demo-sites show me it's slower than geoJson.

Regards
-Fabian Zeindl

[Nicklas Avén]

> What's the benefit? The demo-sites show me it's slower than geoJson.

That is interesting, but sounds a little strange.

If you for instance use the largest of the layers: "Areal Types", the
TWKB version is 817023 bytes, and the geoJSON version is 3712864 bytes.
The server sends the TWKB-version faster than the geoJSON version
because of less bytes to handle (It is compressed directly in the
database at once after reading from disc). In total I get a timiin of
about 1200 ms to get the TWKB-geometry and about 5500 ms to get the
geoJSON from the server.

Then the timing of the parsing can vary some. At this box in Firefox
parsing from TWKB takes about 80-100 ,s and from geoJSON-string to
geoJSON takes about 150 ms. In Ipad I have noticed that geoJSON is
faster to parse than TWKB. About 200 ms for geoJSON and 400 ms for TWKB.
But still the time saved to get 800 kb instead of 3.7 MB takes much more
time than the parsing.

But I am very interested to hear about other experiences since I am
trying to optimize the implementation.

I also think it would be a lot faster if the parsing was integrated in
the rendering so the values had a shortest possible way from twkb-format
to canvas. This geoJSON-step I think slows down things significantly.


Best Regards

Nicklas Avén

[Fabian Zeindl]

On Saturday, 10 August 2013 15:49:17 UTC+2, Nelson Minar wrote:

Long story short, I've got a tiled layer that uses GeoJSON to draw lots of Polylines. Doing it with Leaflet's GeoJSON was very slow. Norbert Renner did some work removing parts of Leaflet's GeoJSON system until he got something that was faster. I ended up using a plugin for Leaflet that uses D3 to render GeoJSON.

https://github.com/NelsonMinar/vector-river-map/blob/master/clients/lib/TileLayer.d3_geoJSON.js

When I try that plugin using var myLayer = new L.TileLayer.d3_geoJSON(null), I get a TypeError t is null in leaflet. Can I not just add the layer, then later add data?

- Fabian

[Nelson Minar]

https://github.com/NelsonMinar/vector-river-map/blob/master/clients/lib/TileLayer.d3_geoJSON.js

When I try that plugin using var myLayer = new L.TileLayer.d3_geoJSON(null), I get a TypeError t is null in leaflet. Can I not just add the layer, then later add data?

That may be a bug in the plugin where it requires something else created the SVG element before the d3_geoJson layer is added. Most Leaflet layers do this automatically but I don't know enough to fix this bug. One workaround is to do this first:

// Add a fake GeoJSON line to coerce Leaflet into creating the <svg> tag that d3_geoJson needs
new L.geoJson({"type": "LineString","coordinates":[[0,0],[0,0]]}).addTo(map); 

[Fabian Zeindl]

Doesn't work either. I'm using d3-v3 and leaflet-0.6.2.

fabian

[Fabian Zeindl]

(I'm getting the same error as before).

fabian

[Fabian Zeindl]

On Aug 28, 2013, at 11:33 , Fabian Zeindl <fabian...@gmail.com> wrote:

Second follow-up: I use a TileLayer provided by Cloudmade:

var url = 'http://{s}.tile.cloudmade.com/' + CLOUDMADE_KEY + '/22677/256/{z}/{x}/{y}.png';
var tileLayer = new L.TileLayer(url, {});
this.map.addLayer(tileLayer);

and have my GeoJson layer showing additional data on top of it.

In Firefox rendering is a lot slower than in Chrome.

Regards
Fabian

[Fabian Zeindl]

I tracked down the error to this:

(line 138 in leaflet-0.6.4(

template: function (str, data) {

     return str.replace(/\{ *([\w_]+) *\}/g, function (str, key) {

It tells me "str" is null.

I'm not sure if I can use the plugin since it says it's for "tiled" layers? I'm not entirely sure what this means, because I have bitmap-map-tiles and just a GeoJson Layer that I reload on pan/zoom. There are no tiles in this layer?