GeoJSON performance? River vector tile map.

[Nelson Minar]

Is anyone here knowledgable about GeoJSON performance in Leaflet? I'm just wrapping up work on a tutorial about building maps with vector GeoJSON tiles. One of the things I did was implement more-or-less the same map of US rivers using Polymaps, D3, and Leaflet (using Glen Robertson's leaflet-tilelayer-geojson). The Leaflet map is the most capable thanks to Leaflet's excellent UI controls, but it's also the slowest to render. Now I'm trying to understand why and what I can do to help Leaflet render faster.

You can compare Polymaps and Leaflet performance at these two URLs:

  http://www.somebits.com/rivers/rivers-leaflet.html#6/38.307/-114.675

  http://www.somebits.com/rivers/rivers-polymaps.html#6.00/38.307/-114.675

Rendering from cached tiles, Leaflet takes about 4 seconds to render that map. Polymaps is about 0.5 seconds. It's a lot of vector data; 1140kB for that view (depending on browser size), with a lot of convoluted lines.

I've poked around with Chrome Developer Tools and don't see any obvious problems in what Leaflet is doing. Just spending a lot of time building up SVG paths. But I'm not expert at Leaflet's code, I could easily have missed something.

The full project source is here:

  https://github.com/NelsonMinar/vector-river-map

Also for an idea of how fast something could be, here's a D3-only renderer:

  http://www.somebits.com/rivers/rivers-d3.html

The D3 version is not really a fair comparison because it's not a complete map, just rendering the vector geometry. But it's awfully fast, a nice lower bound for what performance could be.

[Nelson Minar]

I have another, fairer comparison of Leaflet GeoJSON rendering speed. Here's the same map with two different Leaflet plugins for rendering GeoJSON vector tiles. One uses Leaflet's own GeoJSON library to render the SVG, the other uses D3.

http://www.somebits.com/~nelson/tmp/rivers-leaflet-geojson-nopopup.html#5/38.100/-97.822

http://www.somebits.com/rivers/rivers-d3leaflet.html#5/38.100/-97.822

With everything cached it takes Leaflet/GeoJSON 5 seconds to render that view whereas Leaflet/D3 takes 0.5 seconds.

Any ideas how to improve Leaflet/GeoJSON performance? It's certainly possible that there's something in the way this plugin works that's making things slow. But the code is pretty simple and poking around in the Chrome profiler I don't see much in the plugin itself. Maybe no one's tried to optimize GeoJSON rendering for this much vector data before?

I did notice a lot of time was spent in L.Path.L.Path.extend._initEvents. It appears that Leaflet is adding event listeners to every SVG element even if no one is subscribing to them. That's not wasted work in general; a key part of why I want to use Leaflet is to use its popup-on-click support (I removed it for my performance testing). But maybe there's a way to do this more efficiently?

Grateful if anyone has insight,

  Nelson

[norbert...@gmx.de]

These are interesting comparisons. I've been investigating this and updated with your latest example. I have made some patches for optimization and it takes now less than half the time on my machine:

http://bl.ocks.org/nrenner/raw/5635334/#5/38.100/-97.822
https://gist.github.com/nrenner/5635334

More on this later,
Norbert

[Nelson Minar]

Thank you so much for looking at this! Your changes bring the time down from 5 seconds to 2 seconds, that's very good. It looks like you're changing the way code applies per-feature styles; do you think you this optimization can apply to the default Leaflet API?

--
 
---
You received this message because you are subscribed to a topic in the Google Groups "Leaflet" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/leaflet-js/_WJquNpdmH0/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to leaflet-js+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Norbert Renner]

> do you think you this optimization can apply to the default Leaflet API?

I only tried to avoid calling the most time consuming functions in this
specific example for a better comparison and to see how fast it could
get. So this is rather stripping down unnecessary features for that one
use-case than actually optimizing Leaflet code and thus may only be a
starting point for optimizing Leaflet itself. But I don't want to spend
more time on this right now, so anyone feel free to take this on.

Here's a short description of what I've done:

I added a "console.profile" Statement around
L.TileLayer.GeoJSON#_tilesLoaded to automatically collect CPU profiles
in Chrome for the part that I considered most relevant for the
comparison. In the profile I looked for the most time consuming ("Self")
functions:

(1) L.DomEvent.addListener

in L.Path.L.Path.extend._initEvents (Path.SVG.js)

Disabled Path event registration and therefore interactivity (e.g.
Popups) by setting:
L.Path.options.clickable = false;

(2) L.Util.extend (L.setOptions)

in L.Path.L.Class.extend.setStyle
in L.Path.L.Class.extend.initialize

Worked around using L.setOptions by setting default styles (incl.
application-specific) on svg root element once and simply setting (not
merging) the options property with dynamic styles. The other options
(e.g. clickable) then need to be set individually in initialize. See
"L.Map.include" and "L.Path.include" sections - these are
application-specific and not generally applicable without further work.

(3) L.Mixin.Events.addEventListener

in L.FeatureGroup.L.LayerGroup.extend.addLayer

Disabled FeatureGroup event registration by setting
L.FeatureGroup.EVENTS = '';

(4) "Recalculate Style" in Chrome Timeline

in L.Path.onAdd: this._map._pathRoot.appendChild(this._container);

Minimized by removing the SVG root element from the DOM while adding
elements to it (see L.TileLayer.GeoJSON.prototype._tilesLoaded
interception).


The reason I'm interested in this is that I'm working on the
leaflet-tilelayer-geojson project (still buggy):
http://bl.ocks.org/nrenner/raw/5638517/#5/38.100/-97.822
https://github.com/nrenner/leaflet-tilelayer-vector

And Calvin Metcalf is adding some Web Worker magic:
https://github.com/calvinmetcalf/vector-layers

[Nelson Minar]

Thank you for your experiment and the detailed writeup. I don't have the time to look closely at completing this work right now either, but your start looks very promising.

(4) "Recalculate Style" in Chrome Timeline

  in L.Path.onAdd: this._map._pathRoot.appendChild(this._container);
Minimized by removing the SVG root element from the DOM while adding elements to it (see L.TileLayer.GeoJSON.prototype._tilesLoaded interception)

This seems like a straightforward improvement that doesn't need any further consideration. I guess the drawback is it hides the SVG (all layers) from the screen while tiles are being loaded.

The reason I'm interested in this is that I'm working on the leaflet-tilelayer-geojson project (still buggy):
  http://bl.ocks.org/nrenner/raw/5638517/#5/38.100/-97.822
  https://github.com/nrenner/leaflet-tilelayer-vector
And Calvin Metcalf is adding some Web Worker magic:
  https://github.com/calvinmetcalf/vector-layers

That looks interesting too! I also think this plugin (Glen Robertson's) is the most promising, in that it's the most Leaflet-like and complete. It works fine, it's just a little limited.

I see you've tackled the problem of handling clipped features, merging geometries, etc. Curious what you come up with; it's not clear what the best solution is.

[Norbert Renner]

Sorry for the late answer:

> (4) "Recalculate Style" in Chrome Timeline
>

> in L.Path.onAdd: this._map._pathRoot.__appendChild(this._container);

> Minimized by removing the SVG root element from the DOM while adding

> elements to it (see L.TileLayer.GeoJSON.prototype.___tilesLoaded

> interception)
>
> This seems like a straightforward improvement that doesn't need any
> further consideration. I guess the drawback is it hides the SVG (all
> layers) from the screen while tiles are being loaded.

This actually only works because all tiles are added at once and the UI
is given no time to update so you don't see that the states layer is
temporarily being removed. When adding single tiles all SVG gets
recalculated each time, so it's not an optimization in that case. The
idea is to have a parent group element per tile that can be added and
removed in a single operation. I might have a look into that.

> I see you've tackled the problem of handling clipped features, merging
> geometries, etc. Curious what you come up with; it's not clear what the
> best solution is.

No, I just extracted deduplication of unclipped tiles into a subclass,
the base class takes the tiles as they are. Glen's reunioning branch
does merging of clipped features, but I don't know what the state there
is. My tile source (Mapsplit) does not support clipping yet so I'm
ignoring that case for now.

Norbert

[Fabian Zeindl]

I studied this for two days now, tried to reproduce the steps, went through leaflet's code and did profiling in Firefox and Chrome.

I could not reproduce that the skipping of events-registering for FeatureGroup leads to better performance in Chrome, for example.

It seems to me that there are not a lot of hotspots where leaflet could be optimized without changing functionality, but rather that leaflet's general class-structure produces the overhead.

I noticed that a lot of the time is actually spent in the construction of LatLng objects and garbage-collecting.

[Norbert Renner]

> I could not reproduce that the skipping of events-registering for
> FeatureGroup leads to better performance in Chrome, for example.

I still can reproduce the profiling results in Chromium on my new
machine. For better comparison I added a Gist without optimization but
with Chrome profile recording, see below. But I just noticed that the
line appearance is slightly different, so this might not even be a valid
comparison. Also, the example should actually be updated to current
versions.

The FeatureGroup optimization is the least significant, but seemed so
unnecessary in this case and is easy to fix, which is why this is the
only optimization I actually use in my own project right now.

> It seems to me that there are not a lot of hotspots where leaflet could
> be optimized without changing functionality, but rather that leaflet's
> general class-structure produces the overhead.
> I noticed that a lot of the time is actually spent in the construction
> of LatLng objects and garbage-collecting.

These optimizations reduce unneeded functionality for this specific use
case but are hard to generalize. I don't really know d3, but I also
guess that the biggest overhead compared to d3 is probably the renderer
abstraction using an internal vector object model (the Path classes).

The biggest hotspot in my opinion is that each Path.SVG object is adding
itself to the SVG root which leads to a lot of unnecessary "Recalculate
Style" in the Chrome Timeline, see step (4). I hope this gets solved or
easier to fix with the planned Leaflet Layer refactoring.


non-optimized
http://bl.ocks.org/nrenner/raw/7535077/#5/38.100/-97.800
https://gist.github.com/nrenner/7535077

console.time: addData: 1838.634ms

Profile Total: 1221 ms

Self Function
257 ms L.DomEvent.addEventListener
174 ms L.Util.extend
163 ms (garbage collector)
100 ms setAttribute
94 ms L.Mixin.Events.addEventListener
71 ms addEventListener
...

optimized
http://bl.ocks.org/nrenner/raw/5635334/#5/38.100/-97.800
https://gist.github.com/nrenner/5635334

console.time: addData: 801.183ms

Profile Total: 519 ms

Self Function
77 ms L.Util.extend
76 ms (garbage collector)
64 ms L.Mixin.Events.addEventListener
45 ms L.Path.L.Class.extend.onAdd
40 ms L.Util.splitWords
39 ms setAttribute
...

[Vladimir Agafonkin]

Thanks a lot for the great insights everyone. I'll be looking at this matter closely, because 5s vs 0.5s is crazy. I want Leaflet to be as fast as possible. Yeah, generalized Leaflet vector API makes some performance compromises to make the API easy, but with some careful thought, I think we could optimize it nicely without any API changes.

I'll definitely try event delegation to reduce time spent on adding listeners to individual features, and I also think the planned layer refactoring will allow Leaflet to handle adding/changing/removing groups of features in batch (meaning no multiple relayouts when we can do one, etc.).

[Fabian Zeindl]

Thank you.

I've also noticed that performance is terrible on Firefox / Mac (ONLY on Mac).
I think it has to do with event-listeners. I've filed a bug-report here: https://bugzilla.mozilla.org/show_bug.cgi?id=942579

Regards
Fabian Zeindl

> --
>
> ---
> You received this message because you are subscribed to a topic in the Google Groups "Leaflet" group.

> To unsubscribe from this topic, visit https://groups.google.com/d/topic/leaflet-js/_WJquNpdmH0/unsubscribe.

[Vladimir Agafonkin]

Fabian,

I don't think this has to do with event-listeners — Firefox generally has terrible SVG performance compared to other browsers. OSM editor iD has the same problem.

https://bugzilla.mozilla.org/show_bug.cgi?id=869505

https://bugzilla.mozilla.org/show_bug.cgi?id=837985

[Norbert Renner]

Vladimir,

thanks a lot for picking this up and optimizing performance.

It seems like the rivers example is about 8 times faster now on Chrome
(see below)! Very impressive!

Leaflet v0.7.1:
http://bl.ocks.org/nrenner/raw/8184956
https://gist.github.com/nrenner/8184956

addData: ~ 3100ms

Leaflet master (v0.8-dev):
http://bl.ocks.org/nrenner/raw/8184978
https://gist.github.com/nrenner/8184978

addData: ~ 370ms

[David Blackman]

Wanted to say thanks too, Vlad. Performance on zetashapes.com went from embarrassing to excellent.