Displaying 300k objects

[Jimbo J.]

Hi guys,

I need to display around 300,000 objects using Leaflet 5.0.

The objects are small rectangles completely tiling a large area (e.g. North America) -- please see the attached picture. Each one has some associated metadata.

I add each rectangle as a layer to a LayerGroup (several such groups in all), and then add those layers groups to a LayersControl.

The problem is, this setup is slow on basic pan/zoom operations, and almost freezes on more intensive operations, like hiding a LayerGroup (interestinly, toggling the group _on_ is nowhere as slow).

Is there any way to make that at least usable (I don't need it to be lightning-fast)?

For example, on lower zoom levels it would be OK to merge the rectangles into larger ones, as long as the tiling is kept. MarkerCluster doesn't work for that case (its greedy algorithm doesn't guarantee tiling).

Any suggestions?

Thanks!

[Vladimir Agafonkin]

Hi Jimbo,

That's quite an interesting task! Regarding removeLayer performance, please try the latest master version — it was fixed there. Also, try toggling Canvas backend for vector layers — it can make a big difference when there are lots of layers.

To optimize it further, I guess the only way is to develop some custom square merging algorithm utilizing fast R-tree implementation for storage. 

Cheers!

2013/2/7 Jimbo J. <diane.h...@gmail.com>

--
 
---
You received this message because you are subscribed to the Google Groups "Leaflet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leaflet-js+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Vladimir Agafonkin
http://agafonkin.com/en
+380 (93) 745 44 61

[Jacob Toye]

Ho much control do you have over you map tile rendering service? And how static is your data? You could render this information from a mapserver and use something like Leaflet.UtfGrid (https://github.com/danzel/Leaflet.utfgrid) to add interactivity.

[Bryan McBride]

Sounds like a good case for CartoDB with Leaflet http://developers.cartodb.com/documentation/cartodb-js.html. You might also want to look into TopoJSON to eliminate some polygon redundancy. Leaflet integration demonstrated here: http://bl.ocks.org/4554134.

BRYAN

[Alec Bennett]

Is there sub geometry to these polygons, or does each rectangle just have associated data with it?

This looks like a dataset that could be rasterized, tile, served via MapServer, and then perform a check to see which pixel is clicked, and retrieve that pixels data.  Trying to serve 300,000 rectangular polygons is far more complex than rasterized data in this case.

-Alec

[Alic Wired]

+1 on Alec (and not just because my name's Alic :)

 

That's the best way to make it efficient and scalable. If you don't have access to a a GIS server (like ESRI's ArcGIS, MapDotNet, etc. that would generate image tiles for you from a raster map service), you can write a very simple web service that would generate an image for the image overlay (see L.ImageOverlay in API). You can further optimize it by passing map extent on each map change to your web service to generate just the visible portion of the raster.

 

Mind you, I've only looked at Leaflet JS for a couple of days, so not sure about any optimizations that may be introduced. For example, if Leaflet does internally cache the overlay image outside the map viewport, then you may be better off generating a larger image to make your panning smoother, etc. etc.

[Todd Albert]

I'm displaying 625,000 vector polygons on my map. I'm using Leaflet to display everything - Bing Aerial Tiles and I use TileStache to tile and display the polygons right from a PostGIS database. These polygons all have a parcel-ID associated with them. Those parcel IDs are then associated with data in two other databases that I can access when the user clicks. It's pretty versatile and elegant for my needs.

Check it out:

http://www.madvalorem.com/search/

We plan to scale this up shortly to the two counties to the south – Miami-Dade and Broward (Ft. Lauderdale) counties which will bring us up to 2.2M polygons! We also plan to combine the property data and professional data into a single database to make things a bit easier (and cheaper to maintain).

-Todd

[Rui Leal]

Hello everyone!

Quite related with this large numbers, I'm having a strange issue with about 30k-40k points (in these case, Circles) using Canvas backend. 

Not rendering performance but removeLayer perfomance. I've tested this with Leaflet 0.5.1 and the lastest branch master (grabbed yesterday). 

It appears to still have some very slow performance ...

I'm using a L.geoJSON layer to load some geoJSON data from a server-side component. On a zoom in or zoom out, i remove the previous layer and load another. 

Simply like this:

$.getJSON(url, function(data) {

currentLayer = new L.geoJson(data, {

pointToLayer: function (feature, latlng) {

var circle = new L.Circle(latlng, 1, { ....});

return circle;

}

});

map.addLayer(currentLayer);

});

and removing the layer, simply like this: 

function onChangedZoom(e) {

if(currentLayer !=  undefined) {

map.removeLayer(currentLayer);

}

       ......

}

I'm having terrible performance on my very old test machine, a 2006 Macbook Pro (CoreDuo 2.66 Ghz, 2GB RAM) and similar results (propotionally, i mean) on a colleague Windows machine (i5 2.27GHz, 4GB Ram) ... both on Chrome.

Some numbers on leaflet 0.6-dev:

MacBook Pro:

Layer Create time: 4817ms (L.geoJSON after already having the geoJSON data on the client)

Layer Render time: 4528ms (map.addLayer)

removing layer... 

Removed Layer time: 13857ms (map.removeLayer)

Windows machine:

Layer Create time: 3458ms (L.geoJSON after already having the geoJSON data on the client)

Layer Render time: 4057ms (map.addLayer)

removing layer... 

Removed Layer time: 5316ms (map.removeLayer)

Is it me or removeLayer is really slow ... does it make sense being slower than addLayer?

Any input or suggestion?

Thanks in advance …

Regards, 

Rui Leal

[Chris Galli]

Hi Jimbo,

A very interesting problem! How's the battle going? I see you posted this in Feb. If you have any new info, performance snags, approaches to overcome the initial issues, or new solutions all together, please share. I'd love to hear how you tacked this using Leaflet.js.

Cheers,

-Chris