SVG tiling

[Satish Krishnaraj]

Hi,

Apologies if my requirement is really basic.

I need to implement tiling for upwards of 10,000 svg elements (points, circles, paths, .....). Because of performance I am hesitant to render all the elements at the same time so considering to implement  tiling.
Most of the libraries I have reviewed (polymaps in particular) work with raster images, maps , overlays which are tied to the geo (lat / long) data, and for my application I am not interested in those.

I would appreciate your recommendation in terms of the approach, and libraries to consider to implement the same.

Thanks & Regards,
-Satish

[Andy]

I find 5-10K elements in Chrome on a Macbook AIR to work fine, FF/IE9
are a little clunky but useable.

Possibly you could try dynamically adding/removing elements from the
SVG depending on view vicinity.

[Nelson Minar]

I need to implement tiling for upwards of 10,000 svg elements (points, circles, paths, .....). Because of performance I am hesitant to render all the elements at the same time so considering to implement  tiling.

Try just rendering them all first and see how it works, but I think you're right that 10,000 is too many. My experience with windhistory.com was that even 2500 SVG circles caused a noticeable slowdown of panning (in Chrome on desktops).

One approach is to load all the data once but only draw SVG for the parts that are visible. Mike wrote a simple SVG tiler for me for use in Polymaps, but I think there's nothing Polymaps-specific in d3.geo.tiler. You can see it here:

  http://bl.ocks.org/900050

The key function is tiler.tile(x, y, z) which returns an array of all data points visible within the square defined by that x/y location at that zoom level z.

If you have a whole lot of data, you don't want to load it all at once. Polymaps has support for "vector tiles" that work a lot like raster tiles. Only loads bits of geometry when they are visible. I've never had to do this; in my case it seemed better to just load the data in one big megabyte blob.

[Satish Krishnaraj]

Thanks for quick answer(s).

I reviewed the code mentioned and here are my follow up questions - if I may...........

First off, the data/points are in lat/long (or may be in mercator projection).
However, in my situation all the data points are on plane projection. If at all I have to modify the custom tiler, what would be your recommendation to implement the same (to use plane projection instead of geo*)?

Thanks and regards,
-Satish

[Nelson Minar]

On Tue, Sep 27, 2011 at 9:13 PM, Satish Krishnaraj <sati...@gmail.com> wrote:

However, in my situation all the data points are on plane projection. If at all I have to modify the custom tiler, what would be your recommendation to implement the same (to use plane projection instead of geo*)?

I don't honestly know. I don't think the tiler code much cares about the projection; you could probably replace it with your projection, a no-op projection, or take the projection code out entirely.

[Benjamin West]

I'm also interested in this. I think you should be able to create a
PLANE projection. I did this for OpenLayers in PROJ4JS.
The EQC projection is provided by PROJ4JS.
https://github.com/bewest/proj4js/blob/master/lib/projCode/eqc.js

I tried integrating it with OpenLayers but failed in various ways.
Their code seems to assume too much about being in a spherical
projection where the scale for x and y are dependent on each other.
The result is that their SVG layers fail to show up because they use
their own code for establishing the resolution. When I saw polymaps
and d3 come along, I was excited to give it a try, but I'm still
working through lots of hello world examples on bl.ocks.org.

I suspect some of the polymaps tiling code could be ripped out and
added to d3 in order to do this kind of thing fairly easily in
addition to adding some sort of plane projection. It sounds like one
of the things to watch out for is the need to track the origin
correctly in order to provide an API to setCenter/getCenter and panTo.

I think that's the basic way to go forward, but I still don't have a
good enough understanding to hack it together. I had someone's hex
example converted into drawing something like a tile and had planned
on attempting to add some projection support, however it may be easier
to start with the tiling code from polymaps?

-bewest

On Tue, Sep 27, 2011 at 9:13 PM, Satish Krishnaraj <sati...@gmail.com> wrote:

[Benjamin West]

Sorry, I should have included this stuff as well.

Here are my attempts with OpenLayers and some other things:
https://github.com/bewest/diabetes/tree/master/src/mock
python simple.py # starts a simple server that generates tiles for
openlayers. The tiler uses matplotlib to print all arguments onto the
tile.
I thought perhaps eventually using a single sprite as backgrounds for
various pieces of chart anatomy might be more efficient, but d3 seems
more than capable of generating all the components of the chart in the
browser.

Anyway, https://github.com/bewest/diabetes/blob/master/src/mock/templates/openlayers.html
attempts hi-jacking OpenLayers projection and rendering just enough to
plot a time series that pan/zooms only within the restricted range.
I'm hoping to implement a much cleaner solution in d3.

-bewest

[Benjamin West]

I guess, just to be clear, I think you want tiling in an arbitrary
projection, where x and y are scaled independently, so lat/long means
some other arbitrary unit, right?

-bewest

On Tue, Sep 27, 2011 at 9:13 PM, Satish Krishnaraj <sati...@gmail.com> wrote:

[Satish Krishnaraj]

That is partly correct.

Will put it in another way -

Suppose the user is viewing 800 X 600. However, the svg(s) could be on 2400 X 1800. The user should be able to pan / zoom but with tiling. As the tiles become visible, the appropriate svgs need to be fetched from the server and displayed. In my situation, the svgs are paths/polygons not points / circles. With points / circles, chances of it belonging to  a rectangular tile is high. Whereas with paths / polygons it could be on multiple tiles and that is the real challenge.

Hope I have clarified properly.
Thans & Regards,
-satish