D3 and map libraries

[Nelson Minar]

I'm curious, who else is using D3 to make maps? If you are, what other geo / map libraries are you using?

I've done a few maps with D3 and Javascript like http://windhistory.com/. It works great. But Polymaps has some drawbacks and hasn't had any development in awhile, so I'm looking at alternatives.

All I really need is a library to manage slippy map tile loading, panning, and zooming. Basic raster composition, so I can draw vector data on top with D3 and SVG. The tricky part is georeferencing and clean handling of zooms, pans, etc.

Leaflet and Tile5 are the trendy new Javascript map libraries; anyone using them with D3? There's also OpenLayers and Modest Maps has some Javascript support. Mike's also got a project called Pixymaps that I think isn't quite ready for prime time but could be very nice https://github.com/mbostock/pixymaps

[Mike Bostock]

Tom Carden wrote an experimental tiler with D3:

http://bl.ocks.org/1233904

Mike

[Ian Johnson]

Wow, I'm actually in the middle of a discussion with a friend of mine about when to use d3 vs. OpenLayers!

It seems to me like it would be your best bet. Use it to setup all the tricky GIS parts, and just use d3 to manipulate the svg it generates. OL has really solid development and extensive GIS capabilities, I used it a lot a few years ago and its only gotten better. I haven't had the need to make a GIS capable map with d3 yet, but it's the first place I'd go.

Anyone else have opinions? Is it more complex than it seems to just hook into the SVG made by OpenLayers and start stylin'?

--
Ian Johnson

http://enja.org

[Benjamin West]

I agree with the comment on OpenLayers. They've gone out of their way
to make driving custom map symbols easy.
http://docs.openlayers.org/library/overlays.html
Here's a document they did several years ago comparing SVG and canvas,
http://trac.osgeo.org/openlayers/wiki/Future/OpenLayersWithCanvas so I
also like their transparent and rigorous approach.

I suspect it'd be quite natural to couch d3 inside the right
"subclasses" to do nifty things.

-bewest

[Benjamin West]

Nice, I've been looking for an example like this. I'm curious: in
cube I thought you'd take a tiled approach so there's no need to keep
data around for a visualization that's panned way out of view.

On Mon, Sep 26, 2011 at 9:45 AM, Mike Bostock <mbos...@cs.stanford.edu> wrote:

[Mike Bostock]

> I'm curious: in cube I thought you'd take a tiled approach so there's
> no need to keep data around for a visualization that's panned way
> out of view.

I'll probably add this in the future when we make the visualizations
interactive. It's not necessary with the current implementation
because the visualizations only update (at most) every five minutes,
when new metrics are available. The visualization only stores the data
that is being displayed. Tiling requires deciding on a cache size, and
may be needed with interaction to pan through data quickly; however,
since the server already caches all queries, caching on the client
isn't as important as it might be with other systems.

Mike

[Tom Carden]

TL;DR: +1 on OpenLayers, the swiss army knife of open source mapping. Don't overlook Google Maps if you want plug-and-play. Choose Modest Maps or Leaflet if you want a minimal foundation and tight control over behaviors. Choose Polymaps if you want API harmony with D3 and especially if you love SVG.

Disclaimer: I wrote the Modest Maps JS library with Stamen Design and also, er, "mid-wifed" Polymaps when Mike worked with us there.

...

Each of the open source mapping libraries I'm aware of has a sweet spot:

 - OpenLayers has everything built-in, is very mature and has the largest active community around it. I believe it still uses HTML DOM for tile display but at this point everything is probably pluggable. SVG overlays are possible, and can be styled with CSS or with GIS styling languages... because everything is possible in OL :) Finding current examples and documentation has historically been the trickiest task but the pay-off is usually worth it if you need a stable platform to build on. http://openlayers.org/

 - Modest Maps is pure DOM with IE6 support, probably the smallest and I hope the fastest, but it requires you to roll up your sleeves and do the integration yourself. It hits a sweet spot for Stamen because they are in the business of bespoke visualization design. The talented folks at Development Seed have a library called Wax that helps ease integration with MM and provides basic UI out of the box. Start there if Modest Maps appeals http://mapbox.com/wax/

 - Polymaps is pure SVG and most interestingly supports tiled vector overlays out of the box. It does tiled image maps too, not to be overlooked, because generating tiled vector overlays is stilly a fairly tricky and specialist task. Polymaps and D3 are two great tastes that taste great together, because they share an author and there are examples on the D3 wiki already http://polymaps.org/ :)

 - Leaflet is a new and pretty offering and has nice interaction and UI built in. It's very well done, small in file-size, and to my shame has more features than Modest Maps out of the box (layered maps, data overlays, info bubbles, etc). Mobile support should be good because of the use of CSS transforms. http://leaflet.cloudmade.com/

 - Tile5, as mentioned by others, looks good but I've never used it. It has nice examples!

...

The D3 tile demo I posted doesn't do any "geo" projection stuff yet, but I think it shows the power of D3 and I hope in future we'll see a D3 mapping library (son of polymaps, anyone?) that uses D3's existing geo capabilities with something like this tiling approach.

I don't want to scare you off (if I haven't already!) but be aware that these open source mapping libraries put you on a slippery slope leading to custom tiles, patchy data, obscure licensing agreements, etc. ... whether you choose a full featured library like OpenLayers or a foundational one like Modest Maps, you need to make sure you know where the maps will come from. Google has historically made it difficult to use their maps with other libraries, though I think OpenLayers can do it because it's content to manage Google Maps as a "layer" like any other. OpenStreetMap is a natural choice but can't support everyone*. Look to MapQuest for open tiles on a fast CDN to get you started. Look to Bing or ESRI for a professional solution. Look to TileMill for a custom design solution.

* a note on OpenStreetMap: http://wiki.openstreetmap.org/wiki/Tile_usage_policy ... The project is a fantastic enabler for this kind of stuff but it's not always appropriate to use their tiles, especially not for commercial purposes. That's not a weird licensing constraint, it's just polite: the tiles are there to promote the project and help users improve the data. Visualizations should use tiles that are there for visualizations, something like TileMill is ideal http://mapbox.com/tilemill/ or GeoIQ's acetate tiles http://developer.geoiq.com/tools/acetate/

This is a /very/ long way of saying you should also consider using Google or Bing Maps API (or ESRI) if you want reliable consistent cartography and tons of back-end APIs like geocoding and routing out of the box. A few companies are support OpenStreetMap but their tiles/APIs aren't in wide-spread use, e.g. MapQuest and CloudMade, but they are worth a look if they have a feature you need (like nice cartography or fast servers).

I've been meaning to organize my thoughts on this subject for a while, apologies if this first draft isn't too coherent!

Best,

Tom.

[Tom Carden]

And at the risk of trying too hard to be comprehensive and completely derailing the thread (ha, too late) if anyone is interested in hacking on a D3 tiled map renderer I'd recommend looking at other non-geo projects too, like Microsoft's DeepZoom, the open source Open Zoom project or the more old school Zoomify project. The last two are in Flash but not doing anything you can't do in a browser now. Open Zoom is very well architected and documented.

There's a JS version of DeepZoom called Seajax that does very impressive things with canvas http://expression.microsoft.com/en-us/gg413362.aspx and some mapping stuff too: https://github.com/rsimon/seajax-utils - note that Seajax itself has a typically sideways Microsoft "open source" license that prevents porting it to other languages/systems. Meh!

These libraries share an approach to rendering that uses the full "pyramid" of tiled images to virtually eliminate seams but it can be difficult to tune performance. It's what I was aiming for with my D3 demo but I didn't quite get there. Some day!

Tom.

[CG]

Hi,
I am starting a project that uses an interaction of polymaps and D3
for geo-referenced data visualization and analysis.
It would be interesting to know what kind of (polymaps') drawbacks you
experienced while building windhistory.com.

Till then I am quite happy with the smooth interaction of those 2
products (thank you Mike).

Cheers,
Christophe

On Sep 26, 6:26 pm, Nelson Minar <nel...@monkey.org> wrote:
> I'm curious, who else is using D3 to make maps? If you are, what other geo /
> map libraries are you using?
>

> I've done a few maps with D3 and Javascript likehttp://windhistory.com/. It

[Nelson Minar]

Thanks for all the feedback. Particularly to Tom for all the suggestions! I'm heartened by all the OpenLayers fans: anyone have a simple D3 + OpenLayers example I can crib from?

On Tue, Sep 27, 2011 at 12:58 AM, CG <christop...@gmail.com> wrote:

It would be interesting to know what kind of (polymaps') drawbacks you
experienced while building windhistory.com.

Actually just one drawback; Polymaps is slow on iPads and iPhones. Mobile Safari on iOS doesn't seem to hardware accelerate the SVG transforms Polymaps uses for panning, so dragging the map is very laggy. It's an irritating oversight, I suspect it'd be a week-long project for someone at Apple to fix it.

I also have some concerns about using SVG on mobile devices in general. It's OK on iOS, just not fast. Android only started supporting it with Honeycomb. One appealing thing about Leaflet and Tile5 is they're explicitly designed to be fast on mobile devices.

[D3 Designer]

Regarding Android SVG performance, 5 months later, it appears to have taken a significant step forward with Chrome for Android.

On Tuesday, September 27, 2011 9:45:08 AM UTC-5, Nelson Minar wrote:

Thanks for all the feedback. Particularly to Tom for all the suggestions! I'm heartened by all the OpenLayers fans: anyone have a simple D3 + OpenLayers example I can crib from?

[Ziggy Jonsson]

I created an experimental 72 line D3 slippy map a while back using the
viewBox to zoom and pan: http://bl.ocks.org/1263239. The key idea
was that you would never have to "redraw" svg object upon pan or zoom
as lat/lng locations remain unchanged in SVG coordinates. Works fine
in a good browser but I had problem with printing (due to
approximation errors in floats both in Chrome and Firefox) and don't
use it for production. I'm now using leaflet for speed, both with an
SVG layer (only needs redraw on zoom) and canvas tiles.

[steven citron-pousty]

At foss4g-na there was also a nice discussion of http://jquerygeo.com/

Looked really nice.

Steve

[Mike Bostock]

Here's an example of using D3 with Leaflet:

http://bost.ocks.org/mike/leaflet/

Leaflet already has built-in support for GeoJSON, but you might find
this technique useful regardless.

Cheers,
Mike

[Ziggy Jonsson]

Great example Mike with good detail on custom layers.   I've used leaflet a great deal in practice and use a simple shortcut to avoid a custom layer (more practical). By  calling: map._initPathRoot() , or simply inserting a marker/circle etc, Leaflet provides a standard SVG layer that takes care of transforms, margins etc, making the code slightly simpler.

Here is how the same example would look with the shortcut  (http://bl.ocks.org/2957559)

{{{

var map = new L.Map("map")

    .setView(new L.LatLng(37.8, -96.9), 4)

    .addLayer(new L.TileLayer("http://{s}.tile.cloudmade.com/3eb45b95929d472d8fe4a2a5dafbd314/998/256/{z}/{x}/{y}.png"))

/* Initialize the SVG layer */

map._initPathRoot()     

/* We simply pick up the SVG from the map object */

var svg = d3.select("#map").select("svg"),

    g = svg.append("g");

/* Define the d3 projection */

var path = d3.geo.path().projection(function project(x) {

    var point = map.latLngToLayerPoint(new L.LatLng(x[1], x[0]));

    return [point.x, point.y];

  });

/* Load and project/redraw on zoom */

d3.json("us-states.json", function(collection) {

  var feature = g.selectAll("path")

      .data(collection.features)

    .enter().append("path")

    .attr("d", path);

  map.on("viewreset", function reset() {

    feature.attr("d",path)

  })

});

[Mike Bostock]

Thanks, Ziggy! That's nice to know.

One minor difference between the two is that when you drag using
Leaflet's built-in SVG viewport, it won't redraw the SVG until
mouseup. It appears to use 50% padding on each side. So, for example,
if you drag right more than 480px, then the SVG will be blank until
you mouseup. A small detail, but I was wondering how they got around
computing a bounding box from the geometry!

Mike

[Vladimir Agafonkin]

Hi guys,

The restricted viewport is used for performance reasons — Leaflet features are clipped by a bounding box around the screen (the padding is confugurable) and then simplified for the current zoom, which makes it pretty fast, especially on mobile devices. And it doesn't need to compute bounding box from the geometry — the projected coordinates are relative to a fixed point, and stay the same after panning — only SVG viewbox is updated.

Let me know if you have any questions about Leaflet — I'll be glad to help. :) Regarding the documentation for custom overlays — I'm going to fix this ASAP. And thanks for an awesome tutorial, Mike!

среда, 20 июня 2012 г., 4:55:53 UTC+3 пользователь Mike Bostock написал:

[Mike Bostock]

> Regarding the documentation for custom overlays — I'm going to fix this

Hooray! Thanks for the additional info, too.

Mike

[whereverimayroam]

This is a long post if you don't have the time, skip it, however if you have a few minutes to spare I would REALLY appreciate the help of people who have been there and know better.

I am a web developer who is relatively new to maps programming (I feel that applies x2 after reading all your posts). I am also a motorcycle fan. I building a site to curate my favorite rides. I wanted to skip building a completely custom site in favor of trying to focus on content so I found a wordpress template that pretty closely suits my needs

This is the template:
http://elegantthemes.com/preview/Explorable/

The theme has google maps built in. The home page lets you drop markers on the page (through the wordpress dashboard) and when you click a marker it brings you to the details page for that marker. I will be using those markers to show the places I have traveled to but once you click the marker I want the map to display my route.

I have saved maps of my rides in google's "My Maps". I create the maps by making a directions request and dragging way points to change the directions from the straightest, fastest route that google draws by default to the twistiest, long way round route that I frequently take. My Maps has an export KML feature so technically I could simply link that KML to the Google maps API and display the route that way (and have done so)    ...however there is a hitch.

When I draw the map for a 6 day motorcycle trip goggle only allows 8 waypoints per directions request search (I can use that many or more to draw a single day of riding). So to draw 6 days of riding I have to create 6 different maps. When I display the route for the complete trip however I want to display all 6 days on 1 map. My initial solution was to create a KML Layer for each day and display them all that way however Google also limits the layers possible and currently only displays 1 of the KML Layers I have added (this is probably becase of something I am overlooking). What I decided to do next was extract all the polyline coordinates out of the KML for the day route and chain them all together to draw a single polyline of all the days. (Still working on parsing the network link KML to get the coordinates programmatically instead of the copy paste tests I have been doing but I have had success in displaying all days on 1 map).

Then I found this:
http://nyctaxi.herokuapp.com/

(and these supporting "how I did this" pages)
http://chriswhong.com/data-visualization/taxitechblog1/
http://chriswhong.com/open-data/taxi-techblog-2-leaflet-d3-and-other-frontend-fun/

Ok, so that's cool. Instead of simply just showing the route I could "draw" the route as if it was being ridden right now. On top of that, as you read through my trip log and scroll the text the marker could advance along the route to geolocate the part of the trip you are currently reading about.  Basically the possibilities are really opening up for me but I am becoming concerned that the Google maps API is going to hit a wall before my ideas do.

I really like google maps because of the accuracy of the roads shown. The roads are far more important to me than the look and feel of the map itself. Also, because I have created over 40 maps already of my rides in google I would like to stick with their API (if possible). I don't want to have to start over re plotting my rides. Google's direction services TOS states that I cannot use their directions services responses to drive alternate mapping systems.

Can I even achieve the SVG path and animated marker with Google maps or is it an impossible dream. Can I use the Leaflet/D3 route described in the "how to" article above or am I going to require different frameworks and processes? How would you do this if YOU had to tackle the task?

Any thoughts or advice on how I can get from where I am now to my goal would be GREATLY appreciated. Even if you have nothing to add I would still like to thank you anyway as you are a champ for sticking with me this far.

[Dow Street]

You might get some ideas from this NYT story:

http://www.nytimes.com/newsgraphics/2013/10/13/russia/

--
You received this message because you are subscribed to the Google Groups "d3-js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to d3-js+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.