Transforming flat data into treemap hierarchical structure

1,292 views
Skip to first unread message

Kyle Foreman

unread,
Sep 6, 2011, 10:58:18 AM9/6/11
to d3...@googlegroups.com
Hello all

I'm obtaining data from a MySQL server that I'd like to turn into a treemap. It's only got data for children, like so:

row        name    value     parent
1          A1      .5        A
2          A2      .3        A
3          B1.1    .7        B1
4          B1.2    .1        B1
5          B2      .9        B

So I need to turn that into something like this:

{
 "name": "data",
 "children": [
  {
   "name": "A",
   "children": [
    {"name": "A1", "size": .5},
    {"name": "A2", "size": .3}
   ]
  }
  {
   "name": "B",
   "children": [
    {
"name": "B1", 
"children": [
 {"name": "B1.1", "size": .7},
 {"name": "B1.2", "size": .1}
]
}
    {"name": "B2", "size": .9}
   ]
  }
 ]
}

Can you suggest a way to do this in javascript without having to change how the data is stored? (the current table structure is perfect for other, non-hierarchical visualizations we're using)

It looks like some of the d3.layouts are close but require different inputs than I've got.

Thanks so much for any suggestions!
Kyle

Kyle Foreman

unread,
Sep 6, 2011, 11:11:01 AM9/6/11
to d3...@googlegroups.com
The tree structure does not change, it's just the values of the leafs that change depending on other selections (country, year, etc). So I could have the tree structure in place and just need to fill in the node values. i.e. I could have the structure below in place, then I'd need a way of traversing the tree to insert the values. So I guess that's sort of like the inverse of the treemap.values() function?

{
 "name": "data",
 "children": [
  {
   "name": "A",
   "children": [
    {"name": "A1"},
    {"name": "A2"}
   ]
  }
  {
   "name": "B",
   "children": [
    {
 "name": "B1", 
 "children": [
  {"name": "B1.1"},
  {"name": "B1.2"}
 ]
}
    {"name": "B2"}
   ]
  }
 ]
}


Kyle Foreman

unread,
Sep 7, 2011, 1:13:34 AM9/7/11
to d3...@googlegroups.com
I couldn't figure out a way to do easily in js, so I just wrote some Stata code to precompile the json file for now. That way I can at least get on with debugging the visualization for now. But if you've got any suggestions for creating the tree at runtime I'd like to hear them!

Mike Bostock

unread,
Sep 7, 2011, 1:57:59 AM9/7/11
to d3...@googlegroups.com
It'd probably help to make your names imply the hierarchy more
clearly. In the example data you gave, how do you know that "B1" is a
child of "B"? Also, do you only have rows for leaf nodes, or do you
have rows for internal nodes as well?

Assuming you have only leaf nodes, I'd use a structure like this:

var leaves = [
{name: "A.1", value: .5},
{name: "A.2", value: .3},
{name: "B.1.1", value: .7},
{name: "B.1.2", value: .1},
{name: "B.2", value: .9}
];

Next you can write a memoizing function that returns the node for a
given name. If that node was previously seen, it returns the existing
node rather than creating a new one. Furthermore, it also memoizes all
parent nodes recursively. That looks like this:

var root = {children: []};

function memoize(node) {
var i = node.name.lastIndexOf("."),
p = i < 0 ? root : memoize({name: node.name.substring(0, i),
children: []}),
n = p.children.length;
for (i = -1; ++i < n;) {
if (p.children[i].name === node.name) {
return p.children[i];
}
}
p.children.push(node);
return node;
}

leaves.forEach(memoize);

Mike

Kyle Foreman

unread,
Sep 7, 2011, 8:35:47 PM9/7/11
to d3...@googlegroups.com
Thanks Mike, that's really clean! I will implement that shortly.

In the meantime, I'm having some issues using the data I precompiled. I'm pretty sure it matches the format from the API wiki (https://github.com/mbostock/d3/wiki/Treemap-Layout#wiki-children) exactly, but I must be losing something in translating from the examples. All the examples I've seen use d3.json() to add in the data instead and have a different data format from the API wiki (the flare.json format, which doesn't have arrays of children, but rather has them as properties(?) of the parent). 

Here's what I've gotten together: http://bl.ocks.org/1202253 . No errors are being thrown, but it does not seem to be traversing the tree and appending svg:g elements at all. Any ideas?

Thanks again,
Kyle


Mike Bostock

unread,
Sep 7, 2011, 8:45:05 PM9/7/11
to d3...@googlegroups.com
> All the examples … have a different data format from the API wiki (the

> flare.json format, which doesn't have arrays of children, but rather has
> them as properties(?) of the parent).

That was the case in the long-long ago, but not currently. I updated
flare.json to use the standard representation so that no tricky value
accessor was needed:

https://github.com/mbostock/d3/blob/master/examples/data/flare.json

Related commit here:

https://github.com/mbostock/d3/commit/6bdbe4b8634f56276fa37fa418f7d7d5e629248d

Mike

Mike Bostock

unread,
Sep 7, 2011, 8:47:47 PM9/7/11
to d3...@googlegroups.com
http://bl.ocks.org/1202253

You're passing a map (USA) rather than an array to the data operator
(vis.data). You don't need to call the data operator twice. It'd be
easier if you just passed USA (the root node of your hierarchy) to the
treemap.nodes method, as documented. For example:

var cell = vis.selectAll("g")
.data(treemap.nodes(USA))
.enter().append("svg:g")

Mike

Kyle Foreman

unread,
Sep 8, 2011, 12:00:21 AM9/8/11
to d3...@googlegroups.com
Ah, thanks Mike! I had tried treemap().nodes(USA).... oops! And thanks for pointing out I'm apparently a bit behind on the version I'm using - that's what I get for storing multiple versions on different servers, I suppose :)

Kyle Foreman

unread,
Sep 8, 2011, 12:07:34 AM9/8/11
to d3...@googlegroups.com
Hmm, I'm now getting an error that "TypeError: Object function l(b){var f=e||a(b),j=f[0];j.x=0,j.y=0,j.dx=c[0],j.dy=c[1],e&&a.revalue(j),g(j,c[0]*c[1]/j.value),(e?i:h)(j),d&&(e=f);return f} has no method 'nodes'" when switching to the code that you posted. I'll have to take a look in the morning with fresher eyes!
Best
Kyle

Kyle Foreman

unread,
Sep 8, 2011, 9:33:31 AM9/8/11
to d3...@googlegroups.com
Out of date library strikes again! Alright, now I definitely have the impetus to cleanup my servers.... 

Anyhow, it's working great now after updating d3 - thanks!
Reply all
Reply to author
Forward
0 new messages