Interactive Heatmap Visualization

[des]

Hi all,

I was wondering what are some of the design considerations that are
required when developing a visualization using d3.js that is similar
to the following: http://online.wsj.com/article/SB125993225142676615.html#articleTabs%3Dinteractive

Anybody have any experiences and are willing to share?

[Lars Kotthoff]

> Anybody have any experiences and are willing to share?

It's relatively straightforward. You create a group element for each row and
append the rectangles to that. Code would be something like

// bins is a 2D matrix with the values
// size is the size of each cell
// colour is a function that returns the colour for each value (e.g. a d3.scale)
var hm = svg.append("svg:g"),
hmrows = hm.selectAll("g")
.data(bins)
.enter().append("svg:g")
.attr("transform", function(d, i) {
return "translate(0," + (size * i) + ")";
});
hmrows.selectAll("rect")
.data(function(d) { return d; })
.enter().append("svg:rect")
.attr("transform", function(d, i) {
return "translate(" + (size * i) + ",0)";
})
.attr("width", size)
.attr("height", size)
.attr("fill", function(d) { return colour(d.length); });

There are a couple of pitfalls if you want to make it look fancy. You can change
the stroke width of a cell on hover, but because of the way the cells are drawn,
part of it will overlap with other cells and you will only really see the change
on cells that have been drawn later. I solved this by adding a second grid on
top with no fill (so the heatmap can be seen), but with all the handlers.

The other problem is identifying row and column number of a cell with this
model. The column you can get from the cell itself, but the row is determined by
the parent element. I solved this by adding a property to the generated SVG row
group

.property("row", function(d, i) { return i; })

and getting that in the event handler

var row = parseInt(d3.select(d3.event.target.parentNode).property("row"));

There's probably a more elegant way to do this. With this it should be
relatively straightforward to make something like in the link you posted. Feel
free to hassle me if you get stuck.

Lars

[Kai Chang]

You may want to check out DVL, Vadim's layer on top of d3.js that has many nice features for heatmaps and managing events:

http://vadim.ogievetsky.com/projects/dvl/examples/heatmap/mx_heatmap.html

https://github.com/vadimon/DVL

I've not used it, but the heatmap examples are very functional. If you pull the latest version, you'll see there are some nice sorting operations as well.

[des]

Hi Lars,

I have managed to come out with a heatmap based on this:
https://groups.google.com/group/d3-js/browse_thread/thread/186e67f8991798c6

Similar logic as you have explained but you might want to look at the
implementation (It has a more elegant way to retrieve the row/col)

However, I encountered a problem that will need your help on. I have a
2d array with a set of numerical figures which I want to individually
access and interpolate each of them to a numerical figure that
reflect the color the cell. The bigger the number, the greater the
intensity whereas the lower the number, the lesser the intensity for
the color of the cell.

var numPoints = 16,
size = 300,
numRows = 4,
numCols = 4,

cells = null,
color = d3.interpolateRgb("#fff", "#c09");

var data =
[
[ 55, 44, 147, 161 ],
[ 182, 174, 109, 20 ],
[ 39, 84, 6, 12 ],
[ 137, 88, 65, 101 ]
];

......
// I am able to retrieve the individual numerical figure based on the
row/col eg:d.points[d.row][d.col]
.attr("fill", function(d, i) {
// return color((d.points.length - min) / (max - min));
//alert(interpolateBtw2Num(d.points[d.row][d.col]));
return color((d.points.length));

How should I do the interpolation based on the associated figure?

> top with no fill (so theheatmapcan be seen), but with all the handlers.

[Lars Kotthoff]

> However, I encountered a problem that will need your help on. I have a
> 2d array with a set of numerical figures which I want to individually
> access and interpolate each of them to a numerical figure that
> reflect the color the cell. The bigger the number, the greater the
> intensity whereas the lower the number, the lesser the intensity for
> the color of the cell.

I'm not sure if I'm understanding what you're saying correctly -- do you want to
change the intensity for the colour of a cell (which may be different for each
cell and is determined by some other property)? If so, you could create a new
color scale in the function for fill that ranges from white to the color that
the other scale gives you, e.g.

var color = d3.interpolateRgb("#fff", "#c09");
...

.attr("fill", function(d, i) {

var icolor = d3.interpolateRgb("white", color(d.points.length));
return icolor(d.otherProperty);
}

You could also try using the HSL colour space instead of RGB and change the
saturation based on the figure.

Lars

[des]

Thanks for the help, Lars. This is what I have managed to come out
with so far:

var
canvasSize=600,
numRows = 9,
numCols,
x,y,ticks,_ref,pb, pl, pr, pt,
cells = null,

color = d3.interpolateRgb("#fff", "#c09");

count=0;
_ref = [20, 0, 20, 20], pt = _ref[0], pl = _ref[1], pr = _ref[2], pb =
_ref[3],
size = 500;


var wardsArray = [];
var tpArray = ["A","B","C","D","E","F","G","H","I"];
var scaleYAxis=[105,160,215,270,325,380,435,490,545];
var
scaleXAxis=[105,138,171,204,237,270,303,336,369,402,435,468,501,534];



var resetCellsToData = function(dataArr) {

var resetCells = [];
for (var rowNum = 0; rowNum < numRows; rowNum++) {
resetCells.push([]);
var row = resetCells[resetCells.length - 1];
for (var colNum = 0; colNum < numCols; colNum++) {
row.push({
row: rowNum,
col: colNum,
density: 0,
points: dataArr
});
}
}

return resetCells;
};

var clearCells = function() {
for (var rowNum = 0; rowNum < numRows; rowNum++) {
for (var colNum = 0; colNum < numCols; colNum++) {
cells[rowNum][colNum].density = 0;
cells[rowNum][colNum].points = [];
}
}
};

var randomizeData = function(data) {
//alert(data);
cells = resetCellsToData(data);
};

var selectCell = function(cell,data) {
d3.select(cell).attr("stroke", "#f00").attr("stroke-width", 3);
cell.parentNode.parentNode.appendChild(cell.parentNode);
cell.parentNode.appendChild(cell);
};

var deselectCell = function(cell) {
d3.select(cell).attr("stroke", "#fff").attr("stroke-width", 1);
};

var onCellOver = function(cell, data) {
selectCell(cell,data);
};

var onCellOut = function(cell, data) {
deselectCell(cell);
};

var onCellOverHighlightAxis = function(xAxisPos, yAxisPos) {
//alert('uID'+[xAxisPos]);
d3.select('#textForaxis'+xAxisPos).attr("stroke",
"#000").attr("stroke-width", 0.2);
d3.select('#textForYaxis'+yAxisPos).attr("stroke",
"#000").attr("stroke-width", 0.2);
};

var onCellOutHighlightAxis = function(xAxisPos, yAxisPos) {
//alert('uID'+[xAxisPos]);
d3.select('#textForaxis'+xAxisPos).attr("stroke",
"#000").attr("stroke-width", 0);
d3.select('#textForYaxis'+yAxisPos).attr("stroke",
"#000").attr("stroke-width", 0);
};

var createHeatchart = function() {
var min = 999;
var max = -999;
var l;
var totalSum=0;

for (var rowNum = 0; rowNum < cells.length; rowNum++) {
for (var colNum = 0; colNum < numCols; colNum++) {
l = cells[rowNum][colNum].points.length;
//check to ensure that the same number of cols exists
if(cells[rowNum][colNum].points[rowNum][colNum]==null){
cells[rowNum][colNum].points[rowNum][colNum]=0;
}
totalSum+=(cells[rowNum][colNum].points[rowNum][colNum]);
if (l > max) {
max = l;
}
if (l < min) {
min = l;
}
}
}

var heatchart =
d3.select("div#heatchart").append("svg:svg").attr("width",
canvasSize).attr("height", canvasSize);
x = d3.scale.linear().domain([100, size]).range([0, size]);
y = d3.scale.linear().domain([100, size]).range([size, 0]);


bg =
heatchart.selectAll("g").data(cells).enter().append("svg:g").selectAll("rect").data(function(d)
{
return d;
}).enter().append("svg:rect").attr("x", function(d, i) {
// 0 * (300/4)
return d.col * (size / numCols);
}).attr("y", function(d, i) {
return d.row * (size / numRows);
}).attr("width", size / numCols).attr("height", size /
numRows).attr('transform', "translate(100,15)").attr("fill",
function(d, i) {
return color(((d.points[d.row][d.col]) /
(totalSum))*(2*d.points.length));
}).attr("stroke", "#fff").attr("cell", function(d) {
return "r" + d.row + "c" + d.col;
}).on("mouseover", function(d) {
onCellOverHighlightAxis(d.row,d.col);
onCellOver(this, d.points[d.row][d.col]);
}).on("mouseout", function(d) {
onCellOut(this, d);
onCellOutHighlightAxis(d.row,d.col);
}).on('click', function(d, i) {
return console.log(d.points[d.row][d.col],i);
})
.append('svg:title').attr("id", "tooltiptext").text(function(d,i)
{
return "PS: " + d.points[d.row][d.col] + "%";
});

// Add tick groups
ticks =
heatchart.selectAll('.ticy').data(y.ticks(cells.length)).enter().append('svg:g').attr('transform',
function(d,i) {
return "translate(100, " + ((scaleYAxis[i])) + ")" ;
}).attr('class', 'ticy');
// Add y axis tick labels

ticks.append('svg:text').text(function(d,i) {
return tpArray[i];
}).attr('text-anchor', 'end').attr('dy', -60).attr('dx',
-4).attr('id',function(d,i) {
return 'textForaxis'+i
});

//Add tick groups
ticks =
heatchart.selectAll('.ticx').data(scaleXAxis).enter().append('svg:g').attr('transform',
function(d, i) {
//alert(x.ticks(data[0].length));
return "translate("+ ((scaleXAxis[i])) + ", -30)";
}).attr('class', 'ticx');
//Add x axis tick labels
ticks.append('svg:text').attr('id','textForaxis').text(function(d,
i) {
return (wardsArray[i]);
}).attr('dy', 40).attr('dx', 5).attr('id',function(d,i) {
return 'textForYaxis'+i
});


};

var updateHeatchart = function() {

var min = 999;
var max = -999;
var l;
var totalSum=0;

for (var rowNum = 0; rowNum < cells.length; rowNum++) {
for (var colNum = 0; colNum < numCols; colNum++) {
l = cells[rowNum][colNum].points.length;
//check to ensure that the same number of cols exists
if(cells[rowNum][colNum].points[rowNum][colNum]==null){
cells[rowNum][colNum].points[rowNum][colNum]=0;
}
totalSum+=(cells[rowNum][colNum].points[rowNum][colNum]);

if (l > max) {
max = l;
}
if (l < min) {
min = l;
}
}
}

//reset the tooltip text for all
var tooltipText = d3.selectAll("title")
tooltipText.remove();
//reset the x-axis to the correct number
var ticksX = d3.selectAll('.ticx').data(wardsArray)
ticksX.exit().remove();

/*

d3.select("div#heatchart").select("svg").selectAll("g").data(cells).selectAll("rect").data(function(d,i)
{
return d;
}).attr("fill", function(d, i) {
return color(((d.points[d.row][d.col]) /
(totalSum))*(2*d.points.length));
}).attr("cell", function(d) {
return "r" + d.row + "c" + d.col;
}).on("mouseover", function(d) {
onCellOverHighlightAxis(d.row,d.col);
onCellOver(this, d.points[d.row][d.col]);
}).on("mouseout", function(d) {
onCellOut(this, d);
onCellOutHighlightAxis(d.row,d.col);
}).on('click', function(d, i) {
return console.log(d.points[d.row][d.col],i);
}).append('svg:title').attr("id",
"tooltiptext").text(function(d,i) {
return "PS: " + d.points[d.row][d.col] + "%";
});
*/


d3.select("div#heatchart").select("svg").selectAll("g").data(cells).selectAll("rect").data(function(d,i)
{
return d;
})

.attr("fill", function(d, i) {

return color(((d.points[d.row][d.col]) /
(totalSum))*(2*d.points.length));
}).attr("cell", function(d) {
return "r" + d.row + "c" + d.col;
}).append('svg:title').attr("id",
"tooltiptext").text(function(d,i) {
return "PS: " + d.points[d.row][d.col] + "%";
})
.on("mouseover", function(d) {
onCellOverHighlightAxis(d.row,d.col);
onCellOver(this, d.points[d.row][d.col]);
}).on("mouseout", function(d) {
onCellOut(this, d);
onCellOutHighlightAxis(d.row,d.col);
}).on('click', function(d, i) {
return console.log(d.points[d.row][d.col],i);
});


}

var onDataChangeClick = function() {

var data = [
[10, 10, 0, 7, 33, 8, 16, 20, 50, 0, 0, 22, 0],
[10, 0, 10, 0, 14, 15, 15, 29, 0, 0, 0, 13, 0],
[100, 50, 0, 0, 13, 13, 3, 0, 0, 0, 0, 27, 0],
[25, 0, 29, 23, 0, 28, 0, 0, 36, 38, 40, 100],
[10, 0, 33, 10, 0, 8, 10, 13, 0, 100, 50, 0, 0],
[18, 0, 0, 22, 0, 8, 0, 0, 0, 0, 100, 13],
[10, 0, 0, 0, 20, 0, 5, 0, 0, 0, 0, 0, 22],
[10, 0, 0, 11, 29, 36, 39, 14, 50, 0, 25, 29, 11],
[10, 0, 0, 8, 13, 5, 0, 0, 0, 0, 0, 0, 0]];

var dataDept2 =
[
{
"A052": 0
},{
"A061": 0
},{
"A071": 0
},{
"B046": 0
},{
"B055": 0
},{
"B056": 0
},{
"B065": 0
},{
"B066": 0
},{
"B075": 0
},{
"B076": 0
},{
"B085": 0
},{
"B086": 0
},{
"B095": 0
}
];
//reset the array
wardsArray=[];
//create the ward array
for (i=0; i<dataDept2.length; i++) {
wardsArray[i]=d3.keys(dataDept2[i]);
}

//assign the numCols for the heatmap vis
numCols = dataDept2.length;

randomizeData(data);
updateHeatchart();
};

var init = function() {
var data = [
[0, 0, 0, 7, 33, 8, 16, 20, 50, 0, 0, 22, 0, 8],
[0, 0, 0, 0, 14, 15, 15, 29, 0, 0, 0, 13, 0, 14],
[0, 50, 0, 0, 13, 13, 3, 0, 0, 0, 0, 27, 0, 21],
[25, 0, 29, 23, 0, 28, 0, 0, 36, 38, 40, 100],
[0, 0, 33, 10, 0, 8, 10, 13, 0, 100, 50, 0, 0, 0],
[18, 0, 0, 22, 0, 8, 0, 0, 0, 0, 100, 13],
[0, 0, 0, 0, 20, 0, 5, 0, 0, 0, 0, 0, 22, 50],
[0, 0, 0, 11, 29, 36, 39, 14, 50, 0, 25, 29, 11, 8],
[0, 0, 0, 8, 13, 5, 0, 0, 0, 0, 0, 0, 0, 30]];

var dataDept =
[
{
"A052": 0
},{
"A061": 0
},{
"A071": 0
},{
"B046": 0
},{
"B055": 0
},{
"B056": 0
},{
"B065": 0
},{
"B066": 0
},{
"B075": 0
},{
"B076": 0
},{
"B085": 0
},{
"B086": 0
},{
"B095": 0
},{
"B096": 0
}
];

//create the ward array
for (i=0; i<dataDept.length; i++) {
wardsArray[i]=d3.keys(dataDept[i]);
}
//assign the numCols for the heatmap vis
numCols = dataDept.length;



//alert(data);
randomizeData(data);
createHeatchart();
};

init();


As above, I have managed to come with a heatmap with the ticks
changing dynamically based on the data that is passed in. For
instance, if the 2d array is 9x14 (row,col) the x ticks will reflect
14 times.. If it is a 9x13, it will reflect 13 times, based on the
columns.

However, one problem I faced is that the heatchart is not dynamically
transiting to reflect the correct column number. For instance, if the
2d array passed in is 9x18, the heatchat should transiting to reflect
18 columns when the updateHeatchart() method is called. If the 2d
array is 9x9, then the heatchart will resize accordingly to 9 columns.

How should I go above doing it? I have tweaked the codes several times
and have not achieve any success yet.

Thanks for the help.

[Abhinna Agrawal]

hey,

i was wondering if there was a way to scale this technique to something like 17K x 1K matrix using D3.js . I am unable to render such a large number of data points.

[Kai Chang]

That's a huge image to render. Almost no monitors have 17k horizontal pixels (do any?).

What I would do is simply send the data as an image, and render that image in a <canvas> tag. You'll also need to send data about the x/y/color scales.

Use the color of the pixel under the cursor to determine the approximate data value. Use the .invert() method on the x/y scales to determine the pixel data's value in those two dimensions as well.

--
You received this message because you are subscribed to the Google Groups "d3-js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to d3-js+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.