general questions on d3

[dizzy]

Hi,
I have been looking and trying various libraries over the past month
as a charting solution. I came across d3 today and from reading, I
feel like it might be a good solution...in particular based on how it
seems to delegate to functions for data retrieval. I have some
questions i hope someone will be able to answer.

1. From looking at how series and tick axis data are referenced it
seems like i would he able to keep my data as rich json models and
simply refer to properties of the model objects to feed the d3 drawing
functions. Is that correct?

2. I have unique requirements for the x axis labels. Currently the
only solution with other charting packages has been to generate these
labels on the server side and populate a property of my json object
with them or to iterate through all the data on the client side and
generate an array of labels before actually plotting the data. With
d3 however it looks like i could instead assign a function call and
pass it the the data point being plotted and that function can return
the label to draw. Is this correct?

3. One of the advantages of using html5 canvas is speed. The
disadvantage of canvas is image quality esp when zooming. I've found
that performance can be a problem for svg based chart libraries when
plotting large sets of data (i.e. 20+ yrs of data). I have read that
d3 can handle large sets of data efficiently. Is this correct and how
is it different than other libs in this regard?

4. Are there any examples of candlestick plot(i know protovis has an
example but im guessing d3 is significantly diff?)

Thanks much...im really excited to try this based on what i have
read. Will give it a shot tonight.

[dizzy]

After some testing/prototyping with d3-js, i think i was able to get
the answers to most of these questions...

for #1..Yes

for #3..In terms of time taken to render, despite making use of a lot
of accessors in my test code, i was surprised to observe that i was
able to plot about 15,000 datapoints (about 60 years of daily data) in
about 4-5 seconds or so (without axis ticks and labels however).
similar code i have that does the same thing using the html canvas to
draw takes about 12 to 14 seconds (granted that code base is more
'complete'..meaning it renders more stuff such as the axis ticks and
labels).

for #4..i couldn't find a candle stick example, but i was able to do
this by drawing a line for the wick and then using a rectangle to draw
the body on top of the line. I'm not sure if this is the most
efficient way, but it does accomplish the job. (if it isn't the most
efficient way, then that would be good news as it would mean there is
room for optimization).

that leaves #2...which i haven't tried yet.

I'm really liking this library thus far. it does require more lower
level coding than other libraries, but i like that level of control.
(With other libraries, i've invariably had to either extend core
objects, change the code in the core libraries or branch to accomplish
what i needed. I don't think i will have to do that with this
library...in particular due to the fact that so many of the api
functions accept accessors/delegates).

[Mike Bostock]

Hi Dizzy,

Thanks for the feedback, and glad to hear you're making progress. Your
approach to the candlestick chart sounds reasonable. I think the
answer to #2 is also "yes", but I'm not sure I understand the
question. Like Protovis, D3 uses accessor functions for nearly
everything, which means you can conform your visualization
specification to fit the shape of your data, rather than the other way
around.

Regarding performance of SVG & Canvas: Yep! SVG is fast, too; for
certain tasks such as incremental updates to a complex visualization,
SVG can be faster than Canvas because the browser can be smart about
what needs to redraw. Other tasks, such as caching bitmaps and
pixel-based manipulation, may give Canvas the edge. You can also use
CSS3 transforms with SVG to improve performance. For example, try
rotating this phylogenetic tree:

http://jasondavies.com/tree-of-life

Mike

[Jason Davies]

On Wed, Jun 22, 2011 at 11:29:44AM -0700, dizzy wrote:
> 1. From looking at how series and tick axis data are referenced it
> seems like i would he able to keep my data as rich json models and
> simply refer to properties of the model objects to feed the d3 drawing
> functions. Is that correct?

Yes, for most things D3 allows you to specify an accessor function to
retrieve an arbitrary property (or perform some transformation) for each
datum.

> 2. I have unique requirements for the x axis labels. Currently the
> only solution with other charting packages has been to generate these
> labels on the server side and populate a property of my json object
> with them or to iterate through all the data on the client side and
> generate an array of labels before actually plotting the data. With
> d3 however it looks like i could instead assign a function call and
> pass it the the data point being plotted and that function can return
> the label to draw. Is this correct?

D3 is a completely generic data transformation library so you can decide
on your own way of generating axis labels. Usually each axis has an
associated scale e.g. d3.scale.linear(), and you can call .ticks() to
retrieve an array of tick values appropriate for using for axis labels.

If you have your own peculiar way of generating labels, it should be
easy enough to do: simply bind your data to a group of elements and then
use an accessor function to generate the label for each element.

In summary, you shouldn't need to do any server-side generation of
labels, and they can be generated on-the-fly in the browser.

> 3. One of the advantages of using html5 canvas is speed. The
> disadvantage of canvas is image quality esp when zooming. I've found
> that performance can be a problem for svg based chart libraries when
> plotting large sets of data (i.e. 20+ yrs of data). I have read that
> d3 can handle large sets of data efficiently. Is this correct and how
> is it different than other libs in this regard?

There are indeed differences between canvas and SVG performance-wise.
Typically SVG is better for interactive visualisations because it uses
the browser's built-in scene graph for event handling and animation.

See also Mike's message here which covers some advantages/disadvantages
of pixel-based vs. SVG:
<https://groups.google.com/d/msg/d3-js/KjGW94SyrAg/lc6rDxyisPAJ>

Note that with SVG you can scale your visualisation to arbitrary
resolutions, whereas this is harder to do with canvas.

> 4. Are there any examples of candlestick plot(i know protovis has an
> example but im guessing d3 is significantly diff?)

Not yet, but I'm working on a reusable candlestick chart so one should
appear at some point. You've probably already seen the box plot
example: <http://mbostock.github.com/d3/ex/box.html>.

> Thanks much...im really excited to try this based on what i have
> read. Will give it a shot tonight.

Good stuff, do let us know if you have any questions!
--
Jason Davies, http://www.jasondavies.com/

[dizzy]

Thanks Mike and Jason. The more I get familiar with this library the
more im falling in love with it. What im most excited about is the
ability to inspect and manipulate each plot point or set and
manipulate if necessary....that is so powerful and flexible as it
means you can operate on any type of model or data structure without
having to pre-process to conform to an API expected format and it also
means you don't have to pre-process to apply certain data dependent
rendering rules and logic, it can all be done in one pass when
binding and building the dom..so sweet!
Its also a huge plus to see you guys so active in these forums.

can't wait for the candle stick chart component, though what I have
now is working nicely...and looks great when plotting lots of data
points and zooming {+1 for vector graphics!}