Parallel coordinates, a reusable chart

[Kai Chang]

http://syntagmatic.github.com/parallel-coordinates/

Repository and API Reference:
https://github.com/syntagmatic/parallel-coordinates

I've spent ~10 months hacking together various versions of parallel
coordinates. This is an initial attempt to refactor that work into an
interface based on the reusable charts proposal.

Following nvd3 and parsets, d3.js is the only dependency.

I'll announce major updates to v0.x in this email thread.

[decio]

That's so nice, I will definitely try it out!
Few months ago I was playing around with a Box Chart implementation based on the reusable charts proposal, but abandoned it for lack of time, but your nice work just inspired me to give it a second chance!
All the best,

Decio

[marc fawzi]

That's extremely generous of you, Kai.

Did you abstract the Canvas layer by any chance? I remember you mentioning something about it, and I was wondering if you are anywhere near arriving at a generalized abstraction for canvas emulation on top of D3....?

[Chris Rich]

Awesome!  I'm going to try it out right now!

[Kai Chang]

Marc, I'm not exactly working on a generalized abstraction for canvas,
though I am still exploring canvas to squeeze better performance out
of parallel coordinates.

Previously I've tried rendering lines through a queue, which works
well to scale to arbitrarily large datasets (though it's not *faster*
at rendering):

http://bl.ocks.org/d/3341641/

There are some other techniques I may try this time around, such as
rendering to an off-screen canvas, stroking groups of lines rather
than individually, and using multiple canvases - either one per
dimension pair, or stacked on top of each other for different groups
of data. These will potentially be faster (slightly) in certain
scenarios, while rendering immediately without a queue.

[Chris Rich]

Kai,

  I'm just trying it right now on the dataset I would like to use it on.  However, I had never really looked at the data before and didn't realize that it had 189 dimensions =P.  So, I'm trying to use a subset of the data via the .dimensions() method, but the axis still retain their position on the page based on where they are in the TOTAL number of dimensions and not the subset.  Is there an easy way to correct this that I am missing?  I'm going to look through the file right now, but if there is a quick solution that would be great.  Thanks again!

Chris

[Chris Rich]

Whoops!  I think I found it.  You expose the xscale through the methods.

Chris

[Kai Chang]

For a quick fix, try:

myChart
.autoscale()
.createAxes()

That should reset the xscales, remove the old axes, and put new axes in.

What's probably going on is .render() calls .detectDimensions() if
they haven't been set already. I'd be curious to look at your code,
just to see how you initially hit the problem and/or tried to solve
it.

[Chris Rich]

Hey Kai,

  Yup I had just updated .dimensions from the console and called 

myChart

  .render()

  .createAxes()

  Calling autoscale prior to the rendering fixes it.  Back to playing around with the code!

Chris

[Kai Chang]

Thanks Chris for the feedback. Some methods automatically cascade
changes though, and others don't. I need to make this flow a bit more
rational, and then document it more precisely.

[marc fawzi]

I'm putting 2-3 thousand tiny circles on a map at any given second, while running other D3 animations, so SVG for the circles is out of the question... 

[Chris Rich]

That would be my question too.  As my data has WAY too many dimensions, I was thinking to breaking it up into 2 linked charts.  One on top of the other.

Also, I was trying to mimic the behaviour of having the non-selected ones in the background greyed out.  However, I'm having a bit of trouble.  I thought it would be as easy as changing one of the stroke properties of the canvas.background element.  But it didn't change anything.  Is there a simple way to do this?

Thanks again!

On Saturday, September 8, 2012 3:20:53 PM UTC-7, Will Turman wrote:

This is great! 

Can you show how you link the parallel coordinates chart to a table and vice versa? 

On Thursday, September 6, 2012 5:52:41 AM UTC-7, Kai Chang wrote:

[Kai Chang]

To link to a data table, listen to the "brush" event. The first
argument to the listener is the brushed data which you can use to
update the table:

https://github.com/syntagmatic/parallel-coordinates#parcoords_on

I haven't posted an table-linking example yet because I'm somewhat
conflicted on what to use for the table. I started rolling my own, but
for most applications SlickGrid is the way to go. I started another
thread on this topic.

Update on grey-out shadows soon.

There is a breaking change in 0.1.2. reorderAxes() has been renamed
updateAxes(). It handles transitions for reordering, as well as
adding/removing dimensions.

There's also a new "interactive" mode to make console use more
pleasant. Currently this only affects dimensions(), but I'll flesh
this out more in the future:

https://github.com/syntagmatic/parallel-coordinates#parcoords_interactive

[Kai Chang]

v0.1.3 adds shadows(), which creates the greyed-out background.

There's a new example of coloring by Z-score (Vadim Ogievetsky's
suggestion). Useful for getting a sense of the distribution of the
data, and tracking outliers from a particular dimension. I encourage
you to explore other statistical colorings, if you're so inclined.

http://syntagmatic.github.com/parallel-coordinates/

[Chris Rich]

Hey Kai,

  Due to my number of dimensions (101) and rows (1706), my animations were getting sluggish with the canvas trying to redraw on the brush event.  So, at the bottom of my local copy of parallel-coordinates.js, I added your renderQueue component.

I then changed the pc.render definition to:

rqueue = renderQueue(path_foreground)

        .rate(__.rate)

        .clear(function(){pc.clear('foreground')});

  pc.render = function() {

    // try to autodetect dimensions and create scales

    if (!__.dimensions.length) pc.detectDimensions();

    if (!(__.dimensions[0] in yscale)) pc.autoscale();

    if (brushed) {

        rqueue(brushed.map(function(d){return d}));

    } else {

        rqueue(__.data.map(function(d){return d}));

    }

    events.render.call(this);

    return this;

  };

and added rate = 10 to the __ definition. The animations are MUCH snappier now!

I really like this chart. Thanks again!

Chris

[Kai Chang]

Very nice. I'll work this into the library soon, and expose the
rendering functions publicly. Looks like path_foreground was the only
thing stopping this from being a totally separate plugin.

I added some automated brushes. Click axes titles in the first
"Brushing" section. Couple other goodies, but only minor library
changes.

http://syntagmatic.github.com/parallel-coordinates/

A new version of SVG parallel coordinates came out yesterday. It has a
great format for dimensions, which includes the name, type and scale
for each. Ordinal axis and polyline hovering. Check it out:

http://bl.ocks.org/3709000

[Kai Chang]

Progressive rendering has been added to parallel coordinates. Thanks
Chris for the example code. You can enable it by setting
.mode("queue").

https://github.com/syntagmatic/parallel-coordinates#parcoords_mode
https://github.com/syntagmatic/parallel-coordinates#parcoords_rate

There's an example at the bottom of the project page using the USDA dataset:

http://syntagmatic.github.com/parallel-coordinates/

I've also fixed a nasty bug in Render Queue where it would clobber
data passed in. I'm guessing this is why Chris needed an identity map
before passing data into rqueue. It was very frustrating to track this
bug down, so apologies to anyone who has hit it. Render Queue behaves
correctly now.

[mmicotti]

Hi all,

I have just started to play with parallel coordinates charts, and I find it very useful and nice, so, first of all: thank you everybody for the job you are doing with this kind of chart!
I also have a question for you:  I started working on the standard example, linked by the d3js website (here) and I would like to port on this kind of implementation the possibility to filter by group, clicking on the legend. Is it possible?
I am not a web developer, so I tried looking into the code of the other examples that have this functionality (like this one) but it seems to me that it is not so trivial to introduce this feature in the above example. Could you help me?
Thanks,

marco

[Kai Chang]

Yikes, the version at exposedata.com/parallel is actually one of the
oldest and most complicated. It's hard to implement any feature with
that example.

You'll find it slightly easier to learn from the version at
http://bl.ocks.org/d/3150059/. The true basic example is this:
http://bl.ocks.org/1341021

I'll include an example on
http://syntagmatic.github.com/parallel-coordinates/ of filtering by
group through a second visualization (like a legend) in the next
update.

Underscore.js is useful for programming this feature yourself,
especially _.difference and _.intersection

http://underscorejs.org/#difference
http://underscorejs.org/#intersection

[Will Turman]

Hi Kai, and all who contributed to this project, 

I just wanted to share my implementations of the Parallel Coordinates reusable chart.

I used it to map out 2012 statistical data for Major League Baseball batters with >300 plate appearances, and pitchers with >50 innings pitched. I am thrilled with how it turned out. 

http://charts.graphicbaseball.com/parallelbatting

http://charts.graphicbaseball.com/parallelpitching

I'd eventually like to implement some sort of coloring mechanism based on selected axis, but I'll save that for another day. 

Thank you for putting this project together, and for making it understandable and implementable for a relative noob. 

[Chris Rich]

Those are some pretty charts Will.  Good job!  Those tables looks rad.

Chris

[mmicotti]

Hi all,

I managed to filter by group using the updated version that Kai suggested me in his reply.
Now I have a new kind of problem that I am not able to solve: I need to keep one or more rows of my data as references, excluding them from the axes-filter operation.
I think that this can be useful generally: in my case I use parallel to compare many alternatives on the base of a given set of criteria. So it is useful to have a benchmark that it is always present on the chart.
In a previous version (based on exposedata.com/parallel ) I have put into the html page two parallel object overlapped with two different set of data (reference data below and alternatives data above): now I am not able to do the same and I am also interested to understand if it is possible to exclude from the axes-filter.
Thanks,

marco

[Jean-Barthelemy Mercier]

hi marco,
I was also struggling with the comparison problem using parallel coordinates.
I ended up with this solution : http://mgrafit.com/00-projects/proj02-PCPbaseline/PCPbaseline.html
Hope it will help you... (even if i use a click listener instead of a brush)

Jean

2013/3/1 mmicotti <marco....@gmail.com>

--
You received this message because you are subscribed to the Google Groups "d3-js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to d3-js+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Kai Chang]

Marco, that's a great question. Comparing reference data to a large
dataset is a very compelling way to use the chart. Even in the
nutrient example, I've often wanted to compare two foods directly: Big
Mac vs Whopper, Apple vs Orange, etc.

You might get some implementation clues from this example as well,
which shows a green line that is the average of the selected data:

https://appstorerankings.net/visualizing-the-ios-app-store

I am going to brain dump now. I ended up writing many blocks of text,
so I added section titles.

Visual Database Reference Application
============================

http://bl.ocks.org/3150059

I think of this example as a reference application for a visual
database. What you want is to create and compare arbitrary queries.
Usually, you color-code different views. Maybe calculate new columns
from existing columns. Reduce rows of data by averaging/summing and
display the derived dataset. Text search. CSV import/export.

It should be easy to generate a new view and layer it on the
visualization in just a couple lines of code with d3.parcoords.js. You
could apply any filter function on any dataset, and plot it along with
everything else. Right now it wipes out the old view. The brushing is
still too woven into the component.

I would love more perspectives on the interaction design of this
application as well. What are the features databases have that would
be useful? How can the application be more intuitive? How can it be
extended or refined, such as adding new visualizations, for a
particular dataset or domain?

Ordinal Axes
=========

These should be in the component soon, with brushing. Chris Rich and
Patrick Martin both great examples, so there's no excuse for it to not
be in there yet. I was thinking about working on it, but I brain
dumped instead

For now, check out these examples of ordinal axes:

http://dexvis.wordpress.com/2013/01/28/d3js-parallel-lines-and-football/
http://bl.ocks.org/chrisbrich/4173826

Hypersolid: 4-Dimensional Geometry
==========================

Recently, I'm using parallel coordinates to look at 4-D objects,
polychora. I'm learning about the mappings of lines and surfaces in
parallel coordinates. I have lines working well. Edges of the 4-D
shapes are each represented by 3 points: where the two vertices
intersect between each axis. The intersection point corresponds to the
line (going to infinity). See the "construction" examples as a
starting place, they render the vertices as well.

http://fleetinbeing.net/hypersolid/examples/ - URLs not stable
https://github.com/the-only-milosz/hypersolid

These are all in the examples directory of hypersolid, a 4-D shape viewer.

Actually the intersection can happen outside the axes as well. Once
you start the shapes rotating, you may notice some more dualities:
lines<->points, rotations<->translations, ellipses<->hyperbolas. It's
very easy to see families of parallel lines (groups of points in a
vertical line).

The data for these shapes was collected from:
http://paulbourke.net/geometry/hyperspace/

I don't feel I can explain this all very easily right now, but I plan
to attend Alfred Inselberg's lectures on parallel coordinates in
April. I'll create better examples then of hyperdimensional geometry.
Still learning right now.

Hypersurfaces
==========

Visualizing hyperplanes and hypersurfaces is the next step. I think it
would be possible to see the results of some machine learning
algorithms, such as Support Vector Machines.

If anyone understands this paper on curved surfaces in parallel
coordinates, an interactive example would help the rest of us:
http://www.cs.usc.edu/assets/004/83248.pdf

Network Visualization
===============

With the line<->point duality, many possibilities emerge for exploring
complex spaces. An example is networks/graphs, which is what the
vertices/edges of polychora are. Most network layouts are 2 or
3-dimensional, but you could instead have a static N-dimensional
layout. For example, plot metrics like betweenness or degree as
dimensions.

I'll post an example of a real-world graph eventually, but you can
interpret the hypersolid examples this way.

WebGL 2D
========

I've forked the WebGL-2D project to try out a WebGL renderer instead
of the 2D Canvas. I didn't get around to testing performance, but it
does seem slightly faster (just rendering speed... I turned off render
queue so the interaction is chunky).

With some more optimization, I bet WebGL could be 6-12x as fast.
Rendering is still the biggest bottleneck for larger datasets, 5k+
rows.

http://bl.ocks.org/syntagmatic/raw/5023284/
https://github.com/syntagmatic/webgl-2d

The project was designed to emulate the 2D canvas API, not to be fast.
If you happen to have a fast way to render millions of lines in WebGL,
post it on bl.ocks! This is not a high priority for me in the next few
months, but it would be a nice supercharge someday.

Projective Geometry and a Deeper Understanding
===================================

If you're looking for more insight on parallel coordinates, watch this
lecture on projective geometry.

http://youtu.be/NYK0GBQVngs

There's also a whole book on parallel coordinates which covers
projective geometry, hyperplanes, applications, etc.

http://www.amazon.com/Parallel-Coordinates-Multidimensional-Geometry-Applications/dp/0387215077

Hyperbolic Projection
===============

It would be super-interesting to embed parallel coordinates in a
hyperbolic space, and view it with the poincare disk or beltrami-klein
model. Hyperbolic space has much richer parallelism than Euclidean
space. If they had d3.js in the 19th century, I'm sure geometers would
have posted examples of this on bl.ocks for us.

Norman Wildberger has a great history of Non-Euclidean geometry lecture.

http://youtu.be/zHh9q_nKrbc

He even has an entire lecture series on the topic of "Universal
Hyperbolic Geometry", if your mind is tingling and craves much more.
Use what you learn and post simple examples on bl.ocks! You can do it!

http://www.youtube.com/course?list=EC6ACFCC19EA82CA71

Towards v1.0
==========

Parallel coordinates is a rabbit hole, and it's taking me longer to
move towards version 1.0 than I expected. It's a balance between
something flexible for exploring the geometry and a data exploration
plugin.

There should be a lot of development happening in the geometry-related
areas in April, and then a more stable plugin release in May. I can't
promise v1.0 then, but there will be more features built-in, better
docs, and more examples.

Kai

[Patrick Martin]

Kai,

I've said this before, but will say it again: impressive work.

Parallel coordinates is a great way to visually display a large amount of data visually, at once.

I've been using this at work to communicate certain things such as the relationship between projects, benefits, time estimates at various points in projects versus actual hours spent implementing such projects.  In other cases I use it to show timing breakdowns of various stages of a transaction.  It's a great view with so many applications.  I wish I could show the particulars, but unfortunately they are based on Enterprise data.

My target audience tends to be smart folks.  Business and technical folks.  But they are uninitiated in seeing data laid out in unfamiliar visualizations.

Invariably they are initially overwhelmed by Parallel Coordinates until I isolate a single path and trace through it with them.   Gradually though, they come to appreciate the view.

I think that the ultimate implementation of this visualization would allow for the dynamic inclusion and omission of particular columns or dimensions.  When initially display this view to the uninitiated, I will typically start with a Parallel Coordinate view consisting of 2-3 vertical axis or dimensions.  I will then move on to a more detailed version which may show up to 7 dimensions of data at once.

It would be great if I could dynamically include these from a single interactive view.  I plan to add this capability to my visual eventually.

Also, it would be nice to be able to normalize selected vertical axis along the same extent.  That way it can also serve as a multi-dimensional slope-graph.  The current D3 visual normalizes each vertical axis based upon it's individual high and low values for that particular column.

I have implemented the across-the board normalization logic myself.  Unfortunately, my implementation is all-or-nothing right now and I typically disable this feature and manually re-enable it when generating views which are better normalized to a single consistent scale.

Other useful things might be to support logarithmic scales or statistical scales to produce greater separation between clusters of similarly behaving entries.

Last, but not least, some sort of SQL-like query feature might also be nice for dynamically isolating series of interest.

Keep up the good work and thanks for the provocative discussion!

- Pat

[Patrick Martin]

I forgot a couple of other things which would be quite nice:

Fisheye and linear zooming (which I see in some of your examples)

Also, the ability to select/brush multiple ranges on the same vertical axis.

[mmicotti]

Thank you for all the answers you gave me!
Finally I succeed solving my problem with a working (but not very elegant) solution. In my case the reference alternative that I need to keep always on chart is the first row of my dataset (data), so starting from this example I did just a few changes:

• in html file I added a canvas tag in the chart div, called reference
• in parallel.js I added as well an element called reference, with the same instruction of highlighted
• in parallel.js I added the following line at the end of brush() function, after calling paths:
  
  

path(data[0],reference);

and that's all!
I know that there are many tuning that can be done in order to generalize this trick, so I will try to work on it soon.
In the next days I will also put a working example on bl.ocks, so you can see it live.
thanks again to all,

marco

[Kai Chang]

Added preliminary support for automatic ordinal axes, based on the
examples and suggestions people generously provided.

For string dimensions that are names or ids and not categories, the
current behavior creates way too many tick labels. This slows the
whole chart down.

An example with Wyoming veteran gravesite data (takes a while to load
and CPU intensive, due to the labels)

http://syntagmatic.github.com/parallel-coordinates/examples/veterans.html

I''ll work on saner labeling and factoring out some of this dimension
type logic (currently only numeric/ordinal) for 0.1.8.

[Stephan De Spiegeleire]

Is there anybody here who knows a way to generate a parallel coordinates viz like this one in D3? If so, please contact me. Thanks!

[Simon Russell]

This is the d3 mailing list, so it's all d3.  So I'd start with one of the examples, or the library.

--

[Patrick Martin]

You can generate a D3 Parallel Coordinates visualization without having to manually code by using Dex.

http://dexvis.com

And there is a video showing it in action over here:

http://www.youtube.com/watch?v=-Gnt8y12j64

Forward to 27 minutes in to see the coverage of the Parallel Coordinates visualization.

- Pat

[Nacho Caballero]

Hi Kai,
This is great work. I'll try writing a clickme ractive for it so it becomes accessible from R.

[Nacho Caballero]

As promised: bit.ly/clickme_par_coords Parallel coordinate plots straight from R

[bayram hosari]

Thanks a lot. it is an amazing job.
I started working with parallel coordinates lately and still having some problems. I would like to ask you if there is any example code of linear zooming ?

[Kai Chang]

You mean like this fisheye effect?

http://bl.ocks.org/syntagmatic/3290263

I haven't implemented this for the reusable chart yet.

--

[bayram hosari]

Thanks for your quick replay, but in fact what is i am thinking of, is to zoom on axes by changing the height of the axes (increase / decrease the height). I want to zoom in/out by mouse wheel, but i did not arrive to do it yet. is there any example code about that?

thanks again.

2013/4/24 Kai Chang <kai.s...@gmail.com>

--
You received this message because you are subscribed to a topic in the Google Groups "d3-js" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/d3-js/zsi-w-jd860/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to d3-js+un...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Cordialement,

HOSARI Bayram

[Kai Chang]

Ahh, so you want the zoom behavior on each axis. That would be neat, but I don't yet have an example of this.

https://github.com/mbostock/d3/wiki/Zoom-Behavior

In this version, you can select some subset and click the "Keep" button at the top to rescale and zoom in on that data, which is similar but not quite what you're talking about.

http://bl.ocks.org/syntagmatic/raw/3150059/

[bayram hosari]

Thanks again :) .. that would be useful for my application right now.
If you do not mind me asking some more questions about d3, i would like to ask you about treating data from csv file.
in fact, i want to edit the way parallel coordinates display the intersects of data with axis. i need to draw little circles on the axis where data intersects. and if i have duplicated circles ( duplicated values on the axis) i want to draw circles next to each other as in the image below. i could do that but with preprocessing the data in the csv  by counting the duplicated values for each column. it does what i want but with bad performance with large csv file >1K lines.
Is there any idea to do that with less cost of preprocessing of data?

[Kai Chang]

I suspect slow performance is from rendering so many circles in SVG, rather than any data processing tasks. They get redrawn every time you brush?

[bayram hosari]

i am using polybrush.js (like: lasso select) to select the circles not the normal brush. and i just recolor the selected circles without redrawing them. i think the preprocessing is cost a lot because i need to keep arrays in the memory to save some things like: each value how many times it got repeated (duplicated values) to place circles next to each other, an array of coordinates for each circles (x,y) because i want these coordinates to get selected values in the lasso select.

thanks Kai,

2013/4/24 Kai Chang <kai.s...@gmail.com>

--
Cordialement,

HOSARI Bayram

[Krishna Moorthy]

Hi, 

    I am developing a rickshaw bar graph and i am in need of displaying the y-axis coordinates on top of the bar graph in Onload event some thing like this http://www.rgraph.net/demos/drawing-api-xaxis-basic.html . is there any way to achieve it using rickshaw. please let me know asap. thanks in advance. 

[Mrugank Parikh]

Hi Patrick, can you share in which example shared by Kai you found multiple ranges in the same vertical axis? @kai, if you can also share that link, it would be useful. I have not seen such selection using brush yet.

[Kai Chang]

I'm not aware of an example with multiple selections in the same axis.

--

[Patrick Martin]

Kai,

Fisheye I think should be a right mouse click feature.  It's cool but there is a usability issue.  Ever try to grab one of those axis when fisheye is on?  The axis runs away from you.

[Kai Chang]

Yes, right click might be better than holding down CTRL. Then you can navigate around with just the mouse.

How about for touch devices? Two finger drag?

--

[Patrick Martin]

I thought about it a bit more, and maybe you simply have a button to toggle fisheye mode.  Touch devices will get confusing. And the other problem would be that the process of moving the mouse back to the toggle button would alter the area the fisheye is zoomed in upon.

Which makes me think that a zoom model is more intuitive across devices.  Pinch open and close could zoom in and out on the region of interest which is pretty similar to zoom and pan..

The pinch model would also work well for emerging devices such as the Leap Motion.

But...easy to say, difficult to code.