What's Wrong with This Picture?

[DrQ]

How not to display performance data. (or any data, for that matter)

[DrQ]

On Monday, July 8, 2013 2:25:18 PM UTC-7, DrQ wrote:

How not to display performance data. (or any data, for that matter)

The same post on Linked-in elicited the following responses (copied here in case they're not visible on the web):

Kao-Tai Tsai • Dear Neil: To transform data or not depends on your objectives. I must have done tons of graphs in my career and I have to admit that it is difficult for me to comment without knowing what you want to do.

Mark Powell • Neil, The problems you indicate about use of a log scale (unusual to see it on the abcissa only) seem to be mostly about it being misleading. No argument there, but that issue can be rather easily fixed by adding tick marks at the ordinal values on the log axis. That it is a logarithmic axis becomes obvious then. 
Engineering and science are different worlds. For many natural phenomena, we expect an exponential response (on the ordinate) to the independent variable (on the abcissa), so with a log/linear plot (y axis log only) we expect to see a familiar straight line. In those cases, the linear/linear axes will be misleading. And many natural phenomena exhibit an exponential/exponential relationship, where a log/log plot presents a straight line. 
So, going one step beyond Kao-Tai's comment, it also depends on what the phenomenon is your plotting as to what kind of plot to use. Some folks' objective are indeed to mislead with their plots. 

Laura Meyerovich • The medium (in this case the graph scale) is the message. All too often reports are illustrated by graphs that do not have a message, and so do not add information, whatever scale is used. To Mark's point, if the message is that the process in question is exponential or logarithmic, linear scale would not do. It appears that the first "oldie but baddie" graph has a message that response is at first logarithmic rather than linear or exponential, and flattens out after 254 threads, so exponential scale on abscissa is entirely appropriate. Just turn the graph 90% (assume that the message is "how many threads are required to produce this throughput") - will you still object to exponential scale on what is now a Y-axis?

--------------

I'm truly speechless. Is it me or is it them? Either way, I'll definitely be using these comments as discussion topics in the upcoming GBoot and GCaP classes.

--njg

 

[Mario Jauvin]

I guess I am speechless also. But for another reason altogether. I read your post and a few of the LinkedIn response and I don't get why YOU are speechless :-)!  It must be me!

Mario François Jauvin

MFJ Associates, (613) 686-5130, option 1

Fax 1-888-335-8349

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "Performance Visualization" group.
To unsubscribe from this group and stop receiving emails from it, send an email to perfviz+u...@googlegroups.com.
To post to this group, send email to per...@googlegroups.com.
Visit this group at http://groups.google.com/group/perfviz.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[DrQ]

Come to class and find out.

[Baron Schwartz]

I'm with Neil on this. Have these people never heard of Edward Tufte, or "How To Lie With Charts", or taken any courses in engineering and read any case studies about spectacular failures and their causes??? This stuff matters. This is not casual "agree to disagree" territory.

[Annie Shum]

Ditto, I am with Neil and Barry on this. In all honesty, log scale can be challenging for many.

 

Regards, Annie

Twitter@Insightspedia

--

[spa...@systemdatarecorder.org]

On Tuesday, July 9, 2013 12:25:18 AM UTC+3, DrQ wrote:

How not to display performance data. (or any data, for that matter)

Reminds me of another plot from vmware
 http://blogs.vmware.com/performance/2010/12/performance-and-use-cases-of-vmware-directpath-io-for-networking.html
 
stefan

[DrQ]

Nice example.

At least there are grid lines and axis labels that clearly indicate the use of log10 scaling on the x-axis. No doubt that's what LI commentator Mark Powell was alluding to, but that's not what was done with the example log-plot that I showed. In that case, only rather small, equidistant, tick labels were used.

Actually, in the VMware plot it looks like log scaling might have been done to "compress" the curves into the available CSS plot width. As I mentioned, it makes the curves take on a hockey-stick shape that is artificial and can lead the reader to infer a nonlinear effect when in fact it's probably rather linear.

If it was a marketing objective, however (green is way better?), this might be exactly the desired (false) effect. But I wasn't discussing marketing/sales distortions.