CSV data source -> Exhibit + TimeplotExtension ?

[Dan]

Hello and thanks to everyone who has helped to put together the SIMILE
project. I'm very impressed with Exhibit and am planning to use it as
a data exploration interface for some time series data.

Is it possible to use a CSV data source as input for the Exhibit
Timeplot extension? I see that it's possible when using the standalone
Timeplot code (as in the example with Bush's ratings). I would lke to
use the Exhibit extension instead, as it comes with all of the nice
selection widgets. But when looking at the examples and code for the
Timeplot extension in Exhibit (timeplot-view.js), I couldn't see how
to load a CSV and then use a ColumnSource parameter. The comments in
the code say this:

// There are two ways to specify a series, either by including
all
// the data in one DB item or by connecting one value across
several
// items. Across several is checked first during the set.visit
() and
// addEvent is called if found; if not, addSeries is called.

I know how to load the DB from a JSON file, using a <script> tag in
the head section of the html document:
<link href="test-data.json" type="application/json" rel="exhibit/
data" />

My question is whether it is possible to load this DB from a CSV file?
Or would I need to transform the CSV into a JSON format (which I would
like to avoid)?

thanks
Dan

[Stefano Mazzocchi]

Ryan did an example of something like this in the past, you can see the
result at

http://ryanlee.org/2008/03/rsy/rivalry.html

Not sure this is enough but might get you started.

--
Stefano Mazzocchi Application Catalyst
Metaweb Technologies, Inc. ste...@metaweb.com
-------------------------------------------------------------------

[Dan]

Thanks for the suggestion. Unfortunately, that example uses JSON
format source data : http://ryanlee.org/2008/03/rsy/data.json

> Metaweb Technologies, Inc.                      stef...@metaweb.com
> -------------------------------------------------------------------

[John Callahan]

I noticed that too.  But what a cool example!   Thanks for sharing this link. 

- John

**************************************************
John Callahan
Geospatial Application Developer
Delaware Geological Survey, University of Delaware
227 Academy St, Newark DE 19716-7501
Tel: (302) 831-3584  
Email: john.c...@udel.edu
http://www.dgs.udel.edu
**************************************************

[Ryan Lee]

Exhibit doesn't do CSV like that, no. I wrote up a one-off translator
for this Exhibited Timeplot:

http://ryanlee.org/2009/02/obirh.html

which is a display of Online Bank Interest Rate history pulled from
somebody else's Google Docs spreadsheet. I think you'll have to
normalize your data stream(s) one way or another; it's quite possible
CSV->JSON is the least painful. Anything else currently involves
writing Exhibit / Timeplot code.

--
Ryan Lee rya...@csail.mit.edu
MIT CSAIL Research Staff http://simile.mit.edu/
http://people.csail.mit.edu/ryanlee/

[Dan]

Thanks Ryan for that pointer to your example.

I agree that the CSV->JSON translation would be an easy approach, but
I'd like to
avoid the overhead of JSON formatting in the data files for various
reasons.

I am now experimenting with writing a CSV importer for Exhibit.
Ideally I'd like it to be eligible for contribution into the regular
Exhibit code base, so I'd welcome any guidance from this group about
things to keep in mind (best practices and potential pitfalls). I
expect to re-use the existing "pointTime" "pointValue" and
"seriesConnector" parameters, and introduce a new parameter to specify
if the first column contains the names of the fields in the CSV; and
if not, what the field names are (in an array). Does that sound
reasonable?

Dan

> Ryan Lee                  ryan...@csail.mit.edu

[Larry Kluger]

Hi Dan,

I did a short investigation of writing a csv source method for Timeline. One issue that came up: there is no standard for the csv "format." In particular, Excel's "csv" output is different in various ways from other producers of "csv." Since I wanted to make something generic, I decided to change course.

I'm now working on a perl Excel to JSON program instead that would be run as a batch job.

My tip is that you'll want to test your sw with several different versions of csv including Excel's.

Regards,

Larry
ps. Sorry, I don't have any info on your questions about Exhibit--I focus on Timeline.

---

From: Dan <danb...@gmail.com>
To: SIMILE Widgets <simile-...@googlegroups.com>
Sent: Wednesday, March 11, 2009 7:45:40 PM
Subject: Re: CSV data source -> Exhibit + TimeplotExtension ?

[Ryan Lee]

Hi Dan,

Thanks for your willingness to help out, it's always great to find
another participant. So user CPU costs less than bandwidth? :)

I'll preface the following with this: you should write whatever code you
find satisfies your itch.

I'm all for expanding the reach of Exhibit for making Exhibit-ors' lives
easier, but the conversion efforts to date have taken place outside of
Exhibit. For instance,
http://simile.mit.edu/wiki/Exhibit/How_to_make_a_publications_exhibit
details how to use a BibTeX file as a data source by way of SIMILE's
Babel service (which also does tab-separated values, though I don't know
if there's a built-in to Exhibit for that format).

What I had in mind was some way of tying Exhibit JSON to Timeplot CSV -
maybe adding a third alternative to the Timeplot/Exhibit extension to
index into a Timeplot CSV file(s?), by columns and/or coodinates, or
something similar.

If that doesn't sound at all interesting, I would advise using a form
that mimics the Google Spreadsheet formatting for Exhibit, described here:

http://simile.mit.edu/wiki/Exhibit/How_to_make_an_exhibit_from_data_fed_directly_from_a_Google_Spreadsheet#Formatting_the_Data

From a much higher level view, one of the side effects Exhibit has is
making data accessible. By placing conversion code inside Exhibit
itself, it makes it harder to reuse that data in the same way it was
intended to be read as that intention now resides in code, separate from
data. There are obviously architectural ways around that (Babel-like
services), but I appreciate that Exhibit is closely associated with
JSON. I'd be interested to hear what other developers think.

But again: scratch your itch :)

--
Ryan Lee rya...@csail.mit.edu