Re-using d3.csv returns undefined / empty csv array?

[Daniel]

How is it that currentData is populated with nested data here:

d3.csv("data/data.csv", function(csv) {
currentData = d3.nest()
.key(function(d) { return d.country; })
.key(function(d) { return d.year; })
.rollup(function(d) {
return {x: d[0].val}
})
.map(csv);
});

But if another "d3.csv" is added before it equals "undefined"?


d3.csv("data/countries.csv", function(csv) {

});

d3.csv("data/data.csv", function(csv) {
currentData = d3.nest()
.key(function(d) { return d.country; })
.key(function(d) { return d.year; })
.rollup(function(d) {
return {x: d[0].val}
})
.map(csv);
});


Thanks!

[Mike Bostock]

The `d3.csv` method loads CSV data asynchronously, so your callbacks
will be invoked some time in the future. If you want access to both
CSV files, you can either embed one callback inside the other, or keep
track of whether both CSV's have arrived.

Mike

[Daniel]

> The `d3.csv` method loads CSV data asynchronously, so your callbacks
> will be invoked some time in the future.

The callbacks are invoked fine and I should definitely build some sort
of "gate function" that opens when both csv files are loaded. (If this
is best practice?)

Still I don't understand why this:

d3.csv("data/data_file2.csv", function(csv) {
console_log(csv); => Complete CSV
});

While this:

d3.csv("data/data_file1.csv", function(csv) {
console_log(csv); => Complete CSV
});

d3.csv("data/data_file2.csv", function(csv) {
console_log(csv); => [ ]
});


Thanks for your help :=)

[Daniel]

Sorry for rebumping this, but I really would like to understand why
csv is empty or parsed differently 2:nd time?

d3.csv("file.csv", function(csv){
console.log(csv); => Array of objects
});

d3.csv("file.csv", function(csv){
console.log(csv); => [ ]
});

Thanks.

[Mike Bostock]

> Sorry for rebumping this, but I really would like to understand why
> csv is empty or parsed differently 2:nd time?

Your asynchronous callbacks can be invoked in arbitrary order. So,
even if your code looks like this:

d3.csv("data/data_file1.csv", function(csv) {
console.log("data_file1 loaded");
});

d3.csv("data/data_file2.csv", function(csv) {
console_log("data_file2 loaded");
});

You might see the following on the console:

data_file2 loaded
data_file1 loaded

This is particularly true if data_file1 is larger than data_file2;
they are typically loaded in parallel by your browser, so the larger
one will take longer to load.

Another issue is that console.log lazily displays nested objects and
arrays. If the contents of your array (`csv`) are later changed by
your code, an earlier console.log might display the value of the data
later, rather than at the time you logged in. One way to force
console.log to display the current value of an object is to convert it
to JSON:

console.log("data_file2", JSON.stringify(csv));

Another likely cause is that one of your two files (data_file1.csv) is
invalid CSV. One helpful way is to debug via the developer console.
Try typing this in and seeing what happens:

d3.csv("data_file1.csv", function(csv) { console.log(csv); });

Then run it again (after waiting for the file to load, of course!).
Then point it at your other file and see if it works.

Mike

[Daniel]

Thanks for your advice! - I tried debugging it further but I still get
odd results (though the csv seems to be valid).
Here is small sample archive if you'd like to see the results of
console.log:
http://lapidus.se/external/csv_test.zip

Thanks :)

[Mike Bostock]

I get what appears to be the correct behavior:

test [{"country":"sweden","year":"1950","ind1":"50"},{"country":"norway","year":"1950","ind1":"60"}]
test2 [{"country":"sweden","year":"1950","ind2":"50"},{"country":"norway","year":"1950","ind2":"60"},{"country":"denmark","year":"1960","ind2":"70"}]

I added "test" and "test2" to the front so I could see which file it
was printing. I'm running Chrome 11.0.696.57 beta on Mac OS X, with
python -m SimpleHTTPServer to serve files on my local machine. I also
tried this on Safari 5.0.5 and Firefox 4 and saw identical results.
Are you using a local server? If you run the example out of file:
you'll get security errors.

Mike

[Daniel]

Very helpful of you to print the log for me.
I upgraded to FireFox 4 and it seems to work fine now.

In FF 3.6.17 (with & without local server) the results I got was:

test [{"country":"sweden","year":"1950","ind1":"50"},
{"country":"norway","year": "1950","ind1":"60"}]

test2 [ ]

Perhaps there was a caching error or other security error ...
Well, well - big thanks once again :)

/D

[Mike Bostock]

> In FF 3.6.17 (with & without local server) the results I got was:

Ah, yes. That's a big with regular expression literals in FF 3.6; it's
not creating a new RegExp object each time parseRows runs, so
`lastIndex` isn't being reset to zero. I'll commit a work-around in
1.14.1.

Mike

[Daniel]

Great, thanks :=)