Google BigQuery Dataset Export

2 views
Skip to first unread message

Ankush Agrawal via StackOverflow

unread,
Jul 27, 2014, 9:38:48 PM7/27/14
to google-appengin...@googlegroups.com

I'm trying to use Google BigQuery to download a large dataset for the GitHub Data Challenge. I have designed my query and am able to run it in the console for Google BigQuery, but I am not allowed to export the data as CSV because it is too large. The recommended help tells me to save it to a table. This requires requires me to enable billing on my account and make a payment as far as I can tell.

Is there a way to save datasets as CSV (or JSON) files for export without payment?

For clarification, I do not need this data on Google's cloud and I only need to be able to download it once. No persistent storage required.



Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/24987389/google-bigquery-dataset-export

someone1 via StackOverflow

unread,
Jul 27, 2014, 10:58:56 PM7/27/14
to google-appengin...@googlegroups.com

If you can enable the BigQuery API without enabling billing on your application, you can try using the getQueryResult API call. You're best bet is probably to enable billing (you probably won't be charged for the limited usage you need as you will probably stay within the free tier but if you do get charged it should only be a few cents) and save your query as a Google Storage object. If its too large I don't think you'll be able to use the Web UI effectively.



Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/24987389/google-bigquery-dataset-export/24987872#24987872

Felipe Hoffa via StackOverflow

unread,
Jul 28, 2014, 1:49:19 PM7/28/14
to google-appengin...@googlegroups.com

See this exact topic documentation:

Summary: Use the extract operation. You can export CSV, JSON, or Avro. Exporting is free, but you need to have Google Cloud Storage activated to put the resulting files there.



Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/24987389/google-bigquery-dataset-export/25001172#25001172

N.N. via StackOverflow

unread,
Jul 29, 2014, 9:06:39 AM7/29/14
to google-appengin...@googlegroups.com

use BQ command line tool $ bq query

use the --format flag to save results as CSV.



Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/24987389/google-bigquery-dataset-export/25016012#25016012
Reply all
Reply to author
Forward
0 new messages