Google BigQuery API Updates: Run more queries at once, Table expiration time, and more (Jun 1, 2012)
1,905 views
Skip to first unread message
Michael Manoochehri
Jun 1, 2012, 5:27:51 PM6/1/12
to bigquery...@googlegroups.com
Hello BigQuery Developers:
The BigQuery engineering team has made many updates to the BigQuery API since our public launch last month. Here's a summary of new features:
Updated Model for Running Multiple Queries at a Time:
We've recently updated the project query quota (originally set to 2 concurrent queries at a time). Now, developers are able to run many queries at a single time, as long as the total amount of data processed is 200GB or less. After this threshold is reached, an additional query of any processed data size can be run as well. For example, if you run a set of queries that consume 20 Gb, you will be able to run 10 at a time, plus one more query of any size. If your needs exceed this amount, our sales team would like to hear more about your use case (here).
This change should increase the types of applications you can build with BigQuery, as well as make it easier to develop applications that require both interactive and automated background queries.
Set an Expiration Time for BigQuery Tables:
It's now possible to set an expiration time of a BigQuery table using the "expirationTime" property of a Table Resource, either by creating the table with an expiration time or updating it to have an expiration time. After a table expires, it will be deleted and its storage reclaimed. Of course, if you don't specify an expirationTime, the table will persist indefinitely.
Improved Token-based Pagination Support for List Operations on Tables, Datasets, Jobs, and Projects
If the results of a list operation return more than a single page of data, the result data will have a nextPageToken property. To retrieve the next page of results, make another list call and include the token value as a URL parameter named pageToken.
PATCH Support
Developers can now use the PATCH HTTP method to make modifications to existing Dataset and Table resources.
Links:
- The BigQuery Team
The BigQuery engineering team has made many updates to the BigQuery API since our public launch last month. Here's a summary of new features:
Updated Model for Running Multiple Queries at a Time:
We've recently updated the project query quota (originally set to 2 concurrent queries at a time). Now, developers are able to run many queries at a single time, as long as the total amount of data processed is 200GB or less. After this threshold is reached, an additional query of any processed data size can be run as well. For example, if you run a set of queries that consume 20 Gb, you will be able to run 10 at a time, plus one more query of any size. If your needs exceed this amount, our sales team would like to hear more about your use case (here).
This change should increase the types of applications you can build with BigQuery, as well as make it easier to develop applications that require both interactive and automated background queries.
Set an Expiration Time for BigQuery Tables:
It's now possible to set an expiration time of a BigQuery table using the "expirationTime" property of a Table Resource, either by creating the table with an expiration time or updating it to have an expiration time. After a table expires, it will be deleted and its storage reclaimed. Of course, if you don't specify an expirationTime, the table will persist indefinitely.
Improved Token-based Pagination Support for List Operations on Tables, Datasets, Jobs, and Projects
If the results of a list operation return more than a single page of data, the result data will have a nextPageToken property. To retrieve the next page of results, make another list call and include the token value as a URL parameter named pageToken.
PATCH Support
Developers can now use the PATCH HTTP method to make modifications to existing Dataset and Table resources.
Links:
- BigQuery API Documentation: https://developers.google.com/bigquery/docs/reference/v2/
- BigQuery command line interface: https://developers.google.com/bigquery/docs/cli_tool
- For BigQue| development support and questions, post a question using our Stack Overflow tag "google-bigquery" or visit: http://stackoverflow.com/questions/tagged/google-bigquery
- The BigQuery Team
Reply all
Reply to author
Forward
0 new messages