breaking changes to bigquery package coming soon

45 views

Skip to first unread message

Jonathan Amsterdam

unread,

Oct 14, 2016, 10:21:52 AM10/14/16

to google-api-...@googlegroups.com

Next week we'll complete our overhaul of the BigQuery client (cloud.google.com/go/bigquery). Most of the new surface is already in place, but we'll be removing the old code, resulting in many breaking changes.

We think these changes improve the library in a number of ways:

It's much easier to find the operation you're interested in. Instead of a general Copy method, there are separate methods for loading, extracting, copying and querying.
Modifying table metadata is simpler, requiring only one call.
Iterators over rows and tables now follow the common iterator guidelines for the Go Cloud Client Libraries, so they look more like other iterators in these packages.

After these changes go in, we expect no more major breaking changes to the Go BigQuery client library. (We want to keep the door open to your feedback.)

A summary of the changes follows. You can learn more from the README file in the CL.

To isolate your work from these changes, vendor the cloud.google.com/go/bigquery package. We've created the tag "v0.3.0" for that purpose.

Client.Copy and Client.Read

We'll be removing Client.Copy and Client.Read. Client.Copy supports loading, extracting, querying, and copying. Its replacements target each operation individually:

Table.LoaderFrom constructs a Loader, which you can configure by setting its fields, and then call Run on to start a load job.
Table.ExtractorTo does the same for extraction.
Table.CopierFrom supports copying into a table from one or more other tables.
Query has fields for configuration, as well as a Run method to start a query job.

These methods are already available.

As an example, instead of

client.Copy(ctx, table, gcsReference)

you would write

table.LoaderFrom(gcsReference).Run(ctx)

We'll also remove Client.Read. Instead, use Table.Read to read directly from a table, or Job.Read to read from a query job. These methods are not yet available in their final form.

Table.Patch

We'll remove Table.Patch and the associated type TableMetadataPatch. They are replaced by the easier-to-use Table.Update method, which is already available.

For instance, instead of

p := table.Patch()
p.Description("new description")
metadata, err := p.Apply(ctx)

you would write

metadata, err := table.Update(ctx, bigquery.TableMetadataToUpdate{
Description: "new description",
})

Row Iterator

The Iterator type will be renamed RowIterator and will conform to the standard pattern for the Cloud Client Libraries.

Instead of

var values bigquery.ValueList

for iter.Next(ctx) {
if err := iter.Get(&values); err != nil {

// TODO: Handle error.

}
// TODO: Use values.
}
if iter.Err() != nil { ... }

you would write

var values bigquery.ValueList
for {
err := iter.Next(&values)
if err == iterator.Done {
break
}
if err != nil {
// TODO: Handle error.
}
// Use values.
}

Other Changes

Instead of Dataset.ListTables, call Dataset.Tables to get a TableIterator.
Instead of Client.OpenTable or Client.Table, construct a Dataset with Client.Dataset or Client.DatasetInProject, then call Table on the Dataset.
Instead of Client.CreateTable, construct a Table from a Dataset and call Create on it.

All of these new methods are available now.

Jonathan Amsterdam

unread,

Oct 19, 2016, 1:01:12 PM10/19/16

to google-api-go-announce

These changes are in, along with two more:

Table.NewUploader renamed to Table.Uploader. Instead of passing options to the method, the returned Uploader is configured by setting its fields.
The Load method of ValueLoader now takes a Schema in addition to a slice of Values.