Can't create dataset with API

[Stephane Donze]

Hi there,

I must be missing something, I am trying to create a simple, empty dataset using the API but nothing is working. I first tried the following code using the Java API:

Datasetreference ref = new Datasetreference();

ref.setProjectId(PROJECT_ID);

Dataset dataset = new Dataset();

dataset.setDatasetReference(ref);

dataset = bigquery.datasets().insert(dataset).execute();

System.out.println("dataset=" + dataset);

but I get a GoogleJsonResponseException: 404 Not Found  with no detail on what the error is or what is "not found" . I tried to add a dataset id to the Datasetreference object, and various combinations of parameters in the Dataset object, but I consistently get this 404 Not Found. 

Note that the bigquery object I'm using is working fine for retrieving the list of datasets for example, so the error is really specific to the "insert" operation.

After that I tried to create the dataset manually by using the Web client (using the "Try it" button on the page https://developers.google.com/bigquery/docs/reference/v2/datasets/insert ). On this page I get a different error, which seems to indicate that I am missing a required parameter. However, according to the documentation, only the "projectId" parameter is mandatory. So I don't see what is missing here.... 

400 Bad Request

  

{

 "error": {

  "errors": [

   {

    "domain": "global",

    "reason": "required",

    "message": "Required"

   }

  ],

  "code": 400,

  "message": "Required"

 }

}

Thanks in advance for your help

Stéphane

[Jordan Tigani]

It may be that your project is not found. What is the project id you are using? (and you should need a dataset id, but it sounds like that doesn't have any effect on the problem).

thanks

-jordan

[Stephane Donze]

Hi Jordan,

thanks for your reply. The project id I am using is the string of digits found at the beginning of my OAuth consumer key (so if my consumer key is 1234.apps.googleusercontent.com, I'm using "1234" as a project id). I am not sure if this is the correct ID to use, but it is working fine when I use the API to retrieve the list of datasets, using the following code:

DatasetList list = bigquery.datasets().list(PROJECT_ID).execute();

for (DatasetListDatasets ds : list.getDatasets()) {

Datasetreference ref = ds.getDatasetReference();

System.out.println("Ref=" + ref.getProjectId() + ":" + ref.getDatasetId());

}

Stéphane

On Monday, March 5, 2012 6:15:45 PM UTC-8, jtigani wrote:

It may be that your project is not found. What is the project id you are using? (and you should need a dataset id, but it sounds like that doesn't have any effect on the problem).

thanks

-jordan

[Jordan Tigani]

OK I think you just need to enable "bigquery" in the api console services tab (https://code.google.com/apis/console/?pli=1)

Note that when you use the form at 

https://developers.google.com/bigquery/docs/reference/v2/datasets/insert

you should click on "change request body" and add a datasetReference and datasetId. (this is what was missing when you got the Required message. We have an open bug to fix the required error message, since it is very unhelpful as it is).

Also, as an FYI, the project ID is not a secret -- I was asking because I wanted to look it up in the access logs to see why it was failing.

let me know if enabling bigquery in the api console tab works.

-jordan

[Stéphane Donzé]

Hi Jordan,

my project ID is 647811276564 and I double checked that Bigquery is activated in the API console.

For the Web UI, indeed the datasetId seems to be mandatory (contrary to what is said in the documentation), so when I set both projectId and datasetId in the datasetReference, it seems to work and the dataset is created correctly.

In the code however I still get this 404 Not found error . I think there is a bug in the client library because if I replace my access token with some invalid value, I have the same 404 error instead of an authorization error. 

Again, other Bigquery calls such as datasets().list(PROJECT_ID).execute() work correctly and return the expected results and the expected error codes when I provide an invalid access token... 

Is there a way to trace what are the HTTP calls done by the client library to check what URL is returning this 404 error ? I guess I could maybe subclass the HttpTransport class to put some debug messages in there ?

Thanks

Stéphane

[Jordan Tigani]

Ok looks like we're seeing a wart of the generated client and REST. Basically the issue is that the id is passed in two places -- the url and the object. You're setting it on the object but not the URL. This makes complete sense, but not to the library. I'll follow up with the library-generation team, since this is indeed goofy.

At any rate, you should be able to create the dataset by changing your code slightly to set the project id on the Insert object you get back from calling insert(...). Here is your original code with the changes in yellow:

Datasetreference ref = new Datasetreference();

// ref.setProjectId(PROJECT_ID);

  ref.setDatasetId(DATASET_ID);

  Dataset dataset = new Dataset();

  dataset.setDatasetReference(ref);

  dataset = bigquery.datasets()

      .insert(dataset).setProjectId(PROJECT_ID).execute();

  System.out.println("dataset=" + dataset);

let me know if this works for you

-jordan

[Stephane Donze]

Thanks, that fixed the problem !