unable to use remote_api_shell and bulkloader (all dependent on remote api) with 1.9.30 SDK.

[timh]

Hi

I haven't had to use the remote_api for some time, and now need to use the bulkloader and remote_api_shell.  (Appengine app was first deployed in 2012)

 However utilising the current SDK 1.9.30 and after creating service accounts, it seems that there is an ongoing problem with authentication .'

I get the following when trying use download_data 

[DEBUG   ] [WorkerThread-0] WorkerThread: started

2016-01-03 15:01:19,810 INFO client.py:539 Attempting refresh to obtain initial access_token 

2016-01-03 15:01:20,118 INFO client.py:797 Refreshing access_token 

2016-01-03 15:01:21,733 INFO client.py:571 Refreshing due to a 401 (attempt 1/2) 

2016-01-03 15:01:22,040 INFO client.py:797 Refreshing access_token 

2016-01-03 15:01:23,101 INFO client.py:571 Refreshing due to a 401 (attempt 2/2) 

2016-01-03 15:01:23,415 INFO client.py:797 Refreshing access_token 

repeated infinitely.

I have tried multiple variations and using the remote_api_shell, which eventually times out with to many failed attempts.

Following the docs on the Creating an Application Default Credentials and the only type of account that has a correctly formatted json credential file is a "Application Service Account"

The permissions are "Edit" for the Service Account.

(Note the docs are incorrect as far the cloud console sections, for instance it refers to "APIs & Auth" however that section is called "API Manager"

I am only using the "builtin" remote api definition.

Has anyone been able to use the remote_api_shell with recent SDK's ?

Any thoughts.

 

Cheers

Tim

[timh]

Hmm did some searching, and after further testing found that I could get both appcfg download_data and remote_api_shell to work by using  gcloud login, 

but no amount of effort seems to get an application service account working as outlined https://developers.google.com/identity/protocols/application-default-credentials 

T

[Adam (Cloud Platform Support)]

The permissions for download_app and download_data are restricted to the uploader of the application and the project owners. This is mentioned in the Python docs but not the Java docs. Also in order to use a service account with with gcloud auth activate-service-account, you need to export the key file, but you can only export key files for service accounts you create and not the application's default credentials. These service accounts can only have view or edit permissions, but not owner.

[timh]

Hi Adam

I figured the problem had to be something like that, however there current docs don't even mention download_data at all, and the error you get from appcfg if you haven't logged in with gcloud suggests you need to set GOOGLE_APPLICATION_CREDENTIALS and refer to the docs for setting up an default application service account.  That section does talk about gcloud as one of 5 options.  Though reading through this the application service account seems on the face of the most appropriate choice.

Historically any account with developer status could download the data using appcfg (in because it only required admin access and in the early days you even had your own handler), and as you said the owner could only download the app. 

Somewhat misleading ';-)

Cheers

Tim

[Adam (Cloud Platform Support)]

Yes, unfortunately the bulkloader itself is not documented anywhere in the official docs although it used to be (this is a good use of the 'Send feedback' link on the 'Managing a Python App' and 'Remote API for Python' pages, which I have already submitted).

As an alternative a developer can still use the Cloud Datastore API to access the application's datastore using a service account.