Thread safe client lib on app engine (python)

0 views
Skip to first unread message

Zoltan Molnar via StackOverflow

unread,
Jun 14, 2016, 4:32:06 AM6/14/16
to google-appengin...@googlegroups.com

I found a little sample code about bigquery insert in one of Google's git repositories.

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/appengine/standard/bigquery/main.py

If you see the app.yaml it says this code should be thread safe, but if I'm lokking at the client lib's documentation (https://developers.google.com/api-client-library/python/guide/thread_safety) it should not be thread safe. I'm a little bit confused now, that my following code is thread safe or not? It's running on app engine standard env.

import pprint

from googleapiclient.discovery import build
from oauth2client.client import GoogleCredentials


credentials = GoogleCredentials.get_application_default()

# Create the bigquery api client
service = build('bigquery', 'v2', credentials=credentials)

response = service.datasets().list(projectId='PROJECTID').execute()

pprint.pprint(response)


Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/37806753/thread-safe-client-lib-on-app-engine-python

Tim Hoffman via StackOverflow

unread,
Jun 14, 2016, 6:57:04 AM6/14/16
to google-appengin...@googlegroups.com

If you read the docs you reference it says

The google-api-python-client library is built on top of the httplib2 library, which is not thread-safe. Therefore, if you are running as a multi-threaded application, each thread that you are making requests from must have its own instance of httplib2.Http().

They then go on to show you how to do this. If you follow the instructions then yes it will be.

You sample code is too simple and isn't attempting what was outlined in the docs

# Create a new Http() object for every request
  def build_request(http, *args, **kwargs):
    new_http = httplib2.Http()
    return apiclient.http.HttpRequest(new_http, *args, **kwargs)
  service = build('api_name', 'api_version', requestBuilder=build_request)

  # Pass in a new Http() manually for every request
  service = build('api_name', 'api_version')
  http = httplib2.Http()
  service.stamps().list().execute(http=http)

So if you tried your code in a threaded situation it would not be thread safe. If you are just trying that code from REPL then I doubt you are in a threaded situation.



Please DO NOT REPLY directly to this email but go to StackOverflow:
http://stackoverflow.com/questions/37806753/thread-safe-client-lib-on-app-engine-python/37809988#37809988

Zoltan Molnar via StackOverflow

unread,
Jun 14, 2016, 7:37:04 AM6/14/16
to google-appengin...@googlegroups.com

I found a little sample code about bigquery insert in one of Google's git repositories.

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/appengine/standard/bigquery/main.py

If you see the app.yaml it says this code should be thread safe, but if I'm lokking at the client lib's documentation (https://developers.google.com/api-client-library/python/guide/thread_safety) it should not be thread safe. I'm a little bit confused now, that my following code is thread safe or not? It's running on app engine standard env.

import pprint

from googleapiclient.discovery import build
from oauth2client.client import GoogleCredentials


credentials = GoogleCredentials.get_application_default()

# Create the bigquery api client
service = build('bigquery', 'v2', credentials=credentials)

response = service.datasets().list(projectId='PROJECTID').execute()

pprint.pprint(response)

---- UPDATE ---- After Tim's answer I changed my code to the following. This should be good now: import pprint

from googleapiclient.discovery import build
from oauth2client.contrib.appengine import AppAssertionCredentials
import httplib2


credentials = AppAssertionCredentials(scope='https://www.googleapis.com/auth/bigquery')


# Create the bigquery api client
service = build('bigquery', 'v2')


def get():
    http = credentials.authorize(httplib2.Http())
    response = service.datasets().list(projectId='PROJECTID').execute(http=http)

    pprint.pprint(response)
Reply all
Reply to author
Forward
0 new messages