Google Storage for Developers with Appengine

[Mark Brenneman]

I have been having a tough time getting appengine to work with google storage for developers. I have poked around gslib, google storage documentation, and a few random blog posts and stackexchange pages but can't seem to get it to work.

I am currently trying to import boto into my project, but am getting a syntax error on a "with" command in the boto/connection.py file. Is boto the recommended way to connect to GS? Is Appengine's version of Python too old for the boto package? Any help would be appreciated.

Best,

Mark

Code:

import os

from google.appengine.dist import use_library

use_library('django', '1.2')

import wsgiref.handlers

import logging

from google.appengine.ext import webapp

from google.appengine.ext.webapp import template

from google.appengine.ext import db

""" 

Google Storage Setup

"""

os.environ['BOTO_CONFIG'] = 'boto.cfg'

import boto

import sys

Error:

Traceback (most recent call last): File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/_webapp25.py", line 701, in __call__ handler.get(*groups) 

 File "/base/data/home/apps/callzapapp/1.352600261032840817/main.py", line 44, in get uri.create_bucket() File "/base/data/home/apps/callzapapp/1.352600261032840817/boto/storage_uri.py", line 316, in create_bucket conn = self.connect() 

 File "/base/data/home/apps/callzapapp/1.352600261032840817/boto/storage_uri.py", line 83, in connect from boto.s3.connection import OrdinaryCallingFormat 

 File "/base/data/home/apps/callzapapp/1.352600261032840817/boto/s3/connection.py", line 28, in <module> from boto.connection import AWSAuthConnection File "/base/data/home/apps/callzapapp/1.352600261032840817/boto/connection.py", line 237 with self.mutex: ^

SyntaxError: invalid syntax

[Tim Hoffman]

Hi

"with" statement is actually a feature of later releases of python.

However in 2.5 you can get access to it by doing the following import

from __future__ import with_statement

Rgds

Tim

[Mark Brenneman]

Thanks Tim, that worked, but now I ran into a new problem.

Now, I get a connection error. I have tried finding others with the issues but have not had any luck. Is this a problem with the boto package or does it need to be modified to use with AppEngine?

Upload Code:

fp = tempfile.TemporaryFile()

fp.write('Hello world!\n')

dst_uri = boto.storage_uri(BUCKET_NAME + '/' + DOWNLOAD_FILENAME, 'gs')

# The key-related functions are a consequence of boto's

# interoperability with Amazon S3 (which employs the

# concept of a key mapping to contents).

dst_uri.new_key().set_contents_from_file(fp)

logging.debug('Successfully created "%s/%s"' % (dst_uri.bucket_name, dst_uri.object_name))

Error:

Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/_webapp25.py", line 701, in __call__
    handler.get(*groups)
  File "/base/data/home/apps/callzapapp/1.352621855547207562/main.py", line 54, in get
    dst_uri.new_key().set_contents_from_file(fp)
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/gs/key.py", line 191, in set_contents_from_file
    self.send_file(fp, headers, cb, num_cb)
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/s3/key.py", line 566, in send_file
    query_args=query_args)
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/s3/connection.py", line 423, in make_request
    override_num_retries=override_num_retries)
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/connection.py", line 782, in make_request
    return self._mexe(http_request, sender, override_num_retries)
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/connection.py", line 695, in _mexe
    connection = self.get_http_connection(request.host, self.is_secure)
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/connection.py", line 566, in get_http_connection
    conn = self._pool.get_http_connection(host, is_secure)
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/connection.py", line 241, in get_http_connection
    return self.host_to_pool[key].get()
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/connection.py", line 148, in get
    if self._conn_ready(conn):
  File "/base/data/home/apps/callzapapp/1.352621855547207562/boto/connection.py", line 167, in _conn_ready
    response = conn._HTTPConnection__response
AttributeError: HTTPSConnection instance has no attribute '_HTTPConnection__response'

[tabmarcio]

I have the same problem. BTW this also happens in production with GAE
current python.

m

[tabmarcio]

THe problem I have is the second problem you reported: AttributeError:

HTTPSConnection instance has no attribute
'_HTTPConnection__response'

m

[Hans Then]

Hi,

I have the same problem and not a clue what's wrong. Did you manage to
solve this?

Hans

> > >   File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/_we bapp25.py", line 701, in __call__

[Robert Schuppenies]

Could you try https://groups.google.com/group/gs-discussion? The Google Storage team hangs out over there and they also know about the App Engine integration.

cheers,

robert

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.