the datastore download problem.

[陈友和]

Hi guys, I'm stuck in a problem which drive me crazy. As the url(https://developers.google.com/appengine/docs/python/tools/uploadingdata?hl=zh-CN&csw=1#Configuring_the_Bulk_Loader) introducing the way to download the data from datastore of my app on google app engine, I try this command:

appcfg.py download_data --application=news-leon --kind=csv --url=http://news-leon.appspot.com/remote_api --filename=test.csv

but I always get the these information:

--------------------------

02:49 PM Downloading data records.

[INFO    ] Logging to bulkloader-log-20131113.144907

[INFO    ] Throttling transfers:

[INFO    ] Bandwidth: 250000 bytes/second

[INFO    ] HTTP connections: 8/second

[INFO    ] Entities inserted/fetched/modified: 20/second

[INFO    ] Batch Size: 10

[INFO    ] Opening database: bulkloader-progress-20131113.144907.sql3

[INFO    ] Opening database: bulkloader-results-20131113.144907.sql3

[INFO    ] Connecting to news-leon.appspot.com/remote_api

[ERROR   ] Exception during authentication

Traceback (most recent call last):

  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 3469, in Run

    self.request_manager.Authenticate()

  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 1332, in Authenticate

    remote_api_stub.MaybeInvokeAuthentication()

  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 740, in MaybeInvokeAuthentication

    datastore_stub._server.Send(datastore_stub._path, payload=None)

  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appengine_rpc.py", line 409, in Send

    f = self.opener.open(req)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 394, in open

    response = self._open(req, data)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 412, in _open

    '_open', req)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 372, in _call_chain

    result = func(*args)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1199, in http_open

    return self.do_open(httplib.HTTPConnection, req)

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1174, in do_open

    raise URLError(err)

URLError: <urlopen error [Errno 60] Operation timed out>

[ERROR   ] Authentication Failed: Incorrect credentials or unsupported authentication type (e.g. OpenId).

------------------------------------------------------

I also try:

appcfg.py download_data --application=news-leon --auth_domain=gmail.com --kind=csv --url=https://news-leon.appspot.com/remote_api --filename=test.csv

but still I get the same error. But it is smoothly when I use appcfg.py to update the project.

So could anybody help me?? 

[Lapteuh]

Hi,

First, 'remote_api' does not work with OpenId provider authorization.

Second, there is a mark in the documentation, that HRD will not work this way, keep in mind. You must use 'remote_api' directly from the command line.

Good luck!

среда, 13 ноября 2013 г., 13:05:53 UTC+6 пользователь 陈友和 написал:

[Benjamin Possolo]

have you seen this?

https://groups.google.com/forum/#!topic/google-appengine/1KXscIvH0Lw

[Alan Anderson]

I am having the exact same problem with appcfg.py download_data, and I also have it with appcfg.py create_bulkloader_config. I've tried everything I can think of and some things suggested by others, but I keep running into a wall on this.I am using the HR datastore, The doc "Uploading and Downloading Data" has the following warning.

Warning: This document applies to apps that use the master/slave datastore. If your app uses the High Replication datastore, it is possible to copy data from the app, but Google does not currently support this use case. If you attempt to copy from a High Replication datastore, you'll see a high_replication_warning error in the Admin Console, and the downloaded data might not include recently saved entities.

So it says "it is possible to copy data from the app" and warns "the downloaded data might not include recently saved entities." I can deal with the latter, but how do you get past this authentication problem. Surely they would not take away this vital download_data functionality for a HR datastore...or would they?

[Ronen Azachi]

you should enable remote_api in your app.yaml file

builtins:
- remote_api: on #/_ah/remote_api/

[Andri Firstianto-GCP]

suppose app.yaml for python app engine,

how to set remote api : on for java app engine?

[Nick (Cloud Platform Support)]

Hey Andri,

If you have a question or issue, you should either post it to Stack Overflow with relevant information (if you think that the error is in your lack of understanding somehow), the Public Issue Tracker (if you think you've found an issue on the platform and can post a reproducing example or information necessary to reproduce the behaviour), or in a new thread in this forum (if you have generalized questions which will aid good discussion, not necessarily technical problem reports). Adding a small comment to an old thread gives very little possibility to determine what might be specifically happening to your system.

Best wishes,

Nick