Can't download data

[portentint]

When I use the command appcfg.py download_data --application=foo --
url=http://foo.appspot.com/_ah/remote_api --filename=/output.data

I get:

[INFO ] Logging to bulkloader-log-20111118.153603
[INFO ] Throttling transfers:
[INFO ] Bandwidth: 250000 bytes/second
[INFO ] HTTP connections: 8/second
[INFO ] Entities inserted/fetched/modified: 20/second
[INFO ] Batch Size: 10
[INFO ] Opening database: bulkloader-progress-20111118.153603.sql3
[INFO ] Opening database: bulkloader-results-20111118.153603.sql3
[INFO ] Connecting to foo.appspot.com/_ah/remote_api
Traceback (most recent call last):
File "/usr/local/bin/appcfg.py", line 77, in <module>
run_file(__file__, globals())
File "/usr/local/bin/appcfg.py", line 73, in run_file
execfile(script_path, globals_)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 3877, in <module>
main(sys.argv)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 3868, in main
result = AppCfgApp(argv).Run()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 2442, in Run
self.action(self)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 3633, in __call__
return method()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 3431, in PerformDownload
run_fn(args)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 3343, in RunBulkloader
sys.exit(bulkloader.Run(arg_dict))
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 4375, in Run
return _PerformBulkload(arg_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 4270, in _PerformBulkload
return_code = app.Run()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 3449, in Run
kinds = self.RunPostAuthentication()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 3618, in
RunPostAuthentication
return self.request_manager.GetSchemaKinds()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 1356, in GetSchemaKinds
global_stat = stats.GlobalStat.all().get()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/ext/db/__init__.py", line 2049, in get
results = self.fetch(1, config=config)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/ext/db/__init__.py", line 2102, in fetch
raw = raw_query.Get(limit, offset, config=config)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/datastore.py", line 1673, in Get
batch = batcher.next_batch(limit)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/datastore/datastore_query.py", line 2562, in
next_batch
batch = self.__next_batch.get_result()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/apiproxy_stub_map.py", line 592, in get_result
return self.__get_result_hook(self)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/datastore/datastore_query.py", line 2317, in
__query_result_hook
self._conn.check_rpc_success(rpc)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/datastore/datastore_rpc.py", line 1184, in
check_rpc_success
raise _ToDatastoreError(err)
google.appengine.api.datastore_errors.BadRequestError: app s~foo
cannot access app foo's data

I've tried a lot of different combinations, adding/removing ~s.

The only hint as to what's wrong (maybe) is that when I point a
browser at foo.appspot.com/_ah/remote_api I get:

"You must be logged in as an administrator to access this."

I've searched for an answer, tried a lot of the stuff in the forums
here, logged out of my apps for domains account to make sure there was
no account confusion, and verified that I'm using the Google Accounts
API.

Help?...

Note: My app isn't called 'foo'

[Brandon Istenes]

Having the same problem. GAE team, where y'all been these last four years?

[Patrice (Cloud Platform Support)]

Hi Brandon,

It gives you the exact same error when properly targeting your app? Do you have an ".appcfg_oauth2_tokens_java" file in your user folder? Sometimes deleting this and retrying causes the system to re-authorize you (or authorize you with the right account if there is currently some account confusion).

This could be worth a try. If not, do you mind either posting the exact command you run and the result you get (or confirm you're running exactly the same thing as was originally reported)

Cheers!

[Patrice (Cloud Platform Support)]

Hi again Brandon,

Just realized while I mentioned the ".appcfg_oauth2_tokens_java" file, you can also do the same with ".appcfg_nag" and ".appcfg_cookies" (rename them if you want to be safe and not delete them)

Cheers!

[Brandon Istenes]

Hi Patrice,

Thanks for the response.

Just deleted .appcfg_cookies (the only of the files you mentioned that exists in my home directory) to no avail.

I'm running

bistenes@socos-qs-5:~$ appcfg.py download_data  --kind="User" --url=https://socos-quickstep-601.appspot.com/_ah/remote_api --filename="out.dat"
07:13 PM Downloading data records.
[INFO    ] Logging to bulkloader-log-20150717.191306

[INFO    ] Throttling transfers:
[INFO    ] Bandwidth: 250000 bytes/second
[INFO    ] HTTP connections: 8/second
[INFO    ] Entities inserted/fetched/modified: 20/second
[INFO    ] Batch Size: 10

Please enter login credentials for socos-quickstep-601.appspot.com
Email: bistenes@socos.me
Password for bistenes@socos.me:
Error 401: --- begin server output ---

You must be logged in as an administrator to access this.

--- end server output ---

It's probably worth noting that I'm also experiencing the bug I reported here, which might be related, I dunno: https://code.google.com/p/googleappengine/issues/detail?id=12157

According to apps.google.com my user bist...@socos.me should have "Super Admin" privileges, whatever that means.

Thanks,

Brandon

[Patrice (Cloud Platform Support)]

Hey Brandon,

Thank you for trying that out. It is pretty weird, but I think this is more related to the issue you filed in the Issue tracker than anything else, as this seems to point to a config error somewhere that doesn't list you as admin of your app even though you are.

"Super Admin" is a term for Google Apps, not fully related to Google App Engine, unless you are working with a custom domain. Is "socos-quickstep-601" setup with a custom domain?

I would ask you "can you use the backup function from the Admin console", but I think that since you cannot go on the console, this is pointless. Do you have anyone else setup as an owner that can try running this?

Cheers!

[Brandon Istenes]

Hi Patrice,

"socos-quickstep-601" is set up with socos.me and socoslearning.com. I imagine socoslearning.com is the "custom domain."

I don't have access to the owner's account and it's unlikely that they'll ever do this. If it's definitely necessary to solving this problem that we try that I can see if it can happen, but I'd really rather not.

Any other ideas? Are you able to address this as a bug with the information I've provided?

Thanks,

Brandon

On Friday, November 18, 2011 at 3:54:31 PM UTC-8, portentint wrote:

[Brandon Istenes]

Just verified that I am an "Owner" of the project.

- Brandon

[Patrice (Cloud Platform Support)]

Hi again Brandon.

So I looked into your domains and I can see that soccos.me is a google Apps domain and that you indeed own it. Where does socoslearning.com comes from though?

In general, you can see the custom domains setup on your application by going to "console.developers.google.com", then going to : "compute -> App Engine -> setings". From there you should have a tab that will be "custom domains".

There you can see which custom domain is set to your app engine application. If socoslearning.com is there, then the problem might stem from there, as can be shown by your Issue Tracker that seems to point to a non google Apps domain being linked on your account, so it's possible that removing this will unblock everything.

Let me know if this works, thanks

Cheers!