Getting a 401 Unauthorized while accessing appengine environments using remote api

[Vikram Singh]

I am having some trouble accessing some appengine projects with remote api from my local machine. It fails with a 401 - urllib2.HTTPError: HTTP Error 401: Unauthorized Too many auth attempts.

I am using gcloud version 200.0.0. My credentials have been saved to `~/.config/gcloud/application_default_credentials.json` after I ran `gcloud auth application-default login` and completed the web authentication flow

I am using the following command to start the remote api shell: python /usr/lib/google-cloud-sdk/platform/google_appengine/remote_api_shell.py -s <version>-dot-<my appengine project>.appspot.com

We have around 10 appengine projects. This command works on 8 of them, but consistently fails on 2 of them. As per the IAM & admin page, my email is listed as an owner on one of the two environments where its failing(am thinking that it should at least work on that environment, but its not). I have confirmed that the same code is deployed to all the 10 appengine environments

and the following is also set in app.yaml

builtins:
- remote_api: on

as per https://cloud.google.com/appengine/docs/standard/python/tools/remoteapi

The same thing happens when I try remote access from a gce instance instead of using my local machine. I run `gcloud auth application-default login` on the gce instance, before making any remote api call. Its again fails on the same 2 environments but works on the remaining 8

I am thinking that had it been a configuration issue with the credentials, the remote api command wouldn't have worked at all. But since its working for some appengine projects, it makes me think that it could be a user permission issue in those two projects. But my email is already listed as an owner for one of the projects where its failing(confirmed from the IAM and admin page). I also added my email as an `App Engine admin` over there, but sill the command fails with same error. Not sure what else is needed. Is there anything else that is needed for remote_api_access?

Attaching the stack trace below

Traceback (most recent call last):
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/remote_api_shell.py", line 133, in <module>
    run_file(__file__, globals())
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/remote_api_shell.py", line 129, in run_file
    execfile(_PATHS.script_file(script_name), globals_)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 160, in <module>
    main(sys.argv)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 156, in main
    oauth2=True)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 74, in remote_api_s
hell
    secure=secure, app_id=appid)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 1052, in Co
nfigureRemoteApiForOAuth
    rpc_server_factory=rpc_server_factory)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 1137, in Co
nfigureRemoteApi
    app_id = GetRemoteAppIdFromServer(server, path, rtok)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 842, in Get
RemoteAppIdFromServer
    response = server.Send(path, payload=None, **urlargs)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 259, in Send
    NeedAuth()
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 235, in NeedA
uth
    RaiseHttpError(url, response_info, response, 'Too many auth attempts.')
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 85, in RaiseH
ttpError
    raise urllib2.HTTPError(url, response_info.status, msg, response_info, stream)
urllib2.HTTPError: HTTP Error 401: Unauthorized Too many auth attempts.

[Kamran (Google Cloud Support)]

Hello Vikram,

This seems like a question that would be better posted on Stackoverflow, rather than being a discussion post. This isn't the right place to look for 1-on-1 support, though being a forum meant for general discussion of the platform, and you should take a question like this to stack overflow. Please make sure to include all relevant details that would help somebody debug.

I wish you luck in finding a solution, and have a nice day!

Sincerely, 

[Kamran (Google Cloud Support)]

In case that you feel this might be a bug defect, don't hesitate to open an issue on Google issue tracker and we'll be more than glad to investigate it.

[Vikram Singh]

It seems to be a bug at google's end. Although, I don't have any repro steps to confirm it, and to provide in the public issue tracker. I just know its failing consistently on these two environments. Its as if something on there is preventing remote api access. I have followed all the steps listed here - https://cloud.google.com/appengine/docs/standard/python/tools/remoteapi

Incidentally the environments in which its failing, are one some of the oldest environments we have. Every other environment was created in the past year or two. Maybe there is some incompatiblity between the remoteApiStub from the appengine sdk locally and the appengine/remote access code running on those environments. 

I'll try posting on stack overflow to see if anybody else has experienced something similar lately.