I am getting Internal Server Error 500, which fixes for a while if i deploy it again

5,385 views
Skip to first unread message

Ridhwanul Haque

unread,
Jul 17, 2016, 12:04:08 AM7/17/16
to Google App Engine
I was making this app where it gets data from Google Spreadsheet 
This is the code I used to get data from my Spreadsheet : 

json_key = json.load(open('service.json'))
scope = ['https://spreadsheets.google.com/feeds']
credentials = SignedJwtAssertionCredentials(json_key['client_email'], json_key['private_key'], scope)
self.gc = gspread.authorize(credentials)
self.sheet = self.gc.open('Order Database').sheet1
self.menuSheet = self.gc.open('Menu and Stocking').sheet1
return self.menuSheet.get_all_values()

At the beginning the code was working fine, I was using OAuth 2.0 client IDs.
Later when I was working with my android app, to get the data from the website on that url, I started getting Internal Server 500 error, and I can access to that url even from chrome where other part of my website works fine. While searching for solution, I found some people saying to use a API Keys, so I created a server key and even used that, this is the android  code : 

HttpClient client = new DefaultHttpClient();
HttpGet request =new HttpGet(url);
request.addHeader("x-api-key",apikey);
HttpResponse response = client.execute(request);
BufferReader rd = new BufferReader(new InputStreamReader(response.getEntity().getContent()));
String line = rd.readLine();

The code works fine, at least thats what I think. Every time I get the Internal Server Error 500, if I deploy it again, it starts working fine, but only for a while, soon it starts showing the same error. Looking through the Dashboard of my project I found some detail of the error so I will also include that here : 

HTTPError: 401: <HTML> <HEAD> <TITLE>Unauthorized</TITLE> </HEAD> <BODY BGCOLOR="#FFFFFF" TEXT="#000000"> <H1>Unauthorized</H1> <H2>Error 401</H2> </BODY> </HTML>
    1. at request (/base/data/home/apps/s~gowcanteen/1.394247265273324301/lib/gspread/httpsession.py:75)
    2. at get (/base/data/home/apps/s~gowcanteen/1.394247265273324301/lib/gspread/httpsession.py:79)
    3. at get_cells_feed (/base/data/home/apps/s~gowcanteen/1.394247265273324301/lib/gspread/client.py:250)
    1. at getMenuItems (/base/data/home/apps/s~gowcanteen/1.394247265273324301/server.py:24)
    2. at API_MENU (/base/data/home/apps/s~gowcanteen/1.394247265273324301/main.py:61)
    3. at dispatch_request (/base/data/home/apps/s~gowcanteen/1.394247265273324301/lib/flask/app.py:1461)
    4. at full_dispatch_request (/base/data/home/apps/s~gowcanteen/1.394247265273324301/lib/flask/app.py:1475)
    5. at handle_user_exception (/base/data/home/apps/s~gowcanteen/1.394247265273324301/lib/flask/app.py:1381)
    6. at full_dispatch_request (/base/data/home/apps/s~gowcanteen/1.394247265273324301/lib/flask/app.py:1477)
    7. at wsgi_app (/base/data/home/apps/s~gowcanteen/1.394247265273324301/lib/flask/app.py:1817)

I am new to developing and have very little knowledge about OAuths and Credentials, so there might be issues that I do not know and thats why unable to find the problem and fix it. I have added everything that I could think of that could have been related to this issue, so please I ask for your help.  

Adam (Cloud Platform Support)

unread,
Jul 18, 2016, 1:28:56 PM7/18/16
to Google App Engine
API Keys will not work with the Google Sheets API. You need to use OAuth, as you were doing originally.

The reason you only see 500 Internal Server Errors client-side is because every time some exception occurs, you aren't handling it and just letting your code crash on the back end. A 500 does not mean "try something completely different", it means "go look at the logs to see exactly what exception was thrown, and fix the relevant part of your code".

So the next step would be: Go back to using OAuth 2.0 with Client IDs. When you see 500 errors, don't abandon your approach, but check the request logs in the console (Logging -> Logs) to see what went wrong, and go from there. Feel free to post the stack trace details if you need help interpreting them.
Message has been deleted
Message has been deleted
Message has been deleted
Message has been deleted

Ridhwanul Haque

unread,
Jul 18, 2016, 2:56:26 PM7/18/16
to Google App Engine
I am still using OAuth ID in web app, the python code that I am using to get the data from Google Spreadsheet. API Key was something that I used in my android app, to just try out if it does any change, and that didnt. 
And here is the Logs

00:12:15.755GET500264 B229 msChrome 51/api/menu

59.153.101.143 - - [19/Jul/2016:00:27:15 +0600] "GET /api/menu HTTP/1.1" 500 264 - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36" "gowcanteen.appspot.com" ms=229 cpu_ms=9 cpm_usd=7.9236e-8 loading_request=0 instance=00c61b117c5074171089efe691fd68d35c03ec166588034655bca1bfd8a004a271 app_engine_release=1.9.40 trace_id=c96131dd18bfefe2658ed7980e9c28e9


{
 metadata: {…}  
  protoPayload: {…}  
  insertId: "2016-07-18|11:27:19.295269-07|10.106.46.132|-1425448532"  
  httpRequest: {…}  
  operation: {…}  
}

Exception on /api/menu [GET]
Traceback (most recent call last):
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/main.py", line 59, in API_MENU
    return str(db.getMenuItems())
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/server.py", line 24, in getMenuItems
    return self.menuSheet.get_all_values()
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/models.py", line 339, in get_all_values
    cells = self._fetch_cells()
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/models.py", line 226, in _fetch_cells
    feed = self.client.get_cells_feed(self)
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/client.py", line 250, in get_cells_feed
    r = self.session.get(url)
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/httpsession.py", line 79, in get
    return self.request('GET', url, **kwargs)
  File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/httpsession.py", line 75, in request
    raise HTTPError("%s: %s" % (response.status, response.read()))

HTTPError: 401: <HTML>
<HEAD>
<TITLE>Unauthorized</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Unauthorized</H1>
<H2>Error 401</H2>
</BODY>
</HTML>
 Loading... Scanned up to 2016-07-19 (00:12:15.755)






Message has been deleted

Ridhwanul Haque

unread,
Jul 18, 2016, 2:59:40 PM7/18/16
to Google App Engine
Continuation
Exception on /api/menu [GET] Traceback (most recent call last): File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/flask/app.py", line 1461, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/main.py", line 59, in API_MENU return str(db.getMenuItems()) File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/server.py", line 24, in getMenuItems return self.menuSheet.get_all_values() File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/models.py", line 339, in get_all_values cells = self._fetch_cells() File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/models.py", line 226, in _fetch_cells feed = self.client.get_cells_feed(self) File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/client.py", line 250, in get_cells_feed r = self.session.get(url) File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/httpsession.py", line 79, in get return self.request('GET', url, **kwargs) File "/base/data/home/apps/s~gowcanteen/1.394294787305794312/lib/gspread/httpsession.py", line 75, in request raise HTTPError("%s: %s" % (response.status, response.read())) HTTPError: 401: <HTML> <HEAD> <TITLE>Unauthorized</TITLE> </HEAD> <BODY BGCOLOR="#FFFFFF" TEXT="#000000"> <H1>Unauthorized</H1> <H2>Error 401</H2> </BODY> </HTML>

Ridhwanul Haque

unread,
Jul 18, 2016, 3:01:35 PM7/18/16
to Google App Engine
I reposted the Exception part with a bit better presentation. 

Adam (Cloud Platform Support)

unread,
Jul 22, 2016, 3:06:01 PM7/22/16
to Google App Engine
The error is pretty clear, you're getting 401 unauthorized from your web app. So either your credentials are not good, or the way you are authorizing is incorrect.

Since you're using gspread (a third party library) you might want to revisit their documentation for Using OAuth2 for Authorization. Be sure you are following the steps for your version of oauth2client; it's recommended to use >= 2.0.0.

Josh Moore

unread,
Jul 22, 2016, 3:18:26 PM7/22/16
to google-a...@googlegroups.com
Also, if this happens "after a while", it's possible that the access token is expiring and needs to be refreshed for a new, valid access token.  Access tokens expiration is variable, but is always "very quickly," so you should anticipate these failures and be able to refresh the access token as needed.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
 To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/9ad3f1e8-3829-4786-938d-f8021b39695b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

Josh Moore | Cloud Solutions Engineer | josh...@google.com | 

Ridhwanul Haque

unread,
Jul 23, 2016, 1:00:01 AM7/23/16
to Google App Engine
Sorry, it seems I was using an older method, I have now used the method for oauth 2.0, lets see if it keeps it working. and i did recently (like 3 days ago, but in few hours got the error again) change my access token, but do I have to renew it every day? well i do hope changing to the correct method will help solve the issue. thanks for the help everyone, I will check for some time, if I face the problem again, I will come for help again. 

Ridhwanul Haque

unread,
Jul 23, 2016, 3:57:50 AM7/23/16
to Google App Engine
Looks like i still have the issue, so just confirmation @joshuamo and @Adam I wanna ask by access token you mean the service account key right? or do I have to create new Oauth2 client id (web client) as well. 

Adam (Cloud Platform Support)

unread,
Jul 24, 2016, 6:55:28 PM7/24/16
to Google App Engine
No, the access token is not the service account key. You do not need to keep regenerating the service account key; if you did, it would effectively cause your app to break every few days until you created a new key and pushed a new version of your app.

The access token refresh is generally handled by the client library you are using. Posting your updated code and steps you've taken will help greatly to determine if the problem is with your code or if gspread isn't handling the token refresh properly.
Reply all
Reply to author
Forward
0 new messages