many "Process terminated because the request deadline was exceeded. . (Error code 123)" errors for my go app recently

[Tapir]

I can't get what is problem from the log:

E  GET 500 0 B 100.9 s Googlebot 2 / GET 500 0 B 100.9 s
Googlebot 2 590210c200ff08698603ab0f540001737e74617069722d67616d6573000132000100
  66.249.75.176 - - [27/Apr/2017:23:39:46 +0800] "GET / HTTP/1.1" 500 - -
  "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
  "www.tapirgames.com" ms=100878 cpu_ms=0 cpm_usd=0 loading_request=0 instance=00c61b117c1539d61ab092aaf414fa9a3546b76458a415df22126384274e5d0a59a890fafe6c0872 app_engine_release=1.9.48 trace_id=7fb8718bb041a287d39cdbcd4b720651

 {
 protoPayload: {
  @type: "type.googleapis.com/google.appengine.logging.v1.RequestLog"  
  appId: "s~tapir-games"  
  versionId: "2"  
  requestId: "590210c200ff08698603ab0f540001737e74617069722d67616d6573000132000100"  
  ip: "66.249.75.176"  
  startTime: "2017-04-27T15:39:46.551302Z"  
  endTime: "2017-04-27T15:41:27.430060Z"  
  latency: "100.878758s"  
  method: "GET"  
  resource: "/"  
  httpVersion: "HTTP/1.1"  
  status: 500  
  userAgent: "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"  
  urlMapEntry: "_go_app"  
  host: "www.tapirgames.com"  
  instanceIndex: -1  
  finished: true  
  instanceId: "00c61b117c1539d61ab092aaf414fa9a3546b76458a415df22126384274e5d0a59a890fafe6c0872"  
  line: [
   0: {
    time: "2017-04-27T15:41:27.429917Z"    
    severity: "ERROR"    
    logMessage: "Process terminated because the request deadline was exceeded. (Error code 123)"    
   }
  ]
  appEngineRelease: "1.9.48"  
  traceId: "7fb8718bb041a287d39cdbcd4b720651"  
 }
 insertId: "59021127000691e464bf51ad" 
 httpRequest: {
  status: 500  
 }
 resource: {
  type: "gae_app"  
  labels: {
   zone: "us2"   
   project_id: "tapir-games"   
   version_id: "2"   
   module_id: "default"   
  }
 }
 timestamp: "2017-04-27T15:39:46.551302Z" 
 severity: "ERROR" 
 labels: {
  clone_id: "00c61b117c1539d61ab092aaf414fa9a3546b76458a415df22126384274e5d0a59a890fafe6c0872"  
 }
 logName: "projects/tapir-games/logs/appengine.googleapis.com%2Frequest_log" 
 operation: {
  id: "590210c200ff08698603ab0f540001737e74617069722d67616d6573000132000100"  
  producer: "appengine.googleapis.com/request_id"  
  last: true  
 }
}

23:41:27.429
Process terminated because the request deadline was exceeded. (Error code 123)

Can someone provide me some hints?

[Tapir]

The problem started happening since April 24th.
It never happened before this date.
The app is very simple and I didn't modify it much a week  before and after April 24th.

[Tapir]

btw, the cloud log in firefox is almost unusable.
It is laggy and often refreshes unexpectedly.

[Tapir]

Aha, I think I found the issue.
It is a bug in code.
The code is modified two week ago.

But it is weird that only the oldest log at April 24th prints the program panic stack traces.
Other logs don't print the stack traces.

On Sunday, April 30, 2017 at 9:55:18 PM UTC+8, Tapir wrote:

[Adam (Cloud Platform Support)]

Good to hear you found the problem. Based on the information given, the only thing a reader would be able to deduce is that your request was taking too long for some reason and went over the 60 second deadline.

If you're seeing issue with the logs viewer I'd recommend filing a bug on the issue tracker and provide a screencast of the behavior as well as a HAR capture of the session.