MASSIVE ApiProxy$CancelledException Exceptions since Friday Night.

[Scott Murphy]

I have been experiencing MASSIVE ApiProxy$CancelledException exceptions since Friday night. 

My instances start up times are also EXTREMELY slow now...

Is anyone else feeling this pain?

com.google.apphosting.api.ApiProxy$CancelledException

 The API call datastore_v3.Put() was explicitly cancelled.

Seems like most of the stack traces contain saveSession in them.

[Chris Ramsdale]

Hey Scott,

Are your instances failing to startup within the 60 sec window allocated by App Engine? If so, that could be the reason for the extremely slow startup times. You effectively get into a pattern where a request triggers a new instance, that instance doesn't startup in time, and App Engine continues until an instance starts up within the 60 sec window and can handle the request.

Would you mind enabling App Stats to get further information? This might also shed some light on the ApiProxy$CancelledException exceptions. Meanwhile, I'll try to track down some folks internally that have run into this issue.

-- Chris

Chris Ramsdale

Product Manager, Google App Engine

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/dNZVZJbF8SEJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Carter]

Our example of this problem (unfortunately on our /payment page).
Appreciate any tips in diagnosing and resolving.

2011-11-26 20:28:45.543
/payment/
com.google.apphosting.api.ApiProxy$CancelledException: The API call
taskqueue.BulkAdd() was explicitly cancelled.
at com.google.apphosting.runtime.ApiProxyImpl$5.get(ApiProxyImpl.java:
332)
at com.google.apphosting.runtime.ApiProxyImpl$5.get(ApiProxyImpl.java:
326)
at
com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:
184)
at com.google.apphosting.runtime.ApiProxyImpl.access
$000(ApiProxyImpl.java:63)
at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:
157)
at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:
155)
at java.security.AccessController.doPrivileged(Native Method)
at
com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:
155)
at
com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:
63)
at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:101)
at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:50)
at
com.google.appengine.api.taskqueue.QueueApiHelper.makeSyncCall(QueueApiHelper.java:
24)
at com.google.appengine.api.taskqueue.QueueImpl.add(QueueImpl.java:
468)
at com.google.appengine.api.taskqueue.QueueImpl.add(QueueImpl.java:
414)
at com.google.appengine.api.taskqueue.QueueImpl.add(QueueImpl.java:
399)
at
com.google.apphosting.runtime.jetty.DeferredDatastoreSessionStore.saveSession(DeferredDatastoreSessionStore.java:
44)
at com.google.apphosting.runtime.jetty.SessionManager
$AppEngineSession.save(SessionManager.java:153)
at
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:
41)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:
43)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
418)
at
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:
249)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
542)
at org.mortbay.jetty.HttpConnection
$RequestHandler.headerComplete(HttpConnection.java:923)
at
com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:
76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:
135)
at com.google.apphosting.runtime.JavaRuntime
$RequestRunnable.run(JavaRuntime.java:392)
at com.google.tracing.TraceContext
$TraceContextRunnable.runInContext(TraceContext.java:449)
at com.google.tracing.TraceContext$TraceContextRunnable
$1.run(TraceContext.java:455)
at com.google.tracing.TraceContext.runInContext(TraceContext.java:
695)
at com.google.tracing.TraceContext
$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:
333)
at com.google.tracing.TraceContext
$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:
325)
at com.google.tracing.TraceContext
$TraceContextRunnable.run(TraceContext.java:453)
at com.google.apphosting.runtime.ThreadGroupPool
$PoolEntry.run(ThreadGroupPool.java:162)
at java.lang.Thread.run(Thread.java:679)

[Rui Oliveira]

Hi Scott

I'm with the same problem. Did you get any solution?

Rui