Memcache failure causing either error 500 or extreme site slowdown

[jon]

My app sporadically generates error 500 due to the following memcache exception:

com.google.apphosting.api.ApiProxy$CancelledException: The API call memcache.Get() was explicitly cancelled.
	at com.google.apphosting.runtime.ApiProxyImpl$5.get(ApiProxyImpl.java:367)
	at com.google.apphosting.runtime.ApiProxyImpl$5.get(ApiProxyImpl.java:364)
	at com.google.appengine.tools.appstats.RecordingFuture.get(RecordingFuture.java:108)
	at com.google.appengine.tools.appstats.Recorder.processRecordingFuture(Recorder.java:425)
	at com.google.appengine.tools.appstats.Recorder.processAsyncRpc(Recorder.java:398)
	at com.google.appengine.tools.appstats.RecordingFuture.maybeRecordStats(RecordingFuture.java:140)
	at com.google.appengine.tools.appstats.RecordingFuture.get(RecordingFuture.java:110)
	at com.google.appengine.tools.appstats.RecordingFuture.get(RecordingFuture.java:20)
	at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:28)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.get(MemcacheServiceImpl.java:51)
	at com.google.apphosting.runtime.jetty.MemcacheSessionStore.getSession(MemcacheSessionStore.java:36)
	at com.google.apphosting.runtime.jetty.SessionManager.loadSession(SessionManager.java:338)
	at com.google.apphosting.runtime.jetty.SessionManager.getSession(SessionManager.java:312)
	at org.mortbay.jetty.servlet.AbstractSessionManager.getHttpSession(AbstractSessionManager.java:237)
	at org.mortbay.jetty.Request.getSession(Request.java:1235)
	at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:216)
	at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:193)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
	at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:327)
	at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126)

I raised a production issue 7 hours ago and I haven't heard from Google. The error's still happening.

Can somebody from Google take a look please? To think that just 2 days ago I publicly recommended Google App Engine on Quora.

[jon]

The relevant production issue:

http://code.google.com/p/googleappengine/issues/detail?id=8226