Memcache errors break session-saving. Should fail silently IMO. Any workaround?

[Per]

When using sessions, these are (also) stored in memcache  by GAE. Unfortunately, it seems like any memcache hiccup can crash the process, turning an otherwise fine page into a 500 page on the way out.

To me that looks like a design flaw. After all, Memcache is optional, and its expected to be unavailable or breaking every now and then. As of 1.6 you can even provide a policy to silently ignore memcache errors inside your application. We're using setErrorHandler(ErrorHandlers.getConsistentLogAndContinue(Level.SEVERE)) with Objectify just fine, but it doesn't seem to have any effect on the Session-Save process. This seems to be entirely outside our control.

These errors are really annoying some of our clients. Is there maybe some kind of other error handler I'm missing?

Here's the stracktrace for reference:

com.google.appengine.api.memcache.MemcacheServiceException: Memcache put: Error setting single item (_ahsOH7nik8bCRTbIXblKurKQQ)
	at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$7.transform(AsyncMemcacheServiceImpl.java:426)
	at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$7.transform(AsyncMemcacheServiceImpl.java:418)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$RpcResponseHandler.convertResponse(MemcacheServiceApiHelper.java:60)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.wrap(MemcacheServiceApiHelper.java:112)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.wrap(MemcacheServiceApiHelper.java:105)
	at com.google.appengine.api.utils.FutureWrapper.wrapAndCache(FutureWrapper.java:57)
	at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:98)
	at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:28)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.put(MemcacheServiceImpl.java:81)
	at com.google.apphosting.runtime.jetty.MemcacheSessionStore.saveSession(MemcacheSessionStore.java:39)
	at com.google.apphosting.runtime.jetty.SessionManager$AppEngineSession.save(SessionManager.java:164)
	at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:41)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
	at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
	at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:249)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:326)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
	at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
	at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135)
	at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:477)
	at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:449)
	at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455)
	at com.google.tracing.TraceContext.runInContext(TraceContext.java:695)
	at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333)
	at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325)
	at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453)
	at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
	at java.lang.Thread.run(Thread.java:679)

And here's the code I tried, but which didn't help:

        MemcacheService service = MemcacheServiceFactory.getMemcacheService();
        service.setErrorHandler(new LogAndContinueErrorHandler(Level.SEVERE));

Any help would be appreciated!

Kind regards,
Per

[Joakim]

If silent failure on write is to be acceptable, we need a way to make sure following requests do not read session from memcache, as it could return an old value. I am unaware of any way to accomplish this reliably without hitting the datastore, at which point you might as well not have the session in memcache at all.

One possible alternative is to replace the entire session with storing encrypted data in a cookie, though this only works for smaller amounts of data as the maximum size for an entire cookie is generally said to be 4095 bytes.

Just my 2 cents.
Joakim

[Per]

Hi Joakim,

replacing the session with a cookie is unfortunately not an option for us, our web framework stores a lot of stuff in the session.

I don't even want to use memcache at all, the session ends up in the datastore anyway, so if I had a choice, I would disable memcache here. It's just built that way in GAE.  I'd do anything to stop our users from  getting 500-errors, and I'd like a way to ignore memcache errors here.

I read that the sessions get stored into memcache namespace "_ahs", so I tried to register a LogAndContinue-Errorhandler for that namespace, but unfortunately that doesn't work either, at least not this way:

        MemcacheServiceFactory.getMemcacheService("_ahs").setErrorHandler(new LogAndContinueErrorHandler(Level.INFO) {
            @Override
            public void handleDeserializationError(InvalidValueException thrown) {
                log.error("gotcha!",thrown);
                super.handleDeserializationError(thrown);
            }

            @Override
            public void handleServiceError(MemcacheServiceException thrown) {
                log.error("gotcha!",thrown);
                super.handleServiceError(thrown);
            }
        });


Any idea how else to register a handler to prevent this from killing our requests?

Cheers,
Per