1.1-SNAPSHOT broken in Google App Engine

[Mark Tye]

First, I want to say that I'm really impressed with Lift. It's a
pleasure to work with (now that I've figured it out) and it keeps
getting better.

I'm working on a Lift project that runs on Google App Engine. After an
extended period of development on a local dev server, I uploaded my
app and was surprised to discover it didn't work in the App Engine
environment anymore.

In the logs, I discovered that every request resulted in a HTTP 500
error. The root cause appeared to be a ClassNotFoundException thrown
by com.google.apphosting.runtime.security.UserClassLoader while
deserializing a net.liftweb.http.SessionToServletBridge. (Or possibly
some object that contains a SessionToServletBridge?)

I found an entry in the Google App Engine issue tracker noting
problems with the deserialization of certain objects from a Session:

http://code.google.com/p/googleappengine/issues/detail?id=1290

Some of the comments suggest that these objects are running afoul of
App Engine's JRE White List:

http://code.google.com/appengine/docs/java/jrewhitelist.html

But I looked at the source for SessionToServletBridge and did not see
any forbidden classes in the imports, so I'm not sure if that's the
problem in this case.

I was able get my app working again by reverting from the snapshot to
version 1.1-M4, so presumably, the change that's causing the problem
was introduced after the M4 milestone.

I don't know if compatibility with App Engine is a priority for the
Lift team, but if this doesn't get fixed, I face an ugly choice
between remaining frozen at 1.1-M4 as Lift development moves forward,
or migrating away from App Engine to another cloud solution. Is there
anything I can do to assist the diagnosis/resolution of this issue?

Thanks,
Mark

Here's the stack trace:

javax.servlet.ServletException: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
net.liftweb.http.SessionToServletBridge
at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle
(AppVersionHandlerMap.java:239)
at org.mortbay.jetty.handler.HandlerWrapper.handle
(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
506)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete
(HttpConnection.java:830)
at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable
(RpcRequestParser.java:76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
at
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest
(JettyServletEngineAdapter.java:139)
at com.google.apphosting.runtime.JavaRuntime.handleRequest
(JavaRuntime.java:235)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime
$6.handleBlockingRequest(RuntimePb.java:4823)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime
$6.handleBlockingRequest(RuntimePb.java:4821)
at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest
(BlockingApplicationHandler.java:24)
at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:
359)
at com.google.net.rpc.impl.Server$2.run(Server.java:820)
at com.google.tracing.LocalTraceSpanRunnable.run
(LocalTraceSpanRunnable.java:56)
at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan
(LocalTraceSpanBuilder.java:516)
at com.google.net.rpc.impl.Server.startRpc(Server.java:775)
at com.google.net.rpc.impl.Server.processRequest(Server.java:348)
at com.google.net.rpc.impl.ServerConnection.messageReceived
(ServerConnection.java:436)
at com.google.net.rpc.impl.RpcConnection.parseMessages
(RpcConnection.java:319)
at com.google.net.rpc.impl.RpcConnection.dataReceived
(RpcConnection.java:290)
at com.google.net.async.Connection.handleReadEvent(Connection.java:
428)
at com.google.net.async.EventDispatcher.processNetworkEvents
(EventDispatcher.java:762)
at com.google.net.async.EventDispatcher.internalLoop
(EventDispatcher.java:207)
at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:
101)
at com.google.net.rpc.RpcService.runUntilServerShutdown
(RpcService.java:251)
at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run
(JavaRuntime.java:374)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
net.liftweb.http.SessionToServletBridge
at com.google.apphosting.runtime.jetty.SessionManager.deserialize
(SessionManager.java:362)
at com.google.apphosting.runtime.jetty.SessionManager.loadSession
(SessionManager.java:280)
at com.google.apphosting.runtime.jetty.SessionManager.getSession
(SessionManager.java:255)
at org.mortbay.jetty.servlet.AbstractSessionManager.getHttpSession
(AbstractSessionManager.java:237)
at org.mortbay.jetty.Request.getSession(Request.java:998)
at org.mortbay.jetty.servlet.SessionHandler.handle
(SessionHandler.java:192)
at org.mortbay.jetty.handler.ContextHandler.handle
(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
405)
at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle
(AppVersionHandlerMap.java:237)
... 27 more
Caused by: java.lang.ClassNotFoundException:
net.liftweb.http.SessionToServletBridge
at com.google.apphosting.runtime.security.UserClassLoader$1.run
(UserClassLoader.java:352)
at com.google.apphosting.runtime.security.UserClassLoader$1.run
(UserClassLoader.java:347)
at java.security.AccessController.doPrivileged(Native Method)
at com.google.apphosting.runtime.security.UserClassLoader.findClass
(UserClassLoader.java:347)
at java.lang.ClassLoader.loadClass(Unknown Source)
at
com.google.apphosting.runtime.security.UserClassLoader.loadClassImpl
(UserClassLoader.java:434)
at com.google.apphosting.runtime.security.UserClassLoader.loadClass
(UserClassLoader.java:380)
at java.lang.ClassLoader.loadClass(Unknown Source)
at com.google.apphosting.runtime.jetty.SessionManager
$DelegatingObjectInputStream.resolveClass(SessionManager.java:415)
at java.io.ObjectInputStream.readNonProxyDesc(Unknown Source)
at java.io.ObjectInputStream.readClassDesc(Unknown Source)
at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.readObject(Unknown Source)
at java.util.HashMap.readObject(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
at java.io.ObjectInputStream.readSerialData(Unknown Source)
at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
at java.io.ObjectInputStream.readSerialData(Unknown Source)
at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.readObject(Unknown Source)
at com.google.apphosting.runtime.jetty.SessionManager.deserialize
(SessionManager.java:358)
... 35 more

[Atsuhiko Yamanaka]

Hi,

On Wed, Aug 26, 2009 at 6:35 AM, David
Pollak<feeder.of...@gmail.com> wrote:
> Atsuhiko-san,
>
> Have you seen this note on the Lift list?

I have not seen such a note on this list yet and I had not tried
1.1-SNAPSHOT on
GAE/J in these days, but today I tried it and have also encountered
the reported problem.
I'll check the code and think of what we can.

Sincerely,
--
Atsuhiko Yamanaka
JCraft,Inc.
1-14-20 HONCHO AOBA-KU,
SENDAI, MIYAGI 980-0014 Japan.
Tel +81-22-723-2150
+1-415-578-3454
Skype callto://jcraft/

[Atsuhiko Yamanaka]

Hi,

On Mon, Aug 17, 2009 at 9:18 AM, Mark Tye<mar...@gmail.com> wrote:
...

> I don't know if compatibility with App Engine is a priority for the
> Lift team, but if this doesn't get fixed, I face an ugly choice
> between remaining frozen at 1.1-M4 as Lift development moves forward,
> or migrating away from App Engine to another cloud solution. Is there
> anything I can do to assist the diagnosis/resolution of this issue?

I have found that this problem will occur if you run lift app based on 1.1-M4
and then run lift app based on 1.1-SNAPSHOT.
And then, if you clear a cookie entry for http://yourapp.appsopt.com/
in your browser,
that problem will disappear.

I think there is not a solution in the 1.1-SNAPSHOT code.

How about clearing sessions before updating to 1.1-SNAPSHOT code base?
You know that sessions have been kept in the datastore and memcache,
and following code will clear them.
I have confirmed that I could cure that problem by following code
without deleting
a cookie in my browser.

// clearing sessions in the datastore.
import com.google.appengine.api.datastore.{Query,
DatastoreServiceFactory, Entity}
implicit def foo(i:java.lang.Iterable[Entity]) = new Iterator[Entity]{
val itt = i.iterator
def hasNext:Boolean = itt.hasNext
def next:Entity = itt.next
}
val datastore = DatastoreServiceFactory.getDatastoreService
val results = datastore.prepare(new Query("_ah_SESSION"))
for(session <- results.asIterable){
datastore.delete(session.getKey())
}

// clearing memcache
import java.util.Collections
import javax.cache._
val cacheFactory = CacheManager.getInstance.getCacheFactory
val cache = cacheFactory.createCache(Collections.emptyMap())
val stats = cache.getCacheStatistics
cache.clear

[Mark Tye]

Thanks for taking a look at this issue. If the solution is as simple
as clearing/removing a cookie, that's great news. I'll give it a try
and let you know how it worked. Thanks again!

- Mark

On Aug 26, 2:57 am, Atsuhiko Yamanaka <atsuhiko.yaman...@gmail.com>
wrote:
> Hi,

>
> On Mon, Aug 17, 2009 at 9:18 AM, Mark Tye<mark...@gmail.com> wrote:
>
> ...
>
> > I don't know if compatibility with App Engine is a priority for the
> > Lift team, but if this doesn't get fixed, I face an ugly choice
> > between remaining frozen at 1.1-M4 as Lift development moves forward,
> > or migrating away from App Engine to another cloud solution. Is there
> > anything I can do to assist the diagnosis/resolution of this issue?
>
> I have found that this problem will occur if you run lift app based on 1.1-M4
> and then run lift app based on 1.1-SNAPSHOT.

> And then, if you clear a cookie entry forhttp://yourapp.appsopt.com/

[Mark Tye]

I was able to confirm that clearing the cookie gets rid of the
ClassNotFoundException.

I have a hypothesis as to what happened: It appears that sometime
after the M4 milestone, the SessionToServletBridge moved from the
net.liftweb.http package to net.liftweb.http.provider.servlet. So, the
M4 version of my application serialized a
net.liftweb.http.SessionToServletBridge to a cookie, but the 1.1-
SNAPSHOT version couldn't find a class definition for the object when
deserializing the cookie and threw a ClassNotFoundException.

Anyway, problem resolved! Atsuhiko, thank you very much for your help!
David, thanks for bring the issue to Atsuhiko's attention!

- Mark