payara 4 vs payara 5 / class not found exception and deserialize exception

[Korbinian]

Hi,

I wanted to test out payara 5 and somehow stumbled upon a problem I yet dont get. Our application uses modeshape as JCR library and works without any problems on payara 4 for over a year now.

However, upon starting payara 5 we get deserialize and class not found exceptions. Interestingwise this only happens when the application is trying to load an already existing repo saved under a certain path, if it is creating it new it works the first time. After deploy and reploy it will fail with the same error.

The error is in length:

[2018-08-08T11:43:59.117+0200] [Payara 5.182] [SCHWERWIEGEND] [] [javax.enterprise.web] [tid: _ThreadID=45 _ThreadName=admin-thread-pool::admin-listener(1)] [timeMillis: 1533721439117] [levelValue: 1000] [[

  WebModule[]Exception starting filter FILTERNAME

java.lang.InstantiationException

at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:127)

at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:5093)

at org.apache.catalina.core.StandardContext.start(StandardContext.java:5736)

...

at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)

at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)

at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)

at java.lang.Thread.run(Thread.java:748)

Caused by: javax.servlet.ServletException: org.brixcms.jcr.CannotOpenJcrSessionException: Could not open jcr session for workspace: null

at org.apache.wicket.protocol.http.WicketFilter.init(WicketFilter.java:457)

at org.apache.wicket.protocol.http.WicketFilter.init(WicketFilter.java:355)

at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:267)

at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:123)

... 50 more

Caused by: org.brixcms.jcr.CannotOpenJcrSessionException: Could not open jcr session for workspace: null

at org.brixcms.jcr.AbstractThreadLocalSessionFactory.createSession(AbstractThreadLocalSessionFactory.java:58)

at org.brixcms.jcr.Jcr2WorkspaceManager.createSession(Jcr2WorkspaceManager.java:68)

at org.brixcms.jcr.Jcr2WorkspaceManager.getAccessibleWorkspaceNames(Jcr2WorkspaceManager.java:76)

at org.brixcms.workspace.AbstractSimpleWorkspaceManager.initialize(AbstractSimpleWorkspaceManager.java:171)

at de.whiskyworld.shop.web.AbstractWicketApplication.init(AbstractWicketApplication.java:88)

at de.whiskyworld.shop.web.WicketApplication.init(WicketApplication.java:91)

at org.apache.wicket.Application.initApplication(Application.java:782)

at org.apache.wicket.protocol.http.WicketFilter.init(WicketFilter.java:431)

... 53 more

Caused by: javax.jcr.RepositoryException: Error while starting 'repo' repository: Could not deserialize [-84, -19, 0, 5, 115, 114, 0, 55, 111, 114, 103, 46, 109, 111, 100, 101, 115, 104, 97, 112, 101, 46, 115, 99, 104, 101, 109, 97, 116, 105, 99, 46, 105, 110, 116, 101, 114, 110, 97, 108, 46, 100, 111, 99, 117, 109, 101, 110, 116, 46, 66, 97, 115, 105, 99, 68, 111, 99, 117, 109, 101, 110, 116, 0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 0, 120, 114, 0, 23, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 76, 105, 110, 107, 101, 100, 72, 97, 115, 104, 77, 97, 112, 52, -64, 78, 92, 16, 108, -64, -5, 2, 0, 1, 90, 0, 11, 97, 99, 99, 101, 115, 115, 79, 114, 100, 101, 114, 120, 114, 0, 17, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 72, 97, 115, 104, 77, 97, 112, 5, 7, -38, -63, -61, 22, 96, -47, 3, 0, 2, 70, 0, 10, 108, 111, 97, 100, 70, 97, 99, 116, 111, 114, 73, 0, 9, 116, 104, 114, 101, 115, 104, 111, 108, 100, 120, 112, 63, 64, 0, 0, 0, 0, 0, 12, 119, 8, 0, 0, 0, 16, 0, 0, 0, 2, 116, 0, 8, 109, 101, 116, 97, 100, 97, 116, 97, 115, 113, 0, 126, 0, 0, 63, 64, 0, 0, 0, 0, 0, 12, 119, 8, 0, 0, 0, 16, 0, 0, ... [1.4.196/0]

at org.modeshape.jcr.JcrRepository.login(JcrRepository.java:634)

at org.modeshape.jcr.JcrRepository.login(JcrRepository.java:139)

at org.brixcms.jcr.AbstractThreadLocalSessionFactory.createSession(AbstractThreadLocalSessionFactory.java:55)

... 60 more

Caused by: java.lang.IllegalArgumentException: Could not deserialize [-84, -19, 0, 5, 115, 114, 0, 55, 111, 114, 103, 46, 109, 111, 100, 101, 115, 104, 97, 112, 101, 46, 115, 99, 104, 101, 109, 97, 116, 105, 99, 46, 105, 110, 116, 101, 114, 110, 97, 108, 46, 100, 111, 99, 117, 109, 101, 110, 116, 46, 66, 97, 115, 105, 99, 68, 111, 99, 117, 109, 101, 110, 116, 0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 0, 120, 114, 0, 23, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 76, 105, 110, 107, 101, 100, 72, 97, 115, 104, 77, 97, 112, 52, -64, 78, 92, 16, 108, -64, -5, 2, 0, 1, 90, 0, 11, 97, 99, 99, 101, 115, 115, 79, 114, 100, 101, 114, 120, 114, 0, 17, 106, 97, 118, 97, 46, 117, 116, 105, 108, 46, 72, 97, 115, 104, 77, 97, 112, 5, 7, -38, -63, -61, 22, 96, -47, 3, 0, 2, 70, 0, 10, 108, 111, 97, 100, 70, 97, 99, 116, 111, 114, 73, 0, 9, 116, 104, 114, 101, 115, 104, 111, 108, 100, 120, 112, 63, 64, 0, 0, 0, 0, 0, 12, 119, 8, 0, 0, 0, 16, 0, 0, 0, 2, 116, 0, 8, 109, 101, 116, 97, 100, 97, 116, 97, 115, 113, 0, 126, 0, 0, 63, 64, 0, 0, 0, 0, 0, 12, 119, 8, 0, 0, 0, 16, 0, 0, ... [1.4.196/0]

at org.h2.mvstore.DataUtils.newIllegalArgumentException(DataUtils.java:728)

at org.h2.mvstore.type.ObjectDataType.deserialize(ObjectDataType.java:377)

at org.h2.mvstore.type.ObjectDataType$SerializedObjectType.read(ObjectDataType.java:1556)

at org.h2.mvstore.type.ObjectDataType.read(ObjectDataType.java:232)

at org.h2.mvstore.db.TransactionStore$VersionedValueType.read(TransactionStore.java:1648)

at org.h2.mvstore.Page.read(Page.java:711)

at org.h2.mvstore.Page.read(Page.java:195)

at org.h2.mvstore.MVStore.readPage(MVStore.java:1952)

at org.h2.mvstore.MVMap.readPage(MVMap.java:741)

at org.h2.mvstore.Page.getChildPage(Page.java:217)

at org.h2.mvstore.MVMap.binarySearch(MVMap.java:473)

at org.h2.mvstore.MVMap.binarySearch(MVMap.java:474)

at org.h2.mvstore.MVMap.get(MVMap.java:455)

at org.h2.mvstore.db.TransactionStore$TransactionMap.getValue(TransactionStore.java:1207)

at org.h2.mvstore.db.TransactionStore$TransactionMap.get(TransactionStore.java:1184)

at org.h2.mvstore.db.TransactionStore$TransactionMap.get(TransactionStore.java:1152)

at org.modeshape.persistence.file.FileDb.get(FileDb.java:96)

at org.modeshape.schematic.SchematicDb.getEntry(SchematicDb.java:160)

at org.modeshape.jcr.cache.document.LocalDocumentStore.get(LocalDocumentStore.java:88)

at org.modeshape.jcr.cache.RepositoryCache.<init>(RepositoryCache.java:150)

at org.modeshape.jcr.JcrRepository$RunningState.<init>(JcrRepository.java:1113)

at org.modeshape.jcr.JcrRepository$RunningState.<init>(JcrRepository.java:939)

at org.modeshape.jcr.JcrRepository.doStart(JcrRepository.java:375)

at org.modeshape.jcr.JcrRepository.login(JcrRepository.java:632)

... 62 more

Caused by: java.lang.ClassNotFoundException: org.modeshape.schematic.internal.document.BasicDocument

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at com.sun.enterprise.loader.CurrentBeforeParentClassLoader.loadClass(CurrentBeforeParentClassLoader.java:83)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:348)

at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:686)

at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1866)

at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1749)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2040)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)

at org.h2.mvstore.type.ObjectDataType.deserialize(ObjectDataType.java:375)

... 84 more

]]

What exactly has changed in terms of serializing/ deserializing and URL Class loader from payara 5 vs payara 4? 

Both were using the exact same JDK and the exact same domain configuration (copied over from payara 4 to payara 5)

Best,

KB

[Ondro Mihályi]

Hi,

I'm not aware of any intentional changes in classloading in Payara 5. Many things affect classloading and it's possible that something was changed unintentionally.

Can you find org.modeshape.schematic.internal.document.BasicDocument in your application package? How do you add it to your application? Is it in a JAR in WEB-INF/lib?

Ondro

[Korbinian]

Hi,

thanks for the response. Basically its the same war for both servers - the 

org.modeshape.schematic.internal.document.BasicDocument 

is in WEB-INF/lib in modeshape-schematic-5.4.1.Final.jar

Can it be that payara 5 somehow tries to lazy load these jar's now - or any kind of race condition ? - I mean its quite a simple class so far, see 

https://github.com/ModeShape/modeshape/blob/modeshape-5.4.1.Final/modeshape-schematic/src/main/java/org/modeshape/schematic/internal/document/BasicDocument.java

and its only happening on a deploy where an already existing repo has to get loaded, not when it gets createted initally.... I expected to maybe get some CDI errors, but those class not found really get me;

Best,

KB

[Ondro Mihályi]

I see that your application contains H2 database client

The problem is most probably caused by inclusion of H2 into Payara 5 as another built-in database next to Derby. Payara Server loads H2 classes in a server's classloader and they don't see classes loaded by your applicaiton.

A solution is to disable classloading delegation in your application so that Payara always prefers H2 classes in your app before those in the server. There are multiple ways to do it described in Payara documentation: https://docs.payara.fish/documentation/payara-server/classloading.html

All the best,

Ondro

po 13. 8. 2018 o 9:33 Korbinian <korbini...@gmail.com> napísal(a):

--
You received this message because you are subscribed to the Google Groups "Payara Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to payara-forum...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/payara-forum/3962a2b6-2e07-4ed9-a9b8-4fbebb118f66%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.