Memcached and Kryo Serialization on Tomcat throws NPE

480 views
Skip to first unread message

Rasoul Firoz

unread,
Dec 19, 2016, 5:38:51 PM12/19/16
to memcached-session-manager

I would like to use msm-session-manager and kryo as serialization strategy.

I use tomcat6, java 8 and following libs:

I started with this configuration as suggested by most documentation online (https://github.com/magro/memcached-session-manager/wiki/SetupAndConfiguration#introduction).

tomcat/lib:
spymemcached-2.11.1.jar
memcached-session-manager-tc6-2.0.0.jar
memcached-session-manager-2.0.0.jar

WEB-INF/lib:
asm-5.1.jar
kryo-3.0.3.jar
kryo-serializers-0.37.jar
minlog-1.3.0.jar
reflectasm-1.11.0.jar
msm-kryo-serializer-2.0.0.jar
objenesis-2.4.jar

However, it wasn't able to properly initialize the serializer, so had to move all jars into tomcat/lib dir. Now the configuration is like this:

tomcat/lib:
spymemcached-2.11.1.jar
memcached-session-manager-tc6-2.0.0.jar
memcached-session-manager-2.0.0.jar
asm-5.1.jar
kryo-3.0.3.jar
kryo-serializers-0.37.jar
minlog-1.3.0.jar
reflectasm-1.11.0.jar
msm-kryo-serializer-2.0.0.jar
objenesis-2.4.jar

This is what I have in context.xml

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
      memcachedNodes="n1:localhost:11211"
      requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"     transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
      />

With this configuration, I get the following NPE

WARNING: Could not deserialize session with id EF25541E42C7AE7BFC993FA70D6FC541-n1 from memcached, session will be purged from storage.
de.javakaffee.web.msm.TranscoderDeserializationException: com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
Serialization trace:
criteria (com.quadramed.core.web.LWPagedResult)
    at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:213)
    at de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:167)
    at de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:124)
    at de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1147)
    at de.javakaffee.web.msm.MemcachedSessionService.findSession(MemcachedSessionService.java:597)
    at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:233)
    at org.apache.catalina.connector.Request.doGetSession(Request.java:2419)
    at org.apache.catalina.connector.Request.getSession(Request.java:2157)
    at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:833)
    at com.nuance.him.topaz.filter.ClintegrityMDCFilter.fillMDC(ClintegrityMDCFilter.java:78)
    at com.nuance.him.topaz.filter.ClintegrityMDCFilter.doFilter(ClintegrityMDCFilter.java:56)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at de.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:156)
    at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:124)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
Serialization trace:
criteria (com.quadramed.core.web.LWPagedResult)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
    at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:161)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:686)
    at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:211)
    ... 25 more
Caused by: java.lang.NullPointerException
    at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:234)
    at java.util.ArrayList.ensureCapacity(ArrayList.java:218)
    at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:114)
    at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
    at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
    at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    ... 31 more

I'm not sure what's wrong with my configuration. I tried it with two tomcats and two memcached setup, but still got the same result. This is the config I tried:

<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
      memcachedNodes="n1:host1:11211,n2:host2:11211"
      failoverNodes="n1"
      requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"         transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
      />
  • host1 is localhost in one domain
  • host2 is another machine in a different domain, but full domain path was provided in host2

Any help is much appreciated or any direction on what could be the problem?

Thanks

Martin Grotzke

unread,
Dec 19, 2016, 7:14:51 PM12/19/16
to memcached-se...@googlegroups.com
On 12/19/2016 09:17 PM, Rasoul Firoz wrote:
>
> I would like to use msm-session-manager and kryo as serialization strategy.
>
> I use tomcat6, java 8 and following libs:
>
> I started with this configuration as suggested by most documentation
> online
> (https://github.com/magro/memcached-session-manager/wiki/SetupAndConfiguration#introduction).
>
> |tomcat/lib: spymemcached-2.11.1.jar
> memcached-session-manager-tc6-2.0.0.jar
> memcached-session-manager-2.0.0.jar WEB-INF/lib: asm-5.1.jar
> kryo-3.0.3.jar kryo-serializers-0.37.jar minlog-1.3.0.jar
> reflectasm-1.11.0.jar msm-kryo-serializer-2.0.0.jar objenesis-2.4.jar |
>
> However, it wasn't able to properly initialize the serializer, so had to
> move all jars into tomcat/lib dir.

Placement of libs sounds good so far. What was the exception during
initialization?

Cheers,
Martin
> * host1 is localhost in one domain
> * host2 is another machine in a different domain, but full domain path
> was provided in host2
>
> Any help is much appreciated or any direction on what could be the problem?
>
> Thanks
>
> --
>
> ---
> You received this message because you are subscribed to the Google
> Groups "memcached-session-manager" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to memcached-session-...@googlegroups.com
> <mailto:memcached-session-...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

--
inoio gmbh - http://inoio.de
Schulterblatt 36, 20357 Hamburg
Amtsgericht Hamburg, HRB 123031
Geschäftsführer: Dennis Brakhane, Martin Grotzke, Ole Langbehn

signature.asc

Rasoul Firoz

unread,
Dec 20, 2016, 9:13:00 AM12/20/16
to memcached-session-manager
Thanks Martin for your reply. This is the configuration I had problem with during initialization.

With this configuration:
tomcat/lib:
spymemcached
-2.11.1.jar
memcached
-session-manager-tc6-2.0.0.jar
memcached
-session-manager-2.0.0.jar


WEB
-INF/lib:
asm-5.1.jar
kryo
-3.0.3.jar
kryo
-serializers-0.37.jar
minlog
-1.3.0.jar
reflectasm
-1.11.0.jar
msm
-kryo-serializer-2.0.0.jar
objenesis
-2.4.jar


I get the following exception in stderr.log
Dec 20, 2016 8:55:31 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO
: MemcachedSessionService starts initialization... (configured nodes definition n1:localhost:11211, failover nodes null)
Dec 20, 2016 8:55:31 AM de.javakaffee.web.msm.RequestTrackingHostValve <init>
INFO
: Setting ignorePattern to .*\.(ico|png|gif|jpg|css|js)$
Dec 20, 2016 8:55:31 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode
INFO
: Setting lockingMode to null
Dec 20, 2016 8:55:31 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory
INFO
: Creating transcoder factory de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory
Dec 20, 2016 8:55:31 AM de.javakaffee.web.msm.MemcachedSessionService loadTranscoderFactoryClass
INFO
: Could not load transcoderfactory class with classloader null, trying org.apache.catalina.loader.StandardClassLoader@1251deb
Dec 20, 2016 8:55:31 AM org.apache.catalina.core.StandardContext start
SEVERE
: Error manager.start()
java
.lang.RuntimeException: Could not create transcoder factory.
 at de
.javakaffee.web.msm.MemcachedSessionService.getTranscoderFactory(MemcachedSessionService.java:505)
 at de
.javakaffee.web.msm.MemcachedSessionService.createTranscoderService(MemcachedSessionService.java:497)
 at de
.javakaffee.web.msm.MemcachedSessionService.startInternal(MemcachedSessionService.java:452)
 at de
.javakaffee.web.msm.MemcachedBackupSessionManager.startInternal(MemcachedBackupSessionManager.java:145)
 at de
.javakaffee.web.msm.MemcachedBackupSessionManager.start(MemcachedBackupSessionManager.java:675)
 at org
.apache.catalina.core.StandardContext.start(StandardContext.java:4351)
 at org
.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
 at org
.apache.catalina.core.StandardHost.start(StandardHost.java:719)
 at org
.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
 at org
.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
 at org
.apache.catalina.core.StandardService.start(StandardService.java:516)
 at org
.apache.catalina.core.StandardServer.start(StandardServer.java:710)
 at org
.apache.catalina.startup.Catalina.start(Catalina.java:578)
 at sun
.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun
.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java
.lang.reflect.Method.invoke(Method.java:498)
 at org
.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
 at org
.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.lang.ClassNotFoundException: de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory
 at java
.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java
.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at java
.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at java
.lang.Class.forName0(Native Method)
 at java
.lang.Class.forName(Class.java:348)
 at de
.javakaffee.web.msm.MemcachedSessionService.loadTranscoderFactoryClass(MemcachedSessionService.java:542)
 at de
.javakaffee.web.msm.MemcachedSessionService.createTranscoderFactory(MemcachedSessionService.java:524)
 at de
.javakaffee.web.msm.MemcachedSessionService.getTranscoderFactory(MemcachedSessionService.java:503)
 
... 18 more


Dec 20, 2016 8:55:31 AM org.apache.catalina.core.StandardContext start
SEVERE
: Context [/QDesigner] startup failed due to previous errors
Dec 20, 2016 8:55:31 AM org.apache.catalina.core.StandardContext stop
INFO
: Container org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/QDesigner] has not been started
Dec 20, 2016 8:55:32 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO
: MemcachedSessionService starts initialization... (configured nodes definition n1:localhost:11211, failover nodes null)
Dec 20, 2016 8:55:32 AM de.javakaffee.web.msm.RequestTrackingHostValve <init>
INFO
: Setting ignorePattern to .*\.(ico|png|gif|jpg|css|js)$
Dec 20, 2016 8:55:32 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode
INFO
: Setting lockingMode to null
Dec 20, 2016 8:55:32 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory
INFO
: Creating transcoder factory de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory
Dec 20, 2016 8:55:32 AM de.javakaffee.web.msm.serializer.kryo.KryoTranscoder <init>
INFO
: Starting with initialBufferSize 102400, maxBufferSize 2048000 and defaultSerializerFactory de.javakaffee.web.msm.serializer.kryo.DefaultFieldSerializerFactory
Dec 20, 2016 8:55:32 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO
: --------
- MemcachedSessionService finished initialization:
- sticky: true
- operation timeout: 1000
- node ids: [n1]
- failover node ids: []
- storage key prefix: null
--------


And see this in stdout.log
2016-12-20 08:55:31.301 INFO net.spy.memcached.MemcachedConnection:  Added {QA sa=localhost/127.0.0.1:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2016-12-20 08:55:31.301 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@16373df


Upon tomcat restart, I get this in stderr.log
Dec 20, 2016 8:57:02 AM de.javakaffee.web.msm.MemcachedSessionService loadFromMemcached
WARNING
: Could not deserialize session with id 25669D5AA95A94B92947C85888DF0B59-n1 from memcached, session will be purged from storage.

de
.javakaffee.web.msm.TranscoderDeserializationException: com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
Serialization trace:
criteria
(com.quadramed.core.web.LWPagedResult)
 at de
.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:213)
 at de
.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:167)
 at de
.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:124)
 at de
.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1147)
 at de
.javakaffee.web.msm.MemcachedSessionService.findSession(MemcachedSessionService.java:597)
 at de
.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:233)

 at org
.apache.catalina.connector.Request.doGetSession(Request.java:2283)
 at org
.apache.catalina.connector.Request.getSession(Request.java:2075)

 at org
.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:833)
 at com
.nuance.him.topaz.filter.ClintegrityMDCFilter.fillMDC(ClintegrityMDCFilter.java:78)
 at com
.nuance.him.topaz.filter.ClintegrityMDCFilter.doFilter(ClintegrityMDCFilter.java:56)
 at org
.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
 at org
.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at org
.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 at org
.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at de
.javakaffee.web.msm.RequestTrackingContextValve.invoke(RequestTrackingContextValve.java:99)

 at org
.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
 at de
.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:124)

 at de
.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:156)
 at de
.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:124)

 at org
.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at org
.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:568)
 at org
.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at org
.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
 at org
.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
 at org
.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
 at org
.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)

 at java
.lang.Thread.run(Thread.java:745)
Caused by: com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
Serialization trace:
criteria
(com.quadramed.core.web.LWPagedResult)
 at com
.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
 at com
.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
 at com
.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
 at com
.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:161)
 at com
.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
 at com
.esotericsoftware.kryo.Kryo.readObject(Kryo.java:686)
 at de
.javakaffee.web.msm.serializer.kryo.KryoTranscoder.deserializeAttributes(KryoTranscoder.java:211)

 
... 27 more
Caused by: java.lang.NullPointerException
 at java
.util.ArrayList.ensureExplicitCapacity(ArrayList.java:234)
 at java
.util.ArrayList.ensureCapacity(ArrayList.java:218)
 at com
.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:114)
 at com
.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
 at com
.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
 at com
.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)

 
... 33 more


This configuration is single tomcat and single memcached node running on localhost

Thanks,
Rasoul
Reply all
Reply to author
Forward
0 new messages