Failure to start Diameter stack in cluster mode
156 views
Skip to first unread message
Richard Good
Jun 3, 2012, 7:32:02 AM6/3/12
to mobicents-public
Hi
I have two nodes in a cluster both running the Diameter MUX. I have noticed every now and then on start up one of the nodes fails to start with the error below.
If I bounce the node, it starts up fine.
This line seems the cause of the failure:
Caused by: org.jboss.cache.CacheException: java.io.InvalidClassException: org.jdiameter.api.ApplicationId; local class incompatible: stream classdesc serialVersionUID = 8435889081543623290, local class serialVersionUID = 1
Any ideas why this might happen?
I am using JSLEE2.6.0.FINAL and jdiameter trunk.
Regards
Richard.
LOG FILE:
---------------------------------------------------------
GMS: address is 10.0.1.92:35993 (cluster=DiameterCluster)
---------------------------------------------------------
2012-06-02 09:13:52,741 INFO [org.jboss.cache.RPCManagerImpl] Received new cluster view: [10.0.1.96:45939|3] [10.0.1.96:45939, 10.0.1.92:35993]
2012-06-02 09:13:52,832 WARN [org.jboss.cache.marshall.VersionAwareMarshaller] Unknown replication version [999]. Falling back to the default marshaller installed.
2012-06-02 09:13:52,833 WARN [org.jboss.cache.marshall.VersionAwareMarshaller] Unknown replication version [999]. Falling back to the default marshaller installed.
2012-06-02 09:13:52,834 INFO [org.jboss.cache.statetransfer.DefaultStateTransferIntegrator] Using version 4096
2012-06-02 09:13:52,837 WARN [org.jboss.cache.marshall.VersionAwareMarshaller] Unknown replication version [999]. Falling back to the default marshaller installed.
2012-06-02 09:13:52,846 INFO [org.jboss.cache.RPCManagerImpl] Cache local address is 10.0.1.92:35993
2012-06-02 09:13:52,847 INFO [org.jboss.cache.RPCManagerImpl] Disconnecting and closing the Channel
2012-06-02 09:13:52,865 ERROR [org.jboss.kernel.plugins.dependency.AbstractKernelController] Error installing to Start: name=Mobicents.Diameter.StackMuxMBean state=Create
org.jboss.cache.CacheException: java.lang.reflect.InvocationTargetException
at org.jboss.cache.util.reflect.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:148)
at org.jboss.cache.factories.ComponentRegistry$PrioritizedMethod.invoke(ComponentRegistry.java:1012)
at org.jboss.cache.factories.ComponentRegistry.internalStart(ComponentRegistry.java:778)
at org.jboss.cache.factories.ComponentRegistry.start(ComponentRegistry.java:632)
at org.jboss.cache.invocation.CacheInvocationDelegate.start(CacheInvocationDelegate.java:345)
at org.mobicents.cache.MobicentsCache.startCache(MobicentsCache.java:88)
at org.mobicents.cluster.DefaultMobicentsCluster.startCluster(DefaultMobicentsCluster.java:595)
at org.mobicents.diameter.impl.ha.data.ReplicatedSessionDatasource.<init>(ReplicatedSessionDatasource.java:114)
at org.mobicents.diameter.impl.ha.data.ReplicatedSessionDatasource.<init>(ReplicatedSessionDatasource.java:93)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.picocontainer.injectors.AbstractInjector.newInstance(AbstractInjector.java:147)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:332)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:272)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:354)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:692)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:671)
at org.jdiameter.client.impl.helpers.AssemblerImpl.getComponentInstance(AssemblerImpl.java:131)
at org.jdiameter.client.impl.MetaDataImpl.<init>(MetaDataImpl.java:97)
at org.jdiameter.client.impl.MetaDataImpl.<init>(MetaDataImpl.java:101)
at org.jdiameter.server.impl.MetaDataImpl.<init>(MetaDataImpl.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.picocontainer.injectors.AbstractInjector.newInstance(AbstractInjector.java:147)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:332)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:272)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:354)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:692)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:632)
at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:105)
at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:76)
at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:299)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:325)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:272)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:354)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:692)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:671)
at org.jdiameter.client.impl.helpers.AssemblerImpl.getComponentInstance(AssemblerImpl.java:131)
at org.jdiameter.client.impl.StackImpl.init(StackImpl.java:144)
at org.mobicents.diameter.stack.DiameterStackMultiplexer.initStack(DiameterStackMultiplexer.java:112)
at org.mobicents.diameter.stack.DiameterStackMultiplexer.initStack(DiameterStackMultiplexer.java:101)
at org.mobicents.diameter.stack.DiameterStackMultiplexer.startService(DiameterStackMultiplexer.java:278)
at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:376)
at org.jboss.system.ServiceMBeanSupport.pojoStart(ServiceMBeanSupport.java:216)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:59)
at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:150)
at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction$JoinpointDispatchWrapper.execute(KernelControllerContextAction.java:241)
at org.jboss.kernel.plugins.dependency.ExecutionWrapper.execute(ExecutionWrapper.java:47)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchExecutionWrapper(KernelControllerContextAction.java:109)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchJoinPoint(KernelControllerContextAction.java:70)
at org.jboss.kernel.plugins.dependency.LifecycleAction.installActionInternal(LifecycleAction.java:221)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:54)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:42)
at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.ServiceController.doChange(ServiceController.java:688)
at org.jboss.system.ServiceController.start(ServiceController.java:460)
at org.jboss.system.microcontainer.jmx.ServiceControllerStartStopLifecycleCallback.install(ServiceControllerStartStopLifecycleCallback.java:44)
at sun.reflect.GeneratedMethodAccessor254.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:59)
at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:150)
at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66)
at org.jboss.beans.info.plugins.AbstractBeanInfo.invoke(AbstractBeanInfo.java:300)
at org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext.invoke(AbstractKernelControllerContext.java:286)
at org.jboss.dependency.plugins.AbstractLifecycleCallbackItem.install(AbstractLifecycleCallbackItem.java:87)
at org.jboss.dependency.plugins.AbstractController.handleLifecycleCallbacks(AbstractController.java:1568)
at org.jboss.dependency.plugins.AbstractController.handleInstallLifecycleCallbacks(AbstractController.java:1533)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:943)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:774)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:540)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:121)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:51)
at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
at org.jboss.system.server.profileservice.repository.ProfileDeployAction.install(ProfileDeployAction.java:70)
at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.java:306)
at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
at org.jboss.Main.boot(Main.java:221)
at org.jboss.Main$1.run(Main.java:556)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.jboss.cache.util.reflect.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:144)
... 135 more
Caused by: org.jboss.cache.CacheException: Unable to fetch state on startup
at org.jboss.cache.RPCManagerImpl.start(RPCManagerImpl.java:396)
... 140 more
Caused by: org.jboss.cache.CacheException: java.io.InvalidClassException: org.jdiameter.api.ApplicationId; local class incompatible: stream classdesc serialVersionUID = 8435889081543623290, local class serialVersionUID = 1
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:251)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateState(DefaultStateTransferIntegrator.java:113)
at org.jboss.cache.statetransfer.DefaultStateTransferManager.setState(DefaultStateTransferManager.java:199)
at org.jboss.cache.statetransfer.DefaultStateTransferManager.setState(DefaultStateTransferManager.java:163)
at org.jboss.cache.remoting.jgroups.ChannelMessageListener.setState(ChannelMessageListener.java:190)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:676)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:776)
at org.jgroups.JChannel.up(JChannel.java:1252)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:462)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.handleStateRsp(STATE_TRANSFER.java:421)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:120)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.FC.up(FC.java:473)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:824)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:192)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:616)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:758)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:136)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:309)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:264)
at org.jgroups.protocols.PING.up(PING.java:273)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1285)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1838)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1817)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
... 1 more
Caused by: java.io.InvalidClassException: org.jdiameter.api.ApplicationId; local class incompatible: stream classdesc serialVersionUID = 8435889081543623290, local class serialVersionUID = 1
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
at java.util.HashMap.readObject(HashMap.java:1043)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1004)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1866)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
at org.jboss.cache.marshall.NodeData.readExternal(NodeData.java:112)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:685)
at org.jboss.cache.marshall.CacheMarshaller200.populateFromStream(CacheMarshaller200.java:875)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallLinkedList(CacheMarshaller200.java:818)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:649)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:589)
at org.jboss.cache.marshall.CacheMarshaller200.objectFromObjectStream(CacheMarshaller200.java:161)
at org.jboss.cache.marshall.VersionAwareMarshaller.objectFromObjectStream(VersionAwareMarshaller.java:360)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.readNodesAsList(DefaultStateTransferIntegrator.java:405)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:362)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:234)
... 32 more
I have two nodes in a cluster both running the Diameter MUX. I have noticed every now and then on start up one of the nodes fails to start with the error below.
If I bounce the node, it starts up fine.
This line seems the cause of the failure:
Caused by: org.jboss.cache.CacheException: java.io.InvalidClassException: org.jdiameter.api.ApplicationId; local class incompatible: stream classdesc serialVersionUID = 8435889081543623290, local class serialVersionUID = 1
Any ideas why this might happen?
I am using JSLEE2.6.0.FINAL and jdiameter trunk.
Regards
Richard.
LOG FILE:
---------------------------------------------------------
GMS: address is 10.0.1.92:35993 (cluster=DiameterCluster)
---------------------------------------------------------
2012-06-02 09:13:52,741 INFO [org.jboss.cache.RPCManagerImpl] Received new cluster view: [10.0.1.96:45939|3] [10.0.1.96:45939, 10.0.1.92:35993]
2012-06-02 09:13:52,832 WARN [org.jboss.cache.marshall.VersionAwareMarshaller] Unknown replication version [999]. Falling back to the default marshaller installed.
2012-06-02 09:13:52,833 WARN [org.jboss.cache.marshall.VersionAwareMarshaller] Unknown replication version [999]. Falling back to the default marshaller installed.
2012-06-02 09:13:52,834 INFO [org.jboss.cache.statetransfer.DefaultStateTransferIntegrator] Using version 4096
2012-06-02 09:13:52,837 WARN [org.jboss.cache.marshall.VersionAwareMarshaller] Unknown replication version [999]. Falling back to the default marshaller installed.
2012-06-02 09:13:52,846 INFO [org.jboss.cache.RPCManagerImpl] Cache local address is 10.0.1.92:35993
2012-06-02 09:13:52,847 INFO [org.jboss.cache.RPCManagerImpl] Disconnecting and closing the Channel
2012-06-02 09:13:52,865 ERROR [org.jboss.kernel.plugins.dependency.AbstractKernelController] Error installing to Start: name=Mobicents.Diameter.StackMuxMBean state=Create
org.jboss.cache.CacheException: java.lang.reflect.InvocationTargetException
at org.jboss.cache.util.reflect.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:148)
at org.jboss.cache.factories.ComponentRegistry$PrioritizedMethod.invoke(ComponentRegistry.java:1012)
at org.jboss.cache.factories.ComponentRegistry.internalStart(ComponentRegistry.java:778)
at org.jboss.cache.factories.ComponentRegistry.start(ComponentRegistry.java:632)
at org.jboss.cache.invocation.CacheInvocationDelegate.start(CacheInvocationDelegate.java:345)
at org.mobicents.cache.MobicentsCache.startCache(MobicentsCache.java:88)
at org.mobicents.cluster.DefaultMobicentsCluster.startCluster(DefaultMobicentsCluster.java:595)
at org.mobicents.diameter.impl.ha.data.ReplicatedSessionDatasource.<init>(ReplicatedSessionDatasource.java:114)
at org.mobicents.diameter.impl.ha.data.ReplicatedSessionDatasource.<init>(ReplicatedSessionDatasource.java:93)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.picocontainer.injectors.AbstractInjector.newInstance(AbstractInjector.java:147)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:332)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:272)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:354)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:692)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:671)
at org.jdiameter.client.impl.helpers.AssemblerImpl.getComponentInstance(AssemblerImpl.java:131)
at org.jdiameter.client.impl.MetaDataImpl.<init>(MetaDataImpl.java:97)
at org.jdiameter.client.impl.MetaDataImpl.<init>(MetaDataImpl.java:101)
at org.jdiameter.server.impl.MetaDataImpl.<init>(MetaDataImpl.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.picocontainer.injectors.AbstractInjector.newInstance(AbstractInjector.java:147)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:332)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:272)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:354)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:692)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:632)
at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:105)
at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:76)
at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:299)
at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:325)
at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:272)
at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:354)
at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:692)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:671)
at org.jdiameter.client.impl.helpers.AssemblerImpl.getComponentInstance(AssemblerImpl.java:131)
at org.jdiameter.client.impl.StackImpl.init(StackImpl.java:144)
at org.mobicents.diameter.stack.DiameterStackMultiplexer.initStack(DiameterStackMultiplexer.java:112)
at org.mobicents.diameter.stack.DiameterStackMultiplexer.initStack(DiameterStackMultiplexer.java:101)
at org.mobicents.diameter.stack.DiameterStackMultiplexer.startService(DiameterStackMultiplexer.java:278)
at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:376)
at org.jboss.system.ServiceMBeanSupport.pojoStart(ServiceMBeanSupport.java:216)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:59)
at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:150)
at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction$JoinpointDispatchWrapper.execute(KernelControllerContextAction.java:241)
at org.jboss.kernel.plugins.dependency.ExecutionWrapper.execute(ExecutionWrapper.java:47)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchExecutionWrapper(KernelControllerContextAction.java:109)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchJoinPoint(KernelControllerContextAction.java:70)
at org.jboss.kernel.plugins.dependency.LifecycleAction.installActionInternal(LifecycleAction.java:221)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:54)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:42)
at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.ServiceController.doChange(ServiceController.java:688)
at org.jboss.system.ServiceController.start(ServiceController.java:460)
at org.jboss.system.microcontainer.jmx.ServiceControllerStartStopLifecycleCallback.install(ServiceControllerStartStopLifecycleCallback.java:44)
at sun.reflect.GeneratedMethodAccessor254.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:59)
at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:150)
at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66)
at org.jboss.beans.info.plugins.AbstractBeanInfo.invoke(AbstractBeanInfo.java:300)
at org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext.invoke(AbstractKernelControllerContext.java:286)
at org.jboss.dependency.plugins.AbstractLifecycleCallbackItem.install(AbstractLifecycleCallbackItem.java:87)
at org.jboss.dependency.plugins.AbstractController.handleLifecycleCallbacks(AbstractController.java:1568)
at org.jboss.dependency.plugins.AbstractController.handleInstallLifecycleCallbacks(AbstractController.java:1533)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:943)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:774)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:540)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:121)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:51)
at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
at org.jboss.system.server.profileservice.repository.ProfileDeployAction.install(ProfileDeployAction.java:70)
at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.java:306)
at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
at org.jboss.Main.boot(Main.java:221)
at org.jboss.Main$1.run(Main.java:556)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.jboss.cache.util.reflect.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:144)
... 135 more
Caused by: org.jboss.cache.CacheException: Unable to fetch state on startup
at org.jboss.cache.RPCManagerImpl.start(RPCManagerImpl.java:396)
... 140 more
Caused by: org.jboss.cache.CacheException: java.io.InvalidClassException: org.jdiameter.api.ApplicationId; local class incompatible: stream classdesc serialVersionUID = 8435889081543623290, local class serialVersionUID = 1
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:251)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateState(DefaultStateTransferIntegrator.java:113)
at org.jboss.cache.statetransfer.DefaultStateTransferManager.setState(DefaultStateTransferManager.java:199)
at org.jboss.cache.statetransfer.DefaultStateTransferManager.setState(DefaultStateTransferManager.java:163)
at org.jboss.cache.remoting.jgroups.ChannelMessageListener.setState(ChannelMessageListener.java:190)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:676)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:776)
at org.jgroups.JChannel.up(JChannel.java:1252)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:462)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.handleStateRsp(STATE_TRANSFER.java:421)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:120)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.FC.up(FC.java:473)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:824)
at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:192)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:616)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:758)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:136)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:309)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:264)
at org.jgroups.protocols.PING.up(PING.java:273)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1285)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1838)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1817)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
... 1 more
Caused by: java.io.InvalidClassException: org.jdiameter.api.ApplicationId; local class incompatible: stream classdesc serialVersionUID = 8435889081543623290, local class serialVersionUID = 1
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:604)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
at java.util.HashMap.readObject(HashMap.java:1043)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1004)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1866)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
at org.jboss.cache.marshall.NodeData.readExternal(NodeData.java:112)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:685)
at org.jboss.cache.marshall.CacheMarshaller200.populateFromStream(CacheMarshaller200.java:875)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallLinkedList(CacheMarshaller200.java:818)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:649)
at org.jboss.cache.marshall.CacheMarshaller200.unmarshallObject(CacheMarshaller200.java:589)
at org.jboss.cache.marshall.CacheMarshaller200.objectFromObjectStream(CacheMarshaller200.java:161)
at org.jboss.cache.marshall.VersionAwareMarshaller.objectFromObjectStream(VersionAwareMarshaller.java:360)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.readNodesAsList(DefaultStateTransferIntegrator.java:405)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:362)
at org.jboss.cache.statetransfer.DefaultStateTransferIntegrator.integrateTransientState(DefaultStateTransferIntegrator.java:234)
... 32 more
This email is subject to the disclaimer of Smile Communications (PTY) Ltd. at http://www.smilecoms.com/disclaimer
Alexandre Mendonça
Jun 5, 2012, 11:55:43 AM6/5/12
to mobicent...@googlegroups.com
Hi Richard,
At first look, it seems you may be using different "versions" of org.jdiameter.api.ApplicationId. While it hasn't changed in content/functionality it was added the serialVersionUID = 1L at r22230.
Make sure you are using the same in both nodes.
Richard Good
Jun 5, 2012, 2:21:09 PM6/5/12
to mobicent...@googlegroups.com
Wouldn't this cause the stack to fail starting everytime? My problem occurs only every now and then.
Richard.
Reply all
Reply to author
Forward
0 new messages