java.lang.reflect.InvocationTargetException
112 views
Skip to first unread message
Raziya Bhayani
Jan 10, 2010, 1:23:04 AM1/10/10
to mobicent...@googlegroups.com
Hello,
Heres the scenario. I am trying to run Mobicents JAIN SLEE server from trunk in clustered environment (2 "all" nodes), making SMPP RA HA enabled. As of now, in SmppResourceAdaptor class, I am checking, if it is a head member and bind to SMSC only then. I have a service deployed which has Echo Sbb, it will take the sms fired from SMPPSim and reply it back, just an echo.
For JAIN SLee server I am at 9229 revision. SMPP RA 2.0.0.BETA2, has been modified, to suite our needs. I am using jboss-5.1.0.GA. Everything deployes well. When I send an SMS from Simulator, the code cracks and I get java.lang.reflect.InvocationTargetException. I am attaching server.log of both nodes all and all2. When I shut the first node which throws error, the other one is notified about cluster member being dead.
I dont get consistent behaviour, at times faulttolerant api code works well and only one node binds to SMSC, at other times, both the nodes bind to SMSC. Thats when faulttolerant api code is ignored. Attaching the ra class as well.
If you could please look into this and spot the cause for the same.
Thanks and regards.
Raziya
Heres the scenario. I am trying to run Mobicents JAIN SLEE server from trunk in clustered environment (2 "all" nodes), making SMPP RA HA enabled. As of now, in SmppResourceAdaptor class, I am checking, if it is a head member and bind to SMSC only then. I have a service deployed which has Echo Sbb, it will take the sms fired from SMPPSim and reply it back, just an echo.
For JAIN SLee server I am at 9229 revision. SMPP RA 2.0.0.BETA2, has been modified, to suite our needs. I am using jboss-5.1.0.GA. Everything deployes well. When I send an SMS from Simulator, the code cracks and I get java.lang.reflect.InvocationTargetException. I am attaching server.log of both nodes all and all2. When I shut the first node which throws error, the other one is notified about cluster member being dead.
I dont get consistent behaviour, at times faulttolerant api code works well and only one node binds to SMSC, at other times, both the nodes bind to SMSC. Thats when faulttolerant api code is ignored. Attaching the ra class as well.
If you could please look into this and spot the cause for the same.
Thanks and regards.
Raziya
Eduardo Martins
Jan 10, 2010, 8:00:23 AM1/10/10
to mobicent...@googlegroups.com
We will check, tks for the report, it came in great timing.
--
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
Eduardo Martins
Jan 10, 2010, 1:45:49 PM1/10/10
to mobicent...@googlegroups.com
1) InvocationTargetException - The actual cause is java.lang.NoClassDefFoundError: Lcom/uwyn/jhighlight/renderer/Renderer; when serializing the SMPP Dialog activity handle, this class is probably imported by some class in the RA or its libraries, and thus it should be in the RA library too.
Now I gave a quick look at the RA code, it really needs some changes to be ready for a cluster environment, I discussed that before with Oleg, and we agreed this RA should be temp and a new RA should come from a "standardized" RA Type with Open Cloud. You could try to fix this but I fear it may be more work than expected, the biggest issues I identified quickly are:
- handles have activities as non transient fields, when serialized the activity is serialized too
- activities have some fields which should be transient (by defualt only fields marked as transient and static are not serialized) , for instance the Timer and the ResourceAdaptor, which means currently the whole RA object is being serialized!! The RA is easy to solve, simply mark it as transient and make a setter which is invoked on every related RA method coming from network or SLEE (such as getActivity(ActivityHandle)), the Sip RA does that; the Timer is more a RA design issue, for instance a single RA timer should be used, and then it should be well defined what happens when an activity from node A (with a timer there) is accessed in node B, if that is a possible use case...
2) Both nodes being Head of cluster - this could be from the cluster not starting correctly, either because the nodes don't connect at all (thus you have 2 clusters with a single node) , which is something I have witness myself one time and seems to originate in jgroups or jboss cache 3; or it could be the startup of the nodes at same time where the RA code runs before the nodes actually link. My recommendation is to first completly load one node and then start the other, if the other scenario pops up please report.
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
Now I gave a quick look at the RA code, it really needs some changes to be ready for a cluster environment, I discussed that before with Oleg, and we agreed this RA should be temp and a new RA should come from a "standardized" RA Type with Open Cloud. You could try to fix this but I fear it may be more work than expected, the biggest issues I identified quickly are:
- handles have activities as non transient fields, when serialized the activity is serialized too
- activities have some fields which should be transient (by defualt only fields marked as transient and static are not serialized) , for instance the Timer and the ResourceAdaptor, which means currently the whole RA object is being serialized!! The RA is easy to solve, simply mark it as transient and make a setter which is invoked on every related RA method coming from network or SLEE (such as getActivity(ActivityHandle)), the Sip RA does that; the Timer is more a RA design issue, for instance a single RA timer should be used, and then it should be well defined what happens when an activity from node A (with a timer there) is accessed in node B, if that is a possible use case...
2) Both nodes being Head of cluster - this could be from the cluster not starting correctly, either because the nodes don't connect at all (thus you have 2 clusters with a single node) , which is something I have witness myself one time and seems to originate in jgroups or jboss cache 3; or it could be the startup of the nodes at same time where the RA code runs before the nodes actually link. My recommendation is to first completly load one node and then start the other, if the other scenario pops up please report.
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
Raziya Bhayani
Jan 11, 2010, 11:02:53 PM1/11/10
to mobicent...@googlegroups.com
Eduardo, thanks for your reply.
Please see my comments in-line.
Have fixed this.
If you could let me know how can we go about it, would pitch in to get it standardized.
I was thinking of making most of the things transient and implement JGroups so that the members in cluster group communicate to each other with regards to their current liveliness state. And thus, the peer node would now when one a neighbor is dead and thus bind itself to SMSC. Would this be a good approach?
Yes starting them one after another thus sort this. Thanks.
Please see my comments in-line.
On Mon, Jan 11, 2010 at 12:15 AM, Eduardo Martins <emma...@gmail.com> wrote:
1) InvocationTargetException - The actual cause is java.lang.NoClassDefFoundError: Lcom/uwyn/jhighlight/renderer/Renderer; when serializing the SMPP Dialog activity handle, this class is probably imported by some class in the RA or its libraries, and thus it should be in the RA library too.
Have fixed this.
Now I gave a quick look at the RA code, it really needs some changes to be ready for a cluster environment, I discussed that before with Oleg, and we agreed this RA should be temp and a new RA should come from a "standardized" RA Type with Open Cloud. You could try to fix this but I fear it may be more work than expected, the biggest issues I identified quickly are:
If you could let me know how can we go about it, would pitch in to get it standardized.
- handles have activities as non transient fields, when serialized the activity is serialized too
- activities have some fields which should be transient (by defualt only fields marked as transient and static are not serialized) , for instance the Timer and the ResourceAdaptor, which means currently the whole RA object is being serialized!! The RA is easy to solve, simply mark it as transient and make a setter which is invoked on every related RA method coming from network or SLEE (such as getActivity(ActivityHandle)), the Sip RA does that; the Timer is more a RA design issue, for instance a single RA timer should be used, and then it should be well defined what happens when an activity from node A (with a timer there) is accessed in node B, if that is a possible use case...
I was thinking of making most of the things transient and implement JGroups so that the members in cluster group communicate to each other with regards to their current liveliness state. And thus, the peer node would now when one a neighbor is dead and thus bind itself to SMSC. Would this be a good approach?
2) Both nodes being Head of cluster - this could be from the cluster not starting correctly, either because the nodes don't connect at all (thus you have 2 clusters with a single node) , which is something I have witness myself one time and seems to originate in jgroups or jboss cache 3; or it could be the startup of the nodes at same time where the RA code runs before the nodes actually link. My recommendation is to first completly load one node and then start the other, if the other scenario pops up please report.
Yes starting them one after another thus sort this. Thanks.
Raziya Bhayani
Jan 11, 2010, 11:17:59 PM1/11/10
to mobicent...@googlegroups.com
I understand that the JGroups have already been implemented, what I want to know is, is there a mechanism to notify the other node that the first one has gone down. I see isHeadMember(), in FaultTolerantResourceAdaptorContext, to know if current member is Head Member. When Head Member goes down, and the next member in the queue becomes a Head Member, is there a method / mechanism that would notify this transit of second member to Head member?
Thanks and regards
Raziya
Thanks and regards
Raziya
Raziya Bhayani
Jan 12, 2010, 3:02:22 AM1/12/10
to mobicent...@googlegroups.com
Hello Eduardo,
I have made changes making most of the fields transient, yet I dont get a stable outcome. Sometime it cribs for com/uwyn/jhighlight/renderer/ whose jar I have included in RA libs jar. Sometime it says StackOverflow, at other times, it says IOException. I get altogether different error everytime I run the Server. And AS hangsup printing error stream continually, when I start just one node of the two, and I have to kill it.
Cant get a consistent error or behaviour. Things go astray when the Activity is to be created to fire events. I am attaching log files for both the nodes.
Please advise.
Raziya
I have made changes making most of the fields transient, yet I dont get a stable outcome. Sometime it cribs for com/uwyn/jhighlight/renderer/ whose jar I have included in RA libs jar. Sometime it says StackOverflow, at other times, it says IOException. I get altogether different error everytime I run the Server. And AS hangsup printing error stream continually, when I start just one node of the two, and I have to kill it.
Cant get a consistent error or behaviour. Things go astray when the Activity is to be created to fire events. I am attaching log files for both the nodes.
Please advise.
Thanks and regards.
Raziya
Eduardo Martins
Jan 12, 2010, 8:23:27 AM1/12/10
to mobicent...@googlegroups.com
No, that is not in the cluster framework yet, but the FaulttolerantResourceAdaptorContext gives you acess to replicated data with failover, and there, if you're Head, you can store something that represents being owner of the link, and when the nodes goes down you will get a failOver(key) callback, in the RA object of the node elected to do the failover.
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
Eduardo Martins
Jan 12, 2010, 8:24:11 AM1/12/10
to mobicent...@googlegroups.com
I will have a look, but not sure it will be in the next few days due to final tests of SLEE 2.0.0.CR1, just be patient :-)
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
Sachin Parnami
Jan 12, 2010, 8:34:09 AM1/12/10
to mobicent...@googlegroups.com
On Tue, Jan 12, 2010 at 6:53 PM, Eduardo Martins <emma...@gmail.com> wrote:
No, that is not in the cluster framework yet, but the FaulttolerantResourceAdaptorContext gives you acess to replicated data with failover, and there, if you're Head, you can store something that represents being owner of the link, and when the nodes goes down you will get a failOver(key) callback, in the RA object of the node elected to do the failover.
Can you please explain
>>the RA object of the node elected to do the failover.
How do we specify that? if i have two RA node(Node1 and Node2) implementing faulttolerant api, and if i bring one of the node down(Node1) will the other node's(Node2) failover() gets called ?
Tried but but it doesn't happen.
Tried but but it doesn't happen.
--
Regards,
Sachin Parnami
Sachin Parnami
Jan 12, 2010, 10:02:42 AM1/12/10
to mobicent...@googlegroups.com
Thanks Eduardo
FaultTolerant API works fine i had to put serializable object in ReplicateDataWithFailOver :) , after that i see failover() gets call @Node2 if Node1 dies for some reason.--
Regards,
Sachin Parnami
FaultTolerant API works fine i had to put serializable object in ReplicateDataWithFailOver :) , after that i see failover() gets call @Node2 if Node1 dies for some reason.
Regards,
Sachin Parnami
Eduardo Martins
Jan 12, 2010, 11:25:54 AM1/12/10
to mobicent...@googlegroups.com
Yep, that's it.
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
Raziya Bhayani
Jan 13, 2010, 12:54:44 AM1/13/10
to mobicent...@googlegroups.com
Eduardo,
I finally see failOver() being called. However I was stuck for pretty long time.
What I was doing was, I was checking if current node is a headMember() if it is, then try and bind to SMSC and after which I was calling,
ftRAContext.getReplicatedDataWithFailover().put("DummyKey", "DummyValue")
I could not see failOver() being called at the other node, when the headMember was crashing. I moved this line of code in raActive() and BINGO it works well.
What exactly is the cause of this behaviour? and how exactly does ReplicatedData functionality work?
Thanks and regards,
Raziya
I finally see failOver() being called. However I was stuck for pretty long time.
What I was doing was, I was checking if current node is a headMember() if it is, then try and bind to SMSC and after which I was calling,
ftRAContext.getReplicatedDataWithFailover().put("DummyKey", "DummyValue")
I could not see failOver() being called at the other node, when the headMember was crashing. I moved this line of code in raActive() and BINGO it works well.
What exactly is the cause of this behaviour? and how exactly does ReplicatedData functionality work?
Thanks and regards,
Raziya
Raziya Bhayani
Jan 13, 2010, 11:14:09 AM1/13/10
to mobicent...@googlegroups.com
Hello Eduardo,
While trying to bring up two nodes, in clustered mode, i got following error. Attaching log files for both the nodes.
21:28:10,769 ERROR [AbstractKernelController] Error installing to Instantiated: name=Mobicents.JAINSLEE.Container state=Described
org.jboss.cache.lock.TimeoutException: Replication timeout for 192.168.0.101:53885
at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:768)
at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:716)
at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:721)
at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:161)
at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:135)
at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:107)
at org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:160)
at org.jboss.cache.interceptors.ReplicationInterceptor.visitPutDataMapCommand(ReplicationInterceptor.java:113)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitPutDataMapCommand(AbstractVisitor.java:60)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301)
at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283)
at org.jboss.cache.commands.AbstractVisitor.visitPutDataMapCommand(AbstractVisitor.java:60)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.CacheMgmtInterceptor.visitPutDataMapCommand(CacheMgmtInterceptor.java:97)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at org.jboss.cache.interceptors.InvocationContextInterceptor.visitPutDataMapCommand(InvocationContextInterceptor.java:64)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287)
at org.jboss.cache.invocation.CacheInvocationDelegate.invokePut(CacheInvocationDelegate.java:705)
at org.jboss.cache.invocation.CacheInvocationDelegate.put(CacheInvocationDelegate.java:519)
at org.jboss.cache.invocation.NodeInvocationDelegate.addChild(NodeInvocationDelegate.java:337)
at org.mobicents.cache.CacheData.create(CacheData.java:44)
at org.mobicents.slee.runtime.activity.ActivityContextFactoryImpl.<init>(ActivityContextFactoryImpl.java:38)
at org.mobicents.slee.container.SleeContainer.<init>(SleeContainer.java:243)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
at org.jboss.reflect.plugins.introspection.ReflectionUtils.newInstance(ReflectionUtils.java:149)
at org.jboss.reflect.plugins.introspection.ReflectConstructorInfoImpl.newInstance(ReflectConstructorInfoImpl.java:106)
at org.jboss.joinpoint.plugins.BasicConstructorJoinPoint.dispatch(BasicConstructorJoinPoint.java:80)
at org.jboss.aop.microcontainer.integration.AOPConstructorJoinpoint.createTarget(AOPConstructorJoinpoint.java:282)
at org.jboss.aop.microcontainer.integration.AOPConstructorJoinpoint.dispatch(AOPConstructorJoinpoint.java:103)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction$JoinpointDispatchWrapper.execute(KernelControllerContextAction.java:241)
at org.jboss.kernel.plugins.dependency.ExecutionWrapper.execute(ExecutionWrapper.java:47)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchExecutionWrapper(KernelControllerContextAction.java:109)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchJoinPoint(KernelControllerContextAction.java:70)
at org.jboss.kernel.plugins.dependency.InstantiateAction.installActionInternal(InstantiateAction.java:66)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:54)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:42)
at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:774)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:540)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:121)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:51)
at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
at org.jboss.system.server.profileservice.repository.ProfileDeployAction.install(ProfileDeployAction.java:70)
at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.java:306)
at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
at org.jboss.Main.boot(Main.java:221)
at org.jboss.Main$1.run(Main.java:556)
at java.lang.Thread.run(Thread.java:595)
Thanks and regards,
Raziya
While trying to bring up two nodes, in clustered mode, i got following error. Attaching log files for both the nodes.
21:28:10,769 ERROR [AbstractKernelController] Error installing to Instantiated: name=Mobicents.JAINSLEE.Container state=Described
org.jboss.cache.lock.TimeoutException: Replication timeout for 192.168.0.101:53885
at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:768)
at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:716)
at org.jboss.cache.RPCManagerImpl.callRemoteMethods(RPCManagerImpl.java:721)
at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:161)
at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:135)
at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:107)
at org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:160)
at org.jboss.cache.interceptors.ReplicationInterceptor.visitPutDataMapCommand(ReplicationInterceptor.java:113)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at org.jboss.cache.commands.AbstractVisitor.visitPutDataMapCommand(AbstractVisitor.java:60)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.TxInterceptor.attachGtxAndPassUpChain(TxInterceptor.java:301)
at org.jboss.cache.interceptors.TxInterceptor.handleDefault(TxInterceptor.java:283)
at org.jboss.cache.commands.AbstractVisitor.visitPutDataMapCommand(AbstractVisitor.java:60)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.CacheMgmtInterceptor.visitPutDataMapCommand(CacheMgmtInterceptor.java:97)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at org.jboss.cache.interceptors.InvocationContextInterceptor.visitPutDataMapCommand(InvocationContextInterceptor.java:64)
at org.jboss.cache.commands.write.PutDataMapCommand.acceptVisitor(PutDataMapCommand.java:104)
at org.jboss.cache.interceptors.InterceptorChain.invoke(InterceptorChain.java:287)
at org.jboss.cache.invocation.CacheInvocationDelegate.invokePut(CacheInvocationDelegate.java:705)
at org.jboss.cache.invocation.CacheInvocationDelegate.put(CacheInvocationDelegate.java:519)
at org.jboss.cache.invocation.NodeInvocationDelegate.addChild(NodeInvocationDelegate.java:337)
at org.mobicents.cache.CacheData.create(CacheData.java:44)
at org.mobicents.slee.runtime.activity.ActivityContextFactoryImpl.<init>(ActivityContextFactoryImpl.java:38)
at org.mobicents.slee.container.SleeContainer.<init>(SleeContainer.java:243)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
at org.jboss.reflect.plugins.introspection.ReflectionUtils.newInstance(ReflectionUtils.java:149)
at org.jboss.reflect.plugins.introspection.ReflectConstructorInfoImpl.newInstance(ReflectConstructorInfoImpl.java:106)
at org.jboss.joinpoint.plugins.BasicConstructorJoinPoint.dispatch(BasicConstructorJoinPoint.java:80)
at org.jboss.aop.microcontainer.integration.AOPConstructorJoinpoint.createTarget(AOPConstructorJoinpoint.java:282)
at org.jboss.aop.microcontainer.integration.AOPConstructorJoinpoint.dispatch(AOPConstructorJoinpoint.java:103)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction$JoinpointDispatchWrapper.execute(KernelControllerContextAction.java:241)
at org.jboss.kernel.plugins.dependency.ExecutionWrapper.execute(ExecutionWrapper.java:47)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchExecutionWrapper(KernelControllerContextAction.java:109)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchJoinPoint(KernelControllerContextAction.java:70)
at org.jboss.kernel.plugins.dependency.InstantiateAction.installActionInternal(InstantiateAction.java:66)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:54)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:42)
at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:774)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:540)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:121)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:51)
at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
at org.jboss.system.server.profileservice.repository.ProfileDeployAction.install(ProfileDeployAction.java:70)
at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.java:306)
at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
at org.jboss.Main.boot(Main.java:221)
at org.jboss.Main$1.run(Main.java:556)
at java.lang.Thread.run(Thread.java:595)
Thanks and regards,
Raziya
Eduardo Martins
Jan 13, 2010, 7:51:10 PM1/13/10
to mobicent...@googlegroups.com
We are actually noticing that in JAIN SLEE 2.0 CR1 tests so I will come back on this one later, this happens everytime you start the second node?
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
Eduardo Martins
Jan 13, 2010, 7:52:09 PM1/13/10
to mobicent...@googlegroups.com
You now have it in raActive(), where was it before?
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
-- Eduardo
-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-:-
http://emmartins.blogspot.com
http://www.redhat.com/solutions/telco
Raziya Bhayani
Jan 13, 2010, 9:55:00 PM1/13/10
to mobicent...@googlegroups.com
No not every time. It just popped up for the first time.
Raziya Bhayani
Jan 13, 2010, 10:09:43 PM1/13/10
to mobicent...@googlegroups.com
It was in bindSMSC() which is called from raActive(), but in bindSMSC() it was within the block where it use to check ifHeadMember(){} and hence the consequences.
Before:
raActive(){
....
bindSMSC()
.....
.....
}
bindSMSC(){
if(ftRAContext.isHeadMember()){
//Bind to SMSC
ftRAContext.getReplicatedDataWithFailover().put("DummyKey", "DummyValue"); // So if second node starts, it would not be able to put this key value (since not headMember). Thus failOver() was not being called. Which was the issue I believe. Please correct me if I am wrong.
}
}
Now:
raActive(){
....
ftRAContext.getReplicatedDataWithFailover().put("DummyKey", "DummyValue");
bindSMSC()
.....
.....
}
bindSMSC(){
if(ftRAContext.isHeadMember()){
//bind to SMSC
}
}
Thanks and regards.
Raziya
Before:
raActive(){
....
bindSMSC()
.....
.....
}
bindSMSC(){
if(ftRAContext.isHeadMember()){
//Bind to SMSC
ftRAContext.getReplicatedDataWithFailover().put("DummyKey", "DummyValue"); // So if second node starts, it would not be able to put this key value (since not headMember). Thus failOver() was not being called. Which was the issue I believe. Please correct me if I am wrong.
}
}
Now:
raActive(){
....
ftRAContext.getReplicatedDataWithFailover().put("DummyKey", "DummyValue");
bindSMSC()
.....
.....
}
bindSMSC(){
if(ftRAContext.isHeadMember()){
//bind to SMSC
}
}
Thanks and regards.
Raziya
Reply all
Reply to author
Forward
0 new messages