We are frequently getting the following error on Weblogic server 8.1.1.0. The server freezes every half hr as a result with very little load (5 user activity).
<Aug 19, 2004 12:28:13 AM PDT> <Error> <Cluster> <BEA-000117> <Received a stale
replication request for object –object id.>
BEA indicates that this error arises out of long pauses caused by GC or network problems: http://e-docs.bea.com/wls/docs81/messages/Cluster.html
We are working on migrating our application to Weblogic 8.1 (cluster) and using in-memory replication.
The memory sizing for the VM has been done as:
Admin Server:
-XX:MaxPermSize=64m -XX:NewSize=164m -XX:MaxNewSize=164m -Xmx512m -Xms512m
Managed Servers (Two):
-Xms512m -Xmx512m -XX:MaxPermSize=128m -XX:NewSize=164m -XX:MaxNewSize=164m
Any ideas/suggestions would be greatly appreciated!
Thanks
Abhishek
The thread dump under the unresponsive condition is as follows:
Full thread dump Java HotSpot(TM) Client VM (1.4.1_03-b02 mixed mode):
"ExecuteThread: '2' for queue: 'weblogic.kernel.Non-Blocking'" daemon prio=5 tid
=0x2E003B80 nid=0x778 waiting for monitor entry [2f16f000..2f16fd88]
at weblogic.servlet.internal.session.SessionData.setInternalAttribute(Se
ssionData.java:669)
- waiting to lock <0EFE3858> (a weblogic.servlet.internal.session.Replic
atedSessionData)
at weblogic.servlet.internal.session.ReplicatedSessionData.update(Replic
atedSessionData.java:256)
at weblogic.cluster.replication.ReplicationManager.update(ReplicationMan
ager.java:1257)
at weblogic.cluster.replication.ReplicationManager_WLSkel.invoke(Unknown
Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:466)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:409)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(Authenticate
dSubject.java:353)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:
144)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.jav
a:404)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest
java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
"ExecuteThread: '1' for queue: 'weblogic.kernel.Non-Blocking'" daemon prio=5 tid
=0x2DFF9410 nid=0x62c waiting for monitor entry [2f0ff000..2f0ffd88]
at weblogic.servlet.internal.session.SessionData.setInternalAttribute(Se
ssionData.java:669)
- waiting to lock <0AD30000> (a weblogic.servlet.internal.session.Replic
atedSessionData)
at weblogic.servlet.internal.session.ReplicatedSessionData.update(Replic
atedSessionData.java:256)
at weblogic.cluster.replication.ReplicationManager.update(ReplicationMan
ager.java:1257)
at weblogic.cluster.replication.ReplicationManager_WLSkel.invoke(Unknown
Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:466)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:409)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(Authenticate
dSubject.java:353)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:
144)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.jav
a:404)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest
java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
"ExecuteThread: '0' for queue: 'weblogic.kernel.Non-Blocking'" daemon prio=5 tid
=0x2C05BD58 nid=0x6bc waiting for monitor entry [2f08f000..2f08fd88]
at weblogic.servlet.internal.session.SessionData.setInternalAttribute(Se
ssionData.java:669)
- waiting to lock <0ED3AF98> (a weblogic.servlet.internal.session.Replic
atedSessionData)
at weblogic.servlet.internal.session.ReplicatedSessionData.update(Replic
atedSessionData.java:256)
at weblogic.cluster.replication.ReplicationManager.update(ReplicationMan
ager.java:1257)
at weblogic.cluster.replication.ReplicationManager_WLSkel.invoke(Unknown
Source)
at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:466)
at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:409)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(Authenticate
dSubject.java:353)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:
144)
at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.jav
a:404)
at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest
java:30)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:197)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:170)
"tavant.common.framework.scheduledTask.AttendanceMarkerTask" daemon prio=5 tid=0
x2BCB2170 nid=0x1ec waiting on condition [2f01f000..2f01fd88]
at java.lang.Thread.sleep(Native Method)
at tavant.common.framework.scheduledTask.Task.run(Task.java:118)
at java.lang.Thread.run(Thread.java:536)
"tavant.asa.core.AsaBackgroundTask" daemon prio=5 tid=0x2BC02198 nid=0x7e4 waiti
ng on condition [2efaf000..2efafd88]
at java.lang.Thread.sleep(Native Method)
at tavant.common.framework.scheduledTask.Task.run(Task.java:118)
at java.lang.Thread.run(Thread.java:536)
"tavant.common.framework.notification.MsgNotificationTask" daemon prio=5 tid=0x2
BE64E80 nid=0x758 waiting on condition [2ef3f000..2ef3fd88]
at java.lang.Thread.sleep(Native Method)
at tavant.common.framework.scheduledTask.Task.run(Task.java:118)
at java.lang.Thread.run(Thread.java:536)
"tavant.common.framework.scheduledTask.AttendanceVerifierTask" daemon prio=5 tid
=0x2BAB4AE0 nid=0x59c waiting on condition [2eecf000..2eecfd88]
at java.lang.Thread.sleep(Native Method)
at tavant.common.framework.scheduledTask.Task.run(Task.java:118)
at java.lang.Thread.run(Thread.java:536)
"SSLListenThread.Default" prio=5 tid=0x2BAFE8F8 nid=0x380 runnable [2ee2f000..2e
e2fd88]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
- locked <0D93A070> (a java.net.PlainSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:448)
at javax.net.ssl.impl.SSLServerSocketImpl.accept(Unknown Source)
at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:715)
at weblogic.t3.srvr.ListenThread.run(ListenThread.java:291)
"ListenThread.Default" prio=5 tid=0x2BA99A50 nid=0x710 runnable [2edef000..2edef
d88]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:353)
- locked <0D939E28> (a java.net.PlainSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:448)
at java.net.ServerSocket.accept(ServerSocket.java:419)
at weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java
:24)
at weblogic.t3.srvr.ListenThread.accept(ListenThread.java:715)
at weblogic.t3.srvr.ListenThread.run(ListenThread.java:291)
"TavantTask" daemon prio=5 tid=0x2AE00450 nid=0x724 waiting on condition [2ed7f0
00..2ed7fd88]
at java.lang.Thread.sleep(Native Method)
at tavant.common.framework.scheduledTask.TaskController.run(TaskControll
er.java:122)
"Thread-10" daemon prio=2 tid=0x2DF9C170 nid=0x690 in Object.wait() [2ed0f000..2
ed0fd88]
at java.lang.Object.wait(Native Method)
- waiting on <0D68C448> (a tavant.platform.logging.enterprise.appenders.
JMSAppender$1)
at tavant.platform.logging.enterprise.appenders.Connector.run(Connector.
java:53)
- locked <0D68C448> (a tavant.platform.logging.enterprise.appenders.JMSA
ppender$1)
"Dispatcher-Thread-9" daemon prio=2 tid=0x2DF182B8 nid=0x72c in Object.wait() [2
ec9f000..2ec9fd88]
at java.lang.Object.wait(Native Method)
- waiting on <0D67FDD0> (a org.apache.log4j.helpers.BoundedFIFO)
at java.lang.Object.wait(Object.java:426)
at org.apache.log4j.Dispatcher.run(AsyncAppender.java:293)
- locked <0D67FDD0> (a org.apache.log4j.helpers.BoundedFIFO)
"Thread-8" prio=5 tid=0x2AEFACA8 nid=0x548 in Object.wait() [2d8bf000..2d8bfd88]
at java.lang.Object.wait(Native Method)
- waiting on <0D5DD898> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:429)
- locked <0D5DD898> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:382)
"Thread-7" daemon prio=5 tid=0x2B604D68 nid=0x47c in Object.wait() [2d87f000..2d
87fd88]
at java.lang.Object.wait(Native Method)
- waiting on <0D579198> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:426)
at java.util.TimerThread.mainLoop(Timer.java:403)
- locked <0D579198> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:382)
"weblogic.health.CoreHealthMonitor" daemon prio=5 tid=0x2AFD9700 nid=0x868 waiti
ng on condition [2d83f000..2d83fd88]
at java.lang.Thread.sleep(Native Method)
at weblogic.t3.srvr.CoreHealthMonitorThread.run(CoreHealthMonitorThread.
java:126)
"Thread-6" prio=5 tid=0x2B7F8E20 nid=0x684 in Object.wait() [2d7ff000..2d7ffd88]
at java.lang.Object.wait(Native Method)
- waiting on <0D413B00> (a java.util.TaskQueue)
at java.util.TimerThread.mainLoop(Timer.java:429)
- locked <0D413B00> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:382)
"ExecuteThread: '0' for queue: 'Multicast'" daemon prio=5 tid=0x2AEE6D20 nid=0x8
b0 in Object.wait() [2d7bf000..2d7bfd88]
at java.lang.Object.wait(Native Method)
- waiting on <0D3E8010> (a
...
read more »
