SimpleLogListener and RotateLogListener threads in blocked status
Prajwal Shetty
We recently are facing issues wherein the ISO messages responses were sent with a delay(upto 5 seconds). We use jpos 2.0.3.
Post checking the thread dumps, we found that there were multiple threads blocked while logging.
The thread dumps were as follows,
Thread holding lock
"Thread-12" #61 prio=5 os_prio=0 tid=0x00002b3788055000 nid=0x566f runnable [0x00002b37870e9000]
java.lang.Thread.State: RUNNABLE
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
- locked <0x00000000d577f478> (a java.io.BufferedOutputStream)
at java.io.PrintStream.write(PrintStream.java:480)
- locked <0x00000000d5779188> (a java.io.PrintStream)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
- locked <0x00000000d577f490> (a java.io.OutputStreamWriter)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.newLine(PrintStream.java:546)
- locked <0x00000000d5779188> (a java.io.PrintStream)
at java.io.PrintStream.println(PrintStream.java:807)
- locked <0x00000000d5779188> (a java.io.PrintStream)
at org.jpos.util.LogEvent.dump(LogEvent.java:140)
at org.jpos.util.SimpleLogListener.log(SimpleLogListener.java:51)
- locked <0x00000000d5775368> (a org.jpos.util.SimpleLogListener)
at org.jpos.util.Logger.log(Logger.java:87)
at org.jpos.util.Log.info(Log.java:81)
at trivnet.jpos.q2.handler.OutgoingResponseHandler.convertMapToMsg(OutgoingResponseHandler.java:83)
at trivnet.jpos.q2.handler.OutgoingHandler.process(OutgoingHandler.java:50)
at trivnet.jpos.q2.bean.OutgoingProcessor.run(OutgoingProcessor.java:146)
at java.lang.Thread.run(Thread.java:745)
Threads Waiting on monitor,
"Thread-10" #59 prio=5 os_prio=0 tid=0x00002b3788077800 nid=0x566d waiting for monitor entry [0x00002b3786ee7000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.jpos.util.SimpleLogListener.log(SimpleLogListener.java:50)
- waiting to lock <0x00000000d5775368> (a org.jpos.util.SimpleLogListener)
at org.jpos.util.Logger.log(Logger.java:87)
at org.jpos.util.Log.info(Log.java:81)
at trivnet.jpos.q2.bean.OutgoingProcessor.run(OutgoingProcessor.java:121)
at java.lang.Thread.run(Thread.java:745)
"ExecuteThread: '0' for queue: 'default'" #36 daemon prio=5 os_prio=0 tid=0x000000001e21b800 nid=0x546b waiting for monitor entry [0x00002b37850d3000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.jpos.util.SimpleLogListener.log(SimpleLogListener.java:50)
- waiting to lock <0x00000000d5775368> (a org.jpos.util.SimpleLogListener)
at org.jpos.util.Logger.log(Logger.java:87)
at trivnet.jpos.q2.bean.QJMSReceiver.onMessage(QJMSReceiver.java:227)
at weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4585)
at weblogic.jms.client.JMSSession.execute(JMSSession.java:4271)
at weblogic.jms.client.JMSSession.executeMessage(JMSSession.java:3747)
at weblogic.jms.client.JMSSession.access$000(JMSSession.java:114)
at weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:5096)
at weblogic.work.ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)
at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
Did anyone face the same issue before?
I checked for disk space issue but there wasn't any.
Thanks.
Alejandro Revilla
--
--
jPOS is licensed under AGPL - free for community usage for your open-source project. Licenses are also available for commercial usage. Please support jPOS, contact: sa...@jpos.org
Join us in IRC at http://webchat.freenode.net/?channels=jpos
You received this message because you are subscribed to the "jPOS Users" group.
Please see http://jpos.org/wiki/JPOS_Mailing_List_Readme_first
To post to this group, send email to jpos-...@googlegroups.com
To unsubscribe, send email to jpos-users+...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/jpos-users
---
You received this message because you are subscribed to the Google Groups "jPOS Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jpos-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jpos-users/5ddbc39c-8764-480e-8ac3-3404967974d2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Prajwal Shetty
We will check the gc logs to be sure and update you.
Thanks.
Prajwal Shetty
On Thursday, March 31, 2016 at 7:22:51 PM UTC-6, Alejandro Revilla wrote:
Alejandro Revilla
Prajwal Shetty
We use the following jvm options,
-server -Xloggc:log/gc.log -Xms1024m -Xmx1024m -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -agentpath:$DY_AGENT=name=$DY_NAME,server=$DY_SERVER -Djava.net.preferIPv4Stack=true -Dcom.sun.management.jmxremote -Djava.security.egd=file:/dev/./urandom -jar
The agents used are for profiling app.
We reviewed the GC details during the time of issue and found no unusual behavior there.
Any other points apart from this to consider?
Thanks.
Thanks