I am doing stress test on my client connecting to WAS 6.1. I use XMS
C/C++ in my client application to do Pub/Sub with WAS. I am running a
publisher application with 50 threads publishing on one topic and have
another subscriber application with 50 threads subscribing to the same
topic. I attempt to publish 100000 messages (20 bytes size).
The applications work for 30 minutes or so and then they close with
Connection broken exception. I could see that WAS has stopped with lot
of ffdc created. I see many out of memory exceptions (as shown below).
Any help on WAS crash will be highly appreciated.
Thanks
===========
------Start of DE processing------ = [5/28/09 11:47:42:750 IST] , key =
com.ibm.ws.sib.msgstore.PersistenceException
com.ibm.ws.sib.msgstore.persistence.impl.SpillDispatcher.writeBatch
1:1435:1.30
Exception = com.ibm.ws.sib.msgstore.PersistenceException
Source = com.ibm.ws.sib.msgstore.persistence.impl.SpillDispatcher.writeBatch
probeid = 1:1435:1.30
Stack Dump = com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E:
The data source has produced an unexpected exception:
java.sql.BatchUpdateException: Java exception: ':
java.lang.OutOfMemoryError'.
at
com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl.executeBatch(BatchingContextImpl.java:278)
at
com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatcherThread.writeBatch(SpillDispatcher.java:1388)
at
com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatcherThread.run(SpillDispatcher.java:800)
at java.lang.Thread.run(Thread.java:797)
Caused by: java.sql.BatchUpdateException: Java exception: ':
java.lang.OutOfMemoryError'.
at org.apache.derby.impl.jdbc.EmbedStatement.executeBatch(Unknown Source)
at
com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecuteBatch(WSJdbcPreparedStatement.java:941)
at
com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.executeBatch(WSJdbcStatement.java:705)
at
com.ibm.ws.sib.msgstore.persistence.impl.ItemTable.insert(ItemTable.java:450)
at
com.ibm.ws.sib.msgstore.persistence.impl.TableManager.insert(TableManager.java:687)
at
com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl$1.run(BatchingContextImpl.java:129)
at
com.ibm.ws.sib.msgstore.persistence.impl.DatasourceController.performAction(DatasourceController.java:411)
at
com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl.executeBatch(BatchingContextImpl.java:266)
... 3 more
Caused by: SQL Exception: Java exception: ': java.lang.OutOfMemoryError'.
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeBatchElement(Unknown
Source)
... 11 more
Dump of callerThis =
Object type =
com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatcherThread
com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatcherThread@1ab61ab6
=========
------Start of DE processing------ = [5/28/09 11:51:45:515 IST] , key =
java.lang.OutOfMemoryError com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm 66
Exception = java.lang.OutOfMemoryError
Source = com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm
probeid = 66
Stack Dump = java.lang.OutOfMemoryError
at java.security.AccessController.getProtectionDomains(Native Method)
at java.security.AccessController.getContext(AccessController.java:132)
at java.lang.Thread.initialize(Thread.java:342)
at java.lang.Thread.<init>(Thread.java:274)
at java.lang.Thread.<init>(Thread.java:183)
at com.ibm.ws.util.ThreadPool$Worker.<init>(ThreadPool.java:1453)
at com.ibm.ws.util.ThreadPool.addThread(ThreadPool.java:616)
at com.ibm.ws.util.ThreadPool$3.run(ThreadPool.java:1125)
at
com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:63)
at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:1123)
at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:988)
at com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm(AlarmThreadPool.java:70)
at com.ibm.ejs.util.am.AlarmManagerThread.run(AlarmManagerThread.java:122)
at java.lang.Thread.run(Thread.java:797)
Dump of callerThis =
Object type = com.ibm.ejs.util.am.AlarmThreadPool
com.ibm.ejs.util.am.AlarmThreadPool@28f428f4
===========
increase the Heap Size on WAS servers ,