Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

WAS crashes when running large number of publishers and subscribers

94 views
Skip to first unread message

Shashikanth Rao T

unread,
May 28, 2009, 3:11:48 AM5/28/09
to
Hi all,

I am doing stress test on my client connecting to WAS 6.1. I use XMS
C/C++ in my client application to do Pub/Sub with WAS. I am running a
publisher application with 50 threads publishing on one topic and have
another subscriber application with 50 threads subscribing to the same
topic. I attempt to publish 100000 messages (20 bytes size).

The applications work for 30 minutes or so and then they close with
Connection broken exception. I could see that WAS has stopped with lot
of ffdc created. I see many out of memory exceptions (as shown below).

Any help on WAS crash will be highly appreciated.

Thanks

===========
------Start of DE processing------ = [5/28/09 11:47:42:750 IST] , key =
com.ibm.ws.sib.msgstore.PersistenceException
com.ibm.ws.sib.msgstore.persistence.impl.SpillDispatcher.writeBatch
1:1435:1.30
Exception = com.ibm.ws.sib.msgstore.PersistenceException
Source = com.ibm.ws.sib.msgstore.persistence.impl.SpillDispatcher.writeBatch
probeid = 1:1435:1.30
Stack Dump = com.ibm.ws.sib.msgstore.PersistenceException: CWSIS1501E:
The data source has produced an unexpected exception:
java.sql.BatchUpdateException: Java exception: ':
java.lang.OutOfMemoryError'.
at
com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl.executeBatch(BatchingContextImpl.java:278)
at
com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatcherThread.writeBatch(SpillDispatcher.java:1388)
at
com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatcherThread.run(SpillDispatcher.java:800)
at java.lang.Thread.run(Thread.java:797)
Caused by: java.sql.BatchUpdateException: Java exception: ':
java.lang.OutOfMemoryError'.
at org.apache.derby.impl.jdbc.EmbedStatement.executeBatch(Unknown Source)
at
com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecuteBatch(WSJdbcPreparedStatement.java:941)
at
com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.executeBatch(WSJdbcStatement.java:705)
at
com.ibm.ws.sib.msgstore.persistence.impl.ItemTable.insert(ItemTable.java:450)
at
com.ibm.ws.sib.msgstore.persistence.impl.TableManager.insert(TableManager.java:687)
at
com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl$1.run(BatchingContextImpl.java:129)
at
com.ibm.ws.sib.msgstore.persistence.impl.DatasourceController.performAction(DatasourceController.java:411)
at
com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl.executeBatch(BatchingContextImpl.java:266)
... 3 more
Caused by: SQL Exception: Java exception: ': java.lang.OutOfMemoryError'.
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
Source)
at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeBatchElement(Unknown
Source)
... 11 more

Dump of callerThis =
Object type =
com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatcherThread
com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatcherThread@1ab61ab6
=========

------Start of DE processing------ = [5/28/09 11:51:45:515 IST] , key =
java.lang.OutOfMemoryError com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm 66
Exception = java.lang.OutOfMemoryError
Source = com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm
probeid = 66
Stack Dump = java.lang.OutOfMemoryError
at java.security.AccessController.getProtectionDomains(Native Method)
at java.security.AccessController.getContext(AccessController.java:132)
at java.lang.Thread.initialize(Thread.java:342)
at java.lang.Thread.<init>(Thread.java:274)
at java.lang.Thread.<init>(Thread.java:183)
at com.ibm.ws.util.ThreadPool$Worker.<init>(ThreadPool.java:1453)
at com.ibm.ws.util.ThreadPool.addThread(ThreadPool.java:616)
at com.ibm.ws.util.ThreadPool$3.run(ThreadPool.java:1125)
at
com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:63)
at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:1123)
at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:988)
at com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm(AlarmThreadPool.java:70)
at com.ibm.ejs.util.am.AlarmManagerThread.run(AlarmManagerThread.java:122)
at java.lang.Thread.run(Thread.java:797)

Dump of callerThis =
Object type = com.ibm.ejs.util.am.AlarmThreadPool
com.ibm.ejs.util.am.AlarmThreadPool@28f428f4

===========

noor

unread,
Jun 5, 2009, 9:58:03 AM6/5/09
to
> com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl.executeBatch(B­atchingContextImpl.java:278)
>         at
> com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatc­herThread.writeBatch(SpillDispatcher.java:1388)
>         at
> com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatc­herThread.run(SpillDispatcher.java:800)

>         at java.lang.Thread.run(Thread.java:797)
> Caused by: java.sql.BatchUpdateException: Java exception: ':
> java.lang.OutOfMemoryError'.
>         at org.apache.derby.impl.jdbc.EmbedStatement.executeBatch(Unknown Source)
>         at
> com.ibm.ws.rsadapter.jdbc.WSJdbcPreparedStatement.pmiExecuteBatch(WSJdbcPre­paredStatement.java:941)
>         at
> com.ibm.ws.rsadapter.jdbc.WSJdbcStatement.executeBatch(WSJdbcStatement.java­:705)
>         at
> com.ibm.ws.sib.msgstore.persistence.impl.ItemTable.insert(ItemTable.java:45­0)
>         at
> com.ibm.ws.sib.msgstore.persistence.impl.TableManager.insert(TableManager.j­ava:687)
>         at
> com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl$1.run(Batching­ContextImpl.java:129)
>         at
> com.ibm.ws.sib.msgstore.persistence.impl.DatasourceController.performAction­(DatasourceController.java:411)
>         at
> com.ibm.ws.sib.msgstore.persistence.impl.BatchingContextImpl.executeBatch(B­atchingContextImpl.java:266)

>         ... 3 more
> Caused by: SQL Exception: Java exception: ': java.lang.OutOfMemoryError'.
>         at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
>         at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
>         at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
>         at
> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unkno­wn

> Source)
>         at
> org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
> Source)
>         at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
> Source)
>         at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
> Source)
>         at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
> Source)
>         at
> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeBatchElement(Unkno­wn

> Source)
>         ... 11 more
>
> Dump of callerThis =
> Object type =
> com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatc­herThread
> com.ibm.ws.sib.msgstore.persistence.dispatcher.SpillDispatcher$SpillDispatc­herThread@1ab61ab6

> =========
>
> ------Start of DE processing------ = [5/28/09 11:51:45:515 IST] , key =
> java.lang.OutOfMemoryError com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm 66
> Exception = java.lang.OutOfMemoryError
> Source = com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm
> probeid = 66
> Stack Dump = java.lang.OutOfMemoryError
>         at java.security.AccessController.getProtectionDomains(Native Method)
>         at java.security.AccessController.getContext(AccessController.java:132)
>         at java.lang.Thread.initialize(Thread.java:342)
>         at java.lang.Thread.<init>(Thread.java:274)
>         at java.lang.Thread.<init>(Thread.java:183)
>         at com.ibm.ws.util.ThreadPool$Worker.<init>(ThreadPool.java:1453)
>         at com.ibm.ws.util.ThreadPool.addThread(ThreadPool.java:616)
>         at com.ibm.ws.util.ThreadPool$3.run(ThreadPool.java:1125)
>         at
> com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.jav­a:63)

>         at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:1123)
>         at com.ibm.ws.util.ThreadPool.execute(ThreadPool.java:988)
>         at com.ibm.ejs.util.am.AlarmThreadPool.fireAlarm(AlarmThreadPool.java:70)
>         at com.ibm.ejs.util.am.AlarmManagerThread.run(AlarmManagerThread.java:122)
>         at java.lang.Thread.run(Thread.java:797)
>
> Dump of callerThis =
> Object type = com.ibm.ejs.util.am.AlarmThreadPool
> com.ibm.ejs.util.am.AlarmThreadPool@28f428f4
>
> ===========

increase the Heap Size on WAS servers ,

0 new messages