Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Out of memory error on WebSphere 6.0.2.21 - IBM JRE (build 1.4.2, SR10)

47 views
Skip to first unread message

Srikant Tallapragada

unread,
Jan 21, 2009, 1:34:56 AM1/21/09
to
Hi

I have installed and configured WebSphere 6.0.2.21 on AIX 5.3 ML8 - 2 CPU.

I have set the initial heap size as 1024M and Max heap size as 2048M. After every 2-3 days I keep getting this error and the server stops responding. I have to restart the WAS instance again and a huge core file and a javacore file are created. There are a couple of J2EE apps deployed on the server and there are currently no users accessing this server. Also, there are no other processes or programs that are hitting this server. Any help is appreciated. Here is the error from the SystemErr.log. I am also attaching the javacore file that gets generated everytime this error occurs.

[1/7/09 18:24:39:171 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.hamanager.runtime.CoordinatorComponentImpl.initialize(CoordinatorComponentImpl.java:193)
[1/7/09 18:24:39:172 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.runtime.component.ContainerImpl.initializeComponent(ContainerImpl.java:1160)
[1/7/09 18:24:39:172 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.runtime.component.ContainerImpl.initializeComponents(ContainerImpl.java:979)
[1/7/09 18:24:39:172 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.runtime.component.ServerImpl.initialize(ServerImpl.java:284)
[1/7/09 18:24:39:172 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.runtime.WsServerImpl.bootServerContainer(WsServerImpl.java:173)
[1/7/09 18:24:39:172 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.runtime.WsServerImpl.start(WsServerImpl.java:133)
[1/7/09 18:24:39:173 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.runtime.WsServerImpl.main(WsServerImpl.java:387)
[1/7/09 18:24:39:173 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.runtime.WsServer.main(WsServer.java:53)
[1/7/09 18:24:39:173 GMT+05:30] 0000000a SystemErr R at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[1/7/09 18:24:39:173 GMT+05:30] 0000000a SystemErr R at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
[1/7/09 18:24:39:173 GMT+05:30] 0000000a SystemErr R at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
[1/7/09 18:24:39:174 GMT+05:30] 0000000a SystemErr R at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:60)
[1/7/09 18:24:39:174 GMT+05:30] 0000000a SystemErr R at java.lang.reflect.Method.invoke(Method.java:391)
[1/7/09 18:24:39:174 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.bootstrap.WSLauncher.run(WSLauncher.java:219)
[1/7/09 18:24:39:174 GMT+05:30] 0000000a SystemErr R at java.lang.Thread.run(Thread.java:571)
[1/7/09 18:24:39:174 GMT+05:30] 0000000a SystemErr R Caused by: java.lang.OutOfMemoryError: JVMCI015:OutOfMemoryError, cannot create anymore threads due to memory or resource constraints
[1/7/09 18:24:39:175 GMT+05:30] 0000000a SystemErr R at java.lang.Thread.start(Native Method)
[1/7/09 18:24:39:175 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.hamanager.impl.UserCallbacks.initialize(UserCallbacks.java:159)
[1/7/09 18:24:39:175 GMT+05:30] 0000000a SystemErr R at com.ibm.ws.hamanager.runtime.CoordinatorComponentImpl.initialize(CoordinatorComponentImpl.java:182)
[1/7/09 18:24:39:175 GMT+05:30] 0000000a SystemErr R ... 14 more
[1/7/09 18:25:54:119 GMT+05:30] 0000000a SystemErr R com.ibm.ws.exception.ConfigurationError: com.ibm.ws.exception.ConfigurationError: unable to initialize the CoordinatorComponentImpl
at com.ibm.ws.runtime.WsServerImpl.bootServerContainer(WsServerImpl.java:180)
at com.ibm.ws.runtime.WsServerImpl.start(WsServerImpl.java:133)
at com.ibm.ws.runtime.WsServerImpl.main(WsServerImpl.java:387)
at com.ibm.ws.runtime.WsServer.main(WsServer.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:60)
at java.lang.reflect.Method.invoke(Method.java:391)
at com.ibm.ws.bootstrap.WSLauncher.run(WSLauncher.java:219)
at java.lang.Thread.run(Thread.java:571)
Caused by: com.ibm.ws.exception.ConfigurationError: unable to initialize the CoordinatorComponentImpl
at com.ibm.ws.hamanager.runtime.CoordinatorComponentImpl.initialize(CoordinatorComponentImpl.java:193)
at com.ibm.ws.runtime.component.ContainerImpl.initializeComponent(ContainerImpl.java:1160)
at com.ibm.ws.runtime.component.ContainerImpl.initializeComponents(ContainerImpl.java:979)
at com.ibm.ws.runtime.component.ServerImpl.initialize(ServerImpl.java:284)
at com.ibm.ws.runtime.WsServerImpl.bootServerContainer(WsServerImpl.java:173)
... 10 more
Caused by: java.lang.OutOfMemoryError: JVMCI015:OutOfMemoryError, cannot create anymore threads due to memory or resource constraints
at java.lang.Thread.start(Native Method)
at com.ibm.ws.hamanager.impl.UserCallbacks.initialize(UserCallbacks.java:159)
at com.ibm.ws.hamanager.runtime.CoordinatorComponentImpl.initialize(CoordinatorComponentImpl.java:182)
... 14 more

-Thanks
Srikant

Srikant Tallapragada

unread,
Jan 21, 2009, 1:49:03 AM1/21/09
to
WebSphere version is 6.0.2.31 and not 21 as mentioned earlier.

namdev...@yahoo.com

unread,
Jan 21, 2009, 1:40:48 PM1/21/09
to
Try updating your SDK to 1.4.2 SR11 Cumulative Fix. This is the SDK that is shipped with WAS 6.0.2.31

karun.su...@esisconsulting.com

unread,
Jan 23, 2009, 3:16:19 PM1/23/09
to
Your thread dump seems to be partial. Are you sure this file is not truncated. I should see an "END OF DUMP" string at the last line of the dump.

Looking at the thread dump, wow, you have thousands of thread created. The Application runs out of "Native" Memory and dumps core. This is not your HEAP. This is native memory which is only limited by the OS.

There is good possibility that somewhere in the app you are running into a infinite loop, or you are opening something but never closes it.

Check out the code. Also once you have a good dump, post it here - I'll take a look at it. You don't need to wait for the app to crash. YOu can execute "kill -3 " to genereate the core dump (do this after the app server runs for about a day or two)

- Karun

David Yang

unread,
Jan 27, 2009, 3:34:05 PM1/27/09
to
Hello,
With Java maximum heap size 2048 MB and message *java.lang.OutOfMemoryError: JVMCI015:OutOfMemoryError, cannot create anymore threads due to memory or resource constraints*, you are probably experiencing native heap OOM exhaustion.

See *Webcast replay: Introduction to the Memory Usage of a Java Process* at http://www.ibm.com/support/docview.wss?rs=1045&uid=swg27013757.

For example, the *IBM WebSphere Portal Version 6.0 Tuning Guide* (which uses WAS 6.0.2.x) at http://www.ibm.com/support/docview.wss?rs=688&uid=swg27008511 says they used 1792 MB for the highest Java maximum heap size on AIX 5.

Thank you,
David Yang
WebSphere Portal support

The postings on this site are my own and do not necessarily represent the positions, strategies, or opinions of IBM.

Ben_

unread,
Jan 27, 2009, 3:57:19 PM1/27/09
to
> OutOfMemoryError, cannot create anymore threads

Each thread needs its own piece of native memory (which is different from
the Java heap controlled by the maximum heap size setting).

See also the IBM JVM Diagnostic Guide.

http://www.ibm.com/developerworks/java/jdk/diagnosis/

See also an old discussion on the subject:
http://groups.google.com/group/comp.lang.java.machine/browse_thread/thread/bd8c8229ce19ffd7.

0 new messages