Hung Threads…

[MQS]

100) for any 1 hr time. During this time the server threads are getting into hung state and are not returning to the pool. We could see from the server out logs the number of threads in hung state and could not find any exception. This results in the application becoming unresponsive when the max limit for the Thread pool size is reached. At this point of time we need to restart the server and this makes the users feel annoyed as they were loosing the work that was in progress.

SystemOut.log: 0000001c ThreadMonitor
W WSVR0605W: Thread "WebContainer : 81" (000001e0) has been active for 646905
milliseconds and may be hung. There is/are 119 thread(s) in total in the server
that may be hung.

Is there any ways (like server settings or through code) available to make the hung threads return to the pool without restarting? Also please let me know is there any thing that we need to look on either on our code or on the server settings?

I am providing server settings information below:

Thread pool: minimumSize="50" maximumSize="200" inactivityTimeout="3500" isGrowable="false"

Connection pool: Connection timeout=180seconds Minimum connections=10 Maximum connections=70 Reap time=180seconds Unused timeout=1800seconds Aged timeout=0seconds Purge policy=EntirePool

Heap Size: initialHeapSize="512" maximumHeapSize="1024"

Any suggestion would be of great help to us and is highly appreciated.

Thanks & Regards,
NKarthikeyan

[Ken Hygh]

This is almost certainly a problem in the application code or downstream
components like databases or whatever the application connects to. A
java profiling tool is probably the fastest way to find what threads are
doing that are hung, you can also do a thread dump when a number of
threads are hung and see what's happening in them.

Ken

[jean-mar...@cnafmail.fr]

We have experimented such kind of troubles with one of our application.
Database access were made through SOAP access, and for one client access, application take one unit in the Webcontainer pool and did not release it, so each access reduce number of free pool until server freeze.

Database access have been replaced by jdbc access and solve our problem

[Ben_]

> Is there any ways (like server settings or through code) available to make
> the hung threads return to the pool without restarting?

If the thread is really blocked like a deadlock between two threads, there
is no way to "cancel" the operation and return to a sane state, except
restart.

It may also be that the thread is hanging for a long time but will finally
unblock. In this case, it naturally returns to a sane state.

> Also please let me know is there any thing that we need to look on either
> on our code or on the server settings?

Difficult to say what your application does.

Collect a thread dump (kill -3 pid) and correlate the thread name from the
warning with the thread name in the dump.

See Performance / Hangs in the Troubleshooting Guide:
http://www-01.ibm.com/support/docview.wss?rs=180&uid=swg27005324.

[rajeshs...@gmail.com]

Karthik,

Does your application server connects to third party server like as400
or only to a database.
can you see the time when the application falls and correlate with the
scheduled jobs during that time or a bit earlier.
Regards
Rajesh Saravanan