ORB timeout and hung thread issues

[seem...@rediffmail.com]

We have the web tier and business tier of our application deployed on different WAS 6.1 server instances, there is no clustering as of now. We have stateless session beans in our business tier that are being invoked from the web tier. Now the EJB calls are failing with an ORB timeout error message in the web tier logs. At the same time, there is an error message in the business tier logs, saying that the ORB threads are hung. We tried increasing the ORB timeout setting, but the problem persists.

Here is the error message in the web tier,
org.springframework.remoting.RemoteAccessException: Could not access remote service [ejb/search]; nested exception is java.rmi.RemoteException: CORBA NO_RESPONSE 0x4942fb01 Maybe; nested exception is:
org.omg.CORBA.NO_RESPONSE: Request 845 timed out vmcid: IBM minor code: B01 completed: Maybe

Now the error message in the business tier.
[8/25/08 2:19:18:309 IST] 00000023 ThreadMonitor W WSVR0606W: Thread "ORB.thread.pool : 1" (00000023) was previously reported to be hung but has completed. It was active for approximately 2216802 milliseconds. There is/are 3 thread(s) in total in the server that still may be hung.
[8/25/08 2:19:18:479 IST] 00000083 ThreadMonitor W WSVR0606W: Thread "ORB.thread.pool : 8" (00000083) was previously reported to be hung but has completed. It was active for approximately 2143449 milliseconds. There is/are 2 thread(s) in total in the server that still may be hung.
[8/25/08 2:19:34:053 IST] 000000e0 ThreadMonitor W WSVR0606W: Thread "ORB.thread.pool : 90" (000000e0) was previously reported to be hung but has completed. It was active for approximately 1626228 milliseconds. There is/are 1 thread(s) in total in the server that still may be hung.
[8/25/08 2:20:33:483 IST] 0000008e ThreadMonitor W WSVR0606W: Thread "ORB.thread.pool : 19" (0000008e) was previously reported to be hung but has completed. It was active for approximately 2153126 milliseconds. There is/are 0 thread(s) in total in the server that still may be hung.

Thanks.,
Seema

[Paul Ilechko]

seem...@rediffmail.com wrote:
> We have the web tier and business tier of our application deployed on
> different WAS 6.1 server instances,

Any particular reason why? it's not generally recommended.

> there is no clustering as of now. We have stateless session beans in
> our business tier that are being invoked from the web tier. Now the
> EJB calls are failing with an ORB timeout error message in the web
> tier logs. At the same time, there is an error message in the
> business tier logs, saying that the ORB threads are hung. We tried
> increasing the ORB timeout setting, but the problem persists.

Then I suggest that you figure out why those threads are hanging. It's
probably an application error. Increasing the timeout is not going to
solve the problem.

[ro...@mindspring.com]

Hi, Looking at your messages in the business tier, the threads completed in 2216802 milliseconds (37 minutes?). Are you expecting your session beans to run that long? Maybe why the web side is timing out?

The "hung thread" messsage doesn't always mean a thread is hung, just that it has been running for a certain amount of time. I think that time is also configurable.

Roy