Failover by WLS8.1 using wlclient.jar

[Valeri Gavrilovets]

Hello everybody,

we upgraded our application server to WLS8.1 SP3 and faced one serious
problem that probably is up to new RMI-IIOP protocol. We didn't know
that T3 protocol is replced by IIOP whether wlclient.jar is used.
Our configuration: 2 servers (host1, host2) in the cluster, admin server
on host1. SUN Solaris platform.
Java client on host3 calls an EJB method that takes about 30 minutes.
The server completes its job without problem but client gets the exception:

RemoteException: java.rmi.MarshalException: CORBA COMM_FAILURE
1398079697 No; nested exception is:
org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 209
completed: Noo

And it would not be so bad if we did not seen that the EJB procedure was
called three times! twise on the first server and once on the second
server. It seems to me that the failover had place as if the EJB method
were idemponent. But it is not.

My questions to BEA people:
1. Is that known issue?
2. If t3 protocol is used together with wlclient.jar, is it intended
that client sees no difference as if weblogic.jar were used?
3. Whether it is bug or feature, is there any properties to set in order
to solve this problem?

Other EJB calls that take not so long (less than 1 minute) produces no
problem.

Any responses are welcomed.
Thanx

[Andy Piper]

Valeri Gavrilovets <gavri...@netlife.de> writes:

> Hello everybody,
>
> we upgraded our application server to WLS8.1 SP3 and faced one serious
> problem that probably is up to new RMI-IIOP protocol. We didn't know
> that T3 protocol is replced by IIOP whether wlclient.jar is used.
> Our configuration: 2 servers (host1, host2) in the cluster, admin
> server on host1. SUN Solaris platform.
> Java client on host3 calls an EJB method that takes about 30 minutes.
> The server completes its job without problem but client gets the exception:

The 30 minute call is the root of your issues. Since IIOP has no DGC
the server will timeout connections if they are idle for a certain
period, when the connections are timed out they are closed and you get
the exception below. You can fix the idle timeout by increasing
weblogic.IdleConnectionTimeout (You can also do this per channel or
for IIOP only). Setting it to 0 disables timeouts altogether. The
default is 60s if no requests are outstanding and 4*60s if request are
outstanding.

> RemoteException: java.rmi.MarshalException: CORBA COMM_FAILURE
> 1398079697 No; nested exception is:
> org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 209
> completed: Noo
>
> And it would not be so bad if we did not seen that the EJB procedure
> was called three times! twise on the first server and once on the
> second server. It seems to me that the failover had place as if the
> EJB method were idemponent. But it is not.

Idempotency only comes into play if the request _may_ have
completed. The exception above indicates it didn't complete and
therefore it is ok to retry. I guess you could argue that in this
instance the request _may_ have completed, so this is probably a bug
although I don't think we can fix it since it is the JDK ORB that
decides whether the request completed or not. We might be able to get
Sun to change things in the JDK.

> My questions to BEA people:
> 1. Is that known issue?
> 2. If t3 protocol is used together with wlclient.jar, is it intended
> that client sees no difference as if weblogic.jar were used?

There are documented differences, in particular we do not guarantee to
be bug compatible or to replicate t3 extensions. For most customers
however wlclient.jar should be a plug replacement.

> 3. Whether it is bug or feature, is there any properties to set in
> order to solve this problem?

See above.

andy
--

[Chandrakanth K G]

Thanks Andy, your reply was of great help. In my case an applet was making a long duration EJB method call and I used to get following error on client side.
------------------------------------------------------------------------------------------------

java.rmi.MarshalException: CORBA COMM_FAILURE 1398079697 No; nested exception is:
org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 209 completed: No

at com.sun.corba.se.internal.iiop.ShutdownUtilDelegate.mapSystemException(Unknown Source)
at javax.rmi.CORBA.Util.mapSystemException(Unknown Source)
------------------------------------------------------------------------------------------------

The fix that worked for me is as follows.

1. Open weblogic console (http://yourserver:port/console/)
2. Navigate to the node yourdomain->servers->yourserver using the navigation tree on the left .
3. Click on the "Protocol" tab on the right of the screen and on that screen click on the "Show" hyperlink to get advanced editing options
4. On the Advanced Options screen, modify the entry "Idle Connection Timeout" field from default value of 65 to a suitable higher value
5. Click on the Apply button and restart the server.