Exception Making Web Service Call

[Stephen Byers]

I am running a client tester in eclipse.  The tester makes a web service call with Spring's WebServiceTemplate.  When I turn on debugging, I get the following exception after a Hystrix command time-out.  I get a similar exception when my client makes a Hystrix-wrapped call from tomcat but in that case it is complaining that it cannot "get a SAXParser".  Seems like this would have something to do with the versions of xerces and xalan, but in tomcat I am running xalan2.7.1 and xerces2.9.1.

Any help would be greatly appreciated!

1.0 2014-10-09 17:00:14,049 GMT-0500 WTC-123456-L3 [hystrix-wsCall-5] DEBUG 4427 HystrixCommand.executeCommand Error executing HystrixCommand.run() [TimedOut]. Proceeding to fallback logic ...

java.lang.Exception: Throwable caught while executing.

at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1319)

at com.netflix.hystrix.HystrixCommand.access$2300(HystrixCommand.java:103)

at com.netflix.hystrix.HystrixCommand$5.call(HystrixCommand.java:1186)

at com.netflix.hystrix.strategy.concurrency.HystrixContextCallable.call(HystrixContextCallable.java:51)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:724)

Caused by: java.lang.NoClassDefFoundError: Could not initialize class com.sun.xml.internal.messaging.saaj.soap.EnvelopeFactory

at com.sun.xml.internal.messaging.saaj.soap.ver1_1.SOAPPart1_1Impl.createEnvelopeFromSource(SOAPPart1_1Impl.java:68)

at com.sun.xml.internal.messaging.saaj.soap.SOAPPartImpl.getEnvelope(SOAPPartImpl.java:128)

at org.springframework.ws.soap.saaj.support.SaajUtils.getSaajVersion(SaajUtils.java:155)

at org.springframework.ws.soap.saaj.SaajSoapMessage.getSaajVersion(SaajSoapMessage.java:257)

at org.springframework.ws.soap.saaj.SaajSoapMessage.getImplementation(SaajSoapMessage.java:342)

at org.springframework.ws.soap.saaj.SaajSoapMessage.<init>(SaajSoapMessage.java:117)

at org.springframework.ws.soap.saaj.SaajSoapMessageFactory.createWebServiceMessage(SaajSoapMessageFactory.java:186)

at org.springframework.ws.soap.saaj.SaajSoapMessageFactory.createWebServiceMessage(SaajSoapMessageFactory.java:60)

at org.springframework.ws.transport.AbstractWebServiceConnection.receive(AbstractWebServiceConnection.java:90)

at org.springframework.ws.client.core.WebServiceTemplate.doSendAndReceive(WebServiceTemplate.java:589)

at org.springframework.ws.client.core.WebServiceTemplate.sendAndReceive(WebServiceTemplate.java:537)

at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:384)

at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:378)

at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:370)

at com.fedex.nextgen.client.WebServiceCommand.run(ChainedCallTester.java:87)

at com.fedex.nextgen.client.WebServiceCommand.run(ChainedCallTester.java:1)

at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1281)

... 7 more

[Stephen Byers]

I think I'm getting the error (and another one that is a little easier to see) because work is still being done in the WebServiceTemplate.marshalSendAndReceive() but the method execution gets interrupted by the timeout of the thread.  Ultimately, the error cause is an InterruptedException in the second scenario I see.  Maybe this is expected since the thread was interrupted?:

1.0 2014-10-13 09:13:56,426 GMT-0500 WTC-123456-L3 [hystrix-wsCall-1] DEBUG 4427 HystrixCommand.executeCommand Error executing HystrixCommand.run() [TimedOut]. Proceeding to fallback logic ...

java.lang.Exception: Throwable caught while executing.

at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1319)

at com.netflix.hystrix.HystrixCommand.access$2300(HystrixCommand.java:103)

at com.netflix.hystrix.HystrixCommand$5.call(HystrixCommand.java:1186)

at com.netflix.hystrix.strategy.concurrency.HystrixContextCallable.call(HystrixContextCallable.java:51)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:724)

Caused by: java.lang.ExceptionInInitializerError

at com.sun.xml.internal.messaging.saaj.soap.ver1_1.SOAPPart1_1Impl.createEnvelopeFromSource(SOAPPart1_1Impl.java:68)

at com.sun.xml.internal.messaging.saaj.soap.SOAPPartImpl.getEnvelope(SOAPPartImpl.java:128)

at org.springframework.ws.soap.saaj.support.SaajUtils.getSaajVersion(SaajUtils.java:155)

at org.springframework.ws.soap.saaj.SaajSoapMessage.getSaajVersion(SaajSoapMessage.java:257)

at org.springframework.ws.soap.saaj.SaajSoapMessage.getImplementation(SaajSoapMessage.java:342)

at org.springframework.ws.soap.saaj.SaajSoapMessage.<init>(SaajSoapMessage.java:117)

at org.springframework.ws.soap.saaj.SaajSoapMessageFactory.createWebServiceMessage(SaajSoapMessageFactory.java:186)

at org.springframework.ws.soap.saaj.SaajSoapMessageFactory.createWebServiceMessage(SaajSoapMessageFactory.java:60)

at org.springframework.ws.transport.AbstractWebServiceConnection.receive(AbstractWebServiceConnection.java:90)

at org.springframework.ws.client.core.WebServiceTemplate.doSendAndReceive(WebServiceTemplate.java:589)

at org.springframework.ws.client.core.WebServiceTemplate.sendAndReceive(WebServiceTemplate.java:537)

at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:384)

at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:378)

at org.springframework.ws.client.core.WebServiceTemplate.marshalSendAndReceive(WebServiceTemplate.java:370)

at com.fedex.nextgen.client.WebServiceCommand.run(ChainedCallTester.java:89)

at com.fedex.nextgen.client.WebServiceCommand.run(ChainedCallTester.java:1)

at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:1281)

... 7 more

Caused by: java.lang.RuntimeException: java.lang.InterruptedException

at com.sun.xml.internal.messaging.saaj.util.ParserPool.<init>(ParserPool.java:58)

at com.sun.xml.internal.messaging.saaj.soap.EnvelopeFactory.<clinit>(EnvelopeFactory.java:56)

... 24 more

Caused by: java.lang.InterruptedException

at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)

at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)

at java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:321)

at com.sun.xml.internal.messaging.saaj.util.ParserPool.<init>(ParserPool.java:55)

... 25 more

[Ben Christensen]

When a HystrixCommand timeout occurs by default it interrupts the underlying thread. If you want to turn that off there is configuration for it.

-- 
Ben Christensen - Netflix Edge Engineering
+1.310.782.5511  @benjchristensen

--
You received this message because you are subscribed to the Google Groups "HystrixOSS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hystrixoss+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Stephen Byers]

I did try the setting and the error did go away but, as expected, the timeout did not release control back to the caller because the thread was not interrupted.  I think this is one of those situations where the price to pay for benefit of the timeouts involves an additional stacktrace in the logs.

[kumar]

Stephen,

Were you able to resolve this problem. I have the same issue when I try to debug in eclipse.

[Stephen Byers]

When I turned off thread interrupt on timeout, the transaction did not actually timeout as best I can remember.  That would make sense because the thread was still busy doing work.  Right now I plan on dealing with the extra stacktrace in the logs unless I get the time to dig a bit deeper for other solutions.  I would assume it depends on what all you are doing in the thread as to whether you get additional errors when it realizes an interrupt has occurred.

[kumar]

Thank you for the quick reply. Two followup questions:

What is the configuration to turn off the interrupt on timeout?

Do you know why the thread times out if the transaction is actually still happening?

[Stephen Byers]

1.  You set the interruptOnTimeout property to false (see https://github.com/Netflix/Hystrix/wiki/Configuration#executionisolationthreadinterruptontimeout).

2.  The core idea behind Hystrix is to isolate and ultimately timeout specific sections of the code (especially around remote execution) that could otherwise completely consume your entire application and shut out all subsequent requests.  This allows you to implement proven patterns/best practices for resilient applications such as fail fast, fallback (graceful degredation), etc.  Check out https://github.com/Netflix/Hystrix/wiki/How-To-Use#Common-Patterns

I'm just now learning about it myself so I'm certainly not an expert, but there are lots of great techniques Hystrix brings to developers to create more resilient, cloud-ready apps.

If I've misstated anything above, I'm sure one of the experts will correct me.  I've found this to be a great and active community which gives more credit to what Hystrix has to offer.

[kumar]

Thanks Stephen. Let me try and confirm this. Thank you for pointing me to the documentation.

[Nikhil Talreja]

Did you find any solution to this problem?

I have the same problem and for some reason the getFallback() method is not executed for such cases.