[mule-user] multicasting/async reply router pattern, getting an error

lighthousej

unread,

Jun 8, 2010, 12:26:45 PM6/8/10

to us...@mule.codehaus.org

I have the Manning book Mule In Action. I'm turned to the section on and am using the async reply router pattern applied to these different webservices I have to query and return results from.

I have two soapui instances running on two different physical boxes, both have identical responses (~60kb) but they could be much bigger (10Mb), but I assume the warnings below are due to this size.

I am getting this soap fault in the mule log:
bq. <address> INFO 2010-06-08 10:52:15,398 [http://connector.http.0.receiver.2] org.mule.transport.vm.VMMessageDispatcher: Connected: endpoint.outbound.vm://search.service </address><address> INFO 2010-06-08 10:52:15,409 [http://connector.http.0.dispatcher.1] org.mule.transport.http.HttpClientMessageDispatcher: Connected: endpoint.outbound.[http://localhost:9090/mockservices/searchService]</address><address> INFO 2010-06-08 10:52:15,419 [http://connector.http.0.dispatcher.2] org.mule.transport.http.HttpClientMessageDispatcher: Connected: endpoint.outbound.[http://remote:9090/mockservice/searchService]</address><address> WARN 2010-06-08 10:52:15,468 [http://connector.http.0.dispatcher.1] org.apache.commons.httpclient.cookie.CookieSpec: Invalid cookie state: domain not specified </address><address> WARN 2010-06-08 10:52:15,476 [http://connector.http.0.dispatcher.2] org.apache.commons.httpclient.cookie.CookieSpec: Invalid cookie state: domain not specified </address><address> WARN 2010-06-08 10:52:15,738 [http://connector.http.0.dispatcher.2] org.apache.commons.httpclient.HttpMethodBase: Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended. </address><address> ERROR 2010-06-08 10:52:15,738 [http://connector.http.0.dispatcher.2] org.mule.transport.http.HttpClientMessageDispatcher: <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> </address><address> <soapenv:Body> </address><address> <soapenv:Fault> </address><address> <faultcode>Server</faultcode> </address><address> <faultstring>org.apache.xmlbeans.XmlException: error: Unexpected end of file after null</faultstring> </address><address> </soapenv:Fault> </address><address> </soapenv:Body> </address><address> </soapenv:Envelope></address>
Now soapui has apparently seen this error before: [[SOLVED]XmlException: Unexpected end of file after null|http://www.eviware.com/forum/viewtopic.php?f=5&t=2752&p=9975] but it's unknown to me if it was using mule.
I tried their suggestions about nightly builds and their config change, but it didn't make a difference.
Before I approach them (in fact I cannot register on their forums because my free email accounts apparently don't have MX records to their domains), I wanted to find out if mule (or my config) isn't working right.

Here's the async service:

<address> <service name="PerformSearch"></address>
<address> <inbound></address>
<address> <vm:inbound-endpoint address="vm://search.service" synchronous="true"></address>
<address> <message-properties-transformer></address>
<address> <add-message-property key="MULE_CORRELATION_GROUP_SIZE" value="2"/></address>
<address> </message-properties-transformer></address>
<address> </vm:inbound-endpoint></address>
<address> </inbound></address>
<address> <outbound></address>
<address> <multicasting-router></address>
<address> <outbound-endpoint address="http://localhost:9090/mockservices/searchService" synchronous="true" /></address>
<address> <outbound-endpoint address="http://remote:9090/mockservice/searchService" synchronous="true" /></address>
<address> <reply-to address="vm://search.responses"/></address>
<address> </multicasting-router></address>
<address> </outbound></address>
<address> <async-reply></address>
<address> <vm:inbound-endpoint path="search.responses"/></address>
<address> <collection-async-reply-router/></address>
<address> </async-reply></address>
<address> </service></address>
So, with all of that, I wanted to find out from others if I'm configuring Mule right to do what I want, and that it's soapui (or how I'm using it) that is the problem.

My current suspicion is that mule isn't sending all of the data to the other endpoint, namely because the data is fed through a java.io.InputStream (or similar streaming mechanism). One endpoint connects and gets data, but when the other one does the same, the data is gone (null), hence the error. From reading, the multicasting router *should* copy the data verbatim to make multiple copies but it's not happening? That's my best guess as to what's happening.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

unread,

Jun 8, 2010, 4:40:42 PM6/8/10

to us...@mule.codehaus.org

Yes, we're making progress :)

I think I know what the issue is: in order to aggregate the pair of messages, Mule needs to get something that allows them to be logically grouped together. By default, Mule is using the correlation ID for this. What I'm suspecting is that the messages coming back from the two subsequent multicasted calls don't have the same correlation ID.

Can the 2 HTTP searchServices return the Correlation ID header they receive from Mule in their response? That way, the aggregator will work happily.

If not, another option consists in configuring an expression-message-info-mapping on the collection-async-reply-router in order to have it use something else than the correlation ID as the grouping key. For example, if the SOAP responses coming from the 2 search services contain a discriminant, you could use an XPath expression to group them together based on this value.

HTH

D.

lighthousej

unread,

Jun 8, 2010, 5:25:12 PM6/8/10

to us...@mule.codehaus.org

I think I understand. I was looking at the correlation id because it seemed relevant enough. I changed the enableCorrelation attribute in the multicasting-router element but it didn't make a difference.

The two service calls are contacting a pair of mocked webservice via soapui. Soon they will be contacting real webservices though. How can I get soapui (to keep my testing environment going), and how can I know that the real webservices will respond with the HTTP headers? I realize you might not know the answer to the first or both questions, I am more questioning out loud. I'll try hitting real webservices to see how they respond, perhaps log the incoming responses and HTTP headers if possible.

If not, I think I'd be able to use an XPath. Fortunately the same person wrote both webservices, so the top-level structure a few elements in is generally same enough. Do I just have to identify if a response came from an endpoint in the router, not identify which response goes with which individual endpoint?

One final question: what did adding that transformer do? From what I know, it converts any object (a MuleMessage object?) to a byte array before the router got it? I also read that this specific transformer is used in asynchronous contexts when an efficient stream would normally exhausted after the first use. I had an idea this was happening but I didn't know how to fix it. Is that the cause here?

I'll take a look at the builtin expression-message-info-mapping and custom one to see which one works if it's needed.

Thanks for your help, it's been informative the entire way through.

marking question as answered.

David Dossot

unread,

Jun 8, 2010, 6:33:24 PM6/8/10

to us...@mule.codehaus.org

The two service calls are contacting a pair of mocked webservice via soapui. Soon they will be contacting real webservices though. How can I get soapui (to keep my testing environment going), and how can I know that the real webservices will respond with the HTTP headers? I realize you might not know the answer to the first or both questions, I am more questioning out loud. I'll try hitting real webservices to see how they respond, perhaps log the incoming responses and HTTP headers if possible.

SOAPui will show you the HTTP headers returned alongside the SOAP envelope.

If not, I think I'd be able to use an XPath. Fortunately the same person wrote both webservices, so the top-level structure a few elements in is generally same enough. Do I just have to identify if a response came from an endpoint in the router, not identify which response goes with which individual endpoint?

You will have to identify what pair of responses match with the incoming search request. Ideally, the search web services allow you to add a request ID that it returns inside the response SOAP payload. If that's not the case, maybe a correlation on the search terms, that I hope will be returned in the SOAP response could work. Of course, if simultaneous requests come in with the same search term, you may mix the responses coming back from the 2 search services: if the search is idempotent, that will be fine though.

One final question: what did adding that transformer do? From what I know, it converts any object (a MuleMessage object?) to a byte array before the router got it? I also read that this specific transformer is used in asynchronous contexts when an efficient stream would normally exhausted after the first use. I had an idea this was happening but I didn't know how to fix it. Is that the cause here?

As you figured out, the streaming payload was getting fully consumed by the dispatch occurring on the first endpoint, leaving nothing to consumer for the second endpoint. Turning the stream into a byte[] resolves the issue as the array can be consumed as many times as necessary. Should the multicasting router do this stream deserialization automatically? Maybe, maybe not, it's not such a trivial decision.

Cheers,

D.

lighthousej

unread,

Jun 9, 2010, 8:44:01 AM6/9/10

to us...@mule.codehaus.org

> SOAPui will show you the HTTP headers returned alongside the SOAP envelope.

I do see the request come in with a handful of mule headers: X-MULE_ORIGINATING_ENDPOINT, X-MULE_CORRELATION_ID, etc... and I can see them not going out, but I don't know why soapui isn't returning them, or what I'm doing wrong that's causing them not to show up. I can take that up with the soapui folks because the issue seems to be going out of scope with Mule.

> You will have to identify what pair of responses match with the incoming search request.

To clarify, in the async-reply tag, using the expression-message-info-mapping, I have to identify what would be an acceptable response from any endpoint specified in the source multicasting router? I do not have to tell which response came from which specific outbound endpoint? This is in case I have additional multicast routers and I need to discriminate against their responses here, and likewise exclude these responses there?

I apologize for returning to the subject again, this is all new to me.
You've been very helpful and I just don't want to learn this incorrectly while I have the chance to ask and not go down the wrong path and waste time.

Thank you for your time.

David Dossot

unread,

Jun 9, 2010, 4:23:32 PM6/9/10

to us...@mule.codehaus.org

> SOAPui will show you the HTTP headers returned alongside the SOAP envelope.

I do see the request come in with a handful of mule headers: X-MULE_ORIGINATING_ENDPOINT, X-MULE_CORRELATION_ID, etc... and I can see them not going out, but I don't know why soapui isn't returning them, or what I'm doing wrong that's causing them not to show up. I can take that up with the soapui folks because the issue seems to be going out of scope with Mule.

It's definitively not a SOAPui issue either: returning some headers that you have received is not a standard behavior for web services, it must be implemented specifically like this if you need it. I guess you can do the same with a simple frontal HTTP proxy, like Varnish.

> You will have to identify what pair of responses match with the incoming search request.

To clarify, in the async-reply tag, using the expression-message-info-mapping, I have to identify what would be an acceptable response from any endpoint specified in the source multicasting router? I do not have to tell which response came from which specific outbound endpoint? This is in case I have additional multicast routers and I need to discriminate against their responses here, and likewise exclude these responses there?

What you need is a way to let Mule know that messages coming back from your different search service relate to the same request (so Mule can group them up to the defined correlation group size of 2). This is usually done with correlation ID preservation but if it's not an option, you can then try to look into the actual payload of the message to extract a discriminant value that will allow you to perform this correlation without having a proper ID.

HTH

D.

lighthousej

unread,

Jun 9, 2010, 4:45:24 PM6/9/10

to us...@mule.codehaus.org

> returning some headers that you have received is not a standard behavior for web services, it must be implemented specifically like this if you need it.

That's what I was searching for on the internet to find out for sure, but nobody could tell me for sure.

> you can then try to look into the actual payload of the message to extract a discriminant value that will allow you to perform this correlation without having a proper ID.

I spent some time looking around and haven't found exactly what should happen.

I have the collection router:

<collection-async-reply-router>
<expression-message-info-mapping
messageIdExpression=""
correlationIdExpression=""/>
</collection-async-reply-router>

But I haven't found anything that helps me identify what needs to go in those two sets of quotes.
The documentation seems to mention it but never shows how to use it.

I see a few examples of people using #[...] syntax to some tokens defined somewhere but I can't find where they're defined at, but I'll keep looking.

Thanks again.

David Dossot

unread,

Jun 9, 2010, 4:56:08 PM6/9/10

to us...@mule.codehaus.org

You'll find this page useful :)

http://www.mulesoft.org/documentation/display/MULE2USER/Expressions+Configuration+Reference#ExpressionsConfigurationReference-evalRef

lighthousej

unread,

Jun 9, 2010, 5:17:39 PM6/9/10

to us...@mule.codehaus.org

> {quote:title=Guest wrote:}{quote}

> You'll find this page useful :)
>
> http://www.mulesoft.org/documentation/display/MULE2USER/Expressions+Configuration+Reference#ExpressionsConfigurationReference-evalRef
>