[mule-user] DefaultExceptionStrategy vs RetryPolicy?
richrem
I have written a simple RetryPolicy implementation for use with an SMTP server -- Microsoft Exchange. I have run into the following situation where I'd like to know which approach should be taken. I am using the SmtpConnector, which can usually connect and send emails fine - until there is a idle period of at least 5 minutes. Then, with the next email, I will get the following exception:
ERROR 2008-12-02 09:07:43,539 [SmtpConnector.dispatcher.2] org.mule.DefaultExceptionStrategy: Caught exception in Exception Strategy: [EOF]
com.sun.mail.smtp.SMTPSendFailedException: [EOF]
at com.sun.mail.smtp.SMTPTransport.issueSendCommand(SMTPTransport.java:1388)
at com.sun.mail.smtp.SMTPTransport.mailFrom(SMTPTransport.java:959)
at com.sun.mail.smtp.SMTPTransport.sendMessage(SMTPTransport.java:583)
at org.mule.transport.email.SmtpMessageDispatcher.sendMailMessage(SmtpMessageDispatcher.java:124)
at org.mule.transport.email.SmtpMessageDispatcher.doDispatch(SmtpMessageDispatcher.java:95)
at org.mule.transport.AbstractMessageDispatcher$Worker.run(AbstractMessageDispatcher.java:273)
at org.mule.work.WorkerContext.run(WorkerContext.java:310)
at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1061)
at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:575)
at java.lang.Thread.run(Thread.java:613)
I actually expected to see the retry stuff kick in, but the AbstractConnector has some code that checks if the exception is an instance of ConnectionException before setting the retry stuff in motion. And, if another event needs to send an email, then next one works with the following log entry:
INFO 2008-12-02 15:41:25,402 [SmtpConnector.dispatcher.3] org.mule.transport.email.SmtpMessageDispatcher: Connection closed by remote server. Reconnecting.
It appears to be self-correcting, but there will be lost emails due to the idle period problem. So, I have a couple questions after experimenting some:
- Should the exception thrown above be considered for a Retry? If so, then that exception would need to be converted to a ConnectException for the current logic to work. This would probably include modifying/extending the current SmtpMessageDispatcher - assuming that's where the original exception comes from.
- If not the above approach, then how does one retry sending an email if the exception above occurs - can you dynamically reset the SmtpConnector and tell it to start a new session? What is the best practice here?
- If I go down the path of extending the DefaultExceptionStrategy, which is in play above, I still have the issue of resetting/restarting the SmtpConnector. How do I avoid losing the email that dropped in the bit bucket? It seems like a lot of work to queue the message again and just hope for the best.
Thanks for any input here!
Rich
View this message in context: DefaultExceptionStrategy vs RetryPolicy?
Sent from the Mule - User mailing list archive at Nabble.com.
richrem
I found a very interesting blog
[http://blog.devnull.org/2008/09/11/javamail-and-exchange-woes/] that talks
about the exact situation I am facing.
One point brought up was that the author could not get the
<dispatcher-threading-profile threadTTL="30000"/> attribute to work, which
might help in this case. Is this attribute supposed to kill the thread
after a certain timeout period and/or cause the dispatcher to either 1)
close/reset the connection, or 2) create a new dispatcher?
Is there another attribute on the SmtpConnector or the SmtpMessageDispatcher
that I could configure to basically have the connection closed after each
email is sent?
Cheers,
Rich
--
View this message in context: http://www.nabble.com/DefaultExceptionStrategy-vs-RetryPolicy--tp20803023p20804131.html
Sent from the Mule - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
Antoine Borg
ricston Ltd., BP 2, 1180 Uccle, Brussels, BELGIUM
From: richrem [mailto:ri...@remingtons.us]
Sent: Wednesday, December 03, 2008 12:01 AM
To: us...@mule.codehaus.org
Subject: [mule-user] DefaultExceptionStrategy vs RetryPolicy?
richrem
I've solved the problem noted here in a different way and would like some
feedback whether my solution is reasonable. What I did, rather than
anything else suggested here, was to extend the SmtpMessageDispatcher and
SmtpMessageDispatcherFactory to enable just the right amount of "special"
logic to handle the exception.
It works, but I can spot at least one drawback with this approach -- how
many can you spot?
Thanks for playing along,
Rich
== new dispatcher factory ==
public class ExchangeSmtpMessageDispatcherFactory extends
AbstractMessageDispatcherFactory {
@Override
public MessageDispatcher create(OutboundEndpoint endpoint) throws
MuleException {
return new ExchangeSmtpMessageDispatcher(endpoint);
}
}
== new dispatcher ==
public class ExchangeSmtpMessageDispatcher extends SmtpMessageDispatcher {
public ExchangeSmtpMessageDispatcher(OutboundEndpoint endpoint) {
super(endpoint);
}
@Override
protected void sendMailMessage(Message message) throws MessagingException {
// if we get the SMTPSendFailedException, retry once here and it *should*
work
try {
super.sendMailMessage(message);
} catch (MessagingException e) {
logger.info("caught SMTPSendFailedException: will retry once");
if (e instanceof SMTPSendFailedException)
super.sendMailMessage(message);
}
}
}
== new config ==
<smtp:connector name="SmtpConnector">
<mule:service-overrides
dispatcherFactory="com.acme.mule.transport.email.ExchangeSmtpMessageDispatcherFactory"
/>
</smtp:connector>
--
View this message in context: http://www.nabble.com/DefaultExceptionStrategy-vs-RetryPolicy--tp20803023p20822219.html
Sent from the Mule - User mailing list archive at Nabble.com.
Antoine Borg
1) You may have to refactor code if the dispatcher mechanism in Mule changes
in future versions
2) You're not actually using a retry policy but re-sending the mail if a
specific exception happens. Retry policies may conflict with this mechanism
3) You're only handling one type of exception - if the same problem occurs
with other exception types, you'll have to add conditions for them to your
code.
4) If anything else throws this exception when the conditions are different,
your resolution may screw things up
Having said that, you have a useable solution which functions well enough
for your situation. Some testing may be needed to prove that my points 3 &
4 are void (perhaps you could use the test cases in the Mule source code for
this)
Good stuff!
Antoine Borg, Senior Consultant | Tel: +32 28 504 696
ricston Ltd., BP 2, 1180 Uccle, Brussels, BELGIUM
email: antoin...@ricston.com | blog: blog.ricston.com | web: ricston.com
-----Original Message-----
From: richrem [mailto:ri...@remingtons.us]
AtteH
Any news on this issue? I'm facing the same issue and would not like to
create a custom class for this.
I tried to implement a jms queue that would store the failed email message
and after a while try to resend it again. For some reason this does not
work, because the MimeMessage is transformed automatically?? to String. When
I try to resend the message, all I send is a toString() result of a class...
Log is attached. I'm using Mule 2.0.1
LOG FILE: http://www.nabble.com/file/p20832473/error.txt error.txt
Here is part of my config file where the errorhandling part is defined.
ErrorMessageToString is a custom transformer that checks wether received
class is String or MimeMessage. it seems to be always String. (As the logs
state.)
-------- config part ------------
<smtp:connector name="mySmtp">
<default-connector-exception-strategy>
<vm:outbound-endpoint path="exceptionEndpoint" />
</default-connector-exception-strategy>
</smtp:connector>
<service name="ErrorHandler">
<inbound>
<vm:inbound-endpoint path="exceptionEndpoint" />
</inbound>
<outbound>
<outbound-pass-through-router>
<outbound-endpoint address="jms://my.error.queue"
connector-ref="MyActivemq" name="myErrorQueueOutbound"
transformer-refs="ErrorMessageToString ObjectToJMSMessage" />
</outbound-pass-through-router>
</outbound>
</service>
<service name="CustomerMessageToVendorAfterError">
<inbound>
<inbound-endpoint address="jms://my.error.queue"
connector-ref="MyActivemq" name="myErrorQueueInbound"
transformer-refs="JMSMessageToObject" />
</inbound>
<outbound>
<outbound-pass-through-router>
<vm:outbound-endpoint path="emailAfterErrorToCustomer" />
</outbound-pass-through-router>
</outbound>
</service>
<service name="ErrorEmailSender">
<inbound>
<vm:inbound-endpoint path="emailAfterErrorToCustomer" />
</inbound>
<outbound>
<outbound-pass-through-router>
<smtp:outbound-endpoint connector-ref="mySmtp"
name="emailAfterError" host="${smtp.host}" port="${smtp.port}"
subject="${email.subject}" to="${email.toAddress}"
from="${email.fromAddress}"
password="${smtp.password}" user="${smtp.username}" />
</outbound-pass-through-router>
</outbound>
</service>
--------------- part end -----------------
Any suggestions?
--
View this message in context: http://www.nabble.com/DefaultExceptionStrategy-vs-RetryPolicy--tp20803023p20832473.html
Sent from the Mule - User mailing list archive at Nabble.com.
richrem
Thanks for the feedback and, as you said, good stuff!
My responses are as follows - not necessarily in an attempt to defend my
approach:
> 1) You may have to refactor code if the dispatcher mechanism in Mule
> changes
> in future versions
Agreed. But, any solution that overrides any 3rd party code is susceptible
to that. Rather than hack the SmtpMessageDispatcher directly myself (see
below for reasons against this right now), I think this is a slightly better
approach. I could use the approach in the blog listed in the first entry
for this thread, but then I would have to modify the deployment scenario
(adding a new properties file and new code) for every installation that uses
this code, which could eventually be in the dozens. You couldn't have known
about that, though...
> 2) You're not actually using a retry policy but re-sending the mail if a
> specific exception happens. Retry policies may conflict with this
> mechanism
Agreed. Since the Mule RetryPolicy won't kick in unless the right kind of
exception is thrown and I haven't heard from Mule folk on this, I'd rather
not wait for them. It is a retry policy of sorts, but hard coded for a
specific scenario and not very elegant. I like the fact that it specializes
the behavior for one particular SMTP server, though.
Currently, there is plenty of discussion and action around RetryPolicy,
which could render any modification in this area as moot very shortly. I'm
not sure I like the direction some of the discussion is going, either. In
my mind, this is a perfect example where a simple retry works. Some Mule
developers are saying they don't think retry should be allowed except during
connect - e.g. not in send, dispatch, etc. So, Mule's retry stuff may never
help in this situation. Maybe we also need a SendExceptionRetryPolicy and a
DispatchExceptionRetryPolicy, etc. :-)
Any Mule folks care to comment on this?
However, if I did take this approach, I might be able to come up with a
solution that would be acceptable to Mule and benefit the community. I may
come back to this after I solve a few other problems...
> 3) You're only handling one type of exception - if the same problem occurs
> with other exception types, you'll have to add conditions for them to your
> code.
Right again. No way to predict how many new cases I may run into, but I'm
betting it's not going to be very many. And, the effort required to handle
a new one like this will be trivial. Finally, this would also be a problem
if I hacked the SmtpMessageDispatcher code, too. I'd have it convert the
new exception to a ConnectException so the Retry stuff would kick in.
> 4) If anything else throws this exception when the conditions are
> different,
> your resolution may screw things up
I think I have isolated my use case to a situation where this may not be a
real issue. Only time will tell here. But, if an exception of this type is
thrown at the same point in the code, then a single retry really shouldn't
hurt anything.
Rich
antoine.borg wrote:
>
> It's a bit early and I haven't had a full coffee yet but here goes:
>
> ...
>
> Having said that, you have a useable solution which functions well enough
> for your situation. Some testing may be needed to prove that my points 3
> &
> 4 are void (perhaps you could use the test cases in the Mule source code
> for
> this)
>
>
> Good stuff!
>
--
View this message in context: http://www.nabble.com/DefaultExceptionStrategy-vs-RetryPolicy--tp20803023p20835438.html
richrem
I have a few comments that may help:
1) RetryPolicy stuff is only available in 2.1.x
2) I chose not to store the email off to the side as this only delays the
resend and you *could* run into the same problem with sending the email the
next time. This may not be likely, though, as the delay should be less than
a second or two. My testing indicates that the connection is reestablished
immediately after the EOF by the "normal" logic in the code base when the
next send is attempted. So, each email that runs into this specific
condition, just needs a quick retry. My exact solution may not work for
you, since you are using 2.0.x, but the concept should.
3) I would not wait for Mule folks to fix 2.0.x. They seem to have moved
on.
4) An issue with your approach is that you've introduced a performance issue
and another thing that can go wrong. If you need queues for other reasons,
then this may not be all that valid, but what would happen if the queue is
offline or the network goes away and you can't queue the message? If the
message was critical (in our case, it's an alert email), then even a few
minutes delay could be a problem. Or, worse, you could lose the message.
Now, as to your specifics below:
1) You may not need all the services as you have them defined to accomplish
what you want. Why not just put the original message on the
<vm:outbound-endpoint path="emailAfterErrorToCustomer" /> and bypass the
queue?
2) On the <smtp:outbound-endpoint...> for the ErrorEmailSender service, I
think you want to add the following attribute:
transformer-refs="StringToEmailMessage"
Don't forget to define the StringToEmailMessage transformer as:
<email:string-to-email-transformer name="StringToEmailMessage"/>
HTH,
Rich
AtteH wrote:
>
> Any news on this issue? I'm facing the same issue and would not like to
> create a custom class for this.
> I tried to implement a jms queue that would store the failed email message
> and after a while try to resend it again. For some reason this does not
> work, because the MimeMessage is transformed automatically?? to String.
> When I try to resend the message, all I send is a toString() result of a
> class...
> Log is attached. I'm using Mule 2.0.1
> LOG FILE: http://www.nabble.com/file/p20832473/error.txt error.txt
>
> Here is part of my config file where the errorhandling part is defined.
> ErrorMessageToString is a custom transformer that checks wether received
> class is String or MimeMessage. it seems to be always String. (As the logs
> state.)
>
> --- snip ---
>
> Any suggestions?
>
>
>
--
View this message in context: http://www.nabble.com/DefaultExceptionStrategy-vs-RetryPolicy--tp20803023p20836347.html
AtteH
richrem wrote:
>
> 4) An issue with your approach is that you've introduced a performance
> issue and another thing that can go wrong. If you need queues for other
> reasons, then this may not be all that valid, but what would happen if the
> queue is offline or the network goes away and you can't queue the message?
> If the message was critical (in our case, it's an alert email), then even
> a few minutes delay could be a problem. Or, worse, you could lose the
> message.
>
> Now, as to your specifics below:
>
> 1) You may not need all the services as you have them defined to
> accomplish what you want. Why not just put the original message on the
> <vm:outbound-endpoint path="emailAfterErrorToCustomer" /> and bypass the
> queue?
>
> 2) On the <smtp:outbound-endpoint...> for the ErrorEmailSender service, I
> think you want to add the following attribute:
>
> transformer-refs="StringToEmailMessage"
>
> Don't forget to define the StringToEmailMessage transformer as:
>
> <email:string-to-email-transformer name="StringToEmailMessage"/>
>
>
> HTH,
> Rich
>
Thanks for the reply!
4) I am actually using queue's to deliver the original message in XML format
to me from where I transform it to emailFormat. So the Queue server must be
there all the time.
A simple retry should also work in my case. The most important thing for us
is that the Email must not be lost. If there is a delay, then that is
acceptable. That is why I implemented the queuing solution. What will happen
to your message if the email server actually goes down?
As to the comments to my specs:
1) After the error occurs in the SMTP send, mule transforms the original
message to ExceptionMessage (as DefaultExceptionStrategy is used). I cannot
just pass an ExceptionMessage to the outbound enpoint. Of course I could use
a transformer (I already have one suitable) to transform the
ExceptionMessage to Email Message. I'm going to try this out thou. But I
doubt if this will work any different to the solution I already tried out.
2) The transformer is in use in the original smtp:outbound enpoint. This is
how I figured out it should work:
The original XML message is transformed to String format (a custom format)
-> The message is transformed to EmailMessage -> The email message is sent
using the outbound smtp endpoint -> Email message sending fails -> smtp
connector error handling kicks in -> message is routed to exceptionEndpoint
-> Original MimeMessage is extracted from ExceptionMessage -> MimeMessage
to JMS format and store to queue -> CustomerMessageToVendorAfterError
service starts and picks up the error message from the JMS queue ->
transforms JMS message to MimeMessage -> Message is sent from
emailAfterError connector.
Obviously this fails as the Email message is not in correct format in the
ExceptionMessage
I found this thread:
http://www.mail-archive.com/us...@mule.codehaus.org/msg12637.html
http://www.mail-archive.com/us...@mule.codehaus.org/msg12637.html where this
same automatic transform to string happens.. No resolution unfortunately in
that thread :(
--
View this message in context: http://www.nabble.com/DefaultExceptionStrategy-vs-RetryPolicy--tp20803023p20850755.html
AtteH
Logfile attached, where you can see that a ObjectToString transformer is
executed. Why it is executed?
LOG FILE: http://www.nabble.com/file/p20852316/transformation_log.txt
transformation_log.txt
I added a noActionTransformer:
<smtp:connector name="mySmtp" contentType="text/plain">
<default-connector-exception-strategy>
<vm:outbound-endpoint path="exceptionEndpoint"
transformer-refs="noAction"/>
</default-connector-exception-strategy>
</smtp:connector>
<no-action-transformer name="noAction" />
The ObjectToString is executed before this.
Here is the service where the exception occurs:
<service name="EmailSender">
<inbound>
<vm:inbound-endpoint path="emailToVendor" />
</inbound>
<outbound>
<outbound-pass-through-router>
<smtp:outbound-endpoint connector-ref="mySmtp"
name="email" host="${smtp.host}" port="${smtp.port}"
subject="${email.subject}"
to="${email.toAddress}" from="${email.fromAddress}"
password="${smtp.password}"
user="${smtp.username}" transformer-refs="StringToEmail" />
</outbound-pass-through-router>
</outbound>
</service>
--
View this message in context: http://www.nabble.com/DefaultExceptionStrategy-vs-RetryPolicy--tp20803023p20852316.html
AtteH
I solved my problem and used a bit different solution than richrem did.
I did not want to break my design with JMS queues offering more fault
tolerant message delivery.
So I went and extended AbstractExceptionListener.
I debugged the original AbstractExceptionListener functionality and found
out that the automatic transformation to String happens in the
getErrorMessagePayload function that AbstractExceptionListener uses in
RouteException-function. So I had to rewrite RouteException and
getErrorMessagePayload functions. So actually the only change was that I
changed message.getPayloadAsString() to message.getPayload(). This solved my
problem and now I can extract the original Object from the ExceptionMessage.
protected Object getErrorMessagePayload(MuleMessage message)
{
try
{
logger.debug("ErrorMessagePayload: " + message.getPayload());
return message.getPayload();
}
catch (Exception e)
{
logException(e);
logger.info("Failed to read message payload as string, using raw
payload");
return message.getPayload();
}
}
--
View this message in context: http://www.nabble.com/DefaultExceptionStrategy-vs-RetryPolicy--tp20803023p20892881.html