--
You received this message because you are subscribed to the Google Groups "Particular Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to particularsoftware+unsub...@googlegroups.com.
To post to this group, send email to particularsoftware@googlegroups.com.
Visit this group at https://groups.google.com/group/particularsoftware.
To view this discussion on the web visit https://groups.google.com/d/msgid/particularsoftware/c0163c93-c035-4315-b806-fdb228a49501%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Have you tried NServiceBus V6 yet?
Learn how to scale applications with microservices and NServiceBus 6.
Join Udi Dahan's next Advanced Distributed Systems Design Course:
September 2017 Los Angeles, CA, USA
November 2017 Sydney, Australia
Hi JamesIn general you handler cannot guarantee the idempotency of the whole solution if the BillCustomer() API is itself not idempotent. The call to the payment provider API can fail either before the cusomer is billed or after and the handler won't be able to tell what has happened. So you in case of BillCustomer() error you can either retry (risking double billing) or not (e.g. mark the order as requiring manual intervention) and risk not billing the customer at all.In most cases when dealing with external APIs a client-generated ID is provided for ensuring end-to-end idempotency. Your handler would generate such an ID in a deterministic manner (i.e. derive it from the message ID or order ID) and pass it to the BillCustomer() API. This way you shall be able to safely retry BillCustomer() in case of failure and the payment provider will have to deduplicate the requests based on the ID provided.Does this answer your question?Szymon
2017-09-05 18:04 GMT+02:00 <james...@eccountancy.net>:
What is the best way to handle idempotency (and de-duplication) in my message handler?In my example, I have a Payments endpoint that subscribes to OrderAccepted. It calls a BillCustomer() operation on a PaymentProvider API. It then publishes CustomerBilled as a consequence.Does the idempotency include the published output event?In this scenario my handler operation is not idempotent. If I call BillCustomer() again, the customer will be billed twice. So in order to de-duplicate that side effect I will not call the API again if I receive the OrderAccepted event for a second time. But if I do receive OrderAccepted twice, should I publish the CustomerBilled event again? Or should I silently swallow the input message the second time?If I am genuinely republishing because downstream systems were broken and now fixed, silently swallowing the input and not publishing an output may cause choreography to stop.Conversely, republishing the output message could cause floods of connected handlers to be reinvoked (even detect duplication and do nothing).I am struggling to find opinions on this, so would be grateful for your thoughts.
--
You received this message because you are subscribed to the Google Groups "Particular Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to particularsoftw...@googlegroups.com.
To post to this group, send email to particula...@googlegroups.com.
Visit this group at https://groups.google.com/group/particularsoftware.
To view this discussion on the web visit https://groups.google.com/d/msgid/particularsoftware/c0163c93-c035-4315-b806-fdb228a49501%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Have you tried NServiceBus V6 yet?
Learn how to scale applications with microservices and NServiceBus 6.
Join Udi Dahan's next Advanced Distributed Systems Design Course:
September 2017 Los Angeles, CA, USA
November 2017 Sydney, Australia
--
You received this message because you are subscribed to the Google Groups "Particular Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to particularsoftw...@googlegroups.com.
To post to this group, send email to particula...@googlegroups.com.
Visit this group at https://groups.google.com/group/particularsoftware.
To view this discussion on the web visit https://groups.google.com/d/msgid/particularsoftware/CAE87YxouRd-itdByib_ZEBpBP538euBD%2Bv2%3DcMjEpr0gsA02vw%40mail.gmail.com.
Thank you Szymon and Andreas for responding.
Can I rephrase the question slightly differently, using Andreas' slides as a reference point?
https://skillsmatter.com/skillscasts/2295-reliable-integrations-nservicebus
At 20:58 the slide deck (very quickly) shows a BookShipment command being sent to the service that integrates with Fedex.
It the booking was succesful, a ShipmentBooked event is published.
If we re-sent the BookShipment command, even in a scenerio where the previous execution was succesful, would we expect the ShipmentBooked event to be re-published?
The service handler may know that the previous call to Fedex succeeded, and if Fedex was not idempotent, it would not repeat the operation. It might be costly too. In that scenario it would de-duplicate that call to Fedex.
But would it republish ShipmentBooked?
FYI, in our scenario it is slightly different. Rather than command the Fedex integration service, effectively orchestrating, our Fedex integration would subscribe to OrderSettled, or a similar event. But the principle remains. If the Fedex service is re-triggered, it may or may not call Fedex depending upon Fedex's idempotency guarantee. But as a service, does it republish it's output?
In other words, Do we include the "output" message in our definition of idempotency when the integration service is seen from the outside? Does it's definition of idempotency include both ensuring 1. One delivery is booked with Fedex and 2. A ShipmentBooked event is published?
Or just the first action?
I'd appreciate your thoughts...
On Wednesday, 6 September 2017 07:35:21 UTC+1, andreas.ohlund wrote:
To add to what Szymon said in general you want to avoid mixing transactional and non transactional work on the same message pipeline. Ie try to split off the call to the payment provider by sending a new"BillCustomer" message. That way you are in full control over the recoverability settings for that message. Eg. if there is no way to make the call idempotent (like Szymon suggests) you can turn off all retries and send the message over to the error queue after just a single processing attempt to prevent double billing.With this approach you could use a saga to monitor the status of the billing operation using timeouts. Eg. have a saga started when OrderAccepted arrives. Send the BillCustomer message and then set a timeout to deal with the case where you would not receive a "BillingSucessful" response. When you do receive the response the saga can then publish the OrderBilled event and complete the saga.I gave a talk a few years back on this topic that might be of help:Does this make any sense?Cheers,Andreas
On Tue, Sep 5, 2017 at 6:16 PM Szymon Pobiega <szymon....@particular.net> wrote:
Hi JamesIn general you handler cannot guarantee the idempotency of the whole solution if the BillCustomer() API is itself not idempotent. The call to the payment provider API can fail either before the cusomer is billed or after and the handler won't be able to tell what has happened. So you in case of BillCustomer() error you can either retry (risking double billing) or not (e.g. mark the order as requiring manual intervention) and risk not billing the customer at all.In most cases when dealing with external APIs a client-generated ID is provided for ensuring end-to-end idempotency. Your handler would generate such an ID in a deterministic manner (i.e. derive it from the message ID or order ID) and pass it to the BillCustomer() API. This way you shall be able to safely retry BillCustomer() in case of failure and the payment provider will have to deduplicate the requests based on the ID provided.Does this answer your question?Szymon
2017-09-05 18:04 GMT+02:00 <james...@eccountancy.net>:
What is the best way to handle idempotency (and de-duplication) in my message handler?In my example, I have a Payments endpoint that subscribes to OrderAccepted. It calls a BillCustomer() operation on a PaymentProvider API. It then publishes CustomerBilled as a consequence.Does the idempotency include the published output event?In this scenario my handler operation is not idempotent. If I call BillCustomer() again, the customer will be billed twice. So in order to de-duplicate that side effect I will not call the API again if I receive the OrderAccepted event for a second time. But if I do receive OrderAccepted twice, should I publish the CustomerBilled event again? Or should I silently swallow the input message the second time?If I am genuinely republishing because downstream systems were broken and now fixed, silently swallowing the input and not publishing an output may cause choreography to stop.Conversely, republishing the output message could cause floods of connected handlers to be reinvoked (even detect duplication and do nothing).I am struggling to find opinions on this, so would be grateful for your thoughts.
--
You received this message because you are subscribed to the Google Groups "Particular Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to particularsoftware+unsubscribe@googlegroups.com.
To post to this group, send email to particula...@googlegroups.com.
Visit this group at https://groups.google.com/group/particularsoftware.
To view this discussion on the web visit https://groups.google.com/d/msgid/particularsoftware/c0163c93-c035-4315-b806-fdb228a49501%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
----Have you tried NServiceBus V6 yet?
Learn how to scale applications with microservices and NServiceBus 6.
Join Udi Dahan's next Advanced Distributed Systems Design Course:
September 2017 Los Angeles, CA, USA
November 2017 Sydney, Australia
You received this message because you are subscribed to the Google Groups "Particular Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to particularsoftware+unsubscribe@googlegroups.com.
To post to this group, send email to particula...@googlegroups.com.
Visit this group at https://groups.google.com/group/particularsoftware.
To view this discussion on the web visit https://groups.google.com/d/msgid/particularsoftware/CAE87YxouRd-itdByib_ZEBpBP538euBD%2Bv2%3DcMjEpr0gsA02vw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Particular Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to particularsoftware+unsub...@googlegroups.com.
To post to this group, send email to particularsoftware@googlegroups.com.
Visit this group at https://groups.google.com/group/particularsoftware.
To view this discussion on the web visit https://groups.google.com/d/msgid/particularsoftware/67e8663e-993d-4064-bf78-eb9362a51b4c%40googlegroups.com.