Handling Message Dependencies

[René M. Andersen]

Hi all, 

We have encountered a scenario, which we have not yet found a good way to handle. The scenario seems common enough, to expect that others may already have dealt with it.

Scenario

We have a message type (MyMessage) which results in an entity (Entity1) being inserted into a database. Entity1 references another entity (Entity2) which is identified in MyMessage by an id. If Entity2 already exists Entity1 will reference the existing, if not Entity2 will be inserted in the same unit of work as Entity1. The unit of work is implemented in a message module.

Our service bus instance processes messages using several threads. If two instances of MyMessage is processed simultanously and both reference the same Entity2, two instances of Entity2 will be inserted in the database instead of one.

Does anyone have a nice solution to this problem? 

Currently we are saving Entity2 in the MyMessage consumer (flushing our NHibernate session) and surrounding the saving of Entity2 with a lock...

[Jason Meckley]

I would assume, serialized transactions and overriding the equality of Entity 2 would handle that for your.

[Matt Burton]

How about publishing a new event when entity1 is created and a subscriber can take care of the creation of entity2? That way that subscriber can check if entity2 already exists before creating it and throw the message away if not needed. Then, if concurrency is still an issue after doing this (doubtful) simply move the subscriber that creates entity2 behind another queue, make that endpoint single-threaded, and you're good to go.

My personal rule of thumb is not having more than one interaction with an external system (database, web service, etc...) per message consumer - each can fail and retry independently if you stick to that rule. It also helps untangle dependency issues such as this and make them obvious fairly quickly.

Hope that helps,

Matt

--
You received this message because you are subscribed to the Google Groups "Rhino Tools Dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/rhino-tools-dev/-/mSW-XMMH8tcJ.
To post to this group, send email to rhino-t...@googlegroups.com.
To unsubscribe from this group, send email to rhino-tools-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rhino-tools-dev?hl=en.

[Jason Meckley]

@Matt nice approach! I may have to start implementing that.

[René M. Andersen]

Hi Jason, I understand the part with serialized transactions, but how will overidding equality help in your proposal?

[Jason Meckley]

Equality of the domain objects must be overridden for NH to work properly. if they are not already, then you have some major glitches in the design of your domain model. A NH session contains/is an identity map, ensuring that exactly 1 instance of an entity exists per session. this is why sessoin.Get<>() and session.Load<>() work.

However, thinking about this more, it wouldn't really resolve anything across 2 instances of a consumer because each consumer would have it's own unique session. So entity equality, while it's needed for NH, wouldn't resolve your problem.

Matt's approach would handle this situation for you.

[René M. Andersen]

@Jason: thanks. I will have a look at our domain model to make sure equality is handled correctly.

@Matt; Thank for the suggestion. We will certainly consider your approach.

We are also considering a third approach (NOTE: this approach is only relevant if the race condition occurs on rare occasions):

Our Entity2 object has a "natural id" and we can place a unique constraint on the column in the database. 

When the MyMessage consumer tries to insert Entity2 and another consumer has already inserted it (due to a race condition) an exception will be thrown and the transaction fails. By setting the retry count in the service bus configuration > 1, the message will be processed again. The second time it will correctly find the inserted Entity2 in the database and reference that instead of trying to insert it.