Duplicate keys

[ddevore]

Martin,

I found an interesting behavior while testing against EventsourcingSpec with Reactive. Essentially what is happening in the ReliableChannelSpec test "- must forward the message sender to the destination" messages with duplicate keys are generated. When I say duplicate keys I mean a key of processorId, initiatingChannelId, sequenceNumber and ConfirmingChannelId. For this particular test the following are the keys:

• Key(2147483647,1,1,0) - this is a normal out message.
  
• Key(0,0,0,1) - this is when cmd.ackSequenceNr != SkipAck
  
• Key(2147483647,1,2,0) - this is a normal out message.
  
• Key(0,0,0,1) - this is when cmd.ackSequenceNr != SkipAck
  

As you can see Key(0,0,0,1) is a dup. This doesn't seem to be a problem with the other Journals as they are key/map based and the keys must be unique so dups are ignored. To enforce this behavior in Mongo, I have setup a unique index on processorId, initiatingChannelId, sequenceNumber and ConfirmingChannelId so dups cannot be inserted. I noticed with Casbah as well but was able to get around it by adjusting the WriteConcern to Normal on the Casbah Driver. This setting accounts only for network errors, thus ignores the duplicate key error with Casbah. Reactive isn't as forgiving and the only way I have been able to get around so far is to drop the unique index. If I do drop the unique index all reactive tests pass including the EventsourcingSpec test.

I'm a little nervous not enforcing the unique index, as this could also be an issue w/ the SQL DB's as well.

WDYT?

One thing about the unique index is it does slow things down a touch.

[Martin Krasser]

Hi Duncan,

that's more an issue of how the test is written: the sent messages should have set a valid processorId and sequenceNr but they don't. When re-writing the test to

respondTo(Message("a", processorId = 1, sequenceNr = 12L)) must be("re: a")
respondTo(Message("b", processorId = 1, sequenceNr = 13L)) must be("re: b")

you'll get

out key = Key(2147483647,1,1,0)
ack key = Key(1,0,12,1)
out key = Key(2147483647,1,2,0)
ack key = Key(1,0,13,1)

and the ack keys are unique. ProcessorId and sequenceNr are always valid when the channel is used from within a processor. This is currently the only allowed/supported way to use channels (but the tests are using them standalone). If used from within a processor, you won't see duplicate keys.

I'll soon work on an enhancement that will officially allow standalone channel usage. When implemented and a channel receives a message with an invalid processorId and/or sequenceNr it won't generate an ack (so there's no chance for a duplicate).

So you can safely turn of unique key constraints, for the moment. When ticket #84 is implemented, parts of the channel specs will probably need to be rewritten and then the tests will pass even with unique key constraints enabled.

Cheers,
Martin

Am 03.04.13 07:25, schrieb ddevore:

The preceding email message may contain confidential information of Viridity Energy, Inc. It is not intended for transmission to, or receipt by, any unauthorized persons. If you have received this message in error, please (i) do not read it, (ii) reply to the sender that you received the message in error, and (iii) erase or destroy the message.

-- 
Martin Krasser

blog:    http://krasserm.blogspot.com
code:    http://github.com/krasserm
twitter: http://twitter.com/mrt1nz

[ddevore]

Thanks Martin,

I'll turn the unique id off for just Reactive for now.

Cheers

Duncan