Order of Commands and Events

[António Mota]

Hi all.

Sometime ago I wrote a implementation of the WS-Human Task specification using Axon (2.4.6), that basically uses Commands and Events on a Aggregate that represents a Task (eventually with sub-tasks) and it's possible states and transitions (see picture)

So this of course rely heavily on the guarantee that commands and events are processed in the order they are invoked, for instance, if I send in succession the commands

1. Create
   
2. Activate
   
3. Suspend

I have to make sure the order of events raised correspond to the order of commands

1. Created
   
2. Ready
   
3. Suspended(Ready)
   

Failure to guarantee this will generate errors like ILLEGAL_STATE_FAULT or ILLEGAL_OPERATION_FAULT.

So to make a long story short this was working more or less fine for the last couple of years (a couple of ocasional errors here and there) until one month ago where those errors have grown to several per day. I do did some changes in the Axon configuration:

• changed SimpleCommandBus to AsynchronousCommandBus
  
• tweaked the commandBusTaskExecutor until finally set values of CorePoolSize=100, MaxPoolSize=200, QueueCapacity=100

• changed ClusteringEventBus/DefaultClusterSelector/SimpleCluster to ClusteringEventBus/DefaultClusterSelector/AsynchronousCluster with a SequentialPolicy
  
• tweaked the eventBusTaskExecutor until finally set values of CorePoolSize=100, MaxPoolSize=200, QueueCapacity=100
  

So my question is, will this configuration guarantee the order of processing of commands and of the events those commands generate?

Thanks in advance.

[Allard Buijze]

Hi,

with the AsynchronousCommandBus, it's indeed possible for commands to be handled in a different order than which they were sent, especially with very large thread pools. In that case, multiple threads will take commands off the queue, and chase towards the lock. There is no saying which thread arrives at the lock first, because the OS will be doing a lot of thread switching in the meantime. 

The best way to guarantee ordering of commands is by sending new commands as a result of the previous command's result. Or you'd have to "tweak" the AsynchronousCommandBus and create a single task per aggregateIdentifier, where commands for the same aggregate would just be appended to an existing task.

Hope this helps.

Cheers,

Allard

Op do 1 feb. 2018 om 12:30 schreef António Mota <ams...@gmail.com>:

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Allard Buijze
CTO

E: allard...@axoniq.io
T: +31 6 34 73 99 89

[António Mota]

 

Allard Buijze wrote:

The best way to guarantee ordering of commands is by sending new commands as a result of the previous command's result. 

Hmmm, in some cases I'm using sendAndWait when the pair of commands are always used in conjunction, like the Create and Activate, is that what you're saying?

Cheers.

[Allard Buijze]

If you do "sendAndWait(createCommand)" and then "sendAndWait(updateCommand)", you should get the expected ordering. There is no possibility of commands overtaking eachother.

What I really meant, was using send(createCommand).thenRun(() -> send(updateCommand)). This is a non-blocking approach that achieves a similar goal.

Cheers,

Allard

Op ma 5 feb. 2018 om 18:37 schreef António Mota <ams...@gmail.com>:

[António Mota]

I did replaced all my send() by sendAndWait() in our main Saga, and the result was a disaster...

Very quickly the Saga part of the application froze without any external signal, no error, o time outs, nothing. I finally had to dump all the threads and saw the blocked threads on IdentifierBasedLock$DisposableLock.lock()

​And to make things worse this was actually in Production so I had no other alternative than rollback the build...

Now I'm testing again with the SimpleCommandBus, that was causing problems in a intensive ETL, I'll probably have to find a trade off between performance and load...

​

Melhores cumprimentos / Beir beannacht / Best regards
______________________________________________________
António Manuel dos Santos Mota
http://www.linkedin.com/in/amsmota

https://sites.google.com/site/replytomany/

______________________________________________________

To unsubscribe from this group and stop receiving emails from it, send an email to axonframework+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Allard Buijze
CTO

E: allard...@axoniq.io
T: +31 6 34 73 99 89

--
You received this message because you are subscribed to a topic in the Google Groups "Axon Framework Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/axonframework/OGU37hlrknE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to axonframework+unsubscribe@googlegroups.com.

[Allard Buijze]

Hi Antonio,

it's not unlikely that using sendAndWait in a Saga causes a deadlock somewhere, depending on what your Saga does, exactly. That's basically why I suggested: 

 send(createCommand).thenRun(() -> send(updateCommand)) 

That should work well, even with an asynchronous command bus.

I don't think the trade-off is between performance and load. It's more performance+load vs programming model. Adopting asynchrony all the way would be the way to go here.

Cheers,

Allard 

Op do 8 feb. 2018 om 03:59 schreef António Mota <ams...@gmail.com>:

--

You received this message because you are subscribed to a topic in the Google Groups "Axon Framework Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/axonframework/OGU37hlrknE/unsubscribe.

To unsubscribe from this group and all its topics, send an email to axonframewor...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Axon Framework Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to axonframewor...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.