Rhino ESB - Batches

[racielrod]

Hello,

Is there a way to delay the consumption of a message in Rhino ESB
until either X similar messages are ready or Y time elapsed?
I'm trying to serialize and aggregate several messages in the consumer
so I don't hit my storage for a message, but for a batch instead.

What are my options?

Thanks,
Raciel

[racielrod]

[Jason Meckley]

use a saga. one of the messages will be an "all sent" message. you may want to include a checksum or the like to verify no messages were lost in transport.

[racielrod]

Thanks for the response.

I ended up writing my messages to a MSMQ queue and having a service
that wakes up every couple of seconds, process the queue in batches
and insert the records in the storage. Mainly because I was rushed by
the deadline, however I need to wrap my head around sagas because
implementing this with Rhino-ESB is a personal matter now, regardless
if my company wants or not change what I already put in place.

Should I implement a saga that keeps consuming my messages and putting
them in its state until they reach certain number and then the saga is
processed? I'm not sure...

Thanks,
Raciel

[nightwatch77]

Please take into account that sagas require persistent storage (a
database) and every arriving message will update the saga state
so you'll be doing a database transaction for each individual message.
Weren't you trying to avoid that?
R

[racielrod]

Yes I was, I will read sagas more in detail...
With the little exposure I had to ESBs it seems that they are good in
1-1 message delivery/consumption and I'm not sure they are suitable to
what I was trying to achieve.
The learning experience is being great and most of the times you learn
what something is good for by hitting what that something is NOT good
for.. :)
As I said, I will check sagas more in detail, but it looks that the
solution is not trivial, and I want to keep things simple.

Thanks!
R.

[Jason Meckley]

persistent storage != rdbms

you can also use a persisent hashtable, or nosql both of which are much faster and lighter weight than an rdbms. not to mention there is no relational data with messages, so an rdbms provides little to no value other than a dumping ground between message consumption.

[racielrod]

Jason, sure they are not the same.

But let's go back to my original requirements, all I needed was a
mechanism to serialize and aggregate a high number of writes, so
instead of multiple/most-likely-concurrent writes very often, I have
batches more evenly distributed in time.
When I first looked at ESB, I was pretty aware there were several ways
to resolve this by serializing the entities into any type of storage:
filesystem, key-value-pair db, queues, etc.
I looked into ESB to check if they had a pretty/simple/elegant way to
do this (since I was originally considering MSMQ for this, and ESB
uses queues as transport).

As I said, I delivered something quick and effective but will continue
investigating on the sagas and other options because I personally
found the ESB very suitable for other scenarios.
If achieving the same I just did in a couple of hours is more
complicated with ESB we are probably not using the ESB on this case,
but it had been a great learning experience.

Maybe adding Rx support to the bus will give more flexibility in terms
of the rules you can create for the messages to be delivered/
consumed... just a thought.

[Jason Meckley]

That brings us back to the option of sagas. There is an alternative. You can use to batched messages. The context for using batched messages verses sagas depends on how the information relates to each other. 4 example a shopping cart you would want to implement of the saga but fetching something like favorite actors favorite movies in favorite genres might be done in a batch.