Command Batching examples

[Federico G]

Hello,

I couldn't find any examples online of command batching implementation. Can anyone please point me to some resources on the matter? 

I want to understand which patterns may be uses for command dispatching, validation and transaction management in batch scenarios. For example, atomic processing of the batch vs. atomic processing of each command; handling of errors; returning feedback to the user; application services design for handling  command batches.

Thanks!

[Kasey Speakman]

Here is what I do for batches.

I have a Batch command that contains other commands.

The batch handler validates the batch data: e.g. deduplication, parameter checks, any other check specific to the batch as a whole.

These commands are then executed one by one, and each of their events is added to the Pending list on the Context (Unit of Work, w/e).

Subsequent commands can examine the Pending list in case it will impact how they are handled. (e.g. Something was created earlier in the batch and a subsequent command in the batch needs to use it.)

My strategy is that a failure of any individual command will cause the whole batch to fail. The failure is returned as the reason the batch failed.

Once all the commands have run successfully, the pending events are persisted in a transaction. For SQL persistence, the pending events are translated to SQL statements which all run in the same transaction.

A concrete use case for batches: A trainer may create a course and record training in that course for many people at once.

[Ryan Platte]

Our system (not CQRS) uses Sidekiq's batches: https://github.com/mperham/sidekiq/wiki/Batches

It seems like a pretty solid model. I'd be happy to answer questions you may have.

Ryan

[Kasey Speakman]

Yeah, I should note that mine is not for ETL or batch loading. It's just a way to roll up multiple commands in one transaction.

--
You received this message because you are subscribed to a topic in the Google Groups "DDD/CQRS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dddcqrs/uKq8ui4wIKY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dddcqrs+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/dddcqrs.
For more options, visit https://groups.google.com/d/optout.

[Federico G]

Thanks Kasey and Ryan!

Perhaps Sidekiq's model could be used for CQRS scenarios. 

Kasey, as I understand, you have two levels of command handlers, batch handlers and single command handlers The batch handler performs global validations, and dispatches to single command handlers. Is this correct?

I have three closely related questions:

1. Is the command batch just a collection of commands, or is it a complex command object of its own?  (i.e.:, [new CreateCourseCommand, new RecordTrainingCommand, ..., new RecordTrainingCommand] vs a special batch command with complex attributes (course attributes + array of training attributes)

2. Do the client needs to know its building a batch of commands?

3. Do you need to have a special consideration for batch command dispatching vs individual command dispatching from the client point of view? 

I suppose that the purpose of command batching is to avoid complex commands, is this correct?

Regards.

On Wednesday, August 9, 2017 at 12:36:00 PM UTC-3, Kasey Speakman wrote:

Yeah, I should note that mine is not for ETL or batch loading. It's just a way to roll up multiple commands in one transaction.

On Wed, Aug 9, 2017 at 7:24 AM, Ryan Platte <ryan....@gmail.com> wrote:

Our system (not CQRS) uses Sidekiq's batches: https://github.com/mperham/sidekiq/wiki/Batches

It seems like a pretty solid model. I'd be happy to answer questions you may have.

Ryan

On Monday, August 7, 2017 at 5:41:34 PM UTC-4, Federico G wrote:

Hello,

I couldn't find any examples online of command batching implementation. Can anyone please point me to some resources on the matter? 

I want to understand which patterns may be uses for command dispatching, validation and transaction management in batch scenarios. For example, atomic processing of the batch vs. atomic processing of each command; handling of errors; returning feedback to the user; application services design for handling  command batches.

Thanks!

--
You received this message because you are subscribed to a topic in the Google Groups "DDD/CQRS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dddcqrs/uKq8ui4wIKY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to dddcqrs+u...@googlegroups.com.

[Kasey Speakman]

Answers inline.

Kasey, as I understand, you have two levels of command handlers, batch handlers and single command handlers The batch handler performs global validations, and dispatches to single command handlers. Is this correct?

Yes, but batch handler does not directly dispatch. It simply generates new commands and returns them back to be processed by the dispatcher.

 

1. Is the command batch just a collection of commands, or is it a complex command object of its own?  (i.e.:, [new CreateCourseCommand, new RecordTrainingCommand, ..., new RecordTrainingCommand] vs a special batch command with complex attributes (course attributes + array of training attributes)

Nothing complex, but I do use it in a couple of ways. I have batch commands like BulkRegister which just contains a list of other commands (because all the information needed happens to be in that form anyway). And I have BulkRecordManualTraining which contains enough information to generate the sub-commands on the server side. I have others (which I should probably model differently) which have bare minimum info and load data from the database to generate commands. E.g. CancelRegistrationsForCourse, which loads registrations for a course ... and issues cancel commands for each of them (happens in response to course being deactivated).

 

2. Do the client needs to know its building a batch of commands?

No. In fact my version of "batches" (which I call bulk) are specifically so the client does not have to necessarily know all the sub-commands that are required to achieve the use case. I don't want the client to have to be responsible for managing this. IOW, bulk commands are concrete use cases which happen to be made up of already existing commands.

 

3. Do you need to have a special consideration for batch command dispatching vs individual command dispatching from the client point of view? 

No. They all just look like commands to the client.

 

I suppose that the purpose of command batching is to avoid complex commands, is this correct?

It is to avoid the client being responsible for managing the complexity of a given use-case. It is basically a fan-out mechanism within a single bounded context. (Across BC / service / database boundaries is a different story, probably need a process manager then.)