Implementation Strategy

[Robert M.]

Hi,

I've just started using Retlang in a project and I need an advice
regarding the retlang usage and its invasiveness in the app domain
logic.

I will try to describe a simple scenario that I want to implement
using Retlang and if possible I would like your opinion and maybe some
basic sample code.

A client application can send an order to my order processor
application - which is the subject project. The order processor app
must first validate the message, filter it and then route it to the
appropriate order queue. Of course, all the workflow steps must happen
sequentially but the nature of processing must be asynchronious. For
this I rely on Retlang's capabilities.

In my application, I have defined the following services:

IOrderService with a method called SubmitOrder (Order order)
IOrderValidationService with a method Validate(Order order)
IFilterService with a method Filter(Order order)
IRouterService with a method called Route (Order order)

The processing in all above methods must be async, so practically I
just need to push the message to an internal queue and the actual
action must be done in parallel for many orders. When one of the
methods finishes its work it must trigger the next step in the
workflow (e.g. SubmitOrder calls Validate and exits, when Validate
finishises pushes the order to Filter method and exits and so on).

How would you recommend implementing this kind of scenario using
Retlang? I am trying to hide from the developer using my API interface
contracts above as much implementation detail as I can... He must not
even know that internally the application uses Retlang for message
passing...

Sample code would be most than welcomed since I am not so familiar
with Retlang library...

Many thanks,
Robert

[Mike Rettig]

Here is my a rough example on how I would approach the problem. The OrderService just provides a simple facade to publish the orders.

class OrderService: IOrderService{

  readonly IProcessBus _retlangContext;

  OrderService(IProcessBus retlangContext){ _retlangContext = retlangContext;}

   void SubmitOrder(IOrder order){
   _retlangContext.Publish("new.order", order);
  }
}

Workflows take care of receiving the message, calling the service, and routing the result. I implemented the validation workflow, but the others would be similar.

class OrderValidationWorkflow{
   readonly IOrderValidationService _validator;

  OrderValidationWorkflow(IOrderValidationService validator, IProcessBus retlangBus){
   _validator = validator;
   retlangBus.Subscribe<IOrder>("new.order", OnOrder);
}
  void OnOrder(IMessageHeader header, IOrder order){
    if(_validator.Validate(order))
     _retlangContext.Publish("valid.order", order);
   else
     _retlangContext.Publish("invalid.order", order);
}
}

At runtime, I'd recommend using a dependency injection container such as Spring.NET or Windsor for wiring together the dependencies.

Let me know if this helps at all.

Mike Rettig
Retlang Developer

[Robert M.]

Thanks Mike for info...

Indeed, I am using Windsor for dependency injection.

I have another thought now. I would like that each module(Validation,
Filtering, Routing) to function independently and process its messages
concurrently leveraging the multi-core/multi-CPU machines. I know that
each IProcessBus is considered by Retlang somehow like a thread.

Each module must be serviced by a pool of threads which will process
incoming messages independently of the work of others.

I am thinking to create pooled IProcessBus-es for each module above.
Using the workflow pattern that you indicated above, each workflow
step will publish to the next IProcessBus the messages to be
processed. For the same IProcessBus, does publish and and receive work
in parallel without interfering with each other?

How should I implement buses in order to achieve the maximum message
throughput?

R.

[Mike Rettig]

Each module should use its own IProcessBus. You can use a dedicated thread for each or rely on the thread pool. You will probably want to use the pooled instances as they are lightweight and are already pooled.

http://code.google.com/p/retlang/wiki/ThreadingDesign

The publish/receive for each process bus work independently. For a single process bus, all messages will be received sequentially.

With a bus per module, you will achieve concurrent execution between modules, but sequential processing within the module. With sequential processing, it is easy to keep state for the module without worrying about concurrency.

The resulting design is often called "Pipes and Filters". With Retlang, a IProcessBus is created per filter and the Retlang message bus becomes the pipe.

http://www.enterpriseintegrationpatterns.com/PipesAndFilters.html


Optimal message throughput can depend on a lot of things. Simply multi-threading the problem can be detrimental to overall throughput due to the overhead created by threading. My general rule is to multi-thread only when I have to as it creates overhead and complexity.  Obvious multi-threading opportunities typically result from IO (database, file, network, etc.).

Mike

On Sun, Mar 23, 2008 at 6:24 AM, Robert M. <Robert...@gmail.com> wrote:

Thanks Mike for info...

Indeed, I am using Windsor for dependency injection.

I have another thought now. I would like that each module(Validation,
Filtering, Routing) to function independently and process its messages
concurrently leveraging the multi-core/multi-CPU machines. I know that
each IProcessBus is considered by Retlang somehow like a thread.

Each module must be serviced by a pool of threads which will process
incoming messages independently of the work of others.

I am thinking to create pooled IProcessBus-es for each module above.
Using the workflow pattern that you indicated above, each workflow

step will publish to the next IProcessBus the messages to be/