Comparison Between Akka-Actors and Spring-Batch

[Sriram Narasimhan]

Hi,

Has anyone tried comparing the performance of Spring Batch with Akka
actors ?

I'm working on an application that has a need to batch-process large
feed-files from mainframes. There are many of them, that range between
~ 150MB - 500MB. These feed files have their own frequencies as well ,
some once a day, some twice and some 6-8times.

We started using spring batch, and to speedup we use spring batch-
partition. We still have not achieved any great deal of performance
improvement. Are there any comparison metric, framework level info
available the way the spring-batch and akka do concurrency ?

Thanks,
Sriram

[Patrik Nordwall]

I don't know about any benchmark comparing the two.
Akka shines when it comes to processing small tasks, which is
typically not what you do in batch processing of files, so I don't see
a perfect match for this use case. If you can divide the work into
smaller pieces and you can process those in parallel then Akka could
do a good job. Have you identified the bottleneck? Is it really
related to how the framework handles concurrency?

> --
> You received this message because you are subscribed to the Google Groups "Akka User List" group.
> To post to this group, send email to akka...@googlegroups.com.
> To unsubscribe from this group, send email to akka-user+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/akka-user?hl=en.
>



--
Patrik Nordwall
Typesafe - The software stack for applications that scale
Twitter: patriknw

[√iktor Ҡlang]

Also, I'd be very surprised if the bottleneck isn't the IO.

Viktor Klang

Akka Tech Lead

Typesafe - The software stack for applications that scale

Twitter: @viktorklang

[latr...@yahoo.de]

It depends completely on what you do with your data stream.
If you have chunks of data which can be processed independently you
have a clear case to do processing in parallel.
Then you could have a single IO read-ahead bulk-reader which sends
it's chunks of data to an actor.
That actor can split and collect the data you need to process.

I have done this 10 years ago with a C/C++ program and erlang style
message passing internally.
And it worked perfectly.
In these days Akka is your first choice.
Your optimization target is to get maximum read-speed and catch most
of CPU power.
And read-ahead alone is what I would do with akka.

On 19 Mai, 02:11, Sriram Narasimhan <sriram.iyenga...@gmail.com>
wrote:

[Sriram Iyengar]

Hi all, Thanks for the responses. I'm trying to answer the questions as well.

• The process involves heavy IO (i.e, reading these files).
• The feed reading can be parallelized as the files are contain fixed length records. 
• There is a minimal processing involved as the feed is in EBCDIC format and we change that to ASCII and also handle cobol types like X,9,COMP, COMP3 (packed decimal)
• We need to dump the record of given type into its corresponding table as a row, for processing later.    
  

Few questions,

1. I spent time reading over the weekend and came across how 'Isolated Mutable' design approach can be achieved using ScalaActors/Akka-Actor. 
1. Can the main thread read the large file, on a per-record basis, and pass that record to the actors? Is this a right approach ? 
2. Are there any good suggestions here ? 
   
2. How efficient can the 'DB writing' part be achieved ? (Note : We use Mybatis today with Spring-batch) 

Thanks,

Sriram

[Jason Mason]

i can mostly just echo what has been said already:

unless you can heavily parallelize your design, akka won't buy you much over spring batch because (educated guess here not knowing the details of your process) you will always be IO bound, unless you have some very CPU intensive ETL process involved. 

"efficiency of the DB writing part" is largely orthogonal to akka in my mind. 

[Rick Latrine]

1. I spent time reading over the weekend and came across how 'Isolated Mutable' design approach can be achieved using ScalaActors/Akka-Actor. 

1. Can the main thread read the large file, on a per-record basis, and pass that record to the actors? 

1. Is this a right approach ?
   No, not per record - read chunks of data with read-ahead (as mentioned above)
   Perform parsing to records in a different (maybe multiple) actors.
   

1. Are there any good suggestions here ? 
   

1. How efficient can the 'DB writing' part be achieved ? (Note : We use Mybatis today with Spring-batch)
   

1. Collect the record and do batch inserts - for maximum performance use JDBC.
   So you need a database batch writing actor.
   

[Peter Empen]

Sriram:
After some time:), what did you end up with? Could you share any experience?