I wrote an article in the Effective-ATS series on streamization and
stream-processing in parallel:
http://ats-lang.sourceforge.net/EXAMPLE/EFFECTIVATS/StreamPar/index.htmlThe package StreamPar should make it pretty straightforward to do many
common forms of file-processing in parallel. The StreamPar_binge example
in the above article does the following:
Given the name of a directory and a regular expression (PCRE), all the lines
in the files contained in the directory (and its subdirectories) are printed if they
match the given regular expression.
This is a very common programming pattern. If you ever want to write a program
that processes the files in a given directory, you can use it. And you get parallel
processing, too.
Cheers!