new entry: logrep

1 view
Skip to first unread message

Carlos Bueno

unread,
Jul 19, 2008, 9:49:05 PM7/19/08
to wide-finder
Howdy, I'm working on a new entry for WF2. It's in Python, and it's
actually a production utility called logrep/wtop. It definitely won't
win any line-count contests, but I wanted to end up with something
people can use every day.

http://wtop.googlecode.com/files/wtop-0.6.1-experimental-widefinder.tar.gz
http://code.google.com/p/wtop/wiki/UsingLogrep

I've started simple: if --x-cores is greater than one, it forks N
child processes and feeds them individual lines round-robin. Each
child compiles their own statistics and they are reconciled at the
end.

This seems to hit the wall at 10 threads or 28,000 lines/sec. But for
a first try I think it's pretty good.
Reply all
Reply to author
Forward
0 new messages