Twitter to open source Hadoop-like tool: "Storm"

[Scott Tsai]

Apparently Twitter recently acquired a team with extensive Hadoop
experience and is going release a Hadoop replacement:
1. Critique of Hadoop: http://tech.backtype.com/the-dark-side-of-hadoop
Worth a read even if, like me, you've never used Hadoop

2. http://engineering.twitter.com/2011/08/storm-is-coming-more-details-and-plans.html
They describe Storm as a "“COMPLEX EVENT PROCESSING” system rather
then a plugin Hadoop replacement, ex: Storm doesn't have a builtin
datastore.

One system programming problem mentioned in the second post, "tracking
and cleaning up sub processes" is a problem of the traditional Unix /
POSIX API.
i.e. you can monitor a direct child process but not "child-of-child processes"
I personal would solve it by using Linux specific APIs like "cgroups":
http://libcg.sourceforge.net/

What's your take?

[Tien Ren Chen]

According to my friend who works with Twitter - they don't even start
to use it internally. don't expect too much...

2011/8/6 Scott Tsai <scot...@gmail.com>:

> --
> You are subscribed to the "itrs" group, see: http://groups.google.com/group/itrs
>

[Chia Hao Lo]

Ummm, yet another good example shows that the internal source is the most valuable XD

chlo

2011/8/7 Tien Ren Chen <trche...@gmail.com>