How is weave compared to storm and spark?

[David C Cohen]

Hi,

My application is of a cpu and data intensive distributed computing nature. While I can use weave, I also understand that storm-on-yarn, and spark-on-yarn both can deploy and manage an application on hadoop yarn. What are the benefits of using weave in this context?

[Andreas Neumann]

Hi David,

Storm and Spark are both abstractions with their own programming model. If one of these models (which are quite different between each other) fits your application, then you don’t need Weave. 

Weave in itself is not a programming paradigm like Storm or Spark, but an abstraction over YARN exposing the resources of a YARN cluster with a programming model similar to Java Threads. That helps implement frameworks like Storm of Spark - or your own - without being exposed to all the complexity of YARN.

By the way, Weave was recently incubated (under a new name) as Apache Twill. https://git-wip-us.apache.org/repos/asf?p=incubator-twill.git. We will soon have a new website and wiki, and you can already subscribe to the mailing list for future discussion: dev-subscribe@twill.incubator.apache.org 

Best -Andreas.

On Dec 8, 2013, at 7:43 PM, David C Cohen <buend...@gmail.com> wrote:

Hi,

My application is of a cpu and data intensive distributed computing nature. While I can use weave, I also understand that storm-on-yarn, and spark-on-yarn both can deploy and manage an application on hadoop yarn. What are the benefits of using weave in this context?

--
You received this message because you are subscribed to the Google Groups "weave-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weave-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.