Will future designs of storm-on-yarn launch workers in seperate containers?

[Tzu-Li Tai]

Hi,

Currently, workers are launched within the supervisor containers.

Is it possible that storm-on-yarn will change this design so that workers are launched on separate containers?

Otherwise, for the present design, supervisors need to constantly occupy a large portion of a node's resource. This doesn't take advantage of the allocation granularity that YARN delivers and degrades overall cluster resource utilization.

[Bobby Evans]

I would love to do that.  The current storm on YARN is really just a proof of concept that launches a stand alone storm cluster through YARN.  To be able to do too much more it is going to require much tighter integration with storm.  We would like to start discussion on making it an official part of storm once the security work we have been doing is on the path to be merged into storm.  Hopefully in the not too distant future.

-

Bobby

--
You received this message because you are subscribed to the Google Groups "storm-yarn" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-yarn+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tzu-Li Tai]

Hi Bobby,

Thanks for the insight!