More supervisor containers allocated than expected.

27 views
Skip to first unread message

Sean Zhong

unread,
Jan 28, 2014, 2:22:03 AM1/28/14
to storm...@googlegroups.com
I try to allocate 8 supervisors by 

 bin/storm-yarn addSupervisors  -appId=application_1390868063414_0016 -supervisors=8

But get 12 supervisors returned. Why it behaves like this?

The storm-yarn master log and yarn-site.xml is attached.


Sean
storm_yarn_masterserver.log
yarn-site.xml

Roshan Punnoose

unread,
Jan 28, 2014, 9:45:13 AM1/28/14
to Sean Zhong, storm...@googlegroups.com
Since it is adding supervisors, did you already have 4 supervisors running when you added 8?


--
You received this message because you are subscribed to the Google Groups "storm-yarn" group.
To unsubscribe from this group and stop receiving emails from it, send an email to storm-yarn+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Sean Zhong(clockfly)

unread,
Jan 28, 2014, 10:12:25 AM1/28/14
to Roshan Punnoose, storm...@googlegroups.com
No, the cluster is started with initial supervisors of 0.

Bobby Evans

unread,
Jan 28, 2014, 10:24:23 AM1/28/14
to Sean Zhong(clockfly), Roshan Punnoose, storm...@googlegroups.com
Not really sure.  I can see in the logs that it got a request to put in 8 containers, but 3 times the RM returned 4 new containers for it to use.  My guess is that we have something messed up in our logic that updates the new container request.  There may be a race or something.  We probably want to look at getting some debug information about exactly what we are sending back as part of the heartbeat when it changes.

—Bobby
Reply all
Reply to author
Forward
0 new messages