Renaming Overlord/Middle manager

[Fangjin Yang]

With https://github.com/druid-io/druid/pull/3711 in 0.10.0, I'd like to propose we rename middlemanagers to ingesters, and rename overlord to ingester-coordinator, with the goal that the ingester-coordinator will one day not be needed. The current naming convention just serves to confuse people.

[Charles Allen]

:-1: on middle manager to ingester

Peons do all kinds of stuff other than ingest. "TaskAgent" or similar would be more descriptive

[Himanshu]

regarding MM name change, I agree with Charles on this one. MiddleManagers are not only data ingesters but do other stuff as well, so calling them ingester might be confusing.

regarding overlord name change, I think we should continue to march forward towards removing overlord entirely ... name change and then subsequent removal might present unnecessary perceived changes to end users.

After, https://github.com/druid-io/druid/pull/3711 , we should encourage all users to stop using standalone overlord and once it is proven in production that (coordinator+overlord) in same process are good enough then we can move to stage-2 described in https://github.com/druid-io/druid/issues/3696 and update all documentation to remove concept of overlord.

-- Himanshu

--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-development+unsubscribe@googlegroups.com.
To post to this group, send email to druid-development@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/b288c4d3-ea81-40a2-88a2-5f6429b683c7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Gian Merlino]

I still think middleManagers should have been called Segment Construction Vehicles from the get-go :)

Gian

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CALuYO4eM9rQEY6VyN2QCht0YOjf5Kb%2BzBxfkQ1MN0y0ckn0fQw%40mail.gmail.com.

[Gian Merlino]

I agree with Himanshu, renaming Overlords followed by deprecating them seems like needless churn.

MiddleManagers will be around for a while though and they could use a rename. If nobody wants to go with Segment Creation Vehicle then my second choice is something boring but descriptive like TaskNode.

Gian

On Thu, Dec 15, 2016 at 8:08 AM, Himanshu <g.him...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CALuYO4eM9rQEY6VyN2QCht0YOjf5Kb%2BzBxfkQ1MN0y0ckn0fQw%40mail.gmail.com.

[Eric Tschetter]

I think Segment Construction Vehicles is awesome, but am also willing to listen to arguments that references to Blizzard games should be avoided and we should stick with boring things like TaskNode or TaskAgent.

--Eric

On Fri, Dec 16, 2016 at 1:28 AM Gian Merlino <gi...@imply.io> wrote:

I agree with Himanshu, renaming Overlords followed by deprecating them seems like needless churn.

MiddleManagers will be around for a while though and they could use a rename. If nobody wants to go with Segment Creation Vehicle then my second choice is something boring but descriptive like TaskNode.

Gian

On Thu, Dec 15, 2016 at 8:08 AM, Himanshu <g.him...@gmail.com> wrote:

regarding MM name change, I agree with Charles on this one. MiddleManagers are not only data ingesters but do other stuff as well, so calling them ingester might be confusing.

regarding overlord name change, I think we should continue to march forward towards removing overlord entirely ... name change and then subsequent removal might present unnecessary perceived changes to end users.

After, https://github.com/druid-io/druid/pull/3711 , we should encourage all users to stop using standalone overlord and once it is proven in production that (coordinator+overlord) in same process are good enough then we can move to stage-2 described in https://github.com/druid-io/druid/issues/3696 and update all documentation to remove concept of overlord.

-- Himanshu

On Wed, Dec 14, 2016 at 10:46 AM, Charles Allen <charle...@metamarkets.com> wrote:

:-1: on middle manager to ingester

Peons do all kinds of stuff other than ingest. "TaskAgent" or similar would be more descriptive

On Tuesday, December 13, 2016 at 5:31:53 PM UTC-8, Fangjin Yang wrote:

With https://github.com/druid-io/druid/pull/3711 in 0.10.0, I'd like to propose we rename middlemanagers to ingesters, and rename overlord to ingester-coordinator, with the goal that the ingester-coordinator will one day not be needed. The current naming convention just serves to confuse people.

--


You received this message because you are subscribed to the Google Groups "Druid Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.


To post to this group, send email to druid-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/b288c4d3-ea81-40a2-88a2-5f6429b683c7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--


You received this message because you are subscribed to the Google Groups "Druid Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.


To post to this group, send email to druid-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CALuYO4eM9rQEY6VyN2QCht0YOjf5Kb%2BzBxfkQ1MN0y0ckn0fQw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--


You received this message because you are subscribed to the Google Groups "Druid Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.


To post to this group, send email to druid-de...@googlegroups.com.


To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CACZNdYDtJDfz2G7kTL3WWhoh1SJ0Ky%2BVgmAbcxeVfN6HQCBvZQ%40mail.gmail.com.

[Gian Merlino]

We could also go with WorkerNode, since middleManagers are already called Workers throughout the code base. And WorkerNode is still pretty descriptive of what they do: work on tasks.

Gian

[Julian Hyde]

+1 on renaming. The current names give no clue about their purpose. And for an end user, the purpose is generally things like "the overlord just died. Does that explain why data is not loading?"

It's off-topic, but I'd also support having a mode where all servers can be in a single jvm. Never underestimate how much of druid's usage is in test, not production, environments. 

Julian

To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CACZNdYBtTH%3DHyvKYvvj4HMihPo4v1vhzAar7pF_4siSNpcvrtg%40mail.gmail.com.

[Fangjin]

I'm cool with workers as a name, just something more descriptive than MMs to help the community with debugging.

Bonus fact, almost all the MM logs are quotes from SC.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CACZNdYBtTH%3DHyvKYvvj4HMihPo4v1vhzAar7pF_4siSNpcvrtg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-development+unsubscribe@googlegroups.com.
To post to this group, send email to druid-development@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/8F810FC0-71BD-4AD7-B3EB-8BCD4CCAE6B9%40gmail.com.