Jira (PUP-6165) it should not be possible to disable manifest ordering

2 views
Skip to first unread message

R.I.Pienaar (JIRA)

unread,
Apr 12, 2016, 2:49:04 PM4/12/16
to puppe...@googlegroups.com
R.I.Pienaar created an issue
 
Puppet / Bug PUP-6165
it should not be possible to disable manifest ordering
Issue Type: Bug Bug
Affects Versions: PUP 4.4.1
Assignee: Unassigned
Created: 2016/04/12 11:48 AM
Priority: Normal Normal
Reporter: R.I.Pienaar

Manifest ordering have as far as I can tell been proven to work, problem is with it being configurable it's not something one can really rely on in public code as you just have no way to know if your users have it on or not.

When next allowed by semver I'd propose it be turned on and remain on without the option to configure.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.13#64028-sha1:b7939e9)
Atlassian logo

Sean McDonald (JIRA)

unread,
May 16, 2017, 5:00:06 PM5/16/17
to puppe...@googlegroups.com
Sean McDonald updated an issue
Change By: Sean McDonald
Labels: needs_decision  triaged
This message was sent by Atlassian JIRA (v6.4.14#64029-sha1:ae256fe)
Atlassian logo

Moses Mendoza (JIRA)

unread,
May 16, 2017, 5:00:07 PM5/16/17
to puppe...@googlegroups.com
Moses Mendoza updated an issue
Change By: Moses Mendoza
Team: Agent

Moses Mendoza (JIRA)

unread,
May 16, 2017, 5:00:08 PM5/16/17
to puppe...@googlegroups.com
Moses Mendoza updated an issue
Change By: Moses Mendoza
Labels: needs_decision

Moses Mendoza (JIRA)

unread,
May 16, 2017, 5:00:09 PM5/16/17
to puppe...@googlegroups.com
Moses Mendoza assigned an issue to Eric Sorenson
Change By: Moses Mendoza
Assignee: Eric Sorenson

Moses Mendoza (JIRA)

unread,
May 16, 2017, 5:01:07 PM5/16/17
to puppe...@googlegroups.com

Moses Mendoza (JIRA)

unread,
May 16, 2017, 5:01:08 PM5/16/17
to puppe...@googlegroups.com
Moses Mendoza updated an issue
Change By: Moses Mendoza
Fix Version/s: PUP 6.0.0

Henrik Lindberg (JIRA)

unread,
May 17, 2017, 3:49:02 AM5/17/17
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug PUP-6165
 
Re: it should not be possible to disable manifest ordering

The making of "manifest-order" into the only setting also turns it into a guarantee - if we introduce an opt-in to optimize with parallel execution that guarantee would be broken. Thus, if we do this we miss the future possibility of automatically fully optimizing the agent's time to apply a catalog by parallelizing work. Users would then need to explicitly state what can be "executed in parallel when possible".

Alternatively, we could offer offer a way to place resources in manifest order sequence as if explicitly joined by relationships - that is, what you would need to do manually today in a public module if you care about the order. This could take the form of a function call performed in a class - for example:

class example {
 
  resources_in_sequence()
 
  notify { 'who': message => 'I am on first': }
 
  notify { 'what': message => 'I am on second': }
 
}

or

class example {
 
  with_resources_in_sequence() | | {
 
    notify { 'who': message => 'I am on first': }
 
    notify { 'what': message => 'I am on second': }
 
  }
 
}

Doing the opposite (parallel is ok) would work in analog.

R.I.Pienaar (JIRA)

unread,
May 17, 2017, 3:58:02 AM5/17/17
to puppe...@googlegroups.com
R.I.Pienaar commented on Bug PUP-6165

Henrik Lindberg will that not recreate the issues with include/contain?

class with resources_in_sequence includes other class they don't control, now the whole thing is unknown again?

Generally I think MOAR is such a massive massive win on all levels for Puppet that introducing new hurdlers to it's use is a bad idea, you'd rather annotate which resources are parallel safe rather than which is not. In any way given that we (as a user) have never even considered parallel execution of resources that does seem to me the only viable option.

Moses Mendoza (JIRA)

unread,
May 17, 2017, 2:57:05 PM5/17/17
to puppe...@googlegroups.com
Moses Mendoza assigned an issue to Unassigned
 
Change By: Moses Mendoza
Assignee: Eric Sorenson

Moses Mendoza (JIRA)

unread,
May 17, 2017, 2:57:05 PM5/17/17
to puppe...@googlegroups.com
Moses Mendoza updated an issue
Change By: Moses Mendoza
Labels: needs_decision  triaged

Henrik Lindberg (JIRA)

unread,
May 18, 2017, 10:24:02 AM5/18/17
to puppe...@googlegroups.com
Henrik Lindberg commented on Bug PUP-6165
 
Re: it should not be possible to disable manifest ordering

R.I.Pienaar it would be possible to parallelize what is contained, the contained is always anchored between the start and the end of what it is contained in, so it would not float away to the top.

I get the point about parallell being the exceptional case (especially since it does not exist (yet) )

Note, that MOAR defines the order per class/define & manifest - it does not perform a global ordering based on order of evaluation at compile time. Automatic parallel application can still be performed between unrelated resources.

I can see a hybrid MOAR guarantee + parallell execution opt in + explicit allow of parallell execution in a body/block as being possible to implement should we want to. It does not block our decision to turn on MOAR as the default/guarantee since the MOAR is only a local partial ordering.

Eric Sorenson (JIRA)

unread,
Aug 29, 2018, 3:17:02 PM8/29/18
to puppe...@googlegroups.com
Eric Sorenson commented on Bug PUP-6165

+1 to making this the only option; we made the ordering algorithm pluggable in https://projects.puppetlabs.com/issues/18508 , theoretically to allow different algorithms like a "fuzzer" random order. But nobody ever used this AFAIK and I agree with RI that it'd be good to have a guarantee that it's always on in Puppet 6 and above.

This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93)
Atlassian logo

Josh Cooper (JIRA)

unread,
Aug 29, 2018, 5:58:04 PM8/29/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
 
Change By: Josh Cooper
Sprint: Platform Core Hopper

Josh Cooper (JIRA)

unread,
Aug 29, 2018, 5:58:04 PM8/29/18
to puppe...@googlegroups.com
Josh Cooper updated an issue
Change By: Josh Cooper
Team: Platform Core Coremunity

Jacob Helwig (JIRA)

unread,
Sep 5, 2018, 12:55:04 PM9/5/18
to puppe...@googlegroups.com
Jacob Helwig assigned an issue to Jacob Helwig
Change By: Jacob Helwig
Assignee: Jacob Helwig

Jacob Helwig (JIRA)

unread,
Sep 5, 2018, 12:55:04 PM9/5/18
to puppe...@googlegroups.com
Jacob Helwig updated an issue
Change By: Jacob Helwig
Sprint: Platform Core Hopper KANBAN

Jacob Helwig (JIRA)

unread,
Sep 5, 2018, 4:27:03 PM9/5/18
to puppe...@googlegroups.com
Jacob Helwig commented on Bug PUP-6165
 
Re: it should not be possible to disable manifest ordering

Seems like if the only valid option will be "manifest", that we might as well add a deprecation to 5.5.x, and remove the setting entirely in 6.0.0. PUP-9107 is to deprecate in 5.5.x.

Kenn Hussey (JIRA)

unread,
Sep 10, 2018, 9:30:08 AM9/10/18
to puppe...@googlegroups.com
Kenn Hussey updated an issue
 
Change By: Kenn Hussey
Flagged: Impediment

Jacob Helwig (JIRA)

unread,
Sep 10, 2018, 4:55:03 PM9/10/18
to puppe...@googlegroups.com
Jacob Helwig updated an issue
Change By: Jacob Helwig
Release Notes Summary: The deprecated `ordering` setting has been removed, and catalogs now always have the ordering previously provided by the "manifest" value of this setting.
Release Notes: Deprecation

Jorie Tappa (JIRA)

unread,
Sep 11, 2018, 5:59:05 PM9/11/18
to puppe...@googlegroups.com
Jorie Tappa commented on Bug PUP-6165
 
Re: it should not be possible to disable manifest ordering

merged to master at 2119235cc8f110b8f305fcfec5e589ab655a61c1

Kenn Hussey (JIRA)

unread,
Sep 12, 2018, 9:16:07 AM9/12/18
to puppe...@googlegroups.com
Kenn Hussey updated an issue
 
Change By: Kenn Hussey
Flagged: Impediment

Christopher Wood (JIRA)

unread,
Jan 9, 2019, 1:40:04 PM1/9/19
to puppe...@googlegroups.com
Christopher Wood commented on Bug PUP-6165
 
Re: it should not be possible to disable manifest ordering

As a post-hoc note, I do in fact have random ordering turned on. It is indeed an excellent fuzzer. However I think everybody at work here except for me will be delighted with this change.

 

[root@puppetmaster1 ~]# grep ^order /etc/puppetlabs/puppet/puppet.conf
ordering = random

Nate McCurdy (JIRA)

unread,
Jul 2, 2019, 1:16:03 PM7/2/19
to puppe...@googlegroups.com
Nate McCurdy commented on Bug PUP-6165

For the historical record, I found order = random to be very helpful for canary nodes and non-production tiers. Using random as a fuzzer to catch subtle ordering bugs was a useful tool in my opinion and I will miss it.
Reply all
Reply to author
Forward
0 new messages