Declaring $stage as a class param

[Jeff]

For reasons I wont go into, it looks like I need to start using stages. I have a few classes which i need to apply before doing anything in the 'main' stage.

My problem is that I'm assigning classes using an enc (foreman) so assigning the stage to an assigned class is not as straightforward as it would normally be in a pure-puppet environent. What I've done is declare the $stage variable as part of the class definition, for example:

class myclass (
$myparam = 'myalue',
$stage = 'setup'
){
include stdlib
....
}

This works... around 50% of the time, quite frequently I get compilation errors like..

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet::Parser::Compiler failed with error ArgumentError: Could not find stage setup specified by Class[myclass] on node mynode

Is this method of assigning a stage to a class supported? If not how else can I assign a stage to a class which is being assigned by the enc?

Thanks
Jeff

[jcbollinger]

On Friday, December 12, 2014 6:00:08 AM UTC-6, Jeff wrote:

For reasons I wont go into, it looks like I need to start using stages.

Do be certain to be clear, for you own sake: you have decided that run stages are the best solution to your problem.  No one needs run stages, but they are convenient for certain tasks.  Anything you can do with stages, you can also do without, albeit more verbosely.  Be sure you are aware of stages' limitations and drawbacks.

 

I have a few classes which i need to apply before doing anything in the 'main' stage.

My problem is that I'm assigning classes using an enc (foreman) so assigning the stage to an assigned class is not as straightforward as it would normally be in a pure-puppet environent. What I've done is declare the $stage variable as part of the class definition, for example:

class myclass (
$myparam = 'myalue',
$stage = 'setup'
){
include stdlib
....
}

I would normally say that you should not do that.  Every class has a 'stage' metaparameter without you declaring it, and though it may work for you to also declare it yourself, that's not the sort of thing I would be comfortable relying upon to continue working.  With that said ...

 

This works... around 50% of the time, quite frequently I get compilation errors like..

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet::Parser::Compiler failed with error ArgumentError: Could not find stage setup specified by Class[myclass] on node mynode

... I don't think the problem here is your declaration of a 'stage' parameter.  Instead, it looks like the Stage resource you are referencing is not getting declared.

I am uncertain -- because I avoid stages myself -- whether a given stage's resource declaration needs to be evaluated before any classes are assigned to that stage.  I had supposed not, by analogy with schedules.  In that case, a good alternative would be to declare all relevant Stages at top scope in your site manifest, so that you can be certain that they will be available on every node.

 

Is this method of assigning a stage to a class supported? If not how else can I assign a stage to a class which is being assigned by the enc?

What happens if you set a 'stage' (meta)parameter without declaring stage as a class parameter?  I would expect that to work (as well, at least, as what you're doing now).  If it doesn't then you should file a bug report against Foreman.


John

[Jeff]

Hi John,

Thanks for the detailed reply, see comments below.

Jeff

On Friday, 12 December 2014 18:43:15 UTC, jcbollinger wrote:

On Friday, December 12, 2014 6:00:08 AM UTC-6, Jeff wrote:

For reasons I wont go into, it looks like I need to start using stages.

Do be certain to be clear, for you own sake: you have decided that run stages are the best solution to your problem.  No one needs run stages, but they are convenient for certain tasks.  Anything you can do with stages, you can also do without, albeit more verbosely.  Be sure you are aware of stages' limitations and drawbacks.

 

I have read the limitations of the stages but with the implementation that I have it seems to be the most obvious solution. To expand a bit here, there are a few classes which do fairly low-level stuff (resizing LVM volumes, etc) which it makes sense to do before anything else (installing and starting services which would prevent LVM resizing for example). Whilst this could probably be done with standard ordering its difficult to implement because of organisation issues (different teams writing different manifests). To me, the easiest solution seems to be to apply all this low-level stuff using a stage. I did think about trying to group the resources using a collector but that quickly became complicated - it seems you cannot collect class resources (because theyre not 'real' resources) which means collecting resources by tag. Again - there is varying different types of resources and there is some explicit ordering defined in the manifests which would cause problems. I couldnt see a way of collecting all resources with a given attribute (tag, in this case) so that threw that option out of the window.

If theres a cleaner way of doing it then I am open to suggestions :)

 

I have a few classes which i need to apply before doing anything in the 'main' stage.

My problem is that I'm assigning classes using an enc (foreman) so assigning the stage to an assigned class is not as straightforward as it would normally be in a pure-puppet environent. What I've done is declare the $stage variable as part of the class definition, for example:

class myclass (
$myparam = 'myalue',
$stage = 'setup'
){
include stdlib
....
}

I would normally say that you should not do that.  Every class has a 'stage' metaparameter without you declaring it, and though it may work for you to also declare it yourself, that's not the sort of thing I would be comfortable relying upon to continue working.  With that said ...

 

This works... around 50% of the time, quite frequently I get compilation errors like..

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Puppet::Parser::Compiler failed with error ArgumentError: Could not find stage setup specified by Class[myclass] on node mynode

... I don't think the problem here is your declaration of a 'stage' parameter.  Instead, it looks like the Stage resource you are referencing is not getting declared.

I am uncertain -- because I avoid stages myself -- whether a given stage's resource declaration needs to be evaluated before any classes are assigned to that stage.  I had supposed not, by analogy with schedules.  In that case, a good alternative would be to declare all relevant Stages at top scope in your site manifest, so that you can be certain that they will be available on every node.

 

I think this is the issue, there seems to be some kind of ordering issue during compilation which is causing failures ~50% of the time. Even if i rely on the pre-defined stages in puppetlabs-stdlib I get the same problem. If I define my own stages and include the class from that manifest then it works reliably.. but that doesnt fit in the real world since I call more than one class which need to be part of this new stage.

I havent tried anything with site.pp since thats not really something we typically use in our deployments with foreman. I'll give it a go on Monday and see if this is a solution

 

Is this method of assigning a stage to a class supported? If not how else can I assign a stage to a class which is being assigned by the enc?

What happens if you set a 'stage' (meta)parameter without declaring stage as a class parameter?  I would expect that to work (as well, at least, as what you're doing now).  If it doesn't then you should file a bug report against Foreman.

Theres already a bug open for foreman not being able to set metaparameters but I dont think it really makes much difference.

 

John

[Jeff]

Just a quick update - it seems including stdlib in site.pp is enough to fix the problem. I can use the standard stdlib stages throughout my manifests without any dependency problems.

Thanks for the help.
Jeff