How to bring about serialization in puppet ?

[Mayank]

Hi,
I'm trying to manage some hosts using a single puppetmaster. I
don't know why but puppet on clients seems to be executing the recipes
hosted on puppetmaster in a random order which is breaking apart
dependencies and resulting in a failed run of puppet for first time.
If I do multiple run of puppet thru puppetd --test, everything gets
installed and configured properly. However it's very rare that on
first run I can see puppet managing the configuration and installation
without fail.
Is there any way I can bring about serial execution of puppet
recipes. What I mean by serial execution is that suppose I've a
site.pp with following content:

package {"ruby-devel": ensure=>latest}
package {"rubygems": ensure=>latest}
exec {"install-mysql-gem":
command=>'gem install mysql',
path=>"/bin:/usr/bin:/usr/sbin:/sbin",
require=>[ Package["ruby-devel"], Package["rubygems"]]
}

Now in many cases puppet tries to execute Exec["install-mysql-gem"]
before Package["ruby-devel"] or Package["rubygems"] or both.

Is there a way that I can ensure that puppet renders the file in order
the script is written ?

BTW I'm using CentOS 5.4 with puppet-server-0.24.5-1.el5 and
puppet-0.24.5-1.el5.

Regards,
Mayank

[Peter Meier]

> Is there a way that I can ensure that puppet renders the file in order
> the script is written ?

No. And that's not a bug it's (really!) a feature.

If you have dependencies and they matter, then define (via
require,before,subscribe,notify) and hence _document_ them. If there
are no dependencies, the order doesn't matter.

cheers pete

[Peter Meier]

and:

> Now in many cases puppet tries to execute Exec["install-mysql-gem"]
> before Package["ruby-devel"] or Package["rubygems"] or both.

no that shouldn't be the case. if you define such a require, both
packages are installed _before_ the exec is executed.

Are you really sure that it is the way you explain here?

cheers pete

[mi...@fadedink.co.uk]

> Now in many cases puppet tries to execute Exec["install-mysql-gem"]
> before Package["ruby-devel"] or Package["rubygems"] or both.
>
> Is there a way that I can ensure that puppet renders the file in order
> the script is written ?

Like Pete says, that is what 'require' is supposed to do. Can you run
"puppetd --verbose --debug --test" (when you expect puppet to fail)
and post the output?

Mike

[Mayank]

I'll re run puppetd after I'm done with setting up some more recipes later during the day. Will post the output for sure in case I get into trouble. Thanks for the heads up.

Regards,

--
Mayank

[Jim Bala]

I posted about this exact thing yesterday (Subj: require inside define), only you actually got replies. :)

In my case, it seems as though there's something deep in the heart of Puppet that always causes 'realize'd resources (User, in this case) to be promoted to the top of the task list. The User line gets eval'd before everything else regardless of how many 'require's I put in front of it.

One difference, though, is that I'm using 0.25.4 from the EPEL repo.

-Jim

[Douglas Garstang]

Keep at it. You'll get it all working with requires=> eventually. It
will take forever, your head will hurt, and by the time your done,
you'll have a god-aweful dependency mess that will make you totally
afraid to touch any of it ever again, but you'll get it eventually if
you keep at it! I did!

Doug.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To post to this group, send email to puppet...@googlegroups.com.
To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.

[Daniel Pittman]

Douglas Garstang <doug.g...@gmail.com> writes:
> On Thu, Apr 15, 2010 at 8:44 AM, Jim Bala <usr...@gmail.com> wrote:
>> On Apr 15, 2010, at 1:30 AM, Mayank wrote:
>>
>>> I'm trying to manage some hosts using a single puppetmaster. I don't know
>>> why but puppet on clients seems to be executing the recipes hosted on
>>> puppetmaster in a random order which is breaking apart dependencies and
>>> resulting in a failed run of puppet for first time.

[...]

>>> package {"ruby-devel": ensure=>latest}
>>> package {"rubygems": ensure=>latest}
>>> exec {"install-mysql-gem":
>>>         command=>'gem install mysql',
>>>         path=>"/bin:/usr/bin:/usr/sbin:/sbin",
>>>         require=>[ Package["ruby-devel"], Package["rubygems"]]
>>> }
>>>
>>> Now in many cases puppet tries to execute Exec["install-mysql-gem"]
>>> before Package["ruby-devel"] or Package["rubygems"] or both.

...if you specify the 'require' in your 'exec', and puppet tries to run it
before the packages are installed you have found a puppet bug.

I strongly suspect from ...

>>> Is there a way that I can ensure that puppet renders the file in order the
>>> script is written ?

... this that you don't have that dependency, however, and are looking for a
shortcut way to specify the dependencies through ordering of the input file.

[...]

> Keep at it. You'll get it all working with requires=> eventually. It will
> take forever, your head will hurt, and by the time your done, you'll have a
> god-aweful dependency mess that will make you totally afraid to touch any of
> it ever again, but you'll get it eventually if you keep at it! I did!

...and here I would have to say that, Doug, you are probably getting something
wrong somewhere if the dependencies in your system setup are that complex.[1]

Generally, your dependency chains shouldn't be enormously complex or deep in
puppet, and there are several tricks that make it easy to express complex
dependencies quickly.

One of the big ones is a Class dependency:

# Shouldn't I be using the OS package for this?
exec { "install-mysql-gem":
command => "/sbin/do-magic",
require => Class["ruby"]
}

Then, inside class ruby you install the dependent packages:

class ruby { package ["ruby-devel", "rubygems"]: ensure => latest }

That ensures that *all* the stuff in class 'ruby' is done before the exec for
your gems. Nice, short, and encapsulated.

Then, if you build appropriate classes you can have very short, specific
requirements that can express complex — and transparently changing —
requirements through your codebase.

(For example: using the 'class ruby' structure you can easily implement the
Debian version of it, and the mysql gem thing doesn't need change at all.)

Regards,
Daniel

Footnotes:
[1] ...that, or maybe you work somewhere that does have enormously complex
dependency sequences across your network, and they cannot be simplified
in any way including changing the network. ;)

--
✣ Daniel Pittman ✉ dan...@rimspace.net ☎ +61 401 155 707
♽ made with 100 percent post-consumer electrons

[Nigel Kersten]

So we went there with the more complicated parts of our config, and
then I came back to sanity, with enforcing this really really simple
rule on all our commits. As intra-class require/before statements can
only refer to individual resources. (by definition), the rule is that:

Inter-class require/before statements can only refer to whole classes,
never ever individual resources within those classes.


I mean this is all good programming practice, but it's not something
people always keep in mind when putting out an ops fire and checking
fixes in, but it really makes your dependency situation so much
cleaner. You have well defined interfaces between objects, and you're
free to change internal behavior as you need to without being afraid
to touch stuff.

>
> Doug.
>
> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>
>

--
nigel

[Mayank]

nigel

I can't take chances with re running puppet as what I am trying to accomplish is complete automation of a setup on EC2. I've a script which fires up new instances on EC2. These instances are built using custom AMI (machine images) which have entry for puppetmaster in /etc/hosts and have puppet installed on 'em. I am running puppetd --test thru rc.local and since my puppetmaster is configured to autosign, hence I need this first run of puppetd --test to get the job done. As entire setup is automated so I can't wait for multiple runs of puppet before everything gets installed properly. I've tried to make a dependency tree which should work out for me and will be testing it in next 20 min .... Will revert back with any updates in case something fails this time... I need this first run of puppet to run without fail for me... Praying to God this time to make it a success :)

Regards,
--
Mayank

[Mayank]

Found an error:

Following is my puppet recipe:

class hadoopbase{
        exec {"rubygems-install":
                command => 'wget http://rubyforge.org/frs/download.php/69365/rubygems-1.3.6.tgz && tar -zxvf rubygems-1.3.6.tgz && cd rubygems-1.3.6 && ruby setup.rb && cd .. && rm -rf rubygems-1.3.6* && gem update --system',
                path => "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        exec {"updateyum":
                command => 'yum update -y',
                path => "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        package {"ruby-devel":
                        ensure => latest,
                        require => Exec["rubygems-install"],
        }
        package {"coreutils": ensure => latest}
        package {"subversion": ensure => latest}
        package {"GeoIP-devel": ensure => latest}
        exec {"downloadGeolite":
                command => 'cd /root && wget http://10.214.42.7/GeoLiteCity.dat',
                path => "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        exec {"right_aws_gem_install":
                command => 'gem install --source http://gems.rubyforge.org --version "1.10.0" right_aws',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        exec {"addressable_install":
                command => 'gem install --source http://gems.rubyforge.org --version "2.1.1" addressable',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        exec {"mime_types":
                 command => 'gem install --source http://gems.rubyforge.org --version "1.16" mime-types',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        file {"/tmp/net-geoip":
                source => "puppet:///files/hadoop/gems/net-geoip",
                recurse => true,
                ensure => present,              
        }
        exec { "net-geoip-install":
                command => 'cd /tmp/net-geoip && gem build net-geoip.gemspec && gem install net-geoip-0.0.76.gem',
                path => "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
                require => [ Package["ruby-devel"], Package["GeoIP-devel"], File["/tmp/net-geoip"] ],
        }
}

During first run of puppet thru rc.local I got following error in /var/log/messages

Apr 18 04:11:55 ip-10-250-107-162 puppetd[1812]: (//hadooprole/hadoopbase/Package[ruby-devel]/ensure) change from absent to latest failed: Could not update: Execution of '/usr/bin/yum -d 0 -e 0 -y install ruby-devel' returned 1: Error: Missing Dependency: ruby-libs = 1.8.6.110-2.fc8 is needed by package ruby-devel#012 at /etc/puppet/manifests/classes/hadoopbase.pp:13

However when I checked for ruby-libs package I found:

[root@ip-10-250-107-162 ~]# rpm -qa | grep ruby
ruby-1.8.6.114-1.fc8
ruby-irb-1.8.6.114-1.fc8
rubygems-0.9.4-1.fc8
ruby-libs-1.8.6.114-1.fc8
ruby-rdoc-1.8.6.114-1.fc8

Then I did re-run of puppet thru puppetd --test and it ran without fail and now I've this

[root@ip-10-250-107-162 ~]# rpm -qa | grep ruby
ruby-libs-1.8.6.287-2.fc8
ruby-rdoc-1.8.6.287-2.fc8
ruby-shadow-1.4.1-10.fc8
ruby-1.8.6.287-2.fc8
ruby-devel-1.8.6.287-2.fc8
rubygems-1.2.0-2.fc8
ruby-irb-1.8.6.287-2.fc8

Any ideas as to why ruby-devel was not able to install properly during first run ?

[Patrick]

If you need it to work that bad, you could probably put more than one run in your initialization script.  Also, if you send the second log somewhere, you can improve it over time.  You'll probably want to make it runs with the cache though if you're worried about performance.  Remember that "--test" removes caching.  I use "puppetd --onetime --verbose --no-daemonize".

-Patrick Mohr

[Mr Gabriel]

--
Sent from my mobile device

[Daniel Pittman]

Mayank <mail2...@gmail.com> writes:
> On Sun, Apr 18, 2010 at 1:47 PM, Mayank <mail2...@gmail.com> wrote:
>> On Sun, Apr 18, 2010 at 8:45 AM, Nigel Kersten <nig...@google.com> wrote:

[...]

> Found an error:
>
> Following is my puppet recipe:

[...]

> During first run of puppet thru rc.local I got following error in /var/log/messages
>
> Apr 18 04:11:55 ip-10-250-107-162 puppetd[1812]: (//hadooprole/hadoopbase/Package
> [ruby-devel]/ensure) change from absent to latest failed: Could not update: Execution of
> '/usr/bin/yum -d 0 -e 0 -y install ruby-devel' returned 1: Error: Missing Dependency:
> ruby-libs = 1.8.6.110-2.fc8 is needed by package ruby-devel#012 at /etc/puppet/manifests
> /classes/hadoopbase.pp:13
>
> However when I checked for ruby-libs package I found:

[... that rpm knows about it ...]

> Then I did re-run of puppet thru puppetd --test and it ran without fail and
> now I've this

[...]

> Any ideas as to why ruby-devel was not able to install properly during first
> run?

Nope. I can tell you right now that it isn't a puppet problem, though.[1]

The problem is that puppet executed ...

/usr/bin/yum -d 0 -e 0 -y install ruby-devel

... and yum returned an error, claiming that it couldn't find a dependency of
the package.

Fix your yum setup and puppet will start working. (Maybe you need something
to run a 'yum update' or something like that?)

Daniel

Footnotes:
[1] Technically, it may be puppet *related* problem, since the way puppet
invokes the yum command, or the scheduling of it, may cause the failure
by way of some side-effect.

--
✣ Daniel Pittman ✉ dan...@rimspace.net ☎ +61 401 155 707
♽ made with 100 percent post-consumer electrons

[Patrick]

On Apr 18, 2010, at 1:55 AM, Daniel Pittman wrote:

>> Any ideas as to why ruby-devel was not able to install properly during first
>> run?
>
> Nope. I can tell you right now that it isn't a puppet problem, though.[1]
>
> The problem is that puppet executed ...
>
> /usr/bin/yum -d 0 -e 0 -y install ruby-devel
>
> ... and yum returned an error, claiming that it couldn't find a dependency of
> the package.
>
> Fix your yum setup and puppet will start working. (Maybe you need something
> to run a 'yum update' or something like that?)
>
> Daniel
>
> Footnotes:
> [1] Technically, it may be puppet *related* problem, since the way puppet
> invokes the yum command, or the scheduling of it, may cause the failure
> by way of some side-effect.

As a temporary workaround, try adding an entry to install ruby-libs between ruby-gems-install and ruby-devel. I say temporary because this looks like a bug that will cause you other problems down the road if you don't find the root cause.

Here's one way to diagnose this. Try running puppet on a new system in debug mode. Then run those same commands yourself on a new system replacing -d 0 -e 0 with -d 8 -e 1. Hopefully someone will have an easier idea.

[Mayank]

On Sun, Apr 18, 2010 at 2:39 PM, Patrick <kc7...@gmail.com> wrote:

On Apr 18, 2010, at 1:55 AM, Daniel Pittman wrote:

>> Any ideas as to why ruby-devel was not able to install properly during first
>> run?
>
> Nope.  I can tell you right now that it isn't a puppet problem, though.[1]
>
> The problem is that puppet executed ...
>
>    /usr/bin/yum -d 0 -e 0 -y install ruby-devel
>
> ... and yum returned an error, claiming that it couldn't find a dependency of
> the package.
>
> Fix your yum setup and puppet will start working.  (Maybe you need something
> to run a 'yum update' or something like that?)
>
>        Daniel
>
> Footnotes:
> [1]  Technically, it may be puppet *related* problem, since the way puppet
>     invokes the yum command, or the scheduling of it, may cause the failure
>     by way of some side-effect.

As a temporary workaround, try adding an entry to install ruby-libs between ruby-gems-install and ruby-devel.  I say temporary because this looks like a bug that will cause you other problems down the road if you don't find the root cause.

Here's one way to diagnose this.  Try running puppet on a new system in debug mode.  Then run those same commands yourself on a new system replacing -d 0 -e 0 with -d 8 -e 1.  Hopefully someone will have an easier idea.

Hi,
    I changed my puppet recipe a bit and here it is :

class hadoopbase{
        exec {"rubygems-install":
                command => 'wget http://rubyforge.org/frs/download.php/69365/rubygems-1.3.6.tgz && tar -zxvf rubygems-1.3.6.tgz && cd rubygems-1.3.6 && ruby setup.rb && cd .. && rm -rf rubygems-1.3.6* && gem update --system',
                path => "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        exec {"updateyum":
                command => 'yum update -y',
                path => "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        package {"ruby-devel":
                        ensure => latest,

                        require => [ Exec["updateyum"], Exec["rubygems-install"] ],
        }
        package {"coreutils":
                        ensure => latest,
                        require => Exec["updateyum"],
        }
        package {"subversion":
                        ensure => latest,
                        require => Exec["updateyum"],
        }
        package {"GeoIP-devel":
                        ensure => latest,
                        require => Exec["updateyum"],

        }
        exec {"downloadGeolite":
                command => 'cd /root && wget http://10.214.42.7/GeoLiteCity.dat',
                path => "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",
        }
        exec {"right_aws_gem_install":
                command => 'gem install --source http://gems.rubyforge.org --version "1.10.0" right_aws',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        exec {"addressable_install":
                command => 'gem install --source http://gems.rubyforge.org --version "2.1.1" addressable',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        exec {"mime_types":
                 command => 'gem install --source http://gems.rubyforge.org --version "1.16" mime-types',
                path => "/bin:/usr/bin:/sbin:/usr/sbin",
                require => Package["ruby-devel"],
        }
        file {"/tmp/net-geoip":
                source => "puppet:///files/hadoop/gems/net-geoip",
                recurse => true,
                ensure => present,

                require => [ Package["ruby-devel"], Package["GeoIP-devel"] ]

        }
        exec { "net-geoip-install":
                command => 'cd /tmp/net-geoip && gem build net-geoip.gemspec && gem install net-geoip-0.0.76.gem',
                path => "/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/bin",

                require => File["/tmp/net-geoip"],
        }
}

Now I am able to get puppet work for me in very first run. I had to do a require => Exec["updateyum"] for every package installation in order to ensure that the command yum update -y ran before package is being installed. However this brings me to my initial question. Since I'd mentioned exec{"updateyum": ....} before any package statement so had puppet some way of ensuring serial execution of script I would have been saved from mentioning it every time in package statements.

--
Mayank

[Peter Meier]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> Now I am able to get puppet work for me in very first run. I had to do a
> require => Exec["updateyum"] for every package installation in order to
> ensure that the command yum update -y ran before package is being installed.
> However this brings me to my initial question. Since I'd mentioned
> exec{"updateyum": ....} before any package statement so had puppet some way
> of ensuring serial execution of script I would have been saved from
> mentioning it every time in package statements.

you can shorten that by either wrap your package installation with a
define, or define at the top:

Package{ require => Exec["updateyum"] }

which will introduce a global dependency on the Package resource on that
exec. Note: afair if you set another dependency you will have to reset
it. But it looks like this will be once for your setup.

cheers pete
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvK/6oACgkQbwltcAfKi38bOgCeJQ90OrRpX2KDbG+CbpUI6/yn
34UAoLJsRRztfPQmBdneIzdKNBb3Ouwf
=UIeP
-----END PGP SIGNATURE-----

[Mayank]

On Sun, Apr 18, 2010 at 6:18 PM, Peter Meier <peter...@immerda.ch> wrote:

> Now I am able to get puppet work for me in very first run. I had to do a
> require => Exec["updateyum"] for every package installation in order to
> ensure that the command yum update -y ran before package is being installed.
> However this brings me to my initial question. Since I'd mentioned
> exec{"updateyum": ....} before any package statement so had puppet some way
> of ensuring serial execution of script I would have been saved from
> mentioning it every time in package statements.

you can shorten that by either wrap your package installation with a
define, or define at the top:

Package{ require => Exec["updateyum"] }

which will introduce a global dependency on the Package resource on that
exec. Note: afair if you set another dependency you will have to reset
it. But it looks like this will be once for your setup.

cheers pete

Thanks I'll wrap it up with a define and check it out again. Thanks to everyone for valuable advises. I guess I'll be creating some better recipes in coming days thanks to these advises :)

Regards,
--
Mayank

[Daniel Pittman]

Mayank <mail2...@gmail.com> writes:
> On Sun, Apr 18, 2010 at 6:18 PM, Peter Meier <peter...@immerda.ch> wrote:
>> > Now I am able to get puppet work for me in very first run. I had to do a
>> > require => Exec["updateyum"] for every package installation in order to
>> > ensure that the command yum update -y ran before package is being
>> > installed. However this brings me to my initial question. Since I'd
>> > mentioned exec{"updateyum": ....} before any package statement so had
>> > puppet some way of ensuring serial execution of script I would have been
>> > saved from mentioning it every time in package statements.
>>
>> you can shorten that by either wrap your package installation with a define,
>> or define at the top:
>>
>> Package{ require => Exec["updateyum"] }
>>
>> which will introduce a global dependency on the Package resource on that
>> exec. Note: afair if you set another dependency you will have to reset
>> it. But it looks like this will be once for your setup.
>

> Thanks I'll wrap it up with a define and check it out again. Thanks to
> everyone for valuable advises. I guess I'll be creating some better recipes
> in coming days thanks to these advises :)

I would strongly advise that you use the global dependency, since that avoids
the need to have a non-standard define that acts almost like the standard
package resource.

If you have packages that you require before the package manager update, just
declare them with an explicit empty 'require' attribute and you will override
the global dependency.

Daniel

--
✣ Daniel Pittman ✉ dan...@rimspace.net ☎ +61 401 155 707
♽ made with 100 percent post-consumer electrons