puppet hangs

[Balasubramaniam Natarajan]

Hi

I am a newbie to puppet and I am going through the documents at "http://docs.puppetlabs.com/learning/manifests.html" the exercise were working normally then suddenly puppet hangs when I tried to apply a manifest

root@nscbose:/home/bala/puppet# puppet -V
2.7.11
root@nscbose:/home/bala/puppet# cat /etc/debian_version
wheezy/sid
root@nscbose:/home/bala/puppet# ps aux | grep -i puppet
puppet    4938  3.7  4.9  37764 25288 ?        Ssl  03:01   0:40 /usr/bin/ruby1.8 /usr/bin/puppet master --masterport=8140
root      4945  4.1  3.8  31396 19508 pts/0    Sl+  03:03   0:37 /usr/bin/ruby1.8 /usr/bin/puppet apply 3.file.pp
root      5020  0.0  0.1   4364   848 pts/1    S+   03:19   0:00 grep --color=auto -i puppet
root@nscbose:/home/bala/puppet# cat 3.file.pp
notify {'bignotify':
    message    =>    "This is a very big message and will messup the
            formating of the code, so be wary of using such
            big message, I hope this is very clear for you
            don't ever make such a mistake again
            Notify['title'] syntax"
}
root@nscbose:/home/bala/puppet# puppet apply 3.file.pp
root@nscbose:/var/log/puppet# tail masterhttp.log
    /usr/lib/ruby/1.8/puppet/application/master.rb:144:in `run_command'
    /usr/lib/ruby/1.8/puppet/application.rb:309:in `run'
    /usr/lib/ruby/1.8/puppet/application.rb:413:in `hook'
    /usr/lib/ruby/1.8/puppet/application.rb:309:in `run'
    /usr/lib/ruby/1.8/puppet/application.rb:404:in `exit_on_fail'
    /usr/lib/ruby/1.8/puppet/application.rb:309:in `run'
    /usr/lib/ruby/1.8/puppet/util/command_line.rb:69:in `execute'
    /usr/bin/puppet:4
[2012-09-08 03:00:07] INFO  going to shutdown ...
[2012-09-08 03:00:07] INFO  WEBrick::HTTPServer#start done.
root@nscbose:/var/log/puppet#

--
Regards,
Balasubramaniam Natarajan
www.etutorshop.com/moodle/

[Balasubramaniam Natarajan]

On Sat, Sep 8, 2012 at 3:25 AM, Balasubramaniam Natarajan <bala1...@gmail.com> wrote:

Hi

I am a newbie to puppet and I am going through the documents at "http://docs.puppetlabs.com/learning/manifests.html" the exercise were working normally then suddenly puppet hangs when I tried to apply a manifest

root@nscbose:/home/bala/puppet# ps aux | grep -i puppet
puppet    4938  3.7  4.9  37764 25288 ?        Ssl  03:01   0:40 /usr/bin/ruby1.8 /usr/bin/puppet master --masterport=8140
root      4945  4.1  3.8  31396 19508 pts/0    Sl+  03:03   0:37 /usr/bin/ruby1.8 /usr/bin/puppet apply 3.file.pp
root      5020  0.0  0.1   4364   848 pts/1    S+   03:19   0:00 grep --color=auto -i puppet
root@nscbose:/home/bala/puppet# cat 3.file.pp
notify {'bignotify':
    message    =>    "This is a very big message and will messup the
            formating of the code, so be wary of using such
            big message, I hope this is very clear for you
            don't ever make such a mistake again
            Notify['title'] syntax"
}
root@nscbose:/home/bala/puppet# puppet apply 3.file.pp

Well what do you know, I reboot the machine and puppet started to work once again, I am still wondering why it was it hanging and what could I have done to troubleshoot it without rebooting ? 

ps: When I it CTRL+C it did not die rather I had to kill the process ID of puppet for it to let go of the terminal.

[Balasubramaniam Natarajan]

This has occurred once again,   When I tried to "# puppet apply break_ssh.pp" the puppet terminal just hangs and it does not complete the job please at least point me where I have to start looking for answers.

root@nscbose:/home/bala/puppet# cat 10.ssh_break.pp
file{'/etc/ssh/sshd_config':
    ensure    =>    present,
    mode    =>    600,
    source    =>    '/etc/puppet/files/sshd_config',
}
service{'sshd':
    ensure    =>    running,
    enable    =>    true,
    subscribe    =>    File['/etc/ssh/sshd_config'],
}
root@nscbose:/home/bala/puppet# puppet apply 10.ssh_break.pp

After this command puppet terminal just hangs, from an another terminal I am able to confirm that puppet apply is indeed running.  Nothing on the log files either :-(

 
root@nscbose:/home/bala/puppet# ps aux | grep -i puppet

puppet     766  2.3  5.1  38504 25960 ?        Ssl  00:47   7:02 /usr/bin/ruby1.8 /usr/bin/puppet master --masterport=8140
root      7962  9.3  3.8  31484 19580 pts/2    Sl+  05:39   0:29 /usr/bin/ruby1.8 /usr/bin/puppet apply 10.ssh_break.pp
root      8170  0.0  0.1   4364   848 pts/0    S+   05:44   0:00 grep --color=auto -i puppet

[Balasubramaniam Natarajan]

On Sun, Sep 9, 2012 at 5:46 AM, Balasubramaniam Natarajan <bala1...@gmail.com> wrote:

This has occurred once again,   When I tried to "# puppet apply break_ssh.pp" the puppet terminal just hangs and it does not complete the job please at least point me where I have to start looking for answers.

root@nscbose:/home/bala/puppet# cat 10.ssh_break.pp
file{'/etc/ssh/sshd_config':
    ensure    =>    present,
    mode    =>    600,
    source    =>    '/etc/puppet/files/sshd_config',
}
service{'sshd':
    ensure    =>    running,
    enable    =>    true,
    subscribe    =>    File['/etc/ssh/sshd_config'],
}
root@nscbose:/home/bala/puppet# puppet apply 10.ssh_break.pp

After this command puppet terminal just hangs, from an another terminal I am able to confirm that puppet apply is indeed running.  Nothing on the log files either :-(

 
root@nscbose:/home/bala/puppet# ps aux | grep -i puppet

puppet     766  2.3  5.1  38504 25960 ?        Ssl  00:47   7:02 /usr/bin/ruby1.8 /usr/bin/puppet master --masterport=8140
root      7962  9.3  3.8  31484 19580 pts/2    Sl+  05:39   0:29 /usr/bin/ruby1.8 /usr/bin/puppet apply 10.ssh_break.pp
root      8170  0.0  0.1   4364   848 pts/0    S+   05:44   0:00 grep --color=auto -i puppet

I tried looking at what are the files which puppet is currently using with lsof the same is give here.

http://pastebin.com/Nvn39aBq

[Balasubramaniam Natarajan]

Super found out what the problem was, Under service I had to mention 'ssh' instead of sshd, may be the documentation over here needs to be updated.

root@nscbose:/home/bala/puppet# puppet apply 10.ssh_break.pp

warning: Could not retrieve fact fqdn
notice: /Stage[main]//File[/etc/ssh/sshd_config]/content: content changed '{md5}02b40cbe1d41aebe5dc32e8af41f7d1c' to '{md5}e819ed2e2da115fe9dbc28dcaa280287'
notice: /Stage[main]//Service[ssh]: Triggered 'refresh' from 1 events
notice: Finished catalog run in 2.66 seconds

root@nscbose:/home/bala/puppet# cat 10.ssh_break.pp
file{'/etc/ssh/sshd_config':
    ensure    =>    present,
    mode    =>    600,
    source    =>    '/etc/puppet/files/sshd_config',
}

service{'ssh':

    ensure    =>    running,
    enable    =>    true,
    subscribe    =>    File['/etc/ssh/sshd_config'],

    hasstatus    =>    true,
}
root@nscbose:/home/bala/puppet#
 

[Krzysztof Wilczynski]

Hi,

Not sure why your Puppet run never terminates, and looking at lsof output it looks fairly normal. Said that, can you include strace output for such misbehaving process? Thanks!

KW