Puppet 3.1.1 Windows crashes

[Michael O'Dea]

I noticed that a number of my Windows hosts had stopped dialing in, and upon closer inspection I found that Puppet was crashing in Ruby and wevtapi.dll.  I can't find any reference to this in the issue tracker, although I've got over a dozen occurrences in my environment, all with the same descriptions as below.  I checked to see if the times were similar, in case the server presented some bad data to cause the issue, but the crashes occurred at all different times and days.  

My question is, has anyone else seen this?  Is this a known issue?  If it is fixed in 3.2.1, can I install 3.2.1 clients against a 3.1.x server?  In the meantime, I will be rolling back to an older 2.7.x version that we'd run for some time without issue.  

Thanks in advance.  Fault details below.  

Faulting application name: ruby.exe, version: 1.8.7.371, time stamp: 0x5083a46c

Faulting module name: wevtapi.DLL, version: 6.1.7600.16385, time stamp: 0x4a5bdb2d

Exception code: 0xc0000005

Fault offset: 0x00001360

Faulting process id: 0xa48

Faulting application start time: 0x01ce5e0840c39580

Faulting application path: C:\Program Files (x86)\Puppet Labs\Puppet\sys\ruby\bin\ruby.exe

Faulting module path: C:\Windows\system32\wevtapi.DLL

Report Id: 487d9720-c9fc-11e2-8a25-0a252c8f038f

[Michael O'Dea]

I neglected to mention this crash leaves behind the lockfile, so while the puppet service is still running, all future puppet runs exit saying "another puppet run is in progress".

[Josh Cooper]

Hi Michael,

On Thu, Jun 6, 2013 at 8:01 AM, Michael O'Dea <gmo...@gmail.com> wrote:

I neglected to mention this crash leaves behind the lockfile, so while the puppet service is still running, all future puppet runs exit saying "another puppet run is in progress". 

On Thursday, June 6, 2013 10:55:21 AM UTC-4, Michael O'Dea wrote:

I noticed that a number of my Windows hosts had stopped dialing in, and upon closer inspection I found that Puppet was crashing in Ruby and wevtapi.dll.  I can't find any reference to this in the issue tracker, although I've got over a dozen occurrences in my environment, all with the same descriptions as below.  I checked to see if the times were similar, in case the server presented some bad data to cause the issue, but the crashes occurred at all different times and days.  

My question is, has anyone else seen this? Is this a known issue?

It is now :) https://projects.puppetlabs.com/issues/21109

 

 If it is fixed in 3.2.1, can I install 3.2.1 clients against a 3.1.x server?

In general, running newer agents against older servers is not supported.

 

 In the meantime, I will be rolling back to an older 2.7.x version that we'd run for some time without issue.  

Thanks in advance.  Fault details below.  

Faulting application name: ruby.exe, version: 1.8.7.371, time stamp: 0x5083a46c 

Faulting module name: wevtapi.DLL, version: 6.1.7600.16385, time stamp: 0x4a5bdb2d

Exception code: 0xc0000005

Fault offset: 0x00001360

Faulting process id: 0xa48

Faulting application start time: 0x01ce5e0840c39580

Faulting application path: C:\Program Files (x86)\Puppet Labs\Puppet\sys\ruby\bin\ruby.exe

Faulting module path: C:\Windows\system32\wevtapi.DLL

This is the DLL responsible for windows event logging (the new API in Vista and up). By default, puppet creates an eventlog log destination, and there must be an issue in either puppet or the win32-eventlog gem we are using. What's strange is that the code is unchanged since 2.7.14. So perhaps it's an issue with the message resource dll. I will take a look.

For workarounds, you could change the default log destination to a file while we debug the issue. I don't think this can be controlled via puppet.conf, but you use the registry module to configure the puppet service start arguments to include `--logdest <path>` to log to a file.

Josh

--

Josh Cooper

Developer, Puppet Labs

Join us at PuppetConf 2013, August 22-23 in San Francisco - http://bit.ly/pupconf13
Register now and take advantage of the Early Bird discount - save 25%!

[Michael O'Dea]

Sorry for the long delay - since I'd mitigated I moved on to other things.  Thanks for the quick fix.  Deploying servers is no simple process in my environment so I guess I'll need to stick with the old clients for a while.  

[Josh Cooper]

On Tuesday, June 11, 2013, Michael O'Dea wrote:

Sorry for the long delay - since I'd mitigated I moved on to other things.  Thanks for the quick fix.  Deploying servers is no simple process in my environment so I guess I'll need to stick with the old clients for a while.  

Do you happen to be running on non en_us systems? I'm wondering if there is an issue with multibyte to UTF-16LE conversion of the event record data.

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To post to this group, send email to puppet...@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Josh Cooper]

Do you happen to have a usermode dumps enabled? If not, could you enable it on one of the systems where you were seeing the issue, as described here: http://msdn.microsoft.com/en-us/library/bb787181(VS.85).aspx.

If you do capture a crash, please contact me off list.