Passenger-hosted puppet master breaks after 3.4 upgrade

[Guy Knights]

I just upgraded puppet on my master machine (Ubuntu 12.04, running under Passenger/Apache) and it doesn't work properly any more. The upgrade was from 3.3.2 to 3.4.3.

First of all, after the upgrade I found that the location of config.ru had changed, from /usr/share/puppet/ext/rack/files/config.ru to /usr/share/puppet/ext/rack/config.ru. I've fixed that, however it still doesn't work - if I run a puppet agent against this master, I see html output mixed in with puppet run output (which indicates failure).

There was no output in the apache error log, however I later found some debug lines in /usr/lib/ruby/vendor_ruby/puppet/util.rb which I've uncommented, and now I get the following:

#<RuntimeError: Got 30 failure(s) while initializing: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/yaml; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/yaml

Wrapped exception:

Permission denied - /var/lib/puppet/yaml; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/yaml; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/rrd; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/rrd

Wrapped exception:

Permission denied - /var/lib/puppet/rrd; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/rrd; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/reports; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/reports

Wrapped exception:

Permission denied - /var/lib/puppet/reports; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/reports; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/facts.d; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/facts.d

Wrapped exception:

Permission denied - /var/lib/puppet/facts.d; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/facts.d; Could not set 'file' on ensure: Permission denied - /var/log/puppet/masterhttp.log; Could not set 'file' on ensure: Permission denied - /var/log/puppet/masterhttp.log

Wrapped exception:

Permission denied - /var/log/puppet/masterhttp.log; change from absent to file failed: Could not set 'file' on ensure: Permission denied - /var/log/puppet/masterhttp.log; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/ssl; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/ssl

Wrapped exception:

Permission denied - /var/lib/puppet/ssl; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/ssl; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/state; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/state

Wrapped exception:

Permission denied - /var/lib/puppet/state; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/state; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/lib; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/lib

Wrapped exception:

Permission denied - /var/lib/puppet/lib; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/lib; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/bucket; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/bucket

Wrapped exception:

Permission denied - /var/lib/puppet/bucket; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/bucket; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/server_data; Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/server_data

Wrapped exception:

Permission denied - /var/lib/puppet/server_data; change from absent to directory failed: Could not set 'directory' on ensure: Permission denied - /var/lib/puppet/server_data>

The above directories are all set with owner/group puppet/puppet, I also added the apache user (www-data) to the puppet group when troubleshooting, but it hasn't fixed the problem. I saw some posts which indicated that the owner/group on config.ru might be wrong, but I've checked it and it's set to puppet/puppet.

Does anyone have a solution to this problem or suggestions on where to look? I'm stumped at the moment.

Thanks,

Guy

[Guy Knights]

As an experiment, I downgraded back to 3.3.2. However, after restarting apache I still get the same permission errors I listed in my original post.

[Guy Knights]

I set up a completely new puppet server but with the same puppet/hiera config that we have in our svn repository, and it seems to be working. It appears that the symlink /etc/puppet/rack/config.ru that points to /usr/share/puppet/ext/rack/config.ru must also have its owner/group set to puppet. I was under the impression that the actual file, not the symlink, had to have these permissions, but apparently the symlink does as well.

Anyway, it's working now but I've updated this thread in case anyone else runs into the same problem.