puppetdb report processor failed

[jdt]

Hello,

I recently upgraded from puppetdb 1.6 to 2.2 but now it seems like I'm having issues with the puppetdb report processor.
I keep getting the errors below:
 puppet-master[23130]: Report processor failed: Environment is nil, unable to submit report. This may be due a bug with Puppet. Ensure you are running the latest revision, see PUP-2508 for more details.

I'm running puppet-server 3.6.2 on CentOS 6.x behind an Apache with passenger.
The PuppetDB v 2.2.2 is running on a PostgreSQL 9.3

Before the upgrade it worked fine, does anyone have an idea on what might be wrong? Could it be related to the modified SSL config in puppetdb;s jetty.ini config file? How could I test this?
I noticed it since all the hosts now show as unreported in Puppetboard

Best Regards,
Jo

[kaustubh chaudhari]

what dose puppetdb.log says ?

-Kaustubh

[Ken Barber]

I doubt it's related.

This error is meant to pick up on the bug outlined in
https://tickets.puppetlabs.com/browse/PUP-2508, which was fixed in
3.6.1, or at least it should have been. Basically internally the
environment is not being set in that case, that patch was meant to fix
that. In short, its not meant to happen with 3.6.2 at all, so this is
surprising.

I certainly haven't seen this error in later revisions, or seen this
particular problem in the wild. I would suspect that you hadn't
restarted your Puppet master but I doubt this is true, as the error
only occurs when you upgrade (which needs a restart). Its possibly
related to some other magic, are you using environments in anger in
your setup? ie. do you use environments that are not 'production', and
are you doing anything special like using an ENC or something?

I think at this point we could try to reproduce the problem ourselves
for 3.6.2, but that would take me a little time to set up. Before I do
this, have you tested this with Puppet 3.7.3 yet? Also, is this only
happening in 1 environment, can you reproduce this on another puppet
master perhaps?

ken.

[jdt]

 Hello,

thx for the help, I think it was related to new Package allow_virtual.
I've added a section to the manifest/site.pp to set this for puppet agents running 3.6.1 or higher and now I do get reports into PuppetDB.
So that problem seems to be solved now.
I'm only using the production environment and I'm not using any ENC. I'm using hiera to assign modules to hosts using hiera_include('classes') in the site.pp

I did get several crashes of puppetdb, due to heap space issues. I've bumped up the heap space to 512Mbm then to 768Mb;  but I still got Out-of-Memory
I've bumped up the heap space to 1Gb now, before it was running fine with 256Mb. I've also installed Puppetexplorer on the same box.
Would looking into the generated hprof file help in troubleshooting if 1Gb still isn't enough? I have about 850 active nodes

Best Regards,
Jo

Op maandag 5 januari 2015 16:18:16 UTC+1 schreef Ken Barber:

[jdt]

Hello,

at first sight nothing special.
A bunch of  replace facts, replace catalogs, store reports.
Occasionally I see
 INFO  [o.a.h.i.c.DefaultHttpClient] I/O exception (java.net.SocketException) caught when connecting to the target host: Network is unreachable
2015-01-05 09:44:49,748 INFO  [o.a.h.i.c.DefaultHttpClient] Retrying connect

Not sure what that means

Best Regards,
Jo

Op maandag 5 januari 2015 15:06:41 UTC+1 schreef kaustubh chaudhari: