Re: [Puppet Users] puppetdb postgresql Connection refused

[Nikola Petrov]

On Mon, Nov 05, 2012 at 01:40:18AM -0800, linux@bami wrote:
> Greetings,
>
> we are trying to setup puppetdb, nut our clients get the following error:
>
> Warning: Unable to fetch my node definition, but the agent run will
> continue:
> Warning: Error 400 on SERVER: Could not retrieve facts for lxa7t.unix.lan:
> Failed to submit 'replace facts' command for lxa7t.unix.lan to PuppetDB at
> puppetdb:8081: Connection refused - connect(2)
> Info: Retrieving plugin
> Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryhugepages.rb
> Info: Loading facts in /var/lib/puppet/lib/facter/bstoragemanufacturer.rb
> Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryshmmax.rb
> Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryshmall.rb
> Error: Could not retrieve catalog from remote server: Error 400 on SERVER:
> Failed to submit 'replace facts' command for lxa7t.unix.lan to PuppetDB at
> puppetdb:8081: Connection refused - connect(2)
> Warning: Not using cache on failed catalog
> Error: Could not retrieve catalog; skipping run
>
>
> The puppetdb is remote for the puppetmaster and is using postgresql.
Can you post your puppet.conf master section and puppetdb.conf please?
>
> Direct connect from puppetmaster to the postgresql puppetdb is vaild.
By direct you mean that you can run

puppet agent -t

on the master and it's working? Did you try to connect to the database
only with telnet as described here - http://docs.puppetlabs.com/puppetdb/0.9/install.html#troubleshooting-installation-problems


>
> We don't see any hint in the logfiles.
>
>
> R/ Daniel

Best, Nikola

[linux@bami]

Hi Nikola,

thanks for the response!

here are our configs:

 cat /etc/puppet/puppet.conf

[main]

    server = lx1ml.unix.lan

    logdir = /var/log/puppet

    rundir = /var/run/puppet

    confdir = /app/puppet/etc

    vardir = /app/puppet/var

    ssldir = $vardir/ssl

    modulepath = $confdir/modules

    manifest = $confdir/manifests/site.pp

    pluginsync = true

[master]

    certname = lx1ml.unix.lan

    dns_alt_names = lx1ml.unix.lan

    ssl_client_header = SSL_CLIENT_S_DN

    ssl_client_verify_header = SSL_CLIENT_VERIFY

    reports = tagmail,http

    reportfrom = linux-...@xxxxxx.de

#    reporturl = http://lx1ml.unix.lan:3000/reports/upload

    storeconfigs = true

    storeconfigs_backend = puppetdb

#    facts_terminus = yaml

#    facts_terminus = inventory_active_record

#    dbadapter = mysql

#    dbname = dashboard

#    dbuser = dashboard

#    dbpassword = xxxxx

#    dbserver = puppetdb.unix.lan

#    dbport = 3333

[production]

    modulepath = /app/puppet/env/production/modules

    manifest = /app/puppet/env/production/manifests/site.pp

[development]

    modulepath = /app/puppet/env/development/modules

    manifest = /app/puppet/env/development/manifests/site.pp

cat /etc/puppet/puppetdb.conf

[main]

server = lxj2j.unix.lan

port = 8081

Direkt connect from the puppetmaster to the puppetdb postgresql via psql

[lx1ml ~]# psql -h lxj2j -U puppetdb -d puppetdb

Telnet Connect is working:

[root@lx1ml ~]# telnet lxj2j 8081

Trying 172.20.250.153...

Connected to lxj2j.

Escape character is '^]'.

^CConnection closed by foreign host.

Puppetdb config from lxj2j:

cat /etc/puppetdb/conf.d/jetty.ini

[jetty]

# Hostname to list for clear-text HTTP.  Default is localhost

#host = localhost

# Port to listen on for clear-text HTTP.

port = 8080

ssl-host = lxj2j.unix.lan

ssl-port = 8081

keystore = /etc/puppetdb/ssl/keystore.jks

truststore = /etc/puppetdb/ssl/truststore.jks

removed passwords from output

[root@lxj2j data]# cat /etc/puppetdb/conf.d/database.ini |grep -v "^#"

[database]

classname = org.postgresql.Driver

subprotocol = postgresql

subname = //localhost:5432/puppetdb

log-slow-statements = 10

syntax_pgs = true

gc-interval = 60

username = puppetdb

password = puppetdb

Greetings 

Daniel

Am Montag, 5. November 2012 10:40:18 UTC+1 schrieb linux@bami:

Greetings,

we are trying to setup puppetdb, nut our clients get the following error:

Warning: Unable to fetch my node definition, but the agent run will continue:

Warning: Error 400 on SERVER: Could not retrieve facts for lxa7t.unix.lan: Failed to submit 'replace facts' command for lxa7t.unix.lan to PuppetDB at puppetdb:8081: Connection refused - connect(2)

Info: Retrieving plugin

Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryhugepages.rb

Info: Loading facts in /var/lib/puppet/lib/facter/bstoragemanufacturer.rb

Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryshmmax.rb

Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryshmall.rb

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed to submit 'replace facts' command for lxa7t.unix.lan to PuppetDB at puppetdb:8081: Connection refused - connect(2)

Warning: Not using cache on failed catalog

Error: Could not retrieve catalog; skipping run

The puppetdb is remote for the puppetmaster and is using postgresql.

Direct connect from puppetmaster to the postgresql puppetdb is vaild.

[Nikola Petrov]

Strange ;; everything seems fine to me. Recheck the logs of the master,
the agent and the db please. Can you also make sure that you have a
signed certificate on the puppetdb host as described here -
http://docs.puppetlabs.com/puppetdb/0.9/install.html#step-1-install-and-configure-puppet.
i.e make sure that

puppet agent -t

runs properly there.

P.S. Also did you add a routes.yaml file?

Best, Nikola

[linux@bami]

Hi Nikola,

yes on the puppet master there ist a routes.yaml

[root@lx1ml puppet]# cat routes.yaml

---

master:

  facts:

    terminus: puppetdb

    cache: yaml

And the puppet agent -t runs fine on the puppetdb if we remove the storeconfig settings from our master puppet.conf

Is it possible to increase the jetty log, because we don't the any connection attempts or Connection refused.

greetings 

daniel

[linux@bami]

Hi,

so we figured out the first problem.

We had an alias puppetdb.fqdn which was pointing to out puppet dashboard db. It seems that despite from the entry in the puppetdb.conf, puppet uses puppetdb as database server.

we changed the alias and now we are getting an ssl error:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed to submit 'replace facts' command for lxa7t.unix.lan to PuppetDB at puppetdb:8081: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [unable to get local issuer certificate for /CN=lxj2j.unix.lan]

Warning: Not using cache on failed catalog

Error: Could not retrieve catalog; skipping run

any idea ?! 

thanks and greetings.

Daniel

[Nikola Petrov]

On Mon, Nov 05, 2012 at 05:36:39AM -0800, linux@bami wrote:
> Hi Nikola,
>
> yes on the puppet master there ist a routes.yaml
>
> [root@lx1ml puppet]# cat routes.yaml
> ---
> master:
> facts:
> terminus: puppetdb
> cache: yaml
>
> And the puppet agent -t runs fine on the puppetdb if we remove the
> storeconfig settings from our master puppet.conf
>
> Is it possible to increase the jetty log, because we don't the any
> connection attempts or Connection refused.

Well on my machine the file is at /etc/puppetdb/log4j.properties. There
you can configure the logging level which in my case is INFO and this should
be more than enough. Did you check the system log for your environment
for problems. For example on my ubuntu machine most of the messages from
the master go to /var/log/syslog.

Where are you looking for the puppetdb and puppet logs

>
>
> greetings
>
> daniel

Best, Nikola

>
>
> Am Montag, 5. November 2012 10:40:18 UTC+1 schrieb linux@bami:
> >
> > Greetings,
> >
> > we are trying to setup puppetdb, nut our clients get the following error:
> >
> > Warning: Unable to fetch my node definition, but the agent run will
> > continue:
> > Warning: Error 400 on SERVER: Could not retrieve facts for lxa7t.unix.lan:
> > Failed to submit 'replace facts' command for lxa7t.unix.lan to PuppetDB at
> > puppetdb:8081: Connection refused - connect(2)
> > Info: Retrieving plugin
> > Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryhugepages.rb
> > Info: Loading facts in /var/lib/puppet/lib/facter/bstoragemanufacturer.rb
> > Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryshmmax.rb
> > Info: Loading facts in /var/lib/puppet/lib/facter/bmemoryshmall.rb
> > Error: Could not retrieve catalog from remote server: Error 400 on SERVER:
> > Failed to submit 'replace facts' command for lxa7t.unix.lan to PuppetDB at
> > puppetdb:8081: Connection refused - connect(2)
> > Warning: Not using cache on failed catalog
> > Error: Could not retrieve catalog; skipping run
> >
> >
> > The puppetdb is remote for the puppetmaster and is using postgresql.
> >
> > Direct connect from puppetmaster to the postgresql puppetdb is vaild.
> >
> > We don't see any hint in the logfiles.
> >
> >
> > R/ Daniel
> >
> >
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/OxMm0hZvrTcJ.
> To post to this group, send email to puppet...@googlegroups.com.
> To unsubscribe from this group, send email to puppet-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
>

[Nikola Petrov]

On Mon, Nov 05, 2012 at 06:26:35AM -0800, linux@bami wrote:
> Hi,
>
> so we figured out the first problem.
> We had an alias puppetdb.fqdn which was pointing to out puppet dashboard
> db. It seems that despite from the entry in the puppetdb.conf, puppet uses
> puppetdb as database server.
>
> we changed the alias and now we are getting an ssl error:
>
> Error: Could not retrieve catalog from remote server: Error 400 on SERVER:
> Failed to submit 'replace facts' command for lxa7t.unix.lan to PuppetDB at
> puppetdb:8081: SSL_connect returned=1 errno=0 state=SSLv3 read server
> certificate B: certificate verify failed: [unable to get local issuer
> certificate for /CN=lxj2j.unix.lan]
> Warning: Not using cache on failed catalog
> Error: Could not retrieve catalog; skipping run
>

This looks like an SSL error to me. Try to remove the storeconfigs and
make a clean

puppet agent -t

from puppetdb host before you enable it again.

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/atH0EqHpJrUJ.

[linux@bami]

Hi,

we solved the problem. removed all files from /etc/puppetdb/ssl and ran /usr/sbin/puppetdb-ssl-setup.

Thanks for the support!!

greetings

Daniel

[Nikola Petrov]

On Mon, Nov 05, 2012 at 07:52:02AM -0800, linux@bami wrote:
> Hi,
>
> we solved the problem. removed all files from /etc/puppetdb/ssl and
> ran /usr/sbin/puppetdb-ssl-setup.
>
> Thanks for the support!!
>
> greetings
>
> Daniel
>

I am glad that you managed to solve it.

> > > To post to this group, send email to puppet...@googlegroups.com<javascript:>.

> >
> > > To unsubscribe from this group, send email to

> > puppet-users...@googlegroups.com <javascript:>.

> > > For more options, visit this group at
> > http://groups.google.com/group/puppet-users?hl=en.
> > >
> >
>

> --
> You received this message because you are subscribed to the Google Groups "Puppet Users" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/QuC4XBMIDjoJ.