PuppetDB not working with PostgreSQL

[homerj...@googlemail.com]

Hello,

I'm struggeling with a PuppetDB 3.2.2 Installation on a CentOS 7 x86_64 VM. PuppetDB is starting with the standard hsqldb, but when I switch to postgresql 9.4.5 i get this error message in puppetdb.log and the service did not start:

2015-12-21 09:59:55,656 INFO  [o.e.j.u.log] Logging initialized @44718ms
2015-12-21 10:00:04,209 INFO  [p.t.s.n.nrepl-service] nREPL service disabled, not starting
2015-12-21 10:00:04,210 INFO  [p.t.s.w.jetty9-service] Initializing web server(s).
2015-12-21 10:00:04,333 INFO  [p.p.pdb-routing] Starting PuppetDB, entering maintenance mode
2015-12-21 10:00:04,381 INFO  [p.t.s.w.jetty9-service] Starting web server(s).
2015-12-21 10:00:04,789 INFO  [p.t.s.w.jetty9-core] Starting web server.
2015-12-21 10:00:04,818 INFO  [o.e.j.s.Server] jetty-9.2.z-SNAPSHOT
2015-12-21 10:00:04,902 INFO  [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@370349f{/pdb,null,AVAILABLE}
2015-12-21 10:00:04,973 INFO  [o.e.j.s.ServerConnector] Started ServerConnector@6845b7c8{HTTP/1.1}{localhost:8080}
2015-12-21 10:00:05,152 INFO  [o.e.j.s.ServerConnector] Started ServerConnector@435d2cfe{SSL-HTTP/1.1}{0.0.0.0:8081}
2015-12-21 10:00:05,153 INFO  [o.e.j.s.Server] Started @54218ms
2015-12-21 10:00:05,156 INFO  [p.p.metrics] Starting metrics server
2015-12-21 10:00:05,159 INFO  [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@25f8722c{/metrics,null,AVAILABLE}
2015-12-21 10:00:05,241 INFO  [p.p.c.services] PuppetDB version 3.2.2
2015-12-21 10:00:05,658 INFO  [c.j.b.BoneCP] Shutting down connection pool...
2015-12-21 10:00:05,666 INFO  [c.j.b.BoneCP] Connection pool has been shutdown.
2015-12-21 10:00:05,673 ERROR [p.t.internal] Error during service start!!!
org.postgresql.util.PSQLException: FEHLER: Relation „schema_migrations“ existiert nicht
  Position: 21

The complete log is attached. I can connect to postgresql with the puppetdb user, but no tables get created in the puppetdb-DB. During the puppetdb startup i can see the following lines in the postgresql-log:

< 2015-12-21 10:00:05.638 CET >FEHLER:  Relation „schema_migrations“ existiert nicht bei Zeichen 21
< 2015-12-21 10:00:05.638 CET >ANWEISUNG:  SELECT version FROM schema_migrations ORDER BY version

The puppetdb user can create tables in postgresql and I can see no differences to the puppetdb user on a working local testing-VM with puppet enterprise. I tried the installation with puppetdb-packages from the official puppetlabs yum repo as well as installing with the puppetlabs-module. I got the same errormessage. Changing to nightly builds of puppetdb doesn't change the behaviour.

I'm not sure if the message concerning the missing schema_migrations is the reason for the service not to start, but i cannot find any other hint (debug-logs).

It would be a big help if somebody has a hint for resolving this problem, cause I'm struggeling and searching for 3 days now.

Kind regards
Benjamin

[Russell Mull]

Hi Benjamin, 

This is a recently-introduced regression in PuppetDB. I'm working on it right now, and I'll get you a workaround as soon as I can.  

We're tracking this issue in https://tickets.puppetlabs.com/browse/PDB-2259. There's a workaround listed there, but I should be able to come up with a cleaner one. 

Sorry for the trouble!

-  Russ

[Russell Mull]

I was able to recreate this issue, and it looks like the workaround described in https://tickets.puppetlabs.com/browse/PDB-2259 is your best bet for now. Can you try the given commands in psql and see how it goes? Be sure to connect as the same user puppetdb is using to make sure the tables are created with the correct owner. 

We'll fix this and get a new release out as soon as we can.  

- Russ

[homerj...@googlemail.com]

Hi Russ,

thanks a lot, that fixed my problem!
I'm sorry that I didn't find that ticket myself, but it was created after my extensive google and ticket search and i was concerned with other stuff at that time.

So again, thanks and Merry Christmas!
Benjamin