Failed to upgrade a PuppetDB that runs into a non-standard directories

[Benjamin Parmentier]

Hi,

I actually have a PuppetDB 2.2.2 that runs without any problems into a non standard directories :

/SERVICES/puppetdb/{conf,logs,tmp,var}

I also use a custom script to start/stop PuppetDB.

I extract the command to start puppetdb from the init-script :

/usr/bin/java -Xmx192m -XX:OnOutOfMemoryError=kill -9 %p -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/SERVICES/puppetdb/logs/puppetdb-oom.hprof -Djava.security.egd=file:/dev/urandom -cp /usr/share/puppetdb/puppetdb.jar clojure.main -m com.puppetlabs.puppetdb.core services -cp /SERVICES/puppetdb/conf

I tried to upgrade to the version 2.3.5 and puppetdb won't start anymore... I have this stracktrace in log file :

2015-07-09 17:29:48,677 INFO  [o.e.j.u.log] Logging initialized @41541ms

2015-07-09 17:29:50,738 INFO  [p.t.s.w.jetty9-service] Initializing web server(s).

2015-07-09 17:29:50,752 INFO  [p.t.s.w.jetty9-service] Starting web server(s).

2015-07-09 17:29:51,028 INFO  [p.t.s.w.jetty9-core] Starting web server.

2015-07-09 17:29:51,039 INFO  [o.e.j.s.Server] jetty-9.2.z-SNAPSHOT

2015-07-09 17:29:51,147 INFO  [o.e.j.s.ServerConnector] Started ServerConnector@5442accb{HTTP/1.1}{0.0.0.0:8082}

2015-07-09 17:29:51,282 INFO  [o.e.j.s.ServerConnector] Started ServerConnector@5d63d496{SSL-HTTP/1.1}{0.0.0.0:8081}

2015-07-09 17:29:51,282 INFO  [o.e.j.s.Server] Started @44151ms

2015-07-09 17:29:51,284 WARN  [c.p.p.config] The configuration item `url-prefix` in the [global] section is deprecated. It will be removed in the future.

2015-07-09 17:29:51,334 ERROR [p.t.internal] Error during service start!!!

clojure.lang.ExceptionInfo: Value does not match schema: {:webapps disallowed-key}

        at schema.core$validate.invoke(core.clj:161) ~[na:na]

        at com.puppetlabs.puppetdb.config$configure_puppetdb.invoke(config.clj:217) ~[na:na]

        at com.puppetlabs.puppetdb.config$convert_config.invoke(config.clj:224) ~[na:na]

        at com.puppetlabs.puppetdb.config$process_config_BANG_.invoke(config.clj:364) ~[na:na]

        at com.puppetlabs.puppetdb.cli.services$start_puppetdb.invoke(services.clj:261) ~[na:na]

        at com.puppetlabs.puppetdb.cli.services$reify__21278$service_fnk__17647__auto___positional$reify__21289.start(services.clj:366) ~[na:na]

        at puppetlabs.trapperkeeper.services$eval17483$fn__17497$G__17473__17500.invoke(services.clj:8) ~[na:na]

        at puppetlabs.trapperkeeper.services$eval17483$fn__17497$G__17472__17504.invoke(services.clj:8) ~[na:na]

        at puppetlabs.trapperkeeper.internal$run_lifecycle_fn_BANG_.invoke(internal.clj:152) ~[na:na]

        at puppetlabs.trapperkeeper.internal$run_lifecycle_fns.invoke(internal.clj:180) ~[na:na]

        at puppetlabs.trapperkeeper.internal$build_app_STAR_$reify__19027.start(internal.clj:447) [na:na]

        at puppetlabs.trapperkeeper.internal$boot_services_STAR_$fn__19039.invoke(internal.clj:471) [na:na]

        at puppetlabs.trapperkeeper.internal$boot_services_STAR_.invoke(internal.clj:469) [na:na]

        at puppetlabs.trapperkeeper.core$boot_with_cli_data.invoke(core.clj:113) [na:na]

        at puppetlabs.trapperkeeper.core$run.invoke(core.clj:144) [na:na]

        at puppetlabs.trapperkeeper.core$main.doInvoke(core.clj:159) [na:na]

        at clojure.lang.RestFn.applyTo(RestFn.java:137) [puppetdb.jar:na]

        at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na]

        at com.puppetlabs.puppetdb.cli.services$_main.doInvoke(services.clj:373) [na:na]

        at clojure.lang.RestFn.invoke(RestFn.java:421) [puppetdb.jar:na]

        at clojure.lang.Var.invoke(Var.java:383) [puppetdb.jar:na]

        at clojure.lang.AFn.applyToHelper(AFn.java:156) [puppetdb.jar:na]

        at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na]

        at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na]

        at com.puppetlabs.puppetdb.core$run_command.invoke(core.clj:87) [na:na]

        at com.puppetlabs.puppetdb.core$_main.doInvoke(core.clj:95) [na:na]

        at clojure.lang.RestFn.invoke(RestFn.java:436) [puppetdb.jar:na]

        at clojure.lang.Var.invoke(Var.java:388) [puppetdb.jar:na]

        at clojure.lang.AFn.applyToHelper(AFn.java:160) [puppetdb.jar:na]

        at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na]

        at clojure.core$apply.invoke(core.clj:624) [puppetdb.jar:na]

        at clojure.main$main_opt.invoke(main.clj:315) [puppetdb.jar:na]

        at clojure.main$main.doInvoke(main.clj:420) [puppetdb.jar:na]

        at clojure.lang.RestFn.invoke(RestFn.java:482) [puppetdb.jar:na]

        at clojure.lang.Var.invoke(Var.java:401) [puppetdb.jar:na]

        at clojure.lang.AFn.applyToHelper(AFn.java:171) [puppetdb.jar:na]

        at clojure.lang.Var.applyTo(Var.java:700) [puppetdb.jar:na]

        at clojure.main.main(main.java:37) [puppetdb.jar:na]

2015-07-09 17:29:51,341 INFO  [p.t.internal] Beginning shutdown sequence

2015-07-09 17:29:51,343 INFO  [c.p.p.c.services] Shutdown request received; puppetdb exiting.

2015-07-09 17:29:51,345 INFO  [p.t.s.w.jetty9-service] Shutting down web server(s).

2015-07-09 17:29:51,346 INFO  [p.t.s.w.jetty9-core] Shutting down web server.

2015-07-09 17:29:51,359 INFO  [o.e.j.s.ServerConnector] Stopped ServerConnector@5442accb{HTTP/1.1}{10.19.70.42:8082}

2015-07-09 17:29:51,365 INFO  [o.e.j.s.ServerConnector] Stopped ServerConnector@5d63d496{SSL-HTTP/1.1}{0.0.0.0:8081}

2015-07-09 17:29:51,371 INFO  [p.t.s.w.jetty9-core] Web server shutdown

2015-07-09 17:29:51,372 INFO  [p.t.internal] Finished shutdown sequence

Any ideas ?

Regards,

Benjamin

[Wyatt Alt]

Hey Benjamin,

This is a config validation error. You have a "webapps" setting somewhere in your config directory. That's not a setting PuppetDB will use, and PDB will fail to start if it's present. Remove that setting and things should work fine.

Wyatt

[Benjamin Parmentier]

Indeed... A puppetdb.webapps parameter in my custom config file was the problem. Renaming it solves the issue.

Thanks for your help.