Problems installing PuppetDB from source - the instructions don't actually work

[Simon Tideswell]

See also https://tickets.puppetlabs.com/projects/PDB/issues/PDB-3938?filter=allissues.

Hello

The instructions here https://puppet.com/docs/puppetdb/5.2/install_from_source.html don't actually work! I'm doing this on Ubuntu 18 (Bionic). I've installed lein from the Ubuntu 18 packages rather than pulling it down from github (in case that is relevant).

When performing this step ...

lein with-profile ezbake ezbake stage

I get the response (after some time) of ...

"Could not transfer metadata puppetlabs:puppetdb:5.3.0-SNAPSHOT/maven-metadata.xml from/to snapshots (https://artifactory.delivery.puppetlabs.net/artifactory/list/clojure-snapshots__local/): Connect to artifactory.delivery.puppetlabs.net:443 [artifactory.delivery.puppetlabs.net/192.69.65.54] failed: Connection timed out (Connection timed out)
Failed to retrieve remote metadata puppetlabs:puppetdb:5.3.0-SNAPSHOT/maven-metadata.xml: Could not transfer metadata puppetlabs:puppetdb:5.3.0-SNAPSHOT/maven-metadata.xml from/to snapshots (https://artifactory.delivery.puppetlabs.net/artifactory/list/clojure-snapshots__local/): Connect to artifactory.delivery.puppetlabs.net:443 [artifactory.delivery.puppetlabs.net/192.69.65.54] failed: Connection timed out (Connection timed out)
Error encountered performing task 'ezbake' with profile(s): 'ezbake'
Failed to retrieve remote metadata puppetlabs:puppetdb:5.3.0-SNAPSHOT/maven-metadata.xml: Could not transfer metadata puppetlabs:puppetdb:5.3.0-SNAPSHOT/maven-metadata.xml from/to snapshots (https://artifactory.delivery.puppetlabs.net/artifactory/list/clojure-snapshots__local/): Connect to artifactory.delivery.puppetlabs.net:443 [artifactory.delivery.puppetlabs.net/192.69.65.54] failed: Connection timed out (Connection timed out)"

Indeed the output is correct, https://artifactory.delivery.puppetlabs.net, is inaccessible from any of the networks I've tried, and has been so for a number of days.

Are there instructions that will work for Joe Average anywhere, i.e. someone who does not have access to that host and who isn't a Java maven (seems more appropriate than the word guru in this instance)? On Ubuntu 16 the OS distributor provides a pre-rolled package for puppetdb. Perhaps Canonical are unable to provide a package on Ubuntu 18 for PuppetDB for similar reasons to me (namely inaccessible sources)?

Thanks, Simon.

[Rob Browning]

Simon Tideswell <stide...@gmail.com> writes:

> The instructions here
> https://puppet.com/docs/puppetdb/5.2/install_from_source.html don't

> actually work! I'm doing this on Ubuntu 18 (Bionic). I've installed *lein*

> from the Ubuntu 18 packages rather than pulling it down from github (in
> case that is relevant).

Hmm, sorry you're having trouble. We'll definitely want to make sure
the instructions work, but first I'd like to make sure I understand your
more immediate interests. Are you looking for packages, or perhaps just
being able to build an uberjar you can run directly, or...?

> *"Could not transfer metadata

> puppetlabs:puppetdb:5.3.0-SNAPSHOT/maven-metadata.xml from/to snapshots
> (https://artifactory.delivery.puppetlabs.net/artifactory/list/clojure-snapshots__local/):
> Connect to artifactory.delivery.puppetlabs.net:443
> [artifactory.delivery.puppetlabs.net/192.69.65.54] failed: Connection timed

> out (Connection timed out)*

Ahh right - we're planning to fix this, and I think someone else has
reported a similar problem:
https://tickets.puppetlabs.com/browse/PDB-3922

I suspect, among possibly other things, we'll move the internal
repositories out of the main profile. For now you should be able to
just comment them out, and any remaining failures would be additional
bugs we'll need to address.

In any case, hopefully we'll be able to get things working for you soon.

--
Rob Browning

[Simon Tideswell]

Hello Rob

No, I'm not trying to do anything fancy. I just wanted to use PuppetDB on a Ubuntu 18 server so that I can access the data with PuppetBoard. I can see that Canonical provided a package for PuppetDB on U14. I actually used the same Package on a U16 server with good results. But the U14 and U16 servers were running Puppet 3.8.x. Now that I'm using Puppet 5 on U18 I thought it was high time that I used a newer PuppetDB package. Canonical don't provide one, there doesn't appear to be one at apt.puppetlabs.com and the instructions for compiling from source are broken, so I was a bit stuck. It's not a high priority, but I find PuppetBoard useful (from a sysadmin perspective).

Simon

[Christopher Wood]

On Thu, Jun 21, 2018 at 06:49:01PM -0700, Simon Tideswell wrote:
> Hello Rob
> No, I'm not trying to do anything fancy. I just wanted to use PuppetDB on
> a Ubuntu 18 server so that I can access the data with PuppetBoard. I can

I've had a good experience using PuppetBoard via Docker per the instructions. I encapsulated the "run it" part in a shell script for easy startup.

https://github.com/voxpupuli/puppetboard#docker-images

That will let you run PuppetDB anywhere you please and run PuppetBoard locally on an ad-hoc basis. (Which is nice since it appears that some of the queries make PuppetDB and PostgreSQL work a bit harder than usual.)

> see that Canonical provided a package for PuppetDB on U14. I actually used
> the same Package on a U16 server with good results. But the U14 and U16
> servers were running Puppet 3.8.x. Now that I'm using Puppet 5 on U18 I
> thought it was high time that I used a newer PuppetDB package. Canonical
> don't provide one, there doesn't appear to be one at apt.puppetlabs.com
> and the instructions for compiling from source are broken, so I was a bit
> stuck. It's not a high priority, but I find PuppetBoard useful (from a
> sysadmin perspective).
> Simon
>
> On Saturday, June 16, 2018 at 7:29:50 AM UTC+10, Rob Browning wrote:
>

> Simon Tideswell <[1]stide...@gmail.com> writes:
>
> > The instructions here

> > [2]https://puppet.com/docs/puppetdb/5.2/install_from_source.html don't

> > actually work! I'm doing this on Ubuntu 18 (Bionic). I've installed
> *lein*
> > from the Ubuntu 18 packages rather than pulling it down from github
> (in
> > case that is relevant).
>
> Hmm, sorry you're having trouble.  We'll definitely want to make sure
> the instructions work, but first I'd like to make sure I understand your
> more immediate interests.  Are you looking for packages, or perhaps just
> being able to build an uberjar you can run directly, or...?
>
> > *"Could not transfer metadata
> > puppetlabs:puppetdb:5.3.0-SNAPSHOT/maven-metadata.xml from/to
> snapshots
> >

> ([3]https://artifactory.delivery.puppetlabs.net/artifactory/list/clojure-snapshots__local/):
> > Connect to [4]artifactory.delivery.puppetlabs.net:443
> > [[5]artifactory.delivery.puppetlabs.net/192.69.65.54] failed:

> Connection timed
> > out (Connection timed out)*
>
> Ahh right - we're planning to fix this, and I think someone else has
> reported a similar problem:

> [6]https://tickets.puppetlabs.com/browse/PDB-3922

>
> I suspect, among possibly other things, we'll move the internal
> repositories out of the main profile.  For now you should be able to
> just comment them out, and any remaining failures would be additional
> bugs we'll need to address.
>
> In any case, hopefully we'll be able to get things working for you soon.
>
> --
> Rob Browning
>

> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [7]puppet-users...@googlegroups.com.
> To view this discussion on the web visit
> [8]https://groups.google.com/d/msgid/puppet-users/7a8bf12d-bda1-4865-8097-da192a1e80c5%40googlegroups.com.
> For more options, visit [9]https://groups.google.com/d/optout.
>
> References
>
> Visible links
> 1. javascript:
> 2. https://puppet.com/docs/puppetdb/5.2/install_from_source.html
> 3. https://artifactory.delivery.puppetlabs.net/artifactory/list/clojure-snapshots__local/
> 4. http://artifactory.delivery.puppetlabs.net:443/
> 5. http://artifactory.delivery.puppetlabs.net/192.69.65.54
> 6. https://tickets.puppetlabs.com/browse/PDB-3922
> 7. mailto:puppet-users...@googlegroups.com
> 8. https://groups.google.com/d/msgid/puppet-users/7a8bf12d-bda1-4865-8097-da192a1e80c5%40googlegroups.com?utm_medium=email&utm_source=footer
> 9. https://groups.google.com/d/optout

[Simon Tideswell]

Thanks Chris

I'll take a look at PuppetBoard when the time comes, though I don't remember it being particularly difficult to install. However I still need to get PuppetDB installed first and that's where I'm stuck. I'm working on something else right now, but will come back to this later. It'd be really good if the instructions on the site could be altered so that they are applicable for the 'general public'.

Simon

[Rob Browning]

Simon Tideswell <stide...@gmail.com> writes:

> No, I'm not trying to do anything fancy. I just wanted to use PuppetDB on a
> Ubuntu 18 server so that I can access the data with PuppetBoard. I can see
> that Canonical provided a package for PuppetDB on U14. I actually used the
> same Package on a U16 server with good results. But the U14 and U16 servers
> were running Puppet 3.8.x. Now that I'm using Puppet 5 on U18 I thought it
> was high time that I used a newer PuppetDB package. Canonical don't provide
> one, there doesn't appear to be one at apt.puppetlabs.com and the
> instructions for compiling from source are broken, so I was a bit stuck.
> It's not a high priority, but I find PuppetBoard useful (from a sysadmin
> perspective).

I just noticed something that might be relevant from your earlier
message, and I'll plan to investigate Monday and get back to you, but in
any case, if you'd like an interim fix, and are comfortable just running
puppetdb directly, you should be able to do this:

lein uberjar

and then run puppetdb directly:

java -cp target/puppetdb.jar clojure.main \
-m puppetlabs.puppetdb.core services \
-c your-config-file

Of course you might also want to add some jvm heap arguments, etc.

And unless the data's not very important, I'd suggest sticking with
checkouts of releases, i.e. "git checkout 5.2.2" (before you build the
uberjar) etc.

Hope this helps
--
Rob Browning

[Trevor Vaughan]

Hi Simon,

While this isn't directly related to your question, you might want to pile onto this ticket to ensure that the community can build all of the package artifacts.

https://tickets.puppetlabs.com/browse/LTH-156

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "Puppet Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/b20aa70d-9663-49d2-b5ad-dfe09966eb04%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Trevor Vaughan
Vice President, Onyx Point, Inc

(410) 541-6699 x788

-- This account not approved for unencrypted proprietary information --

[Simon Tideswell]

Hello Rob

Thanks for that information. lein uberjar worked. But it built a jar that does not contain every library necessary to run PuppetDB.

Some, truncated, output from lein ...

puppetserver test depdency unconfigured (ignoring)

-- blah blah blah ---

Warning: The Main-Class specified does not exist within the jar. It may not be executable as expected. A gen-class directive may be missing in the namespace which contains the main method, or the namespace has not been AOT-compiled.

Created /home/simont/puppetdb/target/puppetdb-5.3.0-SNAPSHOT.jar

Warning: The Main-Class specified does not exist within the jar. It may not be executable as expected. A gen-class directive may be missing in the namespace which contains the main method, or the namespace has not been AOT-compiled.

Created /home/simont/puppetdb/target/test/puppetdb-5.3.0-SNAPSHOT-test.jar

Created /home/simont/puppetdb/target/puppetdb.jar

I haven't got a config file yet but this is what happens when I run PuppetDB ...

java -cp target/puppetdb.jar clojure.main -m puppetlabs.puppetdb.core.services 

Exception in thread "main" java.io.FileNotFoundException: Could not locate puppetlabs/puppetdb/core/services__init.class or puppetlabs/puppetdb/core/services.clj on classpath.

at clojure.lang.RT.load(RT.java:463)

at clojure.lang.RT.load(RT.java:426)

at clojure.core$load$fn__6548.invoke(core.clj:6046)

at clojure.core$load.invokeStatic(core.clj:6045)

at clojure.core$load.doInvoke(core.clj:6029)

at clojure.lang.RestFn.invoke(RestFn.java:408)

at clojure.core$load_one.invokeStatic(core.clj:5848)

at clojure.core$load_one.invoke(core.clj:5843)

at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)

at clojure.core$load_lib.invokeStatic(core.clj:5887)

at clojure.core$load_lib.doInvoke(core.clj:5868)

at clojure.lang.RestFn.applyTo(RestFn.java:142)

at clojure.core$apply.invokeStatic(core.clj:659)

at clojure.core$load_libs.invokeStatic(core.clj:5925)

at clojure.core$load_libs.doInvoke(core.clj:5909)

at clojure.lang.RestFn.applyTo(RestFn.java:137)

at clojure.core$apply.invokeStatic(core.clj:659)

at clojure.core$require.invokeStatic(core.clj:5947)

at clojure.main$main_opt.invokeStatic(main.clj:317)

at clojure.main$main_opt.invoke(main.clj:313)

at clojure.main$main.invokeStatic(main.clj:424)

at clojure.main$main.doInvoke(main.clj:387)

at clojure.lang.RestFn.applyTo(RestFn.java:137)

at clojure.lang.Var.applyTo(Var.java:702)

at clojure.main.main(main.java:37)

And inspecting the jar ....

jar -tf target/puppetdb.jar  | grep core | grep services

puppetlabs/trapperkeeper/services/status/status_core.clj

puppetlabs/trapperkeeper/services/watcher/filesystem_watch_core.clj

puppetlabs/trapperkeeper/services/metrics/metrics_core.clj

puppetlabs/trapperkeeper/services/webrouting/webrouting_service_core.clj

puppetlabs/trapperkeeper/services/webserver/jetty9_core.clj

puppetlabs/trapperkeeper/services/scheduler/scheduler_core.clj

META-INF/services/com.fasterxml.jackson.core.ObjectCodec

META-INF/services/com.fasterxml.jackson.core.JsonFactory

This file does exist in the jar however, puppetlabs/puppetdb/core.clj.

Simon

[Simon Tideswell]

Hello

Perhaps I can tackle this a different way. As mentioned in my original post I used the Ubuntu 14 Puppet DB package on Ubuntu 16 because Ubuntu no longer provided a PuppetDB package for Xenial. It's Java so as long as the schema is unchanged it should still work, right? So has the schema changed between PuppetDB 2.3.8 and the version of PuppetDB that would most commonly accompany Puppet 5.4.x? If someone knowledgeable could let me know, or provide the schema conversion SQL then I could do that until a more elegant fix is provided by Puppet themselves. I couldn't find any files that looked like they provided the schema or schema conversion within the source code tree.

Simon

On Thursday, June 14, 2018 at 10:01:14 AM UTC+10, Simon Tideswell wrote:

[Rob Browning]

Simon Tideswell <stide...@gmail.com> writes:

> Perhaps I can tackle this a different way. As mentioned in my original post
> I used the Ubuntu 14 Puppet DB package on Ubuntu 16 because Ubuntu no
> longer provided a PuppetDB package for Xenial. It's Java so as long as the
> schema is unchanged it should still work, right? So has the schema changed
> between PuppetDB 2.3.8 and the version of PuppetDB that would most commonly
> accompany Puppet 5.4.x? If someone knowledgeable could let me know, or
> provide the schema conversion SQL then I could do that until a more elegant
> fix is provided by Puppet themselves. I couldn't find any files that looked
> like they provided the schema or schema conversion within the source code
> tree.

With respect to compatibility among versions:

https://puppet.com/docs/puppetdb/5.2/versioning_policy.html#upgrades

Which means that if you want to go from puppetdb 3 to 6, you'll need to
at least fire up some version of 5 and wait for it to finish all the
database migrations before shutting it down. (At startup puppetdb will
apply any that are necessary.)

And as mentioned, if you can get by without a package, you could also
just build an uberjar of whichever version you like from the source tree
and then run it manually. All you should need is a recent version of
leiningen, openjdk 8 or similar, and the relevant puppetdb source tree.
(I'm happy to help with that process if it seems useful.)

Or, perhaps even simpler than the uberjar in the short run (if it's
feasible), you can run the server directly from the source tree like
this:

lein trampoline run services -c your-puppetdb.ini

In any case commenting out the :repositories section in the project.clj
should fix the artifactory problem, but I don't know if that's enough to
allow you to build packages.

Thanks
--
Rob Browning

[Rob Browning]

Simon Tideswell <stide...@gmail.com> writes:

> Hello Rob
>

> Thanks for that information. *lein uberjar *worked. But it built a jar that

> does not contain every library necessary to run PuppetDB.

Oh, I missed this message before my last reply, and I think I may have
just gotten the invocation slightly wrong. How about this:

java -cp target/puppetdb.jar clojure.main -m puppetlabs.puppetdb.main \
services -c foo.ini

You might also eventually want some other arguments (setting the heap,
etc.). I suspect whatever invocation's in the existing packages might
be fine.

Thanks
--
Rob Browning

[Simon Tideswell]

Hello Rob

Thanks for that. I can now get PuppetDB to start following your instructions. I'm running on Ubuntu 18 which uses PostgreSQL 10 and this version of PostgreSQL doesn't like some aspects of the SQL being performed by the JAR.

Excerpt from the log file ...

-------------------------------------------------------------------------------------------------------------------------------------

2018-07-24 22:05:52,203 INFO  [p.p.c.services] Finished gc packages

2018-07-24 22:05:52,204 INFO  [p.p.c.services] Starting database garbage collection

2018-07-24 22:05:52,250 WARN  [c.z.h.p.ProxyConnection] PDBWritePool - Connection org.postgresql.jdbc.PgConnection@6e32d291 marked as broken because of SQLSTATE(0A000), ErrorCode(0)

java.sql.BatchUpdateException: Batch entry 0 with recursive live_paths(key, path, value) as  (select key, key as path, value     from (select (jsonb_each(stable||volatile)).*             from factsets) as base_case   union all   select sub_path as key,          sub_paths.path||'#~'||sub_path as path,          sub_value as value     from (select *             from (select path,                          case jsonb_typeof(value)                            when 'object'                              then (jsonb_each(value)).key                            when 'array'                              then generate_series(0, jsonb_array_length(value - 1))::text                            end                            as sub_path,                          case jsonb_typeof(value)                            when 'object'                              then (jsonb_each(value)).value                            when 'array'                              then jsonb_array_elements(value)                          end                          as sub_value                     from live_paths) as candidates             where candidates.sub_path is not null)               as sub_paths)  delete from fact_paths fp    where not exists (select 1 from live_paths                        where live_paths.path = fp.path) was aborted: ERROR: set-returning functions are not allowed in CASE

  Hint: You might be able to move the set-returning function into a LATERAL FROM item.

  Position: 559  Call getNextException to see other errors in the batch.

at org.postgresql.jdbc.BatchResultHandler.handleError(BatchResultHandler.java:145)

at org.postgresql.core.ResultHandlerDelegate.handleError(ResultHandlerDelegate.java:50)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2179)

at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:479)

at org.postgresql.jdbc.PgStatement.executeBatch(PgStatement.java:835)

at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:128)

at com.zaxxer.hikari.pool.HikariProxyStatement.executeBatch(HikariProxyStatement.java)

at clojure.java.jdbc$execute_batch.invokeStatic(jdbc.clj:533)

at clojure.java.jdbc$execute_batch.invoke(jdbc.clj:526)

at clojure.java.jdbc$db_do_commands$fn__24916.invoke(jdbc.clj:873)

at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:784)

at clojure.java.jdbc$db_transaction_STAR_.invoke(jdbc.clj:711)

at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:724)

at clojure.java.jdbc$db_transaction_STAR_.invoke(jdbc.clj:711)

at clojure.java.jdbc$db_do_commands.invokeStatic(jdbc.clj:872)

at clojure.java.jdbc$db_do_commands.invoke(jdbc.clj:857)

at puppetlabs.puppetdb.jdbc$do_commands.invokeStatic(jdbc.clj:38)

at puppetlabs.puppetdb.jdbc$do_commands.doInvoke(jdbc.clj:33)

at clojure.lang.RestFn.invoke(RestFn.java:408)

at puppetlabs.puppetdb.scf.storage$delete_unused_fact_paths.invokeStatic(storage.clj:1127)

at puppetlabs.puppetdb.scf.storage$delete_unused_fact_paths.invoke(storage.clj:1123)

at puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_$fn__32259$fn__32262.invoke(storage.clj:1439)

at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__25578$fn__25579.invoke(jdbc.clj:483)

at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:741)

at clojure.java.jdbc$db_transaction_STAR_.invoke(jdbc.clj:711)

at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__25578.invoke(jdbc.clj:482)

at puppetlabs.puppetdb.jdbc$eval25552$retry_sql_STAR___25557$fn__25558$fn__25559.invoke(jdbc.clj:454)

at puppetlabs.puppetdb.jdbc$eval25552$retry_sql_STAR___25557$fn__25558.invoke(jdbc.clj:453)

at puppetlabs.puppetdb.jdbc$eval25552$retry_sql_STAR___25557.invoke(jdbc.clj:444)

at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn.invokeStatic(jdbc.clj:480)

at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn.invoke(jdbc.clj:469)

at puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_$fn__32259.invoke(storage.clj:1438)

at puppetlabs.puppetdb.scf.storage.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source)

at com.codahale.metrics.Timer.time(Timer.java:101)

at puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_.invokeStatic(storage.clj:1432)

at puppetlabs.puppetdb.scf.storage$garbage_collect_BANG_.invoke(storage.clj:1429)

at puppetlabs.puppetdb.cli.services$garbage_collect_BANG_.invokeStatic(services.clj:189)

at puppetlabs.puppetdb.cli.services$garbage_collect_BANG_.invoke(services.clj:180)

at puppetlabs.puppetdb.cli.services$eval40243$clean_up__40248$fn__40252$fn__40262.invoke(services.clj:259)

at puppetlabs.puppetdb.cli.services.proxy$java.lang.Object$Callable$7da976d4.call(Unknown Source)

at com.codahale.metrics.Timer.time(Timer.java:101)

at puppetlabs.puppetdb.cli.services$eval40243$clean_up__40248$fn__40252.invoke(services.clj:258)

at puppetlabs.puppetdb.cli.services$eval40243$clean_up__40248.invoke(services.clj:220)

at puppetlabs.puppetdb.cli.services$collect_garbage.invokeStatic(services.clj:373)

at puppetlabs.puppetdb.cli.services$collect_garbage.invoke(services.clj:368)

at puppetlabs.puppetdb.cli.services$start_puppetdb$fn__40323.invoke(services.clj:434)

at clojure.lang.AFn.run(AFn.java:22)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Caused by: org.postgresql.util.PSQLException: ERROR: set-returning functions are not allowed in CASE

  Hint: You might be able to move the set-returning function into a LATERAL FROM item.

  Position: 559

at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)

at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)

... 51 common frames omitted

2018-07-24 22:05:52,263 ERROR [p.p.c.services] Error during garbage collection

clojure.lang.ExceptionInfo: Rollback failed handling "Batch entry 0 with recursive live_paths(key, path, value) as  (select key, key as path, value     from (select (jsonb_each(stable||volatile)).*             from factsets) as base_case   union all   select sub_path as key,          sub_paths.path||'#~'||sub_path as path,          sub_value as value     from (select *             from (select path,                          case jsonb_typeof(value)                            when 'object'                              then (jsonb_each(value)).key                            when 'array'                              then generate_series(0, jsonb_array_length(value - 1))::text                            end                            as sub_path,                          case jsonb_typeof(value)                            when 'object'                              then (jsonb_each(value)).value                            when 'array'                              then jsonb_array_elements(value)                          end                          as sub_value                     from live_paths) as candidates             where candidates.sub_path is not null)               as sub_paths)  delete from fact_paths fp    where not exists (select 1 from live_paths                        where live_paths.path = fp.path) was aborted: ERROR: set-returning functions are not allowed in CASE

  Hint: You might be able to move the set-returning function into a LATERAL FROM item.

  Position: 559  Call getNextException to see other errors in the batch."

at clojure.core$ex_info.invokeStatic(core.clj:4739)

at clojure.core$ex_info.invoke(core.clj:4739)

at clojure.java.jdbc$db_transaction_STAR_$fn__24869.invoke(jdbc.clj:751)

at clojure.java.jdbc$db_transaction_STAR_.invokeStatic(jdbc.clj:747)

at clojure.java.jdbc$db_transaction_STAR_.invoke(jdbc.clj:711)

at puppetlabs.puppetdb.jdbc$with_transacted_connection_fn$fn__25578.invoke(jdbc.clj:482)

at puppetlabs.puppetdb.jdbc$eval25552$retry_sql_STAR___25557$fn__25558$fn__25559.invoke(jdbc.clj:454)

-------------------------------------------------------------------------------------------------------------------------------------

I can probably use a remote DB running on U16 with PostgreSQL 9.5 and I'm hopeful that this will get me working.

Thanks for your help thus far. I will let you know if I finally get this working.

Simon

[Rob Browning]

Simon Tideswell <stide...@gmail.com> writes:

> Thanks for that. I can now get PuppetDB to start following your
> instructions. I'm running on Ubuntu 18 which uses PostgreSQL 10 and this
> version of PostgreSQL doesn't like some aspects of the SQL being performed
> by the JAR.

Ahh, right, PuppetDB is not compatible with PostgreSQL 10 yet, but we
hope to fix that reasonably soon.

--
Thanks

[Simon Tideswell]

Hello Rob

Do you have a rough idea of the timeframe for Postgres 10 support? Now that I know the process of building a JAR you will find a will tester in me.

Simon

[Simon Tideswell]

"Now that I know the process of building a JAR you will find a willING tester in me."

Arrggh,  I hate it when I do that!!!!

Simon

[Rob Browning]

Simon Tideswell <stide...@gmail.com> writes:

> Do you have a rough idea of the timeframe for Postgres 10 support?

We don't yet, though I think it's possible it'll make it in before the 6.0
release in a couple of months.

Thanks
--
Rob Browning