Jira (PDB-1013) (puppetdb) Can't change 'subname' setting in database.ini

[Kenneth Barber (JIRA)]

Kenneth Barber moved an issue

PuppetDB / PDB-1013 (puppetdb) Can't change 'subname' setting in database.ini Change By: Kenneth Barber Workflow: Forge Engineering  Workflow Key: FM PDB - 1740 1013 Project: Forge Modules [Internal] PuppetDB Add Comment

This message was sent by Atlassian JIRA (v6.3.7#6337-sha1:2ed701e)

[Brian LaMetterey (JIRA)]

Brian LaMetterey updated an issue

PuppetDB / PDB-1013 (puppetdb) Can't change 'subname' setting in database.ini

Change By: Brian LaMetterey Assignee: Colleen Murphy

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013 (puppetdb) Can't change 'subname' setting in database.ini

Change By: Kenneth Barber Component/s: Module Labels: module puppetdb

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber commented on PDB-1013

Re: (puppetdb) Can't change 'subname' setting in database.ini Justin Holguin why do you need to do this? At the moment we already automatically populate that field based on what database a user chooses, what other use case are you trying to achieve here?

Add Comment

[Justin Holguin (JIRA)]

Justin Holguin commented on PDB-1013

Re: (puppetdb) Can't change 'subname' setting in database.ini

Kenneth Barber this all started with me trying to document using SSL between PDB and Postgres for DOC-145/PDB-765. It's been four months now so I don't quite remember all the details, but the problem I was running into is at the bottom of my notes.

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber commented on PDB-1013

Re: (puppetdb) Can't change 'subname' setting in database.ini

Justin Holguin oh that makes sense. Maybe ... the requirement here is more global like 'make puppetdb module support talking to postgresql via SSL'. That way, we're no longer talking about downstream bugs in this effort, but more of a 'lets add a new feature' and the problems that block this will be fixed in turn. That is, adding SSL setup should almost be turnkey, flick a few options and postgresql & puppetdb are talking SSL. Then the documentation becomes minimal? So maybe we just make this a new feature ticket, change the topic and description to be more about 'Module: making puppetdb & postgresql talk via SSL' and then scope it around that. Any documentation changes to the module and our docs would be part of this effort I would imagine. What do you think? Otherwise, or alternatively, if you believe this to be the only work stopping someone shipping some decent documentation for this then maybe we keep it as it is. I really don't know where your work ended up, or if its hard blocking on this and thats the only thing stopping someone from raising a PR?

Add Comment

[Justin Holguin (JIRA)]

Justin Holguin commented on PDB-1013

Re: (puppetdb) Can't change 'subname' setting in database.ini

Kenneth Barber yes, I'd greatly prefer your 'make puppetdb module support talking to postgresql via SSL' idea. As long as it requires a change to the module code (which I'm pretty sure it does), then it might as well be a more comprehensive "use_ssl_transport" option or something, which would indeed minimize the docs work.

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013

Module: postgresql ssl communication should be easily done in the module Change By: Kenneth Barber Summary: (puppetdb) Can't change 'subname' setting Module: postgresql ssl communication should be easily done  in  database.ini  the module

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013 Module: postgresql ssl communication should be easily done in the module Change By: Kenneth Barber

Currently the module doesn't support switching on PostgreSQL SSL support for users who want it. This ticket tracks the changes required to make that easy for users and so the $subname can be set automatically as a part of this also. Original report:  I've managed to use SSL for communication between PuppetDB and Postgres, but I can't effectively manage the 'subname' setting in database.ini while using the puppetlabs-puppetdb module. What ends up happening is that I'll change the subname with my own ini_setting resource, but then the puppetdb module will change it back on the next puppet run. Then my resource changes it again, and so it goes back and forth. I looked through the module source and it doesn't look like there's any configuration option I can set to prevent this behavior. Here's a [gist of my example site.pp|https://gist.github.com/holguinj/8f151921b42b5a2b6321]. Edit: here's another [gist|https://gist.github.com/holguinj/5022d89360230166fc83] with all the steps I took to get to where I am.

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013

Module: postgresql ssl communication should be easily done in the module

Change By: Kenneth Barber Issue Type: Bug New Feature

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013 Module: postgresql ssl communication should be easily done in the module

Change By: Kenneth Barber Story Points: 13

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber commented on PDB-1013

Re: Module: postgresql ssl communication should be easily done in the module Justin Holguin done. Thanks.

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013

Module: postgresql client certificate ssl communication should be easily done in the module Change By: Kenneth Barber Summary: Module: postgresql  client certificate  ssl communication should be easily done in the module

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013 Module: postgresql client certificate ssl communication should be easily done in the module Change By: Kenneth Barber

Currently the module doesn't support switching on PostgreSQL SSL support for users who want it. This ticket tracks the changes required to make that easy for users and so the $subname can be set automatically as a part of this also.

Right now, we have an option for 'database_ssl' and 'read_database_ssl' but these only turn on `ssl=true` in the JDBC subname in the database.ini today.

Original report:  I've managed to use SSL for communication between PuppetDB and Postgres, but I can't effectively manage the 'subname' setting in database.ini while using the puppetlabs-puppetdb module. What ends up happening is that I'll change the subname with my own ini_setting resource, but then the puppetdb module will change it back on the next puppet run. Then my resource changes it again, and so it goes back and forth. I looked through the module source and it doesn't look like there's any configuration option I can set to prevent this behavior. Here's a [gist of my example site.pp|https://gist.github.com/holguinj/8f151921b42b5a2b6321]. Edit: here's another [gist|https://gist.github.com/holguinj/5022d89360230166fc83] with all the steps I took to get to where I am.

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013

Module: postgresql client certificate & general libpq ssl communication should be easily done in the module Change By: Kenneth Barber Summary: Module: postgresql client certificate  & general libpq  ssl communication should be easily done in the module

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013 Module: postgresql client certificate & general libpq ssl communication should be easily done in the module Change By: Kenneth Barber

Currently the module doesn't support switching on PostgreSQL SSL support  using the more configurable libpq factory  for users who want it. This ticket tracks the changes required to make that easy for users and so the $subname can be set automatically as a part of this also. Right now, we have an option for 'database_ssl' and 'read_database_ssl'  in the `puppetdb::server::database_ini` class  but these only turn on `ssl=true` in the JDBC subname in the database.ini today : https://github . com/puppetlabs/puppetlabs-puppetdb/blob/master/manifests/server/database_ini.pp What we want to see is the usage of the libpq parameters as defined in our documentation, for example see the sample line here: https://docs.puppetlabs.com/puppetdb/2.2/postgres_ssl.html#using-your-own-self-signed-ca So that there are options to allow: * sslmode (verify-ca & verify-full) * sslrootcert (a path to the root CA file) * sslcert (path to the client cert file) * sslkey (path to the client key) * sslpassword (password the the client key) To be set with parameters via the module. This could be a top level set of parameters, or perhaps a hash that is passed into a single parameter `database_ssl_options`, and broken into arguments? Source code is the best authoritive 'documentation' for libpq today, it can be found here. If you pick through it, you can see the parameters I've provided in the list above: https://github.com/pgjdbc/pgjdbc/blob/master/org/postgresql/ssl/jdbc4/LibPQFactory.java Combined with this we could also allow proxied configuration via the puppetdb class `puppetdb::database::postgresql` for configuration of SSL for the PostgreSQL module: https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/manifests/database/postgresql.pp. That would allow full end-to-end configuration for SSL potentially, via the `puppetdb` class, whereby the necessary postgresql and puppetdb configuration could be provided up front and configured at once. The design issue I haven't worked out is whether to require client certs up front, or somehow use Puppet's if they are around, but I'm presuming for an MVP we could just ask users to provide these initially. I'm uncertain what work might be required on the postgresql module to enable these capabilities end-to-end, but most (if not all) configuration should already be exposed, so the changes hopefully should be small (and if done generically would help that module anyway).

Original report:  I've managed to use SSL for communication between PuppetDB and Postgres, but I can't effectively manage the 'subname' setting in database.ini while using the puppetlabs-puppetdb module. What ends up happening is that I'll change the subname with my own ini_setting resource, but then the puppetdb module will change it back on the next puppet run. Then my resource changes it again, and so it goes back and forth. I looked through the module source and it doesn't look like there's any configuration option I can set to prevent this behavior. Here's a [gist of my example site.pp|https://gist.github.com/holguinj/8f151921b42b5a2b6321]. Edit: here's another [gist|https://gist.github.com/holguinj/5022d89360230166fc83] with all the steps I took to get to where I am.

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013 Module: postgresql client certificate & general libpq ssl communication should be easily done in the module Change By: Kenneth Barber

Currently the module doesn't support switching on PostgreSQL SSL support using the more configurable libpq factory for users who want it. This ticket tracks the changes required to make that easy for users and so the $subname can be set automatically as a part of this also. Right now, we have an option for  '  {{ database_ssl ' }}  and  '  {{ read_database_ssl ' }}  in the  `  {{ puppetdb::server::database_ini ` }}  class but these only turn on  `  {{ ssl=true ` }}  in the JDBC subname in the database.ini today: https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/manifests/server/database_ini.pp

While this is interesting, it implies the CA for postgresql is the same as Puppet. It also doesn't allow for client based certificate authentication. This is because the factory that is used is basic. What we want to see is the usage of the  libpq  LibPQ  parameters as defined in our documentation, for example see the sample line here: https://docs.puppetlabs.com/puppetdb/2.2/postgres_ssl.html#using-your-own-self-signed-ca So Full documentation for using a new factory is here: http://jdbc.postgresql.org/documentation/93/ssl-factory.html which is very sparse, but the idea would be to use the libpq jdbc factory baked into the JDBC driver now in the later 9.x versions. The options  that  there  are  options  available  to  allow  libpq are as follows :

* sslmode (verify-ca & verify-full) * sslrootcert (a path to the root CA file) * sslcert (path to the client cert file) * sslkey (path to the client key) * sslpassword (password the the client key)

To Our desire would  be  set with  to have the {{subname}} string be auto-created from these  parameters  via  in  the module. This could be a top level set of parameters, or perhaps a hash that is passed into a single parameter  `  {{ database_ssl_options ` }} , and broken into arguments?  Not sure :-). Source code is the best  authoritive  authoritative  'documentation' for  libpq  libpqfactor for jdbc  today , it can be found here . If you pick through it, you can see the parameters I've provided in the list above: https://github.com/pgjdbc/pgjdbc/blob/master/org/postgresql/ssl/jdbc4/LibPQFactory.java

Combined with this we could also allow proxied configuration via the puppetdb class

`  {{ puppetdb::database::postgresql ` }}  for configuration of SSL for the PostgreSQL module: https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/manifests/database/postgresql.pp. That would allow full end-to-end configuration for SSL potentially, via the  `  {{ puppetdb ` }}  class, whereby the necessary postgresql and puppetdb configuration could be provided up front and configured at once.

The design issue I haven't worked out is whether to require client certs up front, or somehow use Puppet's if they are around, but I'm presuming for an MVP we could just ask users to provide these initially. I'm uncertain what work might be required on the postgresql module to enable these capabilities end-to-end, but most (if not all) configuration should already be exposed, so the changes hopefully should be small (and if done generically would help that module anyway). Original report:  I've managed to use SSL for communication between PuppetDB and Postgres, but I can't effectively manage the 'subname' setting in database.ini while using the puppetlabs-puppetdb module. What ends up happening is that I'll change the subname with my own ini_setting resource, but then the puppetdb module will change it back on the next puppet run. Then my resource changes it again, and so it goes back and forth. I looked through the module source and it doesn't look like there's any configuration option I can set to prevent this behavior. Here's a [gist of my example site.pp|https://gist.github.com/holguinj/8f151921b42b5a2b6321]. Edit: here's another [gist|https://gist.github.com/holguinj/5022d89360230166fc83] with all the steps I took to get to where I am.

Add Comment

[Kenneth Barber (JIRA)]

Kenneth Barber updated an issue

PuppetDB / PDB-1013 Module: postgresql client certificate & general libpq ssl communication should be easily done in the module Change By: Kenneth Barber

Currently the module doesn't support switching on PostgreSQL SSL support using the more configurable libpq factory for users who want it. This ticket tracks the changes required to make that easy for users and so the $subname can be set automatically as a part of this also. Right now, we have an option for {{database_ssl}} and {{read_database_ssl}} in the {{puppetdb::server::database_ini}} class but these only turn on {{ssl=true}} in the JDBC subname in the database.ini today: https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/manifests/server/database_ini.pp While this is interesting, it implies the CA for postgresql is the same as Puppet. It also doesn't allow for client based certificate authentication. This is because the factory that is used is basic.

What we want to see is the usage of the LibPQ parameters as defined in our documentation, for example see the sample line here: https://docs.puppetlabs.com/puppetdb/2.2/postgres_ssl.html#using-your-own-self-signed-ca

Full documentation for using a new factory is here: http://jdbc.postgresql.org/documentation/93/ssl-factory.html which is very sparse, but the idea would be to use the libpq jdbc factory baked into the JDBC driver now in the later 9.x versions.

The options that are available to libpq are as follows:

* sslmode (verify-ca & verify-full) * sslrootcert (a path to the root CA file) * sslcert (path to the client cert file) * sslkey (path to the client key) * sslpassword (password the the client key)

Our desire would be to have the {{subname}} string be auto-created from these parameters in the module , so you ended up with a configuration line like this: {code} subname = //<HOST>:<PORT>/<DATABASE>?ssl=true&sslfactory=org . postgresql.ssl.jdbc4.LibPQFactory&sslmode=verify-full&sslrootcert=/etc/puppetdb/ssl/ca.pem {code}  This could be a top level set of parameters  for the class , or perhaps a hash that is passed into a single parameter {{database_ssl_options}}, and broken into arguments? Not sure :-).  Then we would simply proxy these configuration parameters via {{puppetdb::server}} for completeness. Source code is the best authoritative 'documentation' for libpqfactor for jdbc today. If you pick through it, you can see the parameters I've provided in the list above: https://github.com/pgjdbc/pgjdbc/blob/master/org/postgresql/ssl/jdbc4/LibPQFactory.java

Combined with this we could also allow proxied configuration via the puppetdb class {{puppetdb::database::postgresql}} for configuration of SSL for the PostgreSQL module: https://github.com/puppetlabs/puppetlabs-puppetdb/blob/master/manifests/database/postgresql.pp. That would allow full end-to-end configuration for SSL potentially, via the {{puppetdb}} class, whereby the necessary postgresql and puppetdb configuration could be provided up front and configured at once. The design issue I haven't worked out is whether to require client certs up front, or somehow use Puppet's if they are around, but I'm presuming for an MVP we could just ask users to provide these initially. I'm uncertain what work might be required on the postgresql module to enable these capabilities end-to-end, but most (if not all) configuration should already be exposed, so the changes hopefully should be small (and if done generically would help that module anyway). Original report:  I've managed to use SSL for communication between PuppetDB and Postgres, but I can't effectively manage the 'subname' setting in database.ini while using the puppetlabs-puppetdb module. What ends up happening is that I'll change the subname with my own ini_setting resource, but then the puppetdb module will change it back on the next puppet run. Then my resource changes it again, and so it goes back and forth. I looked through the module source and it doesn't look like there's any configuration option I can set to prevent this behavior. Here's a [gist of my example site.pp|https://gist.github.com/holguinj/8f151921b42b5a2b6321]. Edit: here's another [gist|https://gist.github.com/holguinj/5022d89360230166fc83] with all the steps I took to get to where I am.

Add Comment

[Claudia Petty (Jira)]

Claudia Petty updated an issue

PuppetDB / PDB-1013 Module: postgresql client certificate & general libpq ssl communication should be easily done in the module

Change By: Claudia Petty Labels: module new-feature puppetdb

Add Comment

This message was sent by Atlassian Jira (v8.20.21#820021-sha1:38274c8)