Jira (PDB-4934) PuppetDB does not support username@hostname auth for Azure PostgreSQL

6 visualizzazioni
Passa al primo messaggio da leggere

Kevin Reeuwijk (Jira)

da leggere,
19 ott 2020, 13:11:0319/10/20
a puppe...@googlegroups.com
Kevin Reeuwijk created an issue
 
PuppetDB / Bug PDB-4934
PuppetDB does not support username@hostname auth for Azure PostgreSQL
Issue Type: Bug Bug
Affects Versions: PDB 6.11.3
Assignee: Unassigned
Components: PuppetDB
Created: 2020/10/19 10:10 AM
Priority: Normal Normal
Reporter: Kevin Reeuwijk

As a customer, I want to be able to use Azure PostgreSQL as my external PostgreSQL database for Puppet Enterprise.
 
When attempting to use Azure PostgreSQL as an external database for PuppetDB (PE 2019.8.1), I encountered the problem that Azure requires the username for the Postgres connection to be in the username@hostname form, due to the way they publish access to PostgreSQL (as described here). I can manually modify database.ini to set the username to that format, but then you’ll see this in the logs:

clojure.lang.ExceptionInfo: Connected to database as "pe-puppetdb-migrator", not migrator "pe-puppetdb-migrator@pdb01"

It seems we have the same limitations as Chef has (see linked issue).
 
This requirement from Azure stems from their architecture:

Azure Database for PostgreSQL has a gateway in front of the actual database servers that forwards connections from username@hostname to hostname as username.
This means that once the connection is established, you will actually be connected as username, not username@hostname, and any database queries involving users should just use username (e.g. granting permissions).

 
Some issues I’ve encountered while trying to get this to work: * The docs don’t tell you to also create a pe-puppetdb-migrator user

  • You can’t specify username@hostname for the xxx_regular_db_user and xxx_migration_db_user settings in pe.conf, the @hostname part gets cutoff during installation.
  • I can manually re-add the @hostname back to the username in database.ini but then the queries also expect this for the connection, which they should not. And I can probably assume that another puppet run would overwrite the settings in database.ini again.
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935)
Atlassian logo

Kevin Reeuwijk (Jira)

da leggere,
19 ott 2020, 13:26:0319/10/20
a puppe...@googlegroups.com
Kevin Reeuwijk updated an issue
Change By: Kevin Reeuwijk
As a customer, I want to be able to use Azure PostgreSQL as my external PostgreSQL database for Puppet Enterprise.
 
When attempting to use Azure PostgreSQL as an external database for PuppetDB (PE 2019.8.1), I encountered the problem that Azure requires the username for the Postgres connection to be in the {{username@hostname}} form, due to the way they publish access to PostgreSQL (as described [here|https://github.com/chef/chef-server/issues/1559]). I can manually modify {{database.ini}} to set the username to that format, but then you’ll see this in the logs:
{noformat}clojure.lang.ExceptionInfo: Connected to database as "pe-puppetdb-migrator", not migrator "pe-puppetdb-migrator@pdb01"
{noformat}

It seems we have the same limitations as Chef has (see linked issue).
 
This requirement from Azure stems from their architecture:
{noformat}Azure Database for PostgreSQL has a gateway in front of the actual database servers that forwards connections from username@hostname to hostname as username.

This means that once the connection is established, you will actually be connected as username, not username@hostname, and any database queries involving users should just use username (e.g. granting permissions).
{noformat}

 
Some issues I’ve encountered while trying to get this to work:

* The [docs|https://puppet.com/docs/pe/2019.8/installing_postgresql.html#create_pe_databases_on_the_postgresql_instance] don’t tell you to also create a {{pe-puppetdb-migrator}} user
*
The [docs|https://puppet.com/docs/pe/2019.8/installing_postgresql.html#create_pe_databases_on_the_postgresql_instance] assume a Linux OS for the {{psql}} commands to create the users & databases. However, Azure PostgreSQL runs on Windows, which causes the locales to have different names. For Azure PostgreSQL, the {{ENCODING}} line needs to be changed to: {{ENCODING 'utf8' LC_CTYPE 'English_United States.1252' LC_COLLATE 'English_United States.1252' template template0;}}
*
You can’t specify {{username@hostname}} for the {{xxx_regular_db_user}} and {{xxx_migration_db_user}} settings in {{pe.conf}}, the {{@hostname}} part gets cutoff during installation.
* I can manually re-add the {{@hostname}} back to the username in {{database.ini}} but then the queries also expect this for the connection, which they should not. And I can probably assume that another puppet run would overwrite the settings in {{database.ini}} again.

Zachary Kent (Jira)

da leggere,
11 gen 2021, 20:21:0311/01/21
a puppe...@googlegroups.com

Zachary Kent (Jira)

da leggere,
12 gen 2021, 14:14:0412/01/21
a puppe...@googlegroups.com

Bogdan Irimie (Jira)

da leggere,
13 gen 2021, 09:56:0413/01/21
a puppe...@googlegroups.com

Bogdan Irimie (Jira)

da leggere,
13 gen 2021, 09:57:0413/01/21
a puppe...@googlegroups.com

Rob Browning (Jira)

da leggere,
20 gen 2021, 11:31:0220/01/21
a puppe...@googlegroups.com
Rob Browning commented on Bug PDB-4934
 
Re: PuppetDB does not support username@hostname auth for Azure PostgreSQL

After a bit of investigation (and reading some of the chef-server discussion) , I think I might lean toward just adding a new database config option like connection-username that when set, overrides the username that's specified when connecting (i.e. for the hikaricp connection pool), but doesn't affect other uses of the username (i.e. when we're issuing various sql commands that include a username).

While I haven't thought it through yet, I suppose that might mean we'd actually need two options e.g. connection-username and connection-migrator-username.

Bogdan Irimie (Jira)

da leggere,
21 gen 2021, 11:27:0321/01/21
a puppe...@googlegroups.com

Bogdan Irimie (Jira)

da leggere,
27 gen 2021, 09:03:0427/01/21
a puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Sprint: ghost-27.01.2021 , ready for triage 3

Bogdan Irimie (Jira)

da leggere,
3 feb 2021, 10:01:0603/02/21
a puppe...@googlegroups.com
Bogdan Irimie commented on Bug PDB-4934
 
Re: PuppetDB does not support username@hostname auth for Azure PostgreSQL

Rob BrowningKevin Reeuwijk

I managed to reproduce the exception using only PuppetDB and an Azure PostgreSQL

Connected to database as "puppetdb", not migrator "puppetdb@ghost-db"

The exception is thrown by https://github.com/puppetlabs/puppetdb/blob/2cffbb9d898950c2883067cbbf7b7c5d3ba78064/src/puppetlabs/puppetdb/cli/services.clj#L593
because the current user is read from the database and does not contain `@domain`

In PDB-4934 I have trimmed `@domain` when doing the user name check and managed to write and read from Azure PostgreSQL.

Bogdan Irimie (Jira)

da leggere,
4 feb 2021, 06:17:0304/02/21
a puppe...@googlegroups.com
Bogdan Irimie commented on Bug PDB-4934

Rob Browning and Zachary Kent suggested adding two new users: `connection-username ` and `connection-migrator-username`. The users will be used to establish new connections to the database (in the case of Azure, the pattern is `user@domain`). By implementing this we will make a clear distinction between the user that is used to connect to the database (in our case `connection-username` and `connection-migrator-username`) and the user that is used in queries (in our case `username` and `migrator-username`).

 

If the `connection-migrator-username` is not specified in the database config, it will default to `migrator-username`, which in turn default to `username`.

https://github.com/puppetlabs/puppetdb/blob/6.x/src/puppetlabs/puppetdb/config.clj#L393-L399

 

If the `connection-username` is not specified in the database config, it will default to `username`.

 

Example of test: https://github.com/puppetlabs/puppetdb/commit/b489b38ff1484774ea08185db084f9670ee9fac4

Bogdan Irimie (Jira)

da leggere,
10 feb 2021, 09:07:0510/02/21
a puppe...@googlegroups.com
Bogdan Irimie updated an issue
Change By: Bogdan Irimie
Sprint: ghost-27.01.2021, ghost-10.02.2021 , ready for triage 3

Austin Blatt (Jira)

da leggere,
22 feb 2021, 12:15:0122/02/21
a puppe...@googlegroups.com
Austin Blatt updated an issue
Change By: Austin Blatt
Fix Version/s: PDB 6.15.0
Fix Version/s: PDB 7.2.0
Rispondi a tutti
Rispondi all'autore
Inoltra
0 nuovi messaggi