Vault: PostgreSQL dynamic credentials not working

2,775 views
Skip to first unread message

Shashi Gokhale

unread,
May 30, 2019, 2:18:36 AM5/30/19
to Vault

Hello All,


I want to use Vault to generate dynamic credentials for my PostgreSQL database. I did the steps given here: https://www.vaultproject.io/docs/secrets/databases/postgresql.html. Platform versions given below:

Execution platform: CentOS Linux 7.3. Version: Vault v1.1.2 ('0082501623c0b704b87b1fbc84c2d725994bac54'). PostgreSQL version: 10.6 hosted in AWS RDS.

  1. 1. Started Vault server in dev mode with command: "vault server -dev".

  2. 2. Validated that I am able to connect to the PostgreSQL database using psql with command: "psql -h my-rds-end-point.rds.amazonaws.com -p 5432 -d alerts -U master"

  3. 3. Enabled the database secrets engine with command: "vault secrets enable database".

  4. 4. Created database connection config with command: 


  5. A. With username, password in connection_url AND separately.

# vault write database/config/my-postgresql-database \
>     plugin_name=postgresql-database-plugin \
>     allowed_roles="my-role" \
>     connection_url="postgresql://master:pg_master_password @ my-rds-endpoint.rds.amazonaws.com:5432/alerts" \
>     username="master" \
>     password="pg_master_password"
WARNING! The following warnings were returned from Vault:

  * Password found in connection_url, use a templated url to enable root
rotation and prevent read access to password information.

B. With username, password only in connection_url, NOT separately.

# vault write database/config/my-postgresql-database \
>     plugin_name=postgresql-database-plugin \
>     allowed_roles="my-role" \
>     connection_url="postgresql://master:pg_master_password @ my-rds-end-point.rds.amazonaws.com:5432/alerts"
WARNING! The following warnings were returned from Vault:

  * Password found in connection_url, use a templated url to enable root rotation and prevent read access to password information.

C. With username, password removed from connection_url but given separately.

# vault write database/config/my-postgresql-database \
>     plugin_name=postgresql-database-plugin \
>     allowed_roles="my-role" \
> connection_url="postgresql://my-rds-end-point.rds.amazonaws.com:5432/alerts" \
>     username="master" \
>     password="pg_master_password"
Error writing data to database/config/my-postgresql-database: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/database/config/my-postgresql-database
Code: 400. Errors:

* error creating database object: error verifying connection: pq: password authentication failed for user "root"
  1. 5. Created role with command:
vault write database/roles/my-role \
    db_name=my-postgresql-database \
    creation_statements="CREATE ROLE \"vault_role\" WITH LOGIN PASSWORD 'vault_role_password' VALID UNTIL '2019-06-30'; \
        GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"vault_role\";" \
    default_ttl="1h" \
    max_ttl="24h"
  1. 6. Queried credentials with command:
# vault read database/creds/my-role
Key                Value
---                -----
lease_id           database/creds/my-role/WNdcEQ0YYZODGWzYxikRNztl
lease_duration     1h
lease_renewable    true
password           A1a-T19Eh8eKKGOZCLQt
username           v-root-my-role-qWqOv4j34Sa3rQ3g35nJ-1559024979


Observations:

1) When I tried to connect to the database with command below, it failed. I though Vault created a user named 'v-root-my-role-qWqOv4j34Sa3rQ3g35nJ-1559024979'. However when I did "\du" when login as master (the super-user) of the PostgreSQL, I do not find the user in list of users.

# psql -h my-rds-end-point.rds.amazonaws.com -p 5432 -d alerts -U v-root-my-role-qWqOv4j34Sa3rQ3g35nJ-1559024979
Password for user v-root-my-role-qWqOv4j34Sa3rQ3g35nJ-1559024979:
psql: FATAL:  password authentication failed for user "v-root-my-role-qWqOv4j34Sa3rQ3g35nJ-1559024979"
FATAL:  password authentication failed for user "v-root-my-role-qWqOv4j34Sa3rQ3g35nJ-1559024979"

Questions: What am I missing here? How to get a set of credential (username/password) that work? Does Vault actually create a user/role that I can see using "\du" psql command?

2) When I tried to fetch another set of credentials with command below, it failed.

# vault read database/creds/my-role
Error reading database/creds/my-role: Error making API request.

URL: GET http://127.0.0.1:8200/v1/database/creds/my-role
Code: 500. Errors:

* 1 error occurred:
        * pq: role "vault_role" already exists

Questions: What am I missing here? How to fetch multiple credentials?

3) Step4A, 4B is throwing a Warning, while 4C leads to error. What is the correct way to create database connection config mentioned in Step 4 above?


I searched the documentation, help, internet, forums etc. but could not figure out a way to make this use case work.


Can someone please guide, provide hint on how I could configure Vault to be able to generate dynamic credentials to connect to PostgreSQL as a particular role?


Kind regards, Shashi

Paul Knox

unread,
May 30, 2019, 4:55:30 AM5/30/19
to Vault
Hi Shashi,

I'm new to Vault myself and working on this at the minute. Have you tried to put the username/password as placeholders in the request body
e.g.

  "plugin_name": "postgresql-database-plugin", 
  "allowed_roles": "my-role",
  "connection_url": "{{username}}:{{password}}@my-rds-endpoint.rds.amazonaws.com:5432/alerts", 
  "username": "dbusername", 
  "password": "dbpassword"
}

I've not been able to try this yet myself so I'm interested in how you get on.

cheers
Paul

Shashibhushan Gokhale

unread,
May 31, 2019, 7:46:12 AM5/31/19
to vault...@googlegroups.com
Hi Paul,

Thanks for your response. I tried the step you gave. No warning, error message was shown. So, I think you have answered the Observation#3 correctly. 

However, the generated credentials are not useful to login to PostgreSQL. It is failing as before, and as mentioned in Observation #1 in my original post. Observations/Problem #2 also remain.

Please advise if you get this to work.

Kind regards,
Shashi

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to a topic in the Google Groups "Vault" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vault-tool/SaEIFGoWmHg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vault-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/438b7343-594b-41b8-a99c-e50674f1400c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Becca Petrin

unread,
May 31, 2019, 10:08:44 AM5/31/19
to Vault
Hi Shashi,

I'm wondering what state your postgres configuration is currently in. What do you get right now when you do the following commands in order? 
  • vault read database/config/my-postgresql-database
  • vault read database/roles/my-role
  • vault read database/creds/my-role
Thanks!
Becca
To unsubscribe from this group and all its topics, send an email to vault...@googlegroups.com.

Shashi Gokhale

unread,
Jun 3, 2019, 3:22:48 AM6/3/19
to Vault
Thanks for the response Becca. The output of your commands is given below:

My vault server is running in Dev mode i.e. started using 'vault server -dev'.

'''
# vault read database/config/my-postgresql-database
Key                                   Value
---                                   -----
allowed_roles                         [my-role]
connection_details                    map[connection_url:postgresql://{{username}}:{{password}}@my-rds-endpoint.us-east-1.rds.amazonaws.com:5432/alerts username:root]
plugin_name                           postgresql-database-plugin
root_credentials_rotate_statements    []

# vault read database/roles/my-role
Key                      Value
---                      -----
creation_statements      [CREATE ROLE "vault_role" WITH LOGIN PASSWORD 'vault_role_password' VALID UNTIL '2019-06-30';         GRANT SELECT ON ALL TABLES IN SCHEMA public TO "vault_role";]
db_name                  my-postgresql-database
default_ttl              1h
max_ttl                  24h
renew_statements         []
revocation_statements    []
rollback_statements      []
# vault read database/creds/my-role
Error reading database/creds/my-role: Error making API request.

Code: 500. Errors:

* 1 error occurred:
        * pq: role "vault_role" already exists


#
'''

I can get new set of credentials using "vault read database/creds/my-role" after I drop the current role using "drop role vault_role". However, login with the new set of credentials always fails with error of this type: "psql: FATAL:  password authentication failed for user "v-root-my-role-k7aGWc4PAThGOd6Z44q2-1559398107"".

I have followed the steps given at: https://www.vaultproject.io/docs/secrets/databases/postgresql.html, however can't seem to get this to work. 

Are there any steps missing in the documentation? Is this feature not supported when server is started in Development mode? Any clues to make this work are welcome!

Kind regards,
Shashi
Message has been deleted

Becca Petrin

unread,
Jun 4, 2019, 5:32:42 PM6/4/19
to Vault
Hi Shashi,

Rather than hard-coding the username and expiration and such on the role, try leaving the {{example}} fields with curly braces around them intact as they're shown here: https://www.vaultproject.io/docs/secrets/databases/postgresql.html. They're actually really important to have in the statement because otherwise the code can't replace them with the values it needs to generate a user successfully. That should solve it for you!

-Becca
Reply all
Reply to author
Forward
0 new messages