Authentication store private key secure?

[Frank Sagurna]

Hi there,
i am relative new to ansible and for now use it only for some simple tasks (apt updates, create users, change some config files).
I want to try AWX, but i am a little bit confused about the concept how to run it securely. I searched a lot, but did not find an answer to following.

Now i am storing my password protected private key on my desktop, connect to the ansible machine via ssh and run my playbooks on the ansible host.
Authentication on the remote hosts is then done by ssh agent key forwarding with the same user.

If i understand correctly, for AWX i would have to store my private key on the ansible/AWX host. I am feeling a little bit uncomfortable to store my private key in a second location. Or did i miss something?
Is this correct, that i have to store my private key on the ansible/AWX host, and is this secure?

Thx in advance
Frank

[Ben Spencer]

Frank

I'm fairly new to AWX & Ansible as well. With SSH keys, the source of the connection needs to have access to the private key. This can be via an SSH agent (and there are some things like agent forwarding) or it can be that the source directly has the private key. Something with agents are probably possible with AWX but this is not straight forward and complicated do the workers being in a docker container (the workers need the private key). 

So the option is to store the private key and key passphrase within AWX which AWX then manages and makes available to the worker nodes. I do not know how AWX stores the data on the backend or what encryption/protections are put around the passphrase to keep it from prying eyes. Storing the private key and passphrase together in the same location is not the safest idea and I suspect they ultimately are stored in the same database table (not that even a separate table would matter) but hopefully there is some sort of encryption around the passphrase.

With that being said, the key & passphrase storage is not much different than it is outside of AWX. Good key management processes still apply which can/should include some or all of: 

• private keys should have passphrases
• passphrases and keys should regularly be rotated
• Ideally the passphrase for the key would be stored in a secondary location (AWX doesn't seem to support vaults for the passphrase) 
  
• Keys should not be used for multiple purposes (harder to rotate keys; less visibility into the source/actual key used in logs when connection fingerprints are logged)

I'd also say that storing private keys on the desktop that is used for general purpose items (surfing the web; MS office) is not a secure location itself.

While I am also interested in hearing more about the storage/protections of the keys/passphrases in AWX, that is only part of the security puzzle.

[Ryan Petrello]

Hey There!

AWX does indeed store SSH private keys and passphrases in a PostgreSQL database table, and it encrypts the values using Fernet symmetric encryption.  The key used to encrypt these values is composed of the Django `SECRET_KEY` value for your installation (https://docs.djangoproject.com/en/2.1/ref/settings/#secret-key) and the primary key of the credential being encrypted.  The code that implements this in awx is open source and implemented here:

https://github.com/ansible/awx/blob/devel/awx/main/utils/encryption.py

For the reasons Ben mentioned, it's important to practice good key management processes - we recommend that users generate a new private key specific to their awx installation and not use existing/shared keys.  We also highly recommend the use of a passphrase.

When a playbook runs in AWX that requires the use of an SSH key and/or passphrase, AWX fetches the encrypted key value from the database, decrypts it using the `SECRET_KEY`, and writes it to a named pipe (FIFO) so the decrypted value doesn't get written to the disk.  From here, ssh-agent reads the key from the named pipe, and an ansible-playbook process is forked.  When AWX runs ansible-playbook, it does so via a pseudoterminal using the Python library "pexpect".  In this way, if we observe an interactive prompt for an SSH key (e.g., "Enter passphrase for ..."), we take the decrypted passphrase value and write it to stdin.