edit/create encrypted vars in python script (vault encrypt_string)

[Jonathan Renon]

Hello, 
I was wondering if there was any method exposed to create/modify `encrypted string` using ansible python module in a python script.

I was able to read and display `encrypted vars` value using ansible.parsing.dataloade, but I can't figure a way to create a new vault encrypted var.

TL;DR: 

looking for method to do `ansible-vault encrypt_string ...` inside a python script.

Thank you for your help.

[Brian Coca]

Modules should not have direct access to vault secrets, but you can
provide them to module options via the unvault/vault filters: (last
ones in section)

https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#hashing-and-encrypting-strings-and-passwords
--
----------
Brian Coca

[Todd Lewis]

It isn't clear whether you're trying to do this inside or outside of Ansible. If outside, you can have your python script call ansible-vault and pass the data you want to encrypt or decrypt via stdin/stdout. I got this working in

https://gist.github.com/utoddl/66baa4154618ba1fc8ec8127483e7e89

which is a filter I can pass a subset of yaml data through from $EDITOR, and it will decrypt encrypted values, or decrypt encrypted values. It preserves the original yaml's indentation, so greatly simplifies maintenance of secrets in otherwise readable vars files. (ToDo: It doesn't preserve comments.) You should be able to adapt it for your purposes. Here's the blurb from the top of that gist:

"""Filter for "ansible-vault"ing YAML Data Values

This script allows an ansible user to pass lines of yaml data
through it to get string values vaulted and/or already-vaulted values
unvaulted, thus simplifying the maintenance of vaulted strings in
Ansible vars files and eliminating the need to vault entire files.

It takes a single optional positional argument, which is the
Ansible vault identity to use for encryption. This defaults to the
value of the ANSIBLE_VAULT_IDENTITY environment variable. If any
encryption is performed, either the environment variable or the
positional parameter must be provided. (Values from any ansible.cfg
files are insufficient.) """

[Jonathan Renon]

It's to be used outside of ansible, to help change vaulted vars more easily for users.

I'll look at your snippet and see if I can make it work for me, I was trying to avoid running `ansible-vault` using pure python.

[Todd Lewis]

Sounds like we're both addressing the same issue. I've included below our internal docs for using this filter. It addresses vim, but any editor that can pass a subset of lines through a filter should be able to use it.

It's simpler to write your YAML variable definitions and values as you normally do, then pass the "sensitive" ones through a filter that takes care of the fiddly details. Let's start with the same YAML data as in the example above, and use vim on [redacted] to edit it. Again, you should have your ANSIBLE_VAULT_IDENTITY and ANSIBLE_VAULT_IDENTITY_LIST environment variables set. Here's the procedure:

• • • Load the vars file in vim.
    • Go down to the datum that you want to vault.
    • Activate "visual selection" with "shift-v". Your current line should highlight.
    • If applicable, cursor down until all the lines containing the desired data are highlighted. (Some data span multiple lines!)
    • Hit ":". You should be presented with a prompt like ":'<,'>" which represents the range of selected lines that you want to operate on.
    • Type "!av-filter" and hit enter. This will pipe the selected lines through the "av-filter" command, replacing the original line with the av-filter's output. (Use "!av-filter xxxx" if you don't have your ANSIBLE_VAULT_IDENTITY environment variable set, but you do have an "xxxx" vault identity.)
    • Wait for it; ansible-vault is no speed daemon.

Your selected data should be replaced with the ansible-vault encrypted version of the same string, i.e. the blue bits from the section above. If things messed up and you got error messages, read them, then hit "u" to invoke vim's "undo" command. That should put things as they were, and you can go and think about just what you have done.

The same procedure will unvault the value if it's already vaulted. That "!vault" is a YAML tag that indicates the data type; av-filter always encrypts strings and decrypts "!vault", so you never have to tell it whether to encrypt or decrypt. On the back-end of course it's invoking ansible-vault for you and handling the data transfers in and out.

Also, av-filter is in the /[redacted]/bin  directory. If that's not in your path, you've got more work to do. It's a short python script which you can copy to your workstation if you like. (And you get a free trip to pip hell out of it.)

[Jonathan Renon]

Hello, 
I've found a post explaining a way to encrypt using VaultLib from ansible.parsing.vault, using encrypt method and then recreate the flag format of encrypt_string using pyyaml module.

[Todd Lewis]

That is interesting. It's been several years since I could use a single ANSIBLE_VAULT_PASSWORD_FILE, and that shows in my solution. It may be trivial to adapt that post's technique to use ANSIBLE_VAULT_IDENTITY_LIST and ANSIBLE_VAULT_IDENTITY if they are available and avoid `ansible-vault`'s startup costs. However, as infrequently as I end up using `av-filter`, I'm more comfortable sticking with the admittedly slower but likely stable command line interface than hoping the internal API doesn't change. Having said that, even in my `av-filter` implementation I had to include fall-back versions of the `subprocess.run()` calls because of Python changes. [sigh]

But that's also a nice thing about an Open project like Ansible: there are often multiple ways to address an issue, and you get to pick the trade-offs that seem more important to you at the time.

And since you brought it up (indirectly, via that post), I really should make `av-filter` work with ANSIBLE_VAULT_PASSWORD_FILE if ANSIBLE_VAULT_IDENTITY and/or ANSIBLE_VAULT_IDENTITY_LIST are not available. Arguably it should work with the equivalent values coming from the relevant `ansible.cfg` if the environment variables are missing. (That in itself is another can of worms worthy of its own thread. I'm sure other organizations have come up with rational guidelines for when to use env vars vs. ansible.cfg, as we have, and I expect others have come to conclusions different from ours! Again, multiple right answers.)

Thanks for bringing this up! It's always worth looking at different approaches to similar problems.