Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Secret variables / data: encryption

2,517 views
Skip to first unread message

Jan-Piet Mens

unread,
Jun 29, 2013, 6:18:30 PM6/29/13
to ansible...@googlegroups.com
I believe this topic has been requested before, and we talked about it
at some length this afternoon in Antwerp.

People wanting to store "confidential" variables with their playbooks,
say, in a repo, cannot really do that for fear of exposing the secret
variables.

Would there be some interest in having support in Ansible for encrypted
variables? Before investing a lot of time and then having the idea
dropped, I'd like a bit of feedback on this.

I see two possible ways we could go about implementing this. Take
host_vars/ for example, we could add what I'd like to call encrypted
"pouches" of YAML which would be decrypted by Ansible when it attempts
to access the pouch.

host_vars/www normal YAML variables
host_vars/mail.pouch encrypted YAML / base64-encoded

Alternatively, a method proposed by Toshaan which I also rather like is
mix encrypted variables into a regular vars file:

---
name: John Doe
password.crypt: aGVsbG8gd29ybGQgb2YgYW5zaWJsZQo=

Ansible would then decrypt `password.crypt', returning the clear-text in
a variable called `password'.

As to encryption itself, there are a number of possibilities. I'm
tending towards using symmetric encryption instead of public key encryption
because of the possibility of easily sharing the key. With public-key we
would need to decide what system to use, and this would entail
additional Python module requirements for the Ansible manager. If we can
use pycrypto (which is already a dependency for Paramiko) that would be
a bonus.

Thoughts?

-JP

Michael DeHaan

unread,
Jun 29, 2013, 7:51:34 PM6/29/13
to ansible...@googlegroups.com
This is very interesting... I like the 'pouch' idea, though I am not sure I like the name 'pouch' for it :)

The trick is you also want to decrypt it when editing the file, so it needs something for that too, that spawns $EDITOR around editing it, and Ansible needs to recognize if any of these are used to ask for unlock passwords, which sounds like it *MAY* need to be another flag.

Are people going to want to use different passwords for different files?

The idea of handling it with a lookup plugin sounds a little promising as it's not far off from the way the password plugin looks now for generating new passwords.  (Is everybody aware of that?)

Mostly I'm interested in how the user would interact with editing the file, and in what situations and when they are asked for a password.



        -JP

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.





--
Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

Jan-Piet Mens

unread,
Jun 30, 2013, 3:55:20 AM6/30/13
to ansible...@googlegroups.com
> This is very interesting... I like the 'pouch' idea, though I am not sure I
> like the name 'pouch' for it :)

Call it container or whatever -- I think a name that clearly identifies
it as something special is good. Considering how much you like cows,
maby 'udder' is what we need ... ? ;-)

> Are people going to want to use different passwords for different files?

I'm not thinking passwords (as in "type that in"), but rather key files
(stored in a well-known path or specified as cfg variable/environment
variable).

We need something that can work unattended (think cron) and something
which can be shared accross a team of people working on a distributed
copy of the playbook repo.

> The idea of handling it with a lookup plugin sounds a little promising as
> it's not far off from the way the password plugin looks now for generating
> new passwords. (Is everybody aware of that?)

I wasn't, no.

> Mostly I'm interested in how the user would interact with editing the file,
> and in what situations and when they are asked for a password.

I think a small utility which decrypts with said key file, invokes
$EDITOR, and encrypts would suffice. On the other hand, as alluded to,
people could write their own stuff using `openssl' cli.

-JP

Vincent Van der Kussen

unread,
Jun 30, 2013, 4:06:09 AM6/30/13
to ansible...@googlegroups.com
On Sun, Jun 30, 2013 at 9:55 AM, Jan-Piet Mens <jpm...@gmail.com> wrote:
>> This is very interesting... I like the 'pouch' idea, though I am not sure I
>> like the name 'pouch' for it :)
>
> Call it container or whatever -- I think a name that clearly identifies
> it as something special is good. Considering how much you like cows,
> maby 'udder' is what we need ... ? ;-)

Just a suggestion here : vault

>
>> Are people going to want to use different passwords for different files?
>
> I'm not thinking passwords (as in "type that in"), but rather key files
> (stored in a well-known path or specified as cfg variable/environment
> variable).
>
> We need something that can work unattended (think cron) and something
> which can be shared accross a team of people working on a distributed
> copy of the playbook repo.
>
>> The idea of handling it with a lookup plugin sounds a little promising as
>> it's not far off from the way the password plugin looks now for generating
>> new passwords. (Is everybody aware of that?)
>
> I wasn't, no.
>
>> Mostly I'm interested in how the user would interact with editing the file,
>> and in what situations and when they are asked for a password.
>
> I think a small utility which decrypts with said key file, invokes
> $EDITOR, and encrypts would suffice. On the other hand, as alluded to,
> people could write their own stuff using `openssl' cli.

As discussed yesterday, i think what Chef does comes close to what we
want to achieve

http://docs.opscode.com/essentials_data_bags.html

Vincent

Marco Vito Moscaritolo

unread,
Jun 30, 2013, 6:43:42 AM6/30/13
to ansible...@googlegroups.com
We actually use git+git-crypt to encript / decrypt files (SSL Cert,
GPG Keys, ..) and use hashed variable for users password.

Pros:
- Work from now
- Easy to implement
- Choise encrypt system that we prefer

Cons:
- Only GIT compatible (don't work with SVN/BZR/...)
- File is TOTALLY crypted (no just a single variable)

JM2C
> --
> You received this message because you are subscribed to the Google Groups "Ansible Project" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Ciao
Marco
--
web: http://mavimo.org
msn: mav...@tiscali.it
gtalk: mav...@gmail.com
mob: +39 393 9249923
skype: marco.moscaritolo
twitter: http://twitter.com/mavimo

Dave Cottlehuber

unread,
Jun 30, 2013, 9:29:28 AM6/30/13
to ansible...@googlegroups.com
On 30 June 2013 12:43, Marco Vito Moscaritolo <mav...@gmail.com> wrote:
> We actually use git+git-crypt to encript / decrypt files (SSL Cert,
> GPG Keys, ..) and use hashed variable for users password.
>
> Pros:
> - Work from now
> - Easy to implement
> - Choise encrypt system that we prefer
>
> Cons:
> - Only GIT compatible (don't work with SVN/BZR/...)
> - File is TOTALLY crypted (no just a single variable)
>
> JM2C

Sounds interesting! Can you share how to do this? A blog post would be PURRfect.

A+
Dave

Michael DeHaan

unread,
Jun 30, 2013, 11:00:31 AM6/30/13
to ansible...@googlegroups.com
" knife data bag from file BAG_NAME ITEM_NAME.json"

This is perhaps the worst CLI design I have ever seen :)

Perhaps someone can try to boil it down to something more simpler in how we think this would look at CLI level.


Jan-Piet Mens

unread,
Jun 30, 2013, 1:51:34 PM6/30/13
to ansible...@googlegroups.com
> " knife data bag from file BAG_NAME ITEM_NAME.json"
>
> This is perhaps the worst CLI design I have ever seen :)

It *is* the worst I've ever seen. No worries -- we'll make that much
better (which isn't hard to do)

-JP

GW

unread,
Jun 30, 2013, 4:21:48 PM6/30/13
to ansible...@googlegroups.com

Hi,

Why not use the 'password' lookup plugin (with_password or {{ lookup('password',...) }}) or 'file' lookup plugin? This way all confidential variables can be stored in a directory 'credentials' and either you do not commit them at all (the ones who have it could use it) or use .zip/.rar with password or encrypted loopback image and commit the encrypted archive?

Greetings,
  gw 

Michael DeHaan

unread,
Jun 30, 2013, 4:23:05 PM6/30/13
to ansible...@googlegroups.com
The password lookup plugin generates passwords when they are not already set, it is not a means of keeping confidental data that you can hand enter.

The "vault" idea is a neat idea.


--

Petros Moisiadis

unread,
Jun 30, 2013, 4:38:56 PM6/30/13
to ansible...@googlegroups.com
Another approach would be to have a 'secret_vars' playbook section as a list with the names of variables whose values are expected to be in a format like this:
shared_key_configuration_name!base64.b64encode(encrypted_value)

For example, in playbooks:

secret_vars:

    - mysql_root_password
    - some_api_key

and in the inventory or in variable files :

   mysql_root_password: mykey!cXd5OGduMzQ1MjM0I2VyQCMmXiozNGdRVyM=

   some_api_key: anotherkey!M2N2OTIzNDVqdm5zZGZnOTVmcmU5NTR2ei8jIw==

Then, multiple shared key configurations would normally be placed in the ansible configuration, like this:

[shared_key_mykey]

cipher=AES
keyfile=/path/to/mykeyfile

[shared_key_anotherkey]
cipher=DES
keyfile=/path/to/anotherkeyfile

That way it would also be super easy to hide variables that are in secret_vars list from ansible's logs which is also a very important security enhancement measure.

What do you think?


Michael DeHaan

unread,
Jun 30, 2013, 6:08:24 PM6/30/13
to ansible...@googlegroups.com
I kind this much harder to explain than JP Men's suggestion.

We will be going with that :)


Michael DeHaan

unread,
Jun 30, 2013, 6:08:47 PM6/30/13
to ansible...@googlegroups.com
Ignore my inability to plac'e apostrophe's correctly btw :)


Petros Moisiadis

unread,
Jun 30, 2013, 7:19:23 PM6/30/13
to ansible...@googlegroups.com
On second thought, to make things even simpler, 'secret_vars'  is not even needed.
A 'decrypt'  lookup plugin would be just enough to get the decrypted value of a variable in a format like the one I suggested.
That way, the format can be made even simpler by moving away 'shared_key_configuration_name' from it and giving it instead as a parameter to the 'decrypt' lookup plugin call.

Pretty simple I think, without extra playbook syntax or additional, specially-handled files. And perhaps it has the easiest implementation since lookup logic is already available.

Michael DeHaan

unread,
Jun 30, 2013, 7:28:22 PM6/30/13
to ansible...@googlegroups.com
I generally am not a fan of lookup plugin syntax, especially when it has to return a data structure.  It's fine for some things, but the main intention of lookup plugins were to enable things like "with_items" and then they got used for things like file lookups later, and the syntax is ok... but it's something I find myself using rarely.

I really like the vault idea because it would work transparently with vars_files and other things, and *not* require extra syntax.   

I am tentatively thinking

ansible-vault --unlock <path>
ansible-vault --lock <path>

And then when ansible runs if a vault is used, it would prompt for the unlock password for that vault.

The question is do you need multiple vault passwords at the same time or not.

The other slight thing is you want to make sure none of your files get checked into git while locked down though, which might be a little tricky... I would hate to have to have people to know how to set up git triggers for this, but that might be the only way.


Petros Moisiadis

unread,
Jul 1, 2013, 5:24:21 AM7/1/13
to ansible...@googlegroups.com
On 07/01/2013 02:28 AM, Michael DeHaan wrote:
I generally am not a fan of lookup plugin syntax, especially when it has to return a data structure.  It's fine for some things, but the main intention of lookup plugins were to enable things like "with_items" and then they got used for things like file lookups later, and the syntax is ok... but it's something I find myself using rarely.

I really like the vault idea because it would work transparently with vars_files and other things, and *not* require extra syntax.   

I am tentatively thinking

ansible-vault --unlock <path>
ansible-vault --lock <path>

And then when ansible runs if a vault is used, it would prompt for the unlock password for that vault.

The question is do you need multiple vault passwords at the same time or not.

In my opinion, a requirement to always prompt for a password does not fit with the purpose of automating things.
Instead, there can be multiple vault configuration sections in the ansible configuration and ansible-vault could be called without a path (getting the paths from the configuration). For example:

[vault_main]
path=/path/to/vaults/main

[vault_custom]
path=/path/to/vaults/custom
password_file=/path/to/vault_passwords/custom
cipher=Twofish

If 'password_file' is set, it will be used to read the password for unlocking the vault. Only if it is not set would ansible utilities prompt to enter a password.
Running an ansible binary would involve invoking ansible-vault to unlock the vaults in temporary paths (perhaps somewhere inside /tmp or ~/.ansible) with local user readonly access. The script would read vars from there, flag them internally as 'sensitive' so as not to leak them in logs, and, in the end, it would invoke again ansible-vault to remove these temporary paths.
Password files could be shared among machines/users that run ansible. For even better security, they could be stored encrypted with the ssh public key of the local user and be decrypted before being used. ssh-agent could be used to avoid repeated password input prompts in the same fashion that it can be used to ssh to remote hosts.
The 'cipher' setting could be used to support different encryption algorithms. If missing, a default one would be used.
Editing vaults could be done with an 'ansible-vault --edit {vault_name | vault_path}' command that would unlock to a temporary path, spawn an editor and, on exit, encrypt the temporary path and move it to the permanent (locked) vault path.



The other slight thing is you want to make sure none of your files get checked into git while locked down though, which might be a little tricky... I would hate to have to have people to know how to set up git triggers for this, but that might be the only way.

What do you mean by 'make sure none of your files get checked into git while locked down'? Isn't it the desirable behaviour to always check the files into GIT in a locked state?

Jan-Piet Mens

unread,
Jul 1, 2013, 5:36:19 AM7/1/13
to ansible...@googlegroups.com
Thanks for all your comments and ideas. I have submitted a pull-request
[1] which implements this as a filter. I've tried to make it so that
people can use as many keys as they desire. The page with the PR [1]
contains a blurb about how to get started and how to use it.

A utility called ansible-vault is included.

Regards,

-JP


[1] https://github.com/ansible/ansible/pull/3393

Jan-Piet Mens

unread,
Jul 1, 2013, 5:37:12 AM7/1/13
to ansible...@googlegroups.com
> In my opinion, a requirement to always prompt for a password does not
> fit with the purpose of automating things.

Full ACK

-JP

Jan-Piet Mens

unread,
Jul 1, 2013, 5:39:13 AM7/1/13
to ansible...@googlegroups.com
> If 'password_file' is set, it will be used to read the password for
> unlocking the vault. Only if it is not set would ansible utilities
> prompt to enter a password.

I'm not understanding the whole lock/unlock thing. IMO (and that is how
I've done it) encrypted values are contained in vars files and are
decrypted on use.

-JP

Petros Moisiadis

unread,
Jul 1, 2013, 8:44:41 AM7/1/13
to ansible...@googlegroups.com
Correct. Encrypting / Decrypting are better words than Locking /
Unlocking to describe the usage I suggested previously (decrypting vault
files somewhere temporarily while vars are being used), but I used the
latter coming from a discussion with references to that terminology.
Also, although I suggested using temporary files for the decrypted
vaults earlier, on second thought, I think that they should be better
kept temporarily in memory. That way you can be sure that no decrypted
values are left over on unpredicted conditions (e.g. on crashes).

Petros Moisiadis

unread,
Jul 1, 2013, 9:01:04 AM7/1/13
to ansible...@googlegroups.com
Very nice. Thanks!
Are (or can they be) jinja filters also available inside playbooks?

Jan-Piet Mens

unread,
Jul 1, 2013, 9:10:45 AM7/1/13
to ansible...@googlegroups.com
> Are (or can they be) jinja filters also available inside playbooks?

To a certain extent, I believe. For example, this works:

- action: shell echo '{{ mypasswd | vault }}' > /tmp/00

Regards,

-JP

Michael DeHaan

unread,
Jul 1, 2013, 9:57:48 AM7/1/13
to ansible...@googlegroups.com
"encrypted values are contained in vars files and are
decrypted on use."

Where do they get the inputs to decrypt them from?




--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Michael DeHaan

unread,
Jul 1, 2013, 10:01:00 AM7/1/13
to ansible...@googlegroups.com
First off, thanks for the initiative.

This is too early for a core pull request IMHO as there's still a pretty good discussion on the list on how this might work.

I'm also finding the interface somewhat confusing because it's still using lookup plugins (which I asked that it not), and it requires using the keyczar CLI.

If we can dumb things down, I'd like to have this as a core feature, but right now, there's a lot of reading/understanding required for it's use, and it seems to require a bit too much thinking.

Not trying to be negative here, I just think it needs to be really smooth to fit in with the rest of Ansible.

--Michael


--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Jan-Piet Mens

unread,
Jul 1, 2013, 10:19:45 AM7/1/13
to ansible...@googlegroups.com
> I'm also finding the interface somewhat confusing because it's still using
> lookup plugins (which I asked that it not),

I don't understand. It's a Jinja2 filter. Where's the lookup plugin?
(Maybe I'm confusing the terminology.)

> and it requires using the
> keyczar CLI.

For creating a key (or more than one key) initially, yes. I see
absolutely no point in re-inventing the wheel, particularly as crypto
stuff is concerned. People much more clever than I have done that.

After a key is initially created, that's it; no more keyczar CLI. That's
the reason I submitted the ansible-vault utility. (Of course, I could
add key creation to that, which would avoid using the keyczar CLI
entirely.)

> If we can dumb things down, I'd like to have this as a core feature, but
> right now, there's a lot of reading/understanding required for it's use,
> and it seems to require a bit too much thinking.

Pardon me, but I disagree.

1. Create a key.
2. Use it.

I've been verbose in my doc (on the PR page) so as to be as clear as
possible.

-JP

Petros Moisiadis

unread,
Jul 1, 2013, 10:24:10 AM7/1/13
to ansible...@googlegroups.com
Great.
Now what is needed is a way to make sure that decrypted values are not
being leaked in ansible logs, though I am not sure what is the correct
place to do the checks.
Maybe the easiest thing to do would be to add a global "log" attribute
that could be also overridden by tasks. Then it could be set to "log:
no" so that no logs are produced (quite intrusive), set to "log: debug"
to log everything or set to other levels such as "critical", "warning",
"notice", etc with each module having to decide what to log based on the
active log level.

Michael DeHaan

unread,
Jul 1, 2013, 10:56:25 AM7/1/13
to ansible...@googlegroups.com
I guess I'm saying "too many steps" here.

I just want it to be as smooth as Possible.

I think it could be done so you didn't even need the Jinja2 filter, you could just use all the vars files unencrypted during Ansible playbook runtime, but they would be locked when you try to edit them, and could have a full vars_files or two full of content that you could edit plaintext when needed.

Sort of like how ssh-agent doesn't require re-entering your SSH key password, right?






        -JP

Petros Moisiadis

unread,
Jul 1, 2013, 1:36:02 PM7/1/13
to ansible...@googlegroups.com
On 07/01/13 17:56, Michael DeHaan wrote:
> I guess I'm saying "too many steps" here.
>
> I just want it to be as smooth as Possible.
>
> I think it could be done so you didn't even need the Jinja2 filter,
> you could just use all the vars files unencrypted during Ansible
> playbook runtime, but they would be locked when you try to edit them,
> and could have a full vars_files or two full of content that you could
> edit plaintext when needed.
>
The Jinja2 filter is nice. It allows to store mixed encrypted and
unencrypted values and it can be used both in playbooks and templates.
You don't have to encrypt/lock the whole variable file just because you
have some password vars inside it (the usual case). In addition, it
allows to have different encryption keys for different vars without
having to create separate vault files for each key.

keyczar is a good choice too. It has a nice abstraction over handling
key stores and keys, it is tested, and keyczart's code could be
incorporated in the ansible-vault utility in order to have one utility
for all vault tasks, which could be also taught to edit encrypted values
directly inside variable files (e.g. "ansible-vault --key ckey --set
mysecretvar='plaintext' varsfile").

Jan-Piet Mens

unread,
Jul 1, 2013, 1:46:35 PM7/1/13
to ansible...@googlegroups.com
> I just want it to be as smooth as Possible.

It is. :) [Admittedly, with the exception that ansible-vault utility
needs more love.]

> I think it could be done so you didn't even need the Jinja2 filter, you
> could just use all the vars files unencrypted during Ansible playbook
> runtime, but they would be locked when you try to edit them, and could have
> a full vars_files or two full of content that you could edit plaintext when
> needed.

I'm (very slowly) starting to understand what you want: everything is
encrypted and gets magically decrypted on Ansible run. This is way
overkill, and poses a number of problems respectively has a number of
inconveniences:

1. How to detect whether a particular vars file is encrypted (by file
name extension, by content)?
2. Completely disables things like printing/sending a vars file
3. Makes version control near to meaningless -- having all the
crypto-stuff in the files.
4. Requires some 'magic' (like the SSH agent you mention below) which is
able to feed Ansible with keys. Even if we pass a single key to
Ansible, the next point:
5. Makes using more than a key terribly difficult. We discussed using
some form of PKI in Antwerp, but more or less chucked that due to
'what do we encrypt to?' The current user? The current host? The
target host? How do teams share keys, etc. Most complicated, I think.

The original intention was to encrypt really secret stuff, like
passwords for MySQL (my.cnf) and things like that. Most variables do not
need to be encrypted, and I believe the current PR solves that rather
neatly: encrypt only what absolutely needs to be encrypted.

> Sort of like how ssh-agent doesn't require re-entering your SSH key
> password, right?

I'm hearing you, but I don't think this is the way to go.

-JP

Jan-Piet Mens

unread,
Jul 1, 2013, 1:49:57 PM7/1/13
to ansible...@googlegroups.com
> In addition, it
> allows to have different encryption keys for different vars without
> having to create separate vault files for each key.

Bingo!

> keyczar is a good choice too. It has a nice abstraction over handling
> key stores and keys, it is tested, and keyczart's code could be
> incorporated in the ansible-vault utility in order to have one utility
> for all vault tasks, which could be also taught to edit encrypted values
> directly inside variable files (e.g. "ansible-vault --key ckey --set
> mysecretvar='plaintext' varsfile").

Indeed. I would like to augment ansible-vault like so:

ansible-vault -N -k mynewkey # create new key w/ all requirements
echo secret | ansible-vault -k mynewkey - # encrypt

Hide the keyczar CLI utility and making of all directories, etc. Add a
`-l' option to list keys, maybe `-r' to remove, and that's about it.

I like your idea of an --edit function. #feature

-JP

GW

unread,
Jul 1, 2013, 2:59:16 PM7/1/13
to ansible...@googlegroups.com

On Mon, 1 Jul 2013 16:19:45 +0200, Jan-Piet Mens wrote:

I'm also finding the interface somewhat confusing because it's still using lookup plugins (which I asked that it not),
I don't understand. It's a Jinja2 filter. Where's the lookup plugin? (Maybe I'm confusing the terminology.)

But it could probably be simply rewritten to be a lookup plugin, right?

To me the most obvious approach that doesn't add new concepts is to use it as a lookup plugin and storing passwords in individual files, exactly how the password lookup plugin does it. So you would put in your inventory things like:
mysql_password: "{{ lookup('decrypt',  'credentials/path/to') }}"

Although this is pretty similar to what can be done with this Jinja2 filter (if I'm not mistaken):
mysql_password: "{{ lookup('file',  'credentials/path/to')|vault }}"


After thinking about this.. Actually all lookup plugins could have 1:1 mapping with Jinja2 filters, like this:

{{ mysql_config_file|file }}
{{ "mysql/mysqld.conf"|file }}
{{ user_password_path|password }}
{{ "credentials/user_password"|password }}
{{ mysql_password_file|file|vault }}

{{ lookup('XXX', YYY) }} --> {{ YYY|XXX }}

This would look much nicer and more intuitive to use, or not? Thoughts?

Greetings,
gw 

Michael DeHaan

unread,
Jul 1, 2013, 3:04:14 PM7/1/13
to ansible...@googlegroups.com
Somewhat OT, as I have not had a time to read the details on the vault stuff yet, but I would *LOVE* all lookup plugins to work as Jinja2 filters, because I dislike the whole {{ lookup(...) }} thing for being too dense.

Would love a better way.




--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Dag Wieers

unread,
Jul 1, 2013, 5:37:23 PM7/1/13
to ansible...@googlegroups.com
On Mon, 1 Jul 2013, Michael DeHaan wrote:

> Somewhat OT, as I have not had a time to read the details on the vault
> stuff yet, but I would *LOVE* all lookup plugins to work as Jinja2 filters,
> because I dislike the whole {{ lookup(...) }} thing for being too dense.
>
> Would love a better way.

Problem we discovered for the 'mandatory' filter is that filters only get
the values passed, which basically means errors from filters are vague
since they have no context (no filename, no line number, no variable
name, only filter name and value).

This means that if you have a template/playbook with more than one filter,
the error will not tell you which occurence of the filter failed.

I opened a feature request for jinja2, but that's only going to be useful
in the distant future...

https://github.com/mitsuhiko/jinja2/issues/245

--
-- dag wieers, d...@wieers.com, http://dag.wieers.com/
-- dagit linux solutions, con...@dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]

Jan-Piet Mens

unread,
Jul 2, 2013, 5:44:19 AM7/2/13
to ansible...@googlegroups.com
> [1] https://github.com/ansible/ansible/pull/3393

We've had quite a bit of positive feedback on the idea, as well as some
loud doubts, and I want to recap how I think this should be done.

Intention is to protect certain variables in vars files.

---
nameserver: 10.0.0.1
admin: Jane Jolie
dbpassword: "AHchx1Va5fPZdEAY6lg+GEmejsl4tGfCNKxMFP7tM7Y7kl"
webservertype: nginx

Most vars will not be encrypted. This is meant as a protection to vars
which contain sensitive data -- basically clear-text passwords that
nobody wants to have floating around in files that get added to
repositories, etc.

I don't think encrypting vars files completely is useful: makes things
like `grep', and `diff' impossible to use.

It must be kept as simple as possible, and as secure as possible. For
this reason, using AES secret keys has charm. Keyczar implements proven
crypto stuff and is easy to add.

Secret keys mean: the person who has the key can encrypt and decrypt
data. As such, keys will be kept *outside* of the playbook repos (e.g.
/etc/keystore), but they can be distributed, e.g. to team members.
Having keys means Ansible can decrypt data on the fly, without anybody
having to enter passwords. Entering passwords is one thing we don't
want, as it breaks automation.

On the other hand, allowing more than a single key is useful in
multi-tenant situations, but more than a single key is not required.

The interface (ansible-vault) needs to be slimmed down a bit more, which
is not hard to do. I envision two basic steps:

1. Create a new key with all files, dirs, etc. automatically: ansible-vault --createkey keyname
2. Use it to encrypt data to store in a vars file: ansible-vault --key keyname -

The idea (Petros' I think) to add an --edit option to ansible-vault is
not hard, but that means loading the YAML and rewriting it; this would
a) re-order the YAML output and b) loose comments. I'm not sure either
would be satisfactory, but this can be implemented if desired.

Regards,

-JP

Vincent Van der Kussen

unread,
Jul 2, 2013, 6:20:04 AM7/2/13
to ansible...@googlegroups.com
That sums it up pretty nicely.

+1000 !

VIncent

Serge van Ginderachter

unread,
Jul 2, 2013, 6:26:25 AM7/2/13
to ansible...@googlegroups.com

Most vars will not be encrypted. This is meant as a protection to vars
which contain sensitive data -- basically clear-text passwords that
nobody wants to have floating around in files that get added to
repositories, etc.

I don't think encrypting vars files completely is useful: makes things
like `grep', and `diff' impossible to use.

+1 


On the other hand, allowing more than a single key is useful in
multi-tenant situations, but more than a single key is not required.

Does this mean mulltiple keys which each can decrypt all passwords? So different users get different keys, and a key can be revoked without impacting other users?
If yes, +1



Thanks,


Serge

Jan-Piet Mens

unread,
Jul 2, 2013, 6:36:11 AM7/2/13
to ansible...@googlegroups.com
> Does this mean mulltiple keys which each can decrypt all passwords? So
> different users get different keys, and a key can be revoked without
> impacting other users?

Not to start with, no; you have to choose a key to encrypt a particular
value with. If you revoke (i.e. delete) the key, the value can no longer
be decrypted.

The way I envision this is having a key for cust1's vars, and another
for cust2's vars, and I use vault on a per-customer basis.

Keep it simple. :)

-JP

Petros Moisiadis

unread,
Jul 2, 2013, 7:36:46 AM7/2/13