multiple multi-line parsing regressions

[Abhijit Menon-Sen]

Hi.

I have a vars/foo.yml file that contains:

foo: |
---
foo
bar
---

And then I have tasks that look like this:

- name: Load foo
include_vars: foo.yml

- name: Write out foo
copy: content="{{ foo }}" dest=/foo …

With Ansible 1.6.8, this does what I expect. The file contains
"---\nfoo\nbar\n---\n".

With Ansible 1.6.9 (which contained "Further improvements to module
parameter parsing to address additional regressions caused by security
fixes"), the file contains "--- foo bar ---" (no newlines at all!).

With Ansible 1.7.0 and 1.7.1, the file contains what I expect, i.e. they
exhibit the 1.6.8 behaviour.

With devel (v1.6.0-1677-g1dc11c9), all but the last newline are
*tripled*. The file contains: "---\n\n\nfoo\n\n\nbar\n\n\n---\n"

In every case, the include_vars statement results in the same (correct)
multiline string:

ok: [foo] => {"ansible_facts": {"bar": 3, "foo": "---\nfoo\nbar\n---\n"}}

Rewriting the task to use YAML hash syntax for arguments makes the
problem go away, of course:

- name: Write out foo
copy:
content: "{{ foo }}"
dest: /foo
…

Is this a bug, two bugs, three? Should I be avoiding using the key=value
form altogether when my values might contain newlines or quotes (issue
#6294) or anything 'strange'?

The behaviour oscillating like this is a bit worrying.

-- ams

[Michael DeHaan]

How about not using the silly content parameter?  

It's really intended for exceedingly basic one-liner things, and as you know in Ansible that value will get templated out to form a command line to the module, so having weird newlines in there, without the variable, wouldn't make much sense anyway.

Using the "template" module can also be a great solution if you want to trivially just push some value in to the remote side.

-- ams

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/20140912090257.GA31462%40toroid.org.
For more options, visit https://groups.google.com/d/optout.

[Michael DeHaan]

Alternatively:

copy:

   content: "{{ value }}"

   dest: foo

Will pass things along without trying to go through the line-oriented parsing.

[Abhijit Menon-Sen]

At 2014-09-12 10:29:23 -0400, mic...@ansible.com wrote:
>
> Alternatively:
>
> copy:
> content: "{{ value }}"
> dest: foo
>
> Will pass things along without trying to go through the line-oriented
> parsing.

Yes, as I mentioned in my mail, that's the workaround I'm already using.
I do think the documentation should mention something about when to not
use key=value syntax, though.

I find copy's "content" parameter quite convenient to put things like
policy-rc.d scripts and SSL keys from ansible-vault in place. But even
if we disregard that as silly, authorized_key's "key" parameter has
exactly the same problem with parsing.

The documentation for it uses key="{{ lookup … }}" _everywhere_, and
that breaks when your key file starts with «from="12.34.45.56", …»
(which is what github issue #6294 was about).

Would you take a patch to change all the example to use key: "…"
syntax instead?

-- ams

[Michael DeHaan]

Changing all the examples would make very good since for the short term, though I believe we can also deal with fixing the \n issue.

I'm not aware of anything wrong with the key parameter, usually that's one entry per line anyway, and also only used with a single line, but multiple lines don't hurt that file IIRC.

-- ams

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/20140912153407.GA16026%40toroid.org.

[Abhijit Menon-Sen]

At 2014-09-12 15:14:42 -0400, mic...@ansible.com wrote:
>
> Changing all the examples would make very good since for the short
> term, though I believe we can also deal with fixing the \n issue.

OK, I'll submit a pull request to change the examples.

> I'm not aware of anything wrong with the key parameter, usually that's
> one entry per line anyway, and also only used with a single line, but
> multiple lines don't hurt that file IIRC.

No, the problem with authorized_key's key= is not multiple lines, but
quoting. For example, if you want to install the following key:

from="1.2.3.4",command="rsync --server -logDtpre.iLsf --delete . foo/",no-pty,no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-rsa …

using something like:

authorized_key: user=foo key="{{ lookup('file', …) }}"

Then you get:

fatal: [localhost] => A variable inserted a new parameter into the
module args. Be sure to quote variables if they contain equal signs (for
example: "{{var}}").

This has come up a few times, including in the bug report that I linked
to in my earlier post. For this problem, too, using the "key: value"
syntax makes the problem go away. So I'll change those examples too.

Independent of that, if you want to outline your approach to solving the
\n and quoting problems, I'd be happy to work on them.

-- ams

[Abhijit Menon-Sen]

At 2014-09-19 10:09:47 +0530, a...@toroid.org wrote:
>
> At 2014-09-12 15:14:42 -0400, mic...@ansible.com wrote:
> >
> > Changing all the examples would make very good since for the short
> > term, though I believe we can also deal with fixing the \n issue.
>
> OK, I'll submit a pull request to change the examples.

Done: https://github.com/ansible/ansible/pull/9068

Thanks for your comments.

-- ams