problems with quoting for ansible commands

[Kesten Broughton]

What is the proper way to quote a shell command like the following:

 shell: wget --header 'Cookie: gpw=http%3A_url' download_url

Interestingly, the colon in the  http:// in the download url (see details below) is protected somehow, so that it doesn't need any special treatment to parse correctly.  However, the colon after Cookie won't allow ansible to parse the command.  My normal workaround is to quote the colon with alternate single or double quotes from the outer quote.  But as you can see in the output of the actual command below, the line getting fed into wget is --header 'Cookie":" gpw... and this fails.

As a general comment, I find that most of my unproductive time with ansible is spent trying to dance around the quoting rules.  It's a bit of a false premise to have a DSL that is so simple that you don't need any programming skills, yet in reality, you need expert level knowledge of shell expansion rules and on top of that, an extra layer of yaml/ansible translation.  It's one of the main reasons I sometimes just copy over a bash script and execute it rather than fighting the quoted fight.  

I know that distributed shell expansion is just a very hard problem, as it has been the achilles heel of most systems I have tested that try to solve this problem.  And perhaps it is just my skills that are weak in this area.  But I would really appreciate a little more work in this area in general, and as a stopgap, a dozen or so examples of more challenging quoting issues from across the module library.

Here is the actual command issued and the error debug output + error:

 - name: Get the rpm for oracle jdk

   shell: creates="{{_ansible_downloads}}/{{jdk_rpm_file}}"

          chdir="{{_ansible_downloads}}"

          wget --no-cookies --no-check-certificate 

               --header 'Cookie":" gpw_e24=http%3A%2F%2Fwww.oracle.com%2F' 

                "http://download.oracle.com/otn-pub/java/jdk/7u25-b15/jdk-7u25-linux-x64.rpm"

                -O "{{jdk_rpm_file}}"

<Linux-Elasticsearch> REMOTE_MODULE command creates="/tmp/ansible/1.7.0_oracle_jdk_u25.rpm" chdir="/tmp/ansible" wget --no-cookies --no-check-certificate --header 'Cookie":" gpw_e24=http%3A%2F%2Fwww.oracle.com%2F' "http://download.oracle.com/otn-pub/java/jdk/7u25-b15/jdk-7u25-linux-x64.rpm" -O "1.7.0_oracle_jdk_u25.rpm" #USE_SHELL

And based on some ansible-list emails and github issues, i tried variations of the following to no avail:

quoting the entire line, backslash escaping etc. which all gave syntax errors:

"wget --no-cookies --no-check-certificate --header 'Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F' http://download.oracle.com/otn-pub/java/jdk/7u25-b15/jdk-7u25-linux-x64.rpm -O {{jdk_rpm_file}}"

ERROR: Syntax Error while loading YAML script, /Users/kbroughton/vcp/git/21ct/proto/lynx-ansible/roles/centos_common/tasks/centos_oracle_jdk.yml

Note: The error may actually appear before this position: line 21, column 69

tips most welcome!

[Kahlil Hodgson]

On Mon, Oct 21, 2013 at 1:24 PM, Kesten Broughton <solarmobi...@gmail.com> wrote:

What is the proper way to quote a shell command like the following:

 shell: wget --header 'Cookie: gpw=http%3A_url' download_url

Interestingly, the colon in the  http:// in the download url (see details below) is protected somehow, so that it doesn't need any special treatment to parse correctly.  However, the colon after Cookie won't allow ansible to parse the command.  My normal workaround is to quote the colon with alternate single or double quotes from the outer quote.  But as you can see in the output of the actual command below, the line getting fed into wget is --header 'Cookie":" gpw... and this fails.

The problem is the two character sequence ': ', that is, a colon followed by a space, which is a special YAML marker.  I typically get around this using YAML '>' quoting as follows:

 shell: >
    wget --header 'Cookie: gpw=http%3A_url' download_url

Hope this helps,

K

Kahlil (Kal) Hodgson                       GPG: C9A02289
Head of Technology                         (m) +61 (0) 4 2573 0382
DealMax Pty Ltd                            (w) +61 (0) 3 9008 5281

Suite 1415
401 Docklands Drive
Docklands VIC 3008 Australia

"All parts should go together without forcing.  You must remember that
the parts you are reassembling were disassembled by you.  Therefore,
if you can't get them together again, there must be a reason.  By all
means, do not use a hammer."  -- IBM maintenance manual, 1925

[Michael DeHaan]

Basically you just can't have an unquoted colon in the value of a hash element, and can then quote the line.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Kahlil Hodgson]

Sadly, it's the space after the colon that is the problem.

shell: wget --header 'Cookie:gpw=http%3A_url' download_url

will work

K

Kahlil (Kal) Hodgson                       GPG: C9A02289
Head of Technology                         (m) +61 (0) 4 2573 0382
DealMax Pty Ltd                            (w) +61 (0) 3 9008 5281

Suite 1415
401 Docklands Drive
Docklands VIC 3008 Australia

"All parts should go together without forcing.  You must remember that
the parts you are reassembling were disassembled by you.  Therefore,
if you can't get them together again, there must be a reason.  By all
means, do not use a hammer."  -- IBM maintenance manual, 1925

[Kesten Broughton]

Thanks Kahlil

It worked.  It's a little finicky in that it doesn't play "nice" with normal facilities like spanning lines and other shell options.

[Works not]

 shell: 

 >

        wget --header 'Cookie: gpw=http%3A_url' download_url -O output_file

[Works]

 shell: > 

        wget --header 'Cookie: gpw=http%3A_url' download_url -O output_file

[Works]

 shell: > 

        wget --header 'Cookie: gpw=http%3A_url' download_url -O output_file

[Kesten Broughton]

I think my last post got posted incompletely.

Thanks Kahlil

It works, but it's a little finicky in that it doesn't play nice with other ansible constructs

[Works]

 shell: > 

        wget --header 'Cookie: gpw=http%3A_url' download_url -O output_file

        creates=output_file

[Doesn't work: putting other shell options before it]

 shell: creates=output_file 

         > 

          wget --header 'Cookie: gpw=http%3A_url' download_url -O output_file

[Doesn't work: spanning multiple lines]

 shell: > 

            wget --header 'Cookie: gpw=http%3A_url' download_url 

                     -O output_file

But hey, it works enough.  Thanks!