Ansible quoting

[Michael Perzel]

I am having trouble getting ansible to parse my variable values correctly. 

I have defined a variable:

installArguments: "appPoolName='UiService'; dbPassword='password'; dbUserId='user'"

I then have a playbook that makes the following call:

- name: Deploy Launcher

script: deployLauncher.ps1 -env {{ env }} -appName {{ appName }} -artifacts {{ artifacts }} -installArguments @{ '{{ installArguments }}' } -deployNumber {{ deployNumber }} {{ switch }}

If I don't include the single quotes around {{installArguments}} I get a complaint about the equal sign being unquoted.

deployLauncher.ps1 is meant to simply echo back the variables at this point.

param(

    [Parameter(Mandatory=$true,Position=0)][string] $env,

    [Parameter(Mandatory=$true,Position=1)][string] $server,

    [Parameter(Mandatory=$true,Position=2)][string] $appName,

    [Parameter(Mandatory=$true,Position=3)][string[]] $artifacts, #first contains install script

    [Parameter(Mandatory=$false,Position=4)][hashtable] $installArguments = @{},

    [Parameter(Mandatory=$true,Position=5)][string] $deployNumber,

    [Parameter(Mandatory=$false,Position=6)][switch] $switch

)

#./testLauncher  -env Local -server serverName -appName Name -artifacts artifacts.zip -installArguments @{key='value'} -deployNumber 123 -switch

Write-Output "Env: $env Server: $server AppName: $appName Artifacts: $artifacts InstallArguments: $installArguments DeployNumber: $deployNumber Switch: $switch"

When I run the playbook I get the following powershell error.

TASK: [common | debug var=deployLauncher] ************************************* ok: [mdm-wsrv98] => { "deployLauncher": { "changed": true, "invocation": { "module_args": "deployLauncher.ps1 -env LOCAL -appName UiService.Event -artifacts UiService.Event.zip -installArguments @{ 'appPoolName='UiService'; dbPassword='password'; dbUserId='user'' } -deployNumber 456 -switch", "module_name": "script" }, "rc": 0, "stderr": "C:\\Users\\ansible\\AppData\\Local\\Temp\\ansible-tmp-1413821995.23-24997058650291\\de\r\nployLauncher.ps1 : Cannot process argument transformation on parameter \r\n'installArguments'. Cannot convert the \"@{\" value of type \"System.String\" to \r\ntype \"System.Collections.Hashtable\".\r\n + CategoryInfo : InvalidData: (:) [deployLauncher.ps1], ParentCon \r\n tainsErrorRecordException\r\n + FullyQualifiedErrorId : ParameterArgumentTransformationError,deployLaunc \r\n her.ps1\r\n \r\n", "stdout": "", "stdout_lines": [] } }

I think that my issue is that ansible is inserting a single quote after @{ and again after 'user' which isn't valid powershell array syntax. Is there a better way to quote this so that these don't get included?

Thanks!

[Michael DeHaan]

Possibly easier to use the win_script module maybe?

Ansible shouldn't be adding any extra spaces here though, particularly not in recent versions.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/11db0119-bd96-40e1-aaf0-63afbc5f468e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Perzel]

Can you provide a link to the win_script module? I don't see it in the ansible documentation.

I'm using ansible 1.7.2. I've confirmed the issue is caused by my variable including an equal sign. I need to quote my variable {{ "var" }} in the playbook, but when this variable is used in the script module the quotes are included. For example,

var: @{key=value}

script: powershell.ps1 {{ "var" }}

The call to the windows box is:

powershell.ps1 "@{key=value}" 

instead of:

powershell.ps1 @{key=value}

[Michael DeHaan]

Sorry, I think I mispoke here.

"script" is one of those that works on both platforms.

What you have above is more of a quoting question with what arguments you are passing to the script, not the contents, so that won't help...

We can investigate, but perhaps others might know the answer to this one?

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/a2293dfc-2f3b-4c16-88de-ea8f675a91ca%40googlegroups.com.

[J Hawkesworth]

I don't have a direct answer for this - rightly or wrongly the ps1 scripts I have so far don't take any arguments - I consider them to be specific to my roles and at the moment they embed some details that it would probably be best to have as parameters.  However, it looks to me like your deployLauncher.ps1 really wants to be a very general purpose deployment tool and maybe it would be better off as an ansible module.  That way you could perhaps take advantage of ansible's existing syntax for specifying module arguments.

As an aside I believe @Trond Hindenes has a win_package module in the works which might do some of what you want.

[Michael Perzel]

I'll look into this. My work around thus far is using a template for my deploy script. Instead of passing the deploy script arguments, I just assign them at the top:

$build = {{ build }}

$pass = {{ password }}

$arguments = @{ {{ arguments }} }

This then becomes:

$build = 42

$pass = password

$arguments = @{ key1=value1; key2= value2 }

Using the template has given me complete control over the syntax. In our paradigm we have an entirely generic "deploy" powershell script that stages files, unzips them and similar things. It then calls an "install" script that handles all the app specific items. The install scripts are bundled with the specific app.

[Chris Church]

That method of passing arguments is apparently called "splatting" (http://technet.microsoft.com/en-us/magazine/gg675931.aspx).

I have a branch where I've been able to get it working:

https://github.com/cchurch/ansible/tree/powershell_splatting

I've also added integration tests to show examples of it in use:

https://github.com/cchurch/ansible/blob/powershell_splatting/test/integration/roles/test_win_script/tasks/main.yml#L50

I also considered the idea of a Jinja2 filter to convert a YAML/JSON data structure to this format to allow for defining your arguments as you would any other variables:

build_args:

  key1: value1

  key2: value2

You could then run your task as:

script: dostuff.ps1 {{ build_args|splattify }}

Would you (or any other Windows users) be interested in this approach?

--

You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/388179f6-448e-44e7-967f-94e400dc4514%40googlegroups.com.

[Michael Perzel]

Your idea with jinja filter is what I originally tried to get working. I never managed it but I think its the ideal solution (converting yaml hash to a powershell one). 

If I follow your patch you added a parameter that controls whether or not arguments get quoted (and set it to false for powershell scripts)? I'm not particularly worried about nefarious activity within our system so this could work for us.

[Michael Perzel]

I tried taking your changes but it failed with:

<Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><S S="Error">C:\Users\ansible\AppData\Local\Temp\ansible-tmp-1414187414.25-97343749557638\de_x000D__x000A_</S><S S="Error">ployLauncher.ps1 : A positional parameter cannot be found that accepts _x000D__x000A_</S><S S="Error">argument 'False'._x000D__x000A_</S><S S="Error">At line:1 char:1_x000D__x000A_</S><S S="Error">+ &amp; _x000D__x000A_</S><S S="Error">C:\Users\ansible\AppData\Local\Temp\ansible-tmp-1414187414.25-97343749557638\\ _x000D__x000A_</S><S S="Error">..._x000D__x000A_</S><S S="Error">+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~_x000D__x000A_</S><S S="Error">~~~_x000D__x000A_</S><S S="Error"> + CategoryInfo : InvalidArgument: (:) [deployLauncher.ps1], Param _x000D__x000A_</S><S S="Error"> eterBindingException_x000D__x000A_</S><S S="Error"> + FullyQualifiedErrorId : PositionalParameterNotFound,deployLauncher.ps1_x000D__x000A_</S><S S="Error"> _x000D__x000A_</S></Objs>

Looking at the diff between your code and mine there's a few other differences. I'm running 1.7.2,  I assume your branched off something newer.I'll see if I can sort through the issue.

Thanks

[Chris Church]

Could you create a gist (https://gist.github.com/) with the relevant lines from your inventory variables, playbook and script (removing any sensitive information)?  I'm not quite able to piece together a full example of what you're running from the email thread.

My branch is based on devel; I just created another branch based on 1.7.2 with the splatting changes applied: https://github.com/cchurch/ansible/tree/powershell_splatting_v172

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/ba6b122a-1fa5-48c6-9b1d-a8a2f67083ef%40googlegroups.com.

[Michael Perzel]

Take a look at this gist: https://gist.github.com/perzizzle/b3e13af9e94d9fbb7970/download

It contains a simple playbook that calls a deploy.ps1 that should echo back installArguments (which is a powershell array). I tried a few different ways of quoting it with your patch applied but I get argument transformation errors. Next I'll trying doing a git pull on your entire branch, so far I had just been modifying the files you edited. I tested calling deploy.ps1 from a windows server and it does work if provided valid powershell syntax. 

.\deploy.ps1 -installArguments @{key="value"}

I'll take a look at your tests and make sure I'm not doing anything different.

[Michael Perzel]

Alright, I made your changes in your 1.7.2 branch and it works if the variables are defined with quotes in the following order (I updated the gist to reflect this).

  vars:

    installArguments: "@{key='value'}"

although it echo's the hash value correctly (it is just "value" maybe not the best naming) it returns something in stderr. At this point I have no clue what it means but I'm searching the google.

TASK: [debug var=deploy] ******************************************************

ok: [mdm-wsrv99.surescripts-dev.qa] => {

    "deploy": {

        "changed": true,

        "invocation": {

            "module_args": "deploy.ps1 -installArguments \"@{key='value'}\"",

            "module_name": "script"

        },

        "rc": 0,

        "stderr": "#< CLIXML\r\n<Objs Version=\"1.1.0.1\" xmlns=\"http://schemas.microsoft.com/powershell/2004/04\"><Obj S=\"progress\" RefId=\"0\"><TN RefId=\"0\"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N=\"SourceId\">1</I64><PR N=\"Record\"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>",

        "stdout": "Install Arguments: value\n",

        "stdout_lines": [

            "Install Arguments: value"

        ]

    }

}

ok: [mdm-wsrv98.surescripts-dev.qa] => {

    "deploy": {

        "changed": true,

        "invocation": {

            "module_args": "deploy.ps1 -installArguments \"@{key='value'}\"",

            "module_name": "script"

        },

        "rc": 0,

        "stderr": "#< CLIXML\r\n<Objs Version=\"1.1.0.1\" xmlns=\"http://schemas.microsoft.com/powershell/2004/04\"><Obj S=\"progress\" RefId=\"0\"><TN RefId=\"0\"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N=\"SourceId\">1</I64><PR N=\"Record\"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>",

        "stdout": "Install Arguments: value\n",

        "stdout_lines": [

            "Install Arguments: value"

        ]

    }

}

[Michael Perzel]

Are you going to submit a pull request for this into devel? It definitely solved my issue.

On Saturday, October 25, 2014 2:23:55 PM UTC-5, Chris Church wrote:

[Chris Church]

Absolutely, just haven't had a chance to get back to Windows-related things this week.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/118759f9-df6f-4805-886d-00a250457019%40googlegroups.com.

[Michael Perzel]

I ended up creating a new filter that could transform 

 

installArguments:

      key: value

      key2: value2

and return “@{'key2'='value2';'key'='value';}” I can share if anyone is interested.

I just upgraded to ansible 1.8 and made your fix to powershell.py and winrm.py manually again. Is there any target version to get this into the main branch? I poked around in github but couldn't find way to see if you had submitted a pull request for this.

[Chris Church]

Hi Michael,

I had submitted a PR (https://github.com/ansible/ansible/pull/9602) -- I'll see if I can get that merged soon.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/35495c72-0da0-42f9-b854-4ab8f049ccdd%40googlegroups.com.

[Michael Perzel]

This isn't directly related to this thread but it was discussed earlier so it was the best place for it.

I created a filter that converts a yaml dict to a powershell hash. I'm trying to write some tests for it but I'm not sure how to run just my specific test. The file playbook and vars are at ansible/test/integration/roles/test_powershell_filter/ and the filter is at ansible/lib/ansible/runner/filter_plugins.

Test playbook:

https://gist.github.com/perzizzle/8fb875895b218772eede

Test variables

https://gist.github.com/perzizzle/dc73025a2232f2da9b88

Filter

https://gist.github.com/perzizzle/117a6b139e2a97194d5f

When I run "make tests" a whole bunch of them succeed but one fails related to postgres. Is it possible to run just my one test I've created?

Thanks,

[Chris Church]

Add your new test role to test/integration/test_winrm.yml with a matching tag, then run:

TEST_FLAGS="-t test_powershell_filter" make test_winrm

It may be worthwhile to add to the test_win_script role to make sure your filter works for passing parameters to a PowerShell script.

Also, from your test playbook, there is no echo module - maybe you meant to use command instead?

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/db5bd592-8018-4bf7-971d-6f8652fd9737%40googlegroups.com.

[Michael Perzel]

I updated test_winrm.yml to include my role but when I tried to run 

ansible-source/ansible> TEST_FLAGS="-t test_powershell_filter" make test_winrm

make: *** No rule to make target `test_winrm'.  Stop.

Did you need to make changes to the Makefile in order for this to work?

I updated my playbook. I had intended command: echo ....  I'll update test_win_script as well once I get this working.

[Michael Perzel]

I changed my directory so that test_winrm was in the current directory. But now I get an error message complaining it can't find the win_file module. I assume its not finding my ansible-module checkout.

ansible/test/integration> TEST_FLAGS="-t test_powershell_filter -i /etc/ansible/hosts" make test_winrm

ansible-playbook test_winrm.yml -i inventory.winrm -e @integration_config.yml  -v -t test_powershell_filter -i /etc/ansible/hosts

 [WARNING]: The version of gmp you have installed has a known issue regarding

timing vulnerabilities when used with pycrypto. If possible, you should update

it (i.e. yum update gmp).

ERROR: win_file is not a legal parameter in an Ansible task or handler

make: *** [test_winrm] Error 1

I'll keep working through this... 

[Michael Perzel]

Victory!  I'll add a few tests for invalid data and send in a pull request.

TEST_FLAGS="-t test_powershell_filter -i /etc/ansible/hosts" make test_winrm

ansible-playbook test_winrm.yml -i inventory.winrm -e @integration_config.yml  -v -t test_powershell_filter -i /etc/ansible/hosts

 [WARNING]: The version of gmp you have installed has a known issue regarding

timing vulnerabilities when used with pycrypto. If possible, you should update

it (i.e. yum update gmp).

PLAY [windows] ****************************************************************

TASK: [test_powershell_filter | Convert yaml dict to powershell hash] *********

changed: [mdm-wsrv98.surescripts-dev.qa -> 127.0.0.1] => {"changed": true, "cmd": "echo '@{'This'='THIS';'Other'='OTHER';'That'='THAT';}'", "delta": "0:00:00.003494", "end": "2015-02-23 15:52:47.477128", "rc": 0, "start": "2015-02-23 15:52:47.473634", "stderr": "", "stdout": "@{This=THIS;Other=OTHER;That=THAT;}", "warnings": []}

TASK: [test_powershell_filter | Check that actual equals expected] ************

ok: [mdm-wsrv98.surescripts-dev.qa] => {"msg": "all assertions passed"}

[Michael Perzel]

I updated the test_win_script role in my fork to use the filter and confirm the output matches the expected results.

[Michael Perzel]

I recently upgrade to ansible 1.9.2 and am now having issues with the splatting. I looked at https://github.com/cchurch/ansible/blob/e9b6aaf5d8836ce7ffdca855e006c2131fe19632/lib/ansible/runner/shell_plugins/powershell.py since that fixed it last time. THe only change that wasn't still there was in powershell.py. I made the following change and it works now. I haven't gone back to debug closer to see what the difference is.

def _build_file_cmd(cmd_parts, quote_args=True):

    '''Build command line to run a file, given list of file name plus args.'''

    if quote_args:

        cmd_parts = ['"%s"' % x for x in cmd_parts]

    #return ' '.join(_common_args + ['-ExecutionPolicy', 'Unrestricted', '-File'] + cmd_parts)

    return ' '.join(['&'] + cmd_parts)

Before the change this was the output:

TASK: [windowsDeploy | Call deploy.ps1] *************************************** changed: [hostname] => {"changed": true, "rc": 0, "stderr": "C:\\Users\\l_ansible\\AppData\\Local\\Temp\\ansible-tmp-1438627351.4-181936062454137\\\r\nwrapper.ps1 : Cannot process argument transformation on parameter \r\n'installArguments'. Cannot convert the \"System.Collections.Hashtable\" value of \r\ntype \"System.String\" to type \"System.Collections.Hashtable\".\r\n + CategoryInfo : InvalidData: (:) [wrapper.ps1], ParentContainsEr \r\n rorRecordException\r\n + FullyQualifiedErrorId : ParameterArgumentTransformationError,wrapper.ps1\r\n \r\n", "stdout": ""}

I set the debug higher to see the splatted argument and verified it is a correct powershell array/hash. I can fill out an issue at github but thought this would be a good start.

[J Hawkesworth]

Worth trying current dev if you can as I think the way parameters are passed has changed.

All the best, Jon

[Ard-Jan Barnas]

I had the same problem and made the change below to fix the issue. 

[Michael Perzel]

How did you solve the issue (what did you mean by below)? I haven't had time to look into the root cause of this yet.