Passing a list to a callback - quoting issue

[wildfan]

I have wasted way too much time on this seemingly simple solution, and was looking for some help.

We have a Tower job to deploy AWS cloudwatch and configure the logs to monitor that is kicked off via a callback.  I use the following curl-

The callback is:

curl -f -k -H 'Content-Type: application/json' -XPOST -d '{"host_config_key": "xxxxx", "extra_vars": "{\"logs\":{{ eck_logs }},\"env_stage\":\"dev\"}"}'  "https://tower.xxxx.net:443/api/v1/job_templates/xx/callback/"

Which expands to

"curl -f -k -H 'Content-Type: application/json' -XPOST -d '{\"host_config_key\": \"xxx\", \"extra_vars\": \"{\\\"logs\\\":\\\"[{u'file': u'/apps/app1/shared/logs/catalina.out', u'group_name': u'/on-premise/app1/container'}, {u'file': u'/apps/app1/shared/logs/app1-webapp.log', u'group_name': u'/on-premise/app1-webapp/applog'}]\\\",\\\"env_stage\\\":\\\"dev\\\"}\"}' \"https://tower.xxx.net:443/api/v1/job_templates/xx/callback/\""..

The eck_logs variable looks like

"eck_logs": [

            {

                "file": "/apps/app1/shared/logs/catalina.out",

                "group_name": "/on-premise/app1/container"

            },

            {

                "file": "/apps/app1/shared/logs/app1-webapp.log",

                "group_name": "/on-premise/app1/applog"

            }

        ]

The issue i am running into is, i believe all about quoting..  as it is currently, it will pass the variable to the Tower job, however it tacks on the "u" for unicode to the variable resulting in:

logs:

  - ugroup_name: u/on-premise/app1-webapp/container

    ufile: u/apps/app1/shared/logs/catalina.out

  - ugroup_name: u/on-premise/app1/applog

    ufile: u/apps/app1/shared/logs/app1-webapp.log

when i quote the eck_logs variable in the curl command, i then instead get-

command:

curl -f -k -H 'Content-Type: application/json' -XPOST -d '{"host_config_key": "xxx", "extra_vars": "{\"logs\":\"{{ eck_logs }}\",\"env_stage\":\"dev\"}"}'  "https://tower.xxx.net:443/api/v1/job_templates/xxx/callback/"

result

"curl -f -k -H 'Content-Type: application/json' -XPOST -d '{\"host_config_key\": \"xxx\", \"extra_vars\": \"{\\\"logs\\\":\\\"[{u'file': u'/apps/app1/shared/logs/catalina.out', u'group_name': u'/on-premise/app1/container'}, {u'file': u'/apps/app1/shared/logs/app1-webapp.log', u'group_name': u'/on-premise/app1-webapp/applog'}]\\\",\\\"env_stage\\\":\\\"dev\\\"}\"}' \"https://tower.xxx.net:443/api/v1/job_templates/xxx/callback/\""

Variable on Tower:

logs: >-

  [{ufile: u/apps/app1/shared/logs/catalina.out, ugroup_name:

  u/on-premise/app1/container}, {ufile:

  u/apps/app1/shared/logs/app1-webapp.log, ugroup_name:

  u/on-premise/app1/applog}]

I have tried seemingly every filter, quoting, escaping I can and i can't get the desired variable value when passing tower..  

I am looking for the results in my first example for the logs variable (list), but without the leading "u" getting added to the values..

I realize this might be a mixture of a jinja/ansible question , but does anyone have any ideas on what i can try to fix this?

Thanks in advance!

[Matt Martz]

You are just giving your curl command the python data structure of `eck_logs` and effectively you get a text representation of the data which includes the `u` prefix to indicate Unicode.

You likely want to do the following instead:

{{ eck_logs|to_json }}

To output the data in JSON format.

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/0ab058e1-63f5-4079-8037-941ba407971b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Matt Martz
@sivel
sivel.net

[wildfan]

Thanks for the reply..  i had tried using the to_json filter previously with no luck..  ( i apologize for so much output below, but i think it is necessary to show / solve the issue)

When called with this:

 curl -f -k -H 'Content-Type: application/json' -XPOST -d '{"host_config_key": "xxx", "extra_vars": "{\"logs\":\"{{ eck_logs | to_json }}\",\"env_stage\":\"dev\"}"}'  "https://tower.xxx:443/api/v1/job_templates/xxx/callback/"

 Results in 

curl -f -k -H 'Content-Type: application/json' -XPOST -d '{\"host_config_key\": \"xxx\", \"extra_vars\": \"{\\\"logs\\\":\\\"[{\"file\": \"/apps/app1/shared/logs/catalina.out\", \"group_name\": \"/on-premise/app1/container\"}, {\"file\": \"/apps/app1/shared/logs/app1.log\", \"group_name\": \"/on-premise/app1-webapp/applog\"}]\\\",\\\"env_stage\\\":\\\"dev\\\"}\"}' \"https://tower.xxx.net:443/api/v1/job_templates/xxx/callback/\"

which fails on the callback 

curl: (22) The requested URL returned error: 400 BAD REQUEST

When I remove the quoting and escape around "\{{ eck_logs | to_json }}\" it also fails with the same curl error (it is running the following)

curl -f -k -H 'Content-Type: application/json' -XPOST -d '{\"host_config_key\": \"xxx\", \"extra_vars\": \"{\\\"logs\\\":[{\"file\": \"/apps/app1/shared/logs/catalina.out\", \"group_name\": \"/on-premise/app1/container\"}, {\"file\": \"/apps/app1/shared/logs/app1.log\", \"group_name\": \"/on-premise/app1/applog\"}],\\\"env_stage\\\":\\\"dev\\\"}\"}' \"https://tower.xxx.net:443/api/v1/job_templates/xx/callback/\"

It chokes as it adds the quoting around each of the fields..  it needs to pass as the following for it to give me the desired results

\\\"[{'file': '/apps/app1/shared/logs/catalina.out', 'group_name': '/on-premise/app1/container'}, {'file': '/apps/app1/shared/logs/app1-webapp.log', 'group_name': '/on-premise/app1-webapp/applog'}]\\\"

[wildfan]

Sorry for the additional  reply, but i realized i had the incorrect desired output in my last one..  When the curl command is the following, it works and gives me the correct extra-var on the tower side:

curl -f -k -H 'Content-Type: application/json' -XPOST -d '{"host_config_key": "xxx", "extra_vars": "{\"logs\":[{file: /apps/app1/shared/logs/catalina.out, group_name: /on-premise/app1/container}, {file: /apps/app1/shared/logs/app1.log, group_name: /on-premise/app1/webapp/applog}],\"env_stage\":\"dev\"}"}'  "https://tower.xxx.net:443/api/v1/job_templates/xxx/callback/"

env_stage: dev

logs:

  - file: /apps/app1/shared/logs/catalina.out

    group_name: /on-premise/app1/container

  - file: /apps/app1/shared/logs/app1.log

    group_name: /on-premise/app1/webapp/applog

for it to work, the {{ eck_logs }} variable to expand to not include the quoting (ie [{file: /apps/app1/shared/logs/catalina.out, group_name: /on-premise/app1/container}, {file: /apps/app1/shared/logs/app1.log, group_name: /on-premise/app1/webapp/applog}])  or the callback fails.  

thx

-T

[wildfan]

Thanks Matt for steering me towards a solution....I was able to get this working, utilizing the to_json and then stripping out all quotes..  I used an intermediate fact to get to the final desired output as i didnt have luck when doing it inline, but there may be a way to skip that extra step...  

    - block:

      - name: Create fact for log info to format for callback

        set_fact:

          logs: "{{ eck_logs | to_json | replace('\"', '') }}"

      - name: Perform callback to tower to configure ECK logging

        command: >

               curl -f -k -H 'Content-Type: application/json' -XPOST -d '{"host_config_key": "xxx", "extra_vars": "{\"logs\":{{ logs }},\"env_stage\":\"dev\"}"}'  "https://tower.xxx.net:443/api/v1/job_templates/xx/callback/"

      when: eck_logs is defined

thx for pointing me in the right direction, and sorry for all my replies!

-T