to_json filter (sometimes) turns values into strings

[John Riendeau]

Hi all,

It seems like the to_json filter behaves differently depending on:

- Whether it's passed the result of a template call directly, or whether the result of the template call is placed into a fact first

- Whether the template call actually does any substitutions or not

- Whether I'm using ansible community <=v4.10.0 or >=5.0.1

I'm including a test playbook that illustrates the behavior below, along with the relevant portions of the output. I'm relatively new to Ansible, so wanted to check in here before submitting a bug report -- this may be just a nuance I'm not grasping.

When I run the playbook using Ansible 4.10.0, to_json turns the value into a JSON string, if and only if it's passed the result of a template call which does no substitutions. If the template has a variable in it, or if the result of the template call is stored in a fact before being sent to to_json, the JSON looks how I'd expect.

When using Ansible 5.0.1, the value is turned into a string whenever to_json is passed the result of a template call, regardless of whether the template has a variable in it or not.

There's a lot of permutations here, and I'm not sure which, if any, of these behaviors are unexpected. Thanks in advance for any help!

 - John

Test playbook:

---

- name: Test with trivial template

hosts: localhost

tasks:

- copy:

dest: "trivial.json.j2"

content: '{ "hello": "world" }'

- set_fact:

template_result: "{{ lookup('template', 'test.json.j2') }}"

- set_fact:

json_from_fact_trivial_template: "{{ template_result | to_json }}"

json_from_template_trivial_template: "{{ lookup('template', 'test.json.j2') | to_json }}"

- debug:

var: json_from_fact_trivial_template

- debug:

var: json_from_template_trivial_template

- name: Test with template including variable

hosts: localhost

tasks:

- copy:

dest: "withvar.json.j2"

content: '{% raw %}{ "{{ greeting }}": "world" }{% endraw %}'

- set_fact:

greeting: 'howdy'

- set_fact:

template_result: "{{ lookup('template', 'withvar.json.j2') }}"

- set_fact:

json_from_fact_template_withvar: "{{ template_result | to_json }}"

json_from_template_withvar: "{{ lookup('template', 'withvar.json.j2') | to_json }}"

- debug:

var: json_from_fact_template_withvar

- debug:

var: json_from_template_withvar

Output with Ansible 4.10.0:

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "json_from_fact_trivial_template": {
        "hello": "world"
    }
}

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "json_from_template_trivial_template": "\"{ \\\"hello\\\": \\\"world\\\" }\""
}

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "json_from_fact_template_withvar": {
        "howdy": "world"
    }
}

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "json_from_template_withvar": {
        "howdy": "world"
    }
}

Output with Ansible 5.0.1:

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "json_from_fact_trivial_template": {
        "hello": "world"
    }
}

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "json_from_template_trivial_template": "\"{ \\\"hello\\\": \\\"world\\\" }\""
}

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "json_from_fact_template_withvar": {
        "howdy": "world"
    }
}

TASK [debug] ****************************************************************************************************************************************************************
ok: [localhost] => {
    "json_from_template_withvar": "\"{\\\"howdy\\\": \\\"world\\\"}\""
}

[John Riendeau]

Apologies - just noticed that my test playbook had an error in it (was trying to simplify and clarify the test case and forgot to change the template file name in a couple places). It doesn't change the behavior I'm seeing, but this is the corrected version:

---
- name: Test with trivial template
  hosts: localhost
  tasks:
  - copy:
      dest: "trivial.json.j2"
      content: '{ "hello": "world" }'
  - set_fact:

      template_result: "{{ lookup('template', 'trivial.json.j2') }}"

  - set_fact:
      json_from_fact_trivial_template: "{{ template_result | to_json }}"

      json_from_template_trivial_template: "{{ lookup('template', 'trivial.json.j2') | to_json }}"

  - debug:
      var: json_from_fact_trivial_template
  - debug:
      var: json_from_template_trivial_template

- name: Test with template including variable
  hosts: localhost
  tasks:
  - copy:
      dest: "withvar.json.j2"
      content: '{% raw %}{ "{{ greeting }}": "world" }{% endraw %}'
  - set_fact:
      greeting: 'howdy'
  - set_fact:
      template_result: "{{ lookup('template', 'withvar.json.j2') }}"
  - set_fact:
      json_from_fact_template_withvar: "{{ template_result | to_json }}"
      json_from_template_withvar: "{{ lookup('template', 'withvar.json.j2') | to_json }}"
  - debug:
      var: json_from_fact_template_withvar
  - debug:
      var: json_from_template_withvar