Jinja claiming undefined variable for defined variable.

[Kory Brown]

Hiya! So I wrote up this hacky little bit to automate deploying VMs using KVM:

{% for vm in pillar[grains['datacenter'] + '_nodes'] %}
  {% if pillar[grains['datacenter'] + '_nodes'][vm]['generate'] %}
generate-{{ vm }}:
  file.managed:
    - name: /var/preseed/{{ vm }}/preseed.cfg
    - source: salt://kvm/files/preseed
    - user: root
    - group: root
    - mode: 644
    - template: jinja
    - context:
      ip_address: {{ pillar[grains['datacenter'] + '_nodes'][vm]['ip_address'] }}
      netmask: {{ pillar[grains['datacenter'] + '_nodes'][vm]['netmask'] }}
      gateway: {{ pillar[grains['datacenter'] + '_nodes'][vm]['gateway'] }}
      dns_server: {{ pillar[grains['datacenter'] + '_nodes'][vm]['dns_server'] }}
    - require:
      - file: /etc/network/interfaces
      - file: /etc/libvirt/qemu.conf
  cmd.run:
    - name: virt-install --name={{ vm }} --arch=x86_64 --vcpus={{ pillar[grains['datacenter'] + '_nodes'][vm]['num_cpus'] }} --cpu host --ram={{ pillar[grains['datacenter'] + '_nodes'][vm]['ram_in_mb'] }} --os-type=linux --os-variant=debianwheezy --hvm --connect=qemu:///system  --network bridge:br0,model=virtio --location={{ pillar['nodes_global']['installation_media'] }} --disk path=/mnt/virtual_images/{{ vm }}.img,size={{ pillar[grains['datacenter'] + '_nodes'][vm]['hdd_size_in_gb'] }},cache=none,bus=virtio --graphics vnc,port={{ pillar[grains['datacenter'] + '_nodes'][vm]['vnc_port'] }},keymap=en-us --accelerate --noautoconsole --initrd-inject=/var/preseed/{{ vm }}/preseed.cfg --extra-args="auto=true priority=critical"
    - unless: virsh list --all | grep -q {{ vm }}
    - order: last
    - require:
      - file: /etc/network/interfaces
      - file: /etc/libvirt/qemu.conf
      - file: /var/preseed/{{ vm }}/preseed.cfg
      - file: /mnt/virtual_images
  {% endif %}
{% endfor %}

It works fine under version 16, but when I upgraded to 17.1 or develop, it dies on me, claiming that it can't find the {{ vm }} variable:

local:

    Data failed to compile:

----------

    Rendering SLS kvm failed, render error: Undefined jinja variable; line 64 in template

---

[...]

    - unless: virsh list --all | grep -q {{ vm }}

    - order: last

    - require:

      - file: /etc/network/interfaces

      - file: /etc/libvirt/qemu.conf

      - file: /var/preseed/{{ vm }}/preseed.cfg    <======================

      - file: /mnt/virtual_images

  {% endif %}

{% endfor %}

---

Any idea why it's suddenly choking? That variable is clearly defined in the for statement. I've double and triple checked that that machine can properly see the pillar its trying to access (Datacenter is a custom grain), it just seems to choke out on that variable.

[Stephen Wood]

Kory,

Maybe it's possible that the pillar item you have is not returning any value? Have you been able to double check this? I believe this would leave the variable vm as undefined in this case, but I'm not positive on that.

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Stephen Wood

www.heystephenwood.com

[Kory Brown]

Thanks for the response!

That was my initial thinking, but I don't think thats the case. I can run pillar.items on the machine, and I get that pillar returned, I also tried replacing the grains['datacenter'] bit with what I expect that to return and it also chokes.

[Kory Brown]

Any thoughts?

[David Anderson]

You seem to be using 'vm' as both a string and a dict - maybe that's
confusing jinja? You can simplify/clarify your state by doing this instead:

{% for vm_id, vm_config in pillar[grains['datacenter'] +
'_nodes'].items() %}
{% if vm_config['generate'] %}
generate-{{ vm_id }}
...
--
Dave

[Kory Brown]

Thanks for the response!

That makes a lot of sense, I implemented that, and I'm still having the same problem. Here's the new Jinja

{% for vm_id, vm_config in pillar[grains['datacenter'] + '_nodes'].items() %}

  {% if vm_config['generate'] %}

generate-{{ vm_id }}:

  file.managed:

    - name: /var/preseed/{{ vm_id }}/preseed.cfg

    - source: salt://kvm/files/preseed

    - user: root

    - group: root

    - mode: 644

    - template: jinja

    - context:

      ip_address: {{ vm_config['ip_address'] }}

      netmask: {{ vm_config['netmask'] }}

      gateway: {{ vm_config['gateway'] }}

      dns_server: {{ vm_config['dns_server'] }}

    - require:

      - file: /etc/network/interfaces

      - file: /etc/libvirt/qemu.conf

  cmd.run:

    - name: virt-install --name={{ vm_id }} --arch=x86_64 --vcpus={{ vm_config['num_cpus'] }} --cpu host --ram={{ vm_config['ram_in_mb'] }} --os-type=linux --os-variant=debianwheezy --hvm --connect=qemu:///system  --network bridge:br0,model=virtio --location={{ pillar['nodes_global']['installation_media'] }} --disk path=/mnt/virtual_images/{{ vm_id }}.img,size={{ vm_config['hdd_size_in_gb'] }},cache=none,bus=virtio --graphics vnc,port={{ vm_config['vnc_port'] }},keymap=en-us --accelerate --noautoconsole --initrd-inject=/var/preseed/{{ vm_id }}/preseed.cfg --extra-args="auto=true priority=critical"

    - unless: virsh list --all | grep -q {{ vm_id }}

    - order: last

    - require:

      - file: /etc/network/interfaces

      - file: /etc/libvirt/qemu.conf

      - file: /var/preseed/{{ vm_id }}/preseed.cfg

      - file: /mnt/virtual_images

  {% endif %}

{% endfor %}

And the Error: 

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/salt/state.py", line 1992, in render_state
    rendered_sls=mods
  File "/usr/local/lib/python2.7/dist-packages/salt/template.py", line 69, in compile_template
    ret = render(input_data, env, sls, **render_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/salt/renderers/jinja.py", line 42, in render
    tmp_data.get('data', 'Unknown render error in jinja renderer')
SaltRenderError: Undefined jinja variable; line 64 in template
---
[...]
    - unless: virsh list --all | grep -q {{ vm_id }}

    - order: last
    - require:
      - file: /etc/network/interfaces
      - file: /etc/libvirt/qemu.conf

      - file: /var/preseed/{{ vm_id }}/preseed.cfg    <======================

      - file: /mnt/virtual_images
{% endif %}
{% endfor %}
---

Here's an example of the pillar its trying to read:

dpdcn_nodes:
  webadmin:
    role:
        - adminserver
        - webserver
    ip_address: xxx.xxx.xxx.xxx
    netmask: 255.255.255.0
    gateway: xxx.xxx.xxx.xxx
    dns_server: xxx.xxx.xxx.xxx
    vnc_port: 5900
    num_cpus: 2
    ram_in_mb: 12288
    hdd_size_in_gb: 20
    generate: true 

Thanks!

[Ran Leibman]

This could be related to https://github.com/saltstack/salt/issues/7625

[Michael March]

This seems to be a pretty nasty bug..  It's killing me too..

https://groups.google.com/forum/#!topic/salt-users/TcJDzpu4m3E

[Kory Brown]

This does appear to be related... Although it looks like the general consensus there was that the data was being cast to a string, which I don't think is my problem (although it would certainly make sense). To add another layer of odd to this problem, I set up a test file with some jinja, and it actually renders properly:

/tmp/test:
  file.managed:
    - source: salt://kvm/files/test.txt

    - user: root
    - group: root
    - mode: 644
    - template: jinja

 

{% set nodes = pillar[grains['datacenter'] + '_nodes'] -%}
{%- for vm, vm_config in pillar[grains['datacenter'] + '_nodes'].items() %}
{%- if vm_config['generate'] == True %}
{{ vm }}
{{ vm_config }}
{%- endif %}
{%- endfor %}

 

 Gives me:

cws-dev1v1
{'dns_server': '129.120.14.122', 'num_cpus': 2, 'vnc_port': 5900, 'hdd_size_in_gb': 20, 'generate': True, 'ram_in_mbs': 12288, 'netmask': '255.255.255.0', 'role': ['webserver'], 'ip_address': '129.120.108.10', 'gateway': '129.120.108.250'}

[Kory Brown]

I've been reading bug report after bug report, its pretty rough. No one seems to be sure if its a Salt or Jinja error. Thankfully this code just automates generating VMs, so at least its not an everyday process for me

[Michael March]

This might be a Jinja error but everyone seems to agree (in the bug reports I've read) that working templates in 16.X stopped working in 17.X.

Some code between those two versions changed that apparently is causing the grief.

[Ran Leibman]

I don't know if it will solve all the issues but for me the following setup looks stable for now:

salt 0.17.1 installed

copy the https://github.com/saltstack/salt/blob/develop/salt/utils/jinja.py file to {{ grains['saltpath'] }}/utils/jinja.py

restart the minion service

you could do it all via salt-ssh easily ....

hope this will ease your pain =)

--
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/1fkRhpdnyq0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Ran Leibman

[Michael March]

When using Vagrant, where would you drop this file?

[Ran Leibman]

you can tell where saltpath is by:

salt-call grains.item saltpath,

for example on RHEL:

root# salt-call grains.item saltpath

local:

  saltpath: /usr/lib/python2.6/site-packages/salt

so it should be in /usr/lib/python2.6/site-packages/salt/utils/jinja.py

[Kent Tenney]

I also am getting odd jinja failures, running current trunk,
'variable not defined' when it clearly is.

I came across the following in salt/renderers/jinja.py

if not tmp_data.get('result', False):
raise SaltRenderError( tmp_data.get('data', 'Unknown render error in
jinja renderer') )

Why report an error if tmp_data['result'] is True?

If the logic is reversed, instead of the undefined error, I get
a report like:

local:
Data failed to compile:
----------

Pillar failed to render with the following messages:
---------- Rendering Primary Top file failed, render error:
base:
'*':
<snip>

Which seems like it could be associated with the recent switch from
env to saltenv

> You received this message because you are subscribed to the Google Groups
> "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an

[Kory Brown]

I gave this a shot, and no such luck for me it looks like :( 

Oddly enough, other templates are rendering just fine, it seems to primarily be jinja in states thats hosing me.

[Kory Brown]

I looked over this bit,  I think thats possible, but since its raising an error different than what we're getting, I don't know if its related?  I have no real idea, but I kind of get the feel that that reads as "If this returns false, but without a result raise Unknown error"

[Kent Tenney]

On Wed, Nov 13, 2013 at 11:27 AM, Kory Brown <blak...@gmail.com> wrote:
> I looked over this bit, I think thats possible, but since its raising an
> error different than what we're getting, I don't know if its related? I
> have no real idea, but I kind of get the feel that that reads as "If this
> returns false, but without a result raise Unknown error"

OK, that could make sense. But.

What about the improved return when the logic is changed? it complains
about 'base', seeming to implicate env -> saltenv

The render() function used to get
env=''
now it gets
saltenv='base'

[Michael March]

I just tried the 0.17.2 tag and this error still seems to be there.

[Colton Myers]

Michael -- have you tried the `develop` branch?  Curious if it's still just not fixed, or if I missed the cherry-pick of the fix for 0.17.2.

--

Colton Myers

--

[Michael March]

It seems to be 'fixed' on the "develop" branch. 

It is fixed that it actually reports real errors of mine.

[Kory Brown]

Sorry for falling off the face of the earth for a bit!

Just wanted to confirm that 0.17.2 has fixed the issue for me.