How to debug "Failed to render"?

[Steve Kirkpatrick]

Just getting started with reactors.  I have a simple /etc/salt/master.d/reactor.conf file:

reactor:

  - 'salt/minion/*/start':

    - /srv/reactor/MinionStart.sls

  - 'salt/cloud/*/destroyed':

    - /srv/reactor/SCDecomm.sls

My two reactor state files are also simple and almost identical:

[root@salt-use1a-prod reactor]# cat /srv/reactor/MinionStart.sls

touchfile:

  cmd.cmd.run:

    - tgt: salt-use1a-prod

    - arg:

      - touch /tmp/{{ data['id'] }}.start

[root@salt-use1a-prod reactor]# cat /srv/reactor/SCDecomm.sls

touchfile:

  cmd.cmd.run:

    - tgt: salt-use1a-prod

    - arg:

      - touch /tmp/{{ data['id'] }}.destroyed

The only difference in the files: One touch line ends with .start and the other ends with .destroyed.  Not sure how that affects rendering.

When my minion starts, MinionStart.sls renders properly and runs the command successfully:

[DEBUG   ] Gathering reactors for tag salt/minion/sctest3/start

[DEBUG   ] Compiling reactions for tag salt/minion/sctest3/start

[DEBUG   ] Jinja search path: ['/var/cache/salt/master/files/base']

[DEBUG   ] Rendered data from file: /srv/reactor/MinionStart.sls:

touchfile:

  cmd.cmd.run:

    - tgt: salt-use1a-prod

    - arg:

      - touch /tmp/sctest3.start

[DEBUG   ] Results of YAML rendering:

OrderedDict([('touchfile', OrderedDict([('cmd.cmd.run', [OrderedDict([('tgt', 'salt-use1a-prod')]), OrderedDict([('arg', ['touch /tmp/sctest3.start'])])])]))])

But the reactor state fails to render when destroying the minion:

[DEBUG   ] Gathering reactors for tag salt/cloud/sctest3/destroyed

[DEBUG   ] Compiling reactions for tag salt/cloud/sctest3/destroyed

[DEBUG   ] Jinja search path: ['/var/cache/salt/master/files/base']

[ERROR   ] Failed to render "/srv/reactor/SCDecomm.sls"

How do I troubleshoot this?  Maybe I need to turn up the debug to get more info?  How?

Thanks,

Steve.

[Steve Kirkpatrick]

OK, after further troubleshooting, it looks like the bug may be in Salt itself rather than the simple Reactor SLS file I configured.

I modified /etc/salt/master.d/reactor.conf, only swapping the two SLS files:

reactor:

  - 'salt/minion/*/start':

    - /srv/reactor/SCDecomm.sls

  - 'salt/cloud/*/destroyed':

    - /srv/reactor/MinionStart.sls

Didn't touch the reactor state files themselves.  Restarted the salt-master server in debug mode and deployed and deleted an instance using salt-cloud.

Here are the results:

Deploy:

[DEBUG   ] Gathering reactors for tag salt/minion/sctest3/start

[DEBUG   ] Compiling reactions for tag salt/minion/sctest3/start

[DEBUG   ] Jinja search path: ['/var/cache/salt/master/files/base']

[DEBUG   ] Rendered data from file: /srv/reactor/SCDecomm.sls:

touchfile:

  cmd.cmd.run:

    - tgt: salt-use1a-prod

    - arg:

      - touch /tmp/sctest3.destroyed

[DEBUG   ] Results of YAML rendering:

OrderedDict([('touchfile', OrderedDict([('cmd.cmd.run', [OrderedDict([('tgt', 'salt-use1a-prod')]), OrderedDict([('arg', ['touch /tmp/sctest3.destroyed'])])])]))])

Delete:

[DEBUG   ] Gathering reactors for tag salt/cloud/sctest3/destroyed

[DEBUG   ] Compiling reactions for tag salt/cloud/sctest3/destroyed

[DEBUG   ] Jinja search path: ['/var/cache/salt/master/files/base']

[ERROR   ] Failed to render "/srv/reactor/MinionStart.sls"

So Salt has no problem rendering the SCDecomm.sls file, it seems to have a problem rendering any SLS file when the instance is being destroyed.

Any ideas?

Thanks,

Steve.

[Seth House]

On Mon, Apr 28, 2014 at 3:37 PM, Steve Kirkpatrick <skirkp...@ghx.com> wrote:
> My two reactor state files are also simple and almost identical:

Use the tests/eventlisten.py script to watch the raw event that comes
back from salt-cloud when you destroy a VM. I suspect the Jinja is
failing to render because ``data['id']`` does not exist. The name of
the VM is not (necessarily) the same as the minion ID so salt-cloud
sends ``data['name']`` instead.

[Steve Kirkpatrick]

Thanks for the response Seth.  As suggested, I changed data['id'] to data['name'] in SCDecomm.sls and I get the desired result when I use salt-cloud to destroy an instance.

I forgot to mention in my original post that I tried to start eventlisten.py but ended up with this error:

[root@salt-use1a-prod ~]# python eventlisten.py

Exception AttributeError: "'SaltEvent' object has no attribute 'cpub'" in <bound method SaltEvent.__del__ of <salt.utils.event.SaltEvent object at 0x7f7cc8b02190>> ignored

Traceback (most recent call last):

  File "eventlisten.py", line 102, in <module>

    listen(opts)

  File "eventlisten.py", line 74, in listen

    opts

TypeError: __init__() takes at most 3 arguments (4 given)

I am a Python newbie as well so I don't know what the message is telling me.  Am I missing something?

It would be nice to get this working as I play more with the Reactor.

One more question... where do I find a list of all the available data['xxx'] variables?  So far I have been doing copy/paste development.  Would be nice to know what is available without having to find an example reactor sls file that uses it. ;-)

Thanks,

Steve. 

[Seth House]

On Tue, Apr 29, 2014 at 10:52 AM, Steve Kirkpatrick
<skirkp...@ghx.com> wrote:
> I forgot to mention in my original post that I tried to start eventlisten.py
> but ended up with this error:

Ah, dang. I believe that script was broken during one recent release.
If you download the latest version do you still see the error?

https://raw.githubusercontent.com/saltstack/salt/develop/tests/eventlisten.py

Or (depending which Salt version you're running) possibly this one:

https://raw.githubusercontent.com/saltstack/salt/v2014.1/tests/eventlisten.py

> One more question... where do I find a list of all the available data['xxx']
> variables?

I'm with you on that one. Short answer is we don't have a list yet.
There's an issue to get one added. But in the meantime using
eventlisten.py or the state.event runner is your best bet.

https://github.com/saltstack/salt/issues/5625

http://docs.saltstack.com/en/latest/ref/runners/all/salt.runners.state.html#salt.runners.state.event

[Steve Kirkpatrick]

Thanks Seth.  Got the same error on the first link below but the 2nd (v2014.1) worked like a charm.

Ahhh... now I see why I want to run eventlisten.py.  Lots of useful info. :-)

Appreciate the help.  Now, off to learn the next Salt thing...

[Seth House]

First stab at a list of events:

https://github.com/saltstack/salt/pull/12395

On Tue, Apr 29, 2014 at 6:48 PM, Steve Kirkpatrick <skirkp...@ghx.com> wrote:
>> https://github.com/saltstack/salt/issues/5625