Typo in top.sls causes blank output for minion with no indication as to what the error is

[Peter Waller]

Hi All,

Given a top.sls file containing "blah:\n", the salt command reports:

$ salt '*' states.hightstate

<minion id>:

$

In other words, the output is blank. The exit state of salt is 0, indicating success.

The minion shows no useful information unless debugging information is turned on. The only indication that anything is wrong is in the minion log, and with the default log level selected this is it:

[WARNING ] TypeError encountered executing state.highstate: 'NoneType' object is not iterable. See debug log for more info.  Possibly a missing arguments issue:  ArgSpec(args=['test', 'queue'], varargs=None, keywords='kwargs', defaults=(None, False))

In order to find the error the minion has to be stopped and put into debug mode.

My concerns are as follows:

* A simple mistake (having a blank declaration anywhere in my file) caused salt to fail in a non-intuitive way

* The failure was almost silent by default, at the very least it is not helpful to the user to figure out what they did wrong

* The "salt '*'" command invocation returned a 0 exit status and was blank.

* I couldn't track the error down without stopping the minion and putting it into a different log level.

* Even with the maximum amount of debugging information it wasn't apparent exactly what my mistake was. I just happened to find it since my top.sls was small enough for my error to be apparent.

Thanks,

- Peter

Stack trace with --log-level DEBUG and versions follow.

$ salt --versions-report
           Salt: 0.17.2
         Python: 2.7.5+ (default, Sep 19 2013, 13:48:49)
         Jinja2: 2.7
       M2Crypto: 0.21.1
 msgpack-python: 0.3.0
   msgpack-pure: Not Installed
       pycrypto: 2.6
         PyYAML: 3.10
          PyZMQ: 13.1.0
            ZMQ: 3.2.3

[DEBUG   ] Results of YAML rendering:
OrderedDict([('test-something', None)])
[WARNING ] TypeError encountered executing state.highstate: 'NoneType' object is not iterable. See debug log for more info.  Possibly a missing arguments issue:  ArgSpec(args=['test', 'queue'], varargs=None, keywords='kwargs', defaults=(None, False))
[DEBUG   ] TypeError intercepted: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 691, in _thread_return
    return_data = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/modules/state.py", line 261, in highstate
    force=kwargs.get('force', False)
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 2332, in call_highstate
    high, errors = self.render_highstate(matches)
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 2232, in render_highstate
    state, errors = self.render_state(sls, env, mods, matches)
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 2071, in render_state
    self._handle_iorder(state)
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 2082, in _handle_iorder
    for s_dec in state[name]:
TypeError: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 691, in _thread_return
    return_data = func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/modules/state.py", line 261, in highstate
    force=kwargs.get('force', False)
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 2332, in call_highstate
    high, errors = self.render_highstate(matches)
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 2232, in render_highstate
    state, errors = self.render_state(sls, env, mods, matches)
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 2071, in render_state
    self._handle_iorder(state)
  File "/usr/lib/python2.7/dist-packages/salt/state.py", line 2082, in _handle_iorder
    for s_dec in state[name]:
TypeError: 'NoneType' object is not iterable

[Ethan Erchinger]

A few debugging tips are to use something like 'salt-call -l debug state.show_top' or state.highstate test=True and see if you can find the issue.  The former should show your rendered top file.  It also isolates the call to the minion in question, and doesn't require the need to stop, reconfigure, run, debug, fix, stop, reconfigure, run.

I believe there is an issue filed already on the non-zero exit status of salt on failures such as the one you describe, +1 that to vote it up in priority.  #7876 is close.

While not directly related to your issue, I like to essentially never touch my top.sls file, and use the following code to auto-assign states to nodes based on the idea of a "cluster" of nodes.  As such, I only ever edit /srv/salt/cluster/<cluster_name>/init.sls and the relevant included files.  For me, it feels like it introduces a level of stability.

https://gist.github.com/erchn/7629017

[Peter Waller]

Thanks Ethan.

I've reported https://github.com/saltstack/salt/issues/8799 to track the blank output issue.

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.