Need help with pillar problem...

[Rainer Krienke]

Hello,

I have a salt pillar problem and I do not see what I am doing wrong. Hope anyone can give me a hint

I defined a pillar "diskusage.sls" (included in /srv/pillar/base/top.sls ):

diskusage:
  default:
      '/': 90
      '/var':  90
      '/boot': 90

  myhost:
      '/': 50
      '/var':  80
      '/boot': 80

      '/mnt': 30

Then I defined a "beacons.sls" file with jinja code to check how full the relevant minions filesystems are. Via jinja I want to either insert filesystems for the current host if the hostname is found in the diskusage pillar or to use the defined default values from the pillar. My beacons.sls now looks like this:

{% set fsdefault=salt['pillar.get']('diskusage:default') %}
beacons:
  diskusage:
  {% for fs, percent in pillar.get('diskusage:' ~ grains['host'], default=fsdefault).items()  %}
    - {{ fs }}: {{ percent }}
  {% endfor %}
    - onchangeonly: True
    - interval: 30  # seconds

The problem is that a call

# salt 'anyhost*' state.apply

results in an error:

    Data failed to compile:
----------
    Pillar failed to render with the following messages:
----------
    Rendering SLS 'beacons' failed. Please see master log for details.


In the masters log I see:

2018-07-31 14:15:56,717 [salt.utils.templates:180 ][ERROR   ][20621] Rendering exception occurred
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/utils/templates.py", line 169, in render_tmpl
    output = render_str(tmplstr, context, tmplpath)
  File "/usr/lib/python2.7/site-packages/salt/utils/templates.py", line 390, in render_jinja_tmpl
    buf=tmplstr)
SaltRenderError: Jinja variable 'unicode object' has no attribute 'items'
2018-07-31 14:15:56,718 [salt.pillar      :732 ][CRITICAL][20621] Rendering SLS 'beacons' failed, render error:
Jinja variable 'unicode object' has no attribute 'items'
2018-07-31 14:15:56,749 [salt.pillar      :1005][CRITICAL][20621] Pillar render error: Rendering SLS 'beacons' failed. Please see master log for details.


However a call on salt master:

# salt-call slsutil.renderer  /srv/pillar/base/beacons.sls 'jinja'

works fine and returns:

local:
    ----------
    beacons:
        ----------
        diskusage:
            |_
              ----------
              /boot:
                  90
            |_
              ----------
              /var:
                  90
            |_
              ----------
              /:
                  90
            |_
              ----------
              onchangeonly:
                  True
            |_
              ----------
              interval:
                  30

Now I really have no idea whats going wrong here. Can anyone help?

Thanks

Rainer

[federic...@upsight.com]

Hi,

Your problem is probably in this line:

{% for fs, percent in pillar.get('diskusage:' ~ grains['host'], default=fsdefault).items()  %}

You are setting a default string for a dictionary pillar, and maybe you meant to set the default host instead? If so, that would look like this:

{% for fs, percent in pillar.get('diskusage:' ~ grains.get('host', 'fsdefault')).items()  %}

Also make sure "fsdefault" exists in your pillars for the hostname, since it looks like you called it "default":

diskusage:
  default:
      '/': 90
      '/var':  90
      '/boot': 90

On master it probably works because you have the correct host for the master in pillars, and for the rest the default is wrong. It's hard to debug if we don't have the actual pillar values though.

Regards,

[Rainer Krienke]

Hello ,

well what I want to do is to check if filesystems get full on minions. To make this more flexible I created the pillar named "diskusage" (the contents of this pillar is posted above) that contains a default for all minions except for those minions where I want to check eg more or other  filesystems with different limits than the default. For each such host I could create another entry in the diskusage pillar for this host  like shown for the host "myhost".

So the default is the the pillar entry named "default" that is to be used if I do not find the minions hostname in the pillar.

Thats why my first action in beacons.sls is to determine the default dictionary entry (I hope:- I did)) by this jinja code:

{% set fsdefault=salt['pillar.get']('diskusage:default') %}

and then I simply iterate over the contents of the host-entry in the diskusage dictonary I retrieve via

pillar.get('diskusage:' ~ grains['host'] ...)

If pillar.get does not return any value because the host is not in the pillar  I want to use the default value I stored before in fsdefault (a dictionary). This is what I intended to do but something is probably wrong but I do not know what.

On the commandline the command below works fine and this is what should be in the variable "fsdata" like described above.

# salt-call pillar.get 'diskusage:default'

local:
    ----------
    /:
        90
    /boot:
        90
    /var:
        90

So after all I think in the statement:

{% for fs, percent in pillar.get('diskusage:' ~ grains['host'], default=fsdefault).items()  %}

I do not set fsdata to a string but to a dictionary returned by pillar.get, don't I?

Thanks

Rainer

[federic...@upsight.com]

Ah sorry, I missed this line:

{% set fsdefault=salt['pillar.get']('diskusage:default') %}

You can do that too, but I think this is how you set a dict in jinja:

{% set fsdefault=salt['pillar.get']('diskusage:default') | dictsort() %}

Or you could save a few lines, and set the default host in in the grains.get('host') like this, and not use fsdefault variable at all:

{% for fs, percent in pillar.get('diskusage:' ~ grains.get('host', 'default')).items()  %}

On Tuesday, July 31, 2018 at 9:39:13 AM UTC-3, Rainer Krienke wrote:

[Thomas Phipps]

Rainer,

The issue is you are trying to pull pillar from with in pillar. that won't work. kind of a chicken and egg problem. 

A better way is to load the diskusage.sls as a yml file using import_yaml

such as 

{% import_yaml '/srv/pillar/base/diskusage.sls' as diskusage %}

{% set fsdefault=diskusage['diskusage']['default'] %}

beacons:
  diskusage:
  {% for fs, percent in diskusage['diskusage'].get(grains['host'], default=fsdefault).items()  %}

    - {{ fs }}: {{ percent }}
  {% endfor %}
    - onchangeonly: True
    - interval: 30  # seconds

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/43bc42cd-a66e-43a3-a00e-fc8624b9ed22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rainer Krienke]

Hello,

I tried you code version but unfortunately the error message remains the same.

Thanks

Rainer

[Rainer Krienke]

Hello Thomas,

thanks for your explanation and the code. Using your code version I get another error message indicating that the imported diskusage['diskusage'] is not a dictionary. The error message I see in the masters log is the following:

2018-08-01 08:58:18,003 [salt.pillar      :732 ][CRITICAL][23466] Rendering SLS 'beacons' failed, render error:
Jinja error: get() takes no keyword arguments

Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/salt/utils/templates.py", line 380, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 989, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<template>", line 15, in top-level template code
TypeError: get() takes no keyword arguments

; line 15

---
[...]
# check diskusage beacon
# handled in: /srv/reactor/base/diskusage.sls
#
beacons:
  diskusage:
  {% for fs, percent in diskusage['diskusage'].get(grains['host'], default=fsdefault).items()  %}    <======================

    - {{ fs }}: {{ percent }}
  {% endfor %}
    - onchangeonly: True
    - interval: 30  # seconds

Thanks

Rainer

[Rainer Krienke]

Because I did not find a solution for my pillar problem I thought I try to drop the pillar and write {% if .. else .. endif %} jinja statements into the beacons.sls file to distinguish host specific settings like shown below:

beacons:                            # file: /srv/pillar/base/beacons.sls
  diskusage:
{% if grains['host'] == 'myhost1' %}
    - '/': 50%                      # specific for host myhost1
{% else %}
    - '/': 90%                      # the default for any other host
{% endif %}
    - '/var': 90%
    - '/boot': 90%
    - onchangeonly: True
    - interval: 3600  # seconds

Well even this code fails when running salt:

salt '*' state.apply.:

myhost1.mydomain.de

    Data failed to compile:
----------
    Pillar failed to render with the following messages:
----------
    Rendering SLS 'beacons' failed. Please see master log for details.

The master logs do not contain any helpful infos. If I remove the jina code from this beacons.sls file everything works just fine.

Could it be that Jinja code accessing grains is generally not allowed in beacons?

Thanks

Rainer

[Rainer Krienke]

Forget about my last post.

I forgot to run a salt '*' saltutil.refresh_pillar before trying the salt '*' state.apply. After the refresh jinja code directly in the beacons.sls file works as expected, however the initial pillar based versions still do not work.

Sorry

Rainer

[Thomas Phipps]

sorry about that. forgot about that little thing. basically don't pass default=fsdefault. just pass the variable that is meant to be default like the following line. 

 {% for fs, percent in diskusage['diskusage'].get(grains['host'], fsdefault).items()  %}  

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/salt-users/7f7d307d-9826-4378-a644-4064c37b2715%40googlegroups.com.

[Rainer Krienke]

Thank you very much. I removed the named parameter ("default=") and now it works. Great.

Now I still have one wish: to understand the two things that went wrong:

1. Why was the named parameter "default=something" not allowed in the for loop? Python generally allows named parameters to my knowledge and the call is basically something like dict.get(grains['host'], fsdefault) which looks like python.

2. Can you describe what was causing what you called the chicken and egg problem with pillar diskusage. After all  this pillar is statically defined, globally included in top.sls and so I do not understand why accessing it (without direct import in beacons) didn't work?

Would be great if you could try to explain this to me.

Thanks Rainer

[Ethan Erchinger]

For #1, the fsdefault has to be passed that way because .get() takes an optional positional arg, not an optional kwarg.