Making a generic top.sls for pillars based on custom grains and loops

[Sebastien Wains]

Hi,

I already made my top.sls as generic as possible for states.

We assign states in our CMDB and retrieve them with a custom grain.

We end up with a ETNIC_ITOP_STATE custom grain with the list of states to apply to the minion:

ETNIC_ITOP_STATE:
  - frontend

My state top.sls looks like this:

base:
# Assigning the roles found in iTop for the minion
{% set ROLES = salt['grains.get']('ETNIC_ITOP_STATE',[]) %}
{% for ROLE in ROLES %}
  'G@ETNIC_ITOP_STATE:{{ ROLE }}':
     - {{ ROLE }}
{% endfor %}

It works great and we don't have to edit top.sls anymore (which was always a great source of merge conflict before :))

Now I'd like to apply the same kind of logic for pillars top.sls.

Somehow, not all states have pillars named after them (eg: /srv/salt/states/projX/ and /srv/salt/pillars/projX/).

To avoid SLS errors, I should check on master if folder /srv/salt/pillars/projX/ exists for a minion with projX state.

I was thinking of something like this:

base:
{% set ROLES = salt['grains.get']('ETNIC_ITOP_STATE',[]) %}
{% for ROLE in ROLES %}
  'G@ETNIC_ITOP_STATE:{{ ROLE }}':
     - {% if salt['file.directory_exists']("/srv/salt/pillars/{{ ROLE }}") %}{{ ROLE }}{% else %}No_pillar_named_after_state{% endif %}
{% endfor %}

Obviously, file.directory_exists is executed on the minion so it doesn't work. No_pillar_named_after_state is applied instead of projX.

Is there a way to execute file.directory_exists on the master when compiling top.sls?

Thanks,
Seb


 

[Mike Freitas]

Hi Sebastien,

Checkout this talk from SaltConf16. This sounds a lot like what you are trying to accomplish. It uses Pillar Stack, among other things.

https://www.youtube.com/watch?v=F3K706JJ2EI&list=PL9svBjLDUl_-sVwcRliUQ-VGDb2qvwpx_

Mike

[Sebastien Wains]

Thanks, I'll check it out.

[Sebastien Wains]

I confirm the ext_pillar "stack" is the solution to my problem. Doc is here: https://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.stack.html

Stack will ignore any yaml file without producing any error when running pillar.data:

[DEBUG   ] YAML: basedir=/srv/salt/pillars, path=/srv/salt/pillars/core_users/init.sls
[WARNING ] Ignoring pillar stack template "tech_proxy/init.sls": can't find from root dir "/srv/salt/pillars"
[DEBUG   ] YAML: basedir=/srv/salt/pillars, path=/srv/salt/pillars/proj_tracability/init.sls

Thanks!

[Mike Freitas]

Awesome. I'm glad that was helpful. I have started to use pillar_stack a little bit myself so if you run into any issues I will help if I can. I will say that it was pretty straight forward for my use case so i'm sure you will do just fine. Good luck.

Mike