Default values and pillar data

[David Seira]

Hi all,

I'm playing with saltstack but I'm stuck in two questions.

I'm trying to create a formula like the sysctl formula to my servers for testing; I want to use a defaults.yaml file where put all the default values if there is no data configured in the pillars.

The defaults.yaml file looks like:

sysctl:

    params:

        -

         name: net.ipv4_forward

         value: 0

        -

         name: kernel.sysrq

         value: 0

The map.jinja is:

{% import_yaml 'sysctl/defaults.yaml' as defaults_settings %}

{% set sysctl_settings = salt['pillar.get'](

    'sysctl',

    default=defaults_settings.sysctl,

    merge=defaults_settings.sysctl

    )

%}

The pillar for a server:

{% if grains['fqdn'] == 'server1' %}

sysctl:

    params:

        -

         name: net.ipv4_forward

         value: 1

{% endif %}

And the state where apply the config:

{% from "sysctl/map.jinja" import sysctl_settings with context %}

{% for param in sysctl_settings.get('params', {}) %}

sysctl_present_{{ param.name }}:

    sysctl.present:

        - name: {{ param.name }}

        - value: {{ param.value }}

{% endfor %}

The first problem is that when I apply the state to the server1 it only applies the data configured in the pillar, it doesn't take the data in the defaults.yaml (in that case the kernel.sysrq option from yaml only because the net.ipv4_forward should be use the pillar one).

How can I achieve this?

The second problem is that if I remove the server from the pillar, because I want to apply only the default values, and apply the state; salt shows the following error:

Rendering SLS 'base:sysctl.param' failed: Jinja variable 'dict object' has no attribute 'sysctl'

I don't know which is the proper way to deal with this.

Could someone give me an idea?

The master version is 2014.7.0 and minion 2014.7.5 both running on ubuntu server 12.04.

Best regards and thanks,

David

[RabidCicada]

I would be better to provide a c+p of all your code in a format that doesn't potentially munge the formatting (like pastebin or whatever).

I see problems potentially with indentation:

http://docs.saltstack.com/en/latest/topics/troubleshooting/yaml_idiosyncrasies.html

I also see a problem with the merge argument for pillar.get.  It should be true/false....not an object:

http://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.pillar.html

I'm guessing a failure in one of the above results in your problem with the Rendering.  I'd use cp.getfile with jinja templating turned on while running the minion in debug mode (salt_minion -l debug) to see what's coming across: http://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.cp.html?highlight=get_file

~Kyle

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

"Ambush!?....I just ambushed you with a cup of coffee"
-CIA agent (RONIN)

[David Seira]

Hi Kyle,

This is the gist: https://gist.github.com/dseira/5e8911ef898674843732

I don't know if it is possible but my idea is to have default values in the defaults.yaml like (not indented, just data):

net.ipv4_forward:0

kernel.sysrq:0

And then, for every server, if it is needed to have the possibility to redefine that values in the pillar:

net.ipv4_forward:1

In that case, when the state is applied the server should have the next params:

net.ipv4_forward:1

kernel.sysrq: 0

Or, if the server is not defined in the pillar, it takes the defaults:

net.ipv4_forward:0

kernel.sysrq:0

This would be nice because the pillar file would be shorter than having to put all the same params for every server in the pillar; all the common params would be in the defaults.yaml.

I don't know if it is enough clear.

Regards,

David

--
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/2ItXYMaFqpg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.

[Colton Myers]

I think you're on the right track. However, you have one too many "endif" tags in your gist:

https://gist.github.com/dseira/5e8911ef898674843732#file-param-sls-L9

Also you left off some lines in your defaults.yaml compared to your original email (you need to define a value as well as a name in each of the params).

Hope that helps!

--

Colton Myers

Platform Engineer, SaltStack

@basepi on Twitter/Github/IRC

[David Seira]

Hi Colton,

Sorry for the delay. 

The close "endif" was a mistake from the copy/paste.

The defaults.yaml was only just an example, I've updated the gist to be more clear. https://gist.github.com/dseira/5e8911ef898674843732

Right now, I have 2 sysctl params in the defaults.yaml (ip_forward and sysrq) and in the pillar only 1 (ip_forward) because for that server I want to apply that specific sysctl param but I also want to apply the sysrq param that it is in the defaults.yaml. With that configuration we I apply the state it only applies the ip_forward param defined in the pillar for that server; it doesn't take into account the sysrq defined in the defaults.yaml (it seems that it doesn't merge both files).

I don't know where is the mistake.

Is it possible?

Thanks so much,

David

[Andrew Hammond]

David, you're misusing the default keyword here. Try doing something like this (from our salt formula):

{% import_yaml 'salt/defaults.yaml' as defaults %}

{% set salt_map = defaults.salt_map %}

{% do salt_map.update(salt['grains.filter_by']({

    'Debian': {

    },

    'RedHat': {

    }

}, merge=salt['pillar.get']('salt_map:lookup'))) %}

The defaults are pulled from your yaml file.

Then updated based on OS type.

And finally overridden by the pillar's salt_map.lookup section.

A

[David Seira]

Hi Andrew,

I've tested your code but with the same result; it only applies the params that are in the pillar, it seems that it doesn't take into account the params in defaults.yml. In fact, I've also tested the sysctl formula that has more or less the same map.jinja structure but it also fails. It takes the defaults.yml params if I remove the params from the pillar.

I understand the idea of the map.jinja (first takes the defaults, then update based on the OS and later on override from the pillar) but I don't understand why  it doesn't work.

This is the gist: https://gist.github.com/dseira/1adfedf28503fa0ae4c6 (in fact, is the same as the sysctl formula from the github repository)

Are there any way to debug this to understand what is happening? Could be something related to the salt-master version?

salt-master: 2014.7.0

salt-minion: 2014.7.5

Regards,

David

[Colton Myers]

I'm pretty sure I had this working when I was testing for my previous message, but I could be wrong. I'm not sure what's going on here, but maybe we should move the discussion to a github issue?

As far as versions go, I don't know of any specific issues which were fixed between 2014.7.0 and 2015.5.2 (current release), but there may very well have been an issue. You might try upgrading.

--

Colton Myers

Platform Engineer, SaltStack

@basepi on Twitter/Github/IRC

[David Seira]

Hi Colton,

I've also tested with the 2015.5.0 version (master and minion) with the same result: it only applies the defaults.yml if I remove the params section in the pillar, if I keep the params section in the pillar it doesn't merge with the defaults.yml. Are there any way to debug to try to understand what is happening?

It would be nice to move to github issue; this is why I created the github issue (https://github.com/saltstack/salt/issues/24043); you can re-open it.

Regards,

David

[Colton Myers]

Ah, right, sorry. I'm involved in so many separate threads and issues I sometimes forget the history of the issue. I'm re-opening that one now.

--

Colton Myers

Platform Engineer, SaltStack

@basepi on Twitter/Github/IRC

[Fernando Correia]

David, did you ever figure out how to achieve this?

I'm facing the same situation: I want to declare default values, while being able to override them.

(See http://stackoverflow.com/questions/32895808/how-to-have-base-values-in-pillars-that-can-be-overridden)

Incidentally, it looks like issue https://github.com/saltstack/salt/issues/24043 was meant to be reopened, but it's still closed.

Regards,

Fernando

[Onno]

David, Fernando,

You can do this with PillarStack see;
https://docs.saltstack.com/en/develop/ref/pillar/all/salt.pillar.stack.html

It can do waaaay more. Play with it and it will soon play a central
role...

The "old" pillar will be evaluated first and then PillarStack, this
opens up a whole lot of possibilities, including your use case (ours is
a CMDB in pillar and role based matching in PillarStack en top.sls)

Regards,
Onno



Fernando Correia schreef op 2015-10-02 16:03:

> --
> You received this message because you are subscribed to the Google
> Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to salt-users+...@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout [1].
>
>
> Links:
> ------
> [1] https://groups.google.com/d/optout