What is the best way to write reusable/generic states?

[Windfinder Support]

Hi,

I am using Salt for one year now to manage our servers. The
environment of several server types is represented by Salt states. It
works great.

But now for the first time I have to setup an environment (by
environment I mean a collection of existing states) on a single
machine multiple times. The environments on that machine differ in
very few parameters. So it would be great to have generic states and
inject certain parameters (like user name etc.) when pushing the state
instead of replicating all states in order to change only one or two
parameters.

My approach is something like a dynamic render context based on pillar
values and defaults (described here
https://github.com/saltstack/salt/issues/4389#issuecomment-16046283).
The problem is that it is currently not a feature of Salt.

Is there another (better) way to do that today with the existing
features of the current Salt version?

Thanks,
Johannes

[Stevearc]

You can do that with pillars and jinja.  It's maybe not the clearest thing to read, and it requires you to rewrite your sls file to be inside of a for loop, but it gets the job done.

/srv/pillar/foobar.sls

foobars:

  foo1:

    text: This is the first foo

  {% if grains.get('numfoos') == 2 %}

  foo2:

    text: This is the second foo

  {% endif %}

/srv/salt/foobar.sls

{% for name, items in pillar.get('foobars', {}).iteritems() %}

.run-{{ name }}:

  cmd.run:

    - name: echo {{ items.get('text') }}

{% endfor %}

[Paul Eipper]

Would be great if the include command could take parameters:

include:
  git:
    - user: myuser
    - param1: myserver

 

[Steven Arcangeli]

I actually asked a similar question a while back and was pointed at the stateconf renderer. It does some cool things besides that as well, like improving namespacing! After trying both approaches, and in light of the recent 'pillar.get' functionality added in 0.14, I think that using pillars is the easier, more scalable, more readable way to go. But they're not mutually exclusive. You can create a stateconf.set that has pillar values as the default:

dinner.sls

# This is the pillar file for dinners

dinner:

  order: spam

  side: spam

dinner.sls

#!stateconf

# This is the state sls file for dinners

.params

  stateconf.set:

    - order: {{ salt['pillar.get']('dinner.order') }}

    - side: {{ salt['pillar.get']('dinner.side') }}

# --- end of state config ---

.cmd:

  cmd.run:

    - name: echo "I'll have the {{ params.order }} with a side of {{ params.side }}"

my_meals.sls

#!stateconf

# This is the state sls file that is actually included in the top.sls

include:

  - dinner

extend:

  dinner::params:

    stateconf.set:

      - side: eggs

--
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/wKvrHLdOXT4/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[in...@windfinder.com]

I have already looked into the stateconf renderer. It looked promising but I had problems with extending an alreday extended state. But that is what I actually need to get my generic states working.

Basically all I need is the option to pass in data that represent my pillar object when pushing a state or the option to explicitly set that pillar variable inside a state. Imagine the following generic state. It describes a simple checkout of a repository. Now I would like to use that same state to do a checkout with several other users. The only thing I can do is to extend that state N times for N user and change all parameters which basically results in N replicated nearly identical new states. 

git/gitcheckout.sls

{% set git_user = salt['pillar.get']('git:user', 'default_user') %}

{% ser git_pass = salt['pillar.get']('git:pass', 'default_pass') %}

repo_checkout

  git.latest:

    - name: https://{{ git_user }}:{{ git_pass }}@domain.com/repo.git

    - target: /var/repos/{{ git_user }}/repo

    - rev: {{ salt['pillar.get']('git:rev', 'master') }}

    - runas: {{ git_user }}

    - require:

      - user: {{ git_user }}

If it would be possible to set the pillar variable inside a state things could be easier. You store all parameters as a configuration set inside a pillar and just reference to it in your state. 

Pillar data

users:

  andy:

    git:

      user: andy

      pass: password

      rev: master

  joe

    git:

      user: joe 

      pass: anotherpassword

      rev: dev

  etc.

git/gitcheckout_andy.sls

{% set pillar_root = salt['pillar.get']('users:andy') %}

include:

  - git.gitcheckout 

git/gitcheckout_joe.sls

{% set pillar_root = salt['pillar.get']('users:joe') %}

include:

  - git.gitcheckout 

Same thing when pushing a state (-p is an imaginary option to inject the root of the pillar structure)

salt * state.sls git.gitcheckout -p users:andy

[Stevearc]

Sadly, I don't think there is a good way to do that with salt at the moment. The only way I know of to make a perfectly reusable state is to write your own custom state. That isn't so hard by itself, but (unless I missed the memo) you can't actually call other states from your states. You have to call other modules. Which means that if your custom state needs to set up some files, you have to replicate a lot of code that's already in the file state.

I think this is actually a problem worth seriously considering. As engineers, we're constantly taught to think modular. Design individual components that can be swapped, plugged, moved around, etc. So that's how we're used to thinking. But with the salt state system, that's actually not possible. You can't build a completely isolated state and use it as you wish; it's always going to be influenced by the global state configuration. This comes out most clearly when trying to get one state to run twice with different parameters. It just doesn't work. So we end up designing around the constraints of salt, which is totally backwards. Python is about readability and ease for the programmer, not the machine.

I don't want to sound like I'm coming down too hard on salt, because I LOVE salt, but this problem in particular is a pain point. I haven't contributed yet, but I'd be willing to put some time into it if we could come up with a better idea of what it should look like.  Ideas so far:

Making states available to custom states. For example, here is one that would allow installing a s3cmd config file on a per-user basis

s3cmd.py

def installed(name, user='root', home='/root', access_key=None, secret_key=None):

    salt['state.pkg.installed']('s3cmd')

    salt['state.file.managed'](home + '/.s3cfg',

        user=user,

        group=user,

        mode='0600',

        template='jinja',

        source='salt://s3cmd/s3cfg.tmpl',

        context={'access_key':access_key,

            'secret_key':secret_key}

        )

Parameterized include. Same example:

s3cmd.sls

#!(user='root', home='/root', access_key=None, secret_key=None) | jinja | yaml

.pkgs:

  pkg.installed:

    - name: s3cmd

.config:

  file.managed:

    - name: {{ home }}/.s3cfg

    - user: {{ user }}

    - group: {{ group }}

    - mode: 0600

    - template: jinja

    - source: salt://s3cmd/s3cfg.tmpl

    - context:

      access_key: {{ access_key }}

      secret_key: {{ secret_key }}

This one would also require a change in the 'include' semantics to allow multiple includes in a single file. In reality, we'd probably have to create another directive, like 'render'. Does anyone else have better ideas? Because I don't completely like either of the two things I suggested.

[Jason Godden]

On 16/04/13 22:19, Stevearc wrote:

I don't want to sound like I'm coming down too hard on salt, because I LOVE salt, but this problem in particular is a pain point. I haven't contributed yet, but I'd be willing to put some time into it if we could come up with a better idea of what it should look like.  Ideas so far:

Making states available to custom states. For example, here is one that would allow installing a s3cmd config file on a per-user basis

s3cmd.py

def installed(name, user='root', home='/root', access_key=None, secret_key=None):

    salt['state.pkg.installed']('s3cmd')

    salt['state.file.managed'](home + '/.s3cfg',

        user=user,

        group=user,

        mode='0600',

        template='jinja',

        source='salt://s3cmd/s3cfg.tmpl',

        context={'access_key':access_key,

            'secret_key':secret_key}

        )

You can use state.single from the state module to achieve the above. The nice thing is you can also just pass in a YAML HEREDOC from your python code too if you want.

http://salt.readthedocs.org/en/latest/ref/modules/all/salt.modules.state.html

[Jason Godden]

On 17/04/13 08:29, Jason Godden wrote:

Making states available to custom states. For example, here is one that would allow installing a s3cmd config file on a per-user basis

s3cmd.py

def installed(name, user='root', home='/root', access_key=None, secret_key=None):

    salt['state.pkg.installed']('s3cmd')

    salt['state.file.managed'](home + '/.s3cfg',

        user=user,

        group=user,

        mode='0600',

        template='jinja',

        source='salt://s3cmd/s3cfg.tmpl',

        context={'access_key':access_key,

            'secret_key':secret_key}

        )

You can use state.single from the state module to achieve the above. The nice thing is you can also just pass in a YAML HEREDOC from your python code too if you want.

http://salt.readthedocs.org/en/latest/ref/modules/all/salt.modules.state.html

Actually you might have issues with this module as state.running is designed to ensure only one state call can be executing on a host at a time so that may not work. You could use a jinja macro in a parameterized fashion and have multiple states there and/or (to the original question for sa...@windfinder.com) you can actually override pillar on the command line now (which I think is what he/she was after):

somestate.sls:

{% mystate(salt['pillar.get']('opt1'), salt['pillar.get']('opt2')) %}

{% macro mystate(opt1, opt2) %}
pkg.installed:
  opt1

file.managed:
  opt2
{% endmacro %}

salt \* state.sls somestate pillar='{"opt1": "foo", "opt2": "baz"}'

Cheers,

Jason