Saltstack reactor EC2 Jinja challenge

[mdeg...@gmail.com]

Hi,

I'm new to saltstack and trying to get saltstack / saltcloud / ec2 up and running.

My environment:

1. Amazon Linux (current version)

2. Salt-master, salt-minion, salt-cloud 2015.5.5-1 from EPEL 6 repo

Salt works as expected, and I can use salt-cloud to launch minions based on a map file. Here's a sample map file I'm using:

west2-small-internal:

    - test.some.domain.name:

        grains:

            ihost: test

            idomain: some.domain.name

            ifqdn: test.some.domain.name

(Lots of stuff about the provider and profile omitted)

Now I would like to use reactor to flesh out the initial system. I would like to do the following:

1. Set the internal host name to the ifqdn name

2. Bunches of other stuff

In order to do that, I set up a simple reactor.conf file:

# reactor for ec2 bootstrapping

reactor:

    - 'salt/cloud/*/created':

        - '/srv/reactor/hostname.sls'

And the reactor file:

# hostname.sls

hostname_set:

    local.network.system:

        - tgt: {{ data['name'] }}

        - kwarg:

            name: {{ grains['ifqdn'] }}

            hostname: {{ grains['ifqdn'] }}

hostname_rename:

    local.cmd.run:

        - tgt: {{ data['name'] }}

        - kwarg:

            name: hostname {{ grains['ifqdn'] }}

However, the above fails to run. The error is:

SaltRenderError: Jinja variable 'dict object' has no attribute 'ifqdn'

Even though:

sudo salt 'test.some.domain.name' grains.item ifqdn

test.some.domain.name:

    ----------

    ifqdn:

        test.some.domain.name

I'm  obviously doing something wrong, but what?

BTW, if I rewrite the reactor file as a state file:

hostname_set:

    network.system:

        - name: {{ grains['ifqdn'] }}

        - hostname: {{ grains['ifqdn'] }}

hostname_rename:

    cmd.run:

        - name: hostname {{ grains['ifqdn'] }}

        - wait:

            cmd: network

salt sudo 'test.some.domain.name' state.sls hostname

works.

. . . just my confused 2 cents

/mde/

[mdeg...@gmail.com]

Figured it out (I think).

Reactor only has access to the event data (data). So what I ended up doing is writing a reactor file that then called the appropriate state files.

Something like this:

# created_highstate.sls

reactor_hostname:

    local.state.apply:

        - arg:

            - hostname

        - tgt: {{ data['name'] }}

# more stuff to follow

Then create the appropriate state files:

# hostname.sls

hostname_set:

    network.system:

        - name: {{ grains['ifqdn']}}

        - hostname: {{ grains['ifqdn']}}

hostname_rename:

    cmd.run:

        - name: hostname {{ grains['ifqdn'] }}

This seems to work, and obviously makes my environment much more modular. It also has the advantage of not requiring a salt-master restart when I add more components to the initial state. Finally, I can use the state files to get previously created minions up to speed as I evolve my salt-cloud configuration.

Thanks for putting up with the noise.

. . . just my two cents

/mde/

[Oliver Bestwalter]

Hi,

thanks for sharing your discovery process :)

I think problems like the one you encountered are based on the design decision to give all these different kinds of files the .sls extension. I don't think that at the time the decision was made it was clear what a staggering amount of features and subsystem would grow on top of SaltStack, so I understand why it it is that way but it does not mean it has to stay that way :)

The problem with all those sls files is, they ...

a) Can contain completely different languages (e.g. YAML[/Jinja2] or Python ) 

b) can be magically passed completely different sets of data (e.g. pillar/grains/opts, tag/data, ...) 

c) can fulffill completely different roles (e.g. top file, pillar, reactor, etc. ) depending on where they are

I am slowly becoming aware of the shortcomings of this approach especially if you are trying to explain SaltStack to Newcomers, but I did not really start looking around yet if there are solutions already (or if I am just missing something). 

I would like to see this growing system of files to become clearer and more flexible at the same time. It would be clearer for me, if the role of a file would be signified by a fixed name (like 'top') but flexible extensions (to signify the containing language(s)).I guess most file types would only need a fixed name. The extension flexibility only is necessary where different language renderers can be used. E.g. I would like to be abe to have state files with a .py ending if I use the PyObjects renderer for a certain state file. 

For the beginning, maybe an addition to the documentation with an overview matrix of the different file types could bring clarity into that growing set of files and maybe provide insights into how one could structure this growing zoo in a flexible and sensible way. The matrix should contain the involved system and it's purpose (reactor states, etc.), magically available data (pillar, grains, etc. ), default name/location, configuration value to change the default location, ways of overriding data (e.g. passing pillar data with a command line call that clobbers existing data), etc. I don't feel equipped to do that, as I only work intesively with salt for a few months now and only with a tiny subset of the vast Salt ecosystem, but I think this would be a valuable adddition to the documentation.

Cheers

Oliver