How to access external pillar data

[Tanoti]

I've been looking at the external pillar system as we have a requirement to get data from a Cassandra database but for now I want to understand the mechanism and how data gets from the external pillar to the minion.

Following an example in the docs, I have added the following to /etc/salt/master:

ext_pillar:
  - cmd_json: "echo {'somevar':'somevalue'}"

What I can't see in the docs is how I get that pillar data to appear for certain minions targeted by a grain for example.

[David Anderson]

ext_pillar modules simply have a function named ext_pillar() that will
be called every time the salt-master compiles the pillar for a minion.
So when a minion asks the salt-master for its pillar data the flow is
like this:

- Minion sends '_pillar' command to Master

- Master compiles the pillar data for that minion id from the tops files
(if any)

- Master then takes that pillar data and calls each ext_pillar in order

- The ext_pillar module accepts the minion_id, pillar as compiled from
the tops, and the ext_pillar configuration for itself.

- The ext_pillar module does whatever, then returns a dict that will be
used to update() the existing pillar.

- After all ext_pillars are called, the master returns the pillar to the
minion.

The cmd_json ext_pillar is actually really simple and easy to see how it
works:
https://github.com/saltstack/salt/blob/develop/salt/pillar/cmd_json.py#L17
--
Dave

> --
> You received this message because you are subscribed to the Google
> Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to salt-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[martin f krafft]

also sprach David Anderson <da...@dubkat.com> [2014-02-13 17:32 +0100]:

> - Master compiles the pillar data for that minion id from the tops
> files (if any)
[…]

> - The ext_pillar module accepts the minion_id, pillar as compiled
> from the tops, and the ext_pillar configuration for itself.

Just to be clear: with "tops" you are referring to the top.sls file(s)
in pillar_roots, not the master_tops files.

I find this nomenclature very confusing and so prefer to be clear…

--
martin | http://madduck.net/ | http://two.sentenc.es/

"i like wagner's music better than anybody's. it is so loud that one
can talk the whole time without other people hearing what one says."
-- oscar wilde

spamtraps: madduc...@madduck.net

[Tanoti]

Thanks for the information. I've done some playing around with a custom external pillar and seen how it gets passed the minion_id and grains, etc and so can work out if it should respond. It seems to me that in order to reduce the amount of queries being triggered just so external pillar knows if there is data in the db to use, I'm going to have to hard-code some logic? I suppose I was rather hoping that I could have used the pillar/top.sls file with standard matching criteria to then specify which external pillars needed to be run.

Something like:

top.sls
=====
base:
  - 'role:WebServer'
  - match: grain
  - ext_pillar:
    - mypillar:
      - arg1: val
      ....

[David Anderson]

You can do that, but it has nothing to do with the pillar top.sls matching.

In your master config, set up your ext_pillar:

ext_pillar:
- my_pillar:
match_tgt: 'role:WebServer'
match_type: 'grain'


In your "my_pillar.py" ext_pillar you would just use the API for the
minion matcher:


def ext_pillar(minion_id, pillar, **kwargs):
match_tgt = kwargs.get('match_tgt', '')
match_type = kwargs.get('match_type', 'grain')
ckminions = salt.utils.minions.CkMinions(__opts__)
if minion_id in ckminions.check_minions(match_tgt, match_type):
....
return my_pillar_data

--
Dave

[Tanoti]

Many thanks Dave, that information should really be in the docs as it is the missing pieces to the puzzle.