Grains module, filter_by for more than one grain

[Alfredo Palhares]

Hello salt addicts,

I have the following use case

set clipboard=unnamedo

# vi: set ft=sls :
{%
set map = salt["grains.filter_by"]({
"Debian": {
"amd64": {
"package": "my_formula_amd64.deb",
"md5": "3c02ef5f9c4aa28eacdc68b230277af3"
},
"i686": {
"package": "my_formula.deb",
"md5": "efd0792fba61b6c416321c6319a77f99"
},
"i386": {
"package": "my_formula.deb",
"md5": "c011c674fce6dd8880eee31e052ae62b"
}
}
[... other distro families here ..]
}, merge=salt["pillar.get"]("my_formula:lookup"))
%}

Thing is, this only filters by one grain[1] at the time. I want to
filter by osarch too.
What would be a "best practise" method to do this? Nest another
grains.filter_by ? Or just import in on the state with
map[grains["osarch"]] ?

Any input is welcome!

[1] https://github.com/saltstack/salt/blob/23fbd41/salt/modules/grains.py#L349


--
Regards,
Alfredo Palhares

[Seth House]

On Mon, Aug 18, 2014 at 11:17 AM, Alfredo Palhares
<maste...@masterkorp.net> wrote:
> I want to
> filter by osarch too.
> What would be a "best practise" method to do this? Nest another
> grains.filter_by ?

For this use-case nesting a second filter_by call fits perfectly with
the intent of the filter_by function -- grabbing a sub-dictionary of a
dictionary. Plus your lookup table is still plenty legible and
logical.

Oh, and if you find you have lots of duplicate entries between archs
the ``defaults`` argument to filter_by may be helpful.

[Alfredo Palhares]

Hello Seth,

I tried this.

{%
set map = salt["grains.filter_by"]({
salt["grains.filter_by"]({
"Debian": {
"amd64": {
"package": "my_formula_amd64.deb",
"md5": "3c02ef5f9c4aa28eacdc68b230277af3"
},
"i686": {
"package": "my_formula.deb",
"md5": "efd0792fba61b6c416321c6319a77f99"
},
"i386": {
"package": "my_formula.deb",
"md5": "c011c674fce6dd8880eee31e052ae62b"
}

},

[... other distro families here ..]

}, grain="os_family", merge=salt["pillar.get"]("my_formula:lookup)),
grain="osarch" )
%}

It results in a error like.

```
Jinja error: 'NoneType' object has no attribute 'get'
```

Some research[1][2] and it appears to be that error does not return a
dictionary wich obsiously does[3] Also if i get a map that was
previosly filterred by it does error too.

Any input welcome.


[1] http://stackoverflow.com/questions/13884815/python-gae-attribute-error-webapp2
[2] http://stackoverflow.com/questions/14657901/gae-error-attributeerror-nonetype-object-has-no-attribute-image
[3] https://github.com/saltstack/salt/blob/27782ee6cc00c0ef1361fba306f17d8a01472cda/salt/modules/grains.py#L349


--
Regards,
Alfredo Palhares

[Seth House]

Teasing out the right syntax for that is rough. This is what I used to
test. Note, if there is not a match in the outer dictionary it will
return None and throw an error on the second filter_by call. If you're
running this on a system that may not match you might have to break it
up into two separate statements or make use of the ``default`` kwarg.

{%
set map = salt["grains.filter_by"](
salt["grains.filter_by"]({
"Debian": {
"amd64": {
"package": "my_formula_amd64.deb",
"md5": "3c02ef5f9c4aa28eacdc68b230277af3",
},
"i686": {
"package": "my_formula.deb",
"md5": "efd0792fba61b6c416321c6319a77f99",
},
"i386": {
"package": "my_formula.deb",
"md5": "c011c674fce6dd8880eee31e052ae62b",
}

},
"MacOS": {
"x86_64": {

"package": "my_formula_amd64.deb",
"md5": "3c02ef5f9c4aa28eacdc68b230277af3",

},
},
},
grain="os_family",
merge=salt["pillar.get"]("my_formula:lookup", {})),
grain="osarch")
%}

echo_map:
cmd.run:
- name: |
echo 'Map: {{ map | json() }}'

[Alfredo Palhares]

Hello,

Excerpts from Seth House's message of 2014-08-20 06:24:17 +0200:

> Teasing out the right syntax for that is rough. This is what I used to
> test. Note, if there is not a match in the outer dictionary it will
> return None and throw an error on the second filter_by call. If you're
> running this on a system that may not match you might have to break it
> up into two separate statements or make use of the ``default`` kwarg.

The map just gets into None value, obviously the os_family is Debian and
osarch is either on of those so I dont get it.
Ĩ think this is a bug now.
Any more ideas ? If not I will open an issue on github.

--
Regards,
Alfredo Palhares

[Seth House]

On Wed, Aug 20, 2014 at 5:57 AM, Alfredo Palhares
<maste...@masterkorp.net> wrote:
> Any more ideas ? If not I will open an issue on github.

Go through each step individually to find the source of the issue.
``filter_by`` is just a shorthand for doing two or three operations in
one line. Use the template below and change the variable that is
getting echoed to see the result of each step.

{% set lookup_table = {

"Debian": {
"amd64": {
"package": "my_formula_amd64.deb",
"md5": "3c02ef5f9c4aa28eacdc68b230277af3",
},
"i686": {
"package": "my_formula.deb",
"md5": "efd0792fba61b6c416321c6319a77f99",
},
"i386": {
"package": "my_formula.deb",
"md5": "c011c674fce6dd8880eee31e052ae62b",
}
},
"MacOS": {
"x86_64": {
"package": "my_formula_amd64.deb",
"md5": "3c02ef5f9c4aa28eacdc68b230277af3",
},
},

} %}

{% set os_map = lookup_table.get(grains.os_family, {}) %}
{% set arch_map = os_map.get(grains.osarch, {}) %}

{% set map = {} %}
{% do map.update(arch_map) %}
{% do map.update(salt['pillar.get']('my_formula:lookup', {})) %}