How to get the IP of some minions based on grains?

[Nicos Karalis]

I have a system that has many web backends (rails apps mostly) and one load balancer (nginx). Because of some scale issues we are trying to use salt to handle all our architecture.

I want to get all my minions ip with a certain role listed on a template. I'm using publish.publish but it does return me only the fqdn and the fqdn is not bind to a ip inside the hosts file.

I'm using a formula that install nginx on all machines and configure based on the "roles" attribute set as a grain. like this:

balancer and backend minion

/etc/salt/minion:

roles:
  - balancer
  - backend

backend minion
/etc/salt/minion:

roles:
  - backend

top.sls

'*':
  - nginx

'roles:balancer':
  - match: grain
  - nginx.balancer

'roles:backend':
  - match: grain
  - nginx.backend

nginx.balancer and nginx.backend are simple sls files that just send the respective template to the minion. nginx.backend works fine but I'm having some problems inside nginx.balancer.

Basically I render the template below and store on /etc/nginx/sites-enabled/balancer.conf

upstream backends {
  server 192.168.33.11;
  {{ # other servers need to be here }}
}

server {
  listen      80 default_server;
  listen [::]:80 ipv6only=on;
  server_name {{ grains['domain'] }};
 
  location / {
    proxy_pass http://backends;
  }
}

The problem is I can't manage to get the ips of the desired minions. I've tried using publish.publish and mine.get but none of them worked. With this SO question I've tried this solutions:

This one returns an empty like if vars host and hostinfo are empty arrays. 

{% for host, hostinfo in salt['mine.get']('*', 'network.interfaces').items() %}
  server {{ hostinfo['eth0']['inet'][0]['address'] if hostinfo['eth0'].has_key('inet') else hostinfo['br0']['inet'][0]['address'] }};
{% endfor %}

This one does find my minions but returns error telling me that Unable to manage file: Jinja variable 'str object' has no attribute 'ip' meaning host is a string, if I use host instead I get the fqdn of that minion and the config doesn't work because nginx can't resolve that hostname.

{% for host in salt['publish.publish']('roles:backend', 'network.ip_addrs', 'eth0', 'grain') %}
  server {{ host.ip }};
{% endfor %}

Also on the second way I want to find all backends that are not balancers so I tried using this matcher "roles:backend and not roles:load_balancer" but it doesn't return anything.

I dont want to put minions IP on a grain or pillar, they should be dynamically retrieved.

Can anyone help me?

[Florian Ermisch]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Nicos,

first: use `grains.items` to check if thoses custom grains ate actually set. I would guess you're missing a "grains:" (and a bunch of spaces below) in you config files.

2nd: try the compound matcher pattern/expression 'G@roles:backend and not G@roles:balancer'.

Regards, Florian
(Not yet fully awake)

Am 2. August 2015 05:04:55 MESZ, schrieb Nicos Karalis <nicosk...@me.com>:
>I have a system that has many web backends (rails apps mostly) and one
>load
>balancer (nginx). Because of some scale issues we are trying to use
>salt to
>handle all our architecture.
>
>I want to get all my minions ip with a certain role listed on a
>template.

>I'm using *publish.publish* but it does return me only the fqdn and the

>
>fqdn is not bind to a ip inside the hosts file.
>
>I'm using a formula that install nginx on all machines and configure
>based
>on the "roles" attribute set as a grain. like this:
>
>balancer and backend minion
>/etc/salt/minion:
>roles:
> - balancer
> - backend
>
>backend minion
>/etc/salt/minion:
>roles:
> - backend
>
>top.sls
>'*':
> - nginx
>
>'roles:balancer':
> - match: grain
> - nginx.balancer
>
>'roles:backend':
> - match: grain
> - nginx.backend
>

>*nginx.balancer* and *nginx.**backend* are simple sls files that just
>send
>the respective template to the minion. *nginx.backend* works fine but
>I'm
>having some problems inside *nginx.balancer*.

>
>Basically I render the template below and store on
>/etc/nginx/sites-enabled/balancer.conf
>
>upstream backends {
> server 192.168.33.11;
> {{ # other servers need to be here }}
>}
>
>server {
> listen 80 default_server;
> listen [::]:80 ipv6only=on;
> server_name {{ grains['domain'] }};
>
> location / {
> proxy_pass http://backends;
> }
>}
>
>The problem is I can't manage to get the ips of the desired minions.
>I've

>tried using *publish.publish* and *mine.get* but none of them worked.
>With
>this SO question <http://stackoverflow.com/q/17158665/839049> I've
>tried
>this solutions:
>
>This one returns an empty like if vars *host* and *hostinfo* are empty

>arrays.
>{% for host, hostinfo in salt['mine.get']('*',
>'network.interfaces').items()
>%}
> server {{ hostinfo['eth0']['inet'][0]['address'] if hostinfo['eth0'].
>has_key('inet') else hostinfo['br0']['inet'][0]['address'] }};
>{% endfor %}
>
>This one does find my minions but returns error telling me that Unable
>to
>manage file: Jinja variable 'str object' has no attribute 'ip' meaning
>host

>is a string, if I use *host* instead I get the fqdn of that minion and

>the
>config doesn't work because nginx can't resolve that hostname.
>{% for host in salt['publish.publish']('roles:backend',
>'network.ip_addrs',
>'eth0', 'grain') %}
> server {{ host.ip }};
>{% endfor %}
>
>
>Also on the second way I want to find all backends that are not
>balancers
>so I tried using this matcher "roles:backend and not
>roles:load_balancer"
>but it doesn't return anything.
>I dont want to put minions IP on a grain or pillar, they should be
>dynamically retrieved.
>
>Can anyone help me?
>

>--
>You received this message because you are subscribed to the Google
>Groups "Salt-users" group.
>To unsubscribe from this group and stop receiving emails from it, send
>an email to salt-users+...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.
-----BEGIN PGP SIGNATURE-----
Version: APG v1.1.1

iQFTBAEBCAA9BQJVvbDbNhxGbG9yaWFuIEVybWlzY2ggPGZsb3JpYW4uZXJtaXNj
aEBhbHVtbmkudHUtYmVybGluLmRlPgAKCRAu8tzCHoBI/XVLB/0TbEfY3LSXkwQ5
msjHQEh3cG0JGwTWcLhk7+fLR6pdlnJXfNu+M+GxxlG7ul3pJXb4eeuxjKozsgXs
CTGN6Gb5ommKVKJ1PzTbGGByl3wq2k3jklBes3Woxyz6wpV+7VCR2A0f2noOpOUv
LHm9YfI2KLx+ZE/rcU/llCBD8bfvqRk9S9qL3Y3clTQoz7jhhTJ1jzpnyP1UjIW9
Tv54JDOhyhbPEUjKIuSupa26TxqXsREigJUKVoehNp3lDb3PE7jlFtpS0tj3Yv5f
LyKZWT5sWm3IQckjwg7WJejIL7hp41voKNkVvJEHOGtInl0rZgnOxIQUG0gevaH5
OqLP+zfm
=N/6C
-----END PGP SIGNATURE-----

[Nicos Karalis]

first: use `grains.items` to check if thoses custom grains ate actually set. I would guess you're missing a "grains:" (and a bunch of spaces below) in you config files. 

I ran grains.items and I can see that the custom grains are set to all minions 

 2nd: try the compound matcher pattern/expression 'G@roles:backend and not G@roles:balancer'. 

I altered the query but still not working. The expression below does not return anything. I have 1 minion with role backend only and one with backend and balancer. The first should be queried, but not the second. Neither of them are returned.

{% for server, addrs in salt['mine.get']('G@roles:backend and not G@roles:balancer', 'network.ip_addrs', 'compound').items() %}

[Nicos Karalis]

Please ignore the compound matcher problem. I had a typo on my minions and everything is working as it should.

Now the only problem remaining is to get the ip of the other minions

[Florian Ermisch]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Nicos,

did you try to manually send & receive data to/from the salt-mine?
Maybe you got a typo somewhere in your mine_funcs…

Regards, Florian

-----BEGIN PGP SIGNATURE-----
Version: APG v1.1.1

iQFTBAEBCAA9BQJVvvnoNhxGbG9yaWFuIEVybWlzY2ggPGZsb3JpYW4uZXJtaXNj
aEBhbHVtbmkudHUtYmVybGluLmRlPgAKCRAu8tzCHoBI/bAtB/oCNj8owtsnm8OH
sBAphx/ysC13TlTr4wu3LV4zewvt+7FDjTVdczmlKvjfi46VuSdU+XScZPjzPeQb
T9x9UTufL9g10VLg2F1qdKKwWW5GMWEH8FkE15xWvHRLTQePVOToIt/6N4CKcvRP
Xg2PWDjeTkg+/qOl9jlstVYL5H2WxRl7fIiw1KtiGPmZPepQXLeNRX881IjykU1j
kDWTeWypNPMGSQ593qfX70FloPira898fiS1JgPSyfeWrWe9vuyIUy0LNNUHGnrP
6YYx4UJGmuWdHrHBMu3hJy68G7d1oZIMyp+O6jNhqiYsGCxqwBIVYsQ7NGMgJkAH
bSloAW5J
=g2dS
-----END PGP SIGNATURE-----