Starting salt-minion on Computers Running Network Manager.

[Ian Barton]

I want to deploy salt-minion on several laptops. However, they don't connect automatically connect to the network at boot, but use Network Manager. So putting the init script for salt-minion in an init script fails, because it can't contact the salt-master until the user has logged in. I am assuming that the minion daemon requires root privileges, so putting it in one of the users local scripts won't work.

Any suggestions as to how to work around this problem? The laptops are all running Arch Linux.

Ian.

[Corey Quinn]

On Tue, Dec 20, 2011 at 1:43 AM, Ian Barton <i...@wilkesley.com> wrote:

I want to deploy salt-minion on several laptops. However, they don't connect automatically connect to the network at boot, but use Network Manager.

Is "they" referencing the laptops, or the Salt minions?

So putting the init script for salt-minion in an init script fails, because it can't contact the salt-master until the user has logged in. I am assuming that the minion daemon requires root privileges, so putting it in one of the users local scripts won't work.

Hmm.

I'm not familiar with Arch (or, for that matter, desktop Linux), but isn't there a dependency model in place for most distros that allows for "run this after the network is established?"

Alternately, there are things like anacron that I might consider for this use case.  Also, in the event that a salt minion is started when there isn't a network connection available, doesn't it persist and continue to attempt connections?  If not, we should investigate this; "network isn't working, screw you, bye!" isn't a valid way to address this, as network connections (particularly across a WAN) aren't guaranteed to be available.

-- Corey

[Nathaniel Whiteinge]

As Corey mentioned, the minion will keep trying to connect to the master so it should Just Work (TM) when your network comes up. If this isn't happening we should track down why -- if you manually start the minion in the foreground before you bring up your network you can see output as the minion tries to connect.

[Ian Barton]

Thanks for the info. I'll do a bit more experimenting. So far I just get an error in boot.log that the minion has failed to connect and the minion appears to either time out, or just not try again.

On my home network, we have several laptops, which may only be intermittently connected to my server, so I am interested in how salt can deal with computers that may not be able to contact the master on a regular basis.

Ian.

[Jeff Schroeder]

Seems like a valid use case. If it doesn't support this well, we can
make it so that it does.

--
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com

[Ian Barton]

I have done some more experimentation. I am using 0.94 on Arch. If I try and start salt-minion in the background, in the logs I can see:

Thu Dec 22 09:01:17 2011: :: Starting salt-minion

However, once I log in salt-minion isn't running and I can't find any errors in the logs. If I start the minion manually once I have logged in everything works as expected.

It would be nice to be able to specify a retry interval, so the minion will try and contact the master every n seconds if it fails the first time. Also a max number of retires option might be useful, as there may be times when the computer is connected to a different network to the one running master. In this case having the minion trying to connect forever might not be a good idea.

[Ian Barton]

Forgot to add that starting the minion in debug mode before the network comes up results in:

The master address salt.bantercat.co.uk couldn't be validated, please check that the specified master in the config file is correct..

[Ian Barton]

Thanks to http://www.techytalk.info/start-script-on-network-manager-successfull-connection/ this I have found a solution. In /etc/NetworkManager/dispatcher.d create a script like this:

#!/bin/bash

 

IF=$1

STATUS=$2

 

if [ "$IF" == "wlan0" ]

then

    case "$2" in

        up)

        logger -s "NM Script up triggered"

        /etc/rc.d/salt-minion start

        ;;

        down)

        logger -s "NM Script down triggered"

        /etc/rc.d/salt-minion stop

        ;;

        pre-up)

        logger -s "NM Script pre-up triggered"

        #command3

        ;;

        post-down)

        logger -s "NM Script post-down triggered"

        #command4

        ;;

        *)

       ;;

    esac

fi

The minion daemon will now be stopped/started when you log in/out.

[Thomas S Hatch]

While this is a killer solution, allow me to explain exactly what is going on. Basically if the minion cannot find the ip address of the system referenced as the master then it fails out. so the problem here is that the dns of the master does not resolve until after the network as started. There are a few ways to get around this, and yours is very good!

The other would be to add the entry to the hosts file, this way Salt will automatically connect when the network comes up, the other would be to set the ip addr in the config.

In the end though the fact that this was not logged at error level is the real issue here, I will get that fixed!

- Thomas S Hatch