Timeout for running commands via cmd.run* ?

[Erik Johnson]

In the module I am writing, I am trying to run a shell command (using cmd.run) which takes a considerable amount of time to run (1 min or more). However, when this command (rvm pkg install openssl) is run, within a few seconds the salt command returns an empty hash and nothing further is logged. I added extra log.error entries after the call to cmd.run but they don't show up in the logs, so it appears that the command is dying within cmd.run. However, there is no timeout from what I can see in source for cmd.py. This behavior does not seem limited to my module either, as I can replicate it by running the following (both as root and using the new runas feature):

sudo salt '*' cmd.run 'tail -f /var/log/dmesg'

So, is there some timeout that I am running into here?

[Thomas S Hatch]

Yes, salt detaches from the execution, so it does finish on the minion, but the timeout needs to be raised with the -t option:

salt -t 60 '*' cmd.run 'command'

Also keep in mind, that doing a tail -f will NEVER return, since the command blocks and the executed command never completes.

[Erik Johnson]

Cool, thanks for the info. And I did know that tail -f would not return, I was just using it to test the timeout I was noticing in my module. I needed a command that would run for several seconds.
--

-Erik

"For me, it is far better to grasp the universe as it really is than to persist in delusion, however satisfying and reassuring." --Carl Sagan

[Thomas S Hatch]

Ahh, makes sense! Usually I just do a:

salt \* cmd.run 'sleep 500'

The new Job interface that is going to be included will make job tracking much easier and more flexible. If you are running minions out of git you can run the command:

salt \* saltutil.running

to get a summary of all running jobs

[Erik Johnson]

On Sat, Jan 28, 2012 at 3:05 PM, Thomas S Hatch <that...@gmail.com> wrote:

Ahh, makes sense! Usually I just do a:

salt \* cmd.run 'sleep 500'

The new Job interface that is going to be included will make job tracking much easier and more flexible. If you are running minions out of git you can run the command:

salt \* saltutil.running

to get a summary of all running jobs

Nice! Will take a look.

An install of ruby within RVM, as well as installation of dependencies needed to build modules will take some time to complete. Unless I can find a solution with this job interface, I will need to include a warning that the initial highstate needs to be run with -t 600 or something.
 

--

-Erik

"For me, it is far better to grasp the universe as it really is than to persist in delusion, however satisfying and reassuring."  -- Carl Sagan

"Take the risk of thinking for yourself. Much more happiness, truth, beauty and wisdom will come to you that way."  -- Christopher Hitchens

[Thomas S Hatch]

Nice! Will take a look.

An install of ruby within RVM, as well as installation of dependencies needed to build modules will take some time to complete. Unless I can find a solution with this job interface, I will need to include a warning that the initial highstate needs to be run with -t 600 or something.
   

Yes, remember that you can execute the highstate on the minon with salt-call state.highstate to watch it, and you can look up returns with the salt -Q command, so that even after the timeout has caused a {} to be returned you can look up the return once it happens.

[Erik Johnson]

Fantastic. I did not know that. Haven't looked at the API all that much, but I'm assuming the same functionality is available there.

--

-Erik

"For me, it is far better to grasp the universe as it really is than to persist in delusion, however satisfying and reassuring." --Carl Sagan

Thomas S Hatch <that...@gmail.com> wrote:

[Thomas S Hatch]

Salt -Q needs some work, but the salt-call command allows you to run anything on the minion directly that you would run remotely, using it makes debugging states very easy, since for states it also prints out a great deal of extra debugging data