Wait for state.highstate via salt-api?

[Martin Emrich]

Hi!

When I call state.highstate via API (I send e.g. "fun=state.highstate&client=local&tgt=testvm-5689"), it waits only for a few seconds. If the execution takes longer, the call ends prematurely, and I get only an empty result.
The state.highstate call is continuing in the background, but I have no means to find out whether it is finished or whether it was successful.

Is there a way to let the call block until the job is finished, or to do some kind of active-wait for it (check every 5 seconds for a result)?

Thanks

Martin

[Seth House]

The timeout behavior in the current version of Salt + salt-api is a bit wonky. This is fixed in the upcoming Salt release (currently in RC).

A workaround for the current release is to send JSON instead of urlencoded data and to specify a long timeout:

[{"client": "local", "fun": "state.highstate", "tgt": "testvm", "timeout": 600}]

Make sure you send the correct content-type header as well.

Another option is, as you suggested, to execute asynchronously and poll for the result. See the /minion and /jobs URLs documentation.

Finally, a third way (that requires the most set up) is to execute asynchronously and to watch the HTTP event stream for job returns.

There's a partial implementation in sh here:

https://gist.github.com/whiteinge/a646d4d937960d3ef306

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Martin Emrich]

Great, I'll try Option 2 first...

Thanks,

Martin

You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/t3fL3g1rdMc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+...@googlegroups.com.

[Martin Emrich]

Hi!

The change to JSON and including the timeout parameter did not work; the timeout value is silently being ignored. (I could not find any mention of "timeout" in the code, too).
I also tried the /minions and /jobs interface: GET /minions/<minionid> returns more or less the same as "salt '<minionid' grains.items' (but no jobid), /jobs always returns '{"info": [{}], "return": [{}]}'.

I am running Salt 2014.1.7 from EPEL on CentOS6, and salt-api V. 0.8.4.1 (I found the RPM somewhere on github, after I could not get V. 0.8.2 from EPEL running). Are there any newer RPMs?

Thanks

Martin

[Martin Emrich]

I grew impatient ;) and installed salt 2014.7.10rc1 and salt-api 0.8.4.1 on my dev system. Now I can start the job via /minions and get job status via /jobs

Cheers

Martin

[Seth House]

On Mon, Aug 18, 2014 at 8:36 AM, Martin Emrich <frozenh...@gmail.com> wrote:
> I grew impatient ;) and installed salt 2014.7.10rc1 and salt-api 0.8.4.1

That will do. :)

Two notes:

salt-api is now included with Salt core in 2014.7.0rc1 so you should
uninstall the stand-alone package.

If you find you still need to tweak the timeout the argument is passed
through salt-api to the LocalClient class in Salt. Documentation is
here:

http://docs.saltstack.com/en/latest/ref/clients/index.html#salt.client.LocalClient.cmd

[Martin Emrich]

Am Montag, 18. August 2014 17:15:31 UTC+2 schrieb Seth House:

salt-api is now included with Salt core in 2014.7.0rc1 so you should
uninstall the stand-alone package.

Done, had to uninstall and reinstall salt itself to restore the binaries. Sadly the "pip" installation does not provide init scripts, so I still look forward to the RPMs...
 

If you find you still need to tweak the timeout the argument is passed
through salt-api to the LocalClient class in Salt. Documentation is
here:

http://docs.saltstack.com/en/latest/ref/clients/index.html#salt.client.LocalClient.cmd

Yes, with a whopping timeout of 600s, my synchronous state.highstate works.

But I also tried the asynchronous variant (POSTing to /minions and polling /jobs), but here, the job seems to get created, but it does not run. do I have to explicitly "start" the job afterwards?

Ciao

Martin

[Seth House]

On Wed, Aug 20, 2014 at 6:37 AM, Martin Emrich <frozenh...@gmail.com> wrote:
> Yes, with a whopping timeout of 600s, my synchronous state.highstate works.

Woot.

> But I also tried the asynchronous variant (POSTing to /minions and polling
> /jobs), but here, the job seems to get created, but it does not run. do I
> have to explicitly "start" the job afterwards?

No activation requirled. POSTing to /minions is the same as POSTing to
/ except the job is async. The CLI analog is running ``salt --async``
vs just ``salt``.

Do you not see job results via the /job/<jid> URL at all? If you run
the ``saltutil.running`` function on the minion after starting the
initial job you can see what the minion is currently working on. The
/jobs/<jid> URLs are the same as ``salt-run jobs.lookup_jid <jid>``
and you should see the same job output in both places.

[Martin Emrich]

Am Donnerstag, 21. August 2014 02:54:13 UTC+2 schrieb Seth House:

No activation requirled. POSTing to /minions is the same as POSTing to
/ except the job is async. The CLI analog is running ``salt --async``
vs just ``salt``.

Then something is wrong. The POST to /minions returns 200 OK and I get a job ID, e.g. "20140821130255916792". But on the minion itself, nothing happens (highstate should install some packages and stuff, but nothing appears).
 

Do you not see job results via the /job/<jid> URL at all? If you run
the ``saltutil.running`` function on the minion after starting the
initial job you can see what the minion is currently working on. The
/jobs/<jid> URLs are the same as ``salt-run jobs.lookup_jid <jid>``
and you should see the same job output in both places.

I call GET on /jobs/20140821130255916792 (with Accept: application/json, X-Auth-Key, and without content). All I get is "200 OK" and
{
          'info' => [
                      'Exception occurred in runner jobs.list_job: SaltInvocationError: list_job takes at least 1 argument (0 given)'
                    ],
          'return' => [
                        'Exception occurred in runner jobs.lookup_jid: SaltInvocationError: lookup_jid takes at least 1 argument (0 given)'
                      ]
        };

Cheers,

Martin

[Seth House]

Thanks for the report. I can reproduce this. Looks like a regression.
I'll put in a fix as soon as I can this week. You can track the bug
here:

https://github.com/saltstack/salt/issues/15218

[Martin Emrich]

Hi!

Am Sonntag, 24. August 2014 07:42:00 UTC+2 schrieb Seth House:

Thanks for the report. I can reproduce this. Looks like a regression.
I'll put in a fix as soon as I can this week. You can track the bug
here:

https://github.com/saltstack/salt/issues/15218

 Good to know, looking forward to it :)

Ciao

Martin