create a minion-side api using salt engine ?

70 views
Skip to first unread message

bruno binet

unread,
Oct 11, 2016, 4:08:40 AM10/11/16
to salt-users
Hi all,

I would like to run salt states and modules locally on the minion.

I know that I can use the "salt-call --local" command, but it is very slow to execute a single salt state.
So to provide better speed/efficiency, I was wondering if we could leverage the salt engines to create a kind of salt-api locally for the minion, so that we can run salt modules and states without having to pay for the creation of a new process and initialization stuff that occurs when using salt-call.
What do you think?

The documentation about salt engines is very light, so I don't really know if this is actually doable, and what API is available in the salt engine.

Cheers.
Bruno

Bruce Wang

unread,
Oct 11, 2016, 6:13:29 AM10/11/16
to salt-...@googlegroups.com
Hi Bruno,

Masterless minion can run salt-call without master,  it can run highstate just like you have a master/minions setup, see https://docs.saltstack.com/en/latest/topics/tutorials/quickstart.html

I'm not sure why you feel it slow, since it basically the same as  running a minion with master, less the network back and forth.

Could you be more specific about what you want to achieve with salt engine?


Cheers,
Bruce 

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

bruno binet

unread,
Oct 11, 2016, 12:30:46 PM10/11/16
to salt-users
On 11 October 2016 at 12:12, Bruce Wang <br...@brucewang.net> wrote:
Hi Bruno,

Masterless minion can run salt-call without master,  it can run highstate just like you have a master/minions setup, see https://docs.saltstack.com/en/latest/topics/tutorials/quickstart.html

Yes, as said in my previous post, I know about salt-call, but when you want to run a single state, it is slow because (I suppose) it needs to do all the salt initialization stuff before actually executing the state.
 
I'm not sure why you feel it slow, since it basically the same as  running a minion with master, less the network back and forth.

It is not the same, because when executing a job come from the master, the salt minion is already running and hot so it can actually execute the state directly (no initialization stuff to do first), so it is quite fast.
 

Could you be more specific about what you want to achieve with salt engine?

What I would like to achieve is something similar to the salt-api of the salt master, but for the salt minion.
So the engine would basically do the initialization stuff once at startup, and then it would expose an http api (similar to the salt-api) so that executing states through this http api would be much faster than running the salt-call command.

Also the second advantage of such an engine is that a http api would be easy to integrate in a webapp running on the minion, for example if you want to create a system web admin interface based on salt (masterless).

I hope this is more clear now.
Cheers.

Daniel Wallace

unread,
Oct 11, 2016, 1:00:42 PM10/11/16
to salt-...@googlegroups.com
There is nothing like that right now in the salt engines, but I have been thinking about adding something like that.

I will see if I have time this week to do a mock up of a possible engine that could do this so that we could have events/webhooks on masterless minions now that reactor also runs on the minion starting in Carbon.

Dmitry Golubenko

unread,
Oct 12, 2016, 12:06:04 AM10/12/16
to salt-...@googlegroups.com
В Втр, 11/10/2016 в 18:30 +0200, bruno binet пишет:
...
>
> What I would like to achieve is something similar to the salt-api of
> the salt master, but for the salt minion.
>
> So the engine would basically do the initialization stuff once at
> startup, and then it would expose an http api (similar to the
> salt-api) so that executing states through this http api would be much
> faster than running the salt-call command.
>
>
> Also the second advantage of such an engine is that a http api would
> be easy to integrate in a webapp running on the minion, for example if
> you want to create a system web admin interface based on salt
> (masterless).
as a workaround you can run full-featured minion and master with api on
the same system. maybe overkill, but should work out of the box.
> +unsub...@googlegroups.com.

bruno binet

unread,
Oct 12, 2016, 2:31:11 AM10/12/16
to salt-users
On 12 October 2016 at 06:05, Dmitry Golubenko <dmitry.g...@xiag.ch> wrote:
В Втр, 11/10/2016 в 18:30 +0200, bruno binet пишет:
...
>
> What I would like to achieve is something similar to the salt-api of
> the salt master, but for the salt minion.
>
> So the engine would basically do the initialization stuff once at
> startup, and then it would expose an http api (similar to the
> salt-api) so that executing states through this http api would be much
> faster than running the salt-call command.
>
>
> Also the second advantage of such an engine is that a http api would
> be easy to integrate in a webapp running on the minion, for example if
> you want to create a system web admin interface based on salt
> (masterless).
as a workaround you can run full-featured minion and master with api on
the same system. maybe overkill, but should work out of the box.

Yes, but my minion is also already connected to a remote salt master, and I don't want to share the master key on the minion as required in multimaster setups.
 
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+unsubscribe@googlegroups.com.

bruno binet

unread,
Oct 12, 2016, 2:31:58 AM10/12/16
to salt-users
Thanks Daniel, that would be great!

Shane Gibson

unread,
Oct 12, 2016, 10:01:30 AM10/12/16
to Salt-users

So why not run an additional Salt Masterless setup? Just change the ports of the minion side master to not conflict with the default in use already.

You'd need to set up a parallel config directory and insure the new Masterless minion calls have the correct config directory flags when called. If you compile yourself, you can set the default directory differently at compile time.

~~shane

bruno binet

unread,
Oct 13, 2016, 7:42:20 AM10/13/16
to salt-users
I'm not sure I understand the solution you are proposing: how an additional Salt Masterless setup would help in my use case?

Shane Gibson

unread,
Oct 13, 2016, 12:02:07 PM10/13/16
to salt-...@googlegroups.com

Salt master and salt-api will give you a full blown API access to a host.  

Since you have master/minion relationship set up - and if I understand correctly - you want to be able to manage a minion *directly* via an API - not via the master --> minion setup ... you need to do "something else".

If you setup your minion as a salt master w/ salt-api, but change the ports from the default 4505/4506 - add second minion using those new ports (lets pretend 5505/5506) - and you have full salt-api at your disposal to control that minion - outside of your normal "master --> minion" relationship. 

It's a bit heavy handed to just get the API capability - but you don't have to write any code, and the solutions should work.

~~shane 


--
"The farther one goes, the less one knows." —Tao Te Ching

--
You received this message because you are subscribed to a topic in the Google Groups "Salt-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/salt-users/jPVEAYnD5SA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to salt-users+unsubscribe@googlegroups.com.

Daniel Wallace

unread,
Oct 13, 2016, 4:12:59 PM10/13/16
to salt-...@googlegroups.com
These should both be going into develop, so they won't be in Carbon, but will be in nitrogen (and could be dropped into _engines and _modules).


Hope this helps some people :) I know I will be setting up my blog with them to be able to self update without having a master.

bruno binet

unread,
Oct 14, 2016, 5:04:59 AM10/14/16
to salt-users
Thanks Daniel, this sounds great!

Could you also add some documentation to explain how to use the new webhook engine?
What is the http api to which the webhook engine replies?

Also, does it work only for masterless setups, or also in master-minion setups?

Cheers.

Daniel Wallace

unread,
Oct 14, 2016, 10:02:04 AM10/14/16
to salt-...@googlegroups.com
I can add that.

It only returns 200 if it posted the message to the queue, if it is a minion with file_client: local, it sends the event to the minion, otherwise it sends the event to the master.  If it is on the master, it sends the event to the master.  It is an engine so it can run on both masters and minions, starting in Carbon.

It takes the uri, and just passes that as the tag to the event stream, and then takes the payload, and passes it in the data to the event stream, it also passes the headers, so that stuff like the github webhook events can be verified with the hmac signature.

bruno binet

unread,
Oct 14, 2016, 11:17:49 AM10/14/16
to salt-users


On 14 October 2016 at 16:01, Daniel Wallace <dwal...@saltstack.com> wrote:
I can add that.

Thanks
 
It only returns 200 if it posted the message to the queue, if it is a minion with file_client: local, it sends the event to the minion, otherwise it sends the event to the master.

It would be great if we could also choose to force send the message to the minion even if file_client is not "local". Can we imagine a configuration option or a runtime parameter (like listening to two different urls, either dedicated to sending events to either the master or dedicated to sending events yo the minion)?
 
If it is on the master, it sends the event to the master.  It is an engine so it can run on both masters and minions, starting in Carbon.

It takes the uri, and just passes that as the tag to the event stream, and then takes the payload, and passes it in the data to the event stream, it also passes the headers, so that stuff like the github webhook events can be verified with the hmac signature.

Thanks I now understand that we can then use the reactor or thorium system to trigger actions based on those events, that's great!

Daniel Wallace

unread,
Oct 14, 2016, 3:08:57 PM10/14/16
to salt-...@googlegroups.com
On Fri, Oct 14, 2016 at 10:17 AM, bruno binet <bruno...@gmail.com> wrote:


On 14 October 2016 at 16:01, Daniel Wallace <dwal...@saltstack.com> wrote:
I can add that.

Thanks
 
It only returns 200 if it posted the message to the queue, if it is a minion with file_client: local, it sends the event to the minion, otherwise it sends the event to the master.

It would be great if we could also choose to force send the message to the minion even if file_client is not "local". Can we imagine a configuration option or a runtime parameter (like listening to two different urls, either dedicated to sending events to either the master or dedicated to sending events yo the minion)?

I am not interested in adding that at this time.  It is a simple implementation to run this as is, where adding all that will make more edge cases that could break.  And if you have a master, you really should use the salt-api on the master for the orchestration needs like reactor or engines. 
Reply all
Reply to author
Forward
0 new messages