Anyone got Salt master posting events to Elastic stack logstash?

[Paul Mulvihill]

Hi Everyone

I'm after some help. I've got a vanilla instance of salt master (2016.3.4-1el7) and a minion running on a dev box (Centos 7).
I'm trying to get it to send the event logs to an Elastic Stack logstash server but i'm not having much luck.

in the salt-master config file i have the following

engines:
    - logstash:
        host: <IP address here>
        port: 5959

The only additional package i've installed is python-logstash (0.4.6) via PIP

if i telnet to the IP and port (logstash) it does connect and shows a connection. but salt doesn't seem to do anything itself.

The saltstack docs mention adding that engines section and them being dependant on logstash. but i'm not sure what action I need to take for the dependancy. or if i need to install any external package or modules.

I have seen the following issue on git
https://github.com/saltstack/salt/issues/34551
and before i tried the adjustment to the logstash.py file it was throwing a critical for the engine salt.loader.lazyloader
"[CRITICAL] Engine <salt.loader.LazyLoader object at 0xfbe790> could not be started! Error: 'str' object does not support item assignment"
with the adjustment the critical goes away.

I'll accept I may just be getting confused or missing something obvious in the docs. but has anyone actually got this working?
If so could they let me know how, what config, what needed installing etc?

Thanks

[Martin Höfling]

On 24.11.16 11:21, Paul Mulvihill wrote:

Hey Paul,

when using logstash via beats protocol,
https://github.com/martinhoefling/saltbeat might be an option for you.

Best

Martin

> --
> You received this message because you are subscribed to the Google
> Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to salt-users+...@googlegroups.com
> <mailto:salt-users+...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Mircea Ulinic]

Hi,

In some cases, it proves to be easier to push documents in logstash via HTTP. If that suits your needs, you might want to have a look at https://docs.saltstack.com/en/develop/ref/engines/all/salt.engines.http_logstash.html (but I think will be available only in 2016.11)

Cheers,

Mircea

[Paul Mulvihill]

Hi

Thanks for that i'll definitely keep an eye on it.
I'll also have a look into saltbeats.

Has anyone actually tried to use the built in logstash engine (none http) available so far?
It would be nice to hear peoples experiences so far.

Paul

[Paul Mulvihill]

Hi Mircea

Have you actually used any of the engines in salt yet?
Is it as simple as adding the "engines:" config in the master conf file to enable that engine module?

The documentation structure doesn't lend itself to explaining how to set it up. or at least to me it's not clear.
The existence of the "start" line is just confusing me and i'm wondering if that is supposed to be put in some other file.

Thanks

Paul


On Friday, 25 November 2016 22:32:54 UTC, Mircea Ulinic wrote:

[Paul Mulvihill]

update:
Got this working by adding in the config to the master conf file.
Note that it only works for salt/job event entries doesn't work for anything else.
The code specifically only targets events with a tag starting with salt/job

Paul