***Urgent Help Required*** - snort syslog-ng alerts not formatted properly in logstash grok parse errors

[Zishan Fayyaz]

Below is what the syslog-ng.conf looks like;

I have replaced IP address and hostnames with XXXX

# This line specifies where the sguild.log file is located, and informs syslog-ng to tail the file, the program_override inserts the string sguil_alert into the string

source s_sguil { file("/var/log/nsm/securityonion/sguild.log" program_override("sguil_alert")); };

# This line filters on the string “Alert Received”
filter f_sguil { match("Alert Received"); };

# This line tells syslog-ng to send the data read to the IP address of xx.xx.xx.xx, via UDP to port xxxx
destination d_sguil_tcp {
tcp("xx.xx.xx.xx"
port("xxxx")
tls(peer-verify(required-untrusted))
);
};


# This log section tells syslog-ng how to structure the previous ‘source / filter / destination’ and is what actually puts them into play
log {
source(s_sguil);
filter(f_sguil);
destination(d_sguil_tcp);
};


Kibana entry looks like this;
message:<13>Jun 4 12:06:50 XXXX sguil_alert: 12:06:49 pid(30684) Alert Received: 0 3 unknown XXXX-eth6-5 {2018-06-04 12:06:48} 7 606697 {ETPRO TROJAN Common Downloader Header Pattern H} XX.XX.XX.XX XX.XX.XX.XX.XX 6 57654 80 1 2803305 8 6086 6086 @version:1 @timestamp:June 4th 2018, 13:06:50.057 type:snort-tcp tags:_grokparsefailure _id:AWPKsWLEZ5gBSaTkYre- _type:snort-tcp _index:security-snort-tcp-2018.06.04 _score: -

Any help in resolving this is greatly appreciated.

P.S - I'm new to Security Onion & ELK Stack so apologies if this has been resolved, I've followed the 3rd party integration document to get this far.
The reason why we are not using SO with ELK integrated is because they already have the ELK Stack ingesting logs form other sources.

Many Thanks

[Zishan Fayyaz]

Any help moving this forward is very welcome...

[Wes Lambert]

To confirm, you are trying to send this to your own Elastic Stack, correct?  If you are asking about the formatting of the alerts and how they are processed on that side, that is something you will need to check with your team.  You can look at the config files in /etc/logstash/conf.d/ to see how the alerts are processed by Logstash and fed into Elasticsearch.

Thanks,

Wes

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

--

https://twitter.com/therealwlambert

https://securityonion.net/

[Zishan Fayyaz]

Hi Wes,

Yes, I'm sending to our corporate Elastic Stack as apposed to the one bundled with Security Onion.

Is there not a grok filter available to parse the logs as bundled with Security Onion?

The squild.log entry looks like this;
2018-06-05 14:03:29 pid(7227) Alert Received: 0 3 unknown SENSORNAME-eth6-31 {2018-06-05 14:03:28} 29 433104 {ETPRO TROJAN Common Downloader Header Pattern H} XX.XX.XX.XX XX.XX.XX.XX 6 58519 80 1 2803305 8 993 993

And the Kibana message look like as follows;
message:<13>Jun 5 14:03:30 MASTERXXX sguil_alert: 14:03:29 pid(7227) Alert Received: 0 3 unknown SENSORNAME-eth6-31 {2018-06-05 14:03:28} 29 433104 {ETPRO TROJAN Common Downloader Header Pattern H} XX.XX.XX.XX XX.XX.XX.XX 6 58519 80 1 2803305 8 993 993 @version:1 @timestamp:June 5th 2018, 15:03:30.021 type:snort-tcp tags:_grokparsefailure _id:AWPQQpE1IsfJPH1tu9UD _type:snort-tcp _index:security-snort-tcp-2018.06.05 _score: -

I replaced the IP addresses with XX. & SENSORNAME.
I can decipher most of the message, however would like to know if there is already a grok filter for logstash or an alert output field map.

Once again all help is appreciated.

Kind Regards
Zishan

> To unsubscribe from this group and stop receiving emails from it, send an email to security-onio...@googlegroups.com.
>
> To post to this group, send email to securit...@googlegroups.com.

[Wes Lambert]

You'll want to take a look at:

/etc/logstash/conf.d/0000_input_syslogng.conf

/etc/logstash/conf.d/1001_preprocess_syslogng.conf

/etc/ogstash/conf.d/1033_preprocess_snort.conf

Thanks,

Wes

To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.

Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.