Elastalert - Few questions

[Josh Silvestro]

It's me again, but I'm still new to this and can't find answer online or am not asking the right questions.

Working with elastalert, I've copied the ids.yaml that was already in the directory and made a few changes to what I need and renamed it. However, after a bit of play with it, realized the index referenced in the ids.yaml is index: logstash-ids*

So my question, I've sent in firewall logs via syslog and parsed with a logstash conf. How do I created or determine the correct index? <-- this should give you an idea on why I'm having trouble Googling the right answer, ha.

Hopefully that make sense.

[Wes Lambert]

Josh,

Try the following from cmd line:

curl localhost:9200/_cat/indices

This will show you the available indices.

You can also see which indice/index (_index) a particular log is tied to within "Discover" or the complete log entry itself.

Thanks,

Wes

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.
To post to this group, send email to security-onion@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

[Josh Silvestro]

So working backwards I may have found what I need to point me in the right direction. Looking at the 9033_output_snort.conf in the conf.d dir. I see there's a index => logstash-ids-%DATE VARIABLES%.

But I don't assume I can just freely make this index up on the fly just in the conf right?

[Wes Lambert]

Josh,

You are likely looking for:

logstash-firewall-*

or 

logstash-syslog-*

Thanks,

Wes

On Wed, Nov 8, 2017 at 4:58 PM, 'Josh Silvestro' via security-onion <securit...@googlegroups.com> wrote:

So working backwards I may have found what I need to point me in the right direction. Looking at the 9033_output_snort.conf in the conf.d dir. I see there's a index => logstash-ids-%DATE VARIABLES%.

But I don't assume I can just freely make this index up on the fly just in the conf right?

[Josh Silvestro]

OK - so I did add an output in my firewall conf that I made to output to logstash-firewall and I now am seeing events.

[Josh Silvestro]

Thanks for the help. So I have a YML file and was able to successfully send an alert via Slack, but I'm having issues with e-mail.

For the past year I've used SO I've installed postfix and sent e-mail that way. When run the so-email and set the notify e-mail to our sup...@company.com and then the notify user as me josh. Never had any issues.

In the YML file I set

alert:
- "email"

email:
- "sup...@company.com"

but receive no alert, I did try setting it to root@localhost as well, but no success. The error I get is "ERROR:root:Error while running alert email: Error connecting to SMTP host: [Errno 99] Cannot assign requested address"

I've looked through configs but can't seem to find anything to change / configure for this.

[Wes]

Josh,

Maybe try setting smtp_host in the yaml file:

alert:
- "email"

email:
- "sup...@company.com"

smtp_host:
- "your_smtp_server"

http://elastalert.readthedocs.io/en/latest/elastalert.html

Thanks,
Wes

[Josh Silvestro]

I did try that and get a connection refused, it's very odd.

I did find some stuff about elastalert and using a gmail account directly from the alert. For this you need an auth file placed inside the elastalert docker image. I've verified this works! However, after a system reboot that file is gone due to the image.

So is there a path with persistence that so-elastalert has access to? Or do I need to look in to persistence within elastalert image.

[Wes Lambert]

Josh,

To what specific auth file are you referring? Where is it (what specific directory, etc.) required to live?

Thanks,

Wes 

[Josh Silvestro]

Inside the rule.yaml file you specify
smtp_auth_file: /opt/elastalert/auth.yaml

^^ Or you can specify any directory really.

But in playing around with it I realized placing it inside your localhost directory such as /etc/elastalert/auth.yaml will not work which makes sense due to elastalert scripts not having access to that directory, unless I'm way off here.

So I can specify whatever path I'd like to store the auth.yaml in but it has to be inside elastalert image, again unless there's something such as a shared directory I don't know about. I assume since we place alert files in /etc/elastalert/rules on the Security Onion host that directory is somehow referenced by so-elastalert.

Hopefully that makes sense.

The third post down from the top of this page may give you better insight.

https://github.com/Yelp/elastalert/issues/627

[Wes Lambert]

We may modify the mounted volume in the future (/etc/elastalert/rules/ currently). -- how about it you put it in /etc/elastalert/rules?  Does it work from there?  Elastalert should have access to that dir, as that it how it has access to read in rules, etc.

Thanks,

Wes 

[Wes Lambert]

how about it

s/it/if

To post to this group, send email to securit...@googlegroups.com.

[Josh Silvestro]

I had tried that earlier, but this causes elastalert to complain that it's not a valid rule file, then hangs. I just attempted again and it yells:

File "elastalert/config.py", line 465, in load_rules
raise EAException('Error loading file %s: %s' % (rule_file, e))
elastalert.util.EAException: Error loading file /etc/elastalert/rules/smtp_auth_file.yaml: Invalid Rule file: /etc/elastalert/rules/smtp_auth_file.yaml
{'password': '**************', 'user': 'em...@company.com', 'rule_file': '/etc/elastalert/rules/smtp_auth_file.yaml'} is valid under each of {'properties': {'type': {'pattern': '[.]'}}, 'title': 'Custom Rule from Module'}, {'properties': {'type': {'enum': ['any']}}, 'title': 'Any'}

[Josh Silvestro]

Fixed! Renamed the file to a txt instead of yaml and it worked fine! So in case anyone else is looking to add gmail support to their elastalert:

Add to your rule file in /etc/elastalert/file.yaml

alert:
- "email"

email:
- "your...@company.com"
smtp_host: "smtp.gmail.com"
smtp_port: 465
smtp_ssl: true
from_addr: "your...@gmail.com"
smtp_auth_file: '/etc/elastalert/rules/smtp_auth_file.txt'

#########
In the smtp_auth_file.txt add:
user: your...@gmail.com
password: yourpassword

And you should be good to go!

[Wes Lambert]

Josh,

Thanks for confirming, Josh.

With your input, I've updated the wiki to clarify the process for utilizing external email providers.

https://github.com/Security-Onion-Solutions/security-onion/wiki/ElastAlert

Thanks,

Wes

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.

To post to this group, send email to security-onion@googlegroups.com.

[Josh Silvestro]

Awesome! Glad I could help somehow! Thanks for the constant support!

On Thu, Nov 9, 2017 at 4:36 PM, Wes Lambert <wlamb...@gmail.com> wrote:

Josh,

Thanks for confirming, Josh.

With your input, I've updated the wiki to clarify the process for utilizing external email providers.

https://github.com/Security-Onion-Solutions/security-onion/wiki/ElastAlert

Thanks,

Wes

On Thu, Nov 9, 2017 at 4:26 PM, 'Josh Silvestro' via security-onion <security-onion@googlegroups.com> wrote:

Fixed! Renamed the file to a txt instead of yaml and it worked fine! So in case anyone else is looking to add gmail support to their elastalert:

Add to your rule file in /etc/elastalert/file.yaml

alert:
- "email"

email:
- "your...@company.com"
smtp_host: "smtp.gmail.com"
smtp_port: 465
smtp_ssl: true
from_addr: "your...@gmail.com"
smtp_auth_file: '/etc/elastalert/rules/smtp_auth_file.txt'


#########
In the smtp_auth_file.txt add:
user: your...@gmail.com
password: yourpassword


And you should be good to go!

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---
You received this message because you are subscribed to the Google Groups "security-onion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to security-onion+unsubscribe@googlegroups.com.

To post to this group, send email to securit...@googlegroups.com.

Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

--
Follow Security Onion on Twitter!
https://twitter.com/securityonion
---

You received this message because you are subscribed to a topic in the Google Groups "security-onion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/security-onion/jwxmJCsyrTE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to security-onion+unsubscribe@googlegroups.com.

To post to this group, send email to security-onion@googlegroups.com.
Visit this group at https://groups.google.com/group/security-onion.
For more options, visit https://groups.google.com/d/optout.

--

Thank You,

Joshua Silvestro