Wazuh+ELK and Arbitrary Logs
821 views
Skip to first unread message
ola...@gmail.com
Apr 9, 2018, 3:42:49 PM4/9/18
to Wazuh mailing list
Assumptions:
- Any <localfile> entries in ossec.conf on agents should cause those logs to be monitored.
- Enabling <logall> in ossec.conf on the manager should mean that all log entries are entered into ELK, regardless of decoder/rule status
- All entries in ElasticSearch should be visible in Kibana
If any of those assumptions are invalid, this question won't make a lot of sense — so please let me know if that's the case! If they're all valid...
My custom logs aren't visible in Kibana. :(
I have a custom log file registered in the agent ossec.conf as syslog format, per the documentation for single-line logs. I've set <logall>yes</logall> in the manager's ossec.conf and restarted OSSEC on both manager and agent. I can see the log entries in the archives log on the manager. In Kibana, I only see log entries which generated OSSEC alerts. There are no filters active in Kibana. I don't see the arbitrary log's events in ElasticSearch when curling either the wazuh-alerts* or wazuh-monitoring* index.
Anyone have any ideas?
- Any <localfile> entries in ossec.conf on agents should cause those logs to be monitored.
- Enabling <logall> in ossec.conf on the manager should mean that all log entries are entered into ELK, regardless of decoder/rule status
- All entries in ElasticSearch should be visible in Kibana
If any of those assumptions are invalid, this question won't make a lot of sense — so please let me know if that's the case! If they're all valid...
My custom logs aren't visible in Kibana. :(
I have a custom log file registered in the agent ossec.conf as syslog format, per the documentation for single-line logs. I've set <logall>yes</logall> in the manager's ossec.conf and restarted OSSEC on both manager and agent. I can see the log entries in the archives log on the manager. In Kibana, I only see log entries which generated OSSEC alerts. There are no filters active in Kibana. I don't see the arbitrary log's events in ElasticSearch when curling either the wazuh-alerts* or wazuh-monitoring* index.
Anyone have any ideas?
Jose Luis Ruiz
Apr 9, 2018, 4:52:37 PM4/9/18
to Wazuh mailing list, ola...@gmail.com
HI Oladon let me try to help here:
- Any <localfile> entries in ossec.conf on agents should cause those logs to be monitored.
Correct
- Enabling <logall> in ossec.conf on the manager should mean that all log entries are entered into ELK, regardless of decoder/rule status
Not correct, by default we only send to Elastic the file /var/ossec/logs/alerts/alerts.json (https://github.com/wazuh/wazuh/blob/3.2/extensions/logstash/01-wazuh-local.conf#L6) or (https://github.com/wazuh/wazuh/blob/3.2/extensions/filebeat/filebeat.yml#L5), enabling logall only creates a file in /var/ossec/logs/alerts/archives/archives.log with the full amount of logs, but this file is not sent to Elastic
- All entries in ElasticSearch should be visible in Kibana
“Correct”… if you have a patter created for these logs… (Wazuh-APP creates this patter automatically for Wazuh-Manager alerts)
Answering your main question, if you have a custom log and Wazuh has not rules and decoders for your logs, the default installation wont send the logs to your Elasticsearch.
You have two options here:
1.- Create a decode/rule for you custom logs:
You can create a custom decoder/rule to “teach” Wazuh manager to read your custom logs and send to Elasticsearch.
2.- Send the full raw archives.json to Elastic:
In this second case you can send the full raw file to Elastic, this means that the full amount of logs without being processed by the manager is sent to Elastic, Wazuh-APP doesn’t work with this situation and also you will have some content duplicated, the original log coming from the archives.json and also the alert coming from the manager.
We always recommend the first one, create your decoder and rule for your logs.
I hope it helps.
--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/a20643e4-7973-4180-8558-7132e78d2ddc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
ola...@gmail.com
Apr 9, 2018, 5:06:48 PM4/9/18
to Wazuh mailing list
Jose,
Thanks for the quick and very helpful reply!
I have a few follow-up questions... :)
1) I'm setting this up for PCI compliance. Is the expectation/norm that the only items sent to ELK (and thus shown in Kibana) will be alerts generated by Wazuh based on the input logs, as opposed to all security-related events?
2) One of my custom logs is an application authentication log; would you recommend creating decoders/rules only for "abnormal" events like failed logins, or for all events in that log (to enable better correlation with more data)? Is the latter typically done with "level 0" alerts?
Thank you again for your help!
-Oladon
Thanks for the quick and very helpful reply!
I have a few follow-up questions... :)
1) I'm setting this up for PCI compliance. Is the expectation/norm that the only items sent to ELK (and thus shown in Kibana) will be alerts generated by Wazuh based on the input logs, as opposed to all security-related events?
2) One of my custom logs is an application authentication log; would you recommend creating decoders/rules only for "abnormal" events like failed logins, or for all events in that log (to enable better correlation with more data)? Is the latter typically done with "level 0" alerts?
Thank you again for your help!
-Oladon
Jose Luis Ruiz
Apr 9, 2018, 5:24:45 PM4/9/18
to Wazuh mailing list, ola...@gmail.com
Hi Oladon, happy to help:
1- The alerts generated by Wazuh covers security events in a Standard system, in the following link you can see how the different Milestone works with OSSEC/Wazuh components:
2.- We recommend log only for the “abnormal” activity in your system, or some cases the normal activity in a low level, and the “abnormal” increase the level of the alert, but really is totally up to you, if you app generates GB of data per day or only a few MB per day…
--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e5dfd07f-6602-4a3c-8715-47bbc09e44b5%40googlegroups.com.
ola...@gmail.com
Apr 9, 2018, 5:29:45 PM4/9/18
to Wazuh mailing list
Thanks, Jose — appreciate it!
Jose Luis Ruiz
Apr 9, 2018, 5:30:49 PM4/9/18
to Wazuh mailing list, ola...@gmail.com
No Problem,
If you have problem to create your custom decoders and rules, please feel to contact with us!, we will help you :)
On April 9, 2018 at 5:29:48 PM, ola...@gmail.com (ola...@gmail.com) wrote:
Thanks, Jose — appreciate it!
--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/65fd33de-d49f-4cdf-881e-d1bb4d21e0d5%40googlegroups.com.
J. Desipeda
Apr 17, 2018, 4:36:42 AM4/17/18
to Wazuh mailing list
I'm actually facing this problem now. I made a localfile entry in ossec.conf and it shows in ossec.log that the custom log is being monitored and analyzed. I am following this blog:
and the second step is to check whether you can see the file in archives.log.
The problem is I can't find it there and the custom log is not appearing in the web interface.
Apparently the decoder and rule creation step must be done first.
I'll try it now. Thanks. :))
J. Desipeda
Apr 17, 2018, 4:40:03 AM4/17/18
to Wazuh mailing list
Or am I mistaken? Should the monitored logs already appear in archive.log even if there are no decoders and rules made?
Jose Luis Ruiz
Apr 17, 2018, 12:31:26 PM4/17/18
to J. Desipeda, Wazuh mailing list
Hi J.
If you add a custom log it being monitored and analyzed, each new line after applying the configuration should appear in the archives.log.
If you have no decoder/rules that can read these logs, you won’t have alerts in the file alerts.json, and therefore in Elasticsearch.
And easy test that you can do is a “lsof /xxx/xxx/newfile.log,” this log should being read by ossec-logmonitoring.
root@wazuh-manager:/var/log# lsof syslog
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
ossec-log 215 root 6r REG 8,1 0 3161629 syslog
root@wazuh-manager:/var/log#
--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To post to this group, send email to wa...@googlegroups.com.
Visit this group at https://groups.google.com/group/wazuh.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/41fe5d2a-bf4b-4295-bed2-cbf32360c368%40googlegroups.com.
J. Desipeda
Apr 17, 2018, 11:07:16 PM4/17/18
to Wazuh mailing list
My problem yesterday was a custom log in a Windows server. But I did try monitoring a custom Linux log today and checked using lsof and it shows the ossec-log. It's also appearing in the archives.log
I'll retry the custom log in the Windows server. Thanks for the reply! :)
Jani Heikkinen
Oct 25, 2019, 4:42:17 AM10/25/19
to Wazuh mailing list
Hi, I have been checking on this thread, since we are in process of trying to monitor customised json file on Wazuh.
Now I am running into something that puzzles me and I do not understand how to proceed. See below.
I have created my own custom rules:
On wazuh server I have following entries in the /var/ossec/etc/local_rules.xml
<rule id="100002" level="0">
<decoded_as>json</decoded_as>
<field name="timestamp">\.+</field>
<field name="enabled">\.+</field>
<description>Portal messages.</description>
</rule>
<rule id="100003" level="3">
<if_sid>100002</if_sid>
<options>alert_by_email</options>
<field name="enabled">^true$</field>
<description>Portal: User enabled </description>
</rule>
<rule id="100004" level="3">
<if_sid>100002</if_sid>
<options>alert_by_email</options>
<field name="enabled">^false$</field>
<description>Portal: User disabled </description>
</rule>
<decoded_as>json</decoded_as>
<field name="timestamp">\.+</field>
<field name="enabled">\.+</field>
<description>Portal messages.</description>
</rule>
<rule id="100003" level="3">
<if_sid>100002</if_sid>
<options>alert_by_email</options>
<field name="enabled">^true$</field>
<description>Portal: User enabled </description>
</rule>
<rule id="100004" level="3">
<if_sid>100002</if_sid>
<options>alert_by_email</options>
<field name="enabled">^false$</field>
<description>Portal: User disabled </description>
</rule>
on my clients ossec.conf I have following entry:
<localfile>
<log_format>json</log_format>
<location>/var/log/portal.json</location>
</localfile>
<log_format>json</log_format>
<location>/var/log/portal.json</location>
</localfile>
Now, I can see with ossec-logtest that given the right input, rule gets triggered.
If
I add content to my monitored JSON file on the client, I also see an
entry in alerts.log, and I get an email indicating the rule was
triggered.
BUT: I do not see anything in the elasticsearch alerts when trying to search with rule id for the client agent.
As far I can understand, I now have
- custom decoder and rule
- my alert is triggered
- according to the reply of Jose, data should be appearing in Elasticsearch and in Kibana too?
Can you advise, what am I doing wrong since the data does not end in ELK?
Best, Jani Heikkinen
maanantai 9. huhtikuuta 2018 23.24.45 UTC+2 Jose Luis Ruiz kirjoitti:
Hi Oladon, happy to help:1- The alerts generated by Wazuh covers security events in a Standard system, in the following link you can see how the different Milestone works with OSSEC/Wazuh components:2.- We recommend log only for the “abnormal” activity in your system, or some cases the normal activity in a low level, and the “abnormal” increase the level of the alert, but really is totally up to you, if you app generates GB of data per day or only a few MB per day…
On April 9, 2018 at 5:06:50 PM, ola...@gmail.com (ola...@gmail.com) wrote:
Jose,--
Thanks for the quick and very helpful reply!
I have a few follow-up questions... :)
1) I'm setting this up for PCI compliance. Is the expectation/norm that the only items sent to ELK (and thus shown in Kibana) will be alerts generated by Wazuh based on the input logs, as opposed to all security-related events?
2) One of my custom logs is an application authentication log; would you recommend creating decoders/rules only for "abnormal" events like failed logins, or for all events in that log (to enable better correlation with more data)? Is the latter typically done with "level 0" alerts?
Thank you again for your help!
-Oladon
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
Pablo Rodríguez Martín
Oct 29, 2019, 11:57:55 AM10/29/19
to Wazuh mailing list
Hi, Jani.
In order to make sure there is not any kind of problem with the communications between nodes, please check both Wazuh and ELK versions match in the compatibility matrix: https://documentation.wazuh.com/3.10/installation-guide/compatibility_matrix/index.html#api-and-kibana-app
I would also like to ask you to check the following command outputs:
At the Wazuh manager node
systemctl status filebeat --> This will show if the Filebeat service is active and running and will reveal errors about the possible cause of the problem if exists
filebeat test output --> Checks if Filebeat is ready to forward alerts to the Elasticsearch node
At the Elasticsearch node
systemctl status elasticsearch --> This will show if the Elasticsearch service is active and running and will reveal errors about the possible cause of the problem if exists
systemctl status elasticsearch --> This will show if the Elasticsearch service is active and running and will reveal errors about the possible cause of the problem if exists
lsof /var/ossec/logs/alerts/alerts.json --> Checks if the alerts.json file is being read by Filebeat (the lsof command must be installed)
lsof -i -P -n | grep 9200 --> Will display the port's status and if the connection with the manager node is established
I hope this information works for you. Let me know if I may be of further help.
Regards,
Pablo Rodríguez
Jani Heikkinen
Oct 29, 2019, 12:03:23 PM10/29/19
to Pablo Rodríguez Martín, Wazuh mailing list
Hi Pablo, I got already help on this from you guys, but thanks.
The issue was that my custom rule needed options no_full_log.
Elasticsearch was receiving a JSON object but it was expecting the
full_log field to be text, thus the data didn't ever end up in
Elastic.
After changing the adding the no_full_log option things started to work.
Best, Jani
>>> To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/5e93a902-6d50-4b70-8a04-94a4a55d95f5%40googlegroups.com.
The issue was that my custom rule needed options no_full_log.
Elasticsearch was receiving a JSON object but it was expecting the
full_log field to be text, thus the data didn't ever end up in
Elastic.
After changing the adding the no_full_log option things started to work.
Best, Jani
>>> To post to this group, send email to wa...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/wazuh.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e5dfd07f-6602-4a3c-8715-47bbc09e44b5%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>
>>> Visit this group at https://groups.google.com/group/wazuh.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e5dfd07f-6602-4a3c-8715-47bbc09e44b5%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
> You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/5e93a902-6d50-4b70-8a04-94a4a55d95f5%40googlegroups.com.
Pablo Rodríguez Martín
Oct 29, 2019, 1:13:20 PM10/29/19
to Wazuh mailing list
Hi, Jani.
Glad this was solved. Please let us know if you have any other questions.
Best regards,
Pablo Rodríguez
Glad this was solved. Please let us know if you have any other questions.
Best regards,
Pablo Rodríguez
>>> To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
>>> To post to this group, send email to wa...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/wazuh.
>>> To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/e5dfd07f-6602-4a3c-8715-47bbc09e44b5%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages