Firewall device as agents and filter in Dashboard

[Azhar Hj.Mohd Ghazali]

Hi ,

I have one questions :

1) I can view the fortigate log in /var/log/messages :-

Feb  5 15:38:48 gateway date=2021-02-05 time=15:38:47 devname=FGT60E4Q16023616 devid=FGT60E4Q17033614 logid="0001000014" type="traffic" subtype="local" level="notice" vd="root" logtime=1612510727 srcip=121.121.43.73 srcport=56010 srcintf="wan2" srcintfrole="wan" dstip=122.0.30.102 dstport=514 dstintf="root" dstintfrole="undefined" sessionid=2219830550 proto=17 action="deny" policyid=0 policytype="local-in-policy" service="SYSLOG" dstcountry="Malaysia" srccountry="Malaysia" trandisp="noop" app="SYSLOG" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat="unscanned" crscore=5 craction=262144 crlevel="low"

I need to massage all traffic from the devname =  FGT60E4Q16023616 into dashboard and action='allow"

What 'field' shall i need to apply to represent the devname.

Pls help.

Thanks

  

[Yana Zaeva]

Hi,

Even if you are able to see the log in the /var/log/messages file, it does not mean that these logs are being parsed. If the event is not being parsed, it means that it does not have a decoder, so it cannot match a rule thus not counting as an alert. This is why you are probably not seeing this event in Kibana. 

You can check if alerts of these events are being generated by performing this command: 
- cat /var/ossec/logs/alerts/alerts.json | grep FGT60E4Q16023616

If the output is empty, these events are not generating alerts. 

You can also use the /var/ossec/bin/ossec-logtest binary, run it and just paste the log. If the result is 'No decoder matched', it means that the events are not generating alerts. 

Please, let me know if you are able to see these alerts as events. If you can, I will show you how you can add a filter, but if you don't I will help you to write a decoder for your event.

I will also leave you a link where you can find more information about the ossec-logtest binary, for further help: https://documentation.wazuh.com/4.0/user-manual/reference/tools/ossec-logtest.html

Waiting for your reply,

Yana.

[Azhar Hj.Mohd Ghazali]

Hi Yana,

Thanks for the great explanation.

cat /var/ossec/logs/alerts/alerts.json | grep FGT60E4Q16023616

got messages ;

slog","access_control","authentication_failed"],"pci_dss":["10.2.4","10.2.5"],"gpg13":["7.8"],"gdpr":["IV_35.7.d","IV_32.2"],"hipaa":["164.312.b"],"nist_800_53":["AU.14","AC.7"],"tsc":["CC6.1","CC6.8","CC7.2","CC7.3"]},"agent":{"id":"000","name":"wazuh.mykris.net"},"manager":{"name":"wazuh.test"},"id":"1612583772.18972205","full_log":"Feb  6 11:56:10 gateway date=2021-02-06 time=11:56:10 devname=FGT60E4Q16023616 devid=FGT60E4Q17033614 logid=\"0100032002\" type=\"event\" subtype=\"system\" level=\"alert\" vd=\"root\" logtime=1612583770 logdesc=\"Admin login failed\" sn=\"0\" user=\"root\" ui=\"ssh(45.146.164.106)\" method=\"ssh\" srcip=45.146.164.106 dstip=1.0.30.5 action=\"login\" status=\"failed\" reason=\"name_invalid\" msg=\"Administrator root login failed from ssh

IN Kibana also got Hit as alerts;

Now  if my management want to have reports of how many top source IP coming to the firewall, src port, dst port, dst IP etc. How do i filter and decoder for this firewall.

and also how to register this firewall as agents.

Pls guide me. 

For agents i.e Linux, Windows, Mac i no worry since Wazuh can handle very well.

Appreciate your help.

Many thanks.

Azhar

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/acf20870-2188-4ffa-9df6-13381e6801dan%40googlegroups.com.

[Yana Zaeva]

Hi Azhar,

Sorry for the late response. I just check both logs and it seems that you have that alert in the alerts.json file because it is triggering a rule because the log contains a certain word, but it is not being parsed as there is not any decoder for it. You can check this in the ossec-logtest, by just pasting the log (you will see No decoder matched). 

You can use this decoder I have written, in order to parse some information. 

<decoder name="fortigate">

    <prematch>^date=\d\d\d\d-\d\d-\d\d time=\d\d:\d\d:\d\d devname=\w+ devid=\w+</prematch>

</decoder>

<decoder name="fortigate2">

        <parent>fortigate</parent>

        <regex>date=(\d\d\d\d-\d\d-\d\d) time=(\d\d:\d\d:\d\d) devname=(\w+) devid=(\w+) logid="(\d+)" type="(\w+)" subtype="(\w+)" </regex>

        <order>date, time, devname, devid, logid, type, subtype</order>

</decoder>

You will have to add this decoder to the file called: /var/ossec/etc/decoders/local_decoder.xml. 

Also, add this simple rule to the file /var/ossec/etc/rules/local_rules.xml: 

<rule id="100002" level="5">

        <decoded_as>fortigate</decoded_as>

        <description>Custom decoder.</description>

  </rule>

Once you have these decoders and rules, restart the Wazuh manager. Now, if you run the ossec-logtest binary, you will be able to see this, instead of No decoder matched. (See the file ossec-logtest_output). 

As you can see, now the fields of your log are being parsed. That means that when you see these events in Kibana,  you will see the parsed information as in the attached image called Kibana_parsed. 

Once you have this information parsed, you can apply the filters. You can filter by decoder.name, as in the picture query1 or by any parsed field in the log, query2. Any field that you parse in your custom decoder will be seen in Kibana as data.<name of your field>. For example, if in the decoder we are parsing as devname, in Kibana you will find this field under the name data.devname. 

You can improve the decoders that I have sent you and make them parse more information from the log. I will leave you some useful links:

- Rules syntax: https://documentation.wazuh.com/4.0/user-manual/ruleset/ruleset-xml-syntax/rules.html#decoded-as

- Decoder syntax: https://documentation.wazuh.com/4.0/user-manual/ruleset/ruleset-xml-syntax/decoders.html

- Regular expression syntax: https://documentation.wazuh.com/4.0/user-manual/ruleset/ruleset-xml-syntax/regex.html

Hope I was helpful. Do not hesitate to contact us if you have any doubts.

Regards,

Yana.

[Azhar Hj.Mohd Ghazali]

Hi Yana,

Sorry i could not find data.devname, data.scrip, etc in my kibana. I have followed your decoder and restart wazuh.

Pls advise.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/c1b32d7d-c932-41c7-901c-88c7c96195d6n%40googlegroups.com.

[Yana Zaeva]

Hi Azhar,

If you are not seeing this information in Kibana it could be due to reasons:

1. The decoders and rules were not properly added. To ensure that they are in the right place, just run the binary /var/ossec/bin/ossec-logtest, and paste the log. I have pasted the log you sent me in the first message, meaning this one: 

Feb  5 15:38:48 gateway date=2021-02-05 time=15:38:47 devname=FGT60E4Q16023616 devid=FGT60E4Q17033614 logid="0001000014" type="traffic" subtype="local" level="notice" vd="root" logtime=1612510727 srcip=121.121.43.73 srcport=56010 srcintf="wan2" srcintfrole="wan" dstip=122.0.30.102 dstport=514 dstintf="root" dstintfrole="undefined" sessionid=2219830550 proto=17 action="deny" policyid=0 policytype="local-in-policy" service="SYSLOG" dstcountry="Malaysia" srccountry="Malaysia" trandisp="noop" app="SYSLOG" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat="unscanned" crscore=5 craction=262144 crlevel="low"

If the decoders and rules are correctly placed (in the directories /var/ossec/etc/decoders/local_decoder.xml and /var/ossec/etc/rules/local_rules.xml), that you will see the output from the image image1, attached below. Let me know if this is the output you are getting. 

2. In case the output is the same as mine, in Discover, try to filter by decoder.name -> is -> fortigate (which is the name I have set for the decoder) and check the results. If there are logs here, expand one of them in order to see all the fields. If you find little exclamation marks, as in the picture attached (cap2), you will have to go to Management -> Index Patterns and refresh the index pattern. After this, you should be able to see the new fields. 

Also, the decoder I have written for you decodes the first seven fields of the log, meaning it decodes data.date, data.time, data.devname, data.devid, data.logid, data.type and data.subtype. You will have to complement this decoder a little if you want it to parse more information. However, I will try to find one that works completely for you. 

Let me know if the output of the ossec-logtest binary is the same as mine, so we can continue tackling the problem.

Waiting for your reply,

Yana.

[Azhar Hj.Mohd Ghazali]

Hi Yana,

I can't see the exclamation mark under data.details.

Only views below :

For image1 , yes i get same result as yours.

I failed to find data.devname, data.dstip etc

Pls help.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/3dc37318-957d-4e79-b71f-b4b647673b2en%40googlegroups.com.

[Yana Zaeva]

Hi Azhar,

If you obtained the same output as I using the ossec-logtest binary, then the decoder is working properly. These fields should appear above the decoder.name field. 

I have found these decoders, which you could use with your logs. They parse more information than the ones I have sent you before. Delete the old ones and place the file decoders_forti.xml in the directory /var/ossec/etc/decoders/. Regarding the rules, change <decoded_as>fortigate</decoded_as> for <decoded_as>FW_FORTI1000D</decoded_as>. 

Once you have done this, change their ownership with these commands:

chmod 660 /var/ossec/etc/decoders/decoders_forti.xml

chown ossec:ossec /var/ossec/etc/decoders/decoders_forti.xml

Once you have done all of this, restart the manager to apply the changes. 

Let me know if that worked. 

Regards,

Yana.

[Azhar Hj.Mohd Ghazali]

Hi yana,

could not find  "FW_FORTI1000D".

Only see the previous decoder name not "FW_FORTI1000D".

And NO events hit in discover interface;

My devices version is 

FortiGate 60E

can your decoders parse same information for 60E version ?

BR,

Azhar

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/4f60ed0e-ea89-4fba-b439-59c03f2f7f19n%40googlegroups.com.

[Yana Zaeva]

Hi Azhar,

If you are working with a cluster, you have to place the decoder only in one master, and then restart the restart of the managers that belong to the cluster, including the master. These decoders seem to match your logs, but I do not know if they completely do. You can check how many fields are being parsed with them using the ossec-logtest binary, by pasting the log. If there are some fields that are not being parsed, you can always modify these decoders, and add more fields to be parsed. I will leave you here two useful links in order to customize decoders: 

- Decoders syntax: https://documentation.wazuh.com/4.0/user-manual/ruleset/ruleset-xml-syntax/decoders.html

- Regular expression syntax: https://documentation.wazuh.com/4.0/user-manual/ruleset/ruleset-xml-syntax/regex.html

Regarding the decoders themselves, check if you have placed them in the right directory (/var/ossec/etc/decoders/) and make sure that you have restarted the manager afterward. Also, make sure that you have modified the field <decoded_as> in the /var/ossec/etc/rules/local_rules.xml

In order to solve this, please send me the following information:

1. The output of these commands: 

ls -l /var/ossec/etc/decoders/

ls -l /var/ossec/etc/rules/

2. The output of the ossec-logtest binary. 

3. The systemctl status wazuh-manager output after a restart.

Waiting for your reply,

Yana.

[Azhar Hj.Mohd Ghazali]

Hi yana,

As requested,

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/1c6af724-73b5-44b7-9656-d06e91ad4fadn%40googlegroups.com.

[Yana Zaeva]

Hi Azhar,

Sorry for the long reply. My bad, I have modified these decoders in one file, to make them match yours, but sent you the unsaved file. This is why you are obtaining  the line No decoders matched in the ossec-logtest. 

Delete the old ones, and move these new ones in the /var/ossec/etc/decoders/ directory. Then, restart the manager and let me know if when you paste any log in the ossec-logtest binary you obtain a similar output to the image attached below (called output-ossec-logtest).

Waiting for your reply,

Yana.

[Azhar Hj.Mohd Ghazali]

Hi yana,

Greats, The decoder works perfect.

Now how do we set RULE alert.i.e action=accept/deny  for port 22/80/443 SSH/HTTP/HTTPS and send emil to sysadmin.

Appreciate your help.

Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/5df1ebe6-8219-4905-8ccb-7f9152ba88d0n%40googlegroups.com.

[Yana Zaeva]

Hi Azhar,

I have modified the decoders a little bit to make them match the srcport and dstport too. I will attach the new version here. Just delete the old one, set the new one in the /var/ossec/etc/decoders/, and restart the manager. Now the output you should be able to see these two fields. 

Regarding the rules, nos as you will have these two fields, you can create a rule similar to this one: 

<rule id="100110" level="5">

    <if_sid>100002<if_sid>

    <srcport type="osregex">22</srcport>

    <description>Source port $(srcport) is detected.</description>

</rule>

Now, if the srcport is equal to 22, an alert will be triggered. You can apply this rule to every port you want to monitor. You can also change srcport to dstport, to be aware of the destination ports. 

You can find more information about this at the following link: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/rules.html#srcport

Hope I was helpful. Do not hesitate to contact us if you have any queries. 

Regards,

Yana.

[Azhar Hj.Mohd Ghazali]

hi yana,

Only rules is not working, once apply the rule in /rules/local_rules.xml

restarted wazuh manager got this;

in rules as below;

Or need to put in different files ?

In wazuh rules documentation also not clear stated on how to put the stuff (which files / create new file? )

the rule ID can we set any number ? or have standard numbering format ?

i am a bit confused about the rules settings.

Pls help.

Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/ac42c44b-c472-4d86-8619-9f30a81a392fn%40googlegroups.com.

[Yana Zaeva]

Hi Azhar,

Regarding the rules, there are two problems. First of all, this is failing probably because you do not have any decoder called fortigate-custom. The decoders I have sent you before are named FW_FORTI1000D, so if you want this to work, in the local_rules.xml file, in the rule 222000 change the field <decoded_as>fortigate-custom</decoded_as>, for <decoded_as> FW_FORTI1000D</decoded_as>. 

The <decoded_as>NAME</decoded_as> field means that the log was decoded using a decoder whose name is NAME. If there isn't any decoder with this particular name, as in your case, fortigate-custom, you will obtain an error. 

The other issue that you have in the rule file is that the rules 222000 and 100110 have to be concatenated. What I mean is that the rule 222000 is for ANY event which has been decoded with the FW_FORTI1000D decoder. This is the parent rule for all the events from FortiGate. I strongly suggest you uncomment it, thus the next rule (100110) is dependant on 222000. Once you have the rule 222000 uncommented, in the rule 1001100 change the field <if_sid>100002</if_sid> for <if_sid>222000</if_sid> thus making it a child of 222000. I have set the rule 100002 as this was the id I have set for the first rule. 

The <if_sid>NUMBER</if_sid> field means that in order to trigger the current rule the rule whose id is NUMBER had to be triggered before. 

If you do not want to receive all the alerts that 222000 is generating (which are a lot) just lower its level from 3 to 2. This way you will only be alerted by the rule 100110. 

Once you have performed the changes, restart the manager. However, I strongly recommend you using the binary /var/ossec/bin/ossec-logtest to see if there is any problem with the rules and decoders, before actually committing the changes. 

Regarding your questions: 

1) About the ID number, you can set whichever number in the range of 100000 to 999999. You can find this information here: https://documentation.wazuh.com/current/user-manual/ruleset/custom.html#adding-new-decoders-and-rules. In this link there is also an explanation on how to customize rules and decoders. Also, be careful with not setting the same Id for two different rules.

2) About the files with customized rules and decoders. In the link above, you can find more information about this topic, but basically, you can add custom rules to the file /var/ossec/etc/rules/local_rules.xml but also you can create your own rule files, meaning that besides having the local_rules.xml  you can have a field called my_custom_rules.xml, or some other name. The only condition is that this file must be an .xml type and it has to be allocated in the /var/ossec/etc/rules/ directory. The same happens with the decoders, with the only exception that the decoder directory is called /var/ossec/etc/decoders/

Lastly, I would like to leave you here some information about rules syntax, hoping that you will find it useful: https://documentation.wazuh.com/current/user-manual/ruleset/ruleset-xml-syntax/rules.html

Let me know if you have any doubts.

Regards,

Yana.

[Azhar Hj.Mohd Ghazali]

Hi Yana,

Works perfect.

Again many thanks to you for great assistance.

If i have any issue, will raise new topic,

Thanks.

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/98df6964-b4ec-4851-b733-54009dd24cf5n%40googlegroups.com.

[Yana Zaeva]

Hi Azhar,

It is always a pleasure to help! 

Do not hesitate to contact us if you have any query.

Regards,

Yana.