Fortigate logging issue

[R VISHNU]

Dear Team,

I have enabled remote logging and logall in wazuh manager to receive syslogs from FortiGate firewall.

I can get logs if I do tcpdump in wazuh manager installed machine. As mentioned in documents these should logs store under /var/ossec/log/archivearchives.log and archives.json file.

But I have completely verified that file, firewall logs didn't store in that file. Also, these logs aren't visible in kibana. Pls, help us to find where the logs stored? and Do we need to configure anything more in filebeat?.

Thanks,

Vishnu

[Jonathan Martín Valera]

Hi, R VISHNU

Let's see what the problem might be.

First of all, say that if the manager is not receiving the events sent by rsyslog, then you won't see any of it in Kibana, so let's start from the beginning of the process flow.

First, you need to configure rsyslog to send information and Wazuh to receive it. To do this, I strongly recommend you to read this article https://wazuh.com/blog/how-to-configure-rsyslog-client-to-send-events-to-wazuh/ that shows how to configure rsyslog client to send events to Wazuh. You can also find related information in our documentation https://documentation.wazuh.com/3.12/user-manual/capabilities/log-data-collection/how-it-works.html#remote-syslog

Once you have everything set up, Wazuh should receive the events, and these are stored in the /var/ossec/logs/archives/archives.log and /var/ossec/logs/archives/archives.log as long as you have activated the corresponding logall sections in the file ossec.conf. To do this, perform the following steps:

- Go to your manager /var/ossec/etc/ossec.conf file and enable logall in global section.

For archives.log

<logall>yes</logall>

For archives.json

<logall_json>yes</logall_json>

- Restart the manager to load the new configuration.

systemctl restart wazuh-manager

or

/var/ossec/bin/ossec-control restart

- Monitor the file /var/ossec/logs/archives/archives.log to see what events the manager is receiving.

tail -f /var/ossec/logs/archives/archives.log

Note: In case the manager is receiving a lot of events, you can filter it using the `grep` command tool.

- If everything is set up correctly, when there is a change in the log you are sending with rsyslog to Wazuh, you should see the corresponding event(s).

Note: Remember to disable logall in case you no longer need it.

From here, you should see the corresponding events in archives.log/.json sent through rsyslog.

Carry out these steps, and tell me if you have reached this point, or what problems you have encountered :)

Best regards.

[Jonathan Martín Valera]

Hi R VISHNU,

Here is the answer to the question you asked me privately (I think it is convenient to write it here in case any other user might be interested :D)

Okay, so you're telling me you've checked the configuration... and you can't get the events to be stored in the archives.log file.

Let's do a basic setup.

Manager configuration

First, we are going to configure the manager.

Imagine that the manager has IP 172.16.1.30 and the rsyslog client has 172.16.1.20/24.

- Add the following to the /var/ossec/etc/ossec.conf file:

  <remote>
    <connection>syslog</connection>
    <port>514</port>
    <protocol>udp</protocol>
    <allowed-ips>172.16.1.20/24</allowed-ips>
    <local_ip>172.16.1.30</local_ip>
  </remote>

Note: In this case it is sent over port 514 UDP, change this if necessary

- Activate the logall in the /var/ossec/etc/ossec.conf file:

# vi /var/ossec/etc/ossec.conf

<logall>yes</logall>

- Restart wazuh service

systemctl restart wazuh-manager

- After restarting the service, we will monitor the file /var/ossec/logs/archives/archives.log to see the events received by the manager

tail -f /var/ossec/logs/archives/archives.log

Rsyslog client configuration

On the client side, it is necessary to send the information through rsyslog.

To send information to the manager, add the following to the file /etc/rsyslog.conf

# vi /etc/rsyslog.conf

*.* @172.16.1.30:514

- Next, create a new configuration to send the information of the monitored file. For this, create  file, with the following content:

# vi /etc/rsyslog.d/wazuh.conf

$ModLoad imfile
$InputFilePollInterval 1
$InputFileName /var/log/test.log
$InputFileTag testlogs:
$InputFileStateFile testlogs
$InputFileFacility local0
$InputRunFileMonitor

- Make a new test log file:

# touch /var/log/test.log

- Restart the rsylog service to load the new configuration.

systemctl restart rsyslog

- After restarting the rsyslog service, you should already see some events like the following in the archives.log file:

2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu rsyslogd: [origin software="rsyslogd" swVersion="8.2001.0" x-pid="1884" x-info="https://www.rsyslog.com"] exiting on signal 15.
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu systemd[1]: Stopping System Logging Service...
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu systemd[1]: rsyslog.service: Succeeded.
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu systemd[1]: Stopped System Logging Service.
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu systemd[1]: Starting System Logging Service...
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu systemd[1]: Started System Logging Service.
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu rsyslogd: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd.  [v8.2001.0]
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu rsyslogd: rsyslogd's groupid changed to 110
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu rsyslogd: rsyslogd's userid changed to 104
2020 Jun 09 16:07:40 NVD-ubuntu->172.16.1.20 Jun  9 16:07:41 NVD-ubuntu rsyslogd: [origin software="rsyslogd" swVersion="8.2001.0" x-pid="1903" x-info="https://www.rsyslog.com"] start

- Now, to check that everything is correct, we only have to add a new entry in the test log file, to see that the events are reported. For example:

# echo "Hello World" >> /var/log/test.log

In the archives.log file you should see something like the following:

2020 Jun 09 16:09:10 NVD-ubuntu->172.16.1.20 Jun  9 16:09:12 NVD-ubuntu testlogs: Hello World

In the event that nothing is reported, check that you have connectivity through that port, connectivity ...

I hope this answer helps you.

Best regards.

[Jonathan Martín Valera]

Hi R VISHNU,

Please write through the mail thread https://groups.google.com/forum/#!topic/wazuh/J7K5csLn1mk so that everyone can see the questions and answers.

I'm glad it works for you!

Below I show you what a decoder would be like for the first and last two logs you have shared in the docx file.

<decoder name="fortigate">
 <prematch>\d+.\d+.\d+\d+ date=\.*time\.*devname=\.*devid=\.*logid=\.*type=\.*subtype=\.*</prematch>
 <regex>(\d+.\d+.\d+.\d+) date=(\.*) time=(\.*) devname="(\.*)" devid="(\.*)" logid="(\.*)" type="(\.*)" subtype="(\.*)" level="(\.*)" vd="(\.*)" eventtime=(\.*) srcip=(\.*) srcintf=(\.*) srcintfrole="(\.*)" dstip=(\.*) dstintf=(\.*) dstintfrole="(\.*)" sessionid=(\.*) proto=(\.*) action="(\.*)" policyid=(\.*) service="(\.*)" dstcountry="(\.*)" srccountry="(\.*)" trandisp="(\.*)" app="(\.*)" duration=(\.*) sentbyte=(\.*) rcvdbyte=(\.*) sentpkt=(\.*) rcvdpkt=(\.*) appcat="(\.*)"</regex>
 <order>ip,date,time,devname,devid,logid,type,subtype,level,vd,eventtime,srcip,srcintf,srcintfrole,dstip,dstintf,dstintfrole,sessionid,proto,action,policyid,service,dstcountry,srccountry,trandisp,app,duration,sentbyte,rcvdbyte,sentpkt,rcvdpkt,appcat</order>
</decoder>

For example, with the following log:

111.11.11.111 date=2020-05-18 time=10:56:14 devname="Netcon-Cbe" devid="FGTxxxxxxxxxxxxxxxxx" logid="0001000014" type="traffic" subtype="local" level="notice" vd="root" eventtime=1589779574 srcip=xxx.xxx.xxx.xxx srcintf=unknown-0 srcintfrole="undefined" dstip=8.8.8.8 dstintf="wan1" dstintfrole="wan" sessionid=1364104 proto=1 action="accept" policyid=0 service="PING" dstcountry="United States" srccountry="India" trandisp="noop" app="PING" duration=65 sentbyte=300 rcvdbyte=440 sentpkt=5 rcvdpkt=5 appcat="unscanned"

The following decoded fields are obtained:

**Phase 1: Completed pre-decoding.
       full event: '169.254.37.194 date=2020-05-18 time=10:56:14 devname="Netcon-Cbe" devid="FGTxxxxxxxxxxxxxxxxx" logid="0001000014" type="traffic" subtype="local" level="notice" vd="root" eventtime=1589779574 srcip=xxx.xxx.xxx.xxx srcintf=unknown-0 srcintfrole="undefined" dstip=8.8.8.8 dstintf="wan1" dstintfrole="wan" sessionid=1364104 proto=1 action="accept" policyid=0 service="PING" dstcountry="United States" srccountry="India" trandisp="noop" app="PING" duration=65 sentbyte=300 rcvdbyte=440 sentpkt=5 rcvdpkt=5 appcat="unscanned"'
       timestamp: '(null)'
       hostname: 'manager'
       program_name: '(null)'
       log: '169.254.37.194 date=2020-05-18 time=10:56:14 devname="Netcon-Cbe" devid="FGTxxxxxxxxxxxxxxxxx" logid="0001000014" type="traffic" subtype="local" level="notice" vd="root" eventtime=1589779574 srcip=xxx.xxx.xxx.xxx srcintf=unknown-0 srcintfrole="undefined" dstip=8.8.8.8 dstintf="wan1" dstintfrole="wan" sessionid=1364104 proto=1 action="accept" policyid=0 service="PING" dstcountry="United States" srccountry="India" trandisp="noop" app="PING" duration=65 sentbyte=300 rcvdbyte=440 sentpkt=5 rcvdpkt=5 appcat="unscanned"'

**Phase 2: Completed decoding.
       decoder: 'fortigate'
       ip: '111.11.11.11'
       date: '2020-05-18'
       time: '10:56:14'
       devname: 'Netcon-Cbe'
       devid: 'FGTxxxxxxxxxxxxxxxxx'
       logid: '0001000014'
       type: 'traffic'
       subtype: 'local'
       level: 'notice'
       vd: 'root'
       eventtime: '1589779574'
       srcip: 'xxx.xxx.xxx.xxx'
       srcintf: 'unknown-0'
       srcintfrole: 'undefined'
       dstip: '8.8.8.8'
       dstintf: '"wan1"'
       dstintfrole: 'wan'
       sessionid: '1364104'
       proto: '1'
       action: 'accept'
       policyid: '0'
       service: 'PING'
       dstcountry: 'United States'
       srccountry: 'India'
       trandisp: 'noop'
       app: 'PING'
       duration: '65'
       sentbyte: '300'
       rcvdbyte: '440'
       sentpkt: '5'
       rcvdpkt: '5'
       appcat: 'unscanned'

For all other cases, you will have to create additional decoders following the same process. It is very easy, you would only have to add or modify fields according to the regex of the log you want to decode.

For more help related to this topic, I recommend you take a look at this blog article https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/

Best regards.

[Jonathan Martín Valera]

Hi R VISHNU,

I am going to answer you in the public thread, since you may asked privately by mistake. Our intention is to be able to help the maximum number of users possible, and it is common that many times users have the same question and resort to searching for information in mail threads.

In reply to:

Dear Jonathan,

Thanks for helping me with the Decoder script. Can you pls, help me with the Ruleset script too.
Is there any site or materials to learn decoder-script and ruleset script.

Thanks,
Vishnu

To build custom rules, it is necessary to know the condition on which you want to generate the alert, for example, generate a rule to alert when the source IP address is X and ...

To create decoders and custom rules, I recommend you to take a look at this article https://wazuh.com/blog/creating-decoders-and-rules-from-scratch/. Also, you can find more related information in our documentation https://documentation.wazuh.com/3.12/user-manual/ruleset/custom.html

In case you need support to create a set of decoders and rules for your use case, do not hesitate to contact our technical support, who can offer you customized plans for what you need https://wazuh.com/professional-services/.

Best regards.

[R VISHNU]

Step 1: You need to have the right configuration on the ossec.conf file for syslog connection. The location of the file is usually in /var/ossec/etc/ folder. But it could be different based on where you installed it.

In the ossec.conf file, you need to edit the configurations under <connection>syslog</connection>. This part is for network devices. The configuration should account for the following factors:

• <port>514</port> - This part of the configuration specifies which port receives the logs on the manager’s system. By default, its 514 for syslog.

• <protocol>udp</protocol> - This part ensures which protocol your firewall logs are being forwarded from. You can have either tcp or udp.

• <allowed-ips> – You need to specify which IPs should be allowed into the Wazuh-manager. By default, it doesn’t have any or even include the tag. You must add the tag and specify the IP.

• <local_ip>– You need to add the Wazuh-manager’s IP here.

<remote>

<connection>syslog</connection>

<port>514</port>

<protocol>udp</protocol>

<allowed-ips>x.x.x.x/x</allowed-ips>

<local_ip>x.x.x.x</local_ip>

</remote>

Here’s the structure of the code that you can directly copy-paste into the conf file before that replcae the require details. Now restart the Wazuh-manager.

Okay, now that the Wazuh-manger is setup for receiving the firewall logs, you need to set up the Fortigate firewall to send logs.

Step 2:  Create a Site-to-site Tunnel

The logs can freely flowing through the internet, which is not ideal considering it’s a security tool. Hence, to secure the logs properly, we had to create a site-to-site tunnel. 

Note: Site-to-site tunnelling is a private communication channel to ship the logs privately from the device to the manager.

This is the most challenging part of the whole process. You’ll probably face the same challenge if you are hosting it on any cloud server for continued access. We hosted it on AWS, so this step of the blog is specific to AWS. Below are the five steps that we took to successfully complete this device linkage.

2.1. Create a Customer Gateway

Customer gateway is the first step to creating a Site to Site tunnel between your remote networks to cloud. Here, we define the details of which network our Wazuh-manager is connecting to. You can find more details on how to do this at: AWS.Amazon.com.

2.2. Create a Virtual Private Gateway

Next, you need to configure the Target Gateway in AWS. There are two types of Target gateway - Virtual Private Gateway and Transit Gateway. You can find more information on which Gateway you need at: AWS.Amazon.com 

2.3. Enable Route Propagation in your Route Table

Now that we have gates on both ends, we need to configure the route table to include the routes used to travel between both gateways. Check out AWS.Amazon.com for more details. 

2.4. Create a Site-to-Site VPN Connection

In steps 2.1 to 2.3, we created and configured the necessary components to set-up a site-to-site connection. In this step, we set up the actual tunnel by linking the previous components. Follow the steps from AWS.Amazon.com to suit your requirements. After completing the steps, don’t forget to download the configuration file. This file has the necessary configurations that you need to add to Fortigate.

2.5. Configure Fortigate Firewall with VPN Details

Now that the site-to-site vpn is setup, you need to configure the firewall to match with the configuration file you previously downloaded. This is another challenging step in the whole process due to two main reasons. 

• The configuration file that we got from AWS in the previous step had 10.0.0.0/8 as the private IP (LAN) segment. You need to correct it with your private IP segment. In addition, the private IP and public IP (WAN) should match in both the configuration files and the Fortigate VPN details.

• The AWS default interface is composed of 15 random characters, but Fortigate firewall only supports 12 characters. So, it is better to change the name of the interface to your device name. To do so, in your Fortigate console monitor, find the VPN Monitor section and select Ipsec tunnel. Then, select BringUp and ‘All phase 2 Selectors’. Here, you have the option to change the interface name.

That’s it. The connection is now established. The logs are securely transported from the firewall to Wazuh-manager through the secure tunnel. The logs should be reaching the manager on port 514. You can confirm that by running a tcp dump on the network flow on the manager system.

But don’t celebrate yet, we still have work to do. However, the next few steps are relatively easy.

Step 3: Change the Settings on the Firewall Device to Forward System Logs

This part is fairly easy for any network devices. Under logging part of the settings, change the logging location to the remote. This will give you the option to enter an IP address. Then, make sure you have the following accounted for:

1) Add the Wazuh-manager’s IP to forward the logs.

2) Assign same TCP/UDP configuration as you specified in Step 1; ossc.conf file. If you don’t have an option for it, then it’s UDP by default.

Step #4: Decoding

Once the logs reach the manager’s system, the Wazuh-manager captures them and saves them into archives before sending them through decoders. The purpose of a decoder is to take the raw logs and convert them into structured information. The Wazuh-manager, unfortunately, was not decoding our logs, despite having a decoder for this in the default decoders folder. After banging our collective heads, we figured that we are better off creating our own.

There are two ways you can write the decoder, but you should use regex (regular expression) for it. One way is to include all regex in one bit of code.

The other way is to have a single parent for each parameter, also known as sibling decoders.

The logic of how the decoder works are: 

When the log reaches the Wazuh-manager, the prematch is what’s used to qualify which decoder will be used for that log. If a log has content that meets the prematch condition, the decoder name is assigned to the log and sent through the rest of the decoder. Each parent pulls its respective data from the log and presents it in the decoded format of the log.

Having multiple parents is a more robust solution than the former one, because in case a log does not contain a data field or the regex does not match a custom field, such as “policy name”, it is unlikely to work.

Step #5: Rulesset

After decoding, ensure the rules trigger the necessary alerts. This is a continuous task, but this is what differentiates a good SiEM tool from an average one. Create new rules so that the manager triggers alerts for new threats.

For that, you need to understand the logic and linkage of rules with the respective decoder. In the first bit of code, the <decoded_as>Fortigate-26</decoded_as> part is the if-condition that links the decoder to these set of rules. If this condition is met, the logs are processed per each rule in the set. If the logs meet the conditions of any one of the rules, an alert is triggered and sent to elasticsearch.

Step #6: Access the Logs on Kibana

As the log alerts are triggered, filebeat picks up the logs and sends it to elasticsearch. Here, ensure the filebeat yml file has the right configuration so that the logs are reaching Kibana. Once the logs reach elasticsearch, you should be able to query it on the Kibana dashboard under ‘discover’ tab using the Wazuh-alerts template.

Fortigate firewall was the first device we attempted to integrate with Wazuh-manager. If you follow the steps in this blog, you’d know by now how challenging it was. Especially, when there is no how-to resource help that you could just plug and play. Hope this blog was useful to you and stay tuned for our next blog in this series.

[Jonathan Martín Valera]

Hi R VISHNU, the question has finally been resolved?

Best regards.

[R VISHNU]

Hi Jonathan,

It was fixed and working fine.

Thanks,

Vishnu

--
You received this message because you are subscribed to the Google Groups "Wazuh mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/d9f2fe35-fcc0-4783-933d-f0be95165334n%40googlegroups.com.