Imperva decoder

399 views
Skip to first unread message

Nataliia

unread,
Jul 12, 2022, 10:04:58 AM7/12/22
to Wazuh mailing list
Hello Team!

I created decoder for Imperva logs, but it doesn't work in the decoder test - No result found.
My log is:
2022 Jul 12 11:42:46 wazuh-manager->/var/log/messages Jul 12 11:42:46 wazuh-manager : 0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=8218430560168672618 sourceServiceName=staticsso.104.ua siteid=8172810764 suid=1007608 requestClientApplication=Mozilla/5.0 (Linux; Android 11; Redmi Note 8 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Mobile Safari/537.36 deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=true cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=985a08ac-3477-47f4-804e-0d3d8681ce75 cs4Label=VID cs5=bde5ce13e361f6652323cb3d242b0eb7160a3b7452f0a4e596d5b8cde7ce3763ea90592cb34ec6a659ca48297729c984e3e0aca21872c43571ad8ba0171d68c68d7a6f9175d75123cafa3a90c88f7626bbfbca3c8f6cf6395ded9130ed98ad311b0fe104582e5dc560694c4a68028af6 cs5Label=clappsig dproc=Browser cs6=Mobile Chrome cs6Label=clapp ccode=GB cs7=51.4964 cs7Label=latitude cs8=-0.1224 cs8Label=longitude Customer=RGC start=1656892300199 request=staticsso.104.ua/build/sso/img/favicon/favicon-32x32.png ref=https://account.104.ua/ requestMethod=GET cn1=200 app=HTTPS act=REQ_PASSED deviceExternalId=796510916892689834 sip=46.164.130.243 spt=443 in=1518 xff=5.148.126.186 cpt=46552 src=5.148.126.186 ver=TLSv1.3 TLS_AES_128_GCM_SHA256 end=1656892300243#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324   suid=1903608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892216761 request=104.ua/robots.txt requestMethod=GET cn1=200 app=HTTPS act=REQ_CACHED_FRESH deviceExternalId=716457396338874765 in=110 cpt=34316 src=192.99.100.210 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 end=1656892216762#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId= 8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324 suid=2909608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892219823 request=104.ua/ua/news/id/У Вінницькій області 72% мешканців м-14067 requestMethod=GET cn1=301 app=HTTPS act=REQ_PASSED deviceExternalId=716480375800446128 sip=89.162.145.3 spt=443 in=290 xff=192.99.100.210 cpt=34316 src=192.99.100.210 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 end=1656892219980#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId= 8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324 suid= 2909608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892224558 request=104.ua/ua/news requestMethod=GET cn1=200 app=HTTPS act=REQ_PASSED deviceExternalId=825542562284965326 sip=46.164.130.243 spt=443 in=8325 xff=192.99.100.210 cpt=34354 src=192.99.100.210 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 end=1656892224779#000<14>

My decoders:
<decoder name="imperva-dec">
    <prematch>CEF:0|Incapsula|SIEMintegration</prematch>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>fileid=(\d+)</regex>
    <order>fileid</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>sourceServiceName=(\S+)</regex>
    <order>sourceServiceName</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>siteid=(\d+)</regex>
    <order>siteid</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>suid=(\d+)</regex>
    <order>suid</order>
</decoder>

First of all as I see Imperva has multi-line logs. What should I do for Wazuh read multi-line logs correctly?

And even if I try only one log line in decoder test, No result found too.
Help me to set up decoders, please.

victor....@wazuh.com

unread,
Jul 12, 2022, 12:12:38 PM7/12/22
to Wazuh mailing list

Hello Nataliia,
Regarding your event, if we remove the header archives header 2022 Jul 12 11:42:46 wazuh-manager->/var/log/messages we have

Jul 12 11:42:46 wazuh-manager : 0|Incapsula|SIEMintegration|1 ...

The Syslog format follows this structure:

Nov  9 16:06:26 localhost salute: Hello world.

If we compare it with your event, we can see that the program name is empty in your event. This can also be checked in the logtest tool

**Phase 1: Completed pre-decoding.
    full event: 'Jul 12 11:42:46 wazuh-manager : 0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=8218430560168672618 ...
    timestamp: 'Jul 12 11:42:46'
    hostname: 'wazuh-manager'
    program_name: ''
...

Once the Syslog format has been selected as the most suitable for your event, your custom decoder imperva-dec is never triggered.

In order to fix this issue, I recommend you use the out_format option. This allows formatting logs from Logcollector using field substitution. In this use case, something like the following should do the job

  <localfile>
    <log_format>syslog</log_format>
    <out_format>$(timestamp) wazuh-manager ImpervLogs: $(log)</out_format>
    <location>imperva-log-path</location>
  </localfile>

If we use the logtest tool with the final event log we get

**Phase 1: Completed pre-decoding.
    full event: 'Jul 12 01:37:55 wazuh-manager ImpervLogs: Jul 12 11:42:46 wazuh-manager : 0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=8218430560168672618 sourceServiceName=staticsso.104.ua siteid=8172810764 suid=1007608 requestClientApplication=Mozilla/5.0 ...'
    timestamp: 'Jul 12 01:37:55'
    hostname: 'wazuh-manager'
    program_name: 'ImpervLogs'

**Phase 2: Completed decoding.
    name: 'imperva-dec'
    fileid: '8218430560168672618'
    siteid: '8172810764'
    sourceServiceName: 'staticsso.104.ua'
    suid: '1007608'

Regarding Imperva multiline logs, you can use multiline-regex in order to split the logs into multiple events. This specifies a regular expression, match criteria, and replaces option for logs with a variable amount of lines.

If you need help with some of these steps do not hesitate to ask.

​

Nataliia

unread,
Jul 14, 2022, 4:47:06 AM7/14/22
to Wazuh mailing list
Hi Victor!

I've added to localfile out_format string:
  <localfile>
    <log_format>syslog</log_format>
    <out_format>$(timestamp) wazuh-manager ImpervLogs: $(log)</out_format>
    <location>/tmp/processed</location>
  </localfile>

But result the same - logs are not decoded.

вторник, 12 июля 2022 г. в 19:12:38 UTC+3, victor....@wazuh.com:

victor....@wazuh.com

unread,
Jul 15, 2022, 3:38:28 AM7/15/22
to Wazuh mailing list

Try replacing your parent decoder imperva-dec with the following

<decoder name="imperva-dec">
    <program_name>ImpervLogs</program_name>
</decoder>
​

Nataliia

unread,
Jul 15, 2022, 5:47:24 AM7/15/22
to Wazuh mailing list

The same result -  No result found
пятница, 15 июля 2022 г. в 10:38:28 UTC+3, victor....@wazuh.com:

victor....@wazuh.com

unread,
Jul 18, 2022, 5:15:19 AM7/18/22
to Wazuh mailing list

Let’s troubleshoot this issue step by step:
1.- Configure out_format using the configuration specified in my last message.
2.- Enable log_all option

    <logall>yes</logall>

3.- Restart the Wazuh manager
4.- Write the event specified in your first message in the monitored file.
5.- Check in the archives file the full log generated. Check if the header has been created correctly (18 09:07:18 wazuh-manager ImpervLogs:)

2022 Jul 18 09:07:18 wazuh-manager->/tmp/testing5.log Jul 18 09:07:18 wazuh-manager ImpervLogs: Jul 12 11:42:46 wazuh-manager : 0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=8218430560168672618 sourceServiceName=staticsso.104.ua siteid=8172810764 suid=1007608 requestClientApplication=Mozilla/5.0 (Linux; Android 11; Redmi Note 8 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Mobile Safari/537.36 deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=true cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=985a08ac-3477-47f4-804e-0d3d8681ce75 cs4Label=VID cs5=bde5ce13e361f6652323cb3d242b0eb7160a3b7452f0a4e596d5b8cde7ce3763ea90592cb34ec6a659ca48297729c984e3e0aca21872c43571ad8ba0171d68c68d7a6f9175d75123cafa3a90c88f7626bbfbca3c8f6cf6395ded9130ed98ad311b0fe104582e5dc560694c4a68028af6 cs5Label=clappsig dproc=Browser cs6=Mobile Chrome cs6Label=clapp ccode=GB cs7=51.4964 cs7Label=latitude cs8=-0.1224 cs8Label=longitude Customer=RGC start=1656892300199 request=staticsso.104.ua/build/sso/img/favicon/favicon-32x32.png ref=https://account.104.ua/ requestMethod=GET cn1=200 app=HTTPS act=REQ_PASSED deviceExternalId=796510916892689834 sip=46.164.130.243 spt=443 in=1518 xff=5.148.126.186 cpt=46552 src=5.148.126.186 ver=TLSv1.3 TLS_AES_128_GCM_SHA256 end=1656892300243#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324   suid=1903608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892216761 request=104.ua/robots.txt requestMethod=GET cn1=200 app=HTTPS act=REQ_CACHED_FRESH deviceExternalId=716457396338874765 in=110 cpt=34316 src=192.99.100.210 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 end=1656892216762#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId= 8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324 suid=2909608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892219823 request=104.ua/ua/news/id/У Вінницькій області 72% мешканців м-14067 requestMethod=GET cn1=301 app=HTTPS act=REQ_PASSED deviceExternalId=716480375800446128 sip=89.162.145.3 spt=443 in=290 xff=192.99.100.210 cpt=34316 src=192.99.100.210 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 end=1656892219980#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId= 8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324 suid= 2909608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892224558 request=104.ua/ua/news requestMethod=GET cn1=200 app=HTTPS act=REQ_PASSED deviceExternalId=825542562284965326 sip=46.164.130.243 spt=443 in=8325 xff=192.99.100.210 cpt=34354 src=192.99.100.210 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 end=1656892224779#000<14>

6.- Run the logtest tool and check if your decoder works

    full event: 'Jul 18 09:07:18 wazuh-manager ImpervLogs: Jul 12 11:42:46 wazuh-manager : 0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=8218430560168672618 sourceServiceName=staticsso.104.ua siteid=8172810764 suid=1007608 requestClientApplication=Mozilla/5.0 (Linux; Android 11; Redmi Note 8 Pro) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Mobile Safari/537.36 deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=true cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=985a08ac-3477-47f4-804e-0d3d8681ce75 cs4Label=VID cs5=bde5ce13e361f6652323cb3d242b0eb7160a3b7452f0a4e596d5b8cde7ce3763ea90592cb34ec6a659ca48297729c984e3e0aca21872c43571ad8ba0171d68c68d7a6f9175d75123cafa3a90c88f7626bbfbca3c8f6cf6395ded9130ed98ad311b0fe104582e5dc560694c4a68028af6 cs5Label=clappsig dproc=Browser cs6=Mobile Chrome cs6Label=clapp ccode=GB cs7=51.4964 cs7Label=latitude cs8=-0.1224 cs8Label=longitude Customer=RGC start=1656892300199 request=staticsso.104.ua/build/sso/img/favicon/favicon-32x32.png ref=https://account.104.ua/ requestMethod=GET cn1=200 app=HTTPS act=REQ_PASSED deviceExternalId=796510916892689834 sip=46.164.130.243 spt=443 in=1518 xff=5.148.126.186 cpt=46552 src=5.148.126.186 ver=TLSv1.3 TLS_AES_128_GCM_SHA256 end=1656892300243#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324   suid=1903608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892216761 request=104.ua/robots.txt requestMethod=GET cn1=200 app=HTTPS act=REQ_CACHED_FRESH deviceExternalId=716457396338874765 in=110 cpt=34316 src=192.99.100.210 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 end=1656892216762#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId= 8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324 suid=2909608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892219823 request=104.ua/ua/news/id/У Вінницькій області 72% мешканців м-14067 requestMethod=GET cn1=301 app=HTTPS act=REQ_PASSED deviceExternalId=716480375800446128 sip=89.162.145.3 spt=443 in=290 xff=192.99.100.210 cpt=34316 src=192.99.100.210 ver=TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 end=1656892219980#000<14>CEF:0|Incapsula|SIEMintegration|1|1|Normal|0| fileId= 8766000550275392531 sourceServiceName=www.104.ua siteid= 8176510324 suid= 2909608 requestClientApplication=Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) deviceFacility=lon cs2=false cs2Label=Javascript Support cs3=false cs3Label=CO Support cs1=NA cs1Label=Cap Support cs4=681e5b3e-41d2-41d9-a441-20b425b62a1f cs4Label=VID cs5=a069314a28fc3f38df1a7fd08797ff70400c236c3f43c214a588d2c6b92fada93f21b37a01969be556f0370e4534fbd14969aa1f882f5680157c6c2cf9ac15cc cs5Label=clappsig dproc=Unclassified cs6=Bot cs6Label=clapp ccode=CA cicode=Montreal cs7=45.5725 cs7Label=latitude cs8=-73.6195 cs8Label=longitude Customer=RGC start=1656892224558 request=104.ua/ua/news requestMethod=GET cn1=200 app=HTTPS act=REQ_PASSED deviceExternalId=825542562284965326 sip=46.164.130.243 spt=443 in=8325 xff=192.99.100.210 cpt=34354 src=192.99.'
    timestamp: 'Jul 18 09:07:18'
    hostname: 'wazuh-manager'
    program_name: 'ImpervLogs'

**Phase 2: Completed decoding.
    name: 'imperva-dec'
    fileid: '8218430560168672618'
    siteid: '8172810764'
    sourceServiceName: 'staticsso.104.ua'
    suid: '1007608'

If this process does not work in your environment, please indicate which of these steps produces not expected results, sending back again your decoder, the event received in the archives.log, and the logtest result.

My local_decoder.xml

<decoder name="imperva-dec">
    <program_name>ImpervLogs</program_name>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>fileid=(\d+)</regex>
    <order>fileid</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>sourceServiceName=(\S+)</regex>
    <order>sourceServiceName</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>siteid=(\d+)</regex>
    <order>siteid</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>suid=(\d+)</regex>
    <order>suid</order>
</decoder>
​

Nataliia

unread,
Jul 19, 2022, 7:17:26 AM7/19/22
to Wazuh mailing list
1.- Out-format configuring:
  <localfile>
    <log_format>syslog</log_format>
    <out_format>$(timestamp) wazuh-manager ImpervLogs: $(log)</out_format>
    <location>/tmp/processed</location>
  </localfile>


/tmp/processed - directory on the wazuh server, which collected all logs from Imperva.

2 - 3 done
4.- Run service, which allow to collect logs from Impeva to Wazuh server
5.- There are 4 diferent types of headers, which I found in archives.log:
1) 2022 Jul 19 10:37:22 wazuh-manager->/var/log/messages Jul 19 10:37:22 wazuh-manager
2) 2022 Jul 19 10:37:22 wazuh-manager->/var/log/messages Jul 19 10:37:22 wazuh-manager CEF: 0|Incapsula|SIEMintegration|1|1|Normal|0|
3) 2022 Jul 19 10:37:22 8a7132d1c3ac72a6a154859414c2d3cfdbe94342abedaa8e4ad42d24bb6b91d79b748e4c2daccd57004b1aa935fd39bcf81a229b0bea0815fbc6949c395caaf73a1940644c865a31eccbc435e8c1f5313ac0f8510b739c7ce89dd2baa69ddd5a044c->/var/log/messages Jul 19 10:37:22 8a7132d1c3ac72a6a154859414c2d3cfdbe94342abedaa8e4ad42d24bb6b91d79b748e4c2daccd57004b1aa935fd39bcf81a229b0bea0815fbc6949c395caaf73a1940644c865a31eccbc435e8c1f5313ac0f8510b739c7ce89dd2baa69ddd5a044c
4) 2022 Jul 19 10:37:24 03608->/var/log/messages Jul 19 10:37:22 03608
6.- Logs test result - No result found for all of them headers and logs, which writted after this headers.
 
My decoder:
<decoder name="imperva-dec">
    <program_name>ImpervLogs</program_name>
</decoder>

<decoder name="imperva-dec">
    <prematch>ImpervLogs</prematch>

</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>fileid=(\d+)</regex>
    <order>fileid</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>sourceServiceName=(\S+)</regex>
    <order>sourceServiceName</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>siteid=(\d+)</regex>
    <order>siteid</order>
</decoder>

<decoder name="imperva-dec-child">
    <parent>imperva-dec</parent>
    <regex>suid=(\d+)</regex>
    <order>suid</order>
</decoder>

I've added to decoder not all regex because it have not setted up headers yet.

Pieces of logs:
1) 2022 Jul 19 10:37:22 wazuh-manager->/var/log/messages Jul 19 10:37:22 wazuh-manager 6Label=clapp ccode=US cicode=Fremont cs7=37.6435 cs7Label=latitude cs8=-1223.0004
2) 2022 Jul 19 10:37:22 wazuh-manager->/var/log/messages Jul 19 10:37:22 wazuh-manager CEF: 0|Incapsula|SIEMintegration|1|1|Normal|0| fileId=1296401285934712733 sourceServiceName=static.104.ua siteid=51968424 suid=1996678 requestClientApplication=Mozilla/5.0
3) 2022 Jul 19 10:37:22 8a7132d1c3ac72a6a159659414c2d3cfgfe94342abedaa8e4ad42d24bd6b91d79b748e4c2daccd57004b1aa935fd39bcf81a229b0bea0815fbc6949c395caaf73a1940644c865a31eccbc435e8c1f5313ac0f3610b739c7ce89dd2bbb69ddd5a044c->/var/log/messages Jul 19 10:37:22  8a7132d1c3ac72a6a159659414c2d3cfgfe94342abedaa8e4ad42d24bd6b91d79b748e4c2daccd57004b1aa935fd39bcf81a229b0bea0815fbc6949c395caaf73a1940644c865a31eccbc435e8c1f5313ac0f3610b739c7ce89dd2bbb69ddd5a044c  cs5Label=clappsig dproc=Browser cs6=Chrome cs6Label=clapp ccode=US cicode=Los Gatos cs7=37.3695
4) 2022 Jul 19 10:37:24 03608->/var/log/messages Jul 19 10:37:22 03608 requestClientApplication=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36 deviceFacility=sjc cs2=false cs2Label=Javascript
понедельник, 18 июля 2022 г. в 12:15:19 UTC+3, victor....@wazuh.com:

victor....@wazuh.com

unread,
Jul 21, 2022, 3:56:47 AM7/21/22
to Wazuh mailing list

Checking the logs you are receiving, they come from/var/log/messages. The out_format option only will be applied to location value files in the configuration block.

If /tmp/processed is a directory, you should use a regex to monitor all the files contained in that path:

<localfile>
  <log_format>syslog</log_format>
  <out_format>$(timestamp) wazuh-manager ImpervLogs: $(log)</out_format>
  <location>/tmp/processed/*</location>
 </localfile>

This should apply the custom header created by the out_format option for all the events stored in the directory.

​

Nataliia

unread,
Jul 21, 2022, 10:55:03 AM7/21/22
to Wazuh mailing list
I've changed location to  <location>/tmp/processed/*</location>, enabled  log_all option to  <logall>yes</logall> and  restarted the Wazuh manager. But in the archives.log I saw the same headers and in logtest I have the same result.

четверг, 21 июля 2022 г. в 10:56:47 UTC+3, victor....@wazuh.com:

victor....@wazuh.com

unread,
Jul 25, 2022, 3:49:20 AM7/25/22
to Wazuh mailing list
If you see the same header, from the previous files (/var/log/messages) that means that you are not receiving any log in the processed directory. To ensure the out_format option is correctly configured  follow these steps:

  • Enable debug mode for logcollector module

    logcollector.debug=2

  • Create a file empty file/tmp/processed/example_imperva.log

  • Restart wazuh-manager

  • Add a testing log line to the file

    echo "Testing custom header" >> /tmp/processed/example_imperva.log

  • Check that in the ossec.log file appears the following message

    2022/07/25 07:42:23 wazuh-logcollector[7695] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'Testing custom header'

  • Check that in your archives.log appears something similar to

    2022 Jul 25 07:42:23 wazuh-manager->/tmp/processed/example_imperva.log Jul 25 07:42:23 wazuh-manager ImpervLogs: Testing custom header

    If this testing process succeeds, the error is not in the custom header creation but in the Imperva logs creation.

​

Nataliia

unread,
Jul 25, 2022, 9:38:41 AM7/25/22
to Wazuh mailing list
Tell me, please, how can I enable debug mode for logcollector module?

понедельник, 25 июля 2022 г. в 10:49:20 UTC+3, victor....@wazuh.com:

victor....@wazuh.com

unread,
Jul 25, 2022, 12:37:18 PM7/25/22
to Wazuh mailing list

Add the line logcollector.debug=2 to the /var/ossec/etc/local_internal_options.conffile and restart the manager

​

Nataliia

unread,
Jul 28, 2022, 8:57:14 AM7/28/22
to Wazuh mailing list
I've done what you wrote and saw that in the ossec.log file appeared the following message
2022/07/28 15:43:45 wazuh-logcollector[23895] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'echo "Testing custom header"'

But archives.log is empty. Should had I enable logall option befor it?
понедельник, 25 июля 2022 г. в 19:37:18 UTC+3, victor....@wazuh.com:

victor....@wazuh.com

unread,
Aug 2, 2022, 3:45:10 AM8/2/22
to Wazuh mailing list

Yes. You can enable the logall option in the /var/ossec/etc/ossec.conf configuration file:

  <logall>yes</logall>

Then restart your wazuh-manager

Please, send back the logline of the testing message in the archives. If the header is correctly specified, that will means that your configuration is fine, and you should check where your Imperva logs are generating.

​

Nataliia

unread,
Aug 10, 2022, 9:54:34 AM8/10/22
to Wazuh mailing list
Hello!
I've done this steps again, and this time I enabled the logall option.  

In ossec.log I didn't see any line like this - 2022/07/28 15:43:45 wazuh-logcollector[23895] read_syslog.c:104 at read_syslog(): DEBUG: Reading syslog message: 'echo "Testing custom header"'

And in archives.log I didn't see any line this - 2022 Jul 25 07:42:23 wazuh-manager->/tmp/processed/example_imperva.log Jul 25 07:42:23 wazuh-manager ImpervLogs: Testing custom header
вторник, 2 августа 2022 г. в 10:45:10 UTC+3, victor....@wazuh.com:

Nataliia

unread,
Aug 18, 2022, 11:00:00 AM8/18/22
to Wazuh mailing list
Hi there!

Is any updates for my issue?

среда, 10 августа 2022 г. в 16:54:34 UTC+3, Nataliia:
Reply all
Reply to author
Forward
0 new messages