issue with virustotal integration

[Amine HADJAMAR]

Hi,
I try to use virustotal but i don't have any wazuh alert when i try some debugging i get this error:
# /var/ossec/bin/wazuh-integratord -fdd

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:133 at OS_IntegratorD(): DEBUG: sending new alert.
2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:244 at OS_IntegratorD(): DEBUG: skipping: rule doesn't match.
2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:126 at OS_IntegratorD(): DEBUG: jqueue_next()
2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:133 at OS_IntegratorD(): DEBUG: sending new alert.
2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:267 at OS_IntegratorD(): DEBUG: file /tmp/virustotal-1674671184-1953432986.alert was written.
2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:394 at OS_IntegratorD(): DEBUG: Running: integrations /tmp/virustotal-1674671184-1953432986.alert 94e298694ac4a58c323839dcb1ef4cfe36b48bd6beb8a5c7c3aee587117865cf  debug
2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord: Wed Jan 25 19:26:24 CET 2023: # Starting

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord: Wed Jan 25 19:26:24 CET 2023: # API Key

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord: Wed Jan 25 19:26:24 CET 2023: 94e298694ac4a58c323839dcb1ef4cfe36b48bd6beb8a5c7c3aee587117865cf

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord: Wed Jan 25 19:26:24 CET 2023: # File location

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord: Wed Jan 25 19:26:24 CET 2023: /tmp/virustotal-1674671184-1953432986.alert

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord: Wed Jan 25 19:26:24 CET 2023: 'utf-8' codec can't decode byte 0xe8 in position 878: invalid continuation byte

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord: Traceback (most recent call last):

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:   File "/var/ossec/integrations/virustotal.py", line 208, in <module>

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:     main(sys.argv)

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:   File "/var/ossec/integrations/virustotal.py", line 59, in main

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:     json_alert = json.load(alert_file)

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:   File "/var/ossec/framework/python/lib/python3.9/json/__init__.py", line 293, in load

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:     return loads(fp.read(),

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:   File "/var/ossec/framework/python/lib/python3.9/codecs.py", line 322, in decode

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord:     (result, consumed) = self._buffer_decode(data, self.errors, final)

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:403 at OS_IntegratorD(): DEBUG: integratord: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 878: invalid continuation byte

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:412 at OS_IntegratorD(): ERROR: Unable to run integration for virustotal -> integrations
2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:413 at OS_IntegratorD(): ERROR: While running virustotal -> integrations. Output: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 878: invalid continuation byte

2023/01/25 19:26:24 wazuh-integratord[9652] integrator.c:414 at OS_IntegratorD(): ERROR: Exit status was: 1

i use wazuh 4.3 on centos 7

thanks,

[Santiago David Vendramini]

Hi! Can you share your virustotal configuration? Hidding your private api key. Have you checked that the alerts configured to trigger the integration appear in /var/ossec/logs/alerts/alerts.json?

[Amine HADJAMAR]

Hi Santiago,
Thanks for your quick reply,
my ossec.conf:
 
<integration>
  <name>virustotal</name>
  <api_key>MY_API_KEY</api_key>
  <group>syscheck</group>
  <alert_format>json</alert_format>
</integration>
 
There are no relevant alerts in the alerts.json file
The above api key is not mine I have modified it

Thanks,

[Santiago David Vendramini]

I recomend you check this documentation about virustotal integration working with FIM: https://documentation.wazuh.com/current/user-manual/capabilities/virustotal-scan/integration.html?highlight=virustotal#use-case-scanning-a-file. The integration is triggered when some file is added/removed/edited in the directories monitored by syscheck. You will see in alerts.json a syscheck alert and then a virustotal alert. Can you check these conditions?

[Amine HADJAMAR]

Hi,

for the FIM works normally I can read the alert in the file alerts.log, for example:
** Alert 1674762312.631828: - ossec,syscheck,syscheck_entry_added,syscheck_file,pci_dss_11.5,gpg13_4.11,gdpr_II_5.1.f,hipaa_164.312.c.1,hipaa_164.312.c.2,nist_800_53_SI.7,tsc_PI1.4,tsc_PI1.5,tsc_CC6.1,tsc_CC6.8,tsc_CC7.2,tsc_CC7.3,
2023 Jan 26 20:45:12 (HOME1) any->syscheck
Rule: 554 (level 5) -> 'File added to the system.'
File 'c:\test\55555.exe' added
Mode: realtime

Attributes:
 - Size: 0
 - Permissions: Administrateurs (allowed): DELETE|READ_CONTROL|WRITE_DAC|WRITE_OWNER|SYNCHRONIZE|READ_DATA|WRITE_DATA|APPEND_DATA|READ_EA|WRITE_EA|EXECUTE|READ_ATTRIBUTES|WRITE_ATTRIBUTES, Système (allowed): DELETE|READ_CONTROL|WRITE_DAC|WRITE_OWNER|SYNCHRONIZE|READ_DATA|WRITE_DATA|APPEND_DATA|READ_EA|WRITE_EA|EXECUTE|READ_ATTRIBUTES|WRITE_ATTRIBUTES, Utilisateurs (allowed): READ_CONTROL|SYNCHRONIZE|READ_DATA|READ_EA|EXECUTE|READ_ATTRIBUTES, Utilisateurs authentifiés (allowed): DELETE|READ_CONTROL|SYNCHRONIZE|READ_DATA|WRITE_DATA|APPEND_DATA|READ_EA|WRITE_EA|EXECUTE|READ_ATTRIBUTES|WRITE_ATTRIBUTES
 - Date: Thu Jan 26 20:45:04 2023
 - Inode: 0
 - User: Admin (S-1-5-21-1502343767-2352192368-2839685740-1000)
 - MD5: d41d8cd98f00b204e9800998ecf8427e
 - SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
 - SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
 - File attributes: ARCHIVE

but no virustotal alerts appears as the documentation says!

in the integrations.log i still have the error :

Thu Jan 26 20:45:13 CET 2023: # Starting
Thu Jan 26 20:45:13 CET 2023: # API Key
Thu Jan 26 20:45:13 CET 2023: MY_API
Thu Jan 26 20:45:13 CET 2023: # File location
Thu Jan 26 20:45:13 CET 2023: /tmp/virustotal-1674762313--1262485892.alert
Thu Jan 26 20:45:13 CET 2023: 'utf-8' codec can't decode byte 0xe8 in position 888: invalid continuation byte

and the same for the ossec.log file

thanks,

[Santiago David Vendramini]

Can you send me the same alert from the alerts.json file? Because the integration reads the alerts.json file to trigger the integration script. Maybe there is some character there that causes the problem! 

[Amine HADJAMAR]

Hi,

yes i think so, the problem is when the json file is loaded.

json file alert:

{"timestamp":"2023-01-26T20:45:12.750+0100","rule":{"level":5,"description":"File added to the system.","id":"554","firedtimes":3,"mail":false,"groups":["ossec","syscheck","syscheck_entry_added","syscheck_file"],"pci_dss":["11.5"],"gpg13":["4.11"],"gdpr":["II_5.1.f"],"hipaa":["164.312.c.1","164.312.c.2"],"nist_800_53":["SI.7"],"tsc":["PI1.4","PI1.5","CC6.1","CC6.8","CC7.2","CC7.3"]},"agent":{"id":"001","name":"HOME1","ip":"192.168.3.133"},"manager":{"name":"WAZUH.LAB"},"id":"1674762312.631828","full_log":"File 'c:\\test\\55555.exe' added\nMode: realtime\n","syscheck":{"path":"c:\\test\\55555.exe","mode":"realtime","size_after":"0","win_perm_after":[{"name":"Administrateurs","allowed":["DELETE","READ_CONTROL","WRITE_DAC","WRITE_OWNER","SYNCHRONIZE","READ_DATA","WRITE_DATA","APPEND_DATA","READ_EA","WRITE_EA","EXECUTE","READ_ATTRIBUTES","WRITE_ATTRIBUTES"]},{"name":"Système","allowed":["DELETE","READ_CONTROL","WRITE_DAC","WRITE_OWNER","SYNCHRONIZE","READ_DATA","WRITE_DATA","APPEND_DATA","READ_EA","WRITE_EA","EXECUTE","READ_ATTRIBUTES","WRITE_ATTRIBUTES"]},{"name":"Utilisateurs","allowed":["READ_CONTROL","SYNCHRONIZE","READ_DATA","READ_EA","EXECUTE","READ_ATTRIBUTES"]},{"name":"Utilisateurs authentifiés","allowed":["DELETE","READ_CONTROL","SYNCHRONIZE","READ_DATA","WRITE_DATA","APPEND_DATA","READ_EA","WRITE_EA","EXECUTE","READ_ATTRIBUTES","WRITE_ATTRIBUTES"]}],"uid_after":"S-1-5-21-1502343767-2352192368-2839685740-1000","md5_after":"d41d8cd98f00b204e9800998ecf8427e","sha1_after":"da39a3ee5e6b4b0d3255bfef95601890afd80709","sha256_after":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","attrs_after":["ARCHIVE"],"uname_after":"Admin","mtime_after":"2023-01-26T20:45:04","event":"added"},"decoder":{"name":"syscheck_new_entry"},"location":"syscheck"}

attached the json file 

thanks,

[Amine HADJAMAR]

Hi,

After doing some research, I found a python script with errors='ignore' on line 58 of virustotal.py :
open(alert_file_location, errors='ignore') as alert_file
original script : open(alert_file_location) as alert_file
so I tried it and it works fine now.
I dont know if it is a solution or just a workaround and why the original script dont use it?!

[Santiago David Vendramini]

Hi! This change that you mentioned is a fix for these types of errors and will be part of the next release!