Unknown problem somewhere in the system.

[Milan Patel]

Hello, Does any one know why I get this kind of error ...

I want to see rsyslog on wazuh dashboard but I am unable to see that. Please can someone help me to find out what is the real issue behind this ? 

I think I am getting this error for only vhosts/proxy ... rest logs works fine .. not sure why is this I am getting.

** Alert 1698114112.4512443: - syslog,errors,gpg13_4.3,
2023 Oct 24 02:21:52 wazuh50->/var/ossec/logs/archives/archives.log
Rule: 1002 (level 2) -> 'Unknown problem somewhere in the system.'
2023 Oct 24 02:21:50 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:48 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:46 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:44 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:42 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:40 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:38 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:36 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:34 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:32 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:30 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:28 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:26 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:24 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:22 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:20 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:18 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:16 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:14 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:12 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:10 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:08 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:06 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:04 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:02 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:21:00 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:58 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:56 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:54 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:52 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:50 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:48 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:46 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:44 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:42 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:40 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:38 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:36 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:34 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:32 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:30 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:28 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:26 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:24 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:22 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:20 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:18 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:16 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:14 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:12 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:10 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:08 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:06 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:04 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:02 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:20:00 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:58 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:55 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:53 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:51 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:49 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:47 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:45 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:43 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:41 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:39 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:37 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:35 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:33 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:31 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:29 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:27 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:25 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:23 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:21 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:19 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:17 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:15 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:13 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:11 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:09 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:07 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:05 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:03 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:19:01 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:59 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:57 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:55 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:53 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:51 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:49 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:47 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:45 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:43 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:41 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:39 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:37 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:35 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:33 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:31 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:29 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:27 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:25 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:23 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:21 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:19 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:17 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:15 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:13 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:11 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:09 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:07 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:05 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:03 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:18:01 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:59 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:57 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:55 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:53 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:51 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:49 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:47 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:45 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:43 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:41 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:39 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:37 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:35 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:33 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:31 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:29 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:29 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:27 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:25 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:23 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:21 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:19 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:19 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:17 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:15 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:13 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:13 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:11 wazuh50->/var/ossec/logs/archives/archives.log 2023 Oct 24 02:17:11 wazuh50->/var/log/syslog Oct 24 02:17:10 wazuh50 opensearch-dashboards[11766]: {"type":"error","@timestamp":"2023-10-24T02:17:10Z","tags":[],"pid":11766,"level":"error","error":{"message":"Internal Server Error","name":"Error","stack":"Error: Internal Server Error\n    at HapiResponseAdapter.toInternalError (/usr/share/wazuh-dashboard/src/core/server/http/router/response_adapter.js:80:19)\n    at Router.handle (/usr/share/wazuh-dashboard/src/core/server/http/router/router.js:177:34)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)\n    at handler (/usr/share/wazuh-dashboard/src/core/server/http/router/router.js:124:50)\n    at exports.Manager.execute (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/toolkit.js:60:28)\n    at Object.internals.handler (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/handler.js:46:20)\n    at exports.execute (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/handler.js:31:20)\n    at Request._lifecycle (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/request.js:371:32)\n    at Request._execute (/usr/share/wazuh-dashboard/node_modules/@hapi/hapi/lib/request.js:281:9)"},"url":"https://172.25.8.50/api/request","message":"Internal Server Error"}

[Manuel Jose Cano Rojo]

Hi Milan Patel,

The alert you are getting is triggered when a "Bad word" such as failure, error bad, denied, or others, comes in an analyzed event by Wazuh. So it's a very generic alert. Check the logs created by each Wazuh component to find more information. Here I let you know the location of the different .log files, checking them should be useful.

   Wazuh indexer: /var/log/wazuh-indexer 

   Wazuh manager: /var/ossec/logs and /var/log/filebeat 

   Wazuh dashboard: /usr/share/wazuh-dashboard/data/wazuh/logs

Regards,

Manuel.

[Milan Patel]

Hi Manuel,

Thank you so much for replying back.

I am trying very hard to solve my issue but not getting anywhere.

I can see logs like if I do anything in system then I do get those logs on dashboard. Like useradd , user del, doing ssh, exit out of session etc those I can see on dashboard but my apache logs are not showing up on dashboard. I can see all logs are coming to /var/ossec/logs/archives/archives.json | log.

Also have another question. Which log file used by Wazuh-dashboard to read those logs ?
How can I troubleshoot this ?

Based on this file /var/ossec/logs/archives/archives.json or log file for which file should I write rules and decoders. When I do logtest it shows alerts to be generated.

What things we have to check to make sure my wazuh deployment is good without any error.

Sorry for asking such a silly questions but I am new to this tool and trying to understand so that I can deploy it on Production env.

Thanks

Milan Patel

[Manuel Jose Cano Rojo]

Hi Milan Patel,

Don't worry about your questions, we are here to help! 

The Wazuh dashboard logs are stored in this location /usr/share/wazuh-dashboard/data/wazuh/logs/wazuhapp.log.

 

All the default rules and decoders are stored in several .XML files located in /var/ossec/ruleset/, but you should not modify the rules in this folder, since modifications will disappear if you upgrade your Wazuh version. Have a look at this link that will show you how to modify custom decoders and rules.

Regards,

Manuel.