PagerDuty integration skipping alerts.

[S. Grnvld]

Good morning,

Today I noticed that not all level 10 alerts are coming into PagerDuty.

PagerDuty APIv1 & APIv2 return the same result.

We are running a cluster (1 master - 1 worker) with Wazuh 4.0.4 (CentOS 7) and have only 1 integration (PagerDuty).

ossec.conf shows the following (API key marked black):


Thanks for your help in advance!

[Jose Luis Carreras Marin]

Hello S. Grnvld:

Let's analyze this issue in depth, first let's make sure that the alerts are being generated correctly to send them to integratord, which is the daemon that is responsible for sending the alerts to the script, it is possible that some alerts are being discarded because they are missing some parameters . For it, we must activate the debug mode of integratord:

- In the /var/ossec/etc/internal_options.conf file of the manager.

- Change the line integratord.debug=0 -> integratord.debug=2.

- Restart Wazuh manager.

Now in the Wazuh log file, /var/ossec/logs/ossec.log you can see all the debug messages, can you show it to me?

It would also be helpful to know what kind of alerts you are expecting, for example.

In this blogpost you can read in depth all the necessary steps, in case you see something that can help you:

https://wazuh.com/blog/how-to-integrate-external-software-using-integrator/

Integratord documentation:
https://documentation.wazuh.com/current/user-manual/manager/manual-integration.html

Regards, Jose.

[Gaat je niets aan]

Hi Jose,

I did the following:
1. created a new API key (temp) in PagerDuty;

2. adjusted the ossec.conf file;

3. moved back to the old API key in the config file;

4. restarted the managers and alerts started to flow back in.

Seems it got stuck somewhere.

--
You received this message because you are subscribed to a topic in the Google Groups "Wazuh mailing list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/wazuh/u2B8I1UmKwM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to wazuh+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/288fc46c-99b7-49cd-bd5e-5f1ee60906efn%40googlegroups.com.

--

Swen Groeneveld

[Jose Luis Carreras Marin]

Hi Swen Groeneveld,

Great then the problem is solved.
I'm glad, if you have any other issue, don't hesitate to ask the Wazuh team!!!

See you soon!!

[Gaat je niets aan]

Hi Jose,

So this morning I determined that I did again have missed level 10 alerts. As I am running Wazuh 4.0.4. I went through the releases after this version and saw that in the release relating to v 4.1.1; the following was mentioned:

• A bug in Integratord that might lose alerts from Analysisd due to a race condition.

As a result I have updated all to the latest version and hope all will go well over the weekend. 

To view this discussion on the web visit https://groups.google.com/d/msgid/wazuh/98599ba5-61d6-4462-adc6-03623c30e8e9n%40googlegroups.com.

--

Swen Groeneveld

[S. Grnvld]

Negative! I have still missing alerts. I will try to get the logs, my guess here is that the tool is too busy with going through all the events (we got more than 2000 servers and  8258815 events in the last 24 hours)

[Jose Luis Carreras Marin]

Hi Swen Groeneveld

Let's go back and look for these problems then.

As I said in the first message, we should look at the logs to see if they tell us anything.

What kind of events are you missing? Are they File Integrity Monitoring events? If so, we can also enable the syscheck debug (FIM) with syscheck.debug=2.

At first glance, it looks like the events are being flooded.