Seeking Assistance with Prometheus to Splunk Integration Issue: "NO DATA" Error (Error Code 5)

[Aditya Sharma]

Hi Team,

We're currently encountering a roadblock in our efforts to integrate Prometheus alerts into Splunk. Every attempt at integration results in an error labeled "NO DATA," accompanied by error code 5.

Here's a snippet of the data we're trying to integrate:

{
    "receiver": "splunk-webhook",
    "status": "firing",
    "alerts": [{
        "status": "firing",
        "labels": {
            "alertname": "TEST",
            "env": "isdt-sbx",
            "namespace": "isdt-sbxtest",
            "severity": "critical"
        },
        "annotations": {
            "description": "description of the alert",
            "runbook": "http://runbook.biz",
            "summary": "summary of the alert"
        },
        "startsAt": "2024-02-26T12:38:53.724141255Z",
        "endsAt": "0001-01-01T00:00:00Z",
        "generatorURL": "",
        "fingerprint": "e6f0eaf72b9d568c"
    }],
    "groupLabels": {
        "alertname": "TEST",
        "namespace": "isdt-sbxtest"
    },
    "commonLabels": {
        "alertname": "TEST",
        "env": "isdt-sbx",
        "namespace": "isdt-sbxtest",
        "severity": "critical"
    },
    "commonAnnotations": {
        "description": "description of the alert",
        "runbook": "http://runbook.biz",
        "summary": "summary of the alert"
    },
    "externalURL": "https://monitoring.server.net/alertmanager",
    "version": "4",
    "groupKey": "{}/{severity=~\"^(?:critical|Critical|info|Critica)$\"}:{alertname=\"TEST\", namespace=\"isdt-sbxtest\"}",
    "truncatedAlerts": 0
}


Despite our efforts, we haven't been able to pinpoint the root cause of this issue. Our goal is to seamlessly integrate Prometheus alerts into Splunk for enhanced monitoring and analysis.

If anyone in the group has encountered a similar issue or has expertise in Prometheus to Splunk integration, we would greatly appreciate your insights and recommendations on resolving this challenge.

Thanks & Regards,
Aditya Sharma

[Aditya Sharma]