Alertmanager in CrashLoopBackOff state when deployed in Openshift kubernetes dualstack environment

101 views
Skip to first unread message
Message has been deleted

anantha sai ram

unread,
Dec 21, 2022, 10:00:00 AM12/21/22
to promethe...@googlegroups.com

Hi,

Alertmanager pod is stuck with CrashLoopBackOff in Openshift kubernetes dualstack environment when the alertmanager service uses the same port(9094) for TCP and UDP. Same build works fine on Openshift kubernetes non-dualstack environment.

Environment:
Openshift cluster with both IPv4 and IPv6 interface.
Aspenmesh-1.11.8-am4

alertmanager - v0.24.0

Observation:

For alertmanager service, when tried with different ports for TCP(9094) and UDP(9095) the deployment is successful.

Query:

  1. Is there a need to use different ports for UDP and TCP protocols for alertmanager service when deployed in a dualstack k8s environment?
  2. If yes, how does the configured UDP port affect the traffic flow to the alertmanager service?  

This issue is happening on a fresh installation(not upgrade/rollback)
In side-car logs of alertmanager pod,

    warning envoy config    gRPC config for xxxxxx rejected: Error adding/updating listener(s) virtualInbound: error adding listener '0.0.0.0:15006': filter chain '0.0.0.0_9094' has the same matching rules defined as '0.0.0.0_9094'
virtualInbound6: error adding listener '[::]:15006': filter chain '::_9094' has the same matching rules defined as '::_9094' 

In istiod logs

    info    ads     ADS: new connection for node:alertmanager-0
   info    ads     CDS: PUSH request for node:alertmanager-0 size:365.9kB
   info    ads     EDS: PUSH request for node:alertmanager-0 resources:140 size:62.8kB empty:0 cached:140/140
     info    ads     LDS: PUSH request for node:alertmanager-0 size:648.6kB
     info    ads     RDS: PUSH request for node:alertmanager-0 resources:23 size:43.8kB
    warn    ads     ADS:LDS: ACK ERROR alertmanager-0 Internal:Error adding/updating listener(s) virtualInbound: error adding listener '0.0.0.0:15006': filter chain '0.0.0.0_9094' has the same matching rules defined as '0.0.0.0_9094'
   info    ads     ADS: x.x.x.x:xxxx alertmanager-0 terminated rpc error: code = Canceled desc = context canceled 
Reply all
Reply to author
Forward
0 new messages