Hello team,
Following up with our findings:
Right now, at least on this router, Gig0 / Mgmt-Intf VRF are being used for numerous services. This includes TACACS, HTTP for Smart Licensing, and NTP for Google.
At the time of the issue, we saw the input queue of Gig0 was wedged (full, no more packets can be processed):
[ show interface Gig 0 ]
GigabitEthernet0 is up, line protocol is up
Hardware is RP management port, address is 70df.2f98<>
Description: UPLINK TO CORE-C361-DC_G1/0/25
Internet address is <private 10.x>
…
Input queue: 76/75/12/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 4000 bits/sec, 5 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
This should not normally happen. We printed the interface buffer and found NTP protocol communication, from Google IP to Gig0 IP, had saturated the input queue.
We moved the NTP service to a different interface (Po).
Would suggest to do so for all ASRs with a similar configuration.