Monitoring agent failing to authenticate since Jan 22 6pm UTC

[rthe...@virttrade.com]

Metrics haves stopped being reported since 6pm yesterday.  On investigating it seems to show an issue authenticating. Any ideas why this would suddenly happen - has something changed?

● stackdriver-agent.service - LSB: start and stop Stackdriver Agent

   Loaded: loaded (/etc/init.d/stackdriver-agent; generated)

   Active: active (running) since Sun 2021-07-25 14:05:15 UTC; 5 months 29 days ago

     Docs: man:systemd-sysv-generator(8)

    Tasks: 12 (limit: 1125)

   CGroup: /system.slice/stackdriver-agent.service

           └─1336 /opt/stackdriver/collectd/sbin/stackdriver-collectd -C /etc/stackdriver/collectd.conf -P /var/run/stackdriver-agent.pid

Jan 23 14:54:15 dh-3 collectd[1336]: write_gcm: Asking metadata server for auth token

Jan 23 14:54:15 dh-3 collectd[1336]: write_gcm: Error or buffer overflow when building auth_header

Jan 23 14:54:15 dh-3 collectd[1336]: write_gcm: wg_oauth2_get_auth_header failed.

Jan 23 14:54:15 dh-3 collectd[1336]: write_gcm: wg_transmit_unique_segment failed.

Jan 23 14:54:15 dh-3 collectd[1336]: write_gcm: wg_transmit_unique_segments failed. Flushing.

Jan 23 14:55:15 dh-3 collectd[1336]: write_gcm: Asking metadata server for auth token

Jan 23 14:55:15 dh-3 collectd[1336]: write_gcm: Error or buffer overflow when building auth_header

Jan 23 14:55:15 dh-3 collectd[1336]: write_gcm: wg_oauth2_get_auth_header failed.

Jan 23 14:55:15 dh-3 collectd[1336]: write_gcm: wg_transmit_unique_segment failed.

Jan 23 14:55:15 dh-3 collectd[1336]: write_gcm: wg_transmit_unique_segments failed. Flushing.

[rthe...@virttrade.com]

I see there is a new "ops" agent.  Have you turned off support for the legacy agents already?

[Igor Peshansky]

Hi. The standalone monitoring agent is still supported. This is a known issue with the legacy versions (5.5.2-*) of the standalone monitoring agent, triggered by a change in the GCE VM configuration that increases the size of the OAuth token served by the GCE metadata server. The issue has been fixed in agent version 6.1.2, so if you upgrade your deployments to a newer version, you should no longer see this happen.

        Igor

---

Registered in England no 07948120. Registered Office: 8 High Street, Heathfield, TN21 8LS

--
© 2021 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
 
Email preferences: You received this email because you signed up for the Google Stackdriver Discussion Google Group (google-stackdr...@googlegroups.com) to participate in discussions with other members of the GoogleStackdriver community.
---
You received this message because you are subscribed to the Google Groups "Google Stackdriver Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-stackdriver-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-stackdriver-discussion/056385ab-0596-4d33-a638-a86e9ca54eban%40googlegroups.com.

[rthe...@virttrade.com]

Thanks, that indeed seemed to be the problem and upgrading the agent fixed the issue.