Google Workspace Status : Google Meet - AVAILABLE - 2025-10-02 00:08:14 UTC

0 views
Skip to first unread message

back...@gmail.com

unread,
Oct 1, 2025, 8:15:28 PM (11 hours ago) Oct 1
to g-suite-st...@googlegroups.com
Product: Google Meet
Status: AVAILABLE
Published: 2025-10-02 00:08:14 UTC

# Incident Report
## Summary
On Friday, 26 September 2025 at 08:40 US/Pacific, Google Meet users
experienced elevated error rates globally for 1 hour, 1 minute. To our
Google Meet customers who were impacted during this disruption, we
sincerely apologize. This is not the level of quality and reliability
we strive to offer you, and we are taking immediate steps to improve
the platform's performance and availability.
## Root Cause
The root cause of the incident was a recent code change rolled out to
Google Meet's backend API Server. The change caused lock contention
due to an incompatibility with existing legacy code. This rollout was
performed in phases, and the issue was quickly detected by automated
monitoring. An automatic rollback successfully reverted the rollout in
the affected cluster. However, a separate backend API Server
configuration rollout also included the problematic change, which
prolonged the outage in the initially reverted cluster.
## Remediation and Prevention
Google engineers were alerted to the issue through the internal alerts
at 08:49 US/Pacific on 26 September, 2025 and immediately started an
investigation. They identified several recent changes as potential
causes, and began systematically rolling back those changes.
At 09:25 US/Pacific, engineers reverted the Meet API Server binary to
a previous version.
At 09:41 US/Pacific, engineers reverted and paused subsequent Meet
backend API server configuration rollouts across all Google Meet
production clusters thus mitigating the issue.
At 11:52 US/Pacific, engineers reverted the change that caused lock contention.
We sincerely apologize to our customers who were impacted by this
disruption. This is not the level of quality and reliability we strive
to offer, and we are taking the following steps to ensure this does
not happen again. The Google Meet Engineering team is accelerating the
deprecation of legacy code that caused lock contention.
## Detailed Description of Impact
On Friday, 26 September 2025, from 08:40 to 09:41 US/Pacific, users
were either unable to connect to Google Meet or experienced problems
with establishing meetings. Users already in Google Meet sessions were
likely not impacted. The issue affected up to 20 percent of Google
Meet users globally.

Link to official detail of this status:
https://www.google.com/appsstatus/dashboard/incidents/Cy8B1om17oecUkKm3G9Q
Reply all
Reply to author
Forward
0 new messages