Google Workspace Status : Google Meet - AVAILABLE - 2025-10-02 02:01:02 UTC

[back...@gmail.com]

Product: Google Meet
Status: AVAILABLE
Published: 2025-10-02 02:01:02 UTC

# Incident Report
## Summary
On Friday, 26 September 2025 at 08:40 US/Pacific, Google Meet users
experienced elevated error rates globally for 1 hour, 1 minute. To our
Google Meet customers who were impacted during this disruption, we
sincerely apologize. This is not the level of quality and reliability
we strive to offer you, and we are taking immediate steps to improve
the platform's performance and availability.
## Root Cause
The root cause of the incident was a flaw in the error-handling logic
within the experimentation module's configuration evaluation. This
flaw created the conditions for resource contention, where too many
processes attempted to access the same system resource at once.
The trigger was a specific change to the Meet experimentation
configuration file, which caused an elevated error level during
evaluation and led to the contention in the Meet backend. Our
monitoring systems detected the issue right away, and engineers
quickly identified the change as the source. They reverted the update,
which resolved the issue.
## Remediation and Prevention
Google engineers were alerted to the issue through the internal alerts
at 08:49 US/Pacific on 26 September, 2025 and immediately started an
investigation. They identified several recent changes as potential
causes, and began systematically rolling back those changes.
At 09:25 US/Pacific, engineers reverted the Meet API Server binary to
a previous version.
At 09:41 US/Pacific, engineers reverted and paused subsequent Meet
backend API server configuration rollouts across all Google Meet
production clusters thus mitigating the issue.
At 11:52 US/Pacific, engineers reverted the change that caused lock contention.
We sincerely apologize to our customers who were impacted by this
disruption. This is not the level of quality and reliability we strive
to offer, and we are taking the following steps to ensure this does
not happen again. The Google Meet Engineering team is accelerating the
deprecation of legacy code that caused lock contention.
## Detailed Description of Impact
On Friday, 26 September 2025, from 08:40 to 09:41 US/Pacific, users
were either unable to connect to Google Meet or experienced problems
with establishing meetings. Users already in Google Meet sessions were
likely not impacted. The issue affected up to 20 percent of Google
Meet users globally.

Link to official detail of this status:
https://www.google.com/appsstatus/dashboard/incidents/Cy8B1om17oecUkKm3G9Q