Intermittent errors for Managed VM (Beta) deployments
86 views
Skip to first unread message
Google Cloud Platform Status
Mar 9, 2015, 11:04:05 PM3/9/15
to google-appengine...@googlegroups.com
We are currently experiencing an intermittent issue with Google App Engine
Managed VM (Beta) deployments starting Monday 2015-03-09 18:30 (all times
are in US/Pacific). For everyone who is affected, we apologize for any
inconvenience you may be experiencing. We will provide an update by 20:30
with current details.
This issue is related to Cloud Storage incident:
https://status.cloud.google.com/incident/storage/16021 .
Managed VM (Beta) deployments starting Monday 2015-03-09 18:30 (all times
are in US/Pacific). For everyone who is affected, we apologize for any
inconvenience you may be experiencing. We will provide an update by 20:30
with current details.
This issue is related to Cloud Storage incident:
https://status.cloud.google.com/incident/storage/16021 .
Google Cloud Platform Status
Mar 9, 2015, 11:52:57 PM3/9/15
to google-appengine...@googlegroups.com
Managed VM (Beta) deployments have been functional since 19:45. We will
continue monitoring the issue until the Cloud Storage incident is resolved.
continue monitoring the issue until the Cloud Storage incident is resolved.
Google Cloud Platform Status
Mar 10, 2015, 2:59:56 PM3/10/15
to google-appengine...@googlegroups.com
The issue with the Cloud Storage JSON API was resolved as of 19:35 on
Monday 2015-03-09. We are sorry for any issues this may have caused to you
or your users and thank you for your patience and continued support.
We will provide a detailed analysis of this incident under Cloud Storage
incident 16021 once we have completed our internal investigation.
Monday 2015-03-09. We are sorry for any issues this may have caused to you
or your users and thank you for your patience and continued support.
We will provide a detailed analysis of this incident under Cloud Storage
incident 16021 once we have completed our internal investigation.
Google Cloud Platform Status
Mar 11, 2015, 8:28:40 PM3/11/15
to google-appengine...@googlegroups.com
SUMMARY:
On Monday 9 March 2015, deployments of Google App Engine applications
running on Beta Managed VMs experienced intermittent deployment failures
for a duration of 75 minutes. Whilst Managed VMs is still in Beta, we
apologize for any impact this incident may have had on your service or
application.
DETAILED DESCRIPTION OF IMPACT:
Between 18:20 and 19:40 PDT, users of Google App Engine Managed VMs (Beta)
experienced intermittent deployment failures. Users impacted by this
incident would see gcloud or appcfg deploy commands return with error
messages. Over this period, 81% of deployments returned errors.
ROOT CAUSE:
The Managed VMs (Beta) deployment system relies on the Google Cloud Storage
JSON API, which experienced a 5% failure rate during this time period. For
details, see the Cloud Storage incident post:
https://status.cloud.google.com/incident/storage/16021.
REMEDIATION AND PREVENTION:
The primary remediation for this incident is described in the Cloud Storage
incident report. In addition to the steps outlined there, Google engineers
are tuning the systems that monitor Managed VMs application deployments to
alert earlier, which will facilitate speedier resolution.
On Monday 9 March 2015, deployments of Google App Engine applications
running on Beta Managed VMs experienced intermittent deployment failures
for a duration of 75 minutes. Whilst Managed VMs is still in Beta, we
apologize for any impact this incident may have had on your service or
application.
DETAILED DESCRIPTION OF IMPACT:
Between 18:20 and 19:40 PDT, users of Google App Engine Managed VMs (Beta)
experienced intermittent deployment failures. Users impacted by this
incident would see gcloud or appcfg deploy commands return with error
messages. Over this period, 81% of deployments returned errors.
ROOT CAUSE:
The Managed VMs (Beta) deployment system relies on the Google Cloud Storage
JSON API, which experienced a 5% failure rate during this time period. For
details, see the Cloud Storage incident post:
https://status.cloud.google.com/incident/storage/16021.
REMEDIATION AND PREVENTION:
The primary remediation for this incident is described in the Cloud Storage
incident report. In addition to the steps outlined there, Google engineers
are tuning the systems that monitor Managed VMs application deployments to
alert earlier, which will facilitate speedier resolution.
Reply all
Reply to author
Forward
0 new messages