We are experiencing issues with instance DELETE and STOP operations in Google Compute Engine.

298 views
Skip to first unread message

Google Cloud Platform Status

unread,
Jan 29, 2020, 10:35:20 AM1/29/20
to gce-ope...@googlegroups.com
Description: We are experiencing an issue with Google Compute Engine affecting mainly us-central1-a, although other regions might see the issue as well. Instance STOP and DELETE requests may fail or complete with an error, beginning Wednesday, 2020-01-28 08:56 US/Pacific.

Symptoms: Instance STOP and DELETE requests may fail or complete with an error.

Our engineering team continues to investigate the issue.

We will provide an update by Wednesday, 2020-01-29 08:45 US/Pacific with current details.


Diagnosis: Instance STOP and DELETE requests may fail or complete with an error.

Workaround: None at this time.

Google Cloud Platform Status

unread,
Jan 29, 2020, 11:44:34 AM1/29/20
to gce-ope...@googlegroups.com
Description: We are experiencing an issue with stopping and deleting Google Compute Engine instances affecting mainly us-central1-a, with a subset of global operations impacted as well, the majority of which are projects.setCommonInstanceMetadataInstance operations. Impact began Wednesday, 2020-01-29 06:02 US/Pacific. Mitigation is underway and we are currently working through the backlog operations.

Currently running instances are not impacted.

We will provide an update by Wednesday, 2020-01-29 10:00 US/Pacific with current details.


Diagnosis: Instance STOP and DELETE requests may fail or complete with an error in us-central1-a. Errors may be in the form of “rate limit exceeded”.

Setting projects.setCommonInstanceMetadata may also fail or complete with an error, this is a global operation.

Google Cloud Platform Status

unread,
Jan 29, 2020, 12:57:05 PM1/29/20
to gce-ope...@googlegroups.com
Description: We are experiencing an issue with stopping and deleting Google Compute Engine instances affecting mainly us-central1-a, with a subset of global operations impacted as well, the majority of which are projects.setCommonInstanceMetadataInstance operations.

Stop and delete operations between Wednesday, 2020-01-29 04:00 US/Pacific and 09:00 US/Pacific may have failed or appear stuck. Stop and delete operations created after 09:00 US/Pacific should complete as expected, even if they had previously failed or are stuck.

Mitigation is underway and we are currently working through the backlog of operations.

Currently running instances are not impacted. Other zones in us-central1 are not impacted.

Products which depend on GCE may also be unable to stop or delete jobs. Impacted products include GKE, Dataflow, Dataproc, CloudSQL, Memorystore

We will provide an update by Wednesday, 2020-01-29 11:00 US/Pacific with current details.


Diagnosis: Instance STOP and DELETE requests may fail or complete with an error in us-central1-a. Errors may be in the form of “rate limit exceeded”.

Setting projects.setCommonInstanceMetadata may also fail or complete with an error, this is a global operation.

Workaround: New requests should be successful. If there are stuck operations that need to be completed urgently, please try your request again.
Reply all
Reply to author
Forward
0 new messages