Datastore offline as of 3:40pm PST
115 views
Skip to first unread message
blackpawn
Mar 9, 2012, 6:59:36 PM3/9/12
to google-a...@googlegroups.com
The datastore is completely dead since 3:40 and non-HRD apps can't serve any requests besides static files. The status page is showing Anomaly so hopefully this is being looked at. Is there anything we can do to prevent our apps from going nuts during this period spawning tons of instances that just time out but still charge us money for CPU?
blackpawn
Mar 9, 2012, 7:03:12 PM3/9/12
to google-a...@googlegroups.com
... and everything suddenly back to normal. That was a scary 20 minutes! Big thanks to anyone over at Google that fixed :)
Strom
Mar 9, 2012, 7:03:03 PM3/9/12
to Google App Engine
Yes, move to HRD.
blackpawn
Mar 9, 2012, 7:07:20 PM3/9/12
to google-a...@googlegroups.com
Google said a blobstore migration tool is coming so I've held off on finishing move to HRD until that comes out. Looking forward to be switched over for sure.
John
Mar 9, 2012, 7:10:53 PM3/9/12
to google-a...@googlegroups.com
It's recommended that we make use of the CapabilitiesService. This lets you get values like "UNKNOWN" and "DISABLED" for things like "DATASTORE_WRITE". I'm curious what value it would have provided during this period.
On Friday, March 9, 2012 3:59:36 PM UTC-8, blackpawn wrote:
On Friday, March 9, 2012 3:59:36 PM UTC-8, blackpawn wrote:
blackpawn
Mar 9, 2012, 7:17:01 PM3/9/12
to google-a...@googlegroups.com
It seemed like a complete failure. My app does use the capabilities API and handles read only properly but no non-static requests succeeded. Even requests that only wind up hitting memcache were failing too actually. The errors were all "A problem was encountered with the process that handled this request, causing it to exit." and "Request was aborted after waiting too long to attempt to service your request."
devlike
Mar 13, 2012, 2:27:46 AM3/13/12
to google-a...@googlegroups.com
3 days later, the service status page shows all green for the past week. Seems a bit less than honest.
Ikai Lan (Google)
Mar 14, 2012, 3:37:00 PM3/14/12
to google-a...@googlegroups.com
I've updated the status page for the network disruption yesterday:
Dishonesty is not the intention. There's been talk about rebuilding the status site from scratch because a lot of the reporting could use improvement - there's a view some team members share that an incorrect status site is worse than no status site.
Dishonesty is not the intention. There's been talk about rebuilding the status site from scratch because a lot of the reporting could use improvement - there's a view some team members share that an incorrect status site is worse than no status site.
--
Ikai Lan
Developer Programs Engineer, Google App Engine
Developer Programs Engineer, Google App Engine
On Mon, Mar 12, 2012 at 11:27 PM, devlike <richardbr...@gmail.com> wrote:
3 days later, the service status page shows all green for the past week. Seems a bit less than honest.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/pWUz1T_OFAUJ.
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
John
Mar 14, 2012, 7:47:52 PM3/14/12
to google-a...@googlegroups.com
On Friday, March 9, 2012 4:17:01 PM UTC-8, blackpawn wrote:
It seemed like a complete failure. My app does use the capabilities API and handles read only properly but no non-static requests succeeded. Even requests that only wind up hitting memcache were failing too actually. The errors were all "A problem was encountered with the process that handled this request, causing it to exit." and "Request was aborted after waiting too long to attempt to service your request."
Thanks for satisfying my curiosity! Sounds like your code didn't fare any better than ours in this outage.
On Friday, March 9, 2012 4:10:53 PM UTC-8, John wrote:It's recommended that we make use of the CapabilitiesService. This lets you get values like "UNKNOWN" and "DISABLED" for things like "DATASTORE_WRITE". I'm curious what value it would have provided during this period.
On Friday, March 9, 2012 3:59:36 PM UTC-8, blackpawn wrote:The datastore is completely dead since 3:40 and non-HRD apps can't serve any requests besides static files. The status page is showing Anomaly so hopefully this is being looked at. Is there anything we can do to prevent our apps from going nuts during this period spawning tons of instances that just time out but still charge us money for CPU?
blackpawn
Mar 14, 2012, 7:48:31 PM3/14/12
to google-a...@googlegroups.com
Thanks for keeping us up to date Ikai!
On Wednesday, March 14, 2012 12:37:00 PM UTC-7, Ikai Lan wrote:
On Wednesday, March 14, 2012 12:37:00 PM UTC-7, Ikai Lan wrote:
I've updated the status page for the network disruption yesterday:
Dishonesty is not the intention. There's been talk about rebuilding the status site from scratch because a lot of the reporting could use improvement - there's a view some team members share that an incorrect status site is worse than no status site.
--Ikai Lan
Developer Programs Engineer, Google App Engine
On Mon, Mar 12, 2012 at 11:27 PM, devlike <richardbr...@gmail.com> wrote:
3 days later, the service status page shows all green for the past week. Seems a bit less than honest.To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/pWUz1T_OFAUJ.
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages