Backing Up Datastore frequency

[Richard Cheesmar]

Hi, All

I have a few of questions regarding data security and sensible backup routines.

1. How vulnerable are GAE ndb datastores to data loss? Not focusing specifically on any specific reason, more interested though in physical dangers such as power outs or damage to the infrastructure rather than hacking attacks, although this is important too.

2. Is data for a single app based at one location or shared around Google Cloud data centers?

3. What do others recommend as a sane regime for backing up data on the cloud datastore?

Thanks in advance

[Christian F. Howes]

i'll share some things based on my interpretation.....happy to be educated if i get any of this wrong. :)

1.  to my knowledge in 6 years i've never lost data.  there have been outages and slowdowns, but i can't recall a single bit of data loss.  Remember that BigTable (well, i think it's been renamed to Cloud Datastore or something....) is replicated.  I suppose if you have 0 traffic you might get down to 1 copy, but at some traffic level there are at least 2 copies of your data.

2. good question.  i assumed multiple data centers in the same region, but i don't know that for a fact.

3.  i don't have a sane regime to recommend. :(  sorry.

cfh

[Adam]

To answer your second question, data is replicated across multiple data centers globally. The developer docs don't mention much detail about this however it is mentioned in the original blog post when HRD went live. Most of the links are now broken, but the original paper detailing the replication algorithm mentioned in the post, 'Paxos Made Live', is still available. A more recent publication detailing Google's replication strategies is 'Megastore: Providing Scalable, Highly Available Storage for Interactive Services'.

[Jeff Schnitzer]

I don't lose any sleep about Google losing my data. I do lose sleep over accidentally mangling it during a migration. I currently run nightly backups using the datastore admin (and cron), but it's just insurance - it would be a catastrophe if I ever attempted to restore it.

Jeff

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/254ecee8-5b78-47d6-9a98-059f11585b55%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.