Datastore restore failure

49 views
Skip to first unread message

Francois Masurel

unread,
Feb 1, 2012, 11:35:31 AM2/1/12
to google-a...@googlegroups.com
I get this message when I try to restore a datastore backup :

Restore from backup Job Status

There was a problem kicking off the jobs. The error was:

<type 'exceptions.Exception'>: Huge tasks should specify parent entity.

Any help will be greatly appreciated.

Arie Ozarov

unread,
Feb 1, 2012, 4:26:04 PM2/1/12
to google-a...@googlegroups.com
We are looking into it.

Ikai Lan (Google)

unread,
Feb 1, 2012, 7:11:37 PM2/1/12
to google-a...@googlegroups.com
Hi Francois,

I was just talking to Arie about this. So some implementation details (for anyone who is curious): The way the backup tool works is by iterating over all of your data in a mapper job. The number of shards is determined by the number of Kinds * Entities per kind. Entities are serialized (entity.ToPb().SerializeToString() ), batched in a LevelDB format (http://code.google.com/p/leveldb/) and written to the Blobstore (http://code.google.com/p/appengine-mapreduce/source/browse/trunk/python/src/mapreduce/output_writers.py).

What's happening in your case is that there are too many worker shards and the metadata required to describe the job is exceeding the entity size limit, and it's causing a mapreduce exception which is bubbling all the way up to the top. This is a bug in the API and will be fixed. I don't have an ETA for when the fix will go live, however - hopefully sometime in the next few days.

In the meantime, do you have an application ID? Arie on the engineering team would like to take a look at your datastore to look at the metadata entity, but he needs your permission to do so. He might be able to recommend a workaround if there's something that jumps out at us.

--
Ikai Lan 
Developer Programs Engineer, Google App Engine



On Wed, Feb 1, 2012 at 1:26 PM, Arie Ozarov <oza...@google.com> wrote:
We are looking into it.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/roABfczfSxYJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Francois MASUREL

unread,
Feb 2, 2012, 3:11:56 AM2/2/12
to google-a...@googlegroups.com
Hi Ikai,

Thanx for your answer.

Here is my application ID : vinocities

The "data" kind is rather generic and has probably quite a few different properties.

François
Reply all
Reply to author
Forward
0 new messages