Datastore restore failure

[Francois Masurel]

I get this message when I try to restore a datastore backup :

Restore from backup Job Status

There was a problem kicking off the jobs. The error was:

<type 'exceptions.Exception'>: Huge tasks should specify parent entity.

Any help will be greatly appreciated.

[Arie Ozarov]

We are looking into it.

[Ikai Lan (Google)]

Hi Francois,

I was just talking to Arie about this. So some implementation details (for anyone who is curious): The way the backup tool works is by iterating over all of your data in a mapper job. The number of shards is determined by the number of Kinds * Entities per kind. Entities are serialized (entity.ToPb().SerializeToString() ), batched in a LevelDB format (http://code.google.com/p/leveldb/) and written to the Blobstore (http://code.google.com/p/appengine-mapreduce/source/browse/trunk/python/src/mapreduce/output_writers.py).

What's happening in your case is that there are too many worker shards and the metadata required to describe the job is exceeding the entity size limit, and it's causing a mapreduce exception which is bubbling all the way up to the top. This is a bug in the API and will be fixed. I don't have an ETA for when the fix will go live, however - hopefully sometime in the next few days.

In the meantime, do you have an application ID? Arie on the engineering team would like to take a look at your datastore to look at the metadata entity, but he needs your permission to do so. He might be able to recommend a workaround if there's something that jumps out at us.

--

Ikai Lan 
Developer Programs Engineer, Google App Engine

plus.ikailan.com

On Wed, Feb 1, 2012 at 1:26 PM, Arie Ozarov <oza...@google.com> wrote:

We are looking into it.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/roABfczfSxYJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Francois MASUREL]

Hi Ikai,

Thanx for your answer.

Here is my application ID : vinocities

The "data" kind is rather generic and has probably quite a few different properties.

François