Datastore Admin - Delete didn't work, burned $7.85

[GregR]

I have a Entity with 2,356,292 in it.  I tried using the datastore Admin "Delete Entities" to purge it.  It ran for a while, got a bunch of errors, burned $7.85 (mostly in CPU), and didn't actually delete anything.

I've included some of the errors below.

Any idea what's up? I can go back to the old way of deleting, but it would take a long time (and CPU) with this # of entities.

Thanks,
-Greg

·  0.1.0.2 - - [24/Jul/2011:13:20:37 -0700] "POST /_ah/mapreduce/worker_callback HTTP/1.1" 500 2394 "http://1-dot-latest-dot-locationplay.appspot.com/_ah/mapreduce/worker_callback" "AppEngine-Google; (+http://code.google.com/appengine)" "1-dot-latest-dot-locationplay.appspot.com" ms=29696 cpu_ms=306889 api_cpu_ms=305046 cpm_usd=8.524742 queue_name=default task_name=appengine-mrshard-15868331507823D8715B7-11-8 pending_ms=5160 instance=00c61b117c29bef02c280156561831181c935a2e

·  E 2011-07-24 13:20:37.069

The API call datastore_v3.Delete() took too long to respond and was cancelled.

Traceback (most recent call last):

  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 702, in __call__

    handler.post(*groups)

  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/base_handler.py", line 81, in post

    self.handle()

  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/handlers.py", line 184, in handle

    entity, input_reader, ctx, tstate)

  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/handlers.py", line 254, in process_data

    output(ctx)

  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/operation/db.py", line 87, in __call__

    context.mutation_pool.delete(self.entity)

  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/context.py", line 171, in delete

    self.__flush_deletes()

  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/context.py", line 189, in __flush_deletes

    datastore.Delete(self.deletes.items, rpc=self.__create_rpc())

  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/datastore.py", line 550, in Delete

    return DeleteAsync(keys, **kwargs).get_result()

  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 592, in get_result

    return self.__get_result_hook(self)

  File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1464, in __delete_hook

    self.check_rpc_success(rpc)

  File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1072, in check_rpc_success

    rpc.check_success()

  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 558, in check_success

    self.__rpc.CheckSuccess()

  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_rpc.py", line 133, in CheckSuccess

    raise self.exception

DeadlineExceededError: The API call datastore_v3.Delete() took too long to respond and was cancelled.

 

1. 1. 2011-07-24 13:20:37.282 /_ah/mapreduce/worker_callback 500 24793ms 879cpu_ms 529api_cpu_ms 1kb AppEngine-Google; (+http://code.google.com/appengine)

      0.1.0.2 - - [24/Jul/2011:13:20:37 -0700] "POST /_ah/mapreduce/worker_callback HTTP/1.1" 500 1996 "http://1-dot-latest-dot-locationplay.appspot.com/_ah/mapreduce/worker_callback" "AppEngine-Google; (+http://code.google.com/appengine)" "1-dot-latest-dot-locationplay.appspot.com" ms=24793 cpu_ms=879 api_cpu_ms=529 cpm_usd=0.024815 queue_name=default task_name=appengine-mrshard-15868331507823D8715B7-28-8 pending_ms=9561 instance=00c61b117cbc93cdc72672caefcebc6c52198bc4

   2. E 2011-07-24 13:20:37.274

      Traceback (most recent call last):
        File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 702, in __call__
          handler.post(*groups)
        File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/base_handler.py", line 81, in post
          self.handle()
        File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/handlers.py", line 184, in handle
          entity, input_reader, ctx, tstate)
        File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/handlers.py", line 254, in process_data
          output(ctx)
        File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/operation/db.py", line 87, in __call__
          context.mutation_pool.delete(self.entity)
        File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/context.py", line 171, in delete
          self.__flush_deletes()
        File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/mapreduce/context.py", line 189, in __flush_deletes
          datastore.Delete(self.deletes.items, rpc=self.__create_rpc())
        File "/base/python_runtime/python_lib/versions/1/google/appengine/api/datastore.py", line 550, in Delete
          return DeleteAsync(keys, **kwargs).get_result()
        File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 592, in get_result
          return self.__get_result_hook(self)
        File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1464, in __delete_hook
          self.check_rpc_success(rpc)
        File "/base/python_runtime/python_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 1074, in check_rpc_success
          raise _ToDatastoreError(err)
      Timeout

[Robert Kluin]

You're getting datastore timeouts. If you're on master-slave, then
you might try running it again. Perhaps you were just unlucky and ran
durring a latency spike.

Also, does that entity have a lot of indexes or something?

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/IBGRGWdGCUoJ.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>