deleting entities

[ilial]

I have a few hundred entities that I imported from a csv file using
bulkload.Loader. In the future it will be thousands. I need to delete
all the entities.

So I wrote this code:
q = db.GqlQuery("SELECT * FROM Contract")
results = q.fetch(limit=100000) # fetch all
db.delete(results)

Running it, I get the error: All keys must be in the same entity
group.

Is there a better way to delete all the entities?

TIA,

ilia.

[Craig]

I am also struggling with this issue, of course you can delete
entities one by one:

for c in Contract.all():
c.delete()

but I am finding that this introduces another set of issues when the
number of entities is large

[ilial]

Why isn't there a DELETE FROM?

What kind of issues are you having?

[Brett Morgan]

On Mon, Apr 14, 2008 at 11:42 AM, ilial <sam...@gmail.com> wrote:
>
> Why isn't there a DELETE FROM?

Because this isn't SQL? =)

[Craig Carpenter]

First, to explain your issue read the documentation for db.delete(...)

http://code.google.com/appengine/docs/datastore/functions.html#delete

Follow the link to "Keys and Entity Groups" for more details.

My issue is that when processing in bulk you tend to get
DeadlineExceededError's, to avoid that i've limited the number of
items deleted per request and then hit the page repetitively. All
very messy.

[Jeff Hinrichs]

On Apr 13, 8:58 pm, "Craig Carpenter" <craig.carpen...@gmail.com>
wrote:

I too have encountered the same problem.
http://code.google.com/appengine/docs/datastore/creatinggettinganddeletingdata.html#Deleting_an_Entity
The second example fails completely. Always complaining about
"BadRequestError: All keys must be in the same entity group."

I have filed an issue on it, star it, if you want. Report contains
simple test case demonstrating problem.
http://code.google.com/p/googleappengine/issues/detail?id=189

[hads]

On Apr 14, 2:58 pm, "Craig Carpenter" <craig.carpen...@gmail.com>
wrote:

> My issue is that when processing in bulk you tend to get
> DeadlineExceededError's, to avoid that i've limited the number of
> items deleted per request and then hit the page repetitively.   All
> very messy.

I ran across the same issue just now after a bulk import of a large
dataset failed part way through. Basically I had to limit the query to
250 entities and then do a;

while true; do curl http://example.appspot.com/?delete; done

I guess the 250 limit would vary depending on your model objects but
that's what worked for me without causing any warnings in the logs.

hads

[Craig Carpenter]

sorry, but its not a bug. it's by design. read the documentation I
linked to paying attention to the part about transactions

[Brett Morgan]

On the upside you are encouraged to build ajaxy front ends with
progress bars and what not. =)

[Jeff Hinrichs]

It would be more helpful if you could explain why the example listed
in the docs:
http://code.google.com/appengine/docs/datastore/creatinggettinganddeletingdata.html#Deleting_an_Entity

does not work as expected. Even with 10 in the list.


On Apr 13, 9:29 pm, "Craig Carpenter" <craig.carpen...@gmail.com>
wrote:

> sorry, but its not a bug. it's by design. read the documentation I
> linked to paying attention to the part about transactions
>

> On Mon, Apr 14, 2008 at 2:08 PM, dundeemt <dunde...@gmail.com> wrote:
>
> > On Apr 13, 8:58 pm, "Craig Carpenter" <craig.carpen...@gmail.com>
> > wrote:
>
> > > First, to explain your issue read the documentation for db.delete(...)
>
> > >http://code.google.com/appengine/docs/datastore/functions.html#delete
>
> > > Follow the link to "Keys and Entity Groups" for more details.
>
> > > My issue is that when processing in bulk you tend to get
> > > DeadlineExceededError's, to avoid that i've limited the number of
> > > items deleted per request and then hit the page repetitively. All
> > > very messy.
>
> > I too have encountered the same problem.

> > http://code.google.com/appengine/docs/datastore/creatinggettinganddel...

[Craig Carpenter]

all of the entities must be in the same entity group as all the
deletes occur in one transaction. your entities are not in the same
entity group.

two entities are in the same group iff they have the same ancestor

so, for example, when creating your 10 entities you could set their
parent to a given entity E. that way E would be the ancestor of each
of your 10 entities, so they're in the same entity group

they explain it better than I can:
http://code.google.com/appengine/docs/datastore/keysandentitygroups.html#Entity_Groups_Ancestors_and_Paths

[Jeff Hinrichs]

On Apr 14, 1:32 am, "Craig Carpenter" <craig.carpen...@gmail.com>
wrote:

> all of the entities must be in the same entity group as all the
> deletes occur in one transaction. your entities are not in the same
> entity group.
>
> two entities are in the same group iff they have the same ancestor
>
> so, for example, when creating your 10 entities you could set their
> parent to a given entity E. that way E would be the ancestor of each
> of your 10 entities, so they're in the same entity group
>

> they explain it better than I can:http://code.google.com/appengine/docs/datastore/keysandentitygroups.h...

How does one set the ANCESTOR of an entity? I've been through the
docs and can't seem to find anything about setting the parent/
ancestor. There are numerous instances where they refer to selecting
or deleteing.

Thanks,

Jeff

>
> On Mon, Apr 14, 2008 at 6:20 PM, dundeemt <dunde...@gmail.com> wrote:
>
> > It would be more helpful if you could explain why the example listed
> > in the docs:
>

> >http://code.google.com/appengine/docs/datastore/creatinggettinganddel...

[Craig Carpenter]

It can only be set at creation:

http://code.google.com/appengine/docs/datastore/modelclass.html#Model

[ilial]

so how do you set the ancestor when using the bulkload.Loader?

On Apr 14, 4:24 pm, "Craig Carpenter" <craig.carpen...@gmail.com>
wrote:

> It can only be set at creation:
>
> http://code.google.com/appengine/docs/datastore/modelclass.html#Model
>