Bug in counting datastore small ops?

[Lapteuh]

Hi all!

Recently, I had to remove all entities of a certain type in the datastore. There were  ~10 000. I connected via Remote Api and executed the following code:

from model import MyEntityKind

@ndb.toplevel
def delete_all():
    for key in MyEntityKind.query().iter(keys_only=True):
        key.delete_async()


delete_all()

And I get:

OverQuotaError: The API call datastore_v3.RunQuery() required more quota than is available.

And before to run code, small ops quota was zero.

In the administrative console was specified, that ended quotas datastore small operation. This behavior surprised me, because documentation specified Query (keys only) — 1 read + 1 small per key retrieved, and the query had to spend 10 000 key =  1 read + 10 000 small!

Perhaps this is a side effect of using Remote Api or bug! 

Thank you for your attention!

P.S. Sorry for my bad english.

[timh]

Are you sure the over quota is from the query.  

The delete will cause all indexes associated with the deleted entity to be updated.  So that will be more likely the cause of the over quota.

To prove this run the same query without the delete ;-)

T

[Lapteuh]

Thats funny. :)

Cause: I explicitly disabled indexing properties. Yes, even if it were so, it would have been consumed write quota logically. And it is pristine clean. Yes and no further operations datastore I didn't commit.

Thank you for your attention!

воскресенье, 15 декабря 2013 г., 6:43:18 UTC+6 пользователь timh написал:

[Lapteuh]

Hi, again!

This is not a bug in counting small quote, this is side effect of Remote API. More I will not use that. :)

Thanks all!

суббота, 14 декабря 2013 г., 22:32:25 UTC+6 пользователь Lapteuh написал:

[timh]

You didn't mention disabling indexes in your OP.

You are using iter over remote api, I usually use big batches (1000) using a cursor for remote_api work.  It may be the small iter/small batches create a lot of roundtrips and these would add

up as small ops.