Datastore errors, timeouts, general difficulties

[Jonathan Ultis]

I created a simple page that simply instantiates and stores a model
with about 9 primitive properties. The model is filled with the same
data each time, and that data is less than 100 bytes. I don't have
indices on any of the properties.

I did some load tests, storing approximately half a million of the
objects at most. Some of the tests were done during the recent latency
problems. Some were done after latency was stabilized. All tests with
significant concurrency (50) generated fairly high timeout and quota
exceeded error rates. So, I got significant log spew.

My total data size should be around 50 megs of real user data.

Today, my data usage jumped from about 0.5% of my data storage limit
to 67% of my data storage limit without any intervening tests. That
was surprising since that's about 10x what I really should be using.
But, I figured that data limit might count all replicas or something
odd. And, perhaps there's a latency between the data store and the
data size calculation. So, I wrote a delete all page that deletes 50
items, then redirects to itself, to delete the next 50. I just delete
the first 50 items returned. I'm not paging or anything. The script
works locally.

I ran that delete page for a while, periodically getting annoying
timeouts and quota exceeded. I did not run the page in parallel.

Now that I'm using 99% of my data storage limit. I'm afraid I may have
to buy data storage before I can delete more items.

Any ideas?

[Jonathan Ultis]

That should read, after deleting, I'm now using 99% of my data storage
limit with no additional puts.

[Brandon Thomson]

These may be relevant:

http://groups.google.com/group/google-appengine/browse_thread/thread/517c7da78454da66/939a67117246d120?lnk=gst&q=log+storage#939a67117246d120

http://code.google.com/p/googleappengine/issues/detail?id=452

Basically you need to get the entity before you delete or it will be
gone without reducing your storage quota.

[Marzia Niccolai]

Hi,

My apologies for not closing 452.  With the launch of billing, this issue was fixed, we now run a process that correctly accounts for the  amount of datastore usage.  Deletes may not immediately be reflected, but they will be eventually.  So if you did delete all of your data this should be accounted for, minus some delay.

What is likely is that you are seeing we now include indexes when accounting for storage usage, which we did not previously do:
http://code.google.com/appengine/kb/billing.html#indexes

-Marzia

[Jonathan Ultis]

Thanks for the feedback. If indices started to count against quota,
that would certainly account for the change in storage size. Although,
the page I see at

http://code.google.com/appengine/kb/billing.html#indexes

still shows that indices do *not* count against the quota.

This will end up being an expensive storage solution if indices are
billed and there's no way to explicitly exclude properties from the
default indices.

On Mar 9, 11:36 am, Marzia Niccolai <ma...@google.com> wrote:
> Hi,
>
> My apologies for not closing 452.  With the launch of billing, this issue
> was fixed, we now run a process that correctly accounts for the  amount of
> datastore usage.  Deletes may not immediately be reflected, but they will be
> eventually.  So if you did delete all of your data this should be accounted
> for, minus some delay.
>
> What is likely is that you are seeing we now include indexes when accounting
> for storage usage, which we did not previously do:http://code.google.com/appengine/kb/billing.html#indexes
>
> -Marzia
>
> On Sat, Mar 7, 2009 at 7:42 AM, Brandon Thomson <gra...@gmail.com> wrote:
>
> > These may be relevant:
>

> >http://groups.google.com/group/google-appengine/browse_thread/thread/...