count() limited to 1000 entries too?

[Benjamin Golub]

I ported my Django app over to Google App Engine (http://
www.friendfeedstats.com) but have a big problem. FriendFeed Stats
relies on counting. When I had it on my server it was indexing over
100000 entries and used count() just fine. Now that I've ported it to
Google App Engine and let it run for a little while I'm seeing counts
of 1001 all over the place. This leads me to believe that you are
limiting count() queries to 1000.

Is there anyway to fix this? Note that I do not want the actual rows;
just the count.

[Philipp Lenssen]

It doesn't directly help but Google says "Queries have a system-wide
maximum limit of 1000 results. If no limit is specified, or if a limit
larger than the maximum is specified, the maximum is used as the
limit." (http://code.google.com/appengine/docs/datastore/
creatinggettinganddeletingdata.html).

[Benjamin Golub]

I understand that; but a count isn't exactly the same as saying I want
over 1000 rows. I'm hoping this is not by intention.

[ryan]

This actually is the intended behavior, as Philipp pointed out. The
datastore doesn't support histograms, so regardless of whether you ask
for result entities or just a count, it still has to do a full index
scan. Hence the 1000 result cap that's in place right now.

[Benjamin Golub]

Oh well, looks like I'll make the change to point friendfeedstats.com
back to my server. Still though, kudos on a successful launch. I was
able to do port it very very quickly and it runs very fast.

[Greg]

Wouldn't it be simpler (and way more efficient) to maintain a count
field, instead of counting objects?

[Benjamin Golub]

That would be an awful lot of count fields. The site lets you set
durations, show different counts, and filter on user/service. It is
basically impossible to do that.