BadRequestError: offset may not be above 1000

[Aral Balkan]

Hi,

I'm not sure when this changed but it seems that you cannot specify
offsets above 1,000 which kind of kills my backup app.

How are we supposed to iterate over large datasets?

Thanks,
Aral

[Michael]

I have the same problem. I can understand the result limit, but
offset ?
Simple solution is add a record ID .

[Byron Saltysiak]

Perhaps more targetted queries?

--
Sent from Gmail for mobile | mobile.google.com

Byron

[Alexander Kojevnikov]

GAE uses client-side offsets, that means it first runs the query, then
iterates on the result to reach the offset you specified. This implies
that the offset cannot be greater than the result limit:
http://code.google.com/appengine/docs/datastore/creatinggettinganddeletingdata.html#Getting_Entities_Using_a_Query
(read the section called "Executing the Query and Accessing Results")

To iterate over a large dataset you need to have a unique field that
you will filter and order by. I suggest checking out this video:
http://sites.google.com/site/io/building-scalable-web-applications-with-google-app-engine
Closer to the end Brett explains how to compose such a field.

[Emmanuel Okyere]

There are a few tips in the "Updating Existing Entities" section of this article too:

http://code.google.com/appengine/articles/update_schema.html

cheers,

Emmanuel.

---

RECURSION, n. See recursion