Cursor errors after recent upgrade to 4.1.3/1/9.3 from 4.0*/1.8.x

[tim]

Application was running fine for quite some time but after these env updates my application has not been stable.

There are two errors happening that seem related.

1 java.lang.IllegalArgumentException: Cursor does not match query.

After doing:

Query<PageFragmentContentItem2> loadIdCis=getStorage().load()

.type(PageFragmentContentItem2.class)

.filter("loadId", candidateLoadId)

.order("-orderingScalar"); // i use limits here too but removed to narrow down bug

then:

String curStr=user.loadIdToCursor.get(candidateLoadId);

if(curStr!=null) {

Cursor cursor=Cursor.fromWebSafeString(curStr);

loadIdCis.startAt(cursor);

}

iterator = loadIdCis.iterator();

 do a bunch or reading (no writes/updates/deletes on PageFrag), then:

if(!iterator.hasNext()) {

endOfLoadIdReached(user, loadId);

} else {

user.loadIdToCursor.put(loadId, iterator.getCursor().toWebSafeString()); //Exception here

}

Exception stack:

java.lang.IllegalArgumentException: Cursor does not match query.

at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:39)

at com.google.appengine.api.datastore.DatastoreApiHelper$1.convertException(DatastoreApiHelper.java:76)

at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:94)

at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:86)

at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:71)

at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:32)

at com.google.appengine.api.datastore.QueryResultsSourceImpl.peekQueryResultAndIfFirstRecordIndexList(QueryResultsSourceImpl.java:167)

at com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities(QueryResultsSourceImpl.java:109)

at com.google.appengine.api.datastore.QueryResultIteratorImpl.ensureInitialized(QueryResultIteratorImpl.java:135)

at com.google.appengine.api.datastore.QueryResultIteratorImpl.getCursor(QueryResultIteratorImpl.java:86)

at com.googlecode.objectify.impl.ChunkingIterator.getCursor(ChunkingIterator.java:114)

2 The other error is that iter.hasNext() used to return false when the end of the list (using the cursor) was reached but now it seems like it just cycles. This could be because the ex above, though.

I blow away all data before testing so it's not that.

Ideas?

[tim]

Just so others following this have a conclusion: another user found the issue tracking down a different bug. Basically you can't do:

loadIdCis.startAt(cursor); // the "immutable" var loadIdCis is invalid now.

you have to do something like:

loadIdCis=loadIdCis.startAt(cursor);

[Jeff Schnitzer]

Just to be clear - the loadIdCis is not invalid, it just hasn't changed.

Jeff

--
You received this message because you are subscribed to the Google Groups "objectify-appengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appen...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[phine...@gmail.com]

I'm still seeing this (on GAE 1.9.4 and Objectify 4.1.3). In my case, I see "java.lang.IllegalArgumentException: Cursor does not match query." when I'm retrieving with a cursor and my query includes a comparison on a string (>, >=, <, <=). The whole query in pseudo-code goes something like this:

---

final Key<SomeParent> parentKey = Key.create(SomeParent.class, parentId);

Query<SomeChild> query = ofy().load().type(SomeChild.class).ancestor(parentKey);

// don't include these next two lines, and the exception doesn't happen

query = query.filter("someString >", someValue);

query = query.order("someString");

if (null != cursor) {

    query = query.startAt(Cursor.fromWebSafeString(cursor));

}

final QueryResultIterator<Key<SomeChild>> indexKeysIt = query.keys().iterator();

final ArrayList<Long> childIds = new ArrayList<Long>();

while (indexKeysIt.hasNext()) {

childIds.add(Long.parseLong(indexKeysIt.next().getParent().getName()));

}

// get cursor

final Cursor endCursor = indexKeysIt.getCursor();

return new MyResult(childIds, null != endCursor ? endCursor.toWebSafeString() : null);

---

I do have the necessary indexes in place (I think), and this was working 1.8.x/4.0.x (exactly which versions I'm not sure). I am reassigning the "query" variable, which I think is what this post was about. Have I missed something else?

To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appengine+unsub...@googlegroups.com.

[Aleem Mawani]

+1 

I just moved from 4.0 to 4.1.3 and started seeing this issue

[Jeff Schnitzer]

This could be a bug. I will carve out some time this weekend to look at it.

Jeff

To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appen...@googlegroups.com.

[Aleem Mawani]

actually seeing this issue in 4.0.1 as well.

ᐧ

[Aleem Mawani]

Thanks Jeff!

+1 

To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appengine+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "objectify-appengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appengine+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[phine...@gmail.com]

I wanted to meet you part way on this, so I created a standalone JUnit test case that reproduces the problem. There are more comments in the source of the test, but one thing to note that may be useful is that the exception happens only for keys-only queries, as demonstrated by the two methods in the test, one with an straight query for entities (testEntitiesQuery(), which succeeds) and one with a keys-only query (testKeysQuery(), which fails).

Obviously, you need the GAE unit testing jars in your classpath for this to work: appengine-api.jar, appengine-api-labs.jar, appengine-api-stubs.jar and appengine-testing.jar. Thanks for the help, if you've got the time...

+1 

To unsubscribe from this group and stop receiving emails from it, send an email to objectify-appengine+unsubscribe...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[phine...@gmail.com]

One additional thing to mention- I think something within GAE itself may have changed, because if I remember right I was on 4.0.1 and 1.8.x and it was working, then somewhere along the way to 1.9.4 it stopped working, without changing the Objectify version. Then I upgraded to Objectify 4.1.3 to see if that would help, but it didn't. So in which place the fix needs to be made, I don't know. Since I can't give you an exact version of GAE where it changed, don't take this for 100% sure, but hope that helps...

[Tobias Andersson]

I'm on Objectify 4.0rc1. When I upgraded GAE SDK from 1.8.8 to 1.9.4, while keeping the same Objectify version, I started getting this error.

[Mo'in Creemers]

This issue has nothing to do with Objectify.

I have apps that do not use Objectify and are experiencing the issue as described after updating to app engine 1.9.4.

I am certain this is a bug because:

1. my application code has worked before the update.

2. the Cursor DOES match the query (the error message is incorrect).

The best solution I can think of right now is to roll back the update and wait for an update from Google.

[phine...@gmail.com]

I am surprised that something this fundamental would pass unit tests and make it into a GAE SDK release. I haven't seen an issue filed for this here: https://code.google.com/p/googleappengine/issues/list. Do they know about this?

Because I don't know the code in Objectify or GAE, I didn't want to assume it was one or the other. There could be an API change in GAE for example that needs to be accounted for in Objectify somehow. But if you've got apps without Objectify that see this, that's not the case.

[phine...@gmail.com]

I think this is the fix we're waiting for:

https://code.google.com/p/googleappengine/issues/detail?id=10943

[Mo'in Creemers]

Correction: The query does not match because it is incomplete because index information is not included by the Admin SDK.

So the error is probably correct.

This bug has been confirmed by Google.

On Saturday, May 17, 2014 10:26:47 AM UTC+2, Mo'in Creemers wrote:

[phine...@gmail.com]

It's fixed in 1.9.6.