Indexes not deployed (with Maven plugin)

[Moritz]

Hi group,

maybe this is a really stupid question, but I'm a bit stuck. In my
project I'm using the Maven GAE Plugin to deploy the application to
GAE.

However, on the production server, no indexes are defined, although my
entity classes are properly annotated, and on the development server
everything is fine.

Maybe this is an issue with the Maven plugin, but before I investigate
further into that direction, I'd like to understand how Objectify
handles index definitions.

In my project I cannot find a datastore-indexes.xml file. So, how does
Objectify handle it?

Any suggestions are highly appreciated.
Moritz

[Jon Stevens]

Have you tried reading the documentation?

http://code.google.com/p/objectify-appengine/wiki/Concepts#Indexes

jon

[Jeff Schnitzer]

Objectify doesn't do anything with datastore-indexes.xml. That's
strictly between your app and the SDK. Objectify runs queries against
the Low-Level API, and if you don't have the right indexes, those
queries won't work. The same thing happens when using the Low-Level
API or JDO; when you try to execute queries on the development server
that need an index which isn't defined, it suggests the index to you.

The datastore-indexes.xml file should live in the WEB-INF of your project.

Jeff

On Mon, Dec 20, 2010 at 1:34 PM, Moritz <mor...@cloudme.org> wrote:

[Moritz]

Sure, I read the documentation, but maybe I missed the critical part.
My understanding was, that with Objectify all I need to do is to use
the right annotations. Here's how my class looks like:

@Cached
public class ItemInstance extends DomainObject implements
Comparable<ItemInstance> {
private Long itemId;
private Long itemListId;
@Unindexed
private String attribute;
@Unindexed
private boolean ticked;
private String text;
@Transient
private boolean inList = false;
...

And the query is like "find by itemListId order by text".

The exception is:

net.sourceforge.stripes.util.Log warn: Unhandled exception caught by
the Stripes default exception handler.
com.google.appengine.api.datastore.DatastoreNeedIndexException: no
matching index found.. <datastore-index kind="ItemInstance"
ancestor="false" source="manual">
<property name="itemListId" direction="asc"/>
<property name="text" direction="asc"/>
</datastore-index>


at
com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java:
42)
at com.google.appengine.api.datastore.DatastoreApiHelper
$1.convertException(DatastoreApiHelper.java:98)
at
com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:
69)
at
com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:
67)
at
com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:
71)
at
com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:
32)
at
com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities(QueryResultsSourceImpl.java:
69)
at
com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities(QueryResultsSourceImpl.java:
56)
at
com.google.appengine.api.datastore.QueryResultIteratorImpl.ensureLoaded(QueryResultIteratorImpl.java:
156)
at
com.google.appengine.api.datastore.QueryResultIteratorImpl.hasNext(QueryResultIteratorImpl.java:
65)
at
com.googlecode.objectify.helper.TranslatingIterator.hasNext(TranslatingIterator.java:
29)
at com.googlecode.objectify.impl.QueryImpl.list(QueryImpl.java:468)
at
org.cloudme.loclist.dao.ItemInstanceDao.listByItemList(ItemInstanceDao.java:
15)
at
org.cloudme.loclist.item.ItemService.getItemInstances(ItemService.java:
102)
...

Any ideas what's wrong here?

[Jeff Schnitzer]

"find by itemListId order by text" requires a multi-property index.
Multi-property indexes are configured differently than single-property
indexes - it's not enough to just create @Indexed fields on the
relevant parts. You need to build an index containing 'itemListId'
and 'text' by defining it in the datastore-indexes.xml file. The
exception actually provides you the snippet you need to include.

You should read this page end-to-end:

http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html

Try to look through the JDO syntax and see the underlying conceptual
documentation.

BTW one gotcha is that in order for an entity to participate in a
multi-property index, all properties which are part of the
multi-property index must themselves have single-property indexes. So
keep the fields @Indexed.

Jeff

[Moritz]

Jeff, thank you for the explanation. I thought somehow that with
Objectify annotations would be enough. Now it is obvious that I need
the multi-property index defined in the datastore-indexes.xml file.

On 21 Dez., 10:32, Jeff Schnitzer <j...@infohazard.org> wrote:
> "find by itemListId order by text" requires a multi-property index.
> Multi-property indexes are configured differently than single-property
> indexes - it's not enough to just create @Indexed fields on the
> relevant parts.  You need to build an index containing 'itemListId'
> and 'text' by defining it in the datastore-indexes.xml file.  The
> exception actually provides you the snippet you need to include.
>
> You should read this page end-to-end:
>

> http://code.google.com/appengine/docs/java/datastore/queriesandindexe...

[Jeff Schnitzer]

The dichotomy between multi-property indexes and single-property
indexes is a little quirky. It seems strange that you can define a
new multi-index in datastore-indexes.xml and GAE will build it in the
background for you, but single-property indexes require you to re-save
every entity. I can kinda see how Goog arrived at this design but
it's not intuitive to the novice.

Jeff

[Moritz]

There is one more point about this topic: normally, the development
server creates index definitions automatically, even for multi-
property indexes. The definitions are stored in appengine-generated/
datastore-indexes-auto.xml according to the documentation
http://code.google.com/intl/de-DE/appengine/docs/java/config/indexconfig.html#Using_Automatic_Index_Configuration

And, "surprise", it actually works.

Why didn't it work previously? Because I ran "mvn clean gae:deploy".
But with "mvn clean" everything generated by the development server in
the target/ directory is deleted.

Note to self: don't run "mvn clean" or don't use automatic index
definition.