Re: [objectify-appengine] Query doesn't find Entity in JUnit test

[Jeff Schnitzer]

I don't see where you register the Entity and User classes, but I presume you do since save() works.  Also:  You have an extra import javax.persistence.Id in User.java; this isn't hurting anything but it's a good idea to remove the JPA jar from your classpath to avoid future mistakes.

The problem is that you're specifically requesting a User subclass but you haven't indexed the User discriminator.  Objectify indexes nothing by default.  So you have two choices:

 * You can filter for an Entity.class with the specific emailAddress.  This will return only User objects if only Users have an indexed field called emailAddress, but could produce other types if they also have an indexed field called emailAddress.  The advantage of doing it this way is that you don't add any additional indexes.

 * You can index the User subclass discriminator.  @EntitySubclass(index=true).  This will always produce only User objects but requires an additional index for the discriminator.  Under the covers, filtering for a subclass type adds what is effectively an additional filter("^i", "User").

Jeff

On Fri, Dec 7, 2012 at 1:35 AM, Joe J Ernst <joe....@ux1.tv> wrote:

I cannot get my JUnit test to work when I query a class that extends my base Entity.  It works fine when I query the Entity class itself, just not anything that extends it.  I have the @Entity annotation on my base Entity, and the @EntitySubclass annotation on my User (which extends Entity)

I am using Objectify 4.0b1

Here is my code:

Entity.java:

package tv.ux1.entity;

import com.google.appengine.labs.repackaged.org.json.JSONObject;

import com.googlecode.objectify.Key;

import com.googlecode.objectify.annotation.Id;

import com.googlecode.objectify.annotation.Ignore;

import com.googlecode.objectify.annotation.Index;

/**

 * Entity contains methods and members that are common to all entity classes, such as {@link #toJSON} and {@link #toString}.

 * <p/>

 * Copyright 2012: ux1.tv

 *

 * @author Joe Ernst

 *         Date: 7/9/12

 */

@com.googlecode.objectify.annotation.Entity

public class Entity {

    Key key;

    @Id

    Long id;

    @Index

    String name;

    /**

     * @return A JSON representation of this entity.

     */

    public String toJSON() {

        JSONObject json = new JSONObject(this);

        json.remove("class"); // we don't want the class name in the json object

        return json.toString();

    }

    /**

     * Calls {@link #toJSON()}.

     *

     * @return A JSON String representation of this Message.

     */

    public String toString() {

        return this.toJSON();

    }

    public Key getKey() {

        return key;

    }

    public void setKey(Key key) {

        this.key = key;

    }

    public Long getId() {

        return id;

    }

    public void setId(Long id) {

        this.id = id;

    }

    public String getName() {

        return name;

    }

    public void setName(String name) {

        this.name = name;

    }

}

User.java:

package tv.ux1.entity;

import com.googlecode.objectify.annotation.Cache;

import com.googlecode.objectify.annotation.EntitySubclass;

import com.googlecode.objectify.annotation.Index;

import javax.mail.internet.AddressException;

import javax.mail.internet.InternetAddress;

import javax.persistence.Id;

/**

 * A POJO to contain everything relevant to a User.

 *

 * Created by : Joe Ernst

 * Date: 07/03/12

 */

@Cache

@EntitySubclass

public class User extends Entity {

    /**

     * The user's real first name.

     */

    String firstName;

    /**

     * The user's real last name.

     */

    String lastName;

    /**

     * A unique email address that identifies this user.

     */

    @Index

    String emailAddress;

    public User() {

        super();

    }

    public User(String emailAddress) {

        setEmailAddress(emailAddress);

    }

    // Keep the generated methods together for ease of re-generation if necessary

    public String getEmailAddress() {

        return emailAddress;

    }

    public void setEmailAddress(String emailAddress) {

        this.emailAddress = emailAddress;

    }

    public String getFirstName() {

        return firstName;

    }

    public void setFirstName(String firstName) {

        this.firstName = firstName;

    }

    public String getLastName() {

        return lastName;

    }

    public void setLastName(String lastName) {

        this.lastName = lastName;

    }

}

DataAccessObject.java:

package tv.ux1.dao;

import com.googlecode.objectify.Key;

import com.googlecode.objectify.Ref;

import com.googlecode.objectify.Result;

import com.googlecode.objectify.cmd.Query;

import tv.ux1.entity.Entity;

import java.util.logging.Logger;

import static tv.ux1.util.OfyService.ofy;

/**

 * Copyright 2012: ux1.tv

 *

 * @author Joe Ernst

 *         Date: 11/25/12

 */

public class DataAccessObject {

    protected final Logger logger = Logger.getLogger(DataAccessObject.class.getName());

    public Ref<? extends Entity> get(Key<? extends Entity> entityKey) {

        return  ofy().load().key(entityKey);

    }

    public Ref<? extends Entity> get(Class<? extends Entity> entityClass, Long entityId) {

        return ofy().load().type(entityClass).id(entityId);

    }

    public Query<? extends Entity> filter(Class<? extends Entity> entityClass, String field, String value ) {

        return ofy().load().type(entityClass).filter(field, value);

    }

    public Result<Key<Entity>> save(Entity entity) {

        return ofy().save().entity(entity);

    }

    public Result<Void> delete(Key entityKey) {

        return ofy().delete().key(entityKey);

    }

}

and finally,

TestDataAccessObject.java:

package tv.ux1.test.dao;

import com.google.appengine.tools.development.testing.LocalDatastoreServiceTestConfig;

import com.google.appengine.tools.development.testing.LocalServiceTestHelper;

import com.googlecode.objectify.Key;

import com.googlecode.objectify.Result;

import org.junit.After;

import org.junit.Assert;

import org.junit.Before;

import org.junit.Test;

import tv.ux1.dao.DataAccessObject;

import tv.ux1.entity.Entity;

import tv.ux1.entity.User;

import static org.junit.Assert.*;

/**

 * Copyright 2012: ux1.tv

 *

 * @author Joe Ernst

 *         Date: 12/6/12

 */

public class TestDataAccessObject {

    private final LocalServiceTestHelper helper =

            new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig());

    DataAccessObject dao = null;

    /**

     * Creates a new DataAccessObject instance.

     */

    @Before

    public void setUp() {

        helper.setUp();

        dao = new DataAccessObject();

    }

    @After

    public void tearDown() {

        helper.tearDown();

    }

    @Test

    public void createEntity() {

        try {

            Entity entity = new Entity();

            Key<Entity> key = dao.save(entity).now();

            assertNotNull(key);

        } catch (Exception e) {

            e.printStackTrace();

            fail(e.getMessage());

        }

    }

    @Test

    public void queryEntity() {

        try {

            Entity entity = new Entity();

            entity.setName("Foo");

            Key<Entity> key = dao.save(entity).now();

            assertNotNull(key);

            // This works; I can query the Entity by name

            Entity foundEntity = dao.filter(Entity.class, "name", "Foo").first().get();

            assertEquals(entity.getName(), foundEntity.getName());

        } catch (Exception e) {

            e.printStackTrace();

            fail(e.getMessage());

        }

    }

    @Test

    public void queryUser() {

        try {

            User user = new User();

            user.setEmailAddress("f...@bar.com");

            Key key = dao.save(user).now();

            assertNotNull(key);

            // This works; I can get the User by key

            User userByKey = (User)dao.get(key).get();

            assertEquals(user.getEmailAddress(), userByKey.getEmailAddress());

            // This works; I can get the User by Id

            User userById = (User) dao.get(User.class, key.getId()).get();

            assertEquals(user.getEmailAddress(), userById.getEmailAddress());

            // This doesn't work; foundUser is null

            User foundUser = (User)dao.filter(User.class, "emailAddress", "f...@bar.com").first().get();

            assertEquals(user.getEmailAddress(), foundUser.getEmailAddress());

        } catch (Exception e) {

            e.printStackTrace();

            fail(e.getMessage());

        }

    }

}

Am I doing this correctly?

Thanks,

Joe

[Joe J Ernst]

I was thinking that having the @Index annotation on the User's emailAddress field was enough, but if I understand you correctly I also need the (index=true) on the User's @EntitySubclass annotation.  I'll try that later today and verify that it fixes my problem.

Thanks for the response.

-Joe

            user.setEmailAddress("foo@bar.com");

            Key key = dao.save(user).now();

            assertNotNull(key);

            // This works; I can get the User by key

            User userByKey = (User)dao.get(key).get();

            assertEquals(user.getEmailAddress(), userByKey.getEmailAddress());

            // This works; I can get the User by Id

            User userById = (User) dao.get(User.class, key.getId()).get();

            assertEquals(user.getEmailAddress(), userById.getEmailAddress());

            // This doesn't work; foundUser is null

            User foundUser = (User)dao.filter(User.class, "emailAddress", "f...@bar.com").first().get();

            assertEquals(user.getEmailAddress(), foundUser.getEmailAddress());

        } catch (Exception e) {

            e.printStackTrace();

            fail(e.getMessage());

        }

    }

}

[Jeff Schnitzer]

The problem is the difference between these two lines:

ofy().load().type(Entity.class).filter("emailAddress", "f...@bar.com")

ofy().load().type(User.class).filter("emailAddress", "f...@bar.com")

The first line says "give me any Entity with emailAddress=f...@bar.com".  The second says "give me only User entities with emailAddress=f...@bar.com".  In order to select out only User entities (as opposed to other subclasses of Entity) an additional filter is performed on the discriminator.  If you don't index the User discriminator, it won't show up in the result set.

If you don't have any other Entity subclasses with an indexed field "emailAddress", then the result set will always be the same for either query.  But they are not semantically the same queries.

Jeff

[Joe J Ernst]

Thanks for the tip Jeff.  

I understand your comments below, but the problem for me was the lack of the index parameter on the EntitySubclass annotation for the User class. I got this working by adding "(index=true)" to the User class' @EntitySubclass annotation.  What I didn't understand was that the subclasses of Entity aren't indexed, even if I put @Index on the properties I want indexed.  By using @EntitySubclass(index=true) in addition to @Index, I see that the field is now indexed.

Thanks for your help.

-Joe

[Jeff Schnitzer]

It sounds like you're still somewhat confused.

@EntitySubclass(index=true) does not affect indexing of properties *at all*.  Either way, if you put @Index on the emailAddress field, that field *will* be indexed absolutely definitely positively.  You can run the first line below (the one with type(Entity.class)) and you will get back your User object, even with @EntitySubclass(index=false) (the default).

The problem is that by saying type(User.class), you're adding an additional requirement to the query based not on fields but based on the class itself - you want Users only.  This adds a hidden filter on the discriminator, which is stored in the underlying datastore.  Without indexing the discriminator, you queries for that value will not show up.

Jeff

[Joe J Ernst]

Thanks for you patience Jeff!

I think I understand why it worked when I added (index=true); it's because that caused the underlying discriminator to be indexed.  That sounds like extra overhead to me, and  led to the question, did I have my data model correct?

I originally had my base Entity class annotated with @Entity, and the User (and every other Entity subtype) annotated with @EntitySubclass.  Based on your comments, I moved the @Entity annotation to the User class (and every other class that directly extends Entity).  This means my base Entity class has no annotations, even though it is extended by all of my entities. I should only need the @EntitySubclass annotation on classes that further extend User, etc.

I hope that was the right answer, because all of my unit tests are passing and I'm ready to move on the the next concept that I'm struggling with! ;-)  

-Joe

[Jeff Schnitzer]

On Sun, Dec 9, 2012 at 3:57 PM, Joe J Ernst <joe....@ux1.tv> wrote:

Thanks for you patience Jeff!

I think I understand why it worked when I added (index=true); it's because that caused the underlying discriminator to be indexed.  That sounds like extra overhead to me, and  led to the question, did I have my data model correct?

I originally had my base Entity class annotated with @Entity, and the User (and every other Entity subtype) annotated with @EntitySubclass.  Based on your comments, I moved the @Entity annotation to the User class (and every other class that directly extends Entity).  This means my base Entity class has no annotations, even though it is extended by all of my entities. I should only need the @EntitySubclass annotation on classes that further extend User, etc.

I hope that was the right answer, because all of my unit tests are passing and I'm ready to move on the the next concept that I'm struggling with! ;-)  

There is a big difference between a polymorphic hierarchy of entities in the datastore (what you get with @Entity/@EntitySubclass) and a set of separate entities which happen to have a polymorphic Java structure.  @Entity/@EntitySubclass entities all share the same *kind*; a series of @Entity classes have different kinds.

I had assumed that your Entity class was just an example name, but if you actually had a class called Entity, you *definitely* do not want a true polymorphic hierarchy with @EntitySubclass.  Having polymorphic java objects is fine though.

Jeff

[Joe J Ernst]

To help solidify this concept I wrote a small demo hierarchy and JUnit tests.  While I was at it I posted an article, based on what I learned here, on my blog.

I'll gladly accept any feedback.

-Joe