Polymorphic objects

[Ryan Chazen]

Hey - what's the best way to use twig for something like this?

class Person {
 Long id;
 Arm rightArm;
 Arm leftArm;
}

class Arm {
 Long id;
 int numScars;
}

class BrokenArm extends Arm {
 int numFractures;
}

Pretty simple... couple questions:

1) Do I need to add @Entity to these classes?
2) Should I add @Id to 'Long id' ?
3) Do I need to do anything special to be able to store both Arm and BrokenArm classes?

I tried

        ObjectDatastoreFactory.register(Arm.class);
        ObjectDatastoreFactory.register(BrokenArm.class);

but I get the following error:

WARNING: Nested in java.lang.ExceptionInInitializerError:
java.lang.IllegalArgumentException: Kind name  was already mapped to class x.Arm

Line number for the error is the "ObjectDatastoreFactory.register(BrokenArm.class);" line.
I'm guessing I need to add some kind of annotation?

[Ryan Chazen]

Worked out the Kind name issue - it came from specifying @Entity

New issues though

4) If I save a BrokenArm instance, I can only find it again if I search by type for BrokenArm. If I do a find on Arm.class, I don't get back the BrokenArm classes.
Is there a way to specify polymorphism? I tried @Entity(polymorphic=true) but that didn't solve it.

5) If I set the ID explicitly for a new Person, eg. person.id = 7
I then run
            ObjectDatastore datastore = new AnnotationObjectDatastore(false);
            datastore.store().instance(person).now();

When I check the data though, the first person will have id '7', but subsequent inserts of a person with person.id = 7 will generate brand new IDs (8, 12, 30, etc)
I'm guessing twig is checking if that ID already exists, and if it does, generating a new one. Is there an option to force it to use the specified ID, even if that ID exists (ie. don't check for existence)?

[John Patterson]

Hi ryan, answers inline…

On 29/11/2012, at 10:55 PM, Ryan Chazen wrote:

1) Do I need to add @Entity to these classes?

No it is not required.

2) Should I add @Id to 'Long id' ?

Only if you want to.  @Id fields are not required in Twig.  If your application needs to know the id then put one there.

3) Do I need to do anything special to be able to store both Arm and BrokenArm classes?

No these are stored as referenced entities (not embedded) and will have correct type information 

I tried

        ObjectDatastoreFactory.register(Arm.class);
        ObjectDatastoreFactory.register(BrokenArm.class);

but I get the following error:

WARNING: Nested in java.lang.ExceptionInInitializerError:
java.lang.IllegalArgumentException: Kind name  was already mapped to class x.Arm

Line number for the error is the "ObjectDatastoreFactory.register(BrokenArm.class);" line.
I'm guessing I need to add some kind of annotation?

I see you figured this was due to an unnecessary Entity annotation.

[John Patterson]

On 30/11/2012, at 1:44 AM, Ryan Chazen wrote:

Worked out the Kind name issue - it came from specifying @Entity

New issues though

4) If I save a BrokenArm instance, I can only find it again if I search by type for BrokenArm. If I do a find on Arm.class, I don't get back the BrokenArm classes.
Is there a way to specify polymorphism? I tried @Entity(polymorphic=true) but that didn't solve it.

Ah yes polymorphic queries are not supported.  In most cases it is easy enough to do two queries and join the results.

The reason it is problematic is that they two (or more) subclasses must all share a common "kind" in for a single query to work.  This means that a "discriminator" value is needed so that Twig knows what class to actually instantiate.  So far so good.

The problem though, is that a simple Key in another entity that references your subclass does not know which type it is just from the key.

In your example, Person would have a Key that points to "Arm(1)" which may actually be a BrokenArm.  Twig needs to know the exact class when it creates Person (maybe without loading the Arm also).  But that information is only available after loading the Arm entity.  

So basically it is no longer possible to have an "unactivated" arm.  

This restriction could be OK - just document it well.  I just haven't needed to do it myself to bother implementing it.  Another solution is to store the discriminator property also in person.

Patches welcome :)

5) If I set the ID explicitly for a new Person, eg. person.id = 7
I then run
            ObjectDatastore datastore = new AnnotationObjectDatastore(false);
            datastore.store().instance(person).now();

When I check the data though, the first person will have id '7', but subsequent inserts of a person with person.id = 7 will generate brand new IDs (8, 12, 30, etc)

This does not sound right.  If you store() an instance and se the id it will use that id when it is stored.

I'm guessing twig is checking if that ID already exists, and if it does, generating a new one.

No this does not happen.  It would mean that more datastore operations are needed to store any entity.  Twig is very careful to do things in an efficient manner.

The closest thing to what you describe is if you use @Unique annotation, Twig will check for an existing instance but it will throw an exception if one already exists.

Is there an option to force it to use the specified ID, even if that ID exists (ie. don't check for existence)?

If you do really see this behaviour please create a simple test case (extend LocalDatastoreTestCase is easier).  I can only imagine that you do not have @Id set on the id field and so the id is being auto generated.  But then, the id field will also not be set after storing the entity…. so I don't understand what is happening in your app.

John

[Ryan Chazen]

The code I am running is as follows:

class Creature {
 @Id Long id;
}

class Person extends Creature {
 Arm leftArm, rightArm;
}

{
        for (int i = 0; i < 5; i++) {
            Person person = new Person();
            person.id = (long) i;

            ObjectDatastore datastore = new AnnotationObjectDatastore(false);
            datastore.store().instance(person).now();
        }
       

        Thread.sleep(5000);

        ObjectDatastore datastore = new AnnotationObjectDatastore(false);

        List<Person> persons = datastore.find().type(Person.class).returnAll().now();
        for (Person person : persons) {
            System.out.println(person.id);
        }
}

I'll try add it into a test.. I'm guessing the problem is coming because the @Id is inside the superclass?

[John Patterson]

Ok, I've run this code and it works fine.

I get printed 1 2 3 4 5

keep in mind that id 0 will be replaced with an auto-generated id so 1 is probably replaced.

What do you find wrong with this result?

[John Patterson]

On 30/11/2012, at 8:26 PM, John Patterson wrote:

> Ok, I've run this code and it works fine.
>
> I get printed 1 2 3 4 5

I meant 1 2 3 4

[John Patterson]

On 30/11/2012, at 4:16 PM, John Patterson wrote:

3) Do I need to do anything special to be able to store both Arm and BrokenArm classes?

No these are stored as referenced entities (not embedded) and will have correct type information 

BTW, embedded entities can also be polymorphic.  Twig will store a discriminator property "$class" with the registered type name so it knows what type to construct.

[Ryan Chazen]

My bad - I had simplified the test as it seemed to still be returning the error for me. The error comes when you set the leftArm / rightArm also.

It seems ID is not unique across classes? I get the following error too:

java.lang.IllegalArgumentException: Invalid type id 'x.Person' (for id type 'Id.class'): Class x.Arm is not assignable to x.Arm

This is by changing the code above as follows:

        for (int i = 0; i < 5; i++) {
            Person person = new Person();
            person.id = (long) i;

            person.leftArm = new Arm();
            person.leftArm.id = (long) i;

[Ryan Chazen]

Also, I'm not sure why, but when viewing the person entities in /_ah/admin, I see the following IDs after running the code 3-4 times:

2
3
4
102
120

I'm using the RC .jar in the downloads page - have their been bugfixes since in the repo?

[John Patterson]

Firstly can you post at least enough the the stack trace that will show where the exception is being thrown?

and where the hell did Id.class come from?

[John Patterson]

Ok I have run your updated code and it also runs perfectly fine.

The field Arm.id is not annotated as an @Id so the id will be auto-generated (as there is no @Id field)

This is the test case I have run

import java.util.List;

import org.junit.Test;

import com.google.code.twig.annotation.AnnotationObjectDatastore;

import com.google.code.twig.annotation.Id;

public class RyanTest extends LocalDatastoreTestCase

{

{

ObjectDatastoreFactory.register(Person.class);

ObjectDatastoreFactory.register(Arm.class);

}

static class Creature {

@Id Long id;

}

static class Arm {

Long id;

int numScars;

}

static class Person extends Creature {

Arm leftArm, rightArm;

}

@Test

public void run()

{

        for (int i = 0; i < 5; i++) {

            Person person = new Person();

            person.id = (long) i;

            person.leftArm = new Arm();

            person.leftArm.id = (long) i;

            

            ObjectDatastore datastore = new AnnotationObjectDatastore(false);

            datastore.store().instance(person).now();

        }

        

        try

{

Thread.sleep(5000);

}

catch (InterruptedException e)

{

e.printStackTrace();

}

        ObjectDatastore datastore = new AnnotationObjectDatastore(false);

        List<Person> persons = datastore.find().type(Person.class).returnAll().now();

        for (Person person : persons) {

            System.out.println(person.id);

            System.out.println(person.leftArm.id);

        }

}

}

On 30/11/2012, at 8:43 PM, Ryan Chazen wrote:

[John Patterson]

Thats as much time as I can spare on this debugging.  You will need to clearly show what is actually failing with a test case.  Modify the one below if you want

[Ryan Chazen]

Hey John

Sorry for the wasted time - my bad! I was messing around with twig to try and determine what was happening, which seems to be where that strange exception came from.

The actual 'bug' was very simple. If an @Id Long is set to '0', its the same as that Id being null, so twig/datastore creates a brand new unique ID for the entry.
So everytime I ran the test code, it created 1 new entry. Changing the loop to start at 1 rather than at 0 fixed the issue.
The error never showed up for you as you only ran the test code once - if you run it a few more times you will end up with new Ids popping in.

Thanks for your time! I'm going to look into how I can allow for polymorphic queries - Objectify's approach of having a multivalue field with all types an object can be seems to be the best approach.
eg. special kind on any saved entity tagged with @Polymorphic or similar that can be queried - {"x.Arm","x.BrokenArm"}
Do you think that approach would work within twig? (allowing multiple 'kinds' per entity)

[John Patterson]

On 30/11/2012, at 9:55 PM, Ryan Chazen wrote:

> Hey John
>
> Sorry for the wasted time - my bad! I was messing around with twig to try and determine what was happening, which seems to be where that strange exception came from.
>
> The actual 'bug' was very simple. If an @Id Long is set to '0', its the same as that Id being null, so twig/datastore creates a brand new unique ID for the entry.
> So everytime I ran the test code, it created 1 new entry. Changing the loop to start at 1 rather than at 0 fixed the issue.
> The error never showed up for you as you only ran the test code once - if you run it a few more times you will end up with new Ids popping in.
>
> Thanks for your time! I'm going to look into how I can allow for polymorphic queries - Objectify's approach of having a multivalue field with all types an object can be seems to be the best approach.
> eg. special kind on any saved entity tagged with @Polymorphic or similar that can be queried - {"x.Arm","x.BrokenArm"}
> Do you think that approach would work within twig? (allowing multiple 'kinds' per entity)

Yes it would work and in fact if you look at PolymorphicTranslator that is almost exactly what it does (only single value though).

This translator is only used for @Embed'ed objects though, although I can;t see why it wouldn't also work for "top level" objects.

Doing the query part though is a different matter though. It might be more complex… not sure.

Ask yourself if you really need to do a polymorphic query? If so it may be a lot easier to just do two queries!

[Ryan Chazen]

Big issue is for listing objects. Arm and BrokenArm. If I want to give the user a scrolling list of all Arms sorted by name or number of breaks, etc, then 2 queries won't work. You could compromise by having two separate lists or disallowing sorting, but it still makes it much more complex as you now have to deal with Cursor objects from two different queries to do infinite scrolling...

I'll take a look at PolymorphicTranslator - AFAIK underlying datastore treats equal filters on multi-property fields the same way it does for single-property, so simply adding multiple Kind values may make it easy to add in...

[John Patterson]

On 30/11/2012, at 11:26 PM, Ryan Chazen wrote:

> Big issue is for listing objects. Arm and BrokenArm. If I want to give the user a scrolling list of all Arms sorted by name or number of breaks, etc, then 2 queries won't work. You could compromise by having two separate lists or disallowing sorting, but it still makes it much more complex as you now have to deal with Cursor objects from two different queries to do infinite scrolling...

Well if you get something good together then send me a pull request.

Keep in mind the problem of unactivated references to these polymorphic types. Person.arm could never be "unactivated" unless another field was stored with the Arm Key with the type in it.

To me this is the best solution - encoding the reference to an Arm as 2 properties:

arm=KEY("Arm", 1)
arm$class="BrokenArm"

the way I see it, the discriminator value is an "extra" detail of the reference.