Re: [objectify-appengine] Having trouble with @Embed of HashMap<String, T> using 4.0.a3

[Jeff Schnitzer]

At first glance the first problem is that generic fields like
Map<String, ROLE> can't work automatically. Objectify performs static
analysis of the class to determine how to store things, and with
erasure, ROLE is Object.

That said, there should be ways of making this work, not least by
specifying your own @Translate translators. Objectify can't guess
what translator to use, so you have to be explicit.

I'm surprised that you get back an empty value... if something was
saved, you should get something back (although not necessarily what
you want to get back).

Drop the @Embed from the field. @Embed should be specified on the
class that gets embedded. We may remove the ability to put @Embed on
fields before release.

Hope this at least gives you a *little* guidance to start...

Jeff

On Tue, Jun 19, 2012 at 5:20 PM, Wayne Rasmuss <wayne....@gmail.com> wrote:
> I've been having a heck of time with an embedded hash map. I'm using
> Objectify 4.0.a3. I included the jar downloaded from a link in a forum post.
>
> The class with the problem is pasted below. The problem is the roles member
> (private HashMap<String, ROLE> roles = new HashMap<String, ROLE>();) is not
> loaded. Currently I'm linitializing the value with an empty map so that's
> what I get, I've tried not initializing it, and I get roles = null. Finally,
> it seems like the map works in some contexts and not others, so I've
> included the class that references the problems class as well.
>
> I've spent some time with the debugger in the objectify code (acquired from
> GIT) I wasn't able to fetch 4.0.a3. So, I've been debugging 4.0.a1. So far,
> my breakpoints and stepping through code seems solid so I think I've been
> looking at the right code. I've attached some screen shots of my debugger at
> break points and the logs (ofy level = finest) while the object in questions
> is saved and loaded. From the logs it looks like the map is saved correctly,
> but it is loaded empty. The key for the entity is missing the member
> is List("d9c294dc-d800-4c5b-9b13-a488efe725ed")/LinkCollection(1) You should
> be able to go straight to it in the log using that.
>
> I'm going to keep looking the problem, but any help anyone can give would be
> greatly appreciated. I'll be happy to provide more information if required.
>
> Thanks!
> Wayne
>
> *************************************** THE CLASS WITH THE EMPTY MEMBER MAP
> *****************************************************
> /*
>  * Copyright (c) 2012. What's Next Software, LLC all rights reserved
>  */
>
> package whatsnext.exodus.coreservice.bidilinks;
>
> import com.googlecode.objectify.Key;
> import com.googlecode.objectify.annotation.Cache;
> import com.googlecode.objectify.annotation.Embed;
> import com.googlecode.objectify.annotation.Entity;
> import com.googlecode.objectify.annotation.Id;
> import com.googlecode.objectify.annotation.Load;
> import com.googlecode.objectify.annotation.Parent;
> import com.googlecode.objectify.annotation.Unindex;
> import whatsnext.exodus.coreservice.SimpleStore;
>
> import java.util.Collection;
> import java.util.HashMap;
> import java.util.HashSet;
> import java.util.LinkedList;
> import java.util.Map;
> import java.util.Set;
>
> /**
>  * This class contains a bunch of links. Each link may optionally have a
> role. Therefor this class must be careful
>  * to distinguish between items in the set that do not exist and items that
> exist, but do not have roles
>  * @param <T> the type of object that the relationship is to
>  * @param <ROLE> the type of object that represents the role. An enum is
> recommended and may be required in the future
>  */
> @Entity
>
> public class LinkCollection<T, P, ROLE> {
>
>     @Id
>     long id = 1;
>
>     @Parent
>     private Key<P> parent;
>
>     @Embed
>     @Unindex
>     private HashMap<String, ROLE> roles = new HashMap<String, ROLE>(); //<-
> THIS IS WHAT TURNS UP EMPTY
>
>     public LinkCollection(Key<P> parent) {
>         this.parent = parent;
>     }
>
>     private LinkCollection() {
>     }
>
>     boolean resolve(Map<T, ROLE> result, SimpleStore<T> store) {
>         Map<Key<T>, T> resolvedObjects = store.get(getKeys(), null);
>         Set<String> removed = new HashSet<String>();
>         for(Key<T> current : getKeys()) {
>             T currentReconciled = resolvedObjects.get(current);
>             if (currentReconciled != null) {
>
> result.put(currentReconciled,roles.get(current.getString()));
>             } else {
>                 removed.add(current.getString());
>             }
>         }
>         for (String toRemove : removed) {
>             roles.remove(toRemove);
>         }
>         return removed.size() > 0;
>     }
>
>     public ROLE get(Key<T> key) {
>         return roles.get(key.getString());
>     }
>
>     public void setParent(Key<P> parent) {
>         this.parent = parent;
>     }
>
>     @Override
>     public boolean equals(Object obj) {
>         boolean returnValue = false;
>         if (this == obj) {
>             returnValue = true;
>         } else if (obj instanceof LinkCollection) {
>             LinkCollection asMyType = (LinkCollection)obj;
>             returnValue = roles.equals(asMyType.roles);
>         }
>         return returnValue;
>     }
>
>     public boolean remove(Key<T> key) {
>         boolean returnValue = roles.containsKey(key.getString());
>         roles.remove(key.getString());
>         return returnValue;
>     }
>
>     public boolean put(Key<T> target, ROLE role) {
>         boolean returnValue = false;
>         if (role == null) {
>             returnValue = !roles.containsKey(target.getString());
>         }
>         ROLE oldValue = roles.put(target.getString(), role);
>         if (role != null) {
>             returnValue = !role.equals(oldValue);
>         }
>         return returnValue;
>     }
>
>     public Collection<Key<T>> getKeys() {
>         Collection<Key<T>> returnValue = new LinkedList<Key<T>>();
>         for(String asString : roles.keySet()) {
>             returnValue.add(Key.<T>valueOf(asString));
>         }
>         return returnValue;
>     }
>
>     public boolean hasRoleFor(Key<T> parent) {
>         return roles.containsKey(parent.getString());
>     }
> }
>
> *************************************** CLASS REFERENCING THE PROBLEM CLASS
> ABOVE *****************************************************
>
> import com.googlecode.objectify.annotation.Entity;
> import com.googlecode.objectify.annotation.Id;
> import com.googlecode.objectify.annotation.Load;
> import com.googlecode.objectify.annotation.Parent;
> import com.googlecode.objectify.annotation.Unindex;
> import com.googlecode.objectify.impl.ref.StdRef;
> import com.googlecode.objectify.util.ResultNow;
> import whatsnext.exodus.coreservice.SimpleStore;
>
> import java.util.Map;
>
> @Cache
> @Entity
> public class LeaderToFollowerLinks<LEADER, FOLLOWER, ROLE> {
>
>     @SuppressWarnings({"FieldCanBeLocal"})
>     @Id
>     private long id = 1;
>
>     @Parent
>     private Key<LEADER> parent;
>
>     @Unindex
>     private long version = 0;
>
>     @Unindex
>     private Ref<LinkCollection<FOLLOWER, LEADER, ROLE>> roles;  //<- HERE'S
> THE REFERENCE TO THE PROBLEM CLASS. I'VE TRIED EMBED, AND JUST A PLAIN
> REFERENCE
>
>     public LeaderToFollowerLinks(Key<LEADER> parent) {
>         this.parent = parent;
>         Key rolesKey = Key.create(parent, LinkCollection.class, 1l);
>         roles = new StdRef<LinkCollection<FOLLOWER, LEADER,
> ROLE>>(rolesKey);
>     }
>
>     protected LeaderToFollowerLinks() {
>
>     }
>
>     public void putRole(Key<FOLLOWER> target, ROLE role) {
>         if(roles.getValue() == null) {
>             roles.set(new ResultNow<LinkCollection<FOLLOWER, LEADER,
> ROLE>>(new LinkCollection<FOLLOWER, LEADER, ROLE>(parent)));
>         }
>         roles.getValue().put(target, role);
>     }
>
>     public void removeRole(Key<FOLLOWER> follower) {
>         if(roles.getValue() != null) {
>             roles.getValue().remove(follower);
>         }
>     }
>
>     public void resolveLinks(Map<FOLLOWER, ROLE> destination,
> SimpleStore<FOLLOWER> followerStore) {
>         if(roles.getValue()!= null) {
>             roles.getValue().resolve(destination, followerStore);
>         }
>     }
>
>     public long incrementVersion() {
>         return ++version;
>     }
>
>     public long getVersion() {
>         return version;
>     }
>
>     public ROLE getRoleFor(Key<FOLLOWER> follower) {
>         ROLE returnValue = null;
>         if(roles.getValue() != null) {
>             returnValue = roles.getValue().get(follower);
>         }
>         return returnValue;
>     }
>
>     public Key<LEADER> getParent() {
>         return parent;
>     }
>
>     @Override
>     public boolean equals(Object obj) {
>         boolean returnValue = false;
>         if (this == obj) {
>             returnValue = true;
>         } else if (obj instanceof LeaderToFollowerLinks) {
>             LeaderToFollowerLinks asMyType = (LeaderToFollowerLinks)obj;
>             returnValue = parent.equals(asMyType.getParent()) && id ==
> asMyType.id && version == asMyType.version && roles.equals(asMyType.roles);
>         }
>         return returnValue;
>     }
>
>     public boolean hasRoleFor(Key<FOLLOWER> follower) {
>         boolean returnValue = false;
>         if(roles.getValue() != null) {
>             returnValue = roles.getValue().hasRoleFor(follower);
>         }
>         return returnValue;
>     }
>
>     public LinkCollection<FOLLOWER, LEADER, ROLE> getRoles() {
>         return roles.getValue();
>     }
>
> }
>
>

[Wayne Rasmuss]

Thanks for the quick reply. I changed it back to @Serialize just to see if that worked and I found some other bugs along the way. I tried removing the @Embed and then I got an error that the actual type of the value in the map is not a supported property type. I think you basically described the problem above. Any guidance on using the @Translate annotation? I'm not familiar with it. to be honest, I haven't searched much yet, so I may be just fine on my own. Thanks again, I'm really impressed with Objectify

On Tuesday, June 19, 2012 7:32:45 PM UTC-5, Jeff Schnitzer wrote:

At first glance the first problem is that generic fields like
Map<String, ROLE> can't work automatically.  Objectify performs static
analysis of the class to determine how to store things, and with
erasure, ROLE is Object.

That said, there should be ways of making this work, not least by
specifying your own @Translate translators.  Objectify can't guess
what translator to use, so you have to be explicit.

I'm surprised that you get back an empty value... if something was
saved, you should get something back (although not necessarily what
you want to get back).

Drop the @Embed from the field.  @Embed should be specified on the
class that gets embedded.  We may remove the ability to put @Embed on
fields before release.

Hope this at least gives you a *little* guidance to start...

Jeff

[Jeff Schnitzer]

I guess the first question is what is it that you are trying to store
as a ROLE? Is it always an Object that should be broken down as
embedded?

Try putting @Translate(EmbedClassTranslatorFactory.class) on the
Map<String, ROLE> field. This will cause Objectify to always treat
the ROLE as an embedded class.

The concept of a translators is this: There are fundamentally three
representations of entity data in Objectify-land. There are your
entity POJOs. There is the low-level Entity object. And there's an
intermediate form which looks like a node tree (see class Node).
There is a lot of Black Magic involved in converting Entity<->Node,
but the conversion of Node<->POJO is performed by a well-structured
series of Translator objects.

When you register an entity class, Objectify introspects the class
(and any superclasses, and any embedded classes, etc) and constructs a
set of translators that are associated with the fields. The
translators are produced by TranslatorFactory objects. The @Translate
annotation causes Objectify to use a specific TranslatorFactory
instead of trying to figure it out the hard way.*

The issue is that "the hard way" doesn't work when the type is Object
- there's no introspection that can be done on generic fields. So the
@Translate annotation is necessary. Give it the translator factory
you expect - in this case, the EmbedClassTranslatorFactory.

Note that this means you cannot use String or Date or any similar
class as the ROLE.

Jeff

* For anyone paying attention to the code, this is actually something
of a lie. The @Translate annotation gets handled by the
TranslateTranslatorFactory, which just happens to be positioned
carefully in the registry chain. See TranslatorRegistry.java.

[Wayne Rasmuss]

    @Unindex

    @Translate(EmbedClassTranslatorFactory.class)

    private HashMap<String, ROLE> roles = new HashMap<String, ROLE>();

Yields

C:\whatsnext\components\apps\exodus\src\main\java\whatsnext\exodus\coreservice\bidilinks\LinkCollection.java:44: incompatible types

found   : java.lang.Class<com.googlecode.objectify.impl.translate.EmbedClassTranslatorFactory>

required: java.lang.Class<? extends com.googlecode.objectify.impl.translate.TranslatorFactory<?>>

    @Translate(EmbedClassTranslatorFactory.class)

The ofy code for this looked in order (though keep in mind I'm looking at 4.0.a1 source but running thr 4.0.a3 jar) I looked into my build files to make sure there's only one instance of objectify on the class path. I didn't look like it, but I'll do some gradle scripting later to verify.

Also, the error I get with just:

@Unindex

    private HashMap<String, ROLE> roles = new HashMap<String, ROLE>();

is:

Caused by: java.lang.IllegalArgumentException: roles.ahBleG9kdXMtd2hhdHNuZXh0ch4LEgRVc2VyIhQxODU4MDQ3NjQyMjAxMzkxMjQxMQw: whatsnext.exodus.contracts.core.lists.ListRole is not a supported property type.

ListRole is the class that in this case is ROLE, Here's the ListRole class, or enum rather. Right now it's annotated with @Serialize but I've tried it with @Embed and nothing and get the same error.

/*

 * Copyright (c) 2011. What's Next Software, LLC all rights reserved

 */

package whatsnext.exodus.contracts.core.lists;

import com.googlecode.objectify.annotation.Embed;

import com.googlecode.objectify.annotation.Serialize;

import java.io.Serializable;

@Serialize

public enum ListRole implements Serializable{

    owner, member, observer;

    private static final long serialVersionUID = 1L;

}

Thanks Again!

Wayne

[Wayne Rasmuss]

My final solution was to ditch the generic value in the map. That is to say: Map<String, ROLE> became Map<Sting, String> In my case, my values were always enums, so I decided to accept that and use the enum facilities to convert between enum values and string.

Thanks for all the help. Until I posted on the forum I was pretty much focused on the class that was referencing thet class that had there error. But this made it clear the problem was with the value. I'd be happy to try any suggestions to undo my hack if anyone has any suggests on handling an enum as the value in a map type.

Thanks

Wayne

[Jeff Schnitzer]

Enum should just work as-is: Map<String, MyEnum>

Enums are automatically converted to String and back.

Jeff

[Wayne Rasmuss]

Jeff,

Not need to worry about this any more on my account, but I figured you're like me an like to get to the bottom of stuff like this. So, I wrote some simpler classes to work from. They have the same results as before. I went through some of the fixes I tried and recorded the associated errors. 

The simplified classes are below, followed by various attempts at fixes and the encountered errors. I'm not too concerned about getting to the bottom of this right now, but if you have ideas or solutions I'd be happy to try them out.

Wayne

******** Classes ********

public class ClassContainingMap<T> {

    @Id

    Long autoId;

    HashMap<String, T> myMap = new HashMap<String, T>();

    public ClassContainingMap() {

    }

    public void put(String a, T val) {

        myMap.put(a, val);

    }

    public void print() {

        for(Map.Entry<String, T> entry : myMap.entrySet()) {

            System.out.println("Entry: "+entry);

        }

    }

}

public enum SomeEnum {

    val1, val2

}

public class LoaderAndSaver {

    private Key<ClassContainingMap<SomeEnum>> key;

    public void save() {

        ObjectifyService.register(ClassContainingMap.class);

        ClassContainingMap<SomeEnum> anInstace = new ClassContainingMap<SomeEnum>();

        anInstace.put("a", SomeEnum.val1);

        anInstace.put("b", SomeEnum.val2);

        key = ObjectifyService.begin().save().entity(anInstace).now();

    }

    public ClassContainingMap<SomeEnum> load() {

        return ObjectifyService.begin().load().key(key).get();

    }

}

******** Result of Code as is above ********

.

.

. 

Caused by: com.googlecode.objectify.SaveException: Error saving whatsnext.exodus.bootstrapservice.enumgenerictest.ClassContainingMap@1133b5a2: myMap.b: whatsnext.exodus.bootstrapservice.enumgenerictest.SomeEnum is not a supported property type.

at com.googlecode.objectify.impl.Transmog.save(Transmog.java:102)

at com.googlecode.objectify.impl.ConcreteEntityMetadata.save(ConcreteEntityMetadata.java:150)

at com.googlecode.objectify.impl.engine.WriteEngine.save(WriteEngine.java:68)

at com.googlecode.objectify.impl.cmd.SaverImpl.entities(SaverImpl.java:56)

at com.googlecode.objectify.impl.cmd.SaverImpl.entity(SaverImpl.java:33)

at whatsnext.exodus.bootstrapservice.enumgenerictest.LoaderAndSaver.save(LoaderAndSaver.java:21)

at whatsnext.exodus.bootstrapservice.session.SessionResource.retrieve(SessionResource.java:30)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Runtime.java:115)

at org.restlet.resource.ServerResource.doHandle(ServerResource.java:494)

... 73 more

Caused by: java.lang.IllegalArgumentException: myMap.b: whatsnext.exodus.bootstrapservice.enumgenerictest.SomeEnum is not a supported property type.

at com.google.appengine.api.datastore.DataTypeUtils.checkSupportedSingleValue(DataTypeUtils.java:186)

at com.google.appengine.api.datastore.DataTypeUtils.checkSupportedValue(DataTypeUtils.java:159)

at com.google.appengine.api.datastore.DataTypeUtils.checkSupportedValue(DataTypeUtils.java:125)

at com.google.appengine.api.datastore.PropertyContainer.setUnindexedProperty(PropertyContainer.java:122)

at com.googlecode.objectify.impl.Transmog.setEntityProperty(Transmog.java:438)

at com.googlecode.objectify.impl.Transmog.populateFields(Transmog.java:421)

at com.googlecode.objectify.impl.Transmog.populateFields(Transmog.java:427)

at com.googlecode.objectify.impl.Transmog.populateFields(Transmog.java:427)

at com.googlecode.objectify.impl.Transmog.save(Transmog.java:361)

at com.googlecode.objectify.impl.Transmog.save(Transmog.java:97)

... 85 more

******** First attempt at fix ********

Change this line:

@Entity

public class ClassContainingMap<T> {

To this line

@Entity

public class ClassContainingMap<T extends Enum<T>> {

Yields:

at com.googlecode.objectify.impl.ref.StdRef.get(StdRef.java:54)

at whatsnext.exodus.bootstrapservice.enumgenerictest.LoaderAndSaver.load(LoaderAndSaver.java:25)

at whatsnext.exodus.bootstrapservice.session.SessionResource.retrieve(SessionResource.java:31)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Runtime.java:115)

at org.restlet.resource.ServerResource.doHandle(ServerResource.java:494)

... 73 more

Caused by: java.lang.ClassCastException: sun.reflect.generics.reflectiveObjects.TypeVariableImpl cannot be cast to java.lang.Class

at com.googlecode.objectify.impl.translate.EnumTranslatorFactory$1.loadValue(EnumTranslatorFactory.java:29)

at com.googlecode.objectify.impl.translate.EnumTranslatorFactory$1.loadValue(EnumTranslatorFactory.java:1)

at com.googlecode.objectify.impl.translate.ValueTranslator.loadPropertyValue(ValueTranslator.java:41)

at com.googlecode.objectify.impl.translate.PropertyValueNodeTranslator.loadAbstract(PropertyValueNodeTranslator.java:25)

at com.googlecode.objectify.impl.translate.AbstractTranslator.load(AbstractTranslator.java:25)

at com.googlecode.objectify.impl.translate.MapTranslatorFactory$1.loadMapIntoExistingMap(MapTranslatorFactory.java:76)

at com.googlecode.objectify.impl.TranslatableProperty.executeLoad(TranslatableProperty.java:67)

at com.googlecode.objectify.impl.translate.ClassTranslator.loadMap(ClassTranslator.java:80)

at com.googlecode.objectify.impl.translate.MapNodeTranslator.loadAbstract(MapNodeTranslator.java:25)

at com.googlecode.objectify.impl.translate.AbstractTranslator.load(AbstractTranslator.java:25)

at com.googlecode.objectify.impl.Transmog.load(Transmog.java:82)

at com.googlecode.objectify.impl.Transmog.load(Transmog.java:71)

^^^^^^^^^^^^^^^^^^^^^^^ Second Attempt *****************************************************

Add @Translate

    @Translate(EmbedClassTranslatorFactory.class)

    HashMap<String, T> myMap = new HashMap<String, T>();

Yields compiler error (pretty sure there's no duplicate jars on the class path or anything)

 

C:\whatsnext\components\apps\exodus\src\main\java\whatsnext\exodus\bootstrapservice\enumgenerictest\ClassContainingMap.java:23: incompatible types

found   : java.lang.Class<com.googlecode.objectify.impl.translate.EmbedClassTranslatorFactory>

required: java.lang.Class<? extends com.googlecode.objectify.impl.translate.TranslatorFactory<?>>

    @Translate(EmbedClassTranslatorFactory.class)

******************* Third attempt **************************************************************

Add embed to SomeEnum and remove @Translate causing compile error from map

@Embed

public enum SomeEnum {

    val1, val2

}

Yield same as First attempt (highlight below)

Caused by: java.lang.ClassCastException: sun.reflect.generics.reflectiveObjects.TypeVariableImpl cannot be cast to java.lang.Class

******************* Third attempt **************************************************************

Try the @Embed without Attempt One (T extends Enum<T>) above

Yields same problem as original code above

Caused by: java.lang.IllegalArgumentException: myMap.b: whatsnext.exodus.bootstrapservice.enumgenerictest.SomeEnum is not a supported property type.

[Wayne Rasmuss]

I realized I forgot the code calling the example I posted previously. Here it is:

        LoaderAndSaver tester = new LoaderAndSaver();

        tester.save();

        ClassContainingMap<SomeEnum> loaded = tester.load();

        loaded.print();