The explanation:
In the past, a Key<?> contained the POJO entity className en lieu of
Kind. This worked because there was a 1-to-1 mapping between class
names and datastore Kinds, and Objectify could map back and forth
just-in-time.
Polymorphism breaks this 1-to-1 relationship; all the classes in a
polymorphic hierarchy have the same Kind. Thus the new Key<?>
actually wraps a datastore native Key... a much simpler and more
elegant solution. But now think about this:
Key<Car> key = new Key<Car>(Car.class, 123);
...this means that the Key<?> constructor needs to know how to
construct a datastore Key from Car.class. Thus it needs to know the
Kind of a Car... which might be Vehicle... and the only way to know
that is by examining the annotations of Car.class.
But GWT-land does not have annotations. Class.getAnnotation() is not
part of the JRE emulation.
Basically, the polymorphism feature makes it impossible to construct
Key<?> in GWT code. I have tried to think of ways around the problem,
but I'm starting to believe it's intractable. The original strategy
of putting the classname in the Key<?> won't work because there's no
way to go backwards - picture a keys-only query that must return
Key<?> objects, but doesn't have discriminator values from a fetched
Entity to determine what the correct POJO class is. We can't
regenerate the classname.
Soooo... what I'm going to do is provide super-source for Key<?> that
makes the constructors that take a Class<?> throw an
UnsupportedOperationException when called from GWT. You can't
construct a Key<?> in GWT, although you can pass them back and forth
through GWT-RPC. I considered removing the constructors entirely and
forcing Key<?> creation in a KeyFactory (similar to the low-level API)
but that feels like too much API change. Besides, I like "new
Key<Car>(Car.class, 123)" better than
"KeyFactory.<Car>create(Car.class, 123)".
Unless someone has a brilliant insight, this looks like a situation
where we have to make a decision "is Objectify a persistance framework
or a GWT framework?" and the answer must be that persistence comes
first.
Jeff
and the answer must be that persistence comes
first.
It is impossible to offer a client side mechanism that builds a Key<?>
from Class<?> - factories don't help.
However, all hope is not lost. Key<?> now has a constructor that
takes a native datastore Key. Even though Objectify can't figure out
what the Kind is for your classes, you can - if you aren't using
polymorphism, and you aren't using @Entity(name="SomethingElse"), you
can guess what the Kind will be. Furthermore, Key<?> is not a final
class - you can add your own constructor. Nothing stops you from
making a class like this:
public class GWTKey<T> extends Key<T> {
public GWTKey(Key<?> parent, Class<T> clazz, long id) {
// Note: Class<?>.getSimpleName() is not part of GWT's JRE emulation
String kind = clazz.getName().substring(clazz.lastIndexOf('.'));
// You must manage polymorphism yourself
if (kind.equals("Mammal"))
kind = "Animal";
this.raw = KeyFactory.createKey(parent == null ? null :
parent.getRaw(), kind, id);
}
}
Notice that it will be chock-full of application-specific business
logic. Ugly, but it will do the job.
One other thing worth mentioning - I hope you are thinking about the
potential hazards of generating keys client-side. Either you must
validate the key objects carefully or you are assuming that the client
is trustworthy - because anyone could create any Key<?> object and
pass it through your RPC boundary. This could conceivably be abused
to fetch invalid information or to store corrupt information in the
database. This might not be an issue for an internal app with a
trusted user base, but a public API is subject to attack.
Jeff
One other thing worth mentioning - I hope you are thinking about the
potential hazards of generating keys client-side. Either you must
validate the key objects carefully or you are assuming that the client
is trustworthy - because anyone could create any Key<?> object and
pass it through your RPC boundary. This could conceivably be abused
to fetch invalid information or to store corrupt information in the
database. This might not be an issue for an internal app with a
trusted user base, but a public API is subject to attack.
Sorry. I wish there was a better way, but I can't think of one.
Actually, I just did. I'll bet it is possible to create a GWT
generator that creates and hides all that application-specific
business logic inside the generated Key<?> constructor. Unfortunately
my GWT-fu is not strong enough to actually do that right now, and I
don't have time to learn. Anyone else want to take a crack at it?
> This warning is a little over the top, Jeff, at least to me and here's
> why...
>
> Trusting the user means that the user is authenticated and is authorized. A
> clever hacker, a site member, employing a subdomain high-jacking (replacing
> the real session id with their own valid session id) passes both tests
> because as a valid user they were able to obtain a valid session id which
> they then use to replace the real one. In this case and numerous other types
> of hijacking scenarios the user will always appear to be authenticated and
> authorized even though they are up to no good. Even sites sitting behind
> firewalls are subject to these types of attacks and many have been breached
> including Google.
I wasn't considering subdomain hijacking. I was just thinking about
the level of data isolation that is usually required for public-facing
applications; user data must be segregated, and written data must be
protected against malicious injection. It sounds like your
application does not face the public so you can "trust" users not to
be malicious - that's great.
Jeff
On Thu, Mar 3, 2011 at 11:51 AM, Jeff Schwartz <jeffts...@gmail.com> wrote:Sorry. I wish there was a better way, but I can't think of one.
>
> Ugly is putting it mildly, seriously, & I wouldn't bother littering my
> client-side code with it. I'd rather bite the bullet and do a full scale
> refactoring job of the client and server to use RPC calls passing ids
> instead.
Actually, I just did. I'll bet it is possible to create a GWT
generator that creates and hides all that application-specific
business logic inside the generated Key<?> constructor. Unfortunately
my GWT-fu is not strong enough to actually do that right now, and I
don't have time to learn. Anyone else want to take a crack at it?
> This warning is a little over the top, Jeff, at least to me and here'sI wasn't considering subdomain hijacking. I was just thinking about
> why...
>
> Trusting the user means that the user is authenticated and is authorized. A
> clever hacker, a site member, employing a subdomain high-jacking (replacing
> the real session id with their own valid session id) passes both tests
> because as a valid user they were able to obtain a valid session id which
> they then use to replace the real one. In this case and numerous other types
> of hijacking scenarios the user will always appear to be authenticated and
> authorized even though they are up to no good. Even sites sitting behind
> firewalls are subject to these types of attacks and many have been breached
> including Google.
the level of data isolation that is usually required for public-facing
applications; user data must be segregated, and written data must be
protected against malicious injection. It sounds like your
application does not face the public so you can "trust" users not to
be malicious - that's great.
Jeff
You are assuming that only authz'd people would be using a Key. There
is many usecases where that isn't necessary. Anonymous commenting
systems come to mind.
jon
I wasn't assuming that at all. I was merely trying to show that even authenticated and authorized users aren't safe.
Sorry, I guess I didn't make my attack scenario clear. Maybe this
affects your app, maybe not, but if you're creating Key<?> objects on
the client and passing them to the server, there are some attack
scenarios like:
* Say you have an RPC method getThing(Key<?>) that lets you define
what thing to grab. An authenticated, valid malicious user could call
this method making up their own Key<?>. They could request an id for
data that is associated with another user. In a polymorphic system,
they might be able to request access to other different kinds of
things without blowing up the typechecker.
* Say you have an RPC method write(MyEntity) that puts an entity in
the datastore. Say one of the member fields is a Key<?>. A malicious
user could write a MyEntity with a Key pointing to some odd part of
the database; now, when ever your app tries to load the entity pointed
at by that Key, it might throw ClassCastExceptions because it's
getting the wrong kind of data.
All I'm saying is that if you pass keys through untrusted RPC
boundaries, you need to be careful about validating them. You still
have this issue with ids, just less so. Don't trust clients, even if
they're authenticated.
> I realize that you are probably as upset about this breaking change as I am
> and I appreciate that. I agree, persistence rules the day and I would never
> stand in the way of progress so I'll refactor my code when I cut over to the
> new version.
Another option is to stay on Objectify 2.2.3 for a while. Unless you
need async get/put/delete or polymorphism, there's nothing wrong with
the old version. Who knows, I might get around to building that GWT
generator sometime in the next year.
Jeff
All valid scenarios.
I think I just might hold off on upgrading for a while, at least until I actually need pollymorphims or sync.
Also, I wouldn't be so down on the polymorphism feature - it's
something that just eventually comes up in a schema. You don't care
about it until suddenly you need it.
Is your difficulty a question of persistent serialization of old keys,
or a question of creating Key<?>s in GWT client-side? If the later,
you can work around the issue by using the constructor that takes a
native Key. You just need to know how to construct that Key properly
- which really just means knowing what the Kind is called. Unless you
have some pretty magical code, the kind for class Thing is almost
always "Thing".
If the former - run two passes, one that converts Key<?> to native
datastore Key, then upgrade Ofy, then convert back to Key<?>. Or just
leave your data with native Keys. Or (if your keys don't have
parents) just use the simple Long/String. Serializing complex classes
to persistent storage is always just a little bit precarious for
exactly this reason. I don't expect Key<?> to change in a
serialization-incompatible way ever again, but I won't make any hard
promises.
Jeff
http://blog.jdevelop.eu/2010/01/17/use-generators-to-create-boilerplate-code-in-gwt-20/
This would be used to create the client-side source for a class that
maps Class<?> to kind based on the annotations of all entities. The
generated could would just be:
if (clazz == Entity1.class)
return "Entity1";
else ...etc
Then the Key<?> super-source could use this to properly construct a
native datastore Key and wrap it.
Jeff
Key<OnlineOrder> orderKey = new Key<OnlineOrder>(OnlineOrder.class, 123);
Key<OrderLine> lineKey = new Key<OrderLine>(orderKey, OrderLine.class, 456);
... use these lines instead.
Key<OnlineOrder> orderKey = OnlineOrderKeyFactory.createKey(123);
Key<OrderLine> lineKey = OrderLineKeyFactory(orderKey, 456);
new Key<Cat>(Cat.class, id);
...but on the server you would get the normal Key<?> constructor that
does Kind lookup via annotations, on the client you would get a
generated Key<?> constructor that does Kind lookup via "if (clazz ==
Mammal.class) ... else if ...".
This is just going to take some research.
Jeff
I wrote a gwt generator that worked on the example that phillipe previously posted, however you had to have that extra inner class, and it would only work on client code, not shared code.
If you are saying you don't want to have to figure out the base entity that had been sub classed, you don't have to. The annotation processor automatically figures out the base kind of the subclass.
interface HasReferences {Key<? extends HasRefences> getKey();// just an interface,}@Entitypublic class Person implements HasReferences {@Id private long id;Key<? extends HasRefences> getKey() {return PersonKeyFactory.createKey(this.id);}}@Entitypublic class Company implements HasReferences {
@Id private long id;
Key<? extends HasReferences> getKey()
}return CompanyKeyFactory.createKey(this.id);}
// somewhere in GWT:List<HasReferences> hasReferences = getHasReferences();for (HasReferences hr : hasReferences) {Key<? extends HasReferences> k = hr.getKey();}
Right now, Ofy's Key.java is separated between a server version and a
super-source version. This is why there is no Key(Class<?>, long)
constructor on the client.
The client version of Key.java could look like this:
public class Key<T> implements Serializable, Comparable<Key<?>> {
...
static ObjectifyClientKeyFactory keyFact =
GWT.create(ObjectifyCliebntKeyFactory.class);
...
public Key(Class<?> clazz, long id) {
String kind = keyFact.getKind(clazz);
// do the rest of the constructor now that we have the kind
}
}
Now the only trick is how to generate an ObjectifyClientKeyFactory
that looks something like this:
class ObjectifyClientKeyFactory {
static Map<Class<?>, String> kinds = new HashMap<Class<?>, String>();
static {
kinds.put(Thing.class, "Thing");
kinds.put(Animal.class, "Animal");
kinds.put(Mammal.class, "Animal");
kinds.put(Cat.class, "Animal");
}
}
The problem is that you now need the registration information within
the generator. This is somewhat tricky but doesn't seem like an
insurmountable problem... maybe even a good time to introduce
classpath scanning.
Jeff
public <T extends HasLabels> SafeOperationResult<T> tagEntityWithLabel(Key<T> entityKey, LabelEntity label) throws AppSystemException;
The Key would be constructed client-side (we took care to rewrite that code to create and wrap native datastore Keys) and then it would be sent over the RPC layer. Some debugging revealed that the problem lies in our use of Datastore Namespaces. The problem lies in the fact that the GWT super-source of this raw Key has no notion of namespaces, but the server side Key does. So when it is passed over the RPC layer, it is deserialized with the default ("") namespace instead of the actual namespace active on the server side.
Is this something we could fix within Objectify, or is it something that should be handled at GWT?
For now I have resorted to creating more specific tag*WithLabel methods which simply take the object's ID and create the Key<?> server side. If anyone has any ideas as to how we could solve this I'd be very interested :)
--JH
So... all is not well in GWT-land with the new Key<?>. In short: You
cannot create Key<?> objects client-side. I don't see any way to work
around this issue.The explanation:
In the past, a Key<?> contained the POJO entity className en lieu of
Kind. This worked because there was a 1-to-1 mapping between class
names and datastore Kinds, and Objectify could map back and forth
just-in-time.Polymorphism breaks this 1-to-1 relationship; all the classes in a
polymorphic hierarchy have the same Kind. Thus the new Key<?>
actually wraps a datastore native Key... a much simpler and more
elegant solution. But now think about this:Key<Car> key = new Key<Car>(Car.class, 123);
...this means that the Key<?> constructor needs to know how to
construct a datastore Key from Car.class. Thus it needs to know the
Kind of a Car... which might be Vehicle... and the only way to know
that is by examining the annotations of Car.class.But GWT-land does not have annotations. Class.getAnnotation() is not
part of the JRE emulation.Basically, the polymorphism feature makes it impossible to construct
Key<?> in GWT code. I have tried to think of ways around the problem,
but I'm starting to believe it's intractable. The original strategy
of putting the classname in the Key<?> won't work because there's no
way to go backwards - picture a keys-only query that must return
Key<?> objects, but doesn't have discriminator values from a fetched
Entity to determine what the correct POJO class is. We can't
regenerate the classname.Soooo... what I'm going to do is provide super-source for Key<?> that
makes the constructors that take a Class<?> throw an
UnsupportedOperationException when called from GWT. You can't
construct a Key<?> in GWT, although you can pass them back and forth
through GWT-RPC. I considered removing the constructors entirely and
forcing Key<?> creation in a KeyFactory (similar to the low-level API)
but that feels like too much API change. Besides, I like "new
Key<Car>(Car.class, 123)" better than
"KeyFactory.<Car>create(Car.class, 123)".Unless someone has a brilliant insight, this looks like a situation
where we have to make a decision "is Objectify a persistance framework
or a GWT framework?" and the answer must be that persistence comes
first.Jeff