The new Key<?> and GWT
Jeff Schnitzer
cannot create Key<?> objects client-side. I don't see any way to work
around this issue.
The explanation:
In the past, a Key<?> contained the POJO entity className en lieu of
Kind. This worked because there was a 1-to-1 mapping between class
names and datastore Kinds, and Objectify could map back and forth
just-in-time.
Polymorphism breaks this 1-to-1 relationship; all the classes in a
polymorphic hierarchy have the same Kind. Thus the new Key<?>
actually wraps a datastore native Key... a much simpler and more
elegant solution. But now think about this:
Key<Car> key = new Key<Car>(Car.class, 123);
...this means that the Key<?> constructor needs to know how to
construct a datastore Key from Car.class. Thus it needs to know the
Kind of a Car... which might be Vehicle... and the only way to know
that is by examining the annotations of Car.class.
But GWT-land does not have annotations. Class.getAnnotation() is not
part of the JRE emulation.
Basically, the polymorphism feature makes it impossible to construct
Key<?> in GWT code. I have tried to think of ways around the problem,
but I'm starting to believe it's intractable. The original strategy
of putting the classname in the Key<?> won't work because there's no
way to go backwards - picture a keys-only query that must return
Key<?> objects, but doesn't have discriminator values from a fetched
Entity to determine what the correct POJO class is. We can't
regenerate the classname.
Soooo... what I'm going to do is provide super-source for Key<?> that
makes the constructors that take a Class<?> throw an
UnsupportedOperationException when called from GWT. You can't
construct a Key<?> in GWT, although you can pass them back and forth
through GWT-RPC. I considered removing the constructors entirely and
forcing Key<?> creation in a KeyFactory (similar to the low-level API)
but that feels like too much API change. Besides, I like "new
Key<Car>(Car.class, 123)" better than
"KeyFactory.<Car>create(Car.class, 123)".
Unless someone has a brilliant insight, this looks like a situation
where we have to make a decision "is Objectify a persistance framework
or a GWT framework?" and the answer must be that persistence comes
first.
Jeff
Jeff Schwartz
Jeff Schwartz
Jeff Schwartz
http://jefftschwartz.appspot.com/
http://www.linkedin.com/in/jefftschwartz
follow me on twitter: @jefftschwartz
Jeff Schwartz
Jeff Schwartz
Jeff Schwartz
Jeff Schwartz
George Moschovitis
and the answer must be that persistence comes
first.
Jeff Schnitzer
now requires that you know the Kind (to construct the underlying
native Key). The only way you can know the Kind is to examine
annotations (@Subclass and @Entity(name="blah"). But annotations do
not exist in GWT - Class.getAnnotation() is not part of the JRE
emulation.
It is impossible to offer a client side mechanism that builds a Key<?>
from Class<?> - factories don't help.
However, all hope is not lost. Key<?> now has a constructor that
takes a native datastore Key. Even though Objectify can't figure out
what the Kind is for your classes, you can - if you aren't using
polymorphism, and you aren't using @Entity(name="SomethingElse"), you
can guess what the Kind will be. Furthermore, Key<?> is not a final
class - you can add your own constructor. Nothing stops you from
making a class like this:
public class GWTKey<T> extends Key<T> {
public GWTKey(Key<?> parent, Class<T> clazz, long id) {
// Note: Class<?>.getSimpleName() is not part of GWT's JRE emulation
String kind = clazz.getName().substring(clazz.lastIndexOf('.'));
// You must manage polymorphism yourself
if (kind.equals("Mammal"))
kind = "Animal";
this.raw = KeyFactory.createKey(parent == null ? null :
parent.getRaw(), kind, id);
}
}
Notice that it will be chock-full of application-specific business
logic. Ugly, but it will do the job.
One other thing worth mentioning - I hope you are thinking about the
potential hazards of generating keys client-side. Either you must
validate the key objects carefully or you are assuming that the client
is trustworthy - because anyone could create any Key<?> object and
pass it through your RPC boundary. This could conceivably be abused
to fetch invalid information or to store corrupt information in the
database. This might not be an issue for an internal app with a
trusted user base, but a public API is subject to attack.
Jeff
Jeff Schwartz
Ugly is putting it mildly, seriously, & I wouldn't bother littering my client-side code with it. I'd rather bite the bullet and do a full scale refactoring job of the client and server to use RPC calls passing ids instead.
One other thing worth mentioning - I hope you are thinking about the
potential hazards of generating keys client-side. Either you must
validate the key objects carefully or you are assuming that the client
is trustworthy - because anyone could create any Key<?> object and
pass it through your RPC boundary. This could conceivably be abused
to fetch invalid information or to store corrupt information in the
database. This might not be an issue for an internal app with a
trusted user base, but a public API is subject to attack.
This warning is a little over the top, Jeff, at least to me and here's why...
Trusting the user means that the user is authenticated and is authorized. A clever hacker, a site member, employing a subdomain high-jacking (replacing the real session id with their own valid session id) passes both tests because as a valid user they were able to obtain a valid session id which they then use to replace the real one. In this case and numerous other types of hijacking scenarios the user will always appear to be authenticated and authorized even though they are up to no good. Even sites sitting behind firewalls are subject to these types of attacks and many have been breached including Google.
So in the end the most any of us can really do is to follow best practices by using ssl where and when we can, validate session ids and validate the data but then there are no guarantees.
Jeff Schwartz
Jeff Schnitzer
>
> Ugly is putting it mildly, seriously, & I wouldn't bother littering my
> client-side code with it. I'd rather bite the bullet and do a full scale
> refactoring job of the client and server to use RPC calls passing ids
> instead.
Sorry. I wish there was a better way, but I can't think of one.
Actually, I just did. I'll bet it is possible to create a GWT
generator that creates and hides all that application-specific
business logic inside the generated Key<?> constructor. Unfortunately
my GWT-fu is not strong enough to actually do that right now, and I
don't have time to learn. Anyone else want to take a crack at it?
> This warning is a little over the top, Jeff, at least to me and here's
> why...
>
> Trusting the user means that the user is authenticated and is authorized. A
> clever hacker, a site member, employing a subdomain high-jacking (replacing
> the real session id with their own valid session id) passes both tests
> because as a valid user they were able to obtain a valid session id which
> they then use to replace the real one. In this case and numerous other types
> of hijacking scenarios the user will always appear to be authenticated and
> authorized even though they are up to no good. Even sites sitting behind
> firewalls are subject to these types of attacks and many have been breached
> including Google.
I wasn't considering subdomain hijacking. I was just thinking about
the level of data isolation that is usually required for public-facing
applications; user data must be segregated, and written data must be
protected against malicious injection. It sounds like your
application does not face the public so you can "trust" users not to
be malicious - that's great.
Jeff
Jeff Schwartz
On Thu, Mar 3, 2011 at 11:51 AM, Jeff Schwartz <jeffts...@gmail.com> wrote:Sorry. I wish there was a better way, but I can't think of one.
>
> Ugly is putting it mildly, seriously, & I wouldn't bother littering my
> client-side code with it. I'd rather bite the bullet and do a full scale
> refactoring job of the client and server to use RPC calls passing ids
> instead.
Actually, I just did. I'll bet it is possible to create a GWT
generator that creates and hides all that application-specific
business logic inside the generated Key<?> constructor. Unfortunately
my GWT-fu is not strong enough to actually do that right now, and I
don't have time to learn. Anyone else want to take a crack at it?
> This warning is a little over the top, Jeff, at least to me and here'sI wasn't considering subdomain hijacking. I was just thinking about
> why...
>
> Trusting the user means that the user is authenticated and is authorized. A
> clever hacker, a site member, employing a subdomain high-jacking (replacing
> the real session id with their own valid session id) passes both tests
> because as a valid user they were able to obtain a valid session id which
> they then use to replace the real one. In this case and numerous other types
> of hijacking scenarios the user will always appear to be authenticated and
> authorized even though they are up to no good. Even sites sitting behind
> firewalls are subject to these types of attacks and many have been breached
> including Google.
the level of data isolation that is usually required for public-facing
applications; user data must be segregated, and written data must be
protected against malicious injection. It sounds like your
application does not face the public so you can "trust" users not to
be malicious - that's great.
My app is public facing, Jeff, but it uses Google Accounts for authentication and my own datastore entities for authorization. I use innertext on the client most of the time and when I do use innerhtml it is safe. As for GQL injection, I'm using Objectify and its parameter binding methods for filtering on values when querying. So all in all I think I have taken all the steps I should take (except for perhaps using SSL) to segregate and validate user data.
I realize that you are probably as upset about this breaking change as I am and I appreciate that. I agree, persistence rules the day and I would never stand in the way of progress so I'll refactor my code when I cut over to the new version.
Thank you for your continued commitment and support of Objectify. It is a great product.
Jeff Schwartz
Jeff
Jon Stevens
> clever hacker, a site member, employing a subdomain high-jacking (replacing
> the real session id with their own valid session id) passes both tests
> because as a valid user they were able to obtain a valid session id which
> they then use to replace the real one. In this case and numerous other types
> of hijacking scenarios the user will always appear to be authenticated and
> authorized even though they are up to no good. Even sites sitting behind
> firewalls are subject to these types of attacks and many have been breached
> including Google.
You are assuming that only authz'd people would be using a Key. There
is many usecases where that isn't necessary. Anonymous commenting
systems come to mind.
jon
Jeff Schwartz
I wasn't assuming that at all. I was merely trying to show that even authenticated and authorized users aren't safe.
Jeff Schnitzer
>
> My app is public facing, Jeff, but it uses Google Accounts for
> authentication and my own datastore entities for authorization. I use
> innertext on the client most of the time and when I do use innerhtml it is
> safe. As for GQL injection, I'm using Objectify and its parameter binding
> methods for filtering on values when querying. So all in all I think I have
> taken all the steps I should take (except for perhaps using SSL) to
> segregate and validate user data.
Sorry, I guess I didn't make my attack scenario clear. Maybe this
affects your app, maybe not, but if you're creating Key<?> objects on
the client and passing them to the server, there are some attack
scenarios like:
* Say you have an RPC method getThing(Key<?>) that lets you define
what thing to grab. An authenticated, valid malicious user could call
this method making up their own Key<?>. They could request an id for
data that is associated with another user. In a polymorphic system,
they might be able to request access to other different kinds of
things without blowing up the typechecker.
* Say you have an RPC method write(MyEntity) that puts an entity in
the datastore. Say one of the member fields is a Key<?>. A malicious
user could write a MyEntity with a Key pointing to some odd part of
the database; now, when ever your app tries to load the entity pointed
at by that Key, it might throw ClassCastExceptions because it's
getting the wrong kind of data.
All I'm saying is that if you pass keys through untrusted RPC
boundaries, you need to be careful about validating them. You still
have this issue with ids, just less so. Don't trust clients, even if
they're authenticated.
> I realize that you are probably as upset about this breaking change as I am
> and I appreciate that. I agree, persistence rules the day and I would never
> stand in the way of progress so I'll refactor my code when I cut over to the
> new version.
Another option is to stay on Objectify 2.2.3 for a while. Unless you
need async get/put/delete or polymorphism, there's nothing wrong with
the old version. Who knows, I might get around to building that GWT
generator sometime in the next year.
Jeff
Jeff Schwartz
All valid scenarios.
I think I just might hold off on upgrading for a while, at least until I actually need pollymorphims or sync.
Aswath Satrasala
However, a good migration path for the GWT Key<?> will be really desirable.
I don't like to continue on 3.0b1 for long time, as this will miss out the ofy nice api in future.
Can we have fork into Objectify plain and Ofy with Polymorphism. Just an idea.
Regards
-Aswath
Jeff Schnitzer
Google Code supported Git in the style of github so that anyone could
easily make forks and submit pull requests), I personally am not going
to maintain a fork.
Also, I wouldn't be so down on the polymorphism feature - it's
something that just eventually comes up in a schema. You don't care
about it until suddenly you need it.
Is your difficulty a question of persistent serialization of old keys,
or a question of creating Key<?>s in GWT client-side? If the later,
you can work around the issue by using the constructor that takes a
native Key. You just need to know how to construct that Key properly
- which really just means knowing what the Kind is called. Unless you
have some pretty magical code, the kind for class Thing is almost
always "Thing".
If the former - run two passes, one that converts Key<?> to native
datastore Key, then upgrade Ofy, then convert back to Key<?>. Or just
leave your data with native Keys. Or (if your keys don't have
parents) just use the simple Long/String. Serializing complex classes
to persistent storage is always just a little bit precarious for
exactly this reason. I don't expect Key<?> to change in a
serialization-incompatible way ever again, but I won't make any hard
promises.
Jeff
Brendan Doherty
Jeff Schnitzer
http://blog.jdevelop.eu/2010/01/17/use-generators-to-create-boilerplate-code-in-gwt-20/
This would be used to create the client-side source for a class that
maps Class<?> to kind based on the annotations of all entities. The
generated could would just be:
if (clazz == Entity1.class)
return "Entity1";
else ...etc
Then the Key<?> super-source could use this to properly construct a
native datastore Key and wrap it.
Jeff
Philippe Beaudoin
this.raw = KeyFactory.createKey(parent == null ? null : parent.getRaw(), kind, id);
}
}
Key<Cat> key = GWT.create(Cat.MyKey.class).create(null, id);
public class Cat_MyKey_Generated extends Cat.MyKey {
public Cat_MyKey_Generated() {
}
Brendan Doherty
Key<OnlineOrder> orderKey = new Key<OnlineOrder>(OnlineOrder.class, 123);
Key<OrderLine> lineKey = new Key<OrderLine>(orderKey, OrderLine.class, 456);
... use these lines instead.
Key<OnlineOrder> orderKey = OnlineOrderKeyFactory.createKey(123);
Key<OrderLine> lineKey = OrderLineKeyFactory(orderKey, 456);
Thomas Visser
Jeff Schnitzer
be 100% transparent to the user. That is, on both the client and the
server you would construct a Key<?> like this:
new Key<Cat>(Cat.class, id);
...but on the server you would get the normal Key<?> constructor that
does Kind lookup via annotations, on the client you would get a
generated Key<?> constructor that does Kind lookup via "if (clazz ==
Mammal.class) ... else if ...".
This is just going to take some research.
Jeff
Brendan Doherty
I wrote a gwt generator that worked on the example that phillipe previously posted, however you had to have that extra inner class, and it would only work on client code, not shared code.
Brendan Doherty
If you are saying you don't want to have to figure out the base entity that had been sub classed, you don't have to. The annotation processor automatically figures out the base kind of the subclass.
Thomas Visser
Brendan Doherty
interface HasReferences {Key<? extends HasRefences> getKey();// just an interface,}@Entitypublic class Person implements HasReferences {@Id private long id;Key<? extends HasRefences> getKey() {return PersonKeyFactory.createKey(this.id);}}@Entitypublic class Company implements HasReferences {
@Id private long id;
Key<? extends HasReferences> getKey()
}return CompanyKeyFactory.createKey(this.id);}
// somewhere in GWT:List<HasReferences> hasReferences = getHasReferences();for (HasReferences hr : hasReferences) {Key<? extends HasReferences> k = hr.getKey();}
Thomas Visser
Jeff Schnitzer
Right now, Ofy's Key.java is separated between a server version and a
super-source version. This is why there is no Key(Class<?>, long)
constructor on the client.
The client version of Key.java could look like this:
public class Key<T> implements Serializable, Comparable<Key<?>> {
...
static ObjectifyClientKeyFactory keyFact =
GWT.create(ObjectifyCliebntKeyFactory.class);
...
public Key(Class<?> clazz, long id) {
String kind = keyFact.getKind(clazz);
// do the rest of the constructor now that we have the kind
}
}
Now the only trick is how to generate an ObjectifyClientKeyFactory
that looks something like this:
class ObjectifyClientKeyFactory {
static Map<Class<?>, String> kinds = new HashMap<Class<?>, String>();
static {
kinds.put(Thing.class, "Thing");
kinds.put(Animal.class, "Animal");
kinds.put(Mammal.class, "Animal");
kinds.put(Cat.class, "Animal");
}
}
The problem is that you now need the registration information within
the generator. This is somewhat tricky but doesn't seem like an
insurmountable problem... maybe even a good time to introduce
classpath scanning.
Jeff
Brendan Doherty
Jan-Hendrik Kuperus
public <T extends HasLabels> SafeOperationResult<T> tagEntityWithLabel(Key<T> entityKey, LabelEntity label) throws AppSystemException;
The Key would be constructed client-side (we took care to rewrite that code to create and wrap native datastore Keys) and then it would be sent over the RPC layer. Some debugging revealed that the problem lies in our use of Datastore Namespaces. The problem lies in the fact that the GWT super-source of this raw Key has no notion of namespaces, but the server side Key does. So when it is passed over the RPC layer, it is deserialized with the default ("") namespace instead of the actual namespace active on the server side.
Is this something we could fix within Objectify, or is it something that should be handled at GWT?
For now I have resorted to creating more specific tag*WithLabel methods which simply take the object's ID and create the Key<?> server side. If anyone has any ideas as to how we could solve this I'd be very interested :)
--JH
Jeff Schnitzer
Jan-Hendrik Kuperus
Jeff Schnitzer
Jeff Schnitzer
Jan-Hendrik Kuperus
Aswath Satrasala
I am in the process of migrating my app to Objectify3.1.
My application uses namespaces heavily. I would like the GWT key to maintain the namespace during the RPC process.
I tried few things by modifying the Objectify 3.1.
Please find attached patch file that contains the modified files, but, no luck.
I have also created the issue in the tracker.
Please suggest
Thanks
-Aswath
www.AccountingGuru.in
Aswath Satrasala
Attached is the patch file. Any improvements or suggestions are welcome.
If there is a need for you to create Key<?> on the client, following is the sample code
NamespaceManager.set(namespace); // you need to transmit namespace also to the client earlier during module load.
Key<TaxCategory> keyTax = new Key<TaxCategory>(KeyFactory.createKey("TaxCategory", taxCategoryId));
-Aswath
www.AccountingGuru.in
Nate Thompson
On Thursday, March 3, 2011 1:51:47 AM UTC-7, Jeff Schnitzer wrote:
So... all is not well in GWT-land with the new Key<?>. In short: You
cannot create Key<?> objects client-side. I don't see any way to work
around this issue.The explanation:
In the past, a Key<?> contained the POJO entity className en lieu of
Kind. This worked because there was a 1-to-1 mapping between class
names and datastore Kinds, and Objectify could map back and forth
just-in-time.Polymorphism breaks this 1-to-1 relationship; all the classes in a
polymorphic hierarchy have the same Kind. Thus the new Key<?>
actually wraps a datastore native Key... a much simpler and more
elegant solution. But now think about this:Key<Car> key = new Key<Car>(Car.class, 123);
...this means that the Key<?> constructor needs to know how to
construct a datastore Key from Car.class. Thus it needs to know the
Kind of a Car... which might be Vehicle... and the only way to know
that is by examining the annotations of Car.class.But GWT-land does not have annotations. Class.getAnnotation() is not
part of the JRE emulation.Basically, the polymorphism feature makes it impossible to construct
Key<?> in GWT code. I have tried to think of ways around the problem,
but I'm starting to believe it's intractable. The original strategy
of putting the classname in the Key<?> won't work because there's no
way to go backwards - picture a keys-only query that must return
Key<?> objects, but doesn't have discriminator values from a fetched
Entity to determine what the correct POJO class is. We can't
regenerate the classname.Soooo... what I'm going to do is provide super-source for Key<?> that
makes the constructors that take a Class<?> throw an
UnsupportedOperationException when called from GWT. You can't
construct a Key<?> in GWT, although you can pass them back and forth
through GWT-RPC. I considered removing the constructors entirely and
forcing Key<?> creation in a KeyFactory (similar to the low-level API)
but that feels like too much API change. Besides, I like "new
Key<Car>(Car.class, 123)" better than
"KeyFactory.<Car>create(Car.class, 123)".Unless someone has a brilliant insight, this looks like a situation
where we have to make a decision "is Objectify a persistance framework
or a GWT framework?" and the answer must be that persistence comes
first.Jeff