GWT-RPC has a feature that allows shared serializable classes to have additional fields on the server than at the time GWT compilation has happened. This can happen with JPA / JDO which enhance classes on server side.
These additional fields+values are serialized on server side using normal Java serialization, then converted to Base64 and finally will be send between server and client as opaque value. When the server receives an instance of a serializable class that is marked as enhanced (the serialization policy file will have some additional information for these classes so the server knows which classes are enhanced) it will read that opaque base64 string, convert it back to binary and deserialize it using normal Java serialization in order to be able to fill the additional fields only present on server side.
At this step the vulnerability occurs since the server deserializes data that comes from an untrusted source, the browser, without checking if the binary data is actually the data it has send to the client. An attacker can change that data to something totally different which can lead to denial of service or remote code execution.
Deserializing data from untrusted sources is a general issue not just in Java but lately it got some hype because someone found a way for remote code execution if a server has Apache commons-collections library on class path. As this library is used in a lot of servers and applications chances are good that you can execute any code on these servers.
So if you use that feature of GWT-RPC and you have commons-collections on class path on server side then anyone can execute code on your server by simply replaying a modified GWT-RPC request. But even without commons-collections on class path an attacker could cause an endless loop during deserialization which can lead to denial of service (when all server threads are busy).
Further reading:
The proper solution is that GWT needs to sign the serialized data and verify the data before deserialization.
-- J.