RPC policy files usage?
Ed
noserver mode, and I am interested if this might be improved.
Let me explain my use case and show my problems that I experience.
I have 4 GWT app's connecting all to the same backend app. All GWT
app's have two backend interfaces to the backend, and they also have a
dev and test app each. This will produce a total of 24 gwt.rpc files.
Every time that gwt complaints during dev mode about not finding a
gwt.rpc file I copy it manualy to the server such that it's packages
in the war and restart the server. This is costy and takes a lot of
time during development. At this moment I have about 50 gwt.rpc files
in my backend as I don't know which one is old and it takes too long
to find that out every time. The gwt.rpc files contain a lot of
duplicates entries also.
I think that I am doing something wrong as I don't think this is the
correct way.
I will start using the new DeRPC mechanism shortly and understand that
the rpc files aren't needed anymore during dev mode, however because I
am running in noserver mode I don't think this gives me any
improvement.
However, I still looking for a good way to use these files during
testing/production. Please some advice on this?
I think that I should collect all created gwt.rpc files (at least of
the test and production app's) and include them in my war, or not?..
However, the problem I am having with this solution is that I am
building all with maven and means that I need to make the web project
dependent of the gwt app's, such that they are build first before the
war is build. This dependency isn't welcome in my project, unclear and
error phrone.
Can others please share how they solved this?
I then still many policy files that contain duplicated entries. I
would be nice if these could be merged some how.
But let's go back to the goal of the policy files like I understand
it:
Indicates which files are allowed to be serialized.
At this moment this is done through creating policy files that contain
the classes that are allowed to be serialized. Like explained above,
this creates a undesired situation.
This results in creating policy files that have to be pushed from the
front-end to the backend.
But what about calculating the policy in the backend and not create a
dependency between the backend and front-end?
For example: I could return my own implementation of the
SerializationPolicy class that indicates which class can be serialized
depending on the package name....
However, will this work and is this good enough?
I think it is in my case as I use some other security mechanism to
gaurantee a certain security level, but I can't find good information
about the content of the policy files, so it's difficult for me to
make the correct design decisions.
Please some feedback?
Ladislav Gazo
maybe this will give you some relevant answers:
http://code.google.com/p/acris/wiki/SeparateClientAndServer ,
especially the part "Accessing RPC files from server". There is a
solution how to access RPC files in no-server mode remotely also.
ed bras
Thanks for your response.
I do almost the same thing as Acris does, I only have my files located in the classpath (I also use GWT-SL to read my policy files)..
But Acris also doesn't solve the problem of having all these policy files scattered around and assumes that you have them somewhere available in a local or remote context. However, they don't explain how to get them there, which is exactly my problem and not very friendly to my experience.... (they also refer to using ant copy actions to collect them).
Ladislav Gazo
context configuration you need to copy RPC files to the server - and
that is what you apparently don't want. But in remote context these
RPC files are generated on the client (by ProxyCreator), they remain
there and server is able to fetch them using HTTP connection to
appropriate client.
ed bras
> to fetch them using HTTP connection to appropriate client.
But, if you think about it, it will probably be the best option if you want to deal with policy files. The other little disadvantage would be the HTTP communication whereas the files are located on the same server.
But, that doesn't take away the disadvantage of having all these duplicate entries in the policy files.
What about implementing my own SerializationPolicy class and simple filter on, for example the package name, to indicate if an object is allowed to be serialized/deserialized.?
Ladislav Gazo
look into RemoteContextSerializationPolicy you can see how the HTTP
connection is initiated. You don't have to take care about where are
RPCs located because the call itself will tell you.
The drawback of "remote context" is that there is HTTP connection but
you have always an option to switch to "local context" in production.
There will be no overhead then.
Regarding duplicate entries in RPC files... my advice is if your RPC
files are generated correctly, your calls are working correctly,
"don't touch it" :) Resolving serialization problems are one of the
trickiest.... and if there are duplicate entries it is because the
service/permutation apparently needs it there. You may reorganize your
services to lower the number of same objects across them.
Ed
option:
- I run in the following environments: dev, test, acceptance,
production and build.
The dev and build environments are the ones that give me problems as I
run GWTTestCase's in these mode's with a proxy servlet to forward the
call to the running tomcat instance (-noserver mode) (server is
started/stopped through the maven cargo plugin).
The gwt tests run in gwt modules that aren't reachable through HTTP,
so the above Remote context isn't an option.
Sometimes my tests fail during dev or our nightly build because the
polixy file is outdated and need to be replaced... This isn't a
desired situation: the build being dependent on policy file not being
up2date :(
I hope to get some more details through the forum to implement my own
SerializationPolicy. I think that is still my best workable option.
At this moment, these policy files cost me a too much time, and I
tried several options now through the years..
Alex Moffat
unfortunately my situation is somewhat simpler so I don't think what
I've done is directly applicable. I'm using GWT 2.0 so some of this
may does not apply if you are using earlier versions.
I have an simpler situation than you do because I build the server
application after the GWT code is compiled so I have access to the RPC
files and they get placed in the war, or at least into the app server
class path at that time. Thinking about it though I may have a
solution.
The SerializationPolicy is returned from the doGetSerializationPolicy
method in RemoteServiceServlet. The default implementation of this
uses getResourceAsStream to read the file from the web server
classpath. To handle running GWT test cases with no server (and also
to provide additional logging for doUnexpectedFailure) I use a
subclass of RemoteServiceServlet instead of using RemoteServiceServlet
directly. In this subclass I check for a system property that
indicates whether or not I'm running a test case, this is set by the
scripts that execute the test cases. If it is set then instead of the
default behavior I open a URL connection to read the serialization
data from the server running the test case instead of trying to find
it in the classpath of the server running the servlet.
I think a similar approach could be used in your situation. It doesn't
matter where the data is, provided the doGetSerializationPolicy method
can find it. Perhaps you could override that method and load the data
from a well known location in the filesystem if the files are not in
the app server classpath? It may require copying the files around but
at least wouldn't need an app server restart. A fancier alternative
would be to have a simple server that provided access to the
serialization files via http so that the doGetSerializationPolicy
method could use a URL connection to read the data.
Anyway, those are my thoughts at the moment. Perhaps they will spur
others to suggest better alternatives.
Ed
The solution is a bit like discussed above with Ladislav.
I am still not so happy about retrieving this rpc files from different
locations in different environments and also having to configure a
HTTP context in the build/dev environment to be able to retrieve the
policy files....
I still think this should be more simple for "just" dealing with a
"few" policy files.
I think my best option, to make handling the policy files easy, is to
implement my own SerializationPolicy class. But, I have no idea if
that is correct way to go as I can't find correct information about
it :(.
Any idea's on how to do this are more then welcome?
Ed
mmoossen
what we do is at compilation time, with ant, to move the policy files
to the package of the server side implementation of the service, and
to use following loader:
/**
* We do not want that the server goes to fetch files from the
servlet context, so we keep the .gwt.prc files in the class
* path together with the service implementation.<p>
*
* @see
com.google.gwt.user.server.rpc.RemoteServiceServlet#doGetSerializationPolicy(javax.servlet.http.HttpServletRequest,
java.lang.String, java.lang.String)
*/
@Override
protected SerializationPolicy doGetSerializationPolicy(
HttpServletRequest request,
String moduleBaseURL,
String strongName) {
// locate the serialization policy file in the class path
String serializationPolicyFilePath =
getClass().getPackage().getName().replace('.', '/') + "/";
serializationPolicyFilePath +=
SerializationPolicyLoader.getSerializationPolicyFileName(strongName);
SerializationPolicy serializationPolicy = null;
// Open the RPC resource file and read its contents.
InputStream is =
getClass().getClassLoader().getResourceAsStream(serializationPolicyFilePath);
try {
if (is != null) {
try {
serializationPolicy =
SerializationPolicyLoader.loadFromStream(is, null);
} catch (ParseException e) {
this.log("ERROR: Failed to parse the policy file
'" + serializationPolicyFilePath + "'", e);
} catch (IOException e) {
this.log("ERROR: Could not read the policy file '"
+ serializationPolicyFilePath + "'", e);
}
} else {
String message = "ERROR: The serialization policy file
'"
+ serializationPolicyFilePath
+ "' was not found; did you forget to include it
in this deployment?";
this.log(message);
}
} finally {
if (is != null) {
try {
is.close();
} catch (IOException e) {
// Ignore this error
}
}
}
return serializationPolicy;
}
HTH
Michael
Ladislav Gazo
if there is no other option overriding it is your best bet. I think
that the policy should automatically copy RPC files to desired server
location in case of gwt test cases (and automated test env.). It can
read the location from e.g. VM parameter.... For production you can
use local or remote context principles.
ed bras
I am a bit lost here.
I mean, I don't really understand why I even need these policy files and why they aren't more flexible in usage (maybe even optional).
If I am correct, it's there for security reasons, it's used to indicate which objects are allowed to be (de)serialized.
But what if I don't want that, or if I want to do it in another way?
If I look at my gwt app's, security is well taken care of, so I don't think I need these policy files. However, at this moment it's very hard to not-use them :( and it cost developers (a lot of time) to use them in a more complex enterprise-like environment.
Ed
Ed
This issue was driving me nuts.. It costs me so much time to update
these policy files (re-deploy/restart server). I even ended up with
around 100 policy files whereas I only need about 10 (for different
gwt app's against the same backend), but I didn't dare to clean them
as I didn't know which could be removed :(..
Anyway, I implemented my own SerializationPolicy that I like to share
here...
The idea: I copied much of the code from the
StandardSerializationPolicy class.
My implementation is listed below. It's basically the same as the
standard policy file, but instead the serialization and
deserialization whitelists aren't Map's but some general Class that
implements HasContains<Class<?>>.
So the next thing to do is: implement you own policy-rule file that
implement HasContains<Class<?>> and inject it, that's it.
I have two implementation of this HasContains: one that contains an
allowed and not allowed list of classes. And one that, instead of
classes, contains a list of allowed and not allowed Class names. This
latter version allows to use "open end names", by defining a name as
for example: "com.bla.*" that matches any name that start with
"com.bla.". This makes it very friendly to define allowed/not allowed
patterns. You could even add your own implementation to allow more
complex pattern's.
I have tested it and use it online now, and so far I am happy with the
result and safes me quite a bit of time...
The code:
-----
public final class SerializationPolicySimple<P extends
HasContains<Class<?>>> extends SerializationPolicy {
private final P deserializationWhitelist;
private final P serializationWhitelist;
public SerializationPolicySimple(P serializationWhitelist, P
deserializationWhitelist) {
Args.notNull(serializationWhitelist, deserializationWhitelist);
this.deserializationWhitelist = deserializationWhitelist;
this.serializationWhitelist = serializationWhitelist;
}
@Override
public boolean shouldDeserializeFields(Class<?> clazz) {
return isFieldSerializable(clazz, serializationWhitelist);
}
@Override
public boolean shouldSerializeFields(Class<?> clazz) {
return isFieldSerializable(clazz, deserializationWhitelist);
}
@Override
public void validateDeserialize(Class<?> clazz) throws
SerializationException {
Args.exprNotTrueMsg(!isInstantiable(clazz,
deserializationWhitelist), "ClassNotDeserializable", clazz);
}
@Override
public void validateSerialize(Class<?> clazz) throws
SerializationException {
Args.exprNotTrueMsg(!isInstantiable(clazz, serializationWhitelist),
"ClassNotSerializable", clazz);
}
/**
* Field serializable types are primitives and types on the specified
* whitelist.
*/
private boolean isFieldSerializable(Class<?> clazz, P whitelist) {
if (clazz.isPrimitive()) {
return true;
}
else {
return whitelist.contains(clazz);
}
}
/**
* Instantiable types are primitives and types on the specified
whitelist
* which can be instantiated.
*/
private boolean isInstantiable(Class<?> clazz, P whitelist) {
if (clazz.isPrimitive()) {
return true;
}
else {
return whitelist.contains(clazz);
}
}
}
-----
And:
----
public final class ContainsAllowedString implements HasContains<Class<?
>> {
private boolean allowHasPreferenceOverNotAllow;
private final Collection<String> allowed;
private final Collection<String> notAllowed;
/**
* All allowed.
*/
public ContainsAllowedString() {
this(null, null);
}
/**
* The allowed and not allowed lists can both be null, such that all
is allowed.
* All collection items can have an open end through the end
character '*' to match only the beginning of the name for a match.
* For example: entry "com.bla.*" will match any values starting with
"com.bla." like "com.bla.sit".
*
* @param allowed can contain items with an open end (*) to match
only the beginning of a name.
* @param notAllowed can contain items with an open end.
* @see UtilsMiscSafe#isAllowedString(String, Collection, Collection,
boolean)
*/
public ContainsAllowedString(Collection<String> allowed,
Collection<String> notAllowed) {
this.allowed = allowed;
this.notAllowed = notAllowed;
}
public final boolean isAllowHasPreferenceOverNotAllow() {
return allowHasPreferenceOverNotAllow;
}
public final void setAllowHasPreferenceOverNotAllow(boolean yes) {
this.allowHasPreferenceOverNotAllow = yes;
}
public boolean contains(final Class<?> item) {
Args.notNull(item);
return UtilsMiscSafe.isAllowedString(item.getName(), allowed,
notAllowed, allowHasPreferenceOverNotAllow);
}
}
-----
And the method isAllowedString:
----
/**
* In case both the allowed and not allowed collections are empty or
null, all item's are allowed.<br>
* This version is almost the same as {@link #isAllowed(Object,
Collection, Collection, boolean)}, but it can only be used with
String
* as input and can deal with "*" at the end of a string to match
only the beginning of the string.<br>
* Example: suppose allowed contains "com.ited.*" and the item has
value "com.ited.bla.bla". The item will be matched as an item
contained
* in the allowed items. The same counts for the not allowed
collection.
*
* @return true if allowed, false otherwise.
*/
public static boolean isAllowedString(final String item, final
Collection<String> allowed, final Collection<String> notAllowed,
final boolean allowHasPreferenceOverNotAllow) {
Args.notNull(item);
if (isEmpty(allowed) && isEmpty(notAllowed)) { // all allowed
return true;
}
else {
if (hasContent(allowed)) {
final boolean isAllowed = contains(item, allowed);
if (isAllowed && !allowHasPreferenceOverNotAllow &&
hasContent(notAllowed)) { // not allowed has preference
return !contains(item, notAllowed);
}
else { // not allowed is not of interest
return isAllowed;
}
}
else { // only not allowed has content
return !contains(item, notAllowed);
}
}
}
/**
* @return true if the value is contained in the collection, allowing
only the first part to match if the collection item ends with "*".
*/
private static boolean contains(String value, Collection<String> col)
{
for (String item : col) {
if (item.endsWith("*")) {
if (value.startsWith(item.substring(0, item.length() - 1))) { //
found a match
return true;
}
}
else {
if (item.equals(value)) { // found a match
return true;
}
}
}
return false;
}
----
I inject it through Spring, so my spring config will look something
like this:
---------
<!--
============================================================================
The GWT
policy
=============================================================================
-->
<bean id="gwtPolicy"
class="com.ited.gwt.server.SerializationPolicySimple">
<constructor-arg index="0" ref="gwtPolicyAllowed" />
<constructor-arg index="1" ref="gwtPolicyAllowed" />
</bean>
<!--
============================================================================
The objects that are allowed to be serialized/deserialized by
GWT
=============================================================================
-->
<bean id="gwtPolicyAllowed"
class="com.ited.lang.misc.ContainsAllowedString">
<constructor-arg index="0" >
<list>
<value>com.ited.*</value>
<value>com.bv.*</value>
<value>java.lang.*</value>
<value>[Ljava.lang.String;</value>
<value>java.util.*</value>
<value>java.sql.*</value>
</list>
</constructor-arg>
<constructor-arg index="1" >
<list>
<value>java.lang.Object</value>
</list>
</constructor-arg>
</bean>
----------
I might have to optimize bits and pieces in the future but it's good
for now...
Ed