Strange error siena.SienaException: java.lang.IllegalAccessException: Class siena.Util can not access a member of class models.Journal with modifiers "private".

[Raphael André Bauer]

Hi,

just recently I am getting strange errors on GAE. Not on my local play
installation - only on GAE:

SienaException occured : siena.SienaException:
java.lang.IllegalAccessException: Class siena.Util can not access a
member of class models.Journal with modifiers "private".

My Class looks like:

public class Journal extends EnhancedModel {

@Id(Generator.AUTO_INCREMENT)
private Long id;

public void setId(Long id) {
this.id = id;
}

public Long getId() {
return id;
}

And the code that emits the error is:
Journal journal = Journal.all().filter("id", id).get();

I am almost sure this worked before - but somehow it stopped working
at GAE. I am using play siena 2.0.6

If I change private Long id; to protected Long id; it seems to work...

Does anybody have an idea?

Thanks,

Raphael

--
inc: http://ars-machina.raphaelbauer.com
tech: http://ars-codia.raphaelbauer.com
web: http://raphaelbauer.com

[Pascal Voitot Dev]

What Play do you use?
Do you have the full exception stack to see exactly where it crashes?

Pascal

--
You received this message because you are subscribed to the Google Groups "Siena" group.
To post to this group, send email to siena-...@googlegroups.com.
To unsubscribe from this group, send email to siena-discus...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/siena-discuss?hl=en.

[Raphael André Bauer]

On Mon, Dec 12, 2011 at 2:14 PM, Pascal Voitot Dev
<pascal.v...@gmail.com> wrote:
> What Play do you use?

Hmm. 1.2.4...

> Do you have the full exception stack to see exactly where it crashes?

Jupp:
(Thanks :) )...

Execution exception (In /app/controllers/Crawler.java around line 77)

SienaException occured : siena.SienaException:
java.lang.IllegalAccessException: Class siena.Util can not access a
member of class models.Journal with modifiers "private"

play.exceptions.JavaExecutionException: siena.SienaException:

java.lang.IllegalAccessException: Class siena.Util can not access a
member of class models.Journal with modifiers "private"

at play.mvc.ActionInvoker.invoke(ActionInvoker.java:231)
at play.server.ServletWrapper$ServletInvocation.execute(ServletWrapper.java:557)
at play.Invoker$Invocation.run(Invoker.java:278)
at play.server.ServletWrapper$ServletInvocation.run(ServletWrapper.java:548)
at play.Invoker.invokeInThread(Invoker.java:68)
at play.server.ServletWrapper.service(ServletWrapper.java:142)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java:141)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:102)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:249)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:422)
at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:449)
at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455)
at com.google.tracing.TraceContext.runInContext(TraceContext.java:695)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325)
at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453)
at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
at java.lang.Thread.run(Thread.java:679)
Caused by: siena.SienaException: siena.SienaException:

java.lang.IllegalAccessException: Class siena.Util can not access a
member of class models.Journal with modifiers "private"

at siena.gae.GaeMappingUtils.fillModel(GaeMappingUtils.java:518)
at siena.gae.GaeMappingUtils.mapEntities(GaeMappingUtils.java:696)
at siena.gae.GaePersistenceManager.map(GaePersistenceManager.java:1380)
at siena.gae.GaePersistenceManager.doFetchList(GaePersistenceManager.java:1522)
at siena.gae.GaePersistenceManager.fetch(GaePersistenceManager.java:1961)
at siena.AbstractPersistenceManager.get(AbstractPersistenceManager.java:71)
at siena.BaseQuery.get(BaseQuery.java:111)
at play.modules.siena.QueryWrapper.get(QueryWrapper.java:88)
at controllers.Crawler.crawlJournal(Crawler.java:77)
at play.mvc.ActionInvoker.invokeWithContinuation(ActionInvoker.java:548)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:502)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:478)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:473)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:161)
at play.server.ServletWrapper$ServletInvocation.execute(ServletWrapper.java:557)
at play.Invoker$Invocation.run(Invoker.java:278)
at play.server.ServletWrapper$ServletInvocation.run(ServletWrapper.java:548)
at play.Invoker.invokeInThread(Invoker.java:68)
at play.server.ServletWrapper.service(ServletWrapper.java:142)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java:141)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:449)
at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455)
at com.google.tracing.TraceContext.runInContext(TraceContext.java:695)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333)
at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325)
at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453)
... 1 more
Caused by: siena.SienaException: java.lang.IllegalAccessException:

Class siena.Util can not access a member of class models.Journal with
modifiers "private"

at siena.Util.setField(Util.java:286)
at siena.Util.setFromObject(Util.java:294)
at siena.gae.GaeMappingUtils.setFromObject(GaeMappingUtils.java:612)
at siena.gae.GaeMappingUtils.fillModel(GaeMappingUtils.java:515)
... 43 more
Caused by: java.lang.IllegalAccessException: Class siena.Util can not

access a member of class models.Journal with modifiers "private"

at siena.Util.setField(Util.java:284)
... 46 more

[Emanuele Ziglioli]

I thought all persistent field should be public, including the id.
Anyway, I've been experiencing the same error on GAE's production
server and unfortunately haven't been able to reproduce it locally.
It was a very similar case with one model extending another model.
I suspect that perhaps at a certain point I changed the schema so for
some instances there was a mismatch between persisted fields and the
model, it could be additional or missing fields or even different
types.
The error would happen with a query of type asList().add().
I haven't looked whether Siena's unit tests try persisting an entity
with one model and load it with a different one, it would be an
interesting test case.

E.

[Raphael André Bauer]

Hmm...

I am getting a similar failure on another model as well (stacktrace attached).
The strange thing is: if I retry the request it works without any error...

Regarding private / public => the field can be private - right?

Any ideas would be much appreciated...

Best,

Raphael

SienaException occured : java.lang.IllegalAccessException: Class
siena.Util can not access a member of class models.JournalArticle with
modifiers "private"

play.exceptions.JavaExecutionException:

java.lang.IllegalAccessException: Class siena.Util can not access a

member of class models.JournalArticle with modifiers "private"

Caused by: siena.SienaException: java.lang.IllegalAccessException:

Class siena.Util can not access a member of class

models.JournalArticle with modifiers "private"
at siena.Util.setField(Util.java:286)
at siena.gae.GaeMappingUtils.setIdFromKey(GaeMappingUtils.java:158)
at siena.gae.GaeMappingUtils.mapEntitiesKeysOnly(GaeMappingUtils.java:639)
at siena.gae.GaePersistenceManager.mapKeysOnly(GaePersistenceManager.java:1405)
at siena.gae.GaePersistenceManager.doFetchList(GaePersistenceManager.java:1506)
at siena.gae.GaePersistenceManager.fetchKeys(GaePersistenceManager.java:2012)
at siena.BaseQuery.fetchKeys(BaseQuery.java:146)
at play.modules.siena.QueryWrapper.fetchKeys(QueryWrapper.java:126)
at models.utils.search.SearchJanitor.searchItemReturnPublicationsOfDiscreteTimestampNOTCACHED(SearchJanitor.java:344)
at models.utils.search.SearchJanitor.searchItemReturnPublicationsOfDiscreteTimestampCACHED(SearchJanitor.java:289)
at models.utils.search.SearchJanitor.searchItemsLatestPublicationsBasedOnDay(SearchJanitor.java:56)
at models.JournalArticle.searchKeyOnly(JournalArticle.java:254)
at controllers.api.ApiReader.getUserSavedSearches(ApiReader.java:267)

Caused by: java.lang.IllegalAccessException: Class siena.Util can not
access a member of class models.JournalArticle with modifiers
"private"
at siena.Util.setField(Util.java:284)
... 47 more

>
> E.

[Emanuele Ziglioli]

Hi Raphael!

The tricky bit about this error is that it's so difficult to track
down.
In our case it would occur just on the production server on certain
data sets.
If we manage to reproduce it with a unit test it would be solved
already.

What's the query you're trying to do? In our case it was a call to
asList().
asList() would load all the entities so I've stopped using it if I
don't want to load the list but just add an entry for example.

I think the Siena message has nothing to do with the error. It's just
catching another exception, perhaps caused by using reflection.
It could be that the model has changes so some old data has fields
with a different type.
Adding some debug to the Siena code (print the stack trace of the
exception that's beeb caught) could help.

[Emanuele Ziglioli]

Hi again,

I'm seeing this exception again. As Raphael pointed out, it occurs on
a call to setAccessible() in the Util class, setField(), on a private
field:

if(!f.isAccessible()){
f.setAccessible(true);
wasAccess = false;
}

Strangely enough I've had problems in the past with Jaxb on App
Engine, causing exception, but I've never connected the dots. Look at
these pages:

1. here, in order to make jaxb work, one has to comment out calls to
setAccessible()
http://code.google.com/p/googleappengine/issues/detail?id=1267

2. here, someone has measured the impact of setAccessible on GAE:
http://jira.codehaus.org/browse/JRUBY-4246

3. (also on methods) http://jeremyblythe.blogspot.co.nz/2009/11/using-simple-xml-instead-of-jaxb-on.html

So, then I've found:

http://code.google.com/appengine/docs/java/runtime.html
"An application cannot reflect against any other classes not belonging
to itself, and it can not use the setAccessible() method to circumvent
these restrictions."

Now,
1. what's the boundary of an application? is an external jar like
Siena considered 'another' application?
2. why don't we always see exception?

As a workaround, we could just comment out the call to setAccessible()
and mandate that all persistent fields have to be public.

Your thoughts?

[Emanuele Ziglioli]

I've found this thread from Pascal himself where he had to fight
security exceptions on GAE: https://groups.google.com/group/play-framework/browse_thread/thread/68c026c7f0a84778


On Feb 3, 4:12 pm, Emanuele Ziglioli <theb...@emanueleziglioli.it>
wrote:
> Hi again,

[Pascal Voitot Dev]

I added this because some people asked me to be able to have private persisted fields :)
You know that Java has polluted everyone with those dumb accessors which are just a wrong design of the language in my opinion.

So seeing your mail, it appears that setAccessible is just unpredictable in GAE, does it? Weird fact!
Would it be possible to ask directly on GAE list about it?

What are the solution?
Comment them and make all persisted fields public which will bother a few people?
Add an option to comment it but I don't really like that because performance is already bad when using introspection.

No good solution apparently :D

Pascal

[Emanuele Ziglioli]

I think there are problems with GAE's security manager.
I haven't found any message specific with regards to setAccessible()
but googling for it reveals a lot of stack traces with exceptions
generated by that.
I'm not exactly sure why setAccessible() sometimes fails or succeeds.
Subsequent calls seem to behave differently.
It's all hidden in the cloud.... Google hasn't released the source
code since version 1.6.0 (I could use JAD of course...) but,
regardless, I'm never entirely sure that what I run on the development
server is the same that runs on the cloud.

I wanted to try an alternative version of setField(). My version would
try to call Field.set() without checking for isAccessible() and call
setAccessible() in case of an exception:
Unfortunately I haven't been able to run the tests on Siena and due to
my lack of experience with Maven it's taking me a while to figure out
why.
Here's the code:

try {
f.set(object, value);
} catch (Exception e) {
try {

if(!f.isAccessible()){
f.setAccessible(true);
wasAccess = false;
}

} catch (Exception e2) {
throw new SienaException(e2);
} finally {
if(!wasAccess)
f.setAccessible(false);
}
}

[Pascal Voitot Dev]

which alternative version of setField are you talking about?

                       }
               }

[Emanuele Ziglioli]

just that code that I posted, i.e.:

try {
f.set(object, value);
} catch (Exception e) {
try {
if(!f.isAccessible()){
f.setAccessible(true);
wasAccess = false;
}
} catch (Exception e2) {
throw new SienaException(e2);
} finally {
if(!wasAccess)
f.setAccessible(false);
}
}

I'm actually trying it with our codebase and am not noticing
exceptions anymore.
In fact, I don't even bother resetting the access level, now I do:

public static void setField(Object object, Field f, Object value) {
// boolean wasAccess = true;

try {
f.set(object, value);
} catch (Exception e) {
try {
if(!f.isAccessible()){
f.setAccessible(true);

// wasAccess = false;

}
} catch (Exception e2) {
throw new SienaException(e2);
} finally {

// if(!wasAccess)
// f.setAccessible(false);
}
}
}

[Raphael André Bauer]

Hi Emanuele,

that's interesting :)

Best,

Raphael

> --
> You received this message because you are subscribed to the Google Groups "Siena" group.
> To post to this group, send email to siena-...@googlegroups.com.
> To unsubscribe from this group, send email to siena-discus...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/siena-discuss?hl=en.
>

--

[Pascal Voitot Dev]

We could put this in current trunk if you want... I don't see anything against it right now!

Pascal

[Emanuele Ziglioli]

On Feb 7, 10:21 pm, Pascal Voitot Dev <pascal.voitot....@gmail.com>
wrote:

> We could put this in current trunk if you want... I don't see anything
> against it right now!

I'll send you a pull request. I wish I could fix my problems with
Maven and run the unit tests!
My latest version is even simpler:

public static void setField(Object object, Field f, Object
value) {

try {
f.set(object, value);
} catch (Exception e) {
try {

// I don't need to call
f.isAccessible(), apparently that's an expensive call
f.setAccessible(true);
f.set(object, value); // I forgot
this line in my previous code!!

[Emanuele Ziglioli]

Ok, the exception is caught now but what I'm seeing is weird.
Basically it looks like some setField() calls fail even on public
members!!

It looks like in some cases isAccessible() (which I believe is called
internally by setField() ) returns false on public members but
setAccessible() succeeds and then I can set the field!!
It looks like a bug somewhere in their implementation of GAE's
security manager. I should be able to write unit tests to show that

On Feb 9, 5:20 am, Emanuele Ziglioli <theb...@emanueleziglioli.it>
wrote:

[Emanuele Ziglioli]

Here's what looks like an expanation of what's happening:

===================
http://freemarker.blogspot.co.nz/2010/02/freemarker-on-google-app-engine.html#c7806981190571108253

GAE will replace java.lang.reflect.* classes on the fly with its own
doppelgangers defined in
"com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect."
This causes problems when, say, a class tries to reflectively access a
private member of itself, or a package-private member of another class
in its package. This works in a compliant JRE as java.lang.reflect.*
classes aren't subject to security checks, but it fails with GAE as
their doppelgangers won't have the necessary permissions. See this
message for an example:

http://www.mail-archive.com/google-app...@googlegroups.com/msg00865.html

Here, a class named "freemarker.ext.beans.BeansWrapper" tries to
reflectively create an instance of "freemarker.ext.beans.EnumModels"
using a package-private constructor. In a compliant JRE, this works
flawlessly, since they're both in the same package. Under GAE, it
won't work because of intervening doppelganger Constructor instance.
I'm not sure what GAE is trying to achieve by intercepting reflection
(that couldn't be achieved by a carefully tuned security policy file
instead).
=================

What I'm seeing is that, using reflection, some members are not
accessible even if they're public.
So, a call to Field.set() throws an exception.
Then I catch the exception and I manage to call
Field.setAccessible(true). Finally Field.set() succeeds.
So I thought the exception I was seeing was being thrown by
setAccessible() instead is thrown by setField().

Now, I'm going to write a unit test

[Pascal Voitot Dev]

it's really weird and seems to be a real limitation of GAE :D
Crazy stuff!
If you can find a solution, I'll be glad to commit it asap ;)
Pascal

[Emanuele Ziglioli]

Hi Pascal,

I've sent you a pull request on github. Could you please run the tests
to verify that nothing is broken?
I can't run the tests because there are some contain links to your
local hard drive :-)

I've spent some time writing a unit test. Unfortunately I can't
reproduce the bug. What I've noticed is that isAccessible() is always
false. You can verify it by running GaeTest and putting a breakpoint
in setField().

First I've created a standalone unit test. Then I've created a GAE
project with a simple Servlet. In both cases I haven't been able to
reproduce the exception.
I wonder whether it's because we store lists of fields.

Here's the unit test (it doesn't use Siena):

public class ReflectionTest
{
static public class Model {
public long id = 1234;
}

@Test
public void setFieldTest() throws SecurityException,
NoSuchFieldException, IllegalArgumentException,
IllegalAccessException
{
Model model = new Model();

Field f = Model.class.getField("id");

assertTrue( f.isAccessible() );
f.set( model, 9999 );

assertEquals( 9999, model.id );
}
}

Here's the Servlet (it uses the original Util.setField()):

public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {

resp.setContentType("text/plain");

try {
if( model == null ){
model = new Model();
}
Field f = Model.class.getField("id");
resp.getWriter().println(" model.id -> " + model.id );
resp.getWriter().println(" f.isAccessible() -> " +
f.isAccessible() );
//f.set( model, 9999 );
setField( model, f, 9999 );

resp.getWriter().println(" f.set() -> " + "ok" );
resp.getWriter().println(" model.id -> " + model.id );

resp.getWriter().println(" f.isAccessible() -> " +
f.isAccessible() );

} catch (Exception e) {
resp.getWriter().println( e.getCause() );
}


On Feb 10, 9:44 am, Pascal Voitot Dev <pascal.voitot....@gmail.com>
wrote:

[Pascal Voitot Dev]

Hi,
I'll pull your code and run the full tests this weekend!
If the bug can't be reproduced, it's quite hard to know if it's corrected but with it, some people will be able to test in GAE at least ;)

Pascal

[Emanuele Ziglioli]

Interesting explanation here: http://groups.google.com/group/google-appengine-java/msg/128f21a00016ba03

====
The behavior you are seeing is actually not a bug in App Engine but
rather
an unexpected behavior in Java itself. If you try the experiment in
Java
code running outside of App Engine you will see the same behavior.
The source code for java.lang.reflect.AccessibleObject explains the
situation:
// Indicates whether language-level access checks are
overridden // by this object. Initializes to "false". This field is
used by // Field, Method, and Constructor. boolean override;
public boolean isAccessible() {
return override; }
The name "isAccessible" is misleading. The method actually returns
whether accessibility has been overridden to true.
===

[Pascal Voitot Dev]

and finally should we add your cludge to the code? without it, you get problems, isn't it?

Pascal

[Emanuele Ziglioli]

sure, if all tests pass, I haven't seen that exception occurring since
running my patch

On Feb 22, 8:25 pm, Pascal Voitot Dev <pascal.voitot....@gmail.com>
wrote:

> and finally should we add your cludge to the code? without it, you get
> problems, isn't it?
>
> Pascal
>
> On Wed, Feb 22, 2012 at 12:42 AM, Emanuele Ziglioli <
>
>
>
> theb...@emanueleziglioli.it> wrote:
> > Interesting explanation here:

> >http://groups.google.com/group/google-appengine-java/msg/128f21a00016...