Google App Engine (GAE) Support

[Scott Hernandez]

Is there any support on GAE now that java is supported? If so, are
there caveats or concerns while running under GAE? If not, why; is
there another DI framework in play?

Thanks in advance,
Scott

[Dhanji R. Prasanna]

It should work, have you noticed any problems?

[Scott Hernandez]

Yeah, there seem to be some problems. I was mostly asking about the
status of integration. But I gave it a run and tried to get it working
myself from the last 2.0 beta release.

java.security.AccessControlException: access denied
(java.lang.RuntimePermission getClassLoader)
at java.security.AccessControlContext.checkPermission(Unknown Source)
at java.security.AccessController.checkPermission(Unknown Source)
at java.lang.SecurityManager.checkPermission(Unknown Source)
at java.lang.ClassLoader.getSystemClassLoader(Unknown Source)
at com.google.inject.internal.BytecodeGen.getClassLoader
(BytecodeGen.java:123)
at com.google.inject.internal.BytecodeGen.getClassLoader
(BytecodeGen.java:116)
at com.google.inject.internal.BytecodeGen.newFastClass
(BytecodeGen.java:143)
at com.google.inject.DefaultConstructionProxyFactory$2.<init>
(DefaultConstructionProxyFactory.java:73)
at com.google.inject.DefaultConstructionProxyFactory.get
(DefaultConstructionProxyFactory.java:71)
at com.google.inject.ProxyFactory.createConstructionProxy
(ProxyFactory.java:85)
at com.google.inject.ProxyFactory$1.apply(ProxyFactory.java:62)
at com.google.inject.ProxyFactory$1.apply(ProxyFactory.java:60)
...

It is strange, but I think someone might have got to this already in
trunk. I will try building from there, and fixing or reporting the
problems as they come :)

On Apr 8, 8:03 pm, "Dhanji R. Prasanna" <dha...@gmail.com> wrote:
> It should work, have you noticed any problems?
>
> On Thu, Apr 9, 2009 at 12:39 PM, Scott Hernandez

> <scotthernan...@gmail.com>wrote:

[Stuart McCulloch]

2009/4/9 Scott Hernandez <scotthe...@gmail.com>

yes the Google App Engine has a strict security manager - it also doesn't allow you to create threads

btw, this exception should be fixed in trunk (http://code.google.com/p/google-guice/issues/detail?id=336)
 

On Apr 8, 8:03 pm, "Dhanji R. Prasanna" <dha...@gmail.com> wrote:
> It should work, have you noticed any problems?
>
> On Thu, Apr 9, 2009 at 12:39 PM, Scott Hernandez

> <scotthernan...@gmail.com>wrote:

>
>
>
> > Is there any support on GAE now that java is supported? If so, are
> > there caveats or concerns while running under GAE? If not, why; is
> > there another DI framework in play?
>
> > Thanks in advance,
> > Scott

--
Cheers, Stuart

[Scott Hernandez]

Which is the next error I get if I work against trunk. Of course, this
all works in the dev-env cause it doesn't have the same security
policy!

java.security.AccessControlException: access denied
(java.lang.RuntimePermission modifyThreadGroup)

at java.security.AccessControlContext.checkPermission(Unknown Source)
at java.security.AccessController.checkPermission(Unknown Source)
at java.lang.SecurityManager.checkPermission(Unknown Source)

at com.google.apphosting.utils.security.SecurityManagerInstaller
$CustomSecurityManager.checkAccess(SecurityManagerInstaller.java:72)
at java.lang.ThreadGroup.checkAccess(Unknown Source)
at java.lang.Thread.init(Unknown Source)
at java.lang.Thread.<init>(Unknown Source)
at com.google.inject.internal.Finalizer.<init>(Finalizer.java:92)
at com.google.inject.internal.Finalizer.startFinalizer(Finalizer.java:
81)
...

I'm taking a look at the static Finalizer code now. But considering I
don't have any experience with this, I'm not sure how much of a hack I
can make...

On Apr 9, 12:09 am, Stuart McCulloch <mccu...@gmail.com> wrote:
> 2009/4/9 Scott Hernandez <scotthernan...@gmail.com>
> [snip]

[Scott Hernandez]

Once I added a try/catch SecurityException to Finalizer.java
everything now runs on GAE.

I'm not sure what the consequences are of removing the Finalizer
thread, but let's call that google's problem!

Finalizer.java:81
try{
Finalizer finalizer = new Finalizer(finalizableReferenceClass,
frq);
finalizer.start();
return finalizer.queue;
}catch (SecurityException se){
//GAE -- security blows!
}
return null;

[Dhanji R. Prasanna]

Sweet! Thanks for reporting this.

[Brian Pontarelli]

>
> yes the Google App Engine has a strict security manager - it also
> doesn't allow you to create threads

Ouch. Sorta hinders applications that burst/multiplex backend services
or that use schedulers. Wonder if they'll provide a thread pool at
some point.

-bp

[Stuart McCulloch]

2009/4/10 Brian Pontarelli <br...@pontarelli.com>

yep - though there is a basic "cron" scheduler service:

   http://code.google.com/appengine/docs/java/config/cron.html

also, requests have a time limit of 30s - after that they get zapped

   http://code.google.com/appengine/docs/java/runtime.html

--
Cheers, Stuart

[Brian Pontarelli]

On Apr 9, 2009, at 10:21 AM, Stuart McCulloch wrote:

2009/4/10 Brian Pontarelli <br...@pontarelli.com>

> yes the Google App Engine has a strict security manager - it also
> doesn't allow you to create threads

Ouch. Sorta hinders applications that burst/multiplex backend services
or that use schedulers. Wonder if they'll provide a thread pool at
some point.

yep - though there is a basic "cron" scheduler service:

   http://code.google.com/appengine/docs/java/config/cron.html

Nice.

also, requests have a time limit of 30s - after that they get zapped

   http://code.google.com/appengine/docs/java/runtime.html

Even more reason to burst requests to backend service and do things asynchronously inside the application rather than letting the GAE truncate the connection. Adding a task execution service would solve it. You could pin the task to the server that is handling the request or span it, depending on how the tasks are implemented and if they can be distributed. You could also define well known tasks in the appengine XML file and this would allow distribution:

<task>

  <class>com.example.BurstSomeBackendTask</class>

  <!-- Additional task info -->

</task>

public class BurstSomeBackendTask implements Task {

  public void execute(TaskRequest request, TaskResponse response) {

    // do the task here

  }

}

// Client

Map<String, String> data = ...;

TaskService ts = TaskServiceFactory.createTaskService();

TaskResponse response = ts.executeAndWait("com.example.BurstSomeBackendTask", data, 10);

TaskFuture tf = ts.execute("com.example.BurstSomeBackendTask", data);

The data could also be a protobuffer or some other type of transport. 

Plus, if anything wants to do NIO work, it will need to have selector thread(s).

There are many reasons to use threads, so it seems that locking them down would dictate that the GAE would need to provide alternative solutions.

-bp

[Eduardo Nunes]

yes, but the main problem about the cron, is that it executes a URL,
so you have only 30 seconds to do something, at least if they
implemented the same restriction to "cron requests". Does anyone know
something about it?

[Stuart McCulloch]

2009/4/10 Eduardo Nunes <esn...@gmail.com>

yes, but the main problem about the cron, is that it executes a URL,
so you have only 30 seconds to do something, at least if they
implemented the same restriction to "cron requests". Does anyone know
something about it?

yes the same 30s timeout is applied to cron jobs, and you can have a maximum of 20 jobs

--
Cheers, Stuart

[Scott Hernandez]

Seems like you can just setup a re-entrant task where when it gets
close to 30 seconds (or a good place to pause) it calls out to another
app (by url) that immediately calls back (to a place you can even
specify on the request) with all the info needed to start back up and
continue processing. Then you just have your task ping this restart/
resume service, and magically you get a request with where to start
back up. Keep doing this as long as you need to do work.

Using cron to start these processes seems straight forward enough, and
they can go one as long as you want by simpling ping'n yourself
(essentially) continously.

On Apr 9, 11:09 am, Stuart McCulloch <mccu...@gmail.com> wrote:
> 2009/4/10 Eduardo Nunes <esnu...@gmail.com>

[Stuart McCulloch]

2009/4/10 Scott Hernandez <scotthe...@gmail.com>

Seems like you can just setup a re-entrant task where when it gets
close to 30 seconds (or a good place to pause) it calls out to another
app (by url) that immediately calls back (to a place you can even
specify on the request) with all the info needed to start back up and
continue processing. Then you just have your task ping this restart/
resume service, and magically you get a request with where to start
back up. Keep doing this as long as you need to do work.

Using cron to start these processes seems straight forward enough, and
they can go one as long as you want by simpling ping'n yourself
(essentially) continously.

interesting idea... I wonder how easy it would be to create a continuations framework on top of the GAE scheduler?

--
Cheers, Stuart

[Brian Pontarelli]

On Apr 10, 2009, at 1:27 AM, Stuart McCulloch wrote:

2009/4/10 Scott Hernandez <scotthe...@gmail.com>

Seems like you can just setup a re-entrant task where when it gets
close to 30 seconds (or a good place to pause) it calls out to another
app (by url) that immediately calls back (to a place you can even
specify on the request) with all the info needed to start back up and
continue processing. Then you just have your task ping this restart/
resume service, and magically you get a request with where to start
back up. Keep doing this as long as you need to do work.

Using cron to start these processes seems straight forward enough, and
they can go one as long as you want by simpling ping'n yourself
(essentially) continously.

interesting idea... I wonder how easy it would be to create a continuations framework on top of the GAE scheduler?

Holy hack-tastic batman ;)

-bp

[Yogurt Earl]

Has anyone tried guice-servlet (http://code.google.com/p/google-guice/
wiki/Servlets) or warp-servlet (www.wideplay.com/warp::servlet )?

[Stuart McCulloch]

2009/4/14 Yogurt Earl <Yogur...@gmail.com>

Has anyone tried guice-servlet (http://code.google.com/p/google-guice/
wiki/Servlets) or warp-servlet (www.wideplay.com/warp::servlet )?

yes, I've tried a custom build of guice + guice-servlet and it appears to work ok
( I patched guice with the latest google-collections code which supports GAE )

my specific patches are available here:

   http://peaberry.googlecode.com/svn/tags/1.1/lib/build/UPDATE_COLLECTIONS_20090412.txt
   http://peaberry.googlecode.com/svn/tags/1.1/lib/build/GUICE_ISSUE_343_SRC_20090304.txt

the resulting Guice jar (r929+patches) is available in the latest release of peaberry:

   http://peaberry.googlecode.com/files/peaberry-1.1.zip

this zipfile doesn't contain guice-servlet - but you can either grab that from a recent
snapshot of Guice, or you could build it yourself from trunk (note the above patches
don't affect guice-servlet, only the core Guice code)

HTH - happy hacking ;)

--
Cheers, Stuart