Placing GTPL pages and/or groovlets in a jar file

[Scott Murphy]

I am wondering if it is possible to put gtpl pages in a jar file?
I am learning (painfully) that having more files in your web app
degrades your start up time.
It has been recommended to me by the app engine team to make a fat jar
with all my classes and dependencies.

I know you can put resources in a jar file... e.g. mapreduce has
jquery.js in the their jar file
com/google/appengine/tools/mapreduce/jquery-1.4.2.min.js

[Vladimír Oraný]

wait for the end of the week ;) you'll be kindly surprised in upcoming release :)

[Scott Murphy]

wow, news like that makes it feel like christmas is coming a lot
sooner this year
:)

[John]

We've been concerned by the amount of time it takes for a scripts to start executing and have seen some cases where it looked like the script was already compiled, but it still takes a second or more to start executing. We think some time may be taken by calls such as file.exists() and file.canRead() that appear to require a remote call on appengine. We'd like to try packaging the groovlet/gtpl scripts into a jar file to see if there is less delay in script execution.

We have made some progress with subclassing GaelykServlet to configure the GroovyScriptEngine such that it (generally) won't try to access the script file once it has loaded it. The GaelykTemplateServlet is not as easy to extend, since groovy.servlet.TemplateServlet makes its TemplateCacheEntry a private class and its getTemplate method only communicates an inaccessible script resource by throwing ServletException.

Please let me know when there's a gaelyk jar that we can try to see if loading scripts from a jar reduces the delays we observe.

Thanks!
John

[Scott Murphy]

Are you saying that that file.exists() and file.canRead() are called
every
time? even if the script was already compiled?

If so, this should probably be a bug reported with Groovy and be fixed
asap.

Have you looked at this?
https://jira.codehaus.org/browse/GROOVY-4975

[Scott Murphy]

Actually, now that I think about it. It would have to do those
checks to see if the source was newer. :(

Seems like it needs some sort of added configuration to not
perform those checks when in production.

On Mar 3, 11:04 am, Scott Murphy <sc...@pixoto.com> wrote:
> Are you saying that that file.exists() and file.canRead() are called
> every
> time? even if the script was already compiled?
>
> If so, this should probably be a bug reported with Groovy and be fixed
> asap.
>

> Have you looked at this?https://jira.codehaus.org/browse/GROOVY-4975

[John]

I have a simple extension of the GaelykServlet that avoid some of the checks in GroovyScriptEngine. The TemplateServlet is much harder to manipulate from the outside.

class GaelykServletWrapper extends groovyx.gaelyk.GaelykServlet {
   
    @Override
    protected GroovyScriptEngine createGroovyScriptEngine() {
        def result = super.createGroovyScriptEngine()
        if (!Constants.developmentEnvironment) {
            result.config.minimumRecompilationInterval = Integer.MAX_VALUE
        }
        return result

[Scott Murphy]

Hey John,

Did you get noticeable speed improvement after using this? or is it
just speculation at this point?
Prior to doing it, could you consistently see the speed degradation?
Does it happen multiple times per resource or once per instances per
resource?

The reason why I ask is because when using app stats, I can sometimes
notice the following:
a 1.5 second delay before a groovlet starts.
a 1.5 second delay before and/or after a plugin
but these delays are not consistent and I can't replicate them all the
time.

Best,

Scott

[John]

[John]

The symptom we observed is similar: ~1 sec delay before the groovlet script starts executing. We initially believed it was script compilation, but after we saw a groovlet run a second time with a similar delay in a single appengine instance, we started to look for another explanation.

Unfortunately, we don't have a way to verify the behavior with certainty. (We'd need to add logging to the GroovyScriptEngine and I'm not sure that I'll have time to set up the Groovy build.) We'll be doing some more analysis, so maybe we'll have some better-supported conclusions next week.

The app we're looking at doesn't make use of Gaelyk plugins.

On Sunday, March 4, 2012 7:46:20 PM UTC-8, Scott Murphy wrote:

[Scott Murphy]

So I created a test example and it seems to me most of the delays
are happening only on the first load of that specific page on a
new instance. We see delays more frequently on production, but we
also
get a lot of new instances spawning frequently.

I have yet to see first hand a delay on a subsequent page load unless
we are talking less than 40 ms.

I did not see delays around a plugin call, I was mistaken in stating
that earlier.

My test example consisted of a test groovlet, a test plugin, a test
gtpl
page, and a test gtpl include. I did a memcache put call before and
after each transition. I then used app stats to look at the delays
in between calls.

[John]

Thanks for the new information, Scott!

Perhaps we need to focus more on good tools for extracting accurate timing information from the downloaded logs. Have you found anything that works well?

We may be suffering from having too many script files. I'm beginning to think that performance might be improved enough that it makes sense to have fewer files with more code since the delay doesn't seem to be much affected by the length of the script.