How to use soy (google closure) templates on google app engine

[Paul Koerbitz]

Dear Clojurians,

let me briefly introduce myself: I am a phd student from Germany, have
fallen in love with clojure and am toying around with it and google
app engine in my free time.

Now to my question: I am trying to create an google app engine app
using appengine-magic (https://github.com/gcv/appengine-magic) and
google closure templates using clj-soy (https://github.com/levand/clj-
soy). I can't figure out which path I need to pass to the 'build'
function in clj-soy. In the interactive development environment I can
use absolute or relative paths such as

(ns my-project.templates
(:require [clj-soy.templates :as soy]))

(def *templates* (soy/build "war/WEB-INF/templates/template.soy"))
(def *alternative-templates* (soy/build "/full/path/to/project/
directory/war/WEB-INF/templates/template.soy"))

but this won't work with the dev-server or on google app engine.

I read that one way to provide access to resource files would be to
use appengine-magic.core/open-resource-stream but this returns an
java.io.BufferedReader and as far as I understand I can't use that
with clj-soy / closure templates.

Many thanks for creating all these great tools, it is really fun to
work with them.

Paul

[Constantine Vetoshev]

On Wed, Jul 27, 2011 at 1:06 PM, Paul Koerbitz <paul.k...@gmail.com> wrote:
> I read that one way to provide access to resource files would be to
> use appengine-magic.core/open-resource-stream but this returns an
> java.io.BufferedReader and as far as I understand I can't use that
> with clj-soy / closure templates.

I'm not familiar with clj-soy. If it can use templates defined as
strings, then you can consume the BufferedReader returned by
appengine-magic.core/open-resource-stream. appengine-magic already
includes a dependency on Apache Commons IO, so
org.apache.commons.io.IOUtils/toString should work.

[Mark Rathwell]

A few things to try if you want to put in the effort (it doesn't look like clj-soy is done, or maybe just the documentation is not up to date).  

The docs and examples show using clj-soy.template/build, however, that function does not exist.  It looks like clj-soy/from-files may be what replaced that function.  Unfortunately, you do need a File object, or path to a file, for this function (actually used by clj-soy.file/add-file).  What you would need to do is duplicate that logic, but do it for a URL object (or string url path).  

See the docs for the java library that clj-soy is wrapping [1], and note that SoyFileSet$Builder.add() has an overload that takes a URL.  So, I think you could put the templates in your war directory, so they are accessible as static files, and use that url.   

[1] http://closure-templates.googlecode.com/svn/trunk/javadoc-lite/index.html

Hope this helps.

 - Mark

 

On Wed, Jul 27, 2011 at 1:06 PM, Paul Koerbitz <paul.k...@gmail.com> wrote:

[Tarantoga]

Please, have a look at Clojure-WebApp (https://github.com/dbushenko/
Clojure-WebApp)
and a sample web application (https://github.com/dbushenko/
webexample1). The framework is based on the clj-soy.
In a few days I'll commit another web application and will deploy it
on heroku.com, because Clojure-WebApp supports this awsome hosting.

[Paul Koerbitz]

Hello Constantine and Mark,

thank you very much for your suggestions. It does seem indeed that clj-
soy is not finished so maybe extending it would be the best way. I was
also thinking of using 'proxy' to maskerade the BufferedReader as a
File but it does seem overkill for such a simple problem.

I think I'll try to pass the templates in as strings or urls (in that
order). I was sort of hoping that I could just specify a relative path
but I guess that won't work...

thank you
Paul

[Paul Koerbitz]

Tarantoga,

I did take a look at your blog and got some inspiration of that (like
using clj-soy for example ;). The problem with the path is appengine
specific, so Im not sure how much your new framework will help with
this specific problem but I'll gladly take a look and learn.

On Jul 28, 3:27 pm, Tarantoga <d.bushe...@gmail.com> wrote:

[Daniel Renfer]

I've been using https://github.com/r0man/closure-templates-clj and it
has worked pretty well for me so far. I'm still converting all of my
Hiccup templates over to Soy.

One issue I've found so far is that, since I'm using Maven, I have to
re-process the resources and then re-eval the corresponding namespace
whenever I want to make changes to the templates.

Examples at: https://github.com/duck1123/jiksnu/tree/master/src/main/clojure/jiksnu/templates
and https://github.com/duck1123/jiksnu/tree/master/src/main/resources/soy/jiksnu/templates

[Mark Rathwell]

Looks like closure-templates-clj would work fine with a URL or a File.  For clj-soy, looking at it again, I think you would only need to add another condition in clj-soy.file/add-file:

(instance? URL file) (.add builder file)

and also import java.net.URL

[Paul Koerbitz]

Mark,

yes I was thinking something along these lines. Btw the version of clj-
soy that is on clojars appears to be in the 'basic-functionality'
branch, well I am sure you have seen that too.

On question about the URL approach: Would exposing the templates by
putting them in a publicly accessible static directory not be
considered a security risk (in the sense that you should only share
info that you absolutely need to share)? Granted it wouldn't matter
much for the low-profile application that I have in mind, so it's more
a question of general principle.

I haven't had time to look at this so far, I'll let you know what I
get when I do.

cheers
Paul

On Jul 29, 3:40 am, Mark Rathwell <mark.rathw...@gmail.com> wrote:
> Looks like closure-templates-clj would work fine with a URL or a File.  For
> clj-soy, looking at it again, I think you would only need to add another
> condition in clj-soy.file/add-file:
>
> (instance? URL file) (.add builder file)
>
> and also import java.net.URL
>
>
>
>
>
>
>

> On Thu, Jul 28, 2011 at 7:47 PM, Daniel Renfer <d...@kronkltd.net> wrote:
> > I've been usinghttps://github.com/r0man/closure-templates-cljand it

> > has worked pretty well for me so far. I'm still converting all of my
> > Hiccup templates over to Soy.
>
> > One issue I've found so far is that, since I'm using Maven, I have to
> > re-process the resources and then re-eval the corresponding namespace
> > whenever I want to make changes to the templates.
>
> > Examples at:

> >https://github.com/duck1123/jiksnu/tree/master/src/main/clojure/jiksn...
> > and
> >https://github.com/duck1123/jiksnu/tree/master/src/main/resources/soy...
>
> > On Thu, Jul 28, 2011 at 5:50 PM, Paul Koerbitz <paul.koerb...@gmail.com>

[Paul Koerbitz]

Daniel,

closure-templates-clj looks interesting, from a quick look I am not
sure how the paths are resolved.

From deftemplate's docstring

"Define a render fn for a Soy template. The macro expects that the
Soy templates are defined in files matching the hierarchy of the
Clojure namespace in the Soy *directory* on the classpath.

The render fn \"hello-name\" defined in the namespace
\"closure.templates.test.core\" expects a Soy template with the name
\".helloName\" in the \"soy/closure/templates/test/core.soy\" file on
the classpath.

I gather that the .soy files would need to be available on the
classpath at compile time, so that might work with GAE. I'll give it a
shot when I get time.

Right now I am not really sure further input would be helpful (for me)
until I can toy around with these things again, but I very much
appreciate your help and input ;)

cheers
Paul

On Jul 29, 1:47 am, Daniel Renfer <d...@kronkltd.net> wrote:
> I've been usinghttps://github.com/r0man/closure-templates-cljand it

> has worked pretty well for me so far. I'm still converting all of my
> Hiccup templates over to Soy.
>
> One issue I've found so far is that, since I'm using Maven, I have to
> re-process the resources and then re-eval the corresponding namespace
> whenever I want to make changes to the templates.
>

> Examples at:https://github.com/duck1123/jiksnu/tree/master/src/main/clojure/jiksn...
> andhttps://github.com/duck1123/jiksnu/tree/master/src/main/resources/soy...

[Mark Rathwell]

Paul and Constantine,

Yes, it is a security risk, but at this point it was more of a 'make it work...then make it right' type of thing.  I'm going to be in the same boat (of needing this functionality) very soon, so I'm interested in getting this working.

What do you think about this:

In appengine-magic.core add a function to get a resource as a URL:

(defn resource-url [resource-name]

  (-> (clojure.lang.RT/baseLoader) (.getResource resource-name)))

And then modify clj-soy as previously discussed to accept a URL.

 - Mark

[Constantine Vetoshev]

On Fri, Jul 29, 2011 at 11:56 AM, Mark Rathwell <mark.r...@gmail.com> wrote:
> What do you think about this:
> In appengine-magic.core add a function to get a resource as a URL:
> (defn resource-url [resource-name]
>   (-> (clojure.lang.RT/baseLoader) (.getResource resource-name)))

If this works in both the interactive REPL and in production App
Engine, then I'll gladly accept a patch.

[Mark Rathwell]

Great, I'll test this weekend and send a pull request.

Thanks.

[Constantine Vetoshev]

I merged your patch. resource-url is now available in appengine-magic
0.4.3 (which also includes support for App Engine 1.5.2). Thanks for
contributing!

[James Reeves]

On 29 July 2011 16:56, Mark Rathwell <mark.r...@gmail.com> wrote:
> In appengine-magic.core add a function to get a resource as a URL:
> (defn resource-url [resource-name]
>   (-> (clojure.lang.RT/baseLoader) (.getResource resource-name)))

Is there something wrong with using the standard
clojure.java.io/resource function?

- James

[Constantine Vetoshev]

Unfortunately, yes.

1. It's implemented in terms of Thread/currentThread, which I'm not
sure works in production App Engine.

2. In the past, just loading clojure.java.io broke because it uses
classes not on the App Engine classloader whitelist. I'm pretty sure
this hasn't changed.

[Paul Koerbitz]

Mark,

I ended up using closure-templates-clj and it works smoothly on GAE,
thanks for the suggestion. I have thus not looked further into using
clj-soy.

thanks all
Paul

On 29 Jul., 11:56, Mark Rathwell <mark.rathw...@gmail.com> wrote:
> Paul and Constantine,
>
> Yes, it is a security risk, but at this point it was more of a 'make it
> work...then make it right' type of thing.  I'm going to be in the same boat
> (of needing this functionality) very soon, so I'm interested in getting this
> working.
>
> What do you think about this:
>
> In appengine-magic.core add a function to get a resource as a URL:
>
> (defn resource-url [resource-name]
>   (-> (clojure.lang.RT/baseLoader) (.getResource resource-name)))
>
> And then modify clj-soy as previously discussed to accept a URL.
>
>  - Mark
>

[Mark Rathwell]

Hi Paul,

You have tested using closure-template-clj's deftemplate macro on actual GAE with success?  This macro uses clojure.java.io/resource (:only), so it would be interesting to know if it is safe to use.

 - Mark

[Paul Koerbitz]

Hi Mark,

yes I uploaded my 'app' to GAE and it runs, no errors AFAICT, didn't
investigate it any further so not sure if this constitues 'safe to
run', but it works for now.

hth
Paul

On 31 Jul., 15:47, Mark Rathwell <mark.rathw...@gmail.com> wrote:
> Hi Paul,
>
> You have tested using closure-template-clj's deftemplate macro on actual GAE
> with success?  This macro uses clojure.java.io/resource (:only), so it would
> be interesting to know if it is safe to use.
>
>  - Mark
>