Serving static content in a war file

[Allen Johnson]

Hey everyone.

I'm using ring, moustache and lein-ring and am able to serve static
files while running locally (lein ring server) but not when deployed
as a war file. The static files are present in the war file but
ring.middleware.file/wrap-file doesn't seem to be war-file-friendly.

Is there a way to let requests that do not match any of the moustache
routes to fall-through to the web container? Or, anyone know of
middleware that supports this?

Thanks,
Allen

[Michael Ossareh]

ring provides a means to do this:

(defn resource-response

  "Returns a Ring response to serve a packaged resource, or nil if the

  resource does not exist.

  Options:

    :root - take the resource relative to this root"

  [path & [opts]]

  (let [path (str (:root opts "") "/" path)

        path (.replace path "//" "/")

        path (.replaceAll path "^/" "")]

    (if-let [resource (.. Thread currentThread getContextClassLoader (getResourceAsStream path))]

      (response resource))))

from response.clj

 

Thanks,
Allen

[James Reeves]

On 9 March 2011 18:20, Michael Ossareh <oss...@gmail.com> wrote:
> ring provides a means to do this:
> (defn resource-response

Unfortunately most Java servlet containers don't place war resources
on the classpath by default. As far as I can make out, only the
WEB-INF/classes and WEB-INF/lib folders in the war file are on the
classpath.

Currently lein-ring places the :resources-path in the root of the war,
but I'm considering changing this so that it places resources in
WEB-INF/classes by default. This seems to be a common "hack" to get
around the classpath restrictions of servlet containers, but I'm not
entirely sure whether this is the best idea.

Compojure gets around this by getting the resource from the
ServletContext if it detects it on the response, and getting the
resource from the classpath otherwise. But I'm not sure whether this
is a better solution than placing resources in WEB-INF/classes.

A third solution would be to use the standard "lein jar" function, and
to place the resulting jar in WEB-INF/classes. Perhaps using the
ring-java-servlet package to provide the actual servlet class.

I'm uncertain what the best solution is. I'd welcome advice from
anyone more familiar than I am with war-files.

- James

[Michael Ossareh]

On Wed, Mar 9, 2011 at 11:15, James Reeves <jre...@weavejester.com> wrote:

On 9 March 2011 18:20, Michael Ossareh <oss...@gmail.com> wrote:
> ring provides a means to do this:
> (defn resource-response

Unfortunately most Java servlet containers don't place war resources
on the classpath by default. As far as I can make out, only the
WEB-INF/classes and WEB-INF/lib folders in the war file are on the
classpath.

Oooooh - that explains a lot of what I've been battling with recently!

In one case I had a file with some data in it that I wanted to read doing my application startup - I've since moved that data into the database.

In my other case I moved all my enlive templates into directories next to the clj code that calls them. Inadvertently adding them to the classpath.

 

Currently lein-ring places the :resources-path in the root of the war,
but I'm considering changing this so that it places resources in
WEB-INF/classes by default. This seems to be a common "hack" to get
around the classpath restrictions of servlet containers, but I'm not
entirely sure whether this is the best idea.

Compojure gets around this by getting the resource from the
ServletContext if it detects it on the response, and getting the
resource from the classpath otherwise. But I'm not sure whether this
is a better solution than placing resources in WEB-INF/classes.

It certainly works very nicely - when I put a css file into resources/public/css and it works that is a good "it just works" feeling. My other cases were non public files - and having to move them around because they were hard to access felt pretty annoying.

Whichever way it goes having some general purpose functions for finding resources would really be very helpful.

 

A third solution would be to use the standard "lein jar" function, and
to place the resulting jar in WEB-INF/classes. Perhaps using the
ring-java-servlet package to provide the actual servlet class.

This could be done for just the resources right? 

i.e. lein ring war actually creates a myproject.war and a myproject-resources.war. This way the resources, if they do not change that much (images, css, etc), do not have to be deployed each time. Of course, this also means you could just update your resources without deploying your code-containing jar/war - though I believe in the case of tomcat it requires a restart to see changes to a jar in the lib folder?

 

I'm uncertain what the best solution is. I'd welcome advice from
anyone more familiar than I am with war-files.

I'll poke around in my network, see what people think here.

 

- James

[Allen Johnson]

> ring provides a means to do this:
> (defn resource-response
>   "Returns a Ring response to serve a packaged resource, or nil if the
>   resource does not exist.
>   Options:
>     :root - take the resource relative to this root"
>   [path & [opts]]
>   (let [path (str (:root opts "") "/" path)
>         path (.replace path "//" "/")
>         path (.replaceAll path "^/" "")]
>     (if-let [resource (.. Thread currentThread getContextClassLoader
> (getResourceAsStream path))]
>       (response resource))))
> from response.clj

@Michael - sorry for the direct reply. I meant to reply to the group.

I'm unclear on how I'd use this to accomplish what I want. If I have
an app with many static resources (javascript, css, html, etc) that
are bundled in the war file -- would this be an appropriate way to do
it?

;; pseudo code example

(defn static-get [request path]
 (resource-response path))

(def handler
 (app
   ["foo"] {:get foo-get}
   ["bar"] {:get bar-get}
   [[_ path] #"^(.*)"] {:get (delegate static-get path)}))

As long as I can avoid having to update routes for each static
resource that is bundled with the war file I'll be happy.

Thanks,
Allen

[Allen Johnson]

> I'm uncertain what the best solution is. I'd welcome advice from
> anyone more familiar than I am with war-files.

How about letting the servlet container handle those items? Since
servlet containers have been sending content in this manner since the
start couldn't we let all non-matching routes drop down to the
container for handling?

Allen

[James Reeves]

You can set up a separate servlet for handling static routes, but you
need a different url-pattern, e.g. "/static/*". I don't think it's
possible for servlets to cascade in the same way that routes do in
Compojure.

- James

[Saul Hazledine]

On Mar 9, 8:55 pm, James Reeves <jree...@weavejester.com> wrote:

> On 9 March 2011 19:46, Allen Johnson <akjohnso...@gmail.com> wrote:
>
> > How about letting the servlet container handle those items? Since
> > servlet containers have been sending content in this manner since the
> > start couldn't we let all non-matching routes drop down to the
> > container for handling?
>
> You can set up a separate servlet for handling static routes, but you
> need a different url-pattern, e.g. "/static/*". I don't think it's
> possible for servlets to cascade in the same way that routes do in
> Compojure.
>

With Jetty I put my static content into / of the WAR file and it gets
served. I think behaviour might depend on the servlet container though
and with some containers you may need to specifically enable it. Its
standard though for all content outside WEB-INF to be visible and
servable (I just double checked in O'Reilly Java Servlet Programming).

The problem with this approach is that there is a different mechanism
serving static content during development (Ring) compared to after
deployment (the servlet container). One idea to reduce mistakes is to
have a default directory where static content goes during development
(e.g src/html) and an appropriate Ring handler that only serves from
src/html.

Saul

[James Reeves]

On 9 March 2011 21:07, Saul Hazledine <sha...@gmail.com> wrote:
> With Jetty I put my static content into / of the WAR file and it gets
> served. I think behaviour might depend on the servlet container though
> and with some containers you may need to specifically enable it. Its
> standard though for all content outside WEB-INF to be visible and
> servable (I just double checked in O'Reilly Java Servlet Programming).

I guess it must serve the files in the root of the war first, and then
pass control onto the servlet.

> The problem with this approach is that there is a different mechanism
> serving static content during development (Ring) compared to after
> deployment (the servlet container). One idea to reduce mistakes is to
> have a default directory where static content goes during development
> (e.g src/html) and an appropriate Ring handler that only serves from
> src/html.

I'd rather not keep static resources in the source-path...

I'm leaning toward the idea of defaulting to serving resources from
the Ring handler for production, but with the option to override this
behaviour. I'd prefer Ring handlers have as little to do with the
underlying servlet architecture as possible.

- James

[Allen Johnson]

> I'd rather not keep static resources in the source-path...
>
> I'm leaning toward the idea of defaulting to serving resources from
> the Ring handler for production, but with the option to override this
> behaviour. I'd prefer Ring handlers have as little to do with the
> underlying servlet architecture as possible.

How about if lein-ring supported static resources using a
javax.servlet.Filter? This would keep the servlet-api out of any ring
handlers.

;; Example filter

(gen-class
:name myapp.servlet.static_filter
:implements [javax.servlet.Filter]
:state state
:init constructor)

(defn -constructor []
[[] (ref {})])

(defn -init [this config]
(dosync (alter (.state this) :servlet-context (.getServletContext config))))

(defn -destroy [this])

(defn -doFilter [this request response chain]
(let [ctx (:servlet-context @(.state this))]
(if-let [res (.getResourceAsStream ctx (.getPathInfo request))]
(update-response res)
(.doFilter chain request response)))) ; fall-through to ring servlet

This would attempt to lookup the static resource starting in the root
of the war file. If it exists, send it as the response. Otherwise,
fall-through to normal servlet routing. lein-ring would just need to
update it's make-web-xml function to optionally include support for
defining filters.

<filter>
<filter-name>static-filter</filter-name>
<filter-class>myapp.servlet.static_filter</filter-class>
</filter>

<filter-mapping>
<filter-name>static-filter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

Allen

[James Reeves]

On 10 March 2011 03:33, Allen Johnson <akjoh...@gmail.com> wrote:
> How about if lein-ring supported static resources using a
> javax.servlet.Filter? This would keep the servlet-api out of any ring
> handlers.

That assumes all you want to do with resources is to serve them as
static routes. You might want to use them as templates, for example.

- James

[Saul Hazledine]

From James's comment, that may not be suitable for lein-ring. However,
it's a very elegant idea.
For future web applications I'll use this method rather than having
the dynamic content in a subpath (I currently use /app).

Saul

[Saul Hazledine]

On Mar 10, 10:59 am, James Reeves <jree...@weavejester.com> wrote:

> On 10 March 2011 03:33, Allen Johnson <akjohnso...@gmail.com> wrote:
>
> > How about if lein-ring supported static resources using a
> > javax.servlet.Filter? This would keep the servlet-api out of any ring
> > handlers.
>
> That assumes all you want to do with resources is to serve them as
> static routes. You might want to use them as templates, for example.
>

I think there may be a bit of miscommunication here because I (and
possibly Allen) are assuming static HTML and CSS files that don't
require any processing. Am I right in thinking that you'd like all
types of static files as resources?

I can see the reason for this in terms of simplicity and flexibility
(there is just one mechanism for dealing with all resources and its
the same during development and deployment). However, servlet
resources are a royal pain which is why the hack you mentioned earlier
where everything goes into WEB-INF/classes is the usual solution. In
fact, to support dynamically loading clojure source, leiningen-war
even ended up putting the contents of the src directory into WEB-INF/
classes. I never found a better solution but it would be really cool
if you find a better way to do this.

Saul

[Mark Nutter]

On Thu, Mar 10, 2011 at 5:46 AM, Saul Hazledine <sha...@gmail.com> wrote:
> I think there may be a bit of miscommunication here because I (and
> possibly Allen) are assuming static HTML and CSS files that don't
> require any processing. Am I right in thinking that you'd like all
> types of static files as resources?
>
> I can see the reason for this in terms of simplicity and flexibility
> (there is just one mechanism for dealing with all resources and its
> the same during development and deployment). However, servlet
> resources are a royal pain which is why the hack you mentioned earlier
> where everything goes into WEB-INF/classes is the usual solution. In
> fact, to support dynamically loading clojure source, leiningen-war
> even ended up putting the contents of the src directory into WEB-INF/
> classes. I never found a better solution but it would be really cool
> if you find a better way to do this.

What about segregating them according to usage? Use the filter
approach to serve CSS etc that do not require processing, and put
templates or other programmatic resources into WEB-INF/classes (or
some other suitable location).

[Allen Johnson]

> What about segregating them according to usage? Use the filter
> approach to serve CSS etc that do not require processing, and put
> templates or other programmatic resources into WEB-INF/classes (or
> some other suitable location).

+1

My main focus was on static web resources that do not require
additional processing. I agree with Mark that programmatic resources
should be in WEB-INF/classes.

[Allen Johnson]

Just to expand on this a little. Think of Rails as an example where
static resources are in a separate folder and templates are in the
app/views folder:

myapp/
app/controllers/*
app/views/*
public/*

This would translate nicely to a lein-ring project:

myapp/
src/**/*.clj (controllers)
resources/**/* (templates/views)
public/**/* (static resources)
project.clj

When creating the war file, all public items are included at the root
of the war, all resources and classes go to WEB-INF/classes. During
development, lein-ring could wrap the app handler with ring's
wrap-file and serve the static content directly out of the public dir.
When packaged as a war, the web.xml would use a filter like above.

[James Reeves]

On 10 March 2011 15:15, Allen Johnson <akjoh...@gmail.com> wrote:
> When creating the war file, all public items are included at the root
> of the war, all resources and classes go to WEB-INF/classes. During
> development, lein-ring could wrap the app handler with ring's
> wrap-file and serve the static content directly out of the public dir.

Why treat static resources differently to any other sort of resource?

- James

[James Reeves]

On 10 March 2011 10:46, Saul Hazledine <sha...@gmail.com> wrote:
> I can see the reason for this in terms of simplicity and flexibility
> (there is just one mechanism for dealing with all resources and its
> the same during development and deployment). However, servlet
> resources are a royal pain which is why the hack you mentioned earlier
> where everything goes into WEB-INF/classes is the usual solution. In
> fact, to support dynamically loading clojure source, leiningen-war
> even ended up putting the contents of the src directory into WEB-INF/
> classes. I never found a better solution but it would be really cool
> if you find a better way to do this.

I may go with this solution. (And really, I should have looked at
Leiningen-war more closely when I started building lein-ring. It's
obvious you thought about more about the consequences than I did.)

Idiomatic Clojure prefers native constructs over their Java
counterparts. So we use persistent vectors instead of arrays, seqs
instead of iterators, and so forth. We only drop back down to Java to
handle things Clojure has no construct of its own for (e.g. streams),
or for maintaining compatibility with an external library.

So given this philosophy, I'd prefer to do as much as possible in
Ring, and have as little to do with Servlets as possible. We should be
primarily concerned with ensuring Ring behaves consistently, which
means resources should be accessible in the same manner whether being
accessed through a Ring adapter, or when deployed as a WAR file.

- james

[Allen Johnson]

> Why treat static resources differently to any other sort of resource?

IMHO, ring handlers are for dynamic content. Why burden it with static
content when the servlet container can be used for that purpose?

But that aside, I welcome any option. I just want an easy way to serve
static content when in either development mode or war file. I know you
mentioned the /static/* servlet mapping but I get uncomfortable when I
have to give up on "pretty-urls". It's something I can live with -- if
that is the way everyone else is handling it. Just thought I'd offer
my point of view.

Allen

[James Reeves]

On 10 March 2011 15:36, Allen Johnson <akjoh...@gmail.com> wrote:
>> Why treat static resources differently to any other sort of resource?
>
> IMHO, ring handlers are for dynamic content. Why burden it with static
> content when the servlet container can be used for that purpose?

There are a few reasons against:

1. It increases the area of contact between the handler and the
servlet container.
2. It is less simple, in that it requires two different ways of
handling resources.
3. It gives developers less control over how static resources are delivered.
4. It ties the project to using lein-ring, as normal Ring adapters
don't automatically handle static resources.

Which in my mind outweigh the reasons for:

1. It's a little quicker.
2. It's more idiomatic Java.

- James

[James Reeves]

On 10 March 2011 15:36, Allen Johnson <akjoh...@gmail.com> wrote:

> But that aside, I welcome any option. I just want an easy way to serve
> static content when in either development mode or war file.

I'll try and get something up that works for everything (not just
Compojure) within the next few weeks. Possibly by this weekend.

- James

[Sean Allen]

On Thu, Mar 10, 2011 at 10:36 AM, Allen Johnson <akjoh...@gmail.com> wrote:
>> Why treat static resources differently to any other sort of resource?
>
> IMHO, ring handlers are for dynamic content. Why burden it with static
> content when the servlet container can be used for that purpose?
>

I think that is rather limited way to look at it. ring is just a
common interface.
It could be for static/dynamic or whatever. Why would I want to have one
way for static and another for dynamic? That seems like a very
arbitrary division.

It might work for you, but everyone? I doubt that.

[Allen Johnson]

@Sean - sorry for the direct response. Grrr gmail :-\

> I think that is rather limited way to look at it. ring is just a
> common interface.

I concede that. I'm a ring n00b and I'm bringing a Java web
development point-of-view to this.

> It could be for static/dynamic or whatever. Why would I want to have one
> way for static and another for dynamic? That seems like a very
> arbitrary division.

It's not that arbitrary. In Rails, when deploying via passenger on
apache, you let apache serve the static files out of the public dir
and rails only handles the dynamic pages. Similar for PHP, apache
serves static content and only the dynamic stuff goes through mod_php.
Similar division could be said for Django, Java, ASP.NET, CGI, etc.

I think James is right and if he can get something more general
together that will also work with war files then that is what I want
to use. I'm just saying that my division of static/dynamic resources
is not arbitrary.

> It might work for you, but everyone? I doubt that.

Just me, rails, php, django, java, asp.net, cgi, etc... :)

[Michael Ossareh]

Is it easier to have lein ring know what container you're using and it decide where to put static resources according to that? i.e. in project.clj:

 : ring {:container :tomcat }

in the case of  ``lein ring server``  the jetty configuration is implicit.

In the case of a container needing the resources under WEB-INF/classes the war/uberwar task just copies things there?

Flags/hooks (via hooke) could be provided to override default behaviour where preferred. 

Clearly the down side of this is that users of ring would be required to use lein ring.

[Sean Allen]

@Allen- no problem at all. I don't mind direct responses. I didn't
think anything of it.