Contributing

[James Reeves]

Hi folks,

For those interested in contributing to Compojure, here's a list of
things that would be useful to have. I'll start with the simplest
tasks that require the least knowledge of Clojure and Compojure, and
work up to the more complicated stuff:

1. Add a task to the Ant build script to automatically download and
unzip the deps.zip file from:
http://cloud.github.com/downloads/weavejester/compojure/deps.zip

2. Write a Clojure script to pull out all the docstrings and function
names from the Compojure source and automatically compile them into a
HTML API reference like the one at clojure.org/api.

3. Add a Clojure wrapper for another embedded Java HTTP server, such
as Grizzly.

4. Implement a file-based session store, using the multimethods
defined in compojure.http.session. Use the already-implemented :memory
and :cookie session store types as an example.

5. Write a function to create a useful set of routes, such as a login
page. You could create an "authenticate" multimethod so that users
could add their own authentication back end, and perhaps allow them to
specify a template for the page. You wouldn't be able to create a
login page that would be suitable for every site, but it would be nice
to have a basic component to use for mock-ups and simple sites. Other
common pieces of functionality, like tables of paged data, or a list
of comments, could also potentially be factored out into functions.

6. Add a middleware function to add caching functionality to existing
routes. Something like Rack::Cache.

7. Add support to compojure.http.routes for full URLs. If the user
specifies a relative path, like (GET "/" ...), it should work as it
does now. But if the user specifies a full URL, like (GET "http://
www.example.com/" ...) it should match the hostname and protocol.
Additionally, you should be able to specify parameters in the URLs,
like (GET "http://:subdomain.example.com" ...).


Reply to this thread if you're interested in working on any of these
areas, or if you have a suggestion for a new feature you think
Compojure should possess that isn't listed above.

Task no. 5 also requires some brief explanation, which I'll be happy
to expand on. Unlike many frameworks, a route in Compojure is handled
by a function, and several routes can be combined into one using the
routes function. This means we can group together routes for a common
task, such as a GET which shows a login page and a POST that
authenticates the user based on the login form parameters. This is an
interesting feature that isn't present in many other frameworks, and
deserves some exploration. So task no. 5 is really a request for
anyone interested to experiment a little with Compojure's routes, and
see if we can't come up with something to automate a lot of the bog-
standard features most web applications have, or at least create
something that developers can use as a basis to create their own
custom components.

Let me know if you happen to have any questions or need a more
detailed explanation.

- James

[Craig Andera]

For 2, you can check out clojure.contrib.gen-html-doc, which I wrote.
It makes a reference like this [1].

[1] http://clojure.googlegroups.com/web/clj-libs+(3).html

[James Reeves]

On Mar 30, 12:20 am, Craig Andera <craig.and...@gmail.com> wrote:
> For 2, you can check out clojure.contrib.gen-html-doc, which I wrote.
> It makes a reference like this [1].
>
> [1]http://clojure.googlegroups.com/web/clj-libs+(3).html

I'll take a look! I think I'll tweak the style a little, but that's
basically what I wanted.

Other things that can be knocked off the TODO list are tasks 1 and 2,
which Luke Renn has completed. If anyone has any further feature
suggestions, I'd be happy to consider them.

- James

[Luke Renn]

On Mar 30, 5:15 pm, James Reeves <weavejes...@googlemail.com> wrote:
> Other things that can be knocked off the TODO list are tasks 1 and 2,
> which Luke Renn has completed. If anyone has any further feature
> suggestions, I'd be happy to consider them.
>
> - James

It was actually number 3, the Grizzly server implementation. I'm
still cleaning it up and abstracting out the common stuff between
Grizzly and Jetty, but it works well.

Thanks,

Luke

[James Reeves]

On Mar 30, 11:50 pm, Luke Renn <luke.r...@gmail.com> wrote:
> It was actually number 3, the Grizzly server implementation.  I'm
> still cleaning it up and abstracting out the common stuff between
> Grizzly and Jetty, but it works well.

Oops, right, it was 3.

- James

[Sean]

I recently read a post by Zed Shaw where he reviews Django. Go down
to point #10, where he describes Pinax.

http://www.zedshaw.com/blog/2009-03-20.html

Is that sorta like what you have in mind with point 5 on your list?

[youngblood.carl]

I found Pinax to be a touch complicated in its current state, but I
like the idea of composable reusable pieces of functionality as apps
(which was also Django's). I think it would help if the Pinax apps had
a more consistent api, which I know is one of the project's goals.

[Hubert Iwaniuk]

Hi Luke,

Is there a way i could see code of Grizzly integration?
I'm very interested in it, and hopefully can help a bit.

Thanks,
Hubert.

[Luke Renn]

On Mar 31, 10:32 am, Hubert Iwaniuk <hubert.iwan...@gmail.com> wrote:
> Hi Luke,
>
> Is there a way i could see code of Grizzly integration?
> I'm very interested in it, and hopefully can help a bit.
>
> Thanks,
>    Hubert.

Hi Hubert,

Sure, it was actually very easy. I just copied the Jetty back end and
replaced the Jetty calls with Grizzly calls which is why there is not
a formal patch yet (lots of duplication). It took me longer to figure
out why I couldn't set the headers (Compojure API change) than it did
to make Grizzly work :) Jetty doesn't set any headers by default, so
browsers assume text/html. Grizzly defaults to text/plain so your
route will need {:headers {"Content-Type" "text/html"}}.

I'll upload the file grizzly.clj to the group, but you'll need to
change a few other things.

1. Download grizzly.clj and place in src/compojure/server.

2. Edit the build.xml

<arg value="compojure.server.jetty"/>
+ <arg value="compojure.server.grizzly"/>

3. Remove jetty and add grizzly (they can't coexist as of yet).
Edit src/compojure.clj

- 'compojure.server.jetty
+ 'compojure.server.grizzly

3. Download and place into the deps folder:
http://download.java.net/maven/2/com/sun/grizzly/grizzly-http-servlet/1.9.10/grizzly-http-servlet-1.9.10.jar

(James, I sent you the wrong link in my mail, I sent sources,
obviously you only need bin).

4. Build compojure.

5. Add the text/html header to your existing routes.
(defroutes my-app
(GET "/grizzly"
[{:headers {"Content-Type" "text/html"}}
"<html><body><h1>Hello From Grizzly!</h1></body></html>"])
(GET "/*"
(or (serve-file (params :*)) :next))
(ANY "*"
(page-not-found)))

Thanks,

Luke

[Hubert Iwaniuk]

Hi,

That certainly is interesting :)
I'll wait for your code been posted to group.
I wonder if we you really want Servlet integration, GrizzlyAdapter is
nice and removes one layer of abstraction.
Why and is it needed to have both servers (Jetty & Grizzly) available
at the same time?

Cheers,
Hubert.

> 3. Download and place into the deps folder:http://download.java.net/maven/2/com/sun/grizzly/grizzly-http-servlet...

[Luke Renn]

On Mar 31, 2:05 pm, Hubert Iwaniuk <hubert.iwan...@gmail.com> wrote:
> That certainly is interesting :)
> I'll wait for your code been posted to group.
> I wonder if we you really want Servlet integration, GrizzlyAdapter is
> nice and removes one layer of abstraction.
> Why and is it needed to have both servers (Jetty & Grizzly) available
> at the same time?
>
> Cheers,
>    Hubert.

The codes already been posted to the group.

http://compojure.googlegroups.com/web/grizzly.clj

GrizzlyAdapter was used.

I didn't really mean have both available. I just meant that you can't
have compojure.clj include both Jetty and Grizzly yet since it'd be a
namespace clash when you (use 'compojure), thus the change to
compojure.clj. The signatures are the same as the Jetty
implementation.

Like I've said, this is just a way to get Grizzly to serve up
Compojure. I'm still waiting to fully integrate it in a way that's
acceptable to be merged into the official code base.

I don't want to fill this thread up with Grizzly stuff so if you've
got any other questions, comments, or code changes email me directly
or perhaps we should create a new thread.

Thanks,

Luke

[Luke Renn]

> > I wonder if we you really want Servlet integration, GrizzlyAdapter is
> > nice and removes one layer of abstraction.

> GrizzlyAdapter was used.

Ah, I see the confusion. I yet again gave out the wrong jar. The
correct required Grizzly jar is:

grizzly-servlet-webserver-1.9.10.jar

Thanks,

Luke

[Luke Renn]

Actually, I didn't use GrizzlyAdapter, I used ServletAdapter. I guess
I'm confused by your statement that we don't want Servlet integration
(since that's currently how Compojure works). Perhaps I misunderstood
the original request, but you can use now use Grizzly in the same way
Jetty is used now.

Thanks,

Luke

[James Reeves]

On Mar 31, 7:49 pm, Luke Renn <luke.r...@gmail.com> wrote:
> Actually, I didn't use GrizzlyAdapter, I used ServletAdapter.  I guess
> I'm confused by your statement that we don't want Servlet integration
> (since that's currently how Compojure works).  Perhaps I misunderstood
> the original request, but you can use now use Grizzly in the same way
> Jetty is used now.

I think there certainly should be the option of using servlets, as
that allows you to mix Compojure routes with pre-existing servlets
(like ContinuationCometdServlet in Jetty).

But perhaps it would be useful to also allow people to attach routes
directly to the server. So:

(run-server my-routes)

Would be functionality equivalent to:

(run-server "/*" (servlet greet))

But it's not really a big deal, as it doesn't save much code, so I
haven't got around to that yet.

Regarding namespaces, I'm considering moving compojure.http.jetty out
of the default namespaces added to compojure, so that people can
choose which server they want to use.

- James

[James Reeves]

On Mar 31, 1:42 pm, Sean <francoisdev...@gmail.com> wrote:
> I recently read a post by Zed Shaw where he reviews Django.  Go down
> to point #10, where he describes Pinax.
>
> http://www.zedshaw.com/blog/2009-03-20.html
>
> Is that sorta like what you have in mind with point 5 on your list?

Pinax looks very similar to what I want to do, yep! I suspect that we
might be able to get something a little neater-looking than Pinax. For
instance:

(def sql-repository
{:type :sql
:host "localhost"
:user "root"
:db "example"})

(defroutes site
(login-route "/account/login" sql-repository)
...)

I'm far from having on decided exactly how it's going to work, but if
people want to experiment, they might come up with some ideas. Here's
another possible idea for syntax:

(defroutes site
(with-template login-page-template
(login-route "/account/login"))
(with-template main-template
(paged-list "/messages" :messages)))

(run-server (with-repo sql-repository site))

- James

[budu]

Hi, I'm interested principally by number five as I'm already
experimenting with routes to see what's the best way to use them like
everybody I suppose! My project is just a simple personal website by
the way. One thing I've worked on in previous versions was to make the
authentication form embeddable as a sort of control. That way it'll be
more flexible and the authentication form can easily be placed
somewhere in the template or only into specific pages. All of it's
code is kept into controls.auth namespace and I use it in a template
passing it the request map. This is not working well with the new,
more functional, approach to sessions. The more I'm thinking about
this, the more I think it would be a good problem to solve using
monads, I'll look further into that this week-end. I know this is all
kind of overkill for my use case, but I'd like to find the best way to
build reusable components with Compojure.

Also, as all my pages will be displayed using the same template, I'm
looking for a way to specify a default template and reduce repetition
as much as possible. I've written a macro yesterday that partially do
what I want, here it is:

(defmacro page [prefix args & body]
`(defn ~(symbol (str prefix "-page"))
[~'request ~@args]
(with-template ~'meta-data ~'request
(~'with-request-bindings
~'request
~@body))))

The meta-data var is a map specified at the beginning of each pages
and contains things like title, description, etc. For now, my template
is hard-coded into the with-template function, but I'll soon rewrite
it to be more general. The goal here is to make routes definitions
very simple and easily readable.

Another thing is how to handle the root path as when I deploy my
website to Jetty it become "/foo" instead of "/". The routes mechanism
work great, but all links are now wrong. For now I use a simple
function to expand urls beginning with a "~", but it's extremely ugly:

(defn expand [request s]
(.replaceAll s "^~" (.getContextPath (request :servlet-request))))

There's expand call everywhere, so I'll try to automate this trough
the page macro above. Yet this solution doesn't satisfy me, page
functions can also call other function that aren't pages and could
contains urls too, so maybe I need a new macro especially for
expanding urls.

What do you think about all of this, can some of these ideas are worth
integrating into Compojure? Do you have some suggestions on how to
make these ideas more general and solid?

- budu

[James Reeves]

On Apr 3, 11:32 pm, budu <nbudu...@gmail.com> wrote:
> Hi, I'm interested principally by number five as I'm already
> experimenting with routes to see what's the best way to use them like
> everybody I suppose! My project is just a simple personal website by
> the way. One thing I've worked on in previous versions was to make the
> authentication form embeddable as a sort of control. That way it'll be
> more flexible and the authentication form can easily be placed
> somewhere in the template or only into specific pages. All of it's
> code is kept into controls.auth namespace and I use it in a template
> passing it the request map. This is not working well with the new,
> more functional, approach to sessions. The more I'm thinking about
> this, the more I think it would be a good problem to solve using
> monads, I'll look further into that this week-end. I know this is all
> kind of overkill for my use case, but I'd like to find the best way to
> build reusable components with Compojure.

I'd be interested in seeing what you've done with controls.auth so
far.

> Also, as all my pages will be displayed using the same template, I'm
> looking for a way to specify a default template and reduce repetition
> as much as possible. I've written a macro yesterday that partially do
> what I want, here it is:
>
> (defmacro page [prefix args & body]
> `(defn ~(symbol (str prefix "-page"))
> [~'request ~@args]
> (with-template ~'meta-data ~'request
> (~'with-request-bindings
> ~'request
> ~@body))))
>
> The meta-data var is a map specified at the beginning of each pages
> and contains things like title, description, etc.

Could you give an example of how you'd use this? I don't really see
how the page macro would make things more concise.

> Another thing is how to handle the root path as when I deploy my
> website to Jetty it become "/foo" instead of "/". The routes mechanism
> work great, but all links are now wrong.

Hm. I see the problem. Maybe I should take the Rails approach and
write a url-for function that my helper functions can use. Let me
think about this a bit :)

> What do you think about all of this, can some of these ideas are worth
> integrating into Compojure? Do you have some suggestions on how to
> make these ideas more general and solid?

You might need to provide some more examples of usage. You're
essentially providing code for your solutions, without giving us the
code for the problems, if you see what I mean.

- James

[budu]

> I'd be interested in seeing what you've done with controls.auth so
> far.

You'll have to wait a few days, it's a bit of a mess right now and
I'll try to find a way to make it work with the new in-memory
session.

> Could you give an example of how you'd use this? I don't really see
> how the page macro would make things more concise.

Here's an example:

(def meta-data
{:name "blog"
:title "Blog"
:lang "en"
:description "This is a blog."})

(page view-blog [url is-admin?]
[:div#posts
(when is-admin?
[:div#new_post (link-to (str url "/new") "new")])
(mapcat #(view-post url % is-admin?) (dal/posts))])

(defroutes routing
(GET "*" (view-blog-page request
(expand request "~/blog")
(-> session :user :is_admin))))

It feels a little bit hackish due to being forced to define meta-data
at the top of the file and routes at the bottom, but at the same time
it happen to the way I would do things naturally. The real goal (I was
totally confused about the goal in my last post ;-) is to be able to
work on a page with everything right at your fingertips. I'm not sure
that would scale well with more complex website, but for my use case
it's perfect. The worse part is the expand calls that are in the route
definitions, it's ugly now, it will rapidly become a nightmare. Maybe
a macro could capture the request var and have a very short name. I've
tried also to add this in the page macro: (let [e (partial expand
request... which make it a lot more bearable, yet a little bit more
obfuscated.

> You might need to provide some more examples of usage. You're
> essentially providing code for your solutions, without giving us the
> code for the problems, if you see what I mean.

Yeah, it seems I won't ever get rid of that problem!

- budu

[Perry Trolard]

On Apr 3, 2009, at 6:06 PM, James Reeves <weave...@googlemail.com>
wrote:
>>
>>

>> Another thing is how to handle the root path as when I deploy my
>> website to Jetty it become "/foo" instead of "/". The routes
>> mechanism
>> work great, but all links are now wrong.
>
> Hm. I see the problem. Maybe I should take the Rails approach and
> write a url-for function that my helper functions can use. Let me
> think about this a bit :)

I've been meaning to set something up to address this problem, but
hadn't yet gotten to it. I'll be curious to see what you come up with,
James.

FWIW, the use cases I have in mind are

(1) budu's, where internal links remain correct after changing the
servelet context (i.e. context is picked up dynamically)

(2) being able to specify a "context" other than the current
servelet context, say, for when an app is running behind Apache with
mod_proxy (i.e. context is hard-coded)

Best,
Perry

[Adrian Cuthbertson]

I also ran into the new routes problem while migrating my stuff to the
ring version. However this has actually created a _better_ situation
if one uses parameterised route matching. Here's a simple app
illustrating this...

(ns ct
(:use [compojure http html])
(:use [compojure.server jetty]))

(defn post-to
"Helper function to build an action uri for form posts."
[request loc]
(str "/" (:ctx (:route-params request)) loc))

(defn frm
[request]
(let [nm (:name (:form-params request))]
(html [:html [:body
(form-to {:class "forms"} [POST (post-to request "/frm")]
[:ul {:class :app-form}
[:li (label :name "Name:") " " (text-field :name (str nm))]]
(submit-button "Go"))]])))

(defroutes res-routes
(GET "/:ctx/*"
(or (serve-file "./res" (:* (:route-params request))) :next))
(ANY "*"
(page-not-found)))

(defroutes pub-routes
(GET "/:ctx/*"
(or (serve-file "./pub" (:* (:route-params request))) :next))
(ANY "*"
(page-not-found)))

(defroutes adm-routes
(GET "/:ctx/hello"
(prn "route --->" (:route-params request))
(html [:html
[:body [:h1 "Hello World from /adm"]]]))
(GET "/:ctx/frm"
(frm request))
(POST "/:ctx/frm"
(frm request))
(ANY "*"
(html [:html
[:body [:h1 "Adm - Page Not found."]]])))

(defn run-jty []
(run-server {:port 8080}
"/res/*" (servlet res-routes)
"/adm/*" (servlet adm-routes)
"/pub/*" (servlet pub-routes)))

So, here we have three separate servlets each with a different "base"
route. In the three defroutes, by using a parametrised match string
(:ctx), the :ctx captures the base of the uri path and we can use it
for post action uri's etc. See the helper function post-to above.

This means the routes functions become "portable" and can simply be
wired into any servlet without naming clashes. Awesome!

Regards, Adrian

[James Reeves]

Here's another possible solution for giving a bunch of routes a
'context'. I've created a with-context wrapper that checks to see if
(:uri request) has the right prefix, and if it does, it removes the
prefix from the request and passes it on to the contained routes:

(declare *context*)

(defn with-context
[ctx & route-seq]
(let [handler (apply routes route-seq)
pattern (re-pattern (str "^" ctx "(/.*)?"))]
(fn [request]
(if-let [[_ uri] (re-matches pattern (:uri request))]
(binding [*context* ctx]
(handler (assoc request :uri uri)))))))

(defn url [path]
(str *context* path))

(defroutes my-app
(with-context "/foo"
(GET "/"
(str "Hello from " (url "/")))
(GET "/bar"
(str "Hello from " (url "/bar"))))
(ANY "*"
(page-not-found)))

You could also write it as:

(defroutes my-app
(GET "/"
(str "Hello from " (url "/")))
(GET "/bar"
(str "Hello from " (url "/bar"))))

(run-server {:port 8080}
"/foo/*" (servlet (with-context "/foo" my-app))
"/baz/*" (servlet (with-context "/baz" my-app)))

- James

On Apr 4, 5:09 am, Adrian Cuthbertson <adrian.cuthbert...@gmail.com>
wrote:

[Perry Trolard]

with-context is great, James. Thanks.

One question: in pre-Ring Compojure, the path against which a route is
matched does not contain the context -- e.g. routes in an app attached
at context /public/* would only see /afile.html when /public/
afile.html is requested. Am I correct that now the full path,
including the "context," is passed as (:uri request)? If so, why the
change -- decoupling from Jetty, or conformance to the Ring spec, &c?

Thanks,
Perry

On Apr 4, 2009, at 8:26 AM, James Reeves <weave...@googlemail.com>
wrote:

[James Reeves]

On Apr 6, 1:26 am, Perry Trolard <trol...@gmail.com> wrote:
> with-context is great, James. Thanks.
>
> One question: in pre-Ring Compojure, the path against which a route is  
> matched does not contain the context -- e.g. routes in an app attached  
> at context /public/* would only see /afile.html when /public/
> afile.html is requested. Am I correct that now the full path,  
> including the "context," is passed as (:uri request)? If so, why the  
> change -- decoupling from Jetty, or conformance to the Ring spec, &c?

Pretty much all of the above. Ring is independent of the Java servlet
system, so (:uri request) is always the full path. This means a
Compojure route will always act the same, no matter what back-end
server it happens to be connected to. Another result of this approach
is that no information is lost unless explicitly specified. The
'context' remains, unless you remove it with a wrapper like with-
context, and even then you can keep it around in the background as a
var.

The disadvantage is that it's a little more verbose, but I think the
trade-off is worth it. I'd be interested in knowing what other people
think about it, though.

- James

[Perry Trolard]

I'm happy to trade the very small violation of DRY for generalizing
away from Java servlets. & as you've shown in with-context, it's easy
to get the pre-Ring behavior where desirable.

With the pre-Ring branch, it's not exactly that the context part of
the request path is lost, but that knowing how to find it is servlet
API-specific, i.e.:

(-> servlet .getServletConfig .getServletContext .getContextPath)

I'm familiar with Django, & it behaves like post-Ring Compojure: no
matter the "root" of the Django app, routes get passed the entire URI
path (& Django has similar methods of stripping off common prefixes
from it).

In any case, do you plan to include with-context in a Compojure
library? Whatever
it's worth, it'd help folks who were relying on context-less
route-matching migrate to the Ring branch.

Thanks again,
Perry