Start of a thin rest library for compojure

[Philipp Meier]

Hi all,

I started a thin library on top of compojure which provides a
webmachine-style way of defining RESTful resources. What I have is by
no means complete and still growing. However I'd appreciate any
feedback. It's available at

http://github.com/ordnungswidrig/compojure-rest

To get an impression of how a resource is defined, a small example
follows:

(ns test
(:use compojure)
(:use compojure-rest))

(defn hello-resource []
(compojure-rest/make-handler {
:get (fn [req] (str "Hello " ((req :route-params {}) :who
"unknown foreigner")))
:generate-etag (fn [req] ((req :route-params) :who))
:expires (constantly 10000) ; expire in 10 sec
:last-modified (constantly -10000) ; last modified 10 sec ago
:authorized? (fn [req] (not (= "tiger" ((req :route-params
{}) :who))))
:allowed? (fn [req] (not (= "scott" ((req :route-params
{}) :who))))
}))


(defroutes my-app
(ANY "/hello/:who" (hello-resource))
(ANY "/simple/" (compojure-rest/make-handler { :get (fn [req]
"Simple") }))
(ANY "*" (page-not-found)))


(run-server {:port 8080}
"/*" (servlet my-app))


Regards,
-billy.

[James Reeves]

On Nov 5, 2:08 pm, Philipp Meier <phme...@gmail.com> wrote:
> I started a thin library on top of compojure which provides a
> webmachine-style way of defining RESTful resources. What I have is by
> no means complete and still growing. However I'd appreciate any
> feedback.

I think it would be better to make use of middleware in this case. For
example:

(defroutes hello-resource
(GET "/"
(str "Hello (params :who "unknown"))))

(decorate hello-resource
(wrap-etag (comp :who :route-params))
(wrap-expiry (constantly 10000))
(wrap-auth some-auth-function)
...)

To make Compojure more RESTful, I'm planning on decoupling the output
layer from the routes. This would allow you to do this:

(defroutes example
(GET "/person/name"
{:name "Fred", :email "fr...@example.com"}))

(decorate example
wrap-json-output
wrap-xml-output)

So the HTTP client can now choose between JSON and XML outputs. There
will also be middleware that will allow you to return a raw string of
HTML from your routes, in order to remain compatibile with the current
behaviour of Compojure.

(defroutes example
(GET "/person/name"
(html [:h1 "Fred"])))

(decorate example
wrap-raw-output)

- James

[Philipp Meier]

On 5 Nov., 22:06, James Reeves <weavejes...@googlemail.com> wrote:
> On Nov 5, 2:08 pm, Philipp Meier <phme...@gmail.com> wrote:
>
> > I started a thin library on top of compojure which provides a
> > webmachine-style way of defining RESTful resources. What I have is by
> > no means complete and still growing. However I'd appreciate any
> > feedback.
>
> I think it would be better to make use of middleware in this case. For
> example:

>
>   (defroutes hello-resource
>     (GET "/"
>       (str "Hello (params :who "unknown"))))
>
>   (decorate hello-resource
>     (wrap-etag (comp :who :route-params))
>     (wrap-expiry (constantly 10000))
>     (wrap-auth some-auth-function)
>     ...)

Great! I thought about something like you outlined, too, but obviously
my compojure-fu was to low. What I miss with the decorators is to pass
along some context. For example if the generation of the etag or
expiry-date is not ultra-light you'd want to re-use the result while
building the body. Is there a way in compojure to do this?

> To make Compojure more RESTful, I'm planning on decoupling the output
> layer from the routes. This would allow you to do this:
>
>   (defroutes example
>     (GET "/person/name"

>       {:name "Fred", :email "f...@example.com"}))
>
>   (decorate example
>     wrap-json-output
>     wrap-xml-output)

And compojure will negotiate the content-type based on what the output
functions provide? Don't forget that HTTP enables to negotiate the
language, character-set and encoding as well. While character-set and
encoding can be handled transparently by a decorator, the available
languages can only be determined by the output layer.

So the output layer would be at least responsible for

* Providing the content-types it can handle
* Providing the languages it can provide
* Build the actual response

Am I right that the output layer could be specified using decorators
as well, sth. like this:

(defn output-person [req]
(str (({:de "Hallo, " :en "Hello, "} (req :language)) (-> req
params :who "unknown")))

(decorate-output output-person
(wrap-negotiate-languages [:de :en])

Did you start on the implementation yet? Do you plan to branch
compojure or are you going to put it in another library.

-billy.

[James Reeves]

On Nov 6, 8:55 am, Philipp Meier <phme...@gmail.com> wrote:
> Great! I thought about something like you outlined, too, but obviously
> my compojure-fu was to low. What I miss with the decorators is to pass
> along some context. For example if the generation of the etag or
> expiry-date is not ultra-light you'd want to re-use the result while
> building the body. Is there a way in compojure to do this?

Not sure I quite understand... Could you perhaps give an example?

> And compojure will negotiate the content-type based on what the output
> functions provide? Don't forget that HTTP enables to negotiate the
> language, character-set and encoding as well. While character-set and
> encoding can be handled transparently by a decorator, the available
> languages can only be determined by the output layer.

Well, I haven't thought about it much yet, but there are a few ways
around this. You could have a wrapper set of middleware:

(defroutes example
(wrap-output
xml-output
json-output))

Or perhaps each output middleware adds output wrapped in a delay, then
some per-built piece of middleware at the end chooses amongst the
delayed outputs and renders the correct one.

> Did you start on the implementation yet? Do you plan to branch
> compojure or are you going to put it in another library.

TBH I haven't thought that far. My current plan is to factor out much
of Compojure's functionality into separate libraries. At the moment
I'm working on a more efficient HTML generator. If you want to
experiment with ideas for a more RESTful Compojure, I'd be interested
to see what you come up with.

- James

[Philipp Meier]

On 6 Nov., 11:03, James Reeves <weavejes...@googlemail.com> wrote:
> On Nov 6, 8:55 am, Philipp Meier <phme...@gmail.com> wrote:
>
> > Great! I thought about something like you outlined, too, but obviously
> > my compojure-fu was to low. What I miss with the decorators is to pass
> > along some context. For example if the generation of the etag or
> > expiry-date is not ultra-light you'd want to re-use the result while
> > building the body. Is there a way in compojure to do this?
>
> Not sure I quite understand... Could you perhaps give an example?

I was looking for a way to pass information from one decorator to the
next. For example the calculation of an ETag could require the (slow)
lookup of some entity from a database. Later, while generating the
response body you want to re-use the already looked up entity.

Of course the handler can associate some information in the request
where the later handler read look it up:

First handler calls

(handler (assoc-in request ::rest :etag))

Later handeler cal lookup the etag: (-> request ::rest :etag)

> > Did you start on the implementation yet? Do you plan to branch
> > compojure or are you going to put it in another library.
>
> TBH I haven't thought that far. My current plan is to factor out much
> of Compojure's functionality into separate libraries. At the moment
> I'm working on a more efficient HTML generator. If you want to
> experiment with ideas for a more RESTful Compojure, I'd be interested
> to see what you come up with.

I've implemented etag, authentication and authorization decorator and
will push them to my compojure-rest repository soon. At the moment you
can do sth. like:

(defroutes hello-resource
(GET "/"
(str "Hello" (params :who " unknown"))))

(decorate hello-resource
(wrap-etag (comp :who :route-params))

(wrap-expiry 10000) ;; you can pass a value or a function on
request
(wrap-auth (comp not #(some #{%} ["evil", "dark"]) :who :route-
params))
(wrap-allow (comp not #(some #{%} ["scott". "bob",
"ann"]) :who :route-params)))


-billy.

[Philipp Meier]

On 5 Nov., 22:06, James Reeves <weavejes...@googlemail.com> wrote:

> On Nov 5, 2:08 pm, Philipp Meier <phme...@gmail.com> wrote:
>
> > I started a thin library on top of compojure which provides a
> > webmachine-style way of defining RESTful resources. What I have is by
> > no means complete and still growing. However I'd appreciate any
> > feedback.
>
> I think it would be better to make use of middleware in this case. For
> example:
>
>   (defroutes hello-resource
>     (GET "/"
>       (str "Hello (params :who "unknown"))))
>
>   (decorate hello-resource
>     (wrap-etag (comp :who :route-params))
>     (wrap-expiry (constantly 10000))
>     (wrap-auth some-auth-function)
>     ...)

I pushed a new branch of compojure-rest to github:

http://github.com/ordnungswidrig/compojure-rest/tree/decorators

The branch contains middleware for

* etag generation and negotiation
* expiry
* authorization
* authentication

James, I think this is more the way you thought, right? It might not
be the most idiomatic clojure and surely it needs some polish but
I'd happy to receive some feedback on this.

-billy.

[James Reeves]

On Nov 12, 10:58 am, Philipp Meier <phme...@gmail.com> wrote:
> James, I think this is more the way you thought, right? It might not
> be the most idiomatic clojure and surely it needs some polish but
> I'd happy to receive some feedback on this.

Hi there,

I've not taken an extensive look through it, but instead of:

(defn hello-resource [request]
((-> (method-not-allowed)
(wrap-generate-body (fn [r] (str "Hello
" ((request :params) :who "stranger"))))
(wrap-etag (comp :who :params))
...)
request))

Perhaps you want to start off by linking a multimethod that returns
information from the model (product-repository), up to a series of
different outputs:

(def product-resource
(resource product-repository
:json json-outputter
:xml xml-outputter
:html product-html-view))

Or perhaps the outputs are middleware:

(def product-resource
(resource product-repository))

(decorate product-resource
(wrap-output
:json json-outputter
:xml xml-outputter
:html product-html-view))

Perhaps the multimethod looks something like this:

(defmulti product-resource :request-method)

(defmethod product-resource :get [request]
(sql-query "select * from products where id = ?" (->
request :params :id)))

Or something like that...

But I basically I think you need a main function or data structure of
some kind that defines the resource, and then use middleware to add
outputs, caching and so forth.

- James

[Philipp Meier]

On 12 Nov., 23:06, James Reeves <weavejes...@googlemail.com> wrote:
> On Nov 12, 10:58 am, Philipp Meier <phme...@gmail.com> wrote:
>
> > James, I think this is more the way you thought, right? It might not
> > be the most idiomatic clojure and surely it needs some polish but
> > I'd happy to receive some feedback on this.

> Perhaps you want to start off by linking a multimethod that returns
> information from the model (product-repository), up to a series of
> different outputs:
>
> (def product-resource
>   (resource product-repository
>     :json json-outputter
>     :xml  xml-outputter
>     :html product-html-view))

> But I basically I think you need a main function or data structure of
> some kind that defines the resource, and then use middleware to add
> outputs, caching and so forth.

I did not think of how to specify different representations of the
resource, yet. A map :type -> producing-function as you outlined above
sounds fine.

Defining a multimethod on the method makes no sense for a RESTful
resource. A meaningful resource representation must be generated in
the case of a GET only. For a POST, PUT and DELETE the most of the
time a simple numeric status + plaintext explaination will be
sufficient. Sometimes a redirect must be sent i.e. after a POST.

So, at the moment my plan is the following:

* Provide a collection of middleware methods supporting most of the
HTTP aspects like, content-negotiations, authorization and so on. This
middleware functions will be of general use (I hope)
* Based on the above middleware, provide a handler which can be
parametrized with functions to handle POST, DELETE and PUT as well as
producer functions which provide the body representation for specified
content-types. Maybe something like the following:

(def product-resource
(resource :delete (fn [req] delete-in-db ((req :params) :id))
:put (fn [req] create-or-update-in-db
((req :params) :id) (req :body))
:get { :json (fn [req] generic-to-json (get-from-db
((req :params) :id)))
:xml (fn [req] generic-to-xml (get-from-db
((req :params) :id))) }))

-billy.

[James Reeves]

On Nov 15, 9:10 pm, Philipp Meier <phme...@gmail.com> wrote:
> * Based on the above middleware, provide a handler which can be
> parametrized with functions to handle POST, DELETE and PUT as well as
> producer functions which provide the body representation for specified
> content-types. Maybe something like the following:
>
> (def product-resource
>    (resource :delete (fn [req] delete-in-db ((req :params) :id))
>              :put    (fn [req] create-or-update-in-db
> ((req :params) :id) (req :body))
>              :get    { :json (fn [req] generic-to-json (get-from-db
> ((req :params) :id)))
>                        :xml  (fn [req] generic-to-xml  (get-from-db
> ((req :params) :id))) }))

Why not something like:

(defroutes product-resource
(DELETE "/" (delete-in-db (params :id))
(PUT "/" (create-or-update-in-db (params :id) (request :body)))
(GET "/" (get-from-db (params :id)))

(decorate product-resource
(wrap-output
"text/json" write-json-str
"text/xml" some-xml-outputter))

- James

[Philipp Meier]

On 16 Nov., 03:01, James Reeves <weavejes...@googlemail.com> wrote:

> Why not something like:
>
> (defroutes product-resource
>   (DELETE "/" (delete-in-db (params :id))
>   (PUT "/" (create-or-update-in-db (params :id) (request :body)))
>   (GET "/" (get-from-db (params :id)))
>
> (decorate product-resource
>   (wrap-output
>     "text/json" write-json-str
>     "text/xml"  some-xml-outputter))

That looks strange to me because the wrap-output would only be applied
in case of a GET request. However I like the idea that the GET handler
only returns the model and a seperate output handle will do the
presentation in JSON/XML etc. This should by appliable for all HTTP
methods because all HTTP methods can return a body. Having a separate
wrap-output, wrap-delete-output etc. seems verbose.

I'm playing with the idea to use the brand new protocol implementation
to describe the protocol that a reasonable resource must follow.

-billy.