Rethinking route syntax

[James Reeves]

Hi folks,

I've started factoring out the route parser in Clojure to a separate
library, and I've been thinking about ways of improving the syntax.
Currently, it follows the Sinatra/Rails scheme:

"/:account/product/:id"

But I've been thinking that the syntax from the Routes Python library
is a little better:

"/{account}/product/{id}"

The {} are more unambiguous, and you can optionally specify a regular
expression:

"/{account}/product/{id:\\d+}"

Anyone think this is a bad idea? A good idea? Any alternative syntax
for routes I should consider?

- James

[Chouser]

What about a something other than a string? I don't know the
current syntax or use cases well, so this suggestion may not make
sense, but what about a vector of things instead of /'s in
a string? This would let you leverage different literal types,
nice #"" regex escapting, etc:

[:account #".*", "product", :id #"\d+"]

Hm, that looks more cluttered than I was hoping. Maybe more like
destructuring a regex result would be better:

[account id] #"(.*)/product/(\d*)"

Well, just a couple ideas that came to mind.

--chouser

[Shantanu Kumar]

Both of these are excellent ideas. I have been thinking about something like this:

(defn id-matches?

  [value]

  ; match Number / String / Date

  ; do a range-check (from 230 to 693, or from 2008-09-01 to 2009-06-30)

  ; do a regex check, do whatever

  true)

(GET "/{account}/product/{id}" { "id" id-matches? } )

So, while matching the route the value of "id" is passed to the function to determine whether it really matches. I mean, why restrict to just regex checks? You could do a range-check there, or a cache-lookup or whatever -- there are many possibilities.

I would probably add something like this in Taimen 0.2 -- http://code.google.com/p/bitumenframework/

Regards,

Shantanu

[James Reeves]

On Oct 20, 9:38 pm, Chouser <chou...@gmail.com> wrote:
> Hm, that looks more cluttered than I was hoping.  Maybe more like
> destructuring a regex result would be better:
>
>         [account id] #"(.*)/product/(\d*)"

This seems interesting, but:

(GET [account id] #"(.*)/product/(\d*)"
...)

Kinda suggests to me that the groups should be put into local vars,
rather than in (params :account) and (params :id). I wonder if the
parameters in the route should be treated differently to the
parameters in a form or in the query string.

One thing that's always troubled me is the special bindings like
request, params and session that are added in by GET behind the
scenes. But without these local bindings, it seems more verbose. For
instance, if 'routes' was just a fancy 'cond' statement, you could
write:

(defn handler [request]
(routes [request]
(GET #"/products/(.*)" [group]
(controller group (-> request :params :page)))
(ANY #".*"
(page-not-found))))

That's all very explicit, but kinda messy. I guess you could condense
defn and routes:

(defroutes handler [request]
(GET #"/products/(.*)" [group]
(controller group (-> request :params :page)))
(ANY #".*"
(page-not-found))))

I dunno. I don't like (-> request :params :page) just to get a
parameter from the query string. Perhaps if there were more complex
parameters

(defroutes handler [{params :params, :as request}]
(GET #"/products/(.*)" [group]
(controller group (params :page)))
(ANY #".*"
(page-not-found))))

But that seems like a lot of boilerplate, really.

- James

[James Reeves]

On Oct 20, 10:20 pm, Shantanu Kumar <kumar.shant...@gmail.com> wrote:
> Both of these are excellent ideas. I have been thinking about something like
> this:
>
> (defn id-matches?
>   [value]
>   ; match Number / String / Date
>   ; do a range-check (from 230 to 693, or from 2008-09-01 to 2009-06-30)
>   ; do a regex check, do whatever
>   true)
>
> (GET "/{account}/product/{id}" { "id" id-matches? } )

I dunno, I kinda think there doesn't seem to be much difference
between

(GET "/{account}/product/{id}" {"id" id-matches?}

(foobar))

And

(GET "/{account}/product/{id}"

(if (id-matches? (params :id)) (foobar))

I'm not sure I'm really convinced that a route should contain
arbitrary complexity, though I guess I can see it's advantage.

- James

[Shantanu Kumar]

I dunno, I kinda think there doesn't seem to be much difference
between

 (GET "/{account}/product/{id}" {"id" id-matches?}

   (foobar))

And

 (GET "/{account}/product/{id}"
   (if (id-matches? (params :id)) (foobar))

I'm not sure I'm really convinced that a route should contain
arbitrary complexity, though I guess I can see it's advantage.

I was thinking about enterprise environments where people end up with multiple versions (bifurcated on value-range or some such criteria) over a period of time. This kind of a thing helps segregate things at the route level rather than body level. So if I have 4 different versions for the same route pattern, it's sort of easier to have 4 routes rather than 4 cond checks in the body. It also seems to be easier to migrate selected routes from one server to another rather than refactoring the body, have it tested and lose time-to-market in the process.

Regards,

Shantanu

[Shantanu Kumar]

Kinda suggests to me that the groups should be put into local vars,
rather than in (params :account) and (params :id). I wonder if the
parameters in the route should be treated differently to the
parameters in a form or in the query string.

Spring 3.0 MVC treats them differently, and calls them "route variables" -- I think that definition makes sense.

Regards,

Shantanu

[Andrew Boekhoff]

I wonder if something along the lines of the Ruby framework waves
might be appropriate. Users define resources (marked by some portion of the uri
string) and then can determine what representation to render based on matches
against the entire http request.

Something like
(def-resource post "/posts" ;; var-name and uri-dispatch string
(on :uri [#"\d{4}" #"\d+"] ;; matches "posts/2005/3"
(when :content-type "text/html" (render foo))
(when :content-type "text/json" ...))

Of course a lot of syntax twiddling is possible, but I found this approach to
be extremely flexible with a minimum of boilerplate. Ok, perhaps it breaks with
Compojure's Sinatra-like roots too much.

also, as a possible approach to handle boiler plate, I've been using an
approach I've come to call 'autobinding' (no idea if I'm reinventing the wheel
with this or its a YDIW, I'm quite inexperienced with Lisp).
Its basically a macro that does a regex match on symbols.

Here I'll use symbols that start with "&"

(defview myview
&=myview
[:p &bar]))

which expands to something like:

(def view (fn [environment]
(let [&bar (:bar environment)]
(merge environment {:myview [:p &bar]}))))

this is mostly handy for doing things like (render (-> env view1 view2 ...))
but perhaps a similar approach might be an alternative approach for
letting users get to request data with minimum ceremony?

Finally, I've taken to using Compojure for pretty much anything I would have
used Sinatra before I started doing Clojure, so I would just like to add a
thanks for contributing it to the community.

Cheers,
Andy

[Nicolas Buduroi]

Hi, there's lots of great ideas here, but I wonder if it wouldn't be a
better one to go the other way. That is, changing the regex URI
matcher instead of the string one. That way, routes would still behave
the same way as Sinatra when using string routes. What I'm thinking
about would be to simply add a vector of keyword after the regex, like
this:

(GET #"/(.+)/product/(\d+)" [:account :id] ...)

- budu

[Shantanu Kumar]

(GET #"/(.+)/product/(\d+)" [:account :id] ...)

Wouldn't that mean the URI template is actually a Regex? For example,

"hello/*"    ==becomes=> "/hello/(.*)"

Please correct me if I am missing something obvious here -- I guess expecting a user to pass Regex-aware URI template is debatable!

Regards,

Shantanu

[Shantanu Kumar]

Correction:

"/hello/*"    ==becomes=> "/hello/(.*)"

[Perry Trolard]

> (GET #"/(.+)/product/(\d+)" [:account :id] ...)

+1 to budu's idea.

I appreciate the point in James' & Chouser's proposals: that a path
isn't just a string, it's a series of steps, so why not represent it
properly as a sequence? Most of the time this approach is satisfactory
for me, but sometimes I want to specify a path that breaks this model:
maybe the path has multiple numbers of steps (& not just at the end),
or some of whose "steps" are themselves a path, e.g.:

/get-resource/:resource
=>
/get-resource/food/fruit/apples

The above works with Compojure's current syntax (:*), but not with
more than one step that's a path:

/get-resource-with-metadata/md/time/last-modified/rsrc/food/fruit/
apples

For the above I'd want to specify a regex like

/get-resource-with-metadata/md/(.*?)/rsrc/(.*)

This too is currently possible with Compojure, but the index-based
access to the regex groups is just cumbersome enough to be
discouraging. budu's vector of keywords solves this, I think.

To sum: +1 for two ways to specify the path, (1) path-as-sequence
(either with current syntax ("/get-resource/:resource") or with vector
of steps), & (2) a regex with vector of keywords following that names
the groups.

Perry

[Brian Carper]

On Oct 22, 10:04 am, Nicolas Buduroi <nbudu...@gmail.com> wrote:
>
> (GET #"/(.+)/product/(\d+)" [:account :id] ...)
>

+1, or maybe:

(GET [#"/(.+)/product/(\d+)" [:account :id]] ...)

or:

(GET [#"/(.+)/product/(\d+)" :account :id] ...)

(This way has the added benefit of potentially not breaking all
existing Compojure apps out there.)

I prefer this over "{id:\\d+}". Packing named routes and regex routes
into a string is a bit too much. Especially the disparity between
#"\d" and "\\d" is messy and I bet it's going to trip people up.

Honestly, I like the current "/:id" colon-syntax as it is too. It's
very clear that "/:id" becomes (params :id). It's lightweight and it
works fine. It matches visually where "{id}" doesn't; Clojurians are
already used to visually picking out keywords. Slashes are as
explicit a delimiter as curly braces. I hope you keep this syntax
around intact even if you add another option.

[Nicolas Buduroi]

> Honestly, I like the current "/:id" colon-syntax as it is too. It's
> very clear that "/:id" becomes (params :id). It's lightweight and it
> works fine. It matches visually where "{id}" doesn't; Clojurians are
> already used to visually picking out keywords. Slashes are as
> explicit a delimiter as curly braces. I hope you keep this syntax
> around intact even if you add another option.

Same thing here.

> (GET [#"/(.+)/product/(\d+)" :account :id] ...)
>
> (This way has the added benefit of potentially not breaking all
> existing Compojure apps out there.)

Good idea, that would also simplify the implementation. I'll try to
see what I can do if I have some time, shouldn't be that hard.

- budu

[James Reeves]

Here's another idea I've had:

(GET "/groups/:group/products"
[group page]
(show-paged-products group page))

The [group page] define the only bindings the closure has. No request,
no params, no session. Instead, the bindings are taken from
(request :params). As with the current version of Compojure,
(request :params) includes data parsed from the route and the query
string.

At first this seems limiting; without a request map you can't access
the headers or anything else about the request except for the
parameters.

But I'm thinking maybe that's the job of middleware. So if a header or
anything else in the request is potentially relevant, the middleware
can add it to the parameters.

Not sure how good an idea this is. I guess I'll need to think about it
a little more...

- James

[James Reeves]

On Oct 23, 10:17 am, James Reeves <weavejes...@googlemail.com> wrote:
> Here's another idea I've had:
>
>   (GET "/groups/:group/products"
>     [group page]
>     (show-paged-products group page))

A slight refinement of this idea:

(GET "/thread/{id}"
[session/user id]
(if (has-permissions? user id)
(get-thread id)))

The namespace can be used to get other keys in the request map, like
session or headers. If there is no namespace, it defaults to
using :params.

- James

[James Sofra]

Hi,

I have been working on a route generation macro for Compojure. That
generates actions for resources and an individual resource.
It loosely follows the rails routes conventions found on this page
http://guides.rubyonrails.org/routing.html
It also adds nesting without regex.

For example I could say something like this:

(with-route-generators
(defroutes seasonswap
("/resources"
(resources "/people" people/actions
(resource "/car" car/actions))
(GET "/test" (str "Test")))
(ANY "*" (page-not-found))))

and it might expand to something like this:

(defroutes
seasonswap
(GET "/resources/people/remove" [(people/actions :remove)])
(POST "/resources/people" [(people/actions :create)])
(GET "/resources/people/:people-id" [(people/actions :show)])
(GET "/resources/people/:people-id/car/remove" [(car/
actions :remove)])
(POST "/resources/people/:people-id/car" [(car/actions :create)])
(GET "/resources/people/:people-id/car" [(car/actions :show)])
(GET "/resources/test" (str "Test"))
(ANY "*" (page-not-found)))

people/actions and car/actions would be multimethods that define which
actions resources and a resource may respond to, they have predefined
actions and you can specify your own. So the people/actions may have
been defined in a people namespace as something like:

(defmulti actions (fn [action] action))
(defmethod actions :show [action] (view (str "show")))
(defmethod actions :create [action] (view (str "create")))
(defmethod actions :remove [action] (view (str "remove")))

there is a bit more to it than that and I really only started it to
learn how to write macros but thought I would share the idea.

Cheers,
James

[James Sofra]

Sorry, stuffed up that expansion a little bit, should be using
the :destroy keyword and not :remove so it would come out like:

(defroutes
seasonswap
(DELETE "/resources/people/:people-id" [(people/actions :destroy)])

(POST "/resources/people" [(people/actions :create)])
(GET "/resources/people/:people-id" [(people/actions :show)])

(DELETE "/resources/people/:people-id/car" [(car/actions :destroy)])

(POST "/resources/people/:people-id/car" [(car/actions :create)])
(GET "/resources/people/:people-id/car" [(car/actions :show)])
(GET "/resources/test" (str "Test"))
(ANY "*" (page-not-found)))

Cheers,
James

On Oct 27, 12:08 am, James Sofra <james.so...@gmail.com> wrote:
> Hi,
>
> I have been working on a route generation macro for Compojure. That
> generates actions for resources and an individual resource.

> It loosely follows the rails routes conventions found on this pagehttp://guides.rubyonrails.org/routing.html

[Nathan]

@James, I would stick with the :keyword syntax as it is much cleaner.
Instead, why not take an optional map of constraints {:id #"\d+"} and
even have a way to define those common ones globally. Perhaps
arbitrary functions could help verify incoming data, such as find-
slug? or on the route itself: authenticated?

As @Andrew points out with his Waves example, it may be useful to
match on a variety of things: http headers, mime types, file
extensions (.xml). I'm not sure how all these would be used, but it
does get me thinking about multimethod dispatch. Maybe macros on top
of it, or maybe just adopting some of the concepts like :default,
prefer-method, and derive? I'm not sure.

Best.

[Nathan]

Oh, one thing I should add.

Though there doesn't seem to be a clean way to do named regex matches
as used in Django (as an alternative to :keyword syntax), one thing
worth stealing from Django is the ability to include() other routes
from other files/namespaces. The idea that everything under /blog or
on the blog. subdomain has its routes defined elsewhere, and those
routes are relative (as if defined at the root).

[James Reeves]

On Oct 26, 1:08 pm, James Sofra <james.so...@gmail.com> wrote:
> I have been working on a route generation macro for Compojure. That
> generates actions for resources and an individual resource.

> It loosely follows the rails routes conventions found on this pagehttp://guides.rubyonrails.org/routing.html

> It also adds nesting without regex.

I've been thinking about this as well, but I've yet to come up with a
satisfying syntax. But I don't think you need a macro to do this; a
function will probably suffice.

I'm not keep on the idea of hierarchical route definitions. They just
confuse matters, in my opinion. The idea for generating resources
automatically is good, but I wonder if it's flexible enough to be
useful. I rarely find my routes matching up with the Rails ideal
precisely.

- James

[James Reeves]

On Oct 27, 2:32 am, Nathan <nyoung...@gmail.com> wrote:
> @James, I would stick with the :keyword syntax as it is much cleaner.

I was thinking of supporting both. The syntax doesn't overlap, after
all. The only problem with the {} syntax is that it can't be matched
with a regex, so I'll need to update my lexer.

> Instead, why not take an optional map of constraints {:id #"\d+"} and
> even have a way to define those common ones globally.

Take a look at Clout, my standalone routing library I've recently
created. You can do exactly what you describe.

> Perhaps arbitrary functions could help verify incoming data, such

> as find-slug? or on the route itself: authenticated?

Well, you can already do that, either with middleware or by
returning :next in the route body.

> As @Andrew points out with his Waves example, it may be useful to
> match on a variety of things: http headers, mime types, file
> extensions (.xml). I'm not sure how all these would be used, but it
> does get me thinking about multimethod dispatch.

Yes, that is interesting. Maybe we should just have a system for
binding routes to keywords, and then use those keywords to call
multimethods?

- James

[James Reeves]

The only thing Compojure lacks is the idea of relative routes. But
defining 'include' for Compojure wouldn't be very hard:

(defn include [handler]
(fn [request]
(let [subpath (-> request :route-params :*)
request (assoc route :uri subpath)]
(handler request))))

Then you could write:

(GET "/resources/*" (include (GET "/people/:id" ...))

- James

[James Sofra]

> I'm not keep on the idea of hierarchical route definitions. They just
> confuse matters, in my opinion.

You are probably right, I am not a web developer so don't have a well
informed opinion on it like I said I just wrote this for fun and to
learn about writing macros.

> The idea for generating resources
> automatically is good, but I wonder if it's flexible enough to be
> useful. I rarely find my routes matching up with the Rails ideal
> precisely.

Yeah I just used that as a guide I have never used Rails... I did
however build in more flexibility on creating new actions by just
adding new multimethods. The resource route looks up all the keys for
the multimethod and dispatches based on those keys so there are route
generators for the conventional ones
like :show :new :create :edit :destroy :list etc but then default
behavior for an unknown key is to produce a GET route and add the name
of the key to the end of the path. Then if the key is a vector the
first element is key that is used as the HTTP method and the second is
a string to be added to the end of the path. You can see what this
might look like below.

; defined in the a another *.clj file

(defmulti actions (fn [action] action))

(defmethod actions :show [action] (str "show"))
(defmethod actions :status [action] (str "status"))
(defmethod actions [:DELETE "/remove"] [action] (view "remove"))

; the route defs using macro
(with-route-generators
(defroutes seasonswap
(resources "/people" people/actions)
(ANY "*" (page-not-found))))

; expands to something like this
(defroutes
seasonswap
(GET "/people/status" [(people/actions :status)])
(DELETE "/people/remove" [(people/actions [:DELETE "/remove"])])
(GET "/people/:people-id" [(people/actions :show)])
(ANY "*" (page-not-found)))

Also had ideas on how the actions might define which params to pass to
it.

Cheers,
James

[Robert Campbell]

What about adding re-seq support?

For example, what if I want to support
"/tags/keyboard/peripheral/input-device" where the number and order of
tags is variable?

I could implement this as:

(map second (re-seq #"(?!/tags/)/([-\w]+)"
"/tags/keyboard/peripheral/input-device"))

which returns a seq of my tags

("keyboard" "peripheral" "input-device")

The problem I have with the current routes regex support is that it's
bound to re-matches, which only supports hard-coded groupings:
"/tags/(\w+)/(\w+)/(?#...forever..)" which doesn't work when you don't
know how many groups you might have.

There are still times when you want re-matches w/groups for
readability, but I'd like to at least have the option of capturing
these (unknown n) successive matches.

Rob

> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google Groups "Compojure" group.
> To post to this group, send email to comp...@googlegroups.com
> To unsubscribe from this group, send email to compojure+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/compojure?hl=en
> -~----------~----~----~----~------~----~------~--~---
>
>

[Robert Campbell]

Actually playing with it a bit more, I think this:

(GET "/tags/*"
(show-tags (.split (:* (:route-params request)) "/")))

which is currently supported, is more readable than my hypothetical
re-seq version:

(GET #"(?!/tags/)/([-\w]+)"
(show-tags (:route-params request)))

While the second version would remain more concise, I'm not sure it's worth it.

Rob

[Christophe Grand]

Hi,

Chiming in late but I experimented with non-string routes (see
http://moustache.cgrand.net/syntax.html#route) a while ago.
With Moustache the "/tags/*" route would be ["tags" & tags].

(The whole ring application would be (app ["tags" & tags] {:get (show-
tags tags)})).

James's example "/{account}/product/{id:\\d+}" would be [account
"product" [id #"\d+"]]

Christophe

[Robin B]

Can share syntax with Werkzeug routing:

http://werkzeug.pocoo.org/documentation/dev/routing.html

"/<account>/product/<int:id>"

Its a nice DSL because <> is not allowed in urls.

Robin

On Dec 27 2009, 6:24 pm, Robert Campbell <rrc...@gmail.com> wrote:
> What about adding re-seq support?
>
> For example, what if I want to support
> "/tags/keyboard/peripheral/input-device" where the number and order of
> tags is variable?
>
> I could implement this as:
>
> (map second (re-seq #"(?!/tags/)/([-\w]+)"
> "/tags/keyboard/peripheral/input-device"))
>
> which returns a seq of my tags
>
> ("keyboard" "peripheral" "input-device")
>
> The problem I have with the current routes regex support is that it's
> bound to re-matches, which only supports hard-coded groupings:
> "/tags/(\w+)/(\w+)/(?#...forever..)" which doesn't work when you don't
> know how many groups you might have.
>
> There are still times when you want re-matches w/groups for
> readability, but I'd like to at least have the option of capturing
> these (unknown n) successive matches.
>
> Rob
>
> On Tue, Oct 20, 2009 at 8:47 PM, James Reeves
>