I get 404 no matter how I arrange these routes

[Lawrence Krubner]

Something very similar to this code was working up till 2 days ago. Then I decided to redo the way I way the app work -- I moved more of the work to middleware. Then I ran into the problem that when I start the app, and test it from cURL, I get the 404 message that I defined with route/not-found. I get this no matter what URL I try. 

I assumed the problem was with my middleware, since I had just added new middleware. But now, to find the problem, I've commented out all of the middleware, and I'm still getting the same problem. 

I am baffled. Below is my start function. Have I missed something obvious? 

(defn start [map-of-config]

  (try

    (let [port (if (nil? (:port map-of-config))

                 34000

                 (Integer/parseInt (:port map-of-config)))

 app-routes (routes

        (ANY "/" homepage)

          (GET "/v1/token" token)

          (GET "/v1/errors" errors)

    (ANY "/v1/:token/:name-of-collection/object-id/:object-id" request query/fetch)

    (ANY "/v1/:token/:name-of-collection/:document-id" request query/fetch)

    (ANY "/v1/:token/:name-of-collection/" request query/fetch)

    (route/resources "/")

    (route/not-found "Page not found. Check the http verb that you used (GET, POST, PUT, DELETE) and make sure you put a collection name in the URL, and possbly also a document ID. Also, all requests should go to an URL that starts with /v1"))

          app (-> app-routes

;;  (middleware/wrap-prepare-message)

;;                  (middleware/wrap-error-post-with-document-id)

;;                  (middleware/wrap-malformed?)

;;          (middleware/wrap-check-content-type)

;;                  (middleware/wrap-transaction-id)

 ;;                 (middleware/wrap-token-embed-in-json-document)

 ;;                 (middleware/wrap-token-check)

 ;;                 (middleware/wrap-cors-headers)

                  (wrap-json-response)

                  (wrap-keyword-params)

                  (wrap-multipart-params)

                  (wrap-nested-params)

                  (wrap-params)

 (wrap-json-body {:keywords? true})

                  (wrap-content-type))

jetty (run-jetty app {:port port :join? false :max-threads 5000})]

      (timbre/log :trace (str "The port number we will listen to: " port))

      ;; we want to reboot this app from the REPL, so the start function needs to overwrite "server"

      (swap! server (fn [old-value] jetty)))

    (catch Exception e (println e))))

[James Reeves]

You've missed out the binding on most of the routes, which means your body is now treated as the binding.

So for instance, this:

    (ANY "/" homepage)

Means: for any method accessing the URI "/", bind the request map to the symbol "homepage" and return nil.

Instead, you want something like:

    (ANY "/" [] homepage)

- James

--
You received this message because you are subscribed to the Google Groups "Compojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to compojure+...@googlegroups.com.
To post to this group, send email to comp...@googlegroups.com.
Visit this group at http://groups.google.com/group/compojure.
For more options, visit https://groups.google.com/d/optout.

[Lawrence Krubner]

Thank you for that. I made some changes, as you suggested. My current problem is that I can not get the parts of the uri to show up as params. If I call:

"http://localhost:34000/v1/abc202aa-8d02-4cc0-8394-d796c36b3/accounts/"    

And then print the request to the terminal, I can see that the URI is there in the request, but the {} params are empty. What am I doing wrong? Should I suspect my middleware of causing havoc? 

(defroutes        app-routes

                     (ANY "/" request homepage)

                     (GET "/v1/token" request token)

                     (GET "/v1/errors" request errors)

                     (ANY "/v1/:token/:name-of-collection/object-id/:object-id" request query/fetch)

                     (ANY "/v1/:token/:name-of-collection/:document-id" request query/fetch)

                     (ANY "/v1/:token/:name-of-collection/" request query/fetch)

                     (GET "/v1/:token/:name-of-collection" request query/fetch)

                     (route/resources "/")

                     (route/not-found "Page not found. Check the http verb that you used (GET, POST, PUT, DELETE) and make sure you put a collection name in the URL, and poss\

bly also a document ID. Also, all requests should go to an URL that starts with /v1"))

(defn start [map-of-config]

  (try

    (let [port (if (nil? (:port map-of-config))

                 34000

                 (Integer/parseInt (:port map-of-config)))

          app (-> app-routes

                  (middleware/wrap-prepare-message)

                  (middleware/wrap-error-post-with-document-id)

                  (middleware/wrap-malformed?)

                  (middleware/wrap-check-content-type)

                  (middleware/wrap-transaction-id)

                  (middleware/wrap-embed-in-json-document)

                  (middleware/wrap-token-check)

                  (wrap-json-response)

                  (wrap-json-body {:keywords? true})

                  (wrap-keyword-params)

                  (wrap-params)

                  (middleware/wrap-cors-headers))

          jetty (run-jetty app {:port port :join? false :max-threads 5000})]

      (timbre/log :trace (str "The port number we will listen to: " port))

      ;; we want to reboot this app from the REPL, so the start function needs to overwrite "server"

      (swap! server (fn [old-value] jetty)))

    (catch Exception e (println e))))

I end up with a request that looks like: 

{:json-params {:transaction-id "ea6f61b6-a84f-4ab2-afb0-715d66ff92cb", :token "abc202aa-8d02-4cc0-8394-d796c36b3"}, :ssl-client-cert nil, :remote-addr "173.3.205.147", :params {}, :headers {"host" "localhost:34000", "user-agent" "curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8z zlib/1.2.5", "accept" "*/*"}, :server-port 34000, :content-length nil, :form-params {}, :query-params {}, :content-type nil, :character-encoding nil, :uri "/v1/abc202aa-8d02-4cc0-8394-d796c36b3/accounts/", :server-name "localhost", :query-string nil, :body #<HttpInput org.eclipse.jetty.server.HttpInput@3855645f>, :multipart-params {}, :scheme :http, :request-method :get}

[James Reeves]

I don't think there's enough information to say for sure, but it may be the middleware you use. Certainly the request map looks odd; there's no :route-params key that Compojure adds.

How are you printing the request map? Are you printing it in middleware, or inside the matched route?

- James

[Lawrence Krubner]

An example of how I print out the request in the middleware: 

(defn wrap-token-check

  "Enforcing a limit on how many requests a user can make before they get a new token"

  [handler & [opts]]

  (fn [request]

    (clojure.pprint/pprint (str " in wrap-token-check " request))

    (timbre/log :trace (str " in wrap-token-check " request))

    (try

      (if (or (#{"/" "/v1/token" "/v1/errors"} (:uri request)) (tokens/is-valid? request))

          (do

            (tokens/increment-token (:token request))

            (handler request))

          {:status 401

           :body (str "Invalid token: " (:token request))

           :headers {"Content-Type" "application/json"}})

      (catch Exception e (println e)))))

I am reading what is in the Timbre log. 

My current project dependencies: 

  :dependencies [[org.clojure/clojure "1.6.0"]

                 [com.taoensso/timbre "3.2.1"]

                 [dire "0.5.1"]

                 [slingshot "0.10.3"]

                 [ring "1.2.1"]

                 [clj-time "0.6.0"]

                 [org.clojure/data.json "0.2.5"]

                 [compojure "1.3.4"]

                 [com.novemberain/monger "2.0.1"]

                 [org.clojure/tools.namespace "0.2.4"]

                 [manifold "0.1.0"]

                 [me.raynes/fs "1.4.4"]

                 [org.clojure/core.incubator "0.1.3"]

                 [clj-stacktrace "0.2.7"]

                 [overtone/at-at "1.2.0"]

                 [ring/ring-json "0.3.1"]

                 [clj-http "1.1.2"]

                 [org.clojure/core.cache "0.6.4"]

                 [cheshire "5.5.0"]

                 [org.clojure/core.match "0.3.0-alpha4"]]

This was working 3 days ago, and then I re-did the software, and now it is not working. Maddening. I'm not sure what I broke. 

I've tried to comment out my own middleware, but that didn't seem to have an effect. 

[Lawrence Krubner]

I upgraded ring:

                 [ring "1.4.0-RC1"]

but that made no difference

[Lawrence Krubner]

And if I comment all but one of my middleware: 

(defn start [map-of-config]

  (try

    (let [port (if (nil? (:port map-of-config))

                 34000

                 (Integer/parseInt (:port map-of-config)))

          app (-> app-routes

                  (middleware/wrap-prepare-message)

;;                  (middleware/wrap-error-post-with-document-id)

;;                  (middleware/wrap-malformed?)

;;                (middleware/wrap-check-content-type)

;;                (middleware/wrap-transaction-id)

;;                (middleware/wrap-embed-in-json-document)

;;                (middleware/wrap-token-check)

                  (wrap-json-response)

                  (wrap-json-body {:keywords? true})

                  (wrap-keyword-params)

                  (wrap-params)

;;                (middleware/wrap-cors-headers)

)

          jetty (run-jetty app {:port port :join? false :max-threads 5000})]

      (timbre/log :trace (str "The port number we will listen to: " port))

      ;; we want to reboot this app from the REPL, so the start function needs to overwrite "server"

      (swap! server (fn [old-value] jetty)))

    (catch Exception e (println e))))

And print request in that middleware: 

(defn wrap-prepare-message

  [handler & [opts]]

  (fn [request]

    (clojure.pprint/pprint (str " in wrap-prepared-message " request))

    (timbre/log :trace (str " in wrap-prepared-message " request))

    (try

    (let [message {:verb (:request-method request)

                   :object-id (get-in request [:params :object-id])

                   :document-id (get-in request [:params :document-id])

                   :field-to-sort-by (keyword (get-in request [:params :field-to-sort-by] "created-at"))

                   :match-field-1 (get-in request [:params :match-field-1])

                   :match-value-1 (get-in request [:params :match-value-1])

                   :offset (Integer/parseInt (get-in request [:params :offset] "0"))

                   :limit (Integer/parseInt (get-in request [:params :limit] "50"))

                   :document (get-in request [:json-params] {})

                   :name-of-collection (get-in request [:params :name-of-collection])}

          request (assoc request :message message)]

      (handler request))

      (catch Exception e (println e)))))

all the params are still empty: 

{:ssl-client-cert nil, :protocol "HTTP/1.1", :remote-addr "173.3.205.147", :params {}, :headers {"host" "localhost.com:34000", "accept" "*/*", "user-agent" "curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4 OpenSSL/0.9.8z zlib/1.2.5"}, :server-port 34000, :content-length nil, :form-params {}, :query-params {}, :content-type nil, :character-encoding nil, :uri "/v1/9abc4d3d-58f7-493c-8c61-dd701126cdbb/accounts/", :server-name "localhost", :query-string nil, :body #<HttpInputOverHTTP HttpInputOverHTTP@13086580>, :scheme :http, :request-method :get}

Also right now I get a 404, but that's because some of the code, further down the line, is throwing an exception, and that is because some work that should happen in the middleware is not happening, because I commented it out. 

[Lawrence Krubner]

This is interesting. If I re-write my routes so that "request" is passed inside of a vector, then everything stops working: 

(defroutes app-routes

                     (ANY "/" [request] (homepage request))

                     (GET "/v1/token" [request] (token request))

                     (GET "/v1/errors" [request] (errors request))

                     (ANY "/v1/:token/:name-of-collection/object-id/:object-id" [request] (query/fetch request))

                     (ANY "/v1/:token/:name-of-collection/:document-id" [request] (query/fetch request))

                     (ANY "/v1/:token/:name-of-collection/" [request] (query/fetch request))

                     (GET "/v1/:token/:name-of-collection" [request] (query/fetch request))

                     (route/resources "/")

                     (route/not-found "Page not found. Check the http verb that you used (GET, POST, PUT, DELETE) and make sure you put a collection name in the URL, \

and possbly also a document ID. Also, all requests should go to an URL that starts with /v1"))

fetch begins like this, and the :pre assertion fails: 

(defn fetch [request]

  {:pre [

         (map? request)

         (map? (:message request))

        ]}

This throws an Assertion Exception, because "request" is not a map. 

I am sure I will laugh when I find out what I did wrong, but at the moment I am a bit amazed at this. 

[Lawrence Krubner]

Huh, if I do this in query/fetch:

(defn fetch [request]

(println "this is the request: ")

(clojure.pprint/pprint request)

then everything is there as I expect: 

 :params

 {:token "9abc4d3d-58f7-493c-8c61-dd701126cdbb",

  :name-of-collection "accounts",

  "username" "rollio"},

 :route-params

 {:name-of-collection "accounts",

  :token "9abc4d3d-58f7-493c-8c61-dd701126cdbb"},

I don't see how this is possible. Over the last few hours, in an attempt to rule out an error arising from the order of the middleware, I have rearranged the middleware approximately 1 billion times. I've gone through every permutation of commenting stuff out and rearranging the order. And still, I never see :params filled in anywhere in any middleware. But it is there in query/fetch. Maddening. 

Anyway, I will get rid of some of my middleware and move the work to the query namespace. I admit to being beaten this time. 

[James Reeves]

Okay, those are a lot of messages. I think your problem is that you don't completely understand how routes and middleware work.

Let's consider a single route as an example:

  (GET "/:token" request get-token)

Here we're returning a function, get-token, from the route. If you return a function, Compojure passes it the request, and then returns the result of the function. So the above is equivalent to:

  (GET "/:token" request (get-token request))

We could also get the token directly, instead of passing the entire request map:

  (GET "/:token" [token] (get-token token))

By using a vector, we tell Compojure to find a parameter called token. This should explain why:

  (GET "/:token" [request] (get-token request))

doesn't work. Because you're telling Compojure to find a parameter called request, which is obviously not your intention.

Let's go back to this code:

  (GET "/:token" request (get-token request))

The GET macro will transform this into an anonymous function that looks a little like:

  (fn [req]

    (if (= (:request-method req) :get)

      (if-let [route-params (route-matches "/:token" req)]

        (let [request (assoc request :route-params route-params)]

          (get-token request)))))

If either the request-method or route doesn't match, then nil is returned from the function. This indicates the function didn't match.

If the function did match, then the route parameters are associated with the request. In the above code I've assigned them only to :route-params for simplicity, but in practice they're also merged into :params as well.

This should tell you why your middleware wasn't seeing the parameters in the route, because your middleware surrounds the routes, but the route parameters are only assigned inside the route.

There are ways around this. Compojure has a function called wrap-routes, which is designed to apply middleware inside of a route. You can also apply the middleware inside the route, like so:

  (GET "/:token" request (wrap-some-middleware get-token))

However, it's worth noting that Compojure is designed around the idea of controlling the flow of information. If I write something like:

  (GET "/:token" request (get-token request))

Then any key in the request map could potentially affect get-token. But if I write:

  (GET "/:token" [token] (get-token token))

Then only the value of the token parameter affects get-token. By giving the function only what it needs, we limit the number of things that can go wrong in the application.

You don't need to use Compojure's parameter syntax for this, but it's worth bearing in mind that you should be aiming to limit how functions interact. Clojure is designed around the notion of simplicity, which in this sense means reducing the number of interconnections in software.

- James

[Lawrence Krubner]

Thank you so much. Regarding this: 

"This should tell you why your middleware wasn't seeing the parameters in the route, because your middleware surrounds the routes, but the route parameters are only assigned inside the route."

I would have thought I could work around this with some clever mix of calling the middleware either on the request (in some situations) or on the response (in other situations) but it seems I was wrong. Is it Ring or Compojure that takes responsibility for "the route parameters are only assigned inside the route"? I would like to find that exact line of code, so I might figure out how to better alter it. 

[James Reeves]

On 15 June 2015 at 20:38, Lawrence Krubner <lawr...@rollioforce.com> wrote:

Thank you so much. Regarding this: 

"This should tell you why your middleware wasn't seeing the parameters in the route, because your middleware surrounds the routes, but the route parameters are only assigned inside the route."

I would have thought I could work around this with some clever mix of calling the middleware either on the request (in some situations) or on the response (in other situations) but it seems I was wrong. Is it Ring or Compojure that takes responsibility for "the route parameters are only assigned inside the route"? I would like to find that exact line of code, so I might figure out how to better alter it. 

The route parameters are assigned by the route. They have to be, because until you hit the route, you don't know what the route parameters are.

For example, consider the following URI: /foo/10

What are the route parameters for that URI? We don't know yet. We know the URI, but not how to map it to route parameters.

Now consider the following route:

  (GET "/foo/:id" [id] (get-foo id))

Now we can work out that the route parameters are {:id "10"}, because now we have the path template "/foo/:id". Without that template, we can't work out how to destructure the URI into parameters.

This means that middleware outside the route has no access to route parameters that are defined in the route itself.

- James

[Lawrence Krubner]

Thank you for that. I examined this bit of code: 

https://github.com/weavejester/compojure/blob/639fffd3f8b77153644d0c9b87bca5af56be0cb0/src/compojure/core.clj#L244

I will investigate further to see if that can be made to work for my architecture, or whether its time to consider another design.