Feedback on idiomatic API design

[Johan Haleby]

Hi, 

I've just committed an embryo of an open source project to fake http requests by starting an actual (programmable) HTTP server. Currently the API looks like this (which in my eyes doesn't look very Clojure idiomatic):

(let [fake-server (fake-server/start!)
        (fake-route! fake-server "/x" {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))})
        (fake-route! fake-server {:path "/y" :query {:q "something")}} {:status 200 :content-type "application/json" :body (slurp (io/resource "my2.json"))})]
        ; Do actual HTTP request
         (shutdown! fake-server))

fake-server/start! starts the HTTP server on a free port (and thus have side-effects) then you add routes to it by using fake-route!. The first route just returns an HTTP response with status code 200 and content-type "application/json" and the specified response body if a request is made with path "/x". The second line also matches that a query parameter called "q" must be equal to "something. In the end the server is stopped.

I'm thinking of converting all of this into a macro that is used like this:

(with-fake-routes! 

"/x" {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))}

{:path "/y" :query {:q "something")}} {:status 200 :content-type "application/json" :body (slurp (io/resource "my2.json"))})

This looks better imho and it can automatically shutdown the webserver afterwards but there are some potential problems. First of all, since starting a webserver is (relatively) slow it you might want to do this once for a number of tests. I'm thinking that perhaps as an alternative (both options could be available) it could be possible to first start the fake-server and then supply it to with-fake-routes! as an additional parameter. Something like this:

(with-fake-routes! 

        fake-server ; We pass the fake-server as the first argument in order to have multiple tests sharing the same fake-server

"/x" {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))}

 {:path "/y" :query {:q "something")}} {:status 200 :content-type "application/json" :body (slurp (io/resource "my2.json"))})

If so you would be responsible for shutting it down just as in the initial example. 

Another thing that concerns me a bit with the macro is that routes doesn't compose. For example you can't define the route outside of the with-fake-routes! body and just supply it as an argument to the macro (or can you?). I.e. I think it would be quite nice to be able to do something like this:

(let [routes [["/x" {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))}]

              [{:path "/y" :query {:q "something")}} {:status 200 :content-type "application/json" :body (slurp (io/resource "my2.json"))}]]]

     (with-fake-routes routes))

Would this be a good idea? Would it make sense to have overloaded variants of the with-fake-routes! macro to accommodate this as well? Should it be a macro in the first place? What do you think? 

Regards,

/Johan

[Marc Limotte]

I don't think you need a macro here.  In any case, I'd avoid using a macro as late as possible.  See how far you get with just functions, and then maybe at the end, add one macro if you absolutely need it to add just a touch of syntactic sugar.

routes should clearly be some sort of data-structure, rather than side-effect setter functions.  Maybe this:

(with-fake-routes!

  optional-server-instance

  route-map)

Where optional-server-instance, if it exists is, an object returned by (fake-server/start!).  If optional-server-instance is not passed in, then with-fake-routes! creates it's own and is free to call (shutdown!) on it automatically. And route-map is a Map of routes:

{

"/x"
  {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))}
{:path "/y" :query {:q "something")}}
  {:status 200 :content-type "application/json" :body (slurp (io/resource "my2.json"))}

}

Also, at the risk of scope creep, I could foresee wanting the response to be based on the input instead of just a static blob.  So maybe the value of :body could be a string or a function of 1 arg, the route-- in your code test with (fn?).

This gives you a single api, no macros, optional auto-server start/stop or explicit server management.

marc

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Johan Haleby]

Thanks for your feedback, exactly what I wanted.

On Tuesday, March 8, 2016 at 3:16:02 PM UTC+1, mlimotte wrote:

I don't think you need a macro here.  In any case, I'd avoid using a macro as late as possible.  See how far you get with just functions, and then maybe at the end, add one macro if you absolutely need it to add just a touch of syntactic sugar.

routes should clearly be some sort of data-structure, rather than side-effect setter functions.  Maybe this:

(with-fake-routes!

  optional-server-instance

  route-map) 

Where optional-server-instance, if it exists is, an object returned by (fake-server/start!).  If optional-server-instance is not passed in, then with-fake-routes! creates it's own and is free to call (shutdown!) on it automatically. And route-map is a Map of routes:

{

"/x"
  {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))}
{:path "/y" :query {:q "something")}}
  {:status 200 :content-type "application/json" :body (slurp (io/resource "my2.json"))}

}

+1. I'm gonna go for this option.

 

Also, at the risk of scope creep, I could foresee wanting the response to be based on the input instead of just a static blob.  So maybe the value of :body could be a string or a function of 1 arg, the route-- in your code test with (fn?).

That's a good idea indeed. I've already thought about this for matching the request. I'd like this to work:

{

 (fn [request] (= (:path request) "/x")) 

  {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))}

{:path "/y" :query {:q (fn [q] (clojure.string/starts-with? q "some"))}} 

  {:status 200 :content-type "application/json" :body (slurp (io/resource "my2.json"))}

}

Thanks a lot for your help and feedback!

[Johan Haleby]

On Wed, Mar 9, 2016 at 6:20 AM, Johan Haleby <johan....@gmail.com> wrote:

Thanks for your feedback, exactly what I wanted.

On Tuesday, March 8, 2016 at 3:16:02 PM UTC+1, mlimotte wrote:

I don't think you need a macro here.  In any case, I'd avoid using a macro as late as possible.  See how far you get with just functions, and then maybe at the end, add one macro if you absolutely need it to add just a touch of syntactic sugar.

routes should clearly be some sort of data-structure, rather than side-effect setter functions.  Maybe this:

(with-fake-routes!

  optional-server-instance

  route-map)  

Hmm now that I come to think of it I don't see how this would actually work unless you also perform the HTTP request from inside the scope of  with-fake-routes!, otherwise the server instance would be closed before you get the chance to make the request. Since you make an actual HTTP request you need access to the URI generated when starting the fake-server instance (at least if the port is chosen randomly). So either I suppose you would have to do like this (which requires a macro?):

(with-fake-routes! 

  {"/x" {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))}}

  ; Actual HTTP request

  (http/get uri "/x"))

where "uri" is created by the  with-fake-routes! macro or we could return the generated fake-server. But if so with-fake-routes! cannot automatically close the fake-server instance since we need the instance to be alive when we make the call to the generated uri. I suppose it would have to look something like this:

(let [fake-server (with-fake-routes! {"/x" {:status 200 :content-type "application/json" :body (slurp (io/resource "my.json"))}})]

(http/get (:uri fake-server) "/x")

(shutdown! fake-server))

If so I think that the second option is unnecessary since then you might just go with:

(with-fake-routes!

  required-server-instance

  route-map)  

instead of having two options. But then we loose the niceness of having the server instance be automatically created and stopped for us?

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/gieS5hQCUm4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[Marc Limotte]

Yes, I was assuming the HTTP calls happen inside the with-fake-routes! block.  

I missed the part about the random port.  I se 3 options for that:

Assign a port, rather than random

(with-fake-routes! 9999 ...)

But then, of course, you have to worry about port already in use.

An atom

(def the-uri (atom nil))

(with-fake-routes! the-uri

  ...

  (http/get @the-uri "/x"))

A macro

A common convention in Clojure would be to pass it a symbol (e.g. `uri` that is bound by the macro), rather implicitly creating `uri`.

(with-fake-routes! [uri option-server-instance] 

    route-map

    (http/get uri "/x"))

or, with a pre-defined server

(def fake-server ...)
(with-fake-routes!

    route-map

    (http/get (:uri fake-server) "/x"))

marc

[Johan Haleby]

Thanks a lot for your support Marc, really appreciated.

On Wed, Mar 9, 2016 at 5:33 PM, Marc Limotte <msli...@gmail.com> wrote:

Yes, I was assuming the HTTP calls happen inside the with-fake-routes! block.  

I missed the part about the random port.  I se 3 options for that:

Assign a port, rather than random

(with-fake-routes! 9999 ...)

But then, of course, you have to worry about port already in use.

An atom

(def the-uri (atom nil))

(with-fake-routes! the-uri

  ...

  (http/get @the-uri "/x"))

A macro

A common convention in Clojure would be to pass it a symbol (e.g. `uri` that is bound by the macro), rather implicitly creating `uri`.

(with-fake-routes! [uri option-server-instance] 

    route-map

    (http/get uri "/x"))

Didn't know about this convention so thanks for the tip. But is your snippet above actually working code or does the user need escape "uri" and "option-server-instance" using a single-quotes, i.e. 

(with-fake-routes! ['uri 'option-server-instance] ...)

[Marc Limotte]

With the macro approach, they don't need to escape it.

[Gary Verhaegen]

I would suggest a slightly different approach. First, define a record for your fake server, which implements Closeable; put your current "shutdown" code in the close method.

This will allow you to use the existing with-open macro, instead of having to redefine your own, while leaving the option of not using the macro should the user want that.

Then, define a constructor function for that record, which takes as argument a map or vector defining your routes, and returns an instance of the above record. That record should have a :uri property, and encapsulate the actual server.

Then you can do stuff like:

(with-open [srv (fake-server routes)]

  (http/get (:uri srv) ...)

  ...)

while leaving the option of creating and closing the server manually if it makes sense for some use-case. You can also easily reuse routes from one server to another as it is a simple data structure.

[Johan Haleby]

Very interesting approach indeed. I'm going to finish up the previous approach first then I'll look more closely into this. I like it, and its simple!

Thanks!

[Johan Haleby]

On Wed, Mar 9, 2016 at 7:32 PM, Marc Limotte <msli...@gmail.com> wrote:

With the macro approach, they don't need to escape it.

Do you know of any resources of where I can read up on this? I have the macro working with an implicit "uri" generated but I don't know how to make it explicit (i.e. defined by the user) the way you proposed.

[Marc Limotte]

Look at the source for the clojure.core with-open macro.  In the repl: `(source with-open)`.

I think Gary is right.  with-open does exactly what you need, I should have thought of that, and you should probably use it.  But if you want to get your version working, trying to understand what the with-open macro is doing.  Your implementation can be simpler because you only have one explicit binding.  Essentially you'll create a let as a backquoted form and then splice in the explicit symbol from the user: 

   `(let [~sym ...server-instance-or-uri...] ... )

marc

[Johan Haleby]

Thanks a lot for your support and insights. I'm going to rewrite it to use "with-open" as we speak.

[Timothy Baldridge]

"Idiomatic" is always a hard word to define, and I think some of the points made here are good, but let me also provide a few guidelines I try to abide by when writing an API:

Start with data, preferably hash maps. At some point your API will be consumed by someone else's program. Macros make it hard to compose api calls in a sane matter using code. So stick with hash maps and pure data. Something like the following:

{:host "foo.bar.com"

 :port 80

 :path "/some/path/i/want"

 :params {:name :value :key :value2}}

Now if it comes time to modify/process/compose this request we can use normal Clojure functions like assoc/conj to build this request map. Of course, using this approach normally results in a explosion of data, so pretty it up with helper functions: 

(make-request-map "http://foo.bar.com/some/path/i/want" {:name :value})

The key here, is that these helper functions should emit the data you specified in the first step. 

And finally, write macros as a last resort to pretty up the user experience even further. In short:

1) Start with data to allow clojure code to easily access your API

2) Make generating that data simpler by writing helper functions to generate data

3) (Optionally) Write a DSL to make user interaction easier. 

Timothy

“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[Johan Haleby]

Thanks everyone for your input. I just wanted to say that I have an initial version of the library available at my github page if anyone is interested.