Constructors with kv args

[Malcolm Sparks]

modular uses the kv-arg pattern for constructors

(new-web-server :port 8080)

there are good arguments for not doing this- it's tricky to apply these functions.

instead, it might be better to use option maps only

(new-web-server {:port 8080})

If I change the patten, I'd like to do so consistently (and once and for all).

I've heard statements to the effect that 'the community is moving away from kv args' but they seem popular in some places, like Prismatic's libraries.

Any opinions?

[James Henderson]

Hi Malcolm,

I prefer the option maps routes, mainly because it aids composability without trading off ease of calling. As you say, if your function is defined as follows:

(defn new-web-server [& {:keys [port] :as opts}]
  ...)

;; rather than

(defn new-web-server [{:keys [port] :as opts}]
  ...)

;; or even (if the options are themselves optional)

(defn new-web-server [& [{:keys [port] :as opts}]]
  ;; note the extra vector around the parameter destructuring
  ...)

then 'normal' callers (callers that build up the KVs entirely from scratch) call it with:

(new-web-server :port 8080)

;; rather than

(new-web-server {:port 8080})

(I actually think the latter is preferable - it's more obvious to the reader that the function expects a map of options for its final parameter, but I understand that's probably personal preference!)

For callers that are themselves passed a partial map of options (for example, if the calling function's purpose is to merge call-specific options with a set of application defaults), or callers that otherwise want to wrap the call somehow, having the function defined as accepting KVs rather than an option map means calling the function (albeit as a rather contrived example - I'm sure you can imagine similar code in your applications) as:

(defn start-web-server! [server-opts]
  (log "Starting server...")

  (start-server! (apply new-web-server
                        (apply concat (merge {:default :values
                                              ...}
                                             server-opts))))

  (log "Started server!"))

;; rather than

(defn start-web-server! [server-opts]
  (log "Starting server...")

  (start-server! (new-web-server (merge {:default :values
                                         ...}
                                        server-opts)))

  (log "Started server!"))

The behaviour of this function call - the fact that we're calling new-web-server, and that we're merging some default values into the map that we were passed - has been interposed with visual noise that (IMO) obscures what the function is doing, for the sole purpose of shoehorning the args into the format required by the callee. Granted, it's only an 'apply concat' and an 'apply', in this case, but I believe that even this is enough to detract from the meaning of the function if the reader is skimming through.

It might not seem like a big difference, but given how simple the option map alternative is (for both callers and callees), I think it'd be well worth standardising on the option map style.

Obviously, though, this is all very subjective, and just my 2¢ - would be interested to hear other views? 

James

[frozenlock]

I don't like kv-args. 

It's easy to generate a map and pass it as an argument to another function. 

If you go the kv-args way, you have to apply the map. Yuck.

[Adrian Mowat]

As a Ruby developer, I'm quite used to the kv args because they are built into the language but this thread is giving me pause for thought.  Are they just style over substance in Clojure.  I think they may be.

[Zack Maril]

I've spent the past day trying to understand how kv-args are used in modular and how I'm supposed to use them. I don't think kv-args get you much compared to the the cost of diving into understanding `make` and `make-args`.
-Zack

[Dylan Butman]

Having used modular for 3 production applications now. I have to disagree. There are multiple levels of default value specification that I use and each of them greatly contributes to the ease of development and deployment component systems.

constructor 

    - define default values for the component. This is project agnostic. components should define arguments in the most general terms. for example, a webserver should depend on :port, not :webserver-port

system 

    - define project defaults for components. here is where make shines is allowing you to define key substitutions related to your compiled arguments. for example {:port [:webserver :port]}

resources/project.edn 

   - define compile time defaults for components. for me, this often serves as a place to formalize all of the arguments used by the application. This also provides a chance to give structure to arguments. For example, you might have two :port args in your system for different http-listeners. Here you might define [:api :port] and [:webserver :port].

~/.project.edn

   - define environment specific arguments. this is incredible useful for deployment and multiple testing environments. when deploying, I document a step where project/resources/project.edn is copied to ~/.project.edn. This is so much easier for the layman (and for me) when developing and deploying on multiple playforms.

make and make-args' big contribution here is allowing configuration files to be far more complex than the components they provide configuration values for. Without being able to rename nested or long keys to simple values, you'd either limit the scope of configuration files and prevent system scalability, or you'd force components to adopt overly specific naming conventions, which complects environment to functionality.

Also, kv-args have the cool property of being idempotently partially appliable, where the rightmost value takes precedence, which has some cool uses for composing components with partially applied default arguments and then overriding a few arguments down the line. 

[Malcolm Sparks]

Zack: The question of whether to use kv-args or option maps is unrelated to the use of make and make-args, which Dylon rightly points out are intended to allow values specified in system.clj to be overridden in the config file. 

You don't have to use make and make-args at all, and I originally create make and make-args for my own experimental use. Somehow they found their way into the system.clj generation with 'lein new modular'. However, it's obviously not great that you had to spend so much time figuring them out. To be honest, they're not well documented, please accept my apologies for that. One of the aims of modular is to keep components agnostic from the code that assembles them. So if you find 'make' and 'make-args' useful to you in the assembly of your Clojure systems, great. If you don't want to use them, that's fine too. 

I do think though that the configuration mechanism used in the assembly of systems could be improved, retaining the ability to override system configuration with config files/data-sources, but not requiring your system.clj to have knowledge the structure of your config file.

Ideally you want things to be configurable even in situations when you've forgotten to make them so. For example, you might have hard-coded the port of the web server to be 3000, only to find that when you deploy onto your prod machine, someone else has already running another service on 3000 and you don't have the option of going back and modifying the original code (for political or other reasons).

One idea is for the config tree (sourced from a local file or elsewhere) to mirror the structure of the system tree. For example, it's quite possible with modular to have two Jetty web servers running on different ports. But each instance would have to have a unique key in the system map. If the config tree mirrors the system tree, it's then obvious which config setting affects which Jetty web server instance. Of course, you could have a more complex mapping between your configuration tree and the system tree, but the out-of-the-box case should be an easy one-to-one. And you could always process a config file from one map to another.

However, I'm keen that modular doesn't dictate to developers how to do configuration, when there are other really good approaches out there which should fit (env, nomad, etc.). Rather, new projects generated with 'lein new modular' should contain a basic default mechanism that can be swapped out or extended easily. I don't have anything in mind, just hoping to prompt further discussion in this group. Modular should focus on what it does, and stay out of areas best left to other libraries.

So, back to the topic of the thread: constructors with kv-args. I am sympathetic to the arguments of both camps. One compromise I'm considering of to support both approaches: If there's a single argument to a constructor, it's treated as an option map. If there's more that one, it's kv-args. It's just more work in the constructors, and I feel we should cater for users of components rather than their authors. While it's not always a good idea to try to keep everyone happy with compromises, I think this might work. Would they be any major drawbacks with this dual approach? For example, would it cause unforeseen problems and complexity in composition?