A macro for flexible keyword argument handling

[Constantine Vetoshev]

A couple of days ago, I finally had enough of manually extracting
function keyword arguments. defnk is cool and all, but it does nothing
for fn, letfn, defmethod, or any other form with a parameter list. Map
destructuring is also cool, but providing defaults requires writing
each symbol twice, once in :keys and once in :or.

The macro I ended up with, fn-keywords, owes much to Common Lisp's
lambda lists. Instead of an outer wrapper for Clojure's various
function forms, it provides a kind of inner wrapper, really a let form
specialized for keyword arguments. Each keyword is defined in a
keyword spec, which allows for an optional default value, and for an
optional symbol to be bound to true if the keyword was actually
provided in the function call.

Note that keywords arguments may be & rest in style, or provided as
either maps or vectors. This should cover all useful cases for keyword
arguments, including their use in functions with multiple arities.


Example:

(defn test1 [req1 req2 & kw-args]
(fn-keywords [:kw1
[:kw2 "kw2 default"]
[:kw3 "kw3 default" kw3-supplied?]]
kw-args
;; function body
{:req1 req1 :req2 req2 :kw1 kw1 :kw2 kw2
:kw3 kw3 :kw3-supplied? kw3-supplied?}))

> (test1 1 2 :kw3 "there")
{:req1 1, :req2 2, :kw1 nil, :kw2 "kw2 default",
:kw3 "there", :kw3-supplied? true}
> (test1 1 2 :kw1 "there")
{:req1 1, :req2 2, :kw1 "there", :kw2 "kw2 default",
:kw3 "kw3 default", :kw3-supplied? false}
> (test1 1 2 :kw1 "hello" :kw2 "there")
{:req1 1, :req2 2, :kw1 "hello", :kw2 "there",
:kw3 "kw3 default", :kw3-supplied? false}


Macro code:

(defmacro fn-keywords
"Adds flexible keyword handling to any form which has a parameter
list: fn,
defn, defmethod, letfn, and others. Keywords may be passed to the
surrounding
form as & rest arguments, lists, or maps. Lists or maps must be
used for
functions with multiple arities if more than one arity has keyword
parameters. Keywords are bound inside fn-keywords as symbols, with
default
values either specified in the keyword spec or nil. Keyword specs
may consist
of just the bare keyword, which defaults to nil, or may have the
general form
[:keyword-name keyword-default-value* keyword-bound?*]. keyword-
bound? is an
optional symbol bound to true if the keyword was supplied, and to
false
otherwise."
[kw-spec-raw kw-args & body]
(let [kw-spec (map #(if (sequential? %) % [%]) kw-spec-raw)
keywords (map first kw-spec)
symbols (map (comp symbol name) keywords)
defaults (map second kw-spec)
destrmap {:keys (vec symbols)
:or (apply hash-map (interleave symbols defaults))}
supplied (reduce (fn [m [k v]] (assoc m k v)) (sorted-map)
(remove (fn [[_ val]] (nil? val))
(partition 2
(interleave keywords
(map (comp
second rest)
kw-
spec)))))
kw-args-map (gensym)]
`(let [kw-args# ~kw-args
~kw-args-map (if (map? kw-args#) kw-args# (apply hash-map
kw-args#))
~destrmap ~kw-args-map]
~@(if (empty? supplied)
body
`((apply (fn [~@(vals supplied)]
~@body)
(map (fn [x#] (contains? ~kw-args-map x#))
[~@(keys supplied)])))))))


More examples:

(defmulti test2 (fn [x & rest] (class x)))

(defmethod test2 java.lang.Integer [req & kw-args]
(fn-keywords [[:kw1 "default" kw1-supplied?]] kw-args
{:method "integer"
:req req
:kw1 kw1
:kw1-supplied? kw1-supplied?}))

(defmethod test2 java.lang.String [req & kw-args]
(fn-keywords [[:kw1 "default"]] kw-args
{:method "string"
:req req
:kw1 kw1}))

(defn test3
([req1 kw-args]
(fn-keywords [:kw1 :kw2] kw-args
{:req1 req1 :kw1 kw1 :kw2 kw2}))
([req1 req2 kw-args]
(fn-keywords [:kw1 :kw2] kw-args
{:req1 req1 :req2 req2 :kw1 kw1 :kw2 kw2})))

(defn test4 []
(letfn [(inner [& kw-args]
(fn-keywords [:kw1 :kw2] kw-args
{:kw1 kw1 :kw2 kw2}))]
(inner :kw1 "hello" :kw2 "world")))


Credit where it's due: I lifted ideas and bits of code from defnk.
Thanks, Meikel!

Comments welcome.

--
Constantine Vetoshev

[Constantine Vetoshev]

Looks like Google Groups posting software forced line wraps at less
than 80 columns, breaking the code. Lovely. Here's the fn-keywords
macro reformatted for 72 column wrapping.

(defmacro fn-keywords
"Adds flexible keyword handling to any form which has a
parameter list: fn, defn, defmethod, letfn, and
others. Keywords may be passed to the surrounding form as &
rest arguments, lists, or maps. Lists or maps must be used for
functions with multiple arities if more than one arity has
keyword parameters. Keywords are bound inside fn-keywords as
symbols, with default values either specified in the keyword
spec or nil. Keyword specs may consist of just the bare
keyword, which defaults to nil, or may have the general form
[:keyword-name keyword-default-value*

keyword-bound?*]. keyword-bound? is an optional symbol bound

[Laurent PETIT]

Really cool !

Some ideas which may (or may not ?) enhance it even further :

Since it creates new local bindings, maybe make it look more like
other binding forms :

instead of (fn-keyword [kw spec] init-kw-val body) :
(let-keywords [ [kw spec] init-kw-val ] body )

?

or (let-kw) for short ?

Indeed the name (fn-keyword) suggests to me that the expansion will
generate some (fn ...) structure.

Overall, pretty cool, thanks for sharing !

This deserves to be placed in contrib, in my opinion.

Cheers,

--
Laurent

2009/11/16 Constantine Vetoshev <gepa...@gmail.com>:

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en

[Meikel Brandmeyer]

Hi,

On Nov 16, 10:24 am, Laurent PETIT <laurent.pe...@gmail.com> wrote:

> This deserves to be placed in contrib, in my opinion.

In fact it could be used to drive defnk. So defnk could be retained as
a convenience (and for backward compatibility) expanding to (defn ...
(fn-keyword ...)).

+1 to some different name, though. eg. with-keyword-arguments

More random notes:

(apply hash-map (interleave key-seq val-seq)) can be written as
(zipmap key-seq val-seq).

I'm not sure I like the clever interface :kw or [:kw default] or [:kw
default bound?]. Maybe it is trying to do too much? Is bound? really
thaaat interesting?

Is it interesting to allow also map for kw-args?

Sincerely
Meikel

[Timothy Pratley]

On Nov 16, 8:24 pm, Laurent PETIT <laurent.pe...@gmail.com> wrote:
> Really cool !

Yeah, neat :)

> or (let-kw) for short ?

I like that name better

> This deserves to be placed in contrib, in my opinion.

Yes please!


Nice macro Constantine :)

[Constantine Vetoshev]

1. Looks like everyone prefers the let-kw name. Sounds good to me.

2. I tried it with the more let-like form, but I don't like the number
of opening braces required. For a full kw-spec, the form would end up
(let-kw [[[:kw default supplied?]] kw-args] ...). Clojure tends to err
on the side of fewer braces and parentheses, and three opening braces
looks excessive to me. That said, I see the point of keeping it
consistent with the other binding forms.

3. Unnecessary interleave replaced with zipmap.

4. With regard to the generality of the interface: I put all those
features in there because I have wanted to use them. I often find
myself writing or wrapping API-like code, and slicing and dicing
arguments and passing them through to other functions, as vectors or
maps, has been useful.

5. As for putting the macro in clojure-contrib, thanks for the vote of
confidence. :) I'll drop a signed Contributor Agreement in the mail
today, and get an Assembla ticket and patch out soon.

Thanks for all the feedback and suggestions. New version:

(defmacro let-kw

"Adds flexible keyword handling to any form which has a parameter
list: fn, defn, defmethod, letfn, and others. Keywords may be
passed
to the surrounding form as & rest arguments, lists, or maps. Lists
or
maps must be used for functions with multiple arities if more than

one arity has keyword parameters. Keywords are bound inside let-kw

as
symbols, with default values either specified in the keyword spec
or
nil. Keyword specs may consist of just the bare keyword, which
defaults to nil, or may have the general form [:keyword-name

keyword-default-value* keyword-supplied?*]. keyword-supplied? is

an
optional symbol bound to true if the keyword was supplied, and to
false otherwise."
[kw-spec-raw kw-args & body]
(let [kw-spec (map #(if (sequential? %) % [%]) kw-spec-raw)
keywords (map first kw-spec)
symbols (map (comp symbol name) keywords)
defaults (map second kw-spec)

destrmap {:keys (vec symbols) :or (zipmap symbols defaults)}

[Chouser]

On Mon, Nov 16, 2009 at 12:32 PM, Constantine Vetoshev
<gepa...@gmail.com> wrote:
>
> 2. I tried it with the more let-like form, but I don't like the number
> of opening braces required. For a full kw-spec, the form would end up
> (let-kw [[[:kw default supplied?]] kw-args] ...). Clojure tends to err
> on the side of fewer braces and parentheses, and three opening braces
> looks excessive to me. That said, I see the point of keeping it
> consistent with the other binding forms.

'let' is also careful to always have the user provide the symbol
being bound as a symbol. That is you say {:keys [a b c]} not
{:keys [:a :b :c]}. What do you think of having let-kw take
symbols instead of keywords for the names being bound?

--Chouser

[Constantine Vetoshev]

On Nov 16, 4:28 pm, Chouser <chou...@gmail.com> wrote:
> 'let' is also careful to always have the user provide the symbol
> being bound as a symbol.  That is you say {:keys [a b c]} not
> {:keys [:a :b :c]}.  What do you think of having let-kw take
> symbols instead of keywords for the names being bound?

I think that's a good idea.

Updated version:

(defmacro let-kw
"Adds flexible keyword handling to any form which has a parameter
list: fn, defn, defmethod, letfn, and others. Keywords may be
passed to the surrounding form as & rest arguments, lists, or
maps. Lists or maps must be used for functions with multiple
arities if more than one arity has keyword parameters. Keywords are
bound inside let-kw as symbols, with default values either
specified in the keyword spec or nil. Keyword specs may consist of

just the bare keyword symbol, which defaults to nil, or may have

the general form [keyword-name keyword-default-value*
keyword-supplied?*]. keyword-supplied? is an optional symbol
bound to true if the keyword was supplied, and to false otherwise."
[kw-spec-raw kw-args & body]
(let [kw-spec (map #(if (sequential? %) % [%]) kw-spec-raw)

symbols (map first kw-spec)
keywords (map (comp keyword name) symbols)

[nchubrich]

Suppose one went for broke and made a variable binding form that was
1) Maximally permissive
and
2) Notationally minimal
Without saying whether this is \advisable, we might agree it would be
an interesting exercise.
It might look like this:
\All arguments are potentially keywords; and since you can't bind a
value to anything but a symbol, anything in the argument list that is
not a symbol is interpreted as the default value of the preceding
symbol. So if you wrote
(dfnsk demo1 [x y 1 z u ~x] ..)
it could be called either by (demo1 1 2 3 4) or something like
(demo1 :z 3 :x 2). (If you left the arguments incomplete and
unkeyworded it would apply what you put preferentially to the first
arguments in the list without a default value. Thus (demo1 1) would
specify x and (demo1 1 2) would specify x and z.)
There needs to be another way of specifying defaults, because you
may want to apply the same defaults to many different values. So at
the end of the list, after any &rest arguments, you could put any
number of triplet vectors like [0 5 nil] or [p w 10]; where the last
element is the default, and the first two are the range for the
default to be applied to, specified either as a position or a variable
name.
When you mix keyword arguments and positional, there could be two
potential interpretations. Consider
(dfnsk demo2 [p q r s t u v w x y z [p z 0]] ...) If you called it
like (demo2 :u 1 2 4 7), you could mean either 1) skip ahead in the
list to u and fill the rest of the arguments from there or 2)
'consume' u from the list and then continue to call the list as
ordered with a gap at u. So you would need an additional notation
such as (demo2 :skip :u 1 2 4 7), but I'm not sure how to do that
without potential name conflicts.
Finally, you would have a set of predicates that tell you 1) how
many arguments have been supplied and 2) whether a given argument has
been supplied (as Constantine has done). You could make your defaults
if expressions or cond expressions using these predicates (and
whatever else you might want to pend on). You could say, "if x has
not been supplied, make y 2, etc." This could be an alternative (and
I guess more general) way of notating variable arity argument lists.
. I'm not trying to start an argument over whether this would be
too complicated or tricky to use in practice (though this is a good
argument to have too); my immediate question is, can it be any more
general or minimal than this?
Maybe there is some really neat way of encoding arbitrary
preference rules and priorities in an argument list. It's not
occurring to me though.
In any case, the 'complication' is mostly an artifact of the way we
program. If we move a little away from a pure program text and
nothing-but while programming (as IDE's effectively do), we get all
sorts of feedback and help on argument lists, so that it becomes in
part like filling out a form when that helps; and hopefully this
feedback ends up being not too intrusive or distracting. That,
however, is a question for the \en\clojure list.....

[nchubrich]

(preceding message, re-formatted)

Suppose one went for broke and made a variable binding form that was

1) Maximally permissive
and
2) Notationally minimal

Without saying whether this is \advisable, we might agree it would be
an interesting exercise.

It might look like this:

\All arguments are potentially keywords; and since you can't bind a
value to anything but a symbol, anything in the argument list that is
not a symbol is interpreted as the default value of the preceding
symbol. So if you wrote

(dfnsk demo1 [x y 1 z u ~x] ..)

it could be called either by

(demo1 1 2 3 4)

or something like

(demo1 :z 3 :x 2)

(If you left the arguments incomplete and unkeyworded it would apply
what you put preferentially to the first arguments in the list
without
a default value. Thus (demo1 1) would specify x and (demo1 1 2) would
specify x and z.)

There needs to be another way of specifying defaults, because you
may want to apply the same defaults to many different values. So at
the end of the list, after any &rest arguments, you could put any
number of triplet vectors like [0 5 nil] or [p w 10]; where the last
element is the default, and the first two are the range for the
default to be applied to, specified either as a position or a variable
name.

When you mix keyword arguments and positional, there could be two
potential interpretations. Consider

(dfnsk demo2 [p q r s t u v w x y z [p z 0]] ...)

If you called it like

(demo2 :u 1 2 4 7)

you could mean either 1) skip ahead in the list to u and fill the
rest of the arguments from there or 2) 'consume' u from the list and
then continue to call the list as ordered with a gap at u. So you
would need an additional notation
such as

(demo2 :skip :u 1 2 4 7)

[Constantine Vetoshev]

On Nov 17, 11:33 pm, nchubrich <nicholas.chubr...@gmail.com> wrote:
> can it be any more
> general or minimal than this?

Seems to me that your suggestion effectively makes every positional
argument optional. You did note that the scheme calls for supplied-
predicates, but this pushes responsibility for checking required
arguments from the compiler to the programmer. That seems a little too
minimal to me, and barely an improvement from extracting everything
from an "arguments" array.

[nchubrich]

I don't see why you couldn't simply check to make sure that there are
no arguments without either default or supplied values. As I wrote
above, "If you left the arguments incomplete and unkeyworded it would

apply what you put preferentially to the first arguments in the list

without a default value." If there are any null arguments left over,
the macro could throw an exception.

The basic rule is, "accept what you can, and only error our if you
absolutely must."

(Of course this variable application rule might get confusing in
practice, at least without help from the IDE.)

I think it's better than just a flat arguments list, because 1) it
allows you to specify defaults and 2) it gives you the option of
naming arguments by keyword. How can you do that with flat argument
lists?

My point is that the option of naming arguments is implicitly there
any time you write a function, because variables have names; and there
is no way to mistake a default value for a variable. So you might as
well use these built-in distinctors.

You could even do a little branching from within argument lists.
Suppose for instance you had something like

(dfnsk demo-branch [x 0 0 1 y 0 1 0] ....)

There can be only one default per variable, so what are the other
values? These could only be possible passed-in values. x=0, y=1 is
the first branch, x=1, y=0 is the second. Within the function body,
you would have a predicate available that tells you what branch you
are on, so for instance you could write something like

(cond (branch? 1) ...
(branch? 2) ...)

or, if you passed in multiple function bodies after the argument list,
(kind of like the variable arity thing), the first would be for the
first branch, the second for the second, etc.

Another 'generality' that could be added is to match arguments by type
whenever there is a type annotation in the function definition and a
discrepancy in the call; so for instance if you had a number of
symbols that were supposed to be passed last, and you passed a symbol
somewhere else, it would be applied to the first symbol-type argument
in the function list without throwing an error.

[nchubrich]

I should also add something I alluded to in another discussion (under
'positions'); this is the idea of making \any parameter into a rest
parameter. For instance, if you had

(defnsk add [addend augend] ...)

you could call it like (add :addend 1 2 3 :augend 1 2) or (add [1 2 3]
[1 2 3 4]).

Must the poor programmer now check \all of his parameters to see if
they have been magically turned into rest parameters without his
knowing? Of course not! The function body is simply executed
repeatedly, and the results put in a list:

(add [1 2 3] [1 2 3 4]) -> [2 4 6]

If we were being \really permissive, we might want to return \two
results, one the straightforward element-by-element adding, the other
the 'cartesian' adding: [[2 4 6],[[2 3 4 5][3 4 5 6][4 5 6 7]]]. Or
we might allow a flag passed to all functions :cartesian :sequential
to turn this behavior on or off.

If function writers want to explicitly manipulate these sequences, of
course, that is fine; all I am saying is that if they do nothing, the
function \caller gets his sequence code for free.

To assist this way of programming, it might be a good idea to have
another type of 'permissive' sequence abstraction, which you might
call for old time's sake car and cdr:

(car [1 2 3]) -> 1
(cdr [1 2 3]) -> [2 3]
(car 1) -> 1
(cdr 1) -> nil

A direct reference to a sequence as if it were a singular element,
meanwhile, is an invitation to deal with the sequence at the level of
the function caller as we do above (who if he passes elements as a
sequence, presumably knows what he is doing).

I guess this is getting to be a pretty epic macro! I figured it was
worth inviting boos and/or cheers and suggestions before setting out...

[Constantine Vetoshev]

On Nov 20, 8:33 am, nchubrich <nicholas.chubr...@gmail.com> wrote:
> I guess this is getting to be a pretty epic macro!  I figured it was
> worth inviting boos and/or cheers and suggestions before setting out...

Far be it from me to discourage making function invocation semantics
more flexible! Just one thing: you've been calling your macro
"defnsk", which makes it sound like a wrapper for the def form. Don't
leave defmethod, fn, letfn, and every other way of making a function
in the dust. :)

[nchubrich]

No, I'd rather not, but I have to take this in bite-size pieces. I'm
still a bit of a noob at writing macros. I want to make it so that
the user does not have to do anything aside from deal with the
argument lists directly, so that means writing different macros for
defn, letfn, defmethod, etc. Might as well try this on for size \one
way first, and if nobody (including me) likes it, then there's no
reason to continue the crusade....