Augmenting the defn attr-map

[tsuraan]

I'd like to add a :signature entry to the attr-map of defn, that
provides a haskell-style type signature to functions of the
single-arglist-body form. I find the the normal way of providing
hints to a function:

(defn [ #^Class1 var1 #^Class2 var2 #^Class3 var3] ... )

is way too noisy, and the variables get lost between the types. What
I'd like instead is to be able to specify the signature in the
function metadata:

(defn {:signature [ Class1 Class2 Class3 ReturnType ]}
[ var1 var2 var3 ]
...)

I've written a little patch to defn that annotates the arguments
correctly (I think). I'm not sure where the metadata for marking up a
function's return type goes (Is it on the (cons `fn fdecl) part?), but
it probably wouldn't be hard if I knew what I was doing :)

Anyhow, a revised defn (I've called it defn+ to avoid clashes) is as follows:

(defn zip [ a b ] (map vector a b))
(def

#^{:doc "Same as (def name (fn [params* ] exprs*)) or (def
name (fn ([params* ] exprs*)+)) with any doc-string or attrs added
to the var metadata"
:arglists '([name doc-string? attr-map? [params*] body]
[name doc-string? attr-map? ([params*] body)+ attr-map?])}
defn+ (fn defn+ [name & fdecl]
(let [m (if (string? (first fdecl))
{:doc (first fdecl)}
{})
fdecl (if (string? (first fdecl))
(next fdecl)
fdecl)
m (if (map? (first fdecl))
(conj m (first fdecl))
m)
fdecl (if (map? (first fdecl))
(next fdecl)
fdecl)
fdecl (if (vector? (first fdecl))
(if (:signature m)
; we'll apply the types in the signature vector as the
; :tag metadata for each argument
(list
(cons
(apply vector
(for [[ tag var ]
(zip (:signature m) (first fdecl))]
(with-meta var {:tag tag})))
(rest fdecl)))
(list fdecl))
fdecl)
m (if (map? (last fdecl))
(conj m (last fdecl))
m)
fdecl (if (map? (last fdecl))
(butlast fdecl)
fdecl)
m (conj {:arglists (list 'quote (sigs fdecl))} m)]
(list 'def (with-meta name (conj (if (meta name) (meta name) {}) m))
(cons `fn fdecl)))))

(. (var defn+) (setMacro))

Is this something that people would think is worthwhile? I really
prefer haskell's type signatures to clojure's inline hints, but maybe
it's just something that people get used to. Anyhow, my first attempt
at macro hacking, so kind criticism would also be welcome w.r.t.
style, correctness, etc.

[Chas Emerick]

This is an interesting path to take. I'm not at all familiar with
haskell, but a couple of thoughts anyway :-) :

- There's already a lot of moving parts to type hinting, so adding
this optional approach into defn seems like it'd lead to unintended
consequences. That said, there's absolutely nothing wrong with an
alternative def form (defh? as in 'define hinted fn') -- there's
plenty of them throughout contrib and elsewhere.

- Remember destructuring, and along with that, one of the nice thing
about in-place hints, as opposed to a separate definition of expected
types, is that the definitions can be sparse, e.g.:

(defn foo [{blah :foo #^MyType mt-obj :bar} a b c] ...)

It doesn't look like your macro supports hinting destructured args
(understandably enough, doing so given the :signatures approach would
likely be very difficult).

- The thing I don't like about the current type hints is (IMO) the
distracting quality of '#^'. What about something like:

(defh foo [String:s unhinted-arg {int:a :a}] ...) or
(defh foo [s:String unhinted-arg {a:int :a}] ...)

That's far more visually-appealing to me, and has the nice advantages
of being inline in the case of destructuring and supporting sparse
hinting easily. I'll bet the macro would end up being pretty simple,
as well.

Cheers,

- Chas

[tsuraan]

> - There's already a lot of moving parts to type hinting, so adding
> this optional approach into defn seems like it'd lead to unintended
> consequences. That said, there's absolutely nothing wrong with an
> alternative def form (defh? as in 'define hinted fn') -- there's
> plenty of them throughout contrib and elsewhere.

Yeah, I tried to slip my change into a place in defn that probably
wouldn't hurt anything, but I'm still not sure what the full
consequences of my change are. I'm far from a clojure expert :)

> - Remember destructuring, and along with that, one of the nice thing
> about in-place hints, as opposed to a separate definition of expected
> types, is that the definitions can be sparse, e.g.:
>
> (defn foo [{blah :foo #^MyType mt-obj :bar} a b c] ...)
>
> It doesn't look like your macro supports hinting destructured args
> (understandably enough, doing so given the :signatures approach would
> likely be very difficult).

I didn't put it in yet, but I was thinking of just having nil in the
type vector for unhinted variables, so you could have

(defn foo {:signature [ String String nil ]} [ a b c ] ...)

and then c wouldn't be hinted. I hadn't thought of destructuring at
all; I'm guessing that it could be done with

(defn foo {:signature [{:bar Mytype} nil nil nil]} [{blah :foo mt-obj

:bar} a b c] ...)

but that's getting pretty ugly on its own. I'm not sure if it would
be a win to try to do anything fancy like that. I'm also not sure
what the destructuring assignment syntax is for maps right now, so
what I wrote might be total nonsense, syntactically. I hope the idea
is clear, anyhow.

> - The thing I don't like about the current type hints is (IMO) the
> distracting quality of '#^'. What about something like:
>
> (defh foo [String:s unhinted-arg {int:a :a}] ...) or
> (defh foo [s:String unhinted-arg {a:int :a}] ...)
>
> That's far more visually-appealing to me, and has the nice advantages
> of being inline in the case of destructuring and supporting sparse
> hinting easily. I'll bet the macro would end up being pretty simple,
> as well.

I'd hate to see somebody do it, but it's currently valid to define
variables with colons in them, so there could be code out in the wild
with those definition forms already. Other than that, I like the
looks of it. I think the macro would be easy as well, so long as
you're comfortable munging variable names :)

[Mark Volkmann]

I didn't release it was valid to define names with colons in them.
Maybe that shouldn't be allowed. I'd like to be able to specify type
hints using UML-like syntax like the second example from Chas which
was

(defh foo [s:String unhinted-arg {a:int :a}] ...)

--
R. Mark Volkmann
Object Computing, Inc.

[Chas Emerick]

On Aug 12, 2009, at 2:02 PM, tsuraan wrote:

> I didn't put it in yet, but I was thinking of just having nil in the
> type vector for unhinted variables, so you could have
>
> (defn foo {:signature [ String String nil ]} [ a b c ] ...)
>
> and then c wouldn't be hinted. I hadn't thought of destructuring at
> all; I'm guessing that it could be done with

I'd think _ would be more idiomatic for those that like the haskell-
esque style...?

> (defn foo {:signature [{:bar Mytype} nil nil nil]} [{blah :foo mt-obj
> :bar} a b c] ...)
>
> but that's getting pretty ugly on its own. I'm not sure if it would
> be a win to try to do anything fancy like that. I'm also not sure
> what the destructuring assignment syntax is for maps right now, so
> what I wrote might be total nonsense, syntactically. I hope the idea
> is clear, anyhow.

Yeah, I think supporting a parallel argument specification is probably
not tenable. On the other hand, performing a relatively simple
transformation to a set of destructuring forms is probably pretty
straightforward -- I don't even think :as, :keys, etc would require
any special treatment.

>> - The thing I don't like about the current type hints is (IMO) the
>> distracting quality of '#^'. What about something like:
>>
>> (defh foo [String:s unhinted-arg {int:a :a}] ...) or
>> (defh foo [s:String unhinted-arg {a:int :a}] ...)
>>
>> That's far more visually-appealing to me, and has the nice advantages
>> of being inline in the case of destructuring and supporting sparse
>> hinting easily. I'll bet the macro would end up being pretty simple,
>> as well.
>
> I'd hate to see somebody do it, but it's currently valid to define
> variables with colons in them, so there could be code out in the wild
> with those definition forms already. Other than that, I like the
> looks of it. I think the macro would be easy as well, so long as
> you're comfortable munging variable names :)

Well, that's one benefit/motivation to just have another def form,
rather than trying to make defn support everyone's preferred variation/
style.

I knocked out a quickie implementation that appears to work with the
couple of use cases I've thrown at it so far (pasted at the end of
this msg).

Just a little exposition:

com.snowtide.clojure.utils=> (binding [*print-meta* true]
(prn (macroexpand '(deft a [Long:a b
{[Integer:c] :c} {:keys [Double:d]}] (+ a b c d)))))
(def #^{:arglists (quote ([#^Long a b {[#^Integer c] :c} {:keys
[#^Double d]}]))} a (clojure.core/fn ([#^Long a b {[#^Integer c] :c}
{:keys [#^Double d]}] #^{:line 159} (+ a b c d))))

I *think* I've gotten the :tag metadata right (which I've burned
myself on before, e.g. putting Class objects in :tag slots instead of
symbols, etc), as this fn compiles without reflection warnings:

(deft a
[String:b [Double:c :as java.util.List:list] {java.util.Random:d :d}]
(.toCharArray b)
(.size list)
(.floatValue c)
(.nextInt d))

A couple of nice things about this:

- it doesn't interfere with 'normal' type hints (e.g. #^Foo and Foo:b
can coexist in the same declaration if you want to be silly about
things)

- I think this kind of hinting could be bolted right into the existing
destructuring functionality if Rich were so inclined (though I
wouldn't bet on that). This would allow you to use Foo:bar hints in
any binding-form (e.g. let, fn, for, etc etc).

Cheers,

- Chas

-----------
(defn- decorate-hinted-symbol
[sym]
(let [[type arg] (.split (name sym) ":")]
(if arg
(with-meta (symbol arg) {:tag (symbol type)})
sym)))

(defn- decorate-hinted-args
[args]
(cond
(vector? args) (vec (map decorate-hinted-args args))
(map? args) (into {} (map decorate-hinted-args args))
(symbol? args) (decorate-hinted-symbol args)
(keyword? args) args
:else (throw (Exception. (str args)))))

(defmacro deft
[name args & body]
`(defn ~name
~(decorate-hinted-args args)
~@body))

[Chas Emerick]

On Aug 12, 2009, at 3:19 PM, Mark Volkmann wrote:

> I didn't release it was valid to define names with colons in them.
> Maybe that shouldn't be allowed. I'd like to be able to specify type
> hints using UML-like syntax like the second example from Chas which
> was
>
> (defh foo [s:String unhinted-arg {a:int :a}] ...)

I *really* like the look of s:String (or String:s, which I used in my
quickie impl, they're interchangable), but I can easily see colons
being used for some other syntactic purpose down the road that we're
not yet aware of (ranges come to mind, eg. [0:50] as in python). And
of course, colons and other special characters are very helpful in
representing domain-specific semantics directly in code, so losing
those special characters is an expensive choice that should be weighed
very cautiously.

- Chas

[Lauri Pesonen]

2009/8/12 Chas Emerick <ceme...@snowtide.com>:

One possible hack would be to mimic Haskell once again and use a
double colon as the separator between the name and the type, i.e.
s::String. This would still allow for the use of a single colon (and
more than two colons) in normal names, albeit it might be a bit
confusing if both were used in the same context.

I think that special characters in names make code a lot more
readable, and they are useful in DSLs, so I too would hate to see
characters being removed from the set of legal characters.

Actually, it turns out that the reader does not like more than one
colon in the middle (or end) at the moment. This is probably somehow
related to how the namespace qualified keywords are read:

user> 'foo:bar
foo:bar
user> 'foo::bar
; Evaluation aborted.
user> 'foo:::bar
; Evaluation aborted.
user> 'foo:
; Evaluation aborted.
user> 'foo::
; Evaluation aborted.
user> ':foo
:foo
user> '::foo
:user/foo
user> 'foo????
foo????

> - Chas

--
! Lauri

[Chas Emerick]

On Aug 12, 2009, at 3:24 PM, Chas Emerick wrote:

>> I'd hate to see somebody do it, but it's currently valid to define
>> variables with colons in them, so there could be code out in the wild
>> with those definition forms already. Other than that, I like the
>> looks of it. I think the macro would be easy as well, so long as
>> you're comfortable munging variable names :)
>
> Well, that's one benefit/motivation to just have another def form,
> rather than trying to make defn support everyone's preferred
> variation/
> style.
>
> I knocked out a quickie implementation that appears to work with the
> couple of use cases I've thrown at it so far (pasted at the end of
> this msg).

I've been fiddling with this, and produced a slight variant of my
original macro that uses Type:arg-name symbols instead of arg-
name:Type symbols. Example:

(defh a
[b:String [c:Double :as list:java.util.List] {d:java.util.Random :d}]

(.toCharArray b)
(.size list)
(.floatValue c)
(.nextInt d))

I've actually been using it at the REPL here and there, and I've found
it pretty pleasant -- it's a very pretty way to hint args compared to
#^, and arg:Type ordering makes for easy readability (e.g. you can
very easily scan an arg form and see what the names of the args are,
as opposed to "actively" parsing them due to #^ hints or scanning for
the (relatively unobtrusive) colon in the Type:arg style).

Code below.

Cheers,

- Chas

;;;;;;;;;;

(defn- decorate-hinted-symbol
[sym]
(let [[arg type] (.split (name sym) ":")]
(if type

(with-meta (symbol arg) {:tag (symbol type)})
sym)))

(defn- decorate-hinted-args
[args]
(cond
(vector? args) (vec (map decorate-hinted-args args))
(map? args) (into {} (map decorate-hinted-args args))
(symbol? args) (decorate-hinted-symbol args)
(keyword? args) args
:else (throw (Exception. (str args)))))

(defmacro defh

[Stephen C. Gilardi]

On Aug 13, 2009, at 4:52 PM, Chas Emerick wrote:

I've actually been using it at the REPL here and there, and I've found  
it pretty pleasant -- it's a very pretty way to hint args compared to  
#^, and arg:Type ordering makes for easy readability (e.g. you can  
very easily scan an arg form and see what the names of the args are,  
as opposed to "actively" parsing them due to #^ hints or scanning for  
the (relatively unobtrusive) colon in the Type:arg style).

Code below.

Very cool! I like this a lot. Would you be up for writing some doc strings and including it in clojure.contrib.def? 

--Steve

[Chas Emerick]

Yeah, I could do that.

I don't actually think it's practical on its own, though.  My one concern is that using two hinting syntaxes in parallel (e.g. s:String in defh forms, but #^String s everywhere else) is, to put it lightly, painful.

What I might be able to do is define alternatively-hinted versions of let, for, doseq, etc., and rebind clojure.core/destructure during the evaluation of those macros so that the s:String style can get layered on top of binding forms (more) uniformly.

Feels hacky, though....thoughts?

- Chas

[Stephen C. Gilardi]

On Aug 16, 2009, at 7:38 PM, Chas Emerick wrote:

On Aug 16, 2009, at 6:29 PM, Stephen C. Gilardi wrote:

Very cool! I like this a lot. Would you be up for writing some doc strings and including it in clojure.contrib.def? 

Yeah, I could do that.

I don't actually think it's practical on its own, though.  My one concern is that using two hinting syntaxes in parallel (e.g. s:String in defh forms, but #^String s everywhere else) is, to put it lightly, painful.

Good point.

What I might be able to do is define alternatively-hinted versions of let, for, doseq, etc., and rebind clojure.core/destructure during the evaluation of those macros so that the s:String style can get layered on top of binding forms (more) uniformly.

Feels hacky, though....thoughts?

My current favorite is a gen-class'd alternative to clojure.main that calls "bindRoot" on clojure.core/destructure to replace it with your customized version and then calls clojure.main. This way we can play with this pervasively both in the repl and in loaded code with the only tweak being calling your customized entry point instead of clojure.main.

--Steve

[tsuraan]

> (defh a
> [b:String [c:Double :as list:java.util.List] {d:java.util.Random :d}]
> (.toCharArray b)
> (.size list)
> (.floatValue c)
> (.nextInt d))

What's that :as in the [ c:Double :as list:java.util.List ] vector?

[Stephen C. Gilardi]

The entire expression you quoted requests destructuring of a
sequential thing (such as a vector, seq, list, or something seq-able)
as the second argument to a. The :as clause allows you to give a name
to the entire sequential thing.

This desctruturing can be read as:

bind c to the first item in the sequential thing
bind list to the entire sequential thing

Here are a few examples:

user=> (let [[c :as b] [5 4 3 2 1]] (prn c b))
5 [5 4 3 2 1]
nil
user=> (let [[c m :as b] [5 4 3 2 1]] (prn c m b))
5 4 [5 4 3 2 1]
nil
user=> (let [[c m :as b] (java.util.ArrayList. [:a :b :c])] (prn c m
b))
:a :b #<ArrayList [:a, :b, :c]>
nil

The official docs for this are at: http://clojure.org/
special_forms#let .

(and (doc let) contains a pointer there)

--Steve

[Chas Emerick]

On Aug 16, 2009, at 9:27 PM, Stephen C. Gilardi wrote:

>> What I might be able to do is define alternatively-hinted versions
>> of let, for, doseq, etc., and rebind clojure.core/destructure
>> during the evaluation of those macros so that the s:String style
>> can get layered on top of binding forms (more) uniformly.
>>
>> Feels hacky, though....thoughts?
>
> My current favorite is a gen-class'd alternative to clojure.main
> that calls "bindRoot" on clojure.core/destructure to replace it with
> your customized version and then calls clojure.main. This way we can
> play with this pervasively both in the repl and in loaded code with
> the only tweak being calling your customized entry point instead of
> clojure.main.

OK, I'll take a look at that.

Actually, if this were to ever become an accepted default, the real
change would have to be in the reader, so as to enable call-site
hinting (e.g. (.toCharArray s:String). But, I'm definitely not going
to open that can of worms at the moment (and it wouldn't be as simple
and unobtrusive as the alternative entry point you suggest).

- Chas

[tsuraan]

> The official docs for this are at: http://clojure.org/
> special_forms#let .

That's a great link. Thanks!