Functions and vars and meta-data

[Stefan Arentz]

Another one related to my previous question about meta-data.

user> (defn #^{ :xxx 1} foo [] "foo")
#'user/foo
user> (defn #^{ :xxx 2} bar [] "bar")
#'user/bar

I need to do something similar to this:

user> (map #(:xxx (meta %)) [foo bar])
(nil nil)

Basically accessing the meta data of a function of which I only know
the 'name'. (Because that is what my macro takes as one of its
parameters)

I know it works when I use the #' reader macro.

user> (map #(:xxx (meta %)) [#'foo #'bar])
(1 2)

But I'm using this in a bigger macro that takes a bunch of functions
as a parameter. Is there a way to make this work or should I
'translate' the functions that I take by name with (var foo)?

I must admin that I don't fully understand the difference between foo
and #'foo. That is probably why I'm making this beginner mistake :-)

S.

[John Harrop]

On Sat, Nov 7, 2009 at 8:04 PM, Stefan Arentz <ste...@arentz.ca> wrote:

But I'm using this in a bigger macro that takes a bunch of functions
as a parameter. Is there a way to make this work or should I
'translate' the functions that I take by name with (var foo)?

You'll need to translate the symbols into vars using resolve, I think. 

[Stefan Arentz]

Hmmm

user=> (var foo)
#'user/foo

user=> (meta (var foo))
{:ns #<Namespace user>, :name foo, :file "NO_SOURCE_FILE", :line
1, :arglists ([]), :xxx 1}

user=> ((var foo))
"foo"

I'll give this a try in my macro later today :-)

S.

[John Harrop]

Eh. `(var ~argument) should work. You'll need resolve if you ever want to do this in a function though, or use the var object in the macro body itself (as opposed to only using the var object in the code that results from the macro's expansion).

[Alex Osborne]

Stefan Arentz wrote:

> I must admin that I don't fully understand the difference between foo
> and #'foo. That is probably why I'm making this beginner mistake :-)

The difference takes some explanation. So without further ado...

Functions and Metadata: in Vivacious Gory Detail
================================================

There's three types of objects in play here: symbols, vars and functions:

(defn foo [])

(type foo)
=> user$foo_4703 (a class implementing IFn)

(type 'foo)
=> clojure.lang.Symbol

(type #'foo)
=> clojure.lang.Var

A symbol is just a name. A Var is an object that is named by a symbol
and "bound" to a value. Normally Clojure will "evaluate" symbols, for
example when you type this in the REPL:

foo
=> #<user$foo__4703 user$foo__4703@19f1a8a>

The way it does this is by first "resolving" the symbol foo in the
current namespace. A namespace is essentialy just a map from symbols to
vars. So after resolving, it then has a Var object. A Var, is as it's
name suggests, a variable. It consist of a name (symbol + namespace),
metadata map and a value (called the binding). There can actually be
multiple bindings (for thread-local variables and such) but normally
there is only one, the "root binding". So Clojure evaluates the var by
getting the value bound to it.

Now what if you ask Clojure to evaluate a vector of symbols? It
evaluates each symbol (first "resolving" to get a var and then taking
the binding) and gives you back a vector of function objects:

(def some-number 4)

[foo inc some-number]
=> [#<user$foo__4703 user$foo__4703@19f1a8a>
#<core$inc__4633 clojure.core$inc__4633@127e4be>
4]

Now macros. When Clojure sees something like this:

(bar foo inc some-number)

It will first resolve the symbol bar in the current namespace (in this
case the namespace is "user"). Remember resolving gives you a Var, so
in this case the Var #'user/foo. Clojure then looks at the metadata of
the Var to determine whether it is bound to a macro or a function.
Normally Clojure evaluates the arguments [foo inc some-number] producing
[#<user/foo> #<clojure.core/inc> 4] and then calls the binding value
(the actual function object) with them.

Alternatively if the var's metadata says it is bound to a macro, Clojure
doesn't evaluate the arguments. It just calls the binding with the
symbols 'foo 'inc and 'some-number. Clojure will then evaluate the
return value of the macro (usually another bunch of symbols and literals).

So when I write this:

(defmacro mymac [sym]
(println "symbol is:" sym)
sym)

(mymac foo)
=> symbol is: foo
#<user$foo__4703 user$foo__4703@19f1a8a>

Clojure passes mymac a symbol object. We print out the symbol "foo" and
then return it. Clojure then evaluates the return value (the symbol)
producing the actual function object that foo is bound to.

Now for metadata "on" functoins. When you write this:

(defn #^{:xxx 1} greet [] "hello")

The #^{...} syntax means that Clojure creates a list of two symbols
(defn and greet), and empty vector and a string "hello". The second
symbol greet has the metadata {:xxx 1} associated with it. This will be
macro-expanded into this:

(def #^{:xxx 1} greet (fn ([] "hello")))

The greet symbol still keeps it's metadata. Now the def special form:

1. Creates a var.
2. Copies the metadata {:xxx 1} from the symbol greet to the var.
3. Binds the var to the function (fn ([] "hello")).
4. Creates a mapping in the current namespace ("user") from the symbol
greet to the var.

Now notice the function object itself never has any metadata associated
with it? In fact normal Clojure function objects cannot have any metadata:

(with-meta (fn []) {:xxx 1})
=> [Thrown class java.lang.UnsupportedOperationException]

So now suppose we want something that achieves this:

(def #^{:xxx 2} groan (fn ([] "arrrgghhh")))

(get-xxx greet groan)
=> [1 2]

So we need to get at the vars for greet and groan. First thing to note
is that get-xxx has to be a macro, if it were a function greet and groan
would be evaluated to function objects, which don't have metadata. So
what we need is a macro that will take the greet and groan symbols,
resolve them to get vars and then lookup :xxx in the metadata of the
vars. So something like:

(defmacro get-xxx [& syms]
(vec (map #(get (meta (resolve %)) :xxx) syms)))

Now what if we want get-xxx to take a vector? Well no worries we just
change the signature:

(defmacro get-xxx [syms]
(vec (map #(get (meta (resolve %)) :xxx) syms)))

(get-xxx [greet groan])
=> [1 2]

But hang on, what's going on with this?

(let [fns [greet groan]]
(get-xxx fns))
=> java.lang.IllegalArgumentException: Don't know how to create ISeq
from: clojure.lang.Symbol

Remember that the arguments to macros aren't evaluated, so get-xxx is
being passed the symbol "fns" not a vector! Well what if use
macroexpand to explicitly pass it the vector?

(let [fns [greet groan]]
(macroexpand (list 'get-xxx fns)))
=> java.lang.ClassCastException: user$greet__5100 cannot be cast to
clojure.lang.Symbol

Another problem. The let creates a vector function objects not of
symbols. So to get this to work we need to quote greet and groan to
prevent them from being evaluated:

(let [fns ['greet 'groan]]
(macroexpand (list 'get-xxx fns)))
=> [1 2]

Or we can just quote the whole vector:

(let [fns '[greet groan]]
(macroexpand (list 'get-xxx fns)))
=> [1 2]

But at this point get-xxx may just as well have been a function:

(defn get-xxx-fn [syms]
(vec (map #(get (meta (resolve %)) :xxx) syms)))

(let [fns '[greet groan]]
(get-xxx-fn fns))
=> [1 2]

Now, that is one of the reasons why the first rule of Macro Club is:
Don't Write Macros. ;-)

Cheers,

Alex

[Stefan Arentz]

Hi Alex,

Wow! Thank you so much for this excellent explanation! It totally
makes sense now :-)

S.

[Stefan Kamphausen]

Hi Alex,

first of all thank your this exhaustive explanation.

I still don't get some things, though, and kindly ask for more
enlightenment.

On Nov 8, 3:46 am, Alex Osborne <a...@meshy.org> wrote:
> So after resolving, it then has a Var object.  A Var, is as it's

> name suggests, a variable.  It consist of a name (symbol+ namespace),metadatamap and a value (called the binding).

As far as the documentation says, Vars can't have metadata:

"Symbols and collections support metadata," -- http://clojure.org/metadata

"Symbols, Lists, Vector, Sets and Maps can have metadata" --
http://clojure.org/reader

I tried to understand the Java implementation underneath which seems
to be in line with the documentation. E.g. a symbol extends AFn,
which extents Obj which implements IObj, which in turn extends IMeta
which seems to be the relevant implementation of the metadata
interface.

However, there is some metadata code in Var.java which indicates that
Vars *can* have metadata, but according to documentation they can't.

An example also seems to indicate that Vars can have metadata:

user=> (def #^{:hasmeta "yes"} my-var [1 2 3])
#'user/my-var
user=> (meta my-var)
nil
user=> (meta (var my-var))
{:ns #<Namespace user>, :name my-var,
:file "NO_SOURCE_PATH", :line 63, :hasmeta "yes"}


> Now formetadata"on" functoins.  When you write this:

>
> (defn #^{:xxx 1} greet [] "hello")
>
> The #^{...} syntax means that Clojure creates a list of two symbols
> (defn and greet), and empty vector and a string "hello".  The second symbol greet has
> the metadata{:xxx 1} associated with it.

Really? Then why the metadata of the symbol empty?
user=> (meta 'greet)
nil

Or am I not accessing the symbol's metadata that way?

This question is also related to the following part of your
explanation.

> The greet symbol still keeps it's metadata.  Now the def special form:
>
> 1. Creates a var.
> 2. Copies the metadata{:xxx 1} from the symbol greet to the var.
> 3. Binds the var to the function (fn ([] "hello")).
> 4. Creates a mapping in the current namespace ("user") from the symbol
> greet to the var.

All items agreed, except for 2. which I don't grok yet.


I think for me it boils down to

* Is the documentation wrong or my understanding of it?
* What part of the (Java)-code should I read to understand the
implementation of metadata (if I assume that my understanding of how
the classes extent/implement is not correct).

The reason for asking this is, that I *really* want to understand what
I am talking about.

Kind regards,
Stefan

[Alex Osborne]

Stefan Kamphausen <ska...@googlemail.com> writes:

> On Nov 8, 3:46 am, Alex Osborne <a...@meshy.org> wrote:
> As far as the documentation says, Vars can't have metadata:
>
> "Symbols and collections support metadata," -- http://clojure.org/metadata
>
> "Symbols, Lists, Vector, Sets and Maps can have metadata" --
> http://clojure.org/reader

I don't think the documentation is *wrong* per se, it just only seems to
cover the immutable types. Vars, refs, atoms, agents and namespaces can
all have metadata as well, but it works a little differently for them as they
are mutable. You change their metadata using the alter-meta!
function. The with-meta function will not work on them.

> Really? Then why the metadata of the symbol empty?
> user=> (meta 'greet)
> nil
>
> Or am I not accessing the symbol's metadata that way?

You are creating a new symbol 'greet, which doesn't have metadata.
Symbols are immutable types, you can't alter them and you can't alter
their metadata. Instead what you do is create a new symbol with
metadata attached.

It might make more sense if we consider the case of metadata on
collections. I can create a new empty vector with metadata using either
#^ or with-meta:

(meta (with-meta [] {:foo true})) ; => {:foo true}
(meta #^{:foo true} []) ; => {:foo true}

Having done so doesn't change the original empty vector "[]", as vectors
are immutable:

(meta []) ; => nil

The same is true for symbols:

(meta (with-meta 'greet {:a 1})) ; => {:a 1}
(meta 'greet) ; => nil

Note that (perhaps suprisingly) this doesn't work:

(meta #^{:a 1} 'greet) ; => {:a 1}

Why? Well because #^ attaches the metadata to the next read form.
What's the next read form? It's 'greet. But in fact 'greet is just
sugar for (quote greet). So we're actually affixing the metadata to a
list containing two symbols (quote and greet). When the compiler
evaluates (quote greet) it turns it into just the symbol greet and then
throws the list (and thus our metadata) away.

> * What part of the (Java)-code should I read to understand the
> implementation of metadata (if I assume that my understanding of how
> the classes extent/implement is not correct).

I think you are just being confused by the differences in how metadata
works with mutable and immutable types.

There are three interfaces to do with metadata:

IMeta.java meta -- reading metadata
IObj.java with-meta -- "changing" metadata on immutables
IReference.java alter-meta! reset-meta! -- changing metadata on mutables

For the immutable types the implementation is in Obj.java. For mutables
it is in AReference.java.

I hope that helps clarify things a bit.

Alex

[Stefan Kamphausen]

Hi,

that have been some really embarrassing typos in my post (typing too
fast in too stupid an interface, I think).

On Nov 27, 12:52 pm, "Alex Osborne" <a...@meshy.org> wrote:

> Stefan Kamphausen <ska2...@googlemail.com> writes:
> > On Nov 8, 3:46 am, Alex Osborne <a...@meshy.org> wrote:
> > As far as the documentation says, Vars can't have metadata:
>

> > "Symbols and collections support metadata," --http://clojure.org/metadata

>
> > "Symbols, Lists, Vector, Sets and Maps can have metadata" --
> >http://clojure.org/reader
>
> I don't think the documentation is *wrong* per se, it just only seems to
> cover the immutable types.

Which is kind of wrong, isn't it? I strongly believe that this should
be changed.

>  Vars, refs, atoms, agents and namespaces can
> all have metadata as well, but it works a little differently for them as they
> are mutable.  You change their metadata using the alter-meta!
> function.  The with-meta function will not work on them.

While I understand that the mutating functions will not apply to
immutable types, I don't get why I can't create the metadata using the
same interface.

> > Really? Then why the metadata of the symbol empty?
> > user=> (meta 'greet)
> > nil
>
> > Or am I not accessing the symbol's metadata that way?
>
> You are creating a new symbol 'greet,

A *new* symbol? I would have thought I'm getting the original symbol
again.
OK, but I see:

user=> (resolve 'greet)
#'user/greet
user=> (meta (resolve 'greet))
{:ns #<Namespace user>, :name greet,
:file "NO_SOURCE_PATH", :line 66, :arglists ([]), :xxx 1}

Sometimes the 'E' in 'REPL' stands for Enlightenment, and the 'P' must
be Pleasure then. Don't ask me about 'R' and 'L', though. ;-)

> It might make more sense if we consider the case of metadata on
> collections.

I think I already understood them. At least I can align things I read
in documentation and implementation with what I find in experiments on
the REPL.

> Note that (perhaps suprisingly) this doesn't work:
>
>   (meta #^{:a 1} 'greet) ; => {:a 1}
>
> Why?  Well because #^ attaches the metadata to the next read form.
> What's the next read form?  It's 'greet.  But in fact 'greet is just
> sugar for (quote greet).  So we're actually affixing the metadata to a
> list containing two symbols (quote and greet).  When the compiler
> evaluates (quote greet) it turns it into just the symbol greet and then
> throws the list (and thus our metadata) away.

This is subtle! It really feels like one of those things that will
still feel creepy another 50 years from now.
I'll have to meditate on this a bit.

> > * What part of the (Java)-code should I read to understand the
> > implementation of metadata (if I assume that my understanding of how
> > the classes extent/implement is not correct).
>
> I think you are just being confused by the differences in how metadata
> works with mutable and immutable types.

Definitely, and I don't like the distinction too much either.

> There are three interfaces to do with metadata:
>
> IMeta.java       meta -- reading metadata
> IObj.java        with-meta -- "changing" metadata on immutables
> IReference.java  alter-meta! reset-meta! -- changing metadata on mutables

OK, I see. I missed the last one.

> I hope that helps clarify things a bit.

It does. I get a growing feeling of understanding how things work
under the hood here.

Your help is highly appreciated, thank you very much.

Regards,
Stefan

[Meikel Brandmeyer]

Hi,

On Nov 27, 10:19 am, Stefan Kamphausen <ska2...@googlemail.com> wrote:

> As far as the documentation says, Vars can't have metadata:
>

> "Symbols and collections support metadata," --http://clojure.org/metadata
>
> "Symbols, Lists, Vector, Sets and Maps can have metadata" --http://clojure.org/reader

>
> I tried to understand the Java implementation underneath which seems
> to be in line with the documentation.  E.g. a symbol extends AFn,
> which extents Obj which implements IObj, which in turn extends IMeta
> which seems to be the relevant implementation of the metadata
> interface.
>
> However, there is some metadata code in Var.java which indicates that
> Vars *can* have metadata, but according to documentation they can't.

Vars are one of the four reference types implementing IReference. The
other three are Refs, Agents and Atoms. All IReference types support
metadata.

> An example also seems to indicate that Vars can have metadata:
>
> user=> (def #^{:hasmeta "yes"} my-var [1 2 3])
> #'user/my-var
> user=> (meta my-var)
> nil

meta is a normal function. That means in the above case that meta is
called in the vector.

user=> (def #^{:hasmeta "yes"} my-var (with-meta [1 2 3] {:foo :bar}))

#'user/my-var
user=> (meta my-var)

{:foo :bar}

> Really? Then why the metadata of the symbol empty?
> user=> (meta 'greet)
> nil

Because the is a different symbol which happens to have the same name.

user=> (= 'greet 'greet)
true
user=> (identical? 'greet 'greet)
false
user=> (= 'greet (with-meta 'greet {:foo :bar}))
true

> Or am I not accessing the symbol's metadata that way?

You are. But of a different symbol. ;)

> * Is the documentation wrong or my understanding of it?

Hmm.. It seems missing, but I'm sure I read somewhere about the
reference types. Maybe in one Rich's talks? Hmmm..

> * What part of the (Java)-code should I read to understand the
> implementation of metadata (if I assume that my understanding of how
> the classes extent/implement is not correct).

The chain is Var -> Aref -> AReference -> IReference -> IMeta

> The reason for asking this is, that I *really* want to understand what
> I am talking about.

That's a Good Thing. We live to much from assumptions.

Sincerely
Meikel

[Meikel Brandmeyer]

Hi,

On Nov 27, 2:45 pm, Stefan Kamphausen <ska2...@googlemail.com> wrote:

> A *new* symbol?   I would have thought I'm getting the original symbol
> again.

If you get back the original symbol back, it can't carry metadata...

> Definitely, and I don't like the distinction too much either.

But it makes sense. A Var represents an identity and hence always
stays the same. Just the thing it points to might change. So we
obviously need to modify the Var in order to change the meta. If we'd
get a new Var, we would have a new identity. Clojure makes the
distinction between such side-effecting functions and pure functions
very clear.

> It does.  I get a growing feeling of understanding how things work
> under the hood here.

Watch the talk be Rich: http://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hickey

There he explains the different notions and why they are required.

Sincerely
Meikel

[John Harrop]

On Fri, Nov 27, 2009 at 8:45 AM, Stefan Kamphausen <ska...@googlemail.com> wrote:

> Why?  Well because #^ attaches the metadata to the next read form.
> What's the next read form?  It's 'greet.  But in fact 'greet is just
> sugar for (quote greet).  So we're actually affixing the metadata to a
> list containing two symbols (quote and greet).  When the compiler
> evaluates (quote greet) it turns it into just the symbol greet and then
> throws the list (and thus our metadata) away.

This is subtle!  It really feels like one of those things that will
still feel creepy another 50 years from now.
I'll have to meditate on this a bit.

Maybe this ought to be fixed; i.e., if the reader sees #^{meta} 'foo it applies the metadata to foo first, then quotes it, resulting in the same thing as (quote #^{meta} foo).

It would mean the reader would have to see #^{meta} and then store that, then read the next item. Currently, if the next item is also a reader macro, it expands that and then applies the meta to the result. Instead, if the next item was a read macro with an "argument" (like ' or even ^ or @ or whatever) it would apply the meta to the argument, then apply the new reader macro.

[Richard Newman]

> Maybe this ought to be fixed; i.e., if the reader sees #^{meta} 'foo
> it applies the metadata to foo first, then quotes it, resulting in
> the same thing as (quote #^{meta} foo).

Why introduce that special case, when you can simply do the second?

I don't support the view that it's OK for programmers to not know what
they're doing, which in this case means knowing that 'foo reads as
(quote foo).

[John Harrop]

I'm not advocating in favor of programmers not knowing what they're doing. I'm advocating in favor of avoiding as many subtle gotchas as possible. 

[John Harrop]

Interestingly, you can do what I suggested manually:

user=> (meta '#^{:foo 1}foo)

{:foo 1}

It's a bit ugly, but it works.

[Alex Osborne]

Stefan Kamphausen <ska...@googlemail.com> writes:

>> I don't think the documentation is *wrong* per se, it just only seems to
>> cover the immutable types.
>
> Which is kind of wrong, isn't it? I strongly believe that this should
> be changed.

Indeed.

> While I understand that the mutating functions will not apply to
> immutable types, I don't get why I can't create the metadata using the
> same interface.

Because the two operations have completely different semantics. With
immutables a new object is created, with mutables an existing object is
changed. Clojure could hide this behind the same function name, but
you'd still need to know the distinction anyway to work with these two
types of objects. Clojure makes this explicit. See what I say below on
Rich's avoidance of "apparent simplicity but hidden complexity".

> A *new* symbol? I would have thought I'm getting the original symbol
> again.

Yes, one of the differences between symbols and keywords is that symbols
are created fresh each time while keywords are interned:

(identical? 'foo 'foo) ; => false
(identical? :foo :foo) ; => true

Both are immutable, thus symbols can have metadata (as you can have two
symbols that differ only by their metadata) but keywords can't, as two
keywords with the same name are the same object.

>> Note that (perhaps suprisingly) this doesn't work:
>>
>> (meta #^{:a 1} 'greet)

Perhaps I should also note that this *does* work:

(meta '#^{a 1} greet)

As it becomes:

(meta (quote greet)) ; <-- this greet symbol has metadata

> This is subtle! It really feels like one of those things that will
> still feel creepy another 50 years from now.
> I'll have to meditate on this a bit.

Clojure is a very opinionated language and one of those opinions is that
Rich tries very hard to avoid incidental complexity. At times this
means things may at first appear more complex on the surface than in
other languages, but this is because Clojure isn't trying to hide what's
going on under the hood from you. It's one of the things I really enjoy
about it: there's no "magic". Clojure simplicity is real simplicity,
not apparent simplicity created by hiding the complexity under the bed.

It's possible and perhaps should even be expected for most Clojure
programmers to have a good understanding of how things work underneath.
With other languages, even if I've been using them for years I still
have little clue about how things actually get evaluated. If I've taken
a course in compiler writing, I might know about things like abstract
syntax trees, parsers and lexers and all the special cases involved in
parsing if-then-else statements, but how it works in a particular
language is going to be a mystery unless I've written my own compiler.

As Meikel suggested, watch Rich's talks. I think it's important to try
to understand the philosophy behind Clojure's design. Rich has some
very strong messages and even if you don't agree with them, they're
definitely worth listening to and thinking about.

Cheers,

Alex

[Stefan Kamphausen]

Hi,

On Nov 27, 11:06 pm, "Alex Osborne" <a...@meshy.org> wrote:
> > A *new* symbol?   I would have thought I'm getting the original symbol
> > again.
>
> Yes, one of the differences between symbols and keywords is that symbols
> are created fresh each time while keywords are interned:
>
> (identical? 'foo 'foo) ; => false
> (identical? :foo :foo) ; => true

Whoa! Cool example. This is rather an important point, I think. I
mean it's not important, when your "only" programming with Clojure,
which will usally work, and you always have the REPL... But it
becomes more important for those, trying to understand. It's about
grokking a language's design vs. experimenting with the code long
enough so that it seems to work.

> >> Note that (perhaps suprisingly) this doesn't work:
>
> >> (meta #^{:a 1} 'greet)
>
> Perhaps I should also note that this *does* work:
>
>   (meta '#^{a 1} greet)

To be honest, I think it looks even worse. There is some reader macro
which by happy accident works in a certain way together with the other
read syntax. No, I don't think it should work.

> Clojure is a very opinionated language and one of those opinions is that
> Rich tries very hard to avoid incidental complexity.

Which is a Good Idea(tm).

> At times this
> means things may at first appear more complex on the surface than in
> other languages, but this is because Clojure isn't trying to hide what's
> going on under the hood from you. It's one of the things I really enjoy
> about it: there's no "magic".  Clojure simplicity is real simplicity,
> not apparent simplicity created by hiding the complexity under the bed.

Hm, is it possible you're coming from Java here? For me, coming more
from CL than Java, some things in Clojure feel very --let's say--
Perlish: there is so much syntax there. Don't get me wrong, I think
Clojure delivers what arc promised, it does a hell of a job
revolutionizing Lisp. I hardly can disagree with many of the design
principles, I just like the documentation to tell the whole story so
not everyone new to Clojure will have to figure it out for him/
herself.

> It's possible and perhaps should even be expected for most Clojure
> programmers to have a good understanding of how things work underneath.

Agreed. The docs should say so, agreed? I'd very much like to
volunteer on docs, but I just don't know how I could keep up with
development, since I am doing almost all my Clojure stuff after work
when the children are in bed.

> With other languages, even if I've been using them for years I still
> have little clue about how things actually get evaluated.

That may be related to the lesser importance of order of evaluation in
other languages. I mean, Perl has a *lot* of syntax, but usually it
all takes place at the same time, or at least, you as a developer
won't have to care. Anyway, not worth a discussion.

> As Meikel suggested, watch Rich's talks.

I do, and I really like them. One can hear, that Rich's got a story
to tell. They are very valuable for understanding the philosophy
behind many of Clojure's design decisions. The screencast/
presentation/video-thing helps learning a new language nowadays.

>  I think it's important to try to understand the philosophy behind Clojure's design.

Wait, I've heard that before... Deja vu?

> Rich has some
> very strong messages and even if you don't agree with them, they're
> definitely worth listening to and thinking about.

There, another Deja vu. ;-)


Alex, John, Meikel, Richard,
thanks for taking your time.

Kind regards,
Stefan

[Richard Newman]

> Whoa! Cool example. This is rather an important point, I think. I
> mean it's not important, when your "only" programming with Clojure,
> which will usally work, and you always have the REPL... But it
> becomes more important for those, trying to understand. It's about
> grokking a language's design vs. experimenting with the code long
> enough so that it seems to work.

Yeah, pretty awesome :)

I love how almost any technical question on this list can be answered
by a three-line REPL interaction!

>> (meta '#^{a 1} greet)
>
> To be honest, I think it looks even worse. There is some reader macro
> which by happy accident works in a certain way together with the other
> read syntax. No, I don't think it should work.

It's not really a happy accident, it's as specified. ' quotes the next
read form. #^ applies metadata to the next read form. Granted, I have
some experience with reader macros from CL, but I feel this should be
an area that a moderately well-educated programmer should encounter,
perhaps trip up, say "oh yes, I see why that happens", and move on,
rather than viewing it as some kind of fault to be corrected.

When you realize that reader macros and quotation marks are a way of
communicating with the reader that's parsing the text of your code,
things become a little simpler.

> Hm, is it possible you're coming from Java here? For me, coming more
> from CL than Java, some things in Clojure feel very --let's say--
> Perlish: there is so much syntax there.

Coming from CL myself, I have two inputs to this debate.

Firstly, CL has more syntax than one at first thinks (for a Lisp, at
least) — particularly the raft of reader macros (including
configurable ones), but also things like format string keywords. It
simply doesn't have a plurality of *delimiters*: just double-quote,
block comment, and parens, really.

Secondly, I view most of Clojure's syntax — mainly delimiters for
literal maps, sets, and vectors — to be completely worth the sacrifice
of simplicity. The delimiter characters are there, and it would be
foolish to waste them.

The things that makes Perl "Perlish" to me are the special variable
line-noise ($/, $_, $&, etc.), contexts (scalar etc.), and extreme
parse-time lexical manipulation, all of which conspire to make the
meaning of a piece of code enormously context- and runtime-dependent.
(For example, you can't parse Perl without executing it.)

I don't think Clojure's additional delimiters compared to Common Lisp
moves it towards Perl in this way. I hate Perl, and I don't have that
reaction to Clojure! :)

> I just like the documentation to tell the whole story so
> not everyone new to Clojure will have to figure it out for him/
> herself.

As I'm sure you know, this is an unfortunate consequence of Clojure's
youth. Clojure only turned 2 last month, so I think its vibrant
library collection and community are actually surprisingly large and
vigorous. Docs take time to accrete, and Rich is devoted more to
development than to filling in perceived gaps; a lot of knowledge is
tied up in mailing list posts, videos, and presentations. I know Tom
and others have been working to improve the doc situation.

> Alex, John, Meikel, Richard,
> thanks for taking your time.

My pleasure. I love this community!

[Alex Osborne]

Stefan Kamphausen <ska...@googlemail.com> writes:

>>   (meta '#^{a 1} greet)
>
> To be honest, I think it looks even worse. There is some reader macro
> which by happy accident works in a certain way together with the other
> read syntax. No, I don't think it should work.

I agree this is ugly and unintuitive and I wouldn't use it in a real
program, I'd use with-meta instead. I included it as an illustration
that some reader macros like ~, ' and @ are just shorthand for regular
macros: unquote, quote and deref respectively, while others change the
behaviour of the reader, like #^, ; and #_.

Unfortunately syntax-quote (`) currently falls into the latter camp, but
I hope that will change in Clojure-in-Clojure. Try this to see
something scary:

'`(foo)
=> (clojure.core/seq
(clojure.core/concat
(clojure.core/list
(quote user/foo))))

The reason it needs to break the list down like that is so that it can
implement splicing-unquote.

> Hm, is it possible you're coming from Java here? For me, coming more
> from CL than Java, some things in Clojure feel very --let's say--
> Perlish: there is so much syntax there. Don't get me wrong, I think
> Clojure delivers what arc promised, it does a hell of a job
> revolutionizing Lisp.

Yes, I have a mainly non-lisp background, much C, Python, Ruby and
some Java and Haskell and various other languages. I had dabbled in
Common Lisp and Scheme a bit before discovering Clojure but found Common
Lisp very complex and Scheme very verbose and the lack of syntax for
common data structures like hash tables frustrating. So perhaps I fall
towards the slightly more sugary side of the spectrum compared to
Scheme, especially when it comes to data structures but I do agree that
things might be clearer without the ^ and #' reader macros and
restricting #^ just to type hints.

> I hardly can disagree with many of the design
> principles, I just like the documentation to tell the whole story so
> not everyone new to Clojure will have to figure it out for him/
> herself.

Yes. Stuart's book "Programming Clojure" is a much better introduction
to the language. The website is mostly okay as a reference but it is
incomplete and usually out of date.

[DTH]

On Nov 27, 5:46 pm, Richard Newman <holyg...@gmail.com> wrote:
>
> I don't support the view that it's OK for programmers to not know what
> they're doing, which in this case means knowing that 'foo reads as
> (quote foo).

FWIW I *strongly* agree; getting reader macros straight in my head was
a *big* help in macro writing; the special case would reinforce the
"wrong" impression, IMO

On Nov 27, 10:06 pm, "Alex Osborne" <a...@meshy.org> wrote:
> Clojure is a very opinionated language and one of those opinions is that
> Rich tries very hard to avoid incidental complexity.  At times this
> means things may at first appear more complex on the surface than in
> other languages, but this is because Clojure isn't trying to hide what's
> going on under the hood from you. It's one of the things I really enjoy
> about it: there's no "magic".  Clojure simplicity is real simplicity,
> not apparent simplicity created by hiding the complexity under the bed.
>

Yes! One of the greatest aspects of Clojure is the clean consistency
of the base language. One of the things that put me off really using
Scala in anger was that the language itself seemed to have a lot of
corner cases etc. (1) Scala was easier to pick up coming from Java,
but Clojure easier to really get to grips with.

-Dave

1. This is entirely subjective of course; if you consider Scala well-
nigh spherical in its lack of corners, roll on.