Macros and syntax-quote

[Rock]

I was just reading the Macro section in the WikiBook. Regarding the
following piece of code,

`(let [~'frame ~frame]
(.setTitle ~'frame ~title)
(.setVisible ~'frame)
~'frame)

I was what the difference might be with respect to this:

`(let [frame ~frame]
(.setTitle frame ~title)
(.setVisible frame)
frame)

I'm coming from Common Lisp. I've got to admit that, as much as I've
always loved using macros, I have always felt that BACKQUOTE (or
syntax-quote in this case) can be confusing, except for the simpler
cases. I mean, when you get to nested backquotes, for instance, it can
become pretty complicated (as stated by Paul Graham as well).

Is there any chance of getting a more thourough explanation within
Clojure? I'm convinced this is an important issue, especially for
those coming from the likes of Java, Python, Ruby, etc...

[Meikel Brandmeyer]

Hi,

On 20 Nov., 11:29, Rock <rocco.ro...@gmail.com> wrote:
> I was what the difference might be with respect to this:
>
>  `(let [frame ~frame]
>    (.setTitle frame ~title)
>    (.setVisible frame)
>    frame)

This won't work, since - assuming you are in namespace
user - the backquote will expand to...

(let [user/frame (new JFrame)]
(.setTitle user/frame "Sometitle")
(.setVisible user/frame)
user/frame)

... and hence the let will complain, since you are not
allowed to use fully qualified symbols in a let form.

Hence you need ~'frame (capturing frame => bad) or
frame#/gensym (not capturing frame => good).

Hope this helps.

Sincerely
Meikel

[Rock]

Yeah. Thanks. Very clear.

Just out of curiosity, do you know where I can find the code for
syntax-quote in the Clojure source? I would like to see how it's
implemented to possibly get a better understaning of it.

Thanks again.

[Meikel Brandmeyer]

Hi,

On 20 Nov., 12:49, Rock <rocco.ro...@gmail.com> wrote:
> Just out of curiosity, do you know where I can find the code for
> syntax-quote in the Clojure source? I would like to see how it's
> implemented to possibly get a better understaning of it.

You may have a look in Clojure's source in the src/jvm/clojure/lang
directory in the file LispReader.java. Search there for
SyntaxQuoteReader.

Sincerely
Meikel

[Jarkko Oranen]

On Nov 20, 12:29 pm, Rock <rocco.ro...@gmail.com> wrote:
> I was just reading the Macro section in the WikiBook. Regarding the
> following piece of code,
>
> `(let [~'frame ~frame]
>    (.setTitle ~'frame ~title)
>    (.setVisible ~'frame)
>    ~'frame)
>
> I was what the difference might be with respect to this:
>
>  `(let [frame ~frame]
>    (.setTitle frame ~title)
>    (.setVisible frame)
>    frame)
>

Expanding these shows the difference:
and their values frame = 0, title = 1;
the first one becomes
(clojure.core/let [frame 0]
(user/.setTitle frame 1)
(user/.setVisible frame) frame)

Now, this is a valid clojure form (ignoring that integers don't
have .setTitle or .setVisible) that could in theory be evaled.

This kind of expansion should be treated with care. It *captures* the
name frame from the outer scope, shadowing its definition with the
let. This is a dangerous thing to do, and in case it's needed, it
should be clearly documented what variable names are captured. In this
case, you know the context this expansion is done in, so it's not a
problem, but that is not always the case with macros.

Using ~' in a macro shows that you explicitly want to capture
something.

The second one becomes:

(clojure.core/let [user/frame 0]
(user/.setTitle user/frame 1)
(user/.setVisible user/frame) user/frame)

Notice how frame and title are qualified with the namespace.

The crucial difference is:
Namespace-qualified names can't be let.

This means that this will always throw an exception when evaluated.
Namespace qualifying is what makes clojure macros hygienic. That is,
guaranteed not to capture names unless done explicitly with ~' or
through other capturing macros.

the correct way to write this expansion would have been:

(let [my-frame (gensym)] ;notice, not in ` yet, so even if this were
in a macro it will not appear in the macro expansion.
`(let [~my-frame ~frame]
(something-or-other)
~my-frame))

here, the *value* of my-frame is a generated symbol; something
guaranteed to be unique. To that name, we bind the value of frame: the
expansion will become:

(clojure.core/let [G__1278 0]
(user/something-or-other)
G__1278)

which is safe.

Furthermore, another, easier way to produce a similar expansion is to
use automatic gensyms; in a ` form, any symbol prefixed with # is
uniquely expanded within that form (autogensyms don't nest):

`(let [myframe# ~frame]
(something-or-other)
myframe#) ; note, no ~. ~myframe# would mean 0

the expansion is similar to the above:

(clojure.core/let [myframe__1282 0]
(user/something-or-other)
myframe__1282)

> I'm coming from Common Lisp. I've got to admit that, as much as I've
> always loved using macros, I have always felt that BACKQUOTE (or
> syntax-quote in this case) can be confusing, except for the simpler
> cases. I mean, when you get to nested backquotes, for instance, it can
> become pretty complicated (as stated by Paul Graham as well).

Nested syntax-quotes can indeed cause problems with gensyms or just
readability. Fortunately, even though macros may seem like magic, in
the end they are just functions that return forms (clojure code). This
means you can make helper functions (with defn) that return simpler
forms (often by using syntax-quote themselves) and use those helpers
in the macro to produce more complex forms:

(defn helper [x method]
`(~method ~x))

`(do ~@(map helper '(obj another more)
'(.mutate .mutate .mutateMore)))

which will expand into

(do (.mutate obj) (.mutate another) (.mutateMore more))

> Is there any chance of getting a more thourough explanation within
> Clojure? I'm convinced this is an important issue, especially for
> those coming from the likes of Java, Python, Ruby, etc...

I'm not a macro expert myself, but I hope this will help you and
others as well.
It's good to spend some time simply writing macros and using
'macroexpand to see what kind of code they produce. I don't think
there is any other way to learn to write macros.

--
Jarkko

[Rich Hickey]

Yes. Please use auto-gensyms (name#):

`(let [frame# ~frame]
(.setTitle frame# ~title)
(.setVisible frame#)
frame#)

~' should be used for intentional capture only, i.e. rarely.

That section of the Wiki should be re-worked, IMO.

Rich

[Meikel Brandmeyer]

Hi,

On 20 Nov., 13:30, Rich Hickey <richhic...@gmail.com> wrote:
> Yes. Please use auto-gensyms (name#):
>
> `(let [frame# ~frame]
>    (.setTitle frame# ~title)
>    (.setVisible frame#)
>    frame#)

With this specific example, my intention was to show, how
using a macro where a function would do actually causes
a lot of trouble, like multiple evaluation, variable capture,
etc.

> ~' should be used for intentional capture only, i.e. rarely.

I think this was clearly stated in the wiki: only in rare cases,
well-defined situations and well documented.

> That section of the Wiki should be re-worked, IMO.

Yes. It is rather chaotic and not very clear over long
passages. I will tackle this as soon as I find out, who
has to approve my changes to the wiki....

Sincerely
Meikel

[Rich Hickey]

I didn't see any issues with wiki editing, ar eyou still having a
problem?

As far as the section goes, the problem is people will grab code
without reading the text carefully. Given its tutorial nature, I think
it's best for the Wiki to show examples of the right way to do things
rather than the wrong way.

Rich

[Simon Brooke]

A point about using macroexpand which I stumbled badly on yesterday is
that macroexpand expands the macros your macro expands into. Well, of
course it does. But it's been so long since I was learning a new macro
syntax that I'd forgotten this. And when my rather simple macro
expanded into a mess of let* and ifs I found myself staring blankly at
it wondering what the heck I'd done wrong.

[Michael Wood]

On Thu, Nov 20, 2008 at 3:32 PM, Simon Brooke <stil...@googlemail.com> wrote:
> On Nov 20, 12:19 pm, Jarkko Oranen <Chous...@gmail.com> wrote:

[...]

>> I'm not a macro expert myself, but I hope this will help you and
>> others as well.
>> It's good to spend some time simply writing macros and using
>> 'macroexpand to see what kind of code they produce. I don't think
>> there is any other way to learn to write macros.
>
> A point about using macroexpand which I stumbled badly on yesterday is
> that macroexpand expands the macros your macro expands into. Well, of
> course it does. But it's been so long since I was learning a new macro
> syntax that I'd forgotten this. And when my rather simple macro
> expanded into a mess of let* and ifs I found myself staring blankly at
> it wondering what the heck I'd done wrong.

Try macroexpand-1 instead.

--
Michael Wood <esio...@gmail.com>

[Rock]

Maybe there should be a little more info regarding the syntax-quote
expansion algorithm for more involved situations, especially nested
backquotes. There seems to be just the bare minimum in the Reference
section. The rest is gathered in bits and pieces here and there.
Contrast this with the detailed explanation in the CL HyperSpec:

http://www.lispworks.com/documentation/HyperSpec/Body/02_df.htm

It would really be cool if Rich could give us the algorithm in a
similar way (case analysis).

For instance, in the case of nested syntax-quotes, which one gets
expanded first, the inner or the outer one? (just a simple example of
what I'm trying to say)

This is important I believe, because syntax-quotes are an essential
ingredient of macros, and macros are one of the most distinctive
features of Lisp, which set it apart from the rest. I agree that a
very effective way of learning is by trial and error, but it would be
nice to have these things documented precisely as well.

For a more general teatment, here's a good read:

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.5.2566

[Meikel Brandmeyer]

Hi,

On 20 Nov., 14:28, Rich Hickey <richhic...@gmail.com> wrote:
> I didn't see any issues with wiki editing, ar eyou still having a
> problem?

When I go to the wiki, I get the notice, that there is one draft
waiting for approval.

: This is the latest sighted revision, approved on 18 November 2008.
The draft has 1 change awaiting review. (+/-)

In the page history the last two revisions have the tag [sighted
by Mike.lifeguard]. So I suppose there changed something with
those two revisions. The revisions before don't have this tag,
and editing also worked for me before.

> As far as the section goes, the problem is people will grab code
> without reading the text carefully. Given its tutorial nature, I think
> it's best for the Wiki to show examples of the right way to do things
> rather than the wrong way.

I will remove the section. If no one has objections I will have
a look into the pointers provided by Rock and will try to come
up with something similar for Clojure. This time hopefully more
suitable.

Sincerely
Meikel

[Rock]

Thanks for your efforts Meikel. Greatly appreciated.

Rock