First-class functions and macros

[Axel Reichert]

Hello,

both first-class functions and macros are very powerful features. When
learning Lisp I especially loved the former and started to use them very
often. I am still working towards becoming a macro writer/user. One
thing that puzzled me:

(defun square (n)
(* n n))

(mapcar #'square (list 1 2 3 4))

works fine and returns (1 4 9 16). But why cannot macros be used like
first-class functions?

(defmacro cube (n)
(* n n n))

(mapcar #'cube (list 1 2 3 4))

gives

COMMON-LISP-USER::CUBE is a macro, not a function.
[Condition of type UNDEFINED-FUNCTION]

instead of (1 8 27 64). (Of course, "cube" should be a function, not a
macro, but this is just an example.)

I might overlook something obvious to you, but my gut feeling is that
there is a deep theoretical reason for this "limitation". Can anyone
please sketch a brief explanation?

Best regards

Axel

[STE]

On 2023-04-05, Axel Reichert <ma...@axel-reichert.de> wrote:

> (mapcar #'square (list 1 2 3 4))
>
> works fine and returns (1 4 9 16). But why cannot macros be used like
> first-class functions?
>
> (defmacro cube (n)
> (* n n n))

It would be

(defmacro cube (n)
`(* ,n ,n ,n))

> (mapcar #'cube (list 1 2 3 4))
>
> gives
>
> COMMON-LISP-USER::CUBE is a macro, not a function.
> [Condition of type UNDEFINED-FUNCTION]

That's why there are functions and macros, they are not the same things,
they must not be used in the same context for the same purpose.

There is nothing to win to use a macro to write (cube), that's
typically a fonction.

But try to write (while) with a function.

(defmacro while (test &rest body)
`(loop (if (not ,test) (return)) ,@body))

[Spiros Bousbouras]

There is certainly a similarity in that macros are a kind of function which
accepts as argument a piece of code and returns a new piece of code. But
there are differences. Consider the following from the CLHS :

3.1.2.1.2.2 Macro Forms

If the operator names a macro, its associated macro function is applied
to the entire form and the result of that application is used in place of
the original form.

Specifically, a symbol names a macro in a given lexical environment if
macro-function is true of the symbol and that environment. The function
returned by macro-function is a function of two arguments, called the
expansion function. The expansion function is invoked by calling the
macroexpand hook with the expansion function as its first argument, the
entire macro form as its second argument, and an environment object
(corresponding to the current lexical environment) as its third argument.
The macroexpand hook, in turn, calls the expansion function with the form
as its first argument and the environment as its second argument. The
value of the expansion function, which is passed through by the
macroexpand hook, is a form. The returned form is evaluated in place of
the original form.

It probably doesn't count as a "deep theoretical reason" but it explains that
macros and ordinary functions get called by different mechanisms and using
different arguments [some of the arguments passed implicitly] so it wouldn't
be practical to make them interchangeable for MAP and friends. Now you might
ask , do they have to be called using so different mechanisms ? The fact that
a macro must also be able to accept a lexical environment certainly adds to
the functionality. For a specific example see

Remembering information during compilation
https://groups.google.com/g/comp.lang.lisp/c/Rg5Ehxk2yZw

, a thread I started in 2009. [ If you're curious as to why I wanted that ,
see the IMPLEMENTATION section in
http://vlaho.ninja/prog/enhanced-do/README.txt ]

I've never had a chance to [explicitly] use the macroexpand hook [ i.e. the
value of the variable *MACROEXPAND-HOOK* ] but I'm sure that it exists in the
specification because some people have a use for this kind of thing.

Perhaps with sufficient effort one could make functions and macros
interchangeable in more contexts but I can't think of any reason to
expend the effort.

--
Being a pianist as well as a cellist, I learned both parts before we met.
When we first played it together, I kept correcting him. "I think that F
natural should be an F#.... The chord isn't C, E-flat, G, it's C, E natural,
G#...." Prokofiev finally said, "Who wrote this, me or you?"
http://www.cello.org/Newsletter/Articles/rostropovich/rostropovich.htm

[Tom Russ]

On Tuesday, April 4, 2023 at 11:17:39 PM UTC-7, STE wrote:
> On 2023-04-05, Axel Reichert <ma...@axel-reichert.de> wrote:
> > (defmacro cube (n)
> > (* n n n))
> It would be
>
> (defmacro cube (n)
> `(* ,n ,n ,n))

Interestingly, the OP's version will also work, but only when the macro is
applied to a literal numeric value. But if it is used with a variable or other
form that needs to be evaluated, it will fail.
(CUBE 2) ==> 8
(LET (x 2) (CUBE x)) ==> Error. You can't multiply the symbol X.

The reason is that the macro operates on the form that it is passed and
executes the body. This is done at macro-expansion time, so the original
macro would need to compute the cube of N at the time it expands. That
will only be possible if the form is a literal number. That is why you would
really want to use the macro definition Alex provides.

This also provides a hint as to why you can't map the macro, as the mapping
would happen at run time, not at macro-expansion time. But then it is too
late.

[Kaz Kylheku]

On 2023-04-05, Axel Reichert <ma...@axel-reichert.de> wrote:

> Hello,
>
> both first-class functions and macros are very powerful features. When

Macros aren't first class. They get expanded at syntax processing
time. Expanded code contains no more macros; they are gone.

Major dialects of Lisp have "first rate" macro systems,
which isn't the same. :)

> learning Lisp I especially loved the former and started to use them very
> often. I am still working towards becoming a macro writer/user. One
> thing that puzzled me:
>
> (defun square (n)
> (* n n))
>
> (mapcar #'square (list 1 2 3 4))
>
> works fine and returns (1 4 9 16). But why cannot macros be used like
> first-class functions?

First-class function means that a function is a run-time object.
It is applied to arguemnt values.

A macro works with the syntax of the expressions, rather than
their values.

Underneath a macro there is a function, and that itself is first
class. However, if we have a macro called foo, we don't get to
that function using #'foo or foo.

We can use (macro-function 'foo) to access the function.

The function doesn't work like the the macro might suggest. If we
write, say,

(defun mac (a b c))

Then (macro-function 'mac) does not refer to a three-argument
lambda. A macro function, regardless of the destructuring
lambda list of the macro, is a two-argument function whose
first argument is the entire macro call form (its literal
syntax) and an environment parameter. Quick demo:

Macro that takes three argument expressions, inserting
them into a (list ...) call in reverse order:

[1]> (defmacro revlist3 (a b c) `(list ,c ,b ,a))
REVLIST3

We explicitly invoke the expander function on the form (revlist 1 2 3)
to produce (list 3 2 1). (We pass nil as the environment parameter.)

[2]> (funcall (macro-function 'revlist3) '(revlist3 1 2 3) nil)
(LIST 3 2 1)

Now invoke it with some arguments that are expressions; you can see
they are not evaluated, just the syntax is inserted:

[3]> (funcall (macro-function 'revlist3)
'(revlist3 (- 2 1) (+ 1 1) (/ 6 2)) nil)
(LIST (/ 6 2) (+ 1 1) (- 2 1))

This macro-function is first class, to be sure. The mac macro which is
implemented by that function isn't a first class anything though.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazi...@mstdn.ca

[Spiros Bousbouras]

On Thu, 6 Apr 2023 06:17:34 -0000 (UTC)
Kaz Kylheku <864-11...@kylheku.com> wrote:
> Underneath a macro there is a function, and that itself is first
> class. However, if we have a macro called foo, we don't get to
> that function using #'foo or foo.
>
> We can use (macro-function 'foo) to access the function.
>
> The function doesn't work like the the macro might suggest. If we
> write, say,
>
> (defun mac (a b c))
>
> Then (macro-function 'mac) does not refer to a three-argument
> lambda. A macro function, regardless of the destructuring
> lambda list of the macro, is a two-argument function whose
> first argument is the entire macro call form (its literal
> syntax) and an environment parameter. Quick demo:
>
> Macro that takes three argument expressions, inserting
> them into a (list ...) call in reverse order:
>
> [1]> (defmacro revlist3 (a b c) `(list ,c ,b ,a))
> REVLIST3
>
> We explicitly invoke the expander function on the form (revlist 1 2 3)
> to produce (list 3 2 1). (We pass nil as the environment parameter.)
>
> [2]> (funcall (macro-function 'revlist3) '(revlist3 1 2 3) nil)
> (LIST 3 2 1)

Or , to connect it to the example in the opening post ,

(defmacro cube (n) (* n n n))

(mapcar (macro-function 'cube)
'((irrelevant 1) (irrelevant 2) (irrelevant 3)) '(nil nil nil))

returns (1 8 27) .

[Spiros Bousbouras]

On Thu, 6 Apr 2023 06:17:34 -0000 (UTC)
Kaz Kylheku <864-11...@kylheku.com> wrote:

> First-class function means that a function is a run-time object.
> It is applied to arguemnt values.

Actually "first class functions" also means that you can create new
functions at runtime and they have the full power of preexisting ones.

Note that in standard C preexisting functions are also a runtime object of
sorts through function pointers but one wouldn't say that C has first class
functions. If you go beyond standard C , you can load precompiled functions
from dynamic libraries. And if your environment includes a C compiler which
can create shared libraries then you can even create new functions during
runtime and then load them and execute them. But you have to jump through a
lot more hoops to do all that compared to Lisp.

--
If you regard Moore's works as documentaries you will be outraged. If you think of
them as feature-length, full-motion political cartoons - with all the caricature-in-
the-service-of-making-a-point that implies - you'll find them a lot more on-target.
https://www.rifters.com/crawl/?p=9298

[Axel Reichert]

Kaz Kylheku <864-11...@kylheku.com> writes:

> [1]> (defmacro revlist3 (a b c) `(list ,c ,b ,a))
> REVLIST3

[...]

> [3]> (funcall (macro-function 'revlist3)
> '(revlist3 (- 2 1) (+ 1 1) (/ 6 2)) nil)
> (LIST (/ 6 2) (+ 1 1) (- 2 1))

O.K., got it, thanks for the explanation. But wouldn't an "eval" on this
return value do the trick?

(eval (funcall (macro-function 'revlist3)
'(revlist3 (- 2 1) (+ 1 1) (/ 6 2)) nil))

gives

(3 2 1)

, but I have the gut feeling that I am still confused and, for formal
reasons (symmetry between functions and macros), ask for something that
would not be useful from a practical point of view. Or can anyone think
of a concrete example where mapping a macro over a list would make
sense?

Best regards

Axel

[Axel Reichert]

Tom Russ <tar...@google.com> writes:

> On Tuesday, April 4, 2023 at 11:17:39 PM UTC-7, STE wrote:
>> On 2023-04-05, Axel Reichert <ma...@axel-reichert.de> wrote:
>> > (defmacro cube (n)
>> > (* n n n))
>> It would be
>>
>> (defmacro cube (n)
>> `(* ,n ,n ,n))
>
> Interestingly, the OP's version will also work, but only when the macro is
> applied to a literal numeric value. But if it is used with a variable or other
> form that needs to be evaluated, it will fail.
> (CUBE 2) ==> 8
> (LET (x 2) (CUBE x)) ==> Error. You can't multiply the symbol X.

Sure, I should have known this. Embarrassing ...

> This also provides a hint as to why you can't map the macro, as the mapping
> would happen at run time, not at macro-expansion time. But then it is too
> late.

Got it.

Thanks!

Axel

[Bipolar Transistor]

What you are talking about here reminds me of fexprs. They were
basically like functions, that, like macros, received their arguments
unevaluated, but, unlike macros, it would actually be possible to pass
them to higher order functions because they were, well, functions.
They were used in the old times of LISP 1.5, maclisp and interlisp, but
were later ditched in favour of macros because they hindered
compilation. If you want to look at something that developed in a
differend direction, take a peek at the kernel language, or fexpress
language for racket.