How to reduce a list of booleans?

[Daniel Jomphe]

I'm in the following situation:

(and '(true true))
;doesn't work

I tried apply, reduce, and a few other things. Reading the apidocs,
reduce is said to be the proper choice to compute a boolean from a
seq. But:

(reduce and '(true true))
;Exception: Can't take value of a macro: #'clojure.core/and

Also, the following isn't the solution:

(reduce 'and '(true false true))
;true

In any case, I think using reduce with "and" wouldn't be nice because
it won't return false as soon as it can like "and" does. Therefore, I
came up with the following working solution:

(defmacro and-booleans [logical-booleans-list]
`(and ~@logical-booleans-list))

(and-booleans (true false true))
;false

But I wonder if I have overlooked something fundamental, either about
reduce or anything else.

Of course, I could have built my final boolean value imperatively, in
effect reimplementing "and" for lists as a function that wraps the
"ant" built-in macro. But I'm hoping to leave imperative programming
as much behing me as possible.

What would you consider the normal way of solving this small problem
of mine?

[Stephen C. Gilardi]

On Jan 15, 2009, at 1:53 PM, Daniel Jomphe wrote:

What would you consider the normal way of solving this small problem of mine?

(reduce #(and %1 %2) [true false true])

--Steve

[Christophe Grand]

Daniel Jomphe a écrit :

> I'm in the following situation:
>
> (and '(true true))
> ;doesn't work
>
>

(every? identity [true false true])

Christophe

[wwmorgan]

Look at every?. The predicate true? returns true if its argument is
the boolean literal true, and false otherwise. If, instead, you want
to check for logical truth (false and nil are false, everything else
is true), then use identity. As you can see, every? falls out at the
first false result.

user=> (every? true? (list true true true))
true
user=> (every? true? (lazy-cat [true false] (println "not executed")))
false
user=> (every? true? (lazy-cat [true true] (println "executed")))
executed
true
user=> (every? true? [])
true
user=> (every? true? (list true 1))
false
user=> (every? identity (list true 1))
true

[Daniel Jomphe]

> (every? identity [true false true])

So obvious, yet so overlooked each time I read the function names in
the api!

Thanks to both you! Looks like I'll be having a bunch of fun through
this learning curve in the future. :)

[Stuart Sierra]

On Jan 15, 2:05 pm, Daniel Jomphe <danieljom...@gmail.com> wrote:
> > (every? identity [true false true])
>
> So obvious, yet so overlooked each time I read the function names in
> the api!

Hi Daniel,

FYI, the reason "and" doesn't work is that it's a macro, not a
function. Only functions can be used with "apply".

If you had a hypothetical "and function" like this:

(defn and-fn [& args] (every? identity args))

Then you could use apply:

(apply and-fn [true true])
;=> true
(apply and-fn [true false true])
;=> false

But, obviously, just using "every?" in the first place is a lot
simpler.
-Stuart Sierra

[Daniel Jomphe]

Stuart Sierra wrote:

> FYI, the reason "and" doesn't work is that it's a macro, not a
> function.  Only functions can be used with "apply".

When I found that out, I was surprised. My knowledge of lisp comes
from reading, in the past few months:
- ANSI Common Lisp
- Practical Common Lisp
and, some years ago, a few more things.

And still, I wasn't prepared for the fact that macros don't mix 100%
with functions. Either I overlooked an important section in these
books, or they didn't really cover it much.

Even now that I hurt myself against this limitation, I'm still to make
complete sense out of it. I understand that macros aren't known to
functions because functions only get to see their expansions, but it's
fuzzy in my mind how some functions interact well with macros while
some others don't.

It's probably related to the fact one wouldn't want to rely on
repeated macro-expansions while apply/reduce's implementation recurs/
iterates over a seq. So in some way, macros would be limited so that
they are only valid if their call arguments are provided to them at
compile time, not at run time. Yet, it's still hard to see the line
between code-is-data and data-is-code.

[Stuart Sierra]

On Jan 15, 6:04 pm, Daniel Jomphe <danieljom...@gmail.com> wrote:
> And still, I wasn't prepared for the fact that macros don't mix 100%
> with functions. Either I overlooked an important section in these
> books, or they didn't really cover it much.

It's not covered much. You get used to it. Think of it this way --
macros only exist during compilation. Functions like "apply" get
evaluated at run-time. A macro like "and" has no value at run-time,
so you can't use it as an argument to a function.

-Stuart Sierra

[Timothy Pratley]

> fuzzy in my mind how some functions interact well with macros while
> some others don't.

Good:
(some-function (some-macro stuff))

Bad:
(some-function some-macro stuff)

For me I find it easiest to remember as "macros are not first class
functions" ie: they cannot be passed to and executed by other
functions.

[Konrad Hinsen]

On 16.01.2009, at 00:04, Daniel Jomphe wrote:

> When I found that out, I was surprised. My knowledge of lisp comes
> from reading, in the past few months:
> - ANSI Common Lisp
> - Practical Common Lisp
> and, some years ago, a few more things.

The best book to read about macros is in my opinion Paul Graham's "On
Lisp", which is now available for free download:

http://www.paulgraham.com/onlisp.html

It covers mostly Common Lisp plus a bit of Scheme, but many of the
ideas carry over very well to Clojure.

> Even now that I hurt myself against this limitation, I'm still to make
> complete sense out of it. I understand that macros aren't known to
> functions because functions only get to see their expansions, but it's
> fuzzy in my mind how some functions interact well with macros while
> some others don't.

The fundamental restriction is that you cannot pass macros to
arguments as functions, and that is exactly what would be required to
do a reduce over and. The universal workaround is to define a
function that contains nothing but the macro, such as

(fn [x y] (and x y))

which can also be written in Clojure using the shorthand notation

#(and %1 %2)

Here the macro gets expanded inside the function *body*, but what you
pass around is the function *object*.

Konrad.

[Stuart Halloway]

I second the vote for _On Lisp_. I have ported the examples to Clojure:

http://blog.thinkrelevance.com/2008/12/12/on-lisp-clojure

Stuart

[CuppoJava]

I'm also very interested in this topic.

The only reason to write "and" as a macro is for performance reasons.
It's much more convenient, programming-wise, if and is a function.

Is there anyway to write a macro/function that acts as a function when
necessary (so it can be passed as an argument), but expands into a
macro (for speed) when deemed possible?

[Kevin Downey]

and is also written as a macro to short circuit evaluation:

clojure.core/and
([] [x] [x & rest])
Macro
Evaluates exprs one at a time, from left to right. If a form
returns logical false (nil or false), and returns that value and
doesn't evaluate any of the other expressions, otherwise it returns
the value of the last expr. (and) returns true.

--
And what is good, Phaedrus,
And what is not good—
Need we ask anyone to tell us these things?

[Mark Volkmann]

On Fri, Jan 16, 2009 at 12:49 PM, CuppoJava <patrick...@hotmail.com> wrote:
>
> I'm also very interested in this topic.
>
> The only reason to write "and" as a macro is for performance reasons.

That's not the only reason. Another reason is to have the option to
avoid evaluating all of the arguments. Functions always evaluate all
of their arguments. Macros don't always do that.

--
R. Mark Volkmann
Object Computing, Inc.

[Michael Reid]

> Is there anyway to write a macro/function that acts as a function when
> necessary (so it can be passed as an argument), but expands into a
> macro (for speed) when deemed possible?

I don't think this is possible without some sort of support for a
macro/function duality in core Clojure. In the current implementation
(read: you should not rely on this fact) macros' are simply functions
with the #^{:macro true} metadata attached. Since you can't have a
symbol refer to two different values, I don't see any way to do this
except if you re-bind 'and' in a local context:

(let [and (fn and [x & xs]
(if x (if (nil? xs) true (and xs)) false))]
(apply and [true true false]))

Probably the closest you can come to this without redefining and is
Konrad's suggestion:

#(and %1 %2)

/mike.

[Chouser]

On Fri, Jan 16, 2009 at 3:41 PM, Michael Reid <kid....@gmail.com> wrote:
>
>> Is there anyway to write a macro/function that acts as a function when
>> necessary (so it can be passed as an argument), but expands into a
>> macro (for speed) when deemed possible?
>
> I don't think this is possible without some sort of support for a
> macro/function duality in core Clojure.

Which is one way to describe what's provided by definline. It's
marked experimental, has an unusual format and proviso's, but it's
there get a blend of macro performance and function first-classness.

--Chouser

[Daniel Jomphe]

Stuart Sierra said:

> It's not covered much. You get used to it. Think of it this way -- macros only exist during compilation. Functions like "apply" get evaluated at run-time. A macro like "and" has no value at run-time, so you can't use it as an argument to a function.

I'll tell you my current understanding of how Clojure works, so you
can correct me where I may be wrong:

- The compiler has for target a living environment; thus, it's
dynamic.
- There's no interpreter; thus, the following points will be easier to
reason about.
- When code is sent for evaluation, it's first read.
- The reader performs a few substitutions, then gives the resulting
code to the compiler.
- The compiler starts by expanding macros into their substitution.
--- Thus, for functions, there's no such thing as macros.
--- Thus, all that functions see of macros is what came out of macros'
complete expansion.
--- Therefore, like Timothy Pratley wrote so well, a function can have
for parameter a macro call, but not a macro itself.
- After macro expansion, the compiler starts compiling all functions
in need of being compiled, including the functions modified during
macro expansion.
- Then, code is ready for evaluation.

Now, I see that (function macro arg-given-to-macro-by-function) can't
work, because 'function', in its body, during its evaluation, would
have to give 'arg-given...' to 'macro'; but since 'function' is a
function, before eval was inside 'function's body, it needed to
evaluate each one of 'function's parameters; thus, at this precise
moment, 'macro' would have said "hey, now that you're evaluating me,
will you please give me my parameter?", and eval would have answered
"I can't, its 'function's responsibility, but we're not yet inside its
body", and thus, eval would have blocked.

Ok, so although it's a live system, when a new bunch of code is sent
for evaluation, all macros inside it must be expanded before runtime
evaluation can occur. Thus, functions can't act on macros. I thought
that somehow my code could still work because what would have happened
is 'reduce' would have acted on 'and's expansion, but I see and's
expansion couldn't be done because it would have to wait for reduce's
evaluation.

If we were speaking of concurrency, wouldn't this be a deadlock issue?

...I also thought it could be possible that 'and' expands to a partial
application that waits for 'reduce' to provide it its arguments.
Wouldn't this make sense?

Konrad Hinsen said:

> The universal workaround is to define a
> function that contains nothing but the macro, such as
>
> (fn [x y] (and x y))
>
> which can also be written in Clojure using the shorthand notation
>
> #(and %1 %2)
>
> Here the macro gets expanded inside the function *body*, but what you
> pass around is the function *object*.

Wouldn't this defeat the purpose of having 'and's arguments short-
circuited when false is found?

Now, why is this workaround possible? For sure, the wrapper function
is ok as a parameter of a function call. And in the called function's
body, when it's going to call the wrapper function, the arguments for
the macro wrapped in the wrapper function will be ready. And the
wrapper function contains the macro's substitution anyways. So if
we're willing to sacrifice macros' lazy parameters evaluation, we can
wrap them inside functions to achieve this effect. We could also say
that as long as macros are either called or wrapped in any function,
they can go anywhere. In other words, the only limitation seems to be
that runtime code can't use the symbol to which a macro is binded.

Phew, does this all make perfect sense?

Chouser said:

>>> Is there anyway to write a macro/function that acts as a function when
>>> necessary (so it can be passed as an argument), but expands into a
>>> macro (for speed) when deemed possible?

>> I don't think this is possible without some sort of support for a
>> macro/function duality in core Clojure.

> Which is one way to describe what's provided by definline. It's
> marked experimental, has an unusual format and proviso's, but it's
> there get a blend of macro performance and function first-classness.

Cool. Let's see how this develops. It's definitely interesting.

...All those years doing imperative programming, I knew I somehow
wasn't doing what I really always wanted to do. ...Anyways, before I
devote 100% of my programming time to macro-ed functional languages, I
want to get rid of the imperative way of programming as much as
possible, so I thought it might be good for me to do a small detour by
learning Haskell before coming back full strength to Clojure and Qi
+CL. If someone has any opinion about this, I'm all ears open.

[Stuart Sierra]

Hi Daniel,

On Jan 16, 6:54 pm, Daniel Jomphe <danieljom...@gmail.com> wrote:
> I'll tell you my current understanding of how Clojure works, so you
> can correct me where I may be wrong:
>
> - The compiler has for target a living environment; thus, it's
> dynamic.
> - There's no interpreter; thus, the following points will be easier to
> reason about.
> - When code is sent for evaluation, it's first read.
> - The reader performs a few substitutions, then gives the resulting
> code to the compiler.
> - The compiler starts by expanding macros into their substitution.
> --- Thus, for functions, there's no such thing as macros.
> --- Thus, all that functions see of macros is what came out of macros'
> complete expansion.
> --- Therefore, like Timothy Pratley wrote so well, a function can have
> for parameter a macro call, but not a macro itself.

All correct up to this point.

> - After macro expansion, the compiler starts compiling all functions
> in need of being compiled, including the functions modified during
> macro expansion.

Sort of. What's really getting compiled are just the function calls.
The functions themselves were compiled back when they were defined.

> - Then, code is ready for evaluation.

Yes, and at this point, Clojure has done its job and the Java Virtual
Machine takes over.

> Ok, so although it's a live system, when a new bunch of code is sent
> for evaluation, all macros inside it must be expanded before runtime
> evaluation can occur. Thus, functions can't act on macros. I thought
> that somehow my code could still work because what would have happened
> is 'reduce' would have acted on 'and's expansion, but I see and's
> expansion couldn't be done because it would have to wait for reduce's
> evaluation.
>
> If we were speaking of concurrency, wouldn't this be a deadlock issue?

Not really, because macro expansion is not happening concurrently
with compilation or evaluation.

> ...I also thought it could be possible that 'and' expands to a partial
> application that waits for 'reduce' to provide it its arguments.
> Wouldn't this make sense?

No, because "and" doesn't know how many arguments it's going to get.
If you look at the definition of "and", you'll see that it takes a
variable number of arguments and it is recursive. So if you tried to
expand it before knowing the length of the argument list, you would
get an infinite loop.

> Konrad Hinsen said:
>
> > The universal workaround is to define a
> > function that contains nothing but the macro, such as
>
> >         (fn [x y] (and x y))
>
> > which can also be written in Clojure using the shorthand notation
>
> >         #(and %1 %2)
>
> > Here the macro gets expanded inside the function *body*, but what you
> > pass around is the function *object*.
>
> Wouldn't this defeat the purpose of having 'and's arguments short-
> circuited when false is found?

Yes, but there is no alternative if you want a function.

>
> Now, why is this workaround possible? For sure, the wrapper function
> is ok as a parameter of a function call. And in the called function's
> body, when it's going to call the wrapper function, the arguments for
> the macro wrapped in the wrapper function will be ready. And the
> wrapper function contains the macro's substitution anyways. So if
> we're willing to sacrifice macros' lazy parameters evaluation, we can
> wrap them inside functions to achieve this effect. We could also say
> that as long as macros are either called or wrapped in any function,
> they can go anywhere. In other words, the only limitation seems to be
> that runtime code can't use the symbol to which a macro is binded.

Again, sort of. Some macros examine their arguments to decide what
kind of code to produce, and some depend on lazy evaluation, like
"and" or "cond". These macros cannot simply be wrapped in functions.

-Stuart Sierra

[Daniel Jomphe]

Stuart Sierra wrote:

> > - The reader performs a few substitutions, then gives the resulting
> > code to the compiler.
> > - The compiler starts by expanding macros into their substitution.
> > --- Thus, for functions, there's no such thing as macros.
> > --- Thus, all that functions see of macros is what came out of macros'
> > complete expansion.
> > --- Therefore, like Timothy Pratley wrote so well, a function can have
> > for parameter a macro call, but not a macro itself.
>
> All correct up to this point.
>
> > - After macro expansion, the compiler starts compiling all functions
> > in need of being compiled, including the functions modified during
> > macro expansion.
>
> Sort of.  What's really getting compiled are just the function calls.
> The functions themselves were compiled back when they were defined.

Ok, so do we have: read => macro-expand => compile-definitions =>
compile-calls? So that when new code is read, if it contains function
definitions, they'll get compiled before their call(s) is compiled?

Stuart and Konrad respectively said:

> [...] "and" doesn't know how many arguments it's going to get.

> If you look at the definition of "and", you'll see that it takes a
> variable number of arguments and it is recursive. So if you tried to
> expand it before knowing the length of the argument list, you would
> get an infinite loop.

> > > The universal workaround is to define a
> > > function that contains nothing but the macro, such as
> > > (fn [x y] (and x y))
> > > which can also be written in Clojure using the shorthand notation
> > > #(and %1 %2)

Ok!! So *that* would explain why wrapping 'and' *this* way enables it
to be an argument to a function call in (reduce #(and % %2): it
becomes clear how many args are going to be given to 'and's call. Wow,
thanks for such clarity!