loop: iteration variables in the epilogue

[Sam Steingold]

Hi
It appears that the ANSI CL standard does not specify whether loop
iteration variables are available in loop epilogue (let alone what
values they take).

E.g.,

--8<---------------cut here---------------start------------->8---
(loop for c from 0 to 1 for i on '(1 2 3 4 5) finally (return i))
==> (3 4 5) ; CLISP
==> (2 3 4 5) ; SBCL
(loop for c from 0 to 1 for i = '(1 2 3 4 5) then (cdr i) finally (return i))
==> (2 3 4 5) ; CLISP and SBCL
--8<---------------cut here---------------end--------------->8---

1. What do other implementations do?
2. Which behavior is more "reasonable" and why?

Thanks

--
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net http://honestreporting.com
http://iris.org.il http://thereligionofpeace.com http://mideasttruth.com
If you do not move, you will not feel the shackles.

[Barry Margolin]

In article <lzzi7dh...@gnu.org>, Sam Steingold <s...@gnu.org> wrote:

> Hi
> It appears that the ANSI CL standard does not specify whether loop
> iteration variables are available in loop epilogue (let alone what
> values they take).

Probably intentional, since it would be difficult to specify the value
in a way that generalizes well to all the different iteration methods.
So it's left unspecified, and you shouldn't depend on anything in
particular.

E.g. when doing FOR i FROM start TO finish, I wouldn't be surprised if
the value during the epilogue were (+1 finish), similar to the way for
(i = start; i <= finish; i++) works in C-like languages. But FOR i IN
list will probably end with it containing the last element of list. It
will typically depend on whether the iteration method requires a hidden
variable to track the iteration, or it can be done directly on the
iteration variable itself.

>
> E.g.,
>
> --8<---------------cut here---------------start------------->8---
> (loop for c from 0 to 1 for i on '(1 2 3 4 5) finally (return i))
> ==> (3 4 5) ; CLISP
> ==> (2 3 4 5) ; SBCL
> (loop for c from 0 to 1 for i = '(1 2 3 4 5) then (cdr i) finally (return i))
> ==> (2 3 4 5) ; CLISP and SBCL
> --8<---------------cut here---------------end--------------->8---
>
> 1. What do other implementations do?
> 2. Which behavior is more "reasonable" and why?
>
> Thanks

--

Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Robert L.]

[ In "ANSI Common Lisp", Graham makes the following comments: ]

The loop macro was originally designed to help inexperienced
Lisp users write iterative code. Instead of writing Lisp code,
you express your program in a form meant to resemble English,
and this is then translated into Lisp. Unfortunately, loop is
more like English than its designers ever intended: you can
use it in simple cases without quite understanding how it
works, but to understand it in the abstract is almost
impossible.
....
the ANSI standard does not really give a formal specification
of its behavior.
....
The first thing one notices about the loop macro is that it
has syntax. A loop expression contains not subexpressions but
clauses. The clauses are not delimited by parentheses;
instead, each kind has a distinct syntax. In that, loop
resembles traditional Algol-like languages. But the other
distinctive feature of loop, which makes it as unlike Algol as
Lisp, is that the order in which things happen is only
loosely related to the order in which the clauses occur.
....
For such reasons, the use of loop cannot be recommended.





(do ((i 0 (+ i 1))
(xs '(a b c d e f) (cdr xs)))
((= i 2) xs)
(displayln (list i xs)))

(0 (a b c d e f))
(1 (b c d e f))
'(c d e f)

--
When the Israeli bombers and torpedo-planes were sent to attack and destroy the
ship, the Jewish commander, seeing that it was an American vessel, had
misgivings and reported to the High Command, which simply repeated the orders
to attack and sink the Liberty.
I have a graphic from ... the oldest daily newspaper of Israel, ... Haaretz....
The headline reads, "But, sir, it's an American ship." "Never mind. Hit her."
http://archive.org/details/nolies

[Kaz Kylheku]

On 2017-11-23, Barry Margolin <bar...@alum.mit.edu> wrote:
> In article <lzzi7dh...@gnu.org>, Sam Steingold <s...@gnu.org> wrote:
>
>> Hi
>> It appears that the ANSI CL standard does not specify whether loop
>> iteration variables are available in loop epilogue (let alone what
>> values they take).
>
> Probably intentional, since it would be difficult to specify the value
> in a way that generalizes well to all the different iteration methods.
> So it's left unspecified, and you shouldn't depend on anything in
> particular.
>
> E.g. when doing FOR i FROM start TO finish, I wouldn't be surprised if
> the value during the epilogue were (+1 finish), similar to the way for
> (i = start; i <= finish; i++) works in C-like languages. But FOR i IN
> list will probably end with it containing the last element of list. It
> will typically depend on whether the iteration method requires a hidden
> variable to track the iteration, or it can be done directly on the
> iteration variable itself.

The thing is, that in a high level language construct, this kind of
uncertainty is quite bad. It means you can write a harmless loop
construct which executes without diagnostics, which then behaves
differently under another Lisp implementation (even if the CPU and
operating system is the same, ironically, and both implementations have
the same range of fixnum and other implementation choices).

The specification should say that those variables are out of scope of
the epilogue, if their values are uncertain, so that they cannot be
accessed at all.

Failing that, implementations should diagnose (with a warning)
occurrences of variables with uncertain values in the epilogue.

It occurs to me that one easy way to achieve the scoping restriction is
simply to wrap the epilogue forms with symbol-macrolets for each of the
variables, which expand to an error form:)

(symbol-macrolet ((x (sys:loop-scope-warning 'x))
(y (sys:loop-scope-warning 'y)))
epilogue-forms...)

This way, the epilogue-forms don't have to be physically moved out of
the scope where the variables are otherwise visible.

[Pascal J. Bourguignon]

Not really:

(let ((i 42))
(loop for i below 3
finally (return i)))

should return 42, or 3 or 2, or something…


It would be better if it returned 42, but then symbol-macrolet in the
epilogue won't do. More something like:

(block nil
(tagbody
(let ((i 0))
(tagbody
#:again
…
))
#:epilog
(return i)))


--
__Pascal J. Bourguignon
http://www.informatimago.com

[Robert L.]

John Foderaro:

I'm not trying to join a debate on loop. I just wanted to present
the other side of [the issue so that] the intelligent people can
then weigh the arguments on both sides.

I'm not suggesting that loop can be fixed either by adding
parenthesis or coming up with ways of indenting it to make it
understandable. It's a lost cause.


Paul Graham:

I consider Loop one of the worst flaws in CL, and an example
to be borne in mind by both macro writers and language designers.


--
I don't believe in western morality, i.e. don't kill civilians or children....
The only way to fight a moral war is the Jewish way: Destroy their holy sites.
Kill men, women, and children (and cattle). --- Rabbi Manis Friedman
http://archive.org/details/nolies

[Kaz Kylheku]

Here we can still have

(symbol-macrolet ((i (sys:loop-shadow-warning 'i)))
(return i))

Where (sys:loop-shadow-warning 'i) issues a continuable warning and returns
the symbol i.

Of course, the symbol macro expander must not have a tizzy in this case;
but that is easily fixed.

E.g. CLISP doesn't like this right now:

[1]> (symbol-macrolet ((i i)) i)

*** - Program stack overflow. RESET

[Kaz Kylheku]

On 2017-11-23, Kaz Kylheku <217-67...@kylheku.com> wrote:
>
> Here we can still have
>
> (symbol-macrolet ((i (sys:loop-shadow-warning 'i)))
> (return i))
>
> Where (sys:loop-shadow-warning 'i) issues a continuable warning and returns
> the symbol i.
>
> Of course, the symbol macro expander must not have a tizzy in this case;
> but that is easily fixed.

Alas, sys:loop-shadow-warning form has to itself be a macro call, which
yields the symbol i. That macro call be expanded in an environment in
which the symbol-macrolet is no longer visible. That requirement,
unfortunately, is a deal-breaker: won't happen! When i expands to the
(sys:loop-shadow-warning i) form, the code walker will immediately
iterate/recurse on that in the *same* macro environment in which i is
still defined as a symbol macro; so if that macro yields i, an expansion
loop results.

--
TXR Programming Lanuage: http://nongnu.org/txr
Music DIY Mailing List: http://www.kylheku.com/diy
ADA MP-1 Mailing List: http://www.kylheku.com/mp1

[Sam Steingold]

> * Kaz Kylheku <217-679-0842@xlyurxh.pbz> [2017-11-23 19:57:45 +0000]:

>
> The specification should say that those variables are out of scope of
> the epilogue, if their values are uncertain, so that they cannot be
> accessed at all.

Far too much existing user code assumes otherwise.

--
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net

http://no2bds.org http://mideasttruth.com http://think-israel.org
A year spent in artificial intelligence is enough to make one believe in God.

[gabalz]

Hi,

In another discussion (https://sourceforge.net/p/clisp/bugs/667/), Jorg Hohle pointed out chapter 6.1.2.1 of the CL standard: http://clhs.lisp.se/Body/06_aba.htm, which says the following:

"Iteration clauses by themselves do not cause the Loop Facility to return values, but they can be used in conjunction with value-accumulation clauses to return values."

Although not an explicit statement, in my interpretation it means that iteration variables should be available in the loop epilogue.

Best,
`bg`

[Kaz Kylheku]

On 2017-11-26, gabalz <gab...@gmail.com> wrote:
> Hi,
>
> In another discussion (https://sourceforge.net/p/clisp/bugs/667/), Jorg Hohle pointed out chapter 6.1.2.1 of the CL standard: http://clhs.lisp.se/Body/06_aba.htm, which says the following:
>
> "Iteration clauses by themselves do not cause the Loop Facility to
> return values, but they can be used in conjunction with
> value-accumulation clauses to return values."
>
> Although not an explicit statement, in my interpretation it means that
> iteration variables should be available in the loop epilogue.

Not so. What it means is:

for x in '(1 2 3) ;; this is an iteration clause

this clause doesn't make LOOP return anything.

It must be used in conjunction with an accumulation clause, such as:

collecting x ;; this is a value accumulation clause

That one has an implicit location where it accumulates, which is
returned.

The following has an explicit variable y:

collecting x into y
finally (return y) ;; explicit epilogue accessing y

The variables introduced in an accumulation clause clearly need
to be visible in the epilogue to provide access to the accumulated
values. It is not clear that the dummy loop increment variables in
iteration clauses need to be. The above text doesn't help.

Note that if loop increment variables can be accessed in the epilogue,
then loop increment clauses can cause LOOP to return values in
conjunction with epilogue clauses, without being used in conjunction
with accumulation clauses!

[Madhu]

* Sam Steingold <lzzi7dh...@gnu.org> :
Wrote on Thu, 23 Nov 2017 01:22:58 -0500:

> (loop for c from 0 to 1 for i on '(1 2 3 4 5) finally (return i))
> ==> (3 4 5) ; CLISP
> ==> (2 3 4 5) ; SBCL
> (loop for c from 0 to 1 for i = '(1 2 3 4 5) then (cdr i) finally (return i))
> ==> (2 3 4 5) ; CLISP and SBCL
>

> 1. What do other implementations do?

The 3 I checked seem to do what clisp does not do.

> 2. Which behavior is more "reasonable" and why?

I often use iteration variables in the epilogue, and the behaviour I
rely on is that

(loop for c from 0 to 1 for (x . rest) on '(1 2 3 4 5)
finally (return (values x rest)))

=> 2, (3, 4, 5)

I had thought the spec guaranteed me that, and I'm a bit shaken

Note CLISP loop-for-on seems to behave differently if it is followed by
another iteration clause, and behaves like the other implementations

(loop for c from 0 to 1 for i on '(1 2 3 4 5) for (x . rest) = i
finally (return (values i x rest)))

=> (2 3 4 5), 2, (3 4 5)

This difference doesn't seem "reasonable"

[Pascal J. Bourguignon]

It is dangerous (just plainly not conforming) to expect the loop
variables to be available or to have any specific or valid value in the
epilog.

Consider for example:

(loop for i of-type (unsigned-byte 8) from 0 to 255
finally (return (values i (type-of i))))

Most implementations provide a binding with a wrong type,
and sbcl signals a type error.

$ clall -r '(loop for i of-type (unsigned-byte 8) from 0 to 255
finally (return (values i (type-of i))))'
>
Armed Bear Common Lisp --> 256, (INTEGER 0 2147483647)
Clozure Common Lisp --> 256, (INTEGER 0 1152921504606846975)
CLISP --> 256, (INTEGER 0 281474976710655)
ECL --> 256, (INTEGER 256 256)
SBCL The value 256 is not of type (UNSIGNED-BYTE 8)

[Rob Warnock]

Pascal J. Bourguignon <p...@informatimago.com> wrote:

+---------------

| It is dangerous (just plainly not conforming) to expect the loop
| variables to be available or to have any specific or valid value
| in the epilog.

+---------------

Hmmm... That contradicts my reading of the CLHS:

6.1.2.1 Iteration Control
...
All variables are initialized in the loop prologue.
A variable binding has lexical scope unless it is
proclaimed special; thus, by default, the variable
can be accessed only by forms that lie textually
within the loop.

Since any FINALLY clause is still within the LOOP, I would
expect any variables introduced by FOR/AS/AND/etc. to still
be visible in the FINALLY clause.

Now with respect to Sam Steingold's other question, about
the difference in returned values between CLISP and SBCL:

(loop for c from 0 to 1 for i on '(1 2 3 4 5) finally (return i))
==> (3 4 5) ; CLISP
==> (2 3 4 5) ; SBCL

This appears to be a difference in when the variable I is
"stepped", specifically, whether it is assigned the next value
in the iteration *before* or *after* the termination test is
performed. It would appear that CLISP is stepping *before*
the termination test, whereas SBCL (and CMUCL) are stepping
*afterwards*, and then only if the termination test fails.

CLHS 6.1.2.1 "Iteration Control" may also give some guidance
on this, when it says this:

If multiple iteration clauses are used to control iteration,
variable initialization and stepping[1] occur sequentially
by default.

That would imply that CLISP is incorrect in the above example,
since the variable I should not have been stepped again when the
loop terminated on the test of C. Compare with this, which uses
AND instead of FOR to force parallel evaluation in the second
iteration clause:

(loop for c from 0 to 1 AND i on '(1 2 3 4 5) finally (return i))
==> (3 4 5) ; CMUCL [SBCL will probably yield the same]


-Rob

-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <http://rpw3.org/>
San Mateo, CA 94403

[Sam Steingold]

Hi,

> * gabalz <tno...@tznvy.pbz> [2017-11-26 10:24:41 -0800]:

>
> In another discussion (https://sourceforge.net/p/clisp/bugs/667/), Jorg
> Hohle pointed out chapter 6.1.2.1 of the CL standard:
> http://clhs.lisp.se/Body/06_aba.htm, which says the following:
>
> "Iteration clauses by themselves do not cause the Loop Facility to
> return values, but they can be used in conjunction with
> value-accumulation clauses to return values."
>
> Although not an explicit statement, in my interpretation it means that
> iteration variables should be available in the loop epilogue.

In yet another discussion
(https://sourceforge.net/p/clisp/mailman/message/36135304/), the same
Jorg Hohle writes:

> One could make the case that values are not available within the
> epilogue(!), as follows:
> 6.1.2.2 http://clhs.lisp.se/Body/06_abb.htm
> "When a loop form is executed, the local variables are bound and are
> initialized to some value. These local variables exist until loop
> iteration terminates, at which point they cease to exist."
>
> 6.1.1.4 http://clhs.lisp.se/Body/06_aad.htm
> "Loop epilogue
> The loop epilogue contains forms that are executed after iteration
> terminates, such as finally clauses [...]"
>
> Unambiguously, the local variables do not exist anymore within the epilogue!

IOW, loop is not very well specified; it is probably not a very good
idea to use it in portable code.

Thanks.

--
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504
http://steingoldpsychology.com http://www.childpsy.net

http://no2bds.org http://memri.org http://thereligionofpeace.com
You can have it good, soon or cheap. Pick two...

[Kaz Kylheku]

On 2017-11-27, Sam Steingold <s...@gnu.org> wrote:
>> One could make the case that values are not available within the
>> epilogue(!), as follows:
>> 6.1.2.2 http://clhs.lisp.se/Body/06_abb.htm
>> "When a loop form is executed, the local variables are bound and are
>> initialized to some value. These local variables exist until loop
>> iteration terminates, at which point they cease to exist."
>>
>> 6.1.1.4 http://clhs.lisp.se/Body/06_aad.htm
>> "Loop epilogue
>> The loop epilogue contains forms that are executed after iteration
>> terminates, such as finally clauses [...]"
>>
>> Unambiguously, the local variables do not exist anymore within the epilogue!

But the specification cannot reasonably be ruling out usage like:

collect expr into foo
finally (return foo)

So it can't be all variables: iteration vars, not accumulation vars.

> IOW, loop is not very well specified; it is probably not a very good
> idea to use it in portable code.

Or, carry your own loop implementation in the code base.

Add to that "non-lispy clause syntax" and you have compleat explanation of
the loop hatred phenomenon.

[Madhu]

* Sam Steingold <lz8terh...@gnu.org> :
Wrote on Mon, 27 Nov 2017 11:51:46 -0500:

> IOW, loop is not very well specified; it is probably not a very good
> idea to use it in portable code.

This is the wrong conclusion to draw. You would use the well-specified
parts of the loop spec in portable code and avoid the corner cases which
perverse lisp implementations choose to exploit just to create
conflicts.

loop is still more portable, more perspicuous and the loop spec is more
stable than iter or other halfbaked libraries that the loop detractors
roll out which change implementation and semantics on every new release.

[Kaz Kylheku]

On 2017-11-28, Madhu <eno...@meer.net> wrote:
>
> * Sam Steingold <lz8terh...@gnu.org> :
> Wrote on Mon, 27 Nov 2017 11:51:46 -0500:
>
>> IOW, loop is not very well specified; it is probably not a very good
>> idea to use it in portable code.
>
> This is the wrong conclusion to draw. You would use the well-specified
> parts of the loop spec in portable code and avoid the corner cases which
> perverse lisp implementations choose to exploit just to create
> conflicts.

However, it looks like even the basic uses are unspecified, such as:

(loop for x below 10 collecting x finally (return x))

Here we are relying on the loop variable X being still in scope over
the epilogue forms.

6.1.2.2 clearly says "[T]hese local variables exist until loop iteration

terminates, at which point they cease to exist."

When (return x) is being evaluated, iteration has terminated. 6.1.7.2
clearly says: "The finally construct causes the supplied compound-forms
to be evaluated in the loop epilogue after normal iteration terminates."

So it's not simply the case that LOOP is well-defined for normal cases
that occur in everyday code and only murky corner cases are not
well-defined.

Or if it is well-defined, it is not in the manner that developers of the
normal, everyday code like the above, as well as the loop-implementors
who make that code work *think* it is defined.

The reason why such code works, and portably so, is thanks to a shared
understanding between coder and implementor about how it Obviously
Should Work. It probably also has to do with the history of loop as
having consisted of actual implementations.

> loop is still more portable, more perspicuous and the loop spec is more

Though, evidently, some important portability areas are merely
/de facto/.

Which is an English mile better than nothing, sure.

[Marco Antoniotti]

1+

Cheers
--
MA

[Rainer Joswig]

On 2017-11-28 03:50:23 +0000, Kaz Kylheku said:

> On 2017-11-28, Madhu <eno...@meer.net> wrote:
>>
>> * Sam Steingold <lz8terh...@gnu.org> :
>> Wrote on Mon, 27 Nov 2017 11:51:46 -0500:
>>
>>> IOW, loop is not very well specified; it is probably not a very good
>>> idea to use it in portable code.
>>
>> This is the wrong conclusion to draw. You would use the well-specified
>> parts of the loop spec in portable code and avoid the corner cases which
>> perverse lisp implementations choose to exploit just to create
>> conflicts.
>
> However, it looks like even the basic uses are unspecified, such as:
>
> (loop for x below 10 collecting x finally (return x))

That's basic use?

[Sam Steingold]

> * Madhu <rab...@zrre.arg> [2017-11-28 08:18:01 +0500]:

>
> This is the wrong conclusion to draw. You would use the well-specified
> parts of the loop spec in portable code and avoid the corner cases which
> perverse lisp implementations choose to exploit just to create
> conflicts.

DO/DO* is way clearer than loop.

> loop is still more portable, more perspicuous and the loop spec is more
> stable than iter or other halfbaked libraries that the loop detractors
> roll out which change implementation and semantics on every new release.

I have no experience with those.

--
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504

http://steingoldpsychology.com http://www.childpsy.net http://camera.org
http://americancensorship.org http://memri.org http://www.memritv.org
War doesn't determine who's right, just who's left.

[Sam Steingold]

> * Madhu <rab...@zrre.arg> [2017-11-27 08:01:26 +0500]:

>
> * Sam Steingold <lzzi7dh...@gnu.org> :
> Wrote on Thu, 23 Nov 2017 01:22:58 -0500:
>
>> (loop for c from 0 to 1 for i on '(1 2 3 4 5) finally (return i))
>> ==> (3 4 5) ; CLISP
>> ==> (2 3 4 5) ; SBCL
>> (loop for c from 0 to 1 for i = '(1 2 3 4 5) then (cdr i) finally (return i))
>> ==> (2 3 4 5) ; CLISP and SBCL
>>

>> 2. Which behavior is more "reasonable" and why?
>
> I often use iteration variables in the epilogue, and the behaviour I
> rely on is that
>
> (loop for c from 0 to 1 for (x . rest) on '(1 2 3 4 5)
> finally (return (values x rest)))
>
> => 2, (3, 4, 5)
>
> I had thought the spec guaranteed me that, and I'm a bit shaken
>
> Note CLISP loop-for-on seems to behave differently if it is followed by
> another iteration clause, and behaves like the other implementations
>
> (loop for c from 0 to 1 for i on '(1 2 3 4 5) for (x . rest) = i
> finally (return (values i x rest)))
>
> => (2 3 4 5), 2, (3 4 5)
>
> This difference doesn't seem "reasonable"

I am confused, sorry.
What is the "difference"?
X and REST look the same to me (and I is their CONS as expected).

Thanks.

--
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1504

http://steingoldpsychology.com http://www.childpsy.net http://no2bds.org
https://ffii.org http://www.memritv.org http://jij.org http://think-israel.org
Any connection between your reality and mine is purely coincidental.

[Kaz Kylheku]

No, wrong. Please pretend I had written something with a simple
accumulation clause INTO a variable.

[Pascal J. Bourguignon]

Well the into variables will clearly be available in the epilog.

(loop for x below 10

if (oddp x)
collect x into odds
else
collect x into evens
finally (return (values odds evens)))
--> ((1 3 5 7 9) (0 2 4 6 8))

is perfectly conforming.

But:

(loop for x below 10

if (oddp x)
collect x into odds
else
collect x into evens
finally (return (values x odds evens)))
--> (10 (1 3 5 7 9) (0 2 4 6 8))

is not. AFAIU, it could even be an implementation dependent
compilation-time program-error, since x may very well not be in the
epilog scope at all.

[Kaz Kylheku]

It's conforming to everyone's expectations, but not to the ANSI CL
wording which says that the variables EVENS and ODDS are not to be
in scope under the FINALLY clause.

>
> But:
>
> (loop for x below 10
> if (oddp x)
> collect x into odds
> else
> collect x into evens
> finally (return (values x odds evens)))
> --> (10 (1 3 5 7 9) (0 2 4 6 8))
>
> is not. AFAIU, it could even be an implementation dependent
> compilation-time program-error, since x may very well not be in the
> epilog scope at all.

I don't see any text which splits variables into two categories
in this manner, though: accumulation variables (which are to be visible
in the epilogue) and other variables which aren't.

There is explicit wording in the description of COLLECTING and COLLECT
that the user defined variables such as your ODDS and EVENS are
introduced as if by a WITH clause.

The section which describes WITH is the very same one which says that
variables are not in scope after termination!

[Kaz Kylheku]

On 2017-11-29, Kaz Kylheku <217-67...@kylheku.com> wrote:
> The section which describes WITH is the very same one which says that
> variables are not in scope after termination!

In fact, perversely, evcen though "COLLECTING <EXPR>" produces
a machine-generated variable, and procudes an implicit FINALLY clause
which returns the value of that variable, even that variable is not
supposed to be visible!

What that tells us is that there is a contradiction there. The wording
about variables going out of scope upon the termination of the loop is
most likely unintentional (defective).

[Pascal J. Bourguignon]

Yes.

>> But:
>>
>> (loop for x below 10
>> if (oddp x)
>> collect x into odds
>> else
>> collect x into evens
>> finally (return (values x odds evens)))
>> --> (10 (1 3 5 7 9) (0 2 4 6 8))
>>
>> is not. AFAIU, it could even be an implementation dependent
>> compilation-time program-error, since x may very well not be in the
>> epilog scope at all.
>
> I don't see any text which splits variables into two categories
> in this manner, though: accumulation variables (which are to be visible
> in the epilogue) and other variables which aren't.

You are right.

> There is explicit wording in the description of COLLECTING and COLLECT
> that the user defined variables such as your ODDS and EVENS are
> introduced as if by a WITH clause.

Yes.

> The section which describes WITH is the very same one which says that
> variables are not in scope after termination!

Yes.


6.1.3 says "The var argument is bound as if by the construct with. "

and indeed,

6.1.2.2 says "These local variables exist until loop iteration

terminates, at which point they cease to exist."

6.1.7.2 says "The finally construct causes the supplied compound-forms

to be evaluated in the loop epilogue after normal iteration
terminates. "

So to follow formally the specification, we'd have to avoid the finally
clause and write it in the body instead:

(loop for x below 10
if (oddp x)
collect x into odds
else
collect x into evens

when (= 9 x)
return (values odds evens))
;; --> ((1 3 5 7 9) (0 2 4 6 8))

(This one should be conforming!)



It looks like an inconsistency in the specification of LOOP, since those
loop variables are all available in the initially clauses, but not in
the finally clauses for this unfortunate wording "after normal iteration
terminates".

We may think that it wasn't what was intended by the authors.

So it's understandable that implementations extend it, and evaluate the
finally clauses in a scope where those loop variables are available.
Unfortunately, this leaves undefined the binding of the incrementing
loop variables. Shall it be the last value, or the limit, or one beyond
last value, or nil?

[Pascal J. Bourguignon]

Yes.

[Robert L.]

Paul Graham:

I consider Loop one of the worst flaws in CL, and an example
to be borne in mind by both macro writers and language designers.

[ In "ANSI Common Lisp", Graham makes the following comments: ]

The loop macro was originally designed to help inexperienced
Lisp users write iterative code. Instead of writing Lisp code,
you express your program in a form meant to resemble English,
and this is then translated into Lisp. Unfortunately, loop is
more like English than its designers ever intended: you can
use it in simple cases without quite understanding how it
works, but to understand it in the abstract is almost
impossible.
....
the ANSI standard does not really give a formal specification
of its behavior.
....
The first thing one notices about the loop macro is that it
has syntax. A loop expression contains not subexpressions but
clauses. The clauses are not delimited by parentheses;
instead, each kind has a distinct syntax. In that, loop
resembles traditional Algol-like languages. But the other
distinctive feature of loop, which makes it as unlike Algol as
Lisp, is that the order in which things happen is only
loosely related to the order in which the clauses occur.
....
For such reasons, the use of loop cannot be recommended.


--
In Jerusalem, the United Nations (a truly United Nations) will build a Shrine
of the Prophets to serve the federated union of all continents; this will be
the seat of the Supreme Court of Mankind to settle all controversies among the
federated continents, as prophesied by Isaiah.
--- David Ben Gurion (Look, 16 January 1962)
http://archive.org/details/nolies

[Robert L.]

On 11/29/2017, Pascal J. Bourguignon wrote:

> So to follow formally the specification, we'd have to avoid the finally
> clause and write it in the body instead:
>
> (loop for x below 10
> if (oddp x)
> collect x into odds
> else
> collect x into evens
> when (= 9 x)
> return (values odds evens))
> ;; --> ((1 3 5 7 9) (0 2 4 6 8))

Wrong. The actual result:

(1 3 5 7 9)
(0 2 4 6 8)

In Scheme:

(values (range 1 10 2) (range 0 10 2))
===>

(1 3 5 7 9)
(0 2 4 6 8)

--
67% ... of the graduate students at Harvard are Jews. 67%! That's an over
representation which is beyond human belief.
http://archive.org/details/nolies

[Madhu]

* (Rob Warnock) <ovgfir$lsh$1...@dont-email.me> :
Wrote on Mon, 27 Nov 2017 07:41:15 -0000 (UTC):

> Now with respect to Sam Steingold's other question, about
> the difference in returned values between CLISP and SBCL:
>
> (loop for c from 0 to 1 for i on '(1 2 3 4 5) finally (return i))
> ==> (3 4 5) ; CLISP
> ==> (2 3 4 5) ; SBCL
>
> This appears to be a difference in when the variable I is
> "stepped", specifically, whether it is assigned the next value
> in the iteration *before* or *after* the termination test is
> performed. It would appear that CLISP is stepping *before*
> the termination test, whereas SBCL (and CMUCL) are stepping
> *afterwards*, and then only if the termination test fails.
>
> CLHS 6.1.2.1 "Iteration Control" may also give some guidance
> on this, when it says this:
>
> If multiple iteration clauses are used to control iteration,
> variable initialization and stepping[1] occur sequentially
> by default.
>
> That would imply that CLISP is incorrect in the above example,
> since the variable I should not have been stepped again when the
> loop terminated on the test of C. Compare with this, which uses
> AND instead of FOR to force parallel evaluation in the second
> iteration clause:
>
> (loop for c from 0 to 1 AND i on '(1 2 3 4 5) finally (return i))
> ==> (3 4 5) ; CMUCL [SBCL will probably yield the same]
>

6.1.2.2 Has this at the bottom <CLHS/Body/06_aaf.htm>

* Iteration control clauses implicitly perform the following
actions:
-- initialize variables;
-- step variables, generally between each execution of the loop
body;
-- perform termination tests, generally just before the
execution of the loop body.

The variables are stepped between the execution of the loop bodies.
After the termination test, there is no subsequent execution of the loop
body, and if the variable is stepped it is not "between" but "after",
which would be incorrect.

[There is one potential problem with 6.1.1.4, which defines the
"prologue", "loop body", and "epilogue", says

"The loop body contains those forms that are executed during
iteration, including application-specific calculations,
termination tests, and variable stepping[1]."

The footnote [1] is hanging, and this lumps 1) stepping, 2) performance
of termination tests, and the 3) user supplied loop body.

6.1.2.2. above separates these and specifies the order of iteration.]

[Madhu]

* Sam Steingold <lzvahue...@gnu.org> :
Wrote on Mon, 27 Nov 2017 14:15:05 -0500:

>> * Madhu <rab...@zrre.arg> [2017-11-27 08:01:26 +0500]:
>> * Sam Steingold <lzzi7dh...@gnu.org> :
>> Wrote on Thu, 23 Nov 2017 01:22:58 -0500:
>>> (loop for c from 0 to 1 for i on '(1 2 3 4 5) finally (return i))
>>> ==> (3 4 5) ; CLISP
>>> ==> (2 3 4 5) ; SBCL
>>> (loop for c from 0 to 1 for i = '(1 2 3 4 5) then (cdr i) finally (return i))
>>> ==> (2 3 4 5) ; CLISP and SBCL
>>>
>>> 2. Which behavior is more "reasonable" and why?
>>

>> Note CLISP loop-for-on seems to behave differently if it is followed by
>> another iteration clause, and behaves like the other implementations
>>
>> (loop for c from 0 to 1 for i on '(1 2 3 4 5) for (x . rest) = i
>> finally (return (values i x rest)))
>>
>> => (2 3 4 5), 2, (3 4 5)
>>
>> This difference doesn't seem "reasonable"
>
> I am confused, sorry.
> What is the "difference"?
> X and REST look the same to me (and I is their CONS as expected).

The difference is in the value of I
It is (2 3 4 5) not (3 4 5) as you would expect from your example