Scheme vs Common Lisp: war stories

[Matthew Koichi Grimes]

Hi all, I've posted a question regarding Scheme and Common Lisp to
programmers.stackexchange.com, but so far it has gotten few answers.
I'm hoping that the people of comp.lang.lisp might have more insight
to offer on the subject.

The question is as follows:

There are no shortage of vague "Scheme vs Common Lisp" questions on
both StackOverflow and programmers.SE, so I want to make this one more
focused. The question is for people who have coded in both languages:

While coding in Scheme, what specific elements of your Common Lisp
coding experience did you miss most? Or, inversely, while coding in
Common Lisp, what did you miss from coding in Scheme?

I don't necessarily mean just language features. The following are all
valid things to miss, as far as the question is concerned:

* Specific libraries.
* Specific features of development environments like SLIME,
DrRacket, etc.
* Features of particular implementations, like Gambit's ability to
write blocks of C code directly into your Scheme source.
* And of course, language features.

Examples of the sort of answers I'm hoping for:

* "I was trying to implement X in Common Lisp, and if I had
Scheme's first-class continuations, I totally would've just done Y,
but instead I had to do Z, which was more of a pain."
* "Scripting the build process in my Scheme project got
increasingly painful as my source tree grew and I linked in more and
more C libraries. For my next project, I moved back to Common Lisp."
* "I have a large existing C++ codebase, and for me, being able to
embed C++ calls directly in my Gambit Scheme code was totally worth
any shortcomings that Scheme may have vs Common Lisp, even including
lack of SWIG support."

So, I'm hoping for war stories, rather than general sentiments like
"Scheme is a simpler language" etc.

This question was originally posted here:
http://programmers.stackexchange.com/questions/41045/scheme-vs-common-lisp-war-stories

Best,
-- Matt

[Tim Bradshaw]

On 2011-03-24 20:13:02 +0000, Matthew Koichi Grimes said:

> Hi all, I've posted a question regarding Scheme and Common Lisp to
> programmers.stackexchange.com, but so far it has gotten few answers.
> I'm hoping that the people of comp.lang.lisp might have more insight
> to offer on the subject.

The only insight to be had is that there is no insight to be had on
this subject.

[Raffael Cavallaro]

On 2011-03-25 07:00:14 -0400, Tim Bradshaw said:

> The only insight to be had is that there is no insight to be had on
> this subject.

I suspect that Rob Warnock would have some insights on this topic,
moving as he did from working with scheme to working with common lisp.
For example:

<http://groups.google.com/group/comp.lang.lisp/msg/3daa0462a439a768>

Whether his experience is generalizable to all potential scheme or lisp
users is another question; whether the OPs question will lead to more
light than heat is yet another.

warmest regards,

Ralph

--
Raffael Cavallaro

[Alessio Stalla]

On 25 Mar, 15:15, Raffael Cavallaro

<raffaelcavall...@pas.despam.s.il.vous.plait.mac.com> wrote:
> On 2011-03-25 07:00:14 -0400, Tim Bradshaw said:
>
> > The only insight to be had is that there is no insight to be had on
> > this subject.
>
> I suspect that Rob Warnock would have some insights on this topic,
> moving as he did from working with scheme to working with common lisp.
> For example:
>
> <http://groups.google.com/group/comp.lang.lisp/msg/3daa0462a439a768>

[troll]
OMFG Scheme has no READ? Then it's totally not a Lisp!!!
[/troll]

...a little more seriously... really? No READ? That sounds like a real
WTF to me.

[Tim Bradshaw]

On 2011-03-25 19:43:58 +0000, Alessio Stalla said:

> ...a little more seriously... really? No READ? That sounds like a real
> WTF to me.

It has read. I didn't look back further than R3RS (its hard to search
for older reports and get instant hits) but I would expect it has
always been there.

[Larry Coleman]

On Mar 24, 4:13 pm, Matthew Koichi Grimes

<matthew.koichi.gri...@gmail.com> wrote:
> Hi all, I've posted a question regarding Scheme and Common Lisp to
> programmers.stackexchange.com, but so far it has gotten few answers.
> I'm hoping that the people of comp.lang.lisp might have more insight
> to offer on the subject.
>
> The question is as follows:
>
> There are no shortage of vague "Scheme vs Common Lisp" questions on
> both StackOverflow and programmers.SE, so I want to make this one more
> focused. The question is for people who have coded in both languages:
>
> While coding in Scheme, what specific elements of your Common Lisp
> coding experience did you miss most? Or, inversely, while coding in
> Common Lisp, what did you miss from coding in Scheme?
>

You must be new here. Anyone who posts here on a regular basis would
have already complained loudly had they had that experience.

> I don't necessarily mean just language features. The following are all
> valid things to miss, as far as the question is concerned:
>
>     * Specific libraries.
>     * Specific features of development environments like SLIME,
> DrRacket, etc.
>     * Features of particular implementations, like Gambit's ability to
> write blocks of C code directly into your Scheme source.

BTW, ECL can also do this.

>     * And of course, language features.
>
> Examples of the sort of answers I'm hoping for:
>
>     * "I was trying to implement X in Common Lisp, and if I had
> Scheme's first-class continuations, I totally would've just done Y,
> but instead I had to do Z, which was more of a pain."
>     * "Scripting the build process in my Scheme project got
> increasingly painful as my source tree grew and I linked in more and
> more C libraries. For my next project, I moved back to Common Lisp."
>     * "I have a large existing C++ codebase, and for me, being able to
> embed C++ calls directly in my Gambit Scheme code was totally worth
> any shortcomings that Scheme may have vs Common Lisp, even including
> lack of SWIG support."
>
> So, I'm hoping for war stories, rather than general sentiments like
> "Scheme is a simpler language" etc.
>

> This question was originally posted here:http://programmers.stackexchange.com/questions/41045/scheme-vs-common...
>
> Best,
> -- Matt

I'll ask the question I should have asked on programmers.stackexchange
here: Why do you need to know? Do you have a project that you're
considering using Scheme or CL for? Is the concern that you may pick
the wrong one after you get too far into it to switch in a reasonable
amount of time?

If you are considering using Scheme or CL for a major project, my
advice would be to try both. The two languages have very different
philosophies, so it's likely that one or the other will annoy you
pretty quickly. Then you can just go with the other.

Larry

[Alessio Stalla]

Ok. It just seemed too strange if it hadn't it.

[Pascal Costanza]

On 24/03/2011 21:13, Matthew Koichi Grimes wrote:
> Hi all, I've posted a question regarding Scheme and Common Lisp to
> programmers.stackexchange.com, but so far it has gotten few answers.
> I'm hoping that the people of comp.lang.lisp might have more insight
> to offer on the subject.
>
> The question is as follows:
>
> There are no shortage of vague "Scheme vs Common Lisp" questions on
> both StackOverflow and programmers.SE, so I want to make this one more
> focused. The question is for people who have coded in both languages:
>
> While coding in Scheme, what specific elements of your Common Lisp
> coding experience did you miss most? Or, inversely, while coding in
> Common Lisp, what did you miss from coding in Scheme?

When programming in Scheme, I usually get a few nameclashes between
variable names and function names. I almost never get that in Common
Lisp. So when coding in Scheme, I miss the Lisp-2 "feature", and
packages/namespaces. That's actually the most important reason why I
mostly stick to Common Lisp. (There are more, but this is the most
important one.)

I also miss a more extensive set of iteration constructs. Programming
with "just" recursion is like programming in assembly language, I don't
want that.

Defmacro is also missing. There are cases where syntax-rules is too
simplistic, but syntax-case is a far too complicated alternative.
However, that's already a weaker issue, because syntax-rules actually
works surprisingly often.

There doesn't seem to be a good Scheme implementation that offers the
same breadth of support for symmetric multiprocessing as LispWorks does,
which is currently critical for my work.

When programming in Common Lisp, I sometimes miss reflective access to
the current execution stack. I sometimes wish call/cc were available,
although I'm convinced by now that call/cc is the wrong interface. I
think Smalltalk has a much more convincing story in this field.

I definitely miss a reliable way in Common Lisp to request tail call
optimization ("proper" tail recursion).

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

[Rob Warnock]

Alessio Stalla <alessi...@gmail.com> wrote:
+---------------

+---------------

Selective quoting can always be used to distort what someone has said.
What you left out was the introductory phrase:

...what pulled *me* from Scheme to CL were things like these, all
nicely pre-compiled & optimized in any conforming implementation:

followed by a long [and incomplete] list of standard CL features,
*some* of which are also in Scheme, such as READ. [Quiz for the reader:
What other features of Scheme were also mentioned in that list (possibly
with different names in Scheme)? And given what is said below, why do
you think they were in the list in the first place?]

But even for READ, Scheme only offers you the choice of (READ) or
(READ port) [ports are like CL streams], and in particular Scheme READ
*doesn't* offer you a choice for what the EOF-value is -- in Scheme
it's *some* system-defined end of file object that can never be read
in using READ, but it's *not* specified what it is, and it's not
necessarily unique, that is, (AND (EOF-OBJECT? OBJ1) (EOF-OBJECT? OBJ2))
does *not* imply that (EQ? OBJ1 OBJ2) or even (EQV? OBJ1 OBJ2).
[And since Scheme has no exception system, EOF on READ will never
throw an exception.]

So, no, Scheme *doesn't* have CL's READ. Nor READ-FROM-STRING [though
the latter can be synthesized by WRITE-ing the string to a temp file
and then READ-ing from it]. Nor does Scheme's STRING->NUMBER have
CL's PARSE-INTEGER's :START/:END/:RADIX/:JUNK-ALLOWED options [though
it *does* read floats].

And that is really the main point of that 2007 posting -- that in CL
much more of the edge-case behavior is *defined* [though certainly not
all, as we know], *plus* much more of it is tweakable by the programmer
using *standard* options [e.g., :KEY/:TEST/:START/:END and the like] that
are reasonably pervasive [though, again, not uniformly nor completely]
throughout the language.

-Rob

-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <http://rpw3.org/>
San Mateo, CA 94403

[Bakul Shah]

On 3/25/11 6:28 PM, Rob Warnock wrote:
>
> ...what pulled *me* from Scheme to CL were things like these, all

> nicely pre-compiled& optimized in any conforming implementation:

This situation has improved since then.

> But even for READ, Scheme only offers you the choice of (READ) or
> (READ port) [ports are like CL streams], and in particular Scheme READ
> *doesn't* offer you a choice for what the EOF-value is -- in Scheme
> it's *some* system-defined end of file object that can never be read
> in using READ, but it's *not* specified what it is, and it's not
> necessarily unique, that is, (AND (EOF-OBJECT? OBJ1) (EOF-OBJECT? OBJ2))
> does *not* imply that (EQ? OBJ1 OBJ2) or even (EQV? OBJ1 OBJ2).
> [And since Scheme has no exception system, EOF on READ will never
> throw an exception.]

Haven't found a need for any eof-object comparison or uniqueness. What
I don't like is that it is considered an object.

> So, no, Scheme *doesn't* have CL's READ. Nor READ-FROM-STRING [though
> the latter can be synthesized by WRITE-ing the string to a temp file
> and then READ-ing from it]. Nor does Scheme's STRING->NUMBER have
> CL's PARSE-INTEGER's :START/:END/:RADIX/:JUNK-ALLOWED options [though
> it *does* read floats].

SRFI-6 adds string-ports.

> And that is really the main point of that 2007 posting -- that in CL
> much more of the edge-case behavior is *defined* [though certainly not
> all, as we know], *plus* much more of it is tweakable by the programmer
> using *standard* options [e.g., :KEY/:TEST/:START/:END and the like] that
> are reasonably pervasive [though, again, not uniformly nor completely]
> throughout the language.

Scheme certainly doesn't have these many options but what edge cases are
insufficiently defined in Scheme? [A serious question]

[Tim Bradshaw]

On 2011-03-26 01:28:37 +0000, Rob Warnock said:

> [And since Scheme has no exception system, EOF on READ will never
> throw an exception.]

As of R6RS at least, Scheme does have an exception system.

[WJ]

Bakul Shah wrote:

> > and then READ-ing from it]. Nor does Scheme's STRING->NUMBER have
> > CL's PARSE-INTEGER's :START/:END/:RADIX/:JUNK-ALLOWED options [though

> > it does read floats].
>
> SRFI-6 adds string-ports.

1:=> (string->number "100" 16)
256

[TheFlyingDutchman]

Someone asks for a comparison of Scheme and Common Lisp and your only
contribution is this??

[William D Clinger]

Rob Warnock wrote:
> [Quiz for the reader:
> What other features of Scheme were also mentioned in that list (possibly
> with different names in Scheme)? And given what is said below, why do
> you think they were in the list in the first place?]

Okay, I'll attempt to answer Rob's first question but not his
second, using the following key:

not available (Scheme offers no comparable feature)
IEEE (it's part of IEEE/ANSI Scheme)
R5RS (it's part of R5RS Scheme)
R6RS (it's part of R6RS Scheme)
SRFI-n (it's part of SRFI-n)
EASY (base) (fairly easy to build from the named base)

- DEFMACRO
EASY (R6RS)
EASY (SRFI-72)
SRFI-96
- The almost-universal provision of :START/:END (and :START1/:END1/
:START2/:END2, when appropriate) keyword args on sequence functions.
not available
- MAKE-HASH-TABLE, GETHASH, (SETF GETHASH), & REMHASH
R6RS
SRFI-69
- Specialized vectors
SRFI-4 (homogeneous numeric vectors only)
SRFI-66 (octet vectors only)
R6RS (bytevectors only)
- multi-dimensional arrays
SRFI-25
- POSITION, MISMATCH, SEARCH, & SUBSEQ
SRFI-1 (for lists only)
EASY (SRFI-1)
- PARSE-INTEGER (in all its glory)
EASY (IEEE) (already provides number->string)
EASY (R5RS) (already provides number->string)
EASY (R6RS) (already provides number->string)
- CONCATENATE
EASY (IEEE) (already provided for lists, strings)
EASY (R5RS) (already provided for lists, strings)
EASY (R6RS) (already provided for lists, strings)
- COERCE
EASY (R5RS)
EASY (R6RS)
- MAP
EASY (IEEE) (already provided for lists)
EASY (R5RS) (already provided for lists)
EASY (R6RS) (already provided for lists, vectors)
- READ-FROM-STRING
EASY (R6RS) (already provides string ports)
EASY (SRFI-6) (already provides string ports)
- WITH-INPUT-FROM-STRING
not available
- FORMAT
SRFI-28
SRFI-48
SRFI-54
- WITH-OUTPUT-TO-STRING
not available
- LOOP
not available
- READ
IEEE
R5RS
R6RS
- Reader macros
not available

Will

[WJ]

William D. Clinger wrote:

> - DEFMACRO
> EASY (R6RS)
> EASY (SRFI-72)
> SRFI-96

Both Bigloo and Guile have define-macro, which is equivalent to
defmacro.

[William D Clinger]

Yes, many implementations of Scheme go well beyond
the listed features. Since the part of Rob Warnock's
list that I addressed was limited to features provided
by every implementation of the ANSI standard for Common
Lisp, I thought it only fair to limit my list to features
provided by the IEEE/ANSI standard for Scheme or by one
of the more widely implemented de facto standards that
have built upon the official IEEE/ANSI standard.

Will

[fortunatus]

When programming in Common Lisp I miss in a big way having a super-
simple GUI library, like came with PLT Scheme. ("PLT Scheme is now
Racket"...) I use a Lisp web server instead, which has its own
benefits and drawbacks.

When programming in Common Lisp I miss the "Lisp 1" style of Scheme
where function values and continuations are stored in the same symbol
slot as other data values.

When programming in Common Lisp I do miss continuations sometimes.

When programming in either Scheme or Common Lisp I am happy and
relieved to have a full arithmetic stack, especially unbounded
integers and unbounded rationals. (Corollary: I miss that when
working in Emacs Lisp!)

When programming in Scheme I miss structures in a big way.

When programming in Scheme I miss generic functions.

When programming in Scheme I miss the richer set of standard functions
from Common Lisp - I hate reliance on SRFI's, though I'm sure glad
they exist!

When programming in Scheme I miss the portability of Common Lisp - in
Scheme I generally must rely implementation details (such as SRFI's)
so I end up sticking with one Scheme. I use several Common Lisps and
manage porting code pretty well.

When programming in Scheme I miss the more general "dirty" macros of
Common Lisp.

When programming in Scheme I miss the more general loading and
compiling (system building) of Common Lisp.

When programming in Scheme I enjoy using recursion as my interative
means(since DO is kind of crazy). But I miss LOOP because I end up
writing a lot more code at a more basic, decomposed level.

[Paul Rubin]

fortunatus <daniel....@excite.com> writes:
> When programming in Scheme I enjoy using recursion as my interative
> means(since DO is kind of crazy). But I miss LOOP because I end up
> writing a lot more code at a more basic, decomposed level.

Does SRFI-1 mostly take care of that, by letting you write most loops as
folds or something similar? In Haskell at least, loops and naked
recursion are both code smells. There is usually a combinator (i.e.
something like map or fold) that does what you want to use the recursion
for.

[WJ]

fortunatus wrote:

> When programming in Scheme I miss structures in a big way.

SRFI-9 provides records.

> When programming in Scheme I miss the more general "dirty" macros of
> Common Lisp.

Bigloo, Racket, and Guile have defmacro or define-macro.

[WJ]

fortunatus wrote:

> When programming in Scheme I enjoy using recursion as my interative
> means(since DO is kind of crazy). But I miss LOOP because I end up
> writing a lot more code at a more basic, decomposed level.

http://groups.google.com/group/comp.lang.lisp/msg/3b9391273aceb7a5

[fortunatus]

NEAT!!

[Raymond Wiker]

fortunatus <daniel....@excite.com> writes:

Except if you care about efficiency, or actually know how to use
loop and format...

[fortunatus]

I thank the gods for SRFIs - actually I thank a well organized
community!

[Blake McBride]

Problems with Scheme:

Contradiction in separation of NIL and #f

Lisp has long used NIL as "false" or #f. This was very convenient
since lisp is a list processing language, traversing through a list is
very often done. Having NIL and #f equal means one could use:

(while lst ...
or
(if lst ...

With the separation of NIL and #f in scheme one is forced into:

(while (not (null? lst)) ...
or
(if (not (null? lst)) ...

The argument for separating NIL and false was in the name of clarity
as opposed to depending on a coincidence. It would be much more clear
(to a beginner?) what was going on. (Although, if you really want to
be clear, perhaps we should all program in ADA. ADA is a language in
which virtually every line of code has to do with clarity and
virtually none has to do with solving the problem at hand!)

The problem is that scheme has eliminated, rather arbitrarily, one
instance of dependence on a trick but left so many others that it
renders the NIL / false separation no more than a constant annoyance.
For example, if NIL and #f are kept apart, why allow anything except
#t equal true? Yes, I know it is very convenient (just like #f and
NIL being equal), but it is also "unclear". Having:

(if "abc" 6 7)

return 6 is absolute nonsense. Scheme has chosen clarity and
correctness in one instance and convenience in another. It is very
inconsistent. If you wanted to be truly consistent:

(+ "abc" 6)

returns a type error. So:

(if "abc" 6 7)

should also return a type error. The same is true for scheme's:

and
or
not
cond
etc.

Look, ADA already has the market on clarity and correctness. Lisp and
scheme have the elegant, expressiveness, and power market. Why
arbitrarily go down the clarity path with the separation of NIL and #f
- it is a mistake. The only other "consistent" choice is to separate
#t from all else, make all logic operations require #t or #f, and make
scheme the ADA of lisp.


----------------------------------------------------------------------

Unspecified returns

Scheme has many functions which have "unspecified" returns. This is
somewhat arbitrary, and runs contrary to the whole s-expression style
of programming. For example, if you have two variables you want to
set to the same value it would be nice to say:

(set! abc (set! def 55))

like you can in Common Lisp. But in Scheme set! doesn't return a
defined result so you are forced into:

(set! abc 55)
(set! def abc)

In Scheme you would have to write:

(set! abc 55)
(if (> def abc)
......

instead of:

(if (> def (set! abc 55))
.....

If you think it's no big deal then why the heck does "cond" and "if"
return a result? It doesn't in most other languages. The reason "if"
returns a result is to take advantage of the nested s-expression style
of programming that lends itself to lisp style languages rather than a
line by line style of programming like Fortran. If Scheme were
consistent then either "if" would return no result or set! would.

Worse yet, there are many function Scheme which return valid responses
most of the time and "unspecified" values at other times. You don't
know whether you are coming or going.

What do you do if a "case" falls off and returns "unspecified"?
Wouldn't nil be a little more predictable?

The same is true for eqv?, eq?, equal?, set-cdr!, set-car!, memq,
assq, string-set!, pair?, display, etc......

The bottom line is that describing the return of a function as
"unspecified", especially only under certain circumstances, leaves
programs which ultimately run in "unspecified" ways. All functions
should either return a specified, predictable result or they should
signal an error, period.

I would further add that Scheme should define argument evaluation
order. Compilers can be written to support a particular order just as
well as another order. This is no argument for unspecified argument
evaluations order. Having the argument evaluation order unspecified
is a huge and totally unnecessary opportunity for porting bugs.

The issues here are consistency, predictability, and convenience.

----------------------------------------------------------------------

Macro System

My complaint here is that the Scheme macro system is so complicated
and difficult to use that most of the power of lisp macros is lost. I
also contend that it has important limitations.

Scheme's macro system is theoretically pure in some sense related to
capturing of variables. But for that advantage you end up with:

1. While the Scheme macro system is simple and easy to use for simple
macros it quickly becomes inordinately complicated for more complex
macros. I believe that a very large percentage of Common Lisp
programmers are able to create and understand moderately complex macros
with a lot of ease. On the other hand I think the number of people in the
world who understand complex macros in Scheme are extremely rare. Fully
understanding Scheme's macro facility basically requires a Phd.

You can pick up virtually any language, learn a "Hello World" program
and then pick up various pieces as you begin to understand the
language incrementally. Each particular piece typically takes a
couple of minutes to understand. Because of their abstraction, macros
might take a bit longer. You could probably get a pretty good working
knowledge of lisp macros in a day. Scheme macros, on the other hand,
in all except the simplest cases, are unbelievably complex. I doubt
you could get a good handle on it in less than a week. Scheme macros
are too complex.

If you doubt my "complexity" arguments try taking a look at the
portable Scheme macro system available on the net. I haven't seen
much that was more complicated that that in my life.

2. I understand why you don't want variable capture - usually.
Sometimes you do. Until R6 Scheme had no standard way of capturing
variables. It seems like they've added this capability in R6. One
problem is that it is unbelievably complicated to use. You really have
to understand what is going on under the covers. Common Lisp makes
this sort of thing easy and straight forward.

3. I am under the impression that there are levels of complexity in
macros that the Scheme macro system flat out can't handle - at least
not in a portable way. In my own experience trying to use the Scheme
macro system I often and quickly hit a wall in which the complexity of
the solution exceeds my understanding. In nearly every case I asked
an expert and I almost always get one of two answers. They either
tell me to use some implementation specific facility or they tell me
it can't be done with it. I've never experienced this with Common
Lisp macros.

4. Understand that Lisp macros suffer from variable capture. It's
amazing how we've lived with this for 30+ years now. Having to deal
with variable capture is a small price to pay for the power, ease of
use, and flexibility of Common Lisp macros.

The bottom line is that the Scheme macro system is more of an
experiment in complexity than a useful tool.

----------------------------------------------------------------------

OO and Modules

The first is Scheme's lack of an object oriented system (OO). In
today's world OO programming is as fundamental to programming as
"while" loops are instead of goto statements. OO concepts are
probably the biggest and most important addition to computer languages
since structured programming (multithreading is next!!). Not having a
standard OO facility renders Scheme somewhat archaic.

And, if/when one is added it has got to be powerful - like CLOS. It
has got to have multiple inheritance. It has got to have a full MOP.

I started to write a CLOS like OO system in very portable Scheme. I
am about 80% done but haven't worked on it in a long time. I get
discouraged about adding a good and portable OO system to Scheme
because of Scheme's many other issues.

The second issue has to do with a standard module system. I know R6
has defines a library system. This is fine. The problem is that
virtually none of the Scheme implementations has implemented it and
virtually all of the Scheme implementers has publicly stated that
they will not support R6RS! Where exactly does that leave us?

----------------------------------------------------------------------

In conclusion, Scheme has, IMO, many important shortcomings. I
understand that I don't set the standards and these are just basically
my opinions. Although I believe these shortcomings are relatively
easy to fix, the real problem is the political element. No matter
what you think there is always someone out there who thinks the
opposite. The net result is that language development ends up
unnecessarily slow.

Common Lisp, again IMO, did a lot of the right things. There are,
however, two significant problems with Common Lisp as follows.

One big technical error (IMO) was the LISPn thing. I know a bunch of
you guys want to jump on me about this with all kinds of technical BS
trying to rationalize your view. The truth of the matter is that the
highest and most respected experts on Common Lisp are utterly unclear
about the issue. So how does that make you an expert? You are more
of a fan club than a technical expert (again read the articles from
the authors of Common Lisp). The real truth is that the number one
reason Common Lisp is LISPn rather than LISP1 is historical and for
purposes of backward compatibility (all of which is hardly applicable
anymore). It isn't technically better. The experts were able to
point out trade-offs with either design. They even stated that LISP1
encouraged more elegant programming style.

The second mistake of Common Lisp is not requiring support for tail
recursion without taking up stack space. This causes the use of
inelegant loops when the size of the data is large or unknown in order
not to fill the stack. I know some Common Lisps support this, but
unless it is part of the standard and something that can be relied
upon, it shouldn't be depended on.

It is easy to create a Common Lisp package that merges function and
variable name spaces without incurring runtime efficiency (I've done
it). Fixing the recursion issue is a significant problem. I'd love
to see a new ANSI Common Lisp that just addresses the recursion issue.
Changing the LISPn issue would be such a radical change that Common Lisp
would be rendered a new language. Using a package to merge the function
and variable namespaces is sufficient for those who want it.

Just some opinions.

Blake McBride

[Pascal J. Bourguignon]

Blake McBride <bl...@arahant.com> writes:

> Problems with Scheme:

Perhaps it's not too late to tell to the r7rs commitee?

Or if it's too late, be sure to argue all those point on the r8rs
commitee.

In the mean time, there's Common Lisp, and you definitely know why I
prefer it over scheme ;-)

--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

[Alan Bawden]

Blake McBride <bl...@arahant.com> writes:

> Problems with Scheme:
>
> Contradiction in separation of NIL and #f

> [...]

When you switch from programming in Common Lisp to programming in Scheme
you quickly find that this doesn't really make all that much of a
difference. Sure, there are a bunch of Common Lisp cliches that you have
to learn to stop using. But there are a bunch of new cliches that only
work in Scheme that you can now start using. In the long run, it's nothing
that prevents you from getting work done. It's just different.

> ----------------------------------------------------------------------
>
> Unspecified returns
>
> Scheme has many functions which have "unspecified" returns.[...]

Mostly because Scheme was designed by a committee that couldn't agree on
what was the right thing to return. For example, many Scheme
implementations return the new value of the variable in a set! expression
(as you advocate), but at least one major Scheme implementation returns the
-old- value of the variable in that case! So there was an argument. The
compromise: leave it unspecified.

Common Lisp has its own set of unspecified behaviors. As does any other
language I have ever used that was defined by a committee.

But it's nothing that prevents you from getting work done, and the good
news is that someday some future version of the language standard (Common
Lisp, Scheme, C, ...) can define the behavior.

> ----------------------------------------------------------------------
>
> Macro System
>
> My complaint here is that the Scheme macro system is so complicated
> and difficult to use that most of the power of lisp macros is lost. I
> also contend that it has important limitations.

>[...]

Indeed, it's far more complicated that it needs to be. It makes easy
problems trivial, and hard problems nearly impossible.

The tragedy is that it didn't have to be that bad. There are techniques
that could have worked just as well -- techniques that look almost exactly
like Common Lisp-style defmacro-with-backquote macros, and that are just as
flexible.

But my side lost that argument. Oh well.

> ----------------------------------------------------------------------
>
> OO and Modules
> [...]

When I switched from working in Scheme back to working in Common Lisp, the
biggest plus was being able to use a decent object system again. Scheme's
lack of an object system really is something that prevents you from getting
work done. (Scheme also lacked a standard record structure facility until
R6RS -- how crazy is that?)

I quit using Scheme before experiencing the R6RS module system, so I can't
really judge it. But I can say that the Common Lisp's excuse for a "module
system" is no joy to work with. But you can learn how to use it in ways
that don't keep you from getting work done.

> Common Lisp, again IMO, did a lot of the right things. There are,
> however, two significant problems with Common Lisp as follows.
>
> One big technical error (IMO) was the LISPn thing. I know a bunch of
> you guys want to jump on me about this with all kinds of technical BS
> trying to rationalize your view. The truth of the matter is that the
> highest and most respected experts on Common Lisp are utterly unclear
> about the issue. So how does that make you an expert? You are more
> of a fan club than a technical expert (again read the articles from
> the authors of Common Lisp). The real truth is that the number one
> reason Common Lisp is LISPn rather than LISP1 is historical and for
> purposes of backward compatibility (all of which is hardly applicable
> anymore). It isn't technically better. The experts were able to
> point out trade-offs with either design. They even stated that LISP1
> encouraged more elegant programming style.

True.

This was the biggest downside to returning to Common Lisp after years of
programming in Scheme. The fact that the language has to maintain parallel
mechanisms for working in two separate namespaces is a huge source of
unnecessary complexity.

> The second mistake of Common Lisp is not requiring support for tail
> recursion without taking up stack space. This causes the use of
> inelegant loops when the size of the data is large or unknown in order
> not to fill the stack. I know some Common Lisps support this, but
> unless it is part of the standard and something that can be relied
> upon, it shouldn't be depended on.

Also true. But for some reason this doesn't bother me all that much.
Perhaps because for several years I used a Scheme implementation
(Scheme->C) that failed to do tail recursion correctly, so I learned not to
really depend on it even in Scheme. But I did miss it occasionally, and I
do miss it in Common Lisp occasionally as well. But again, it doesn't keep
me from getting work done.

> [...]
> Just some opinions.

And pretty much all on the mark. Except I wouldn't bother with the #F
vs. '() thing -- that one really doesn't matter compared to the rest.

- Alan

[D Herring]

On 06/17/2012 12:11 AM, Alan Bawden wrote:

> And pretty much all on the mark. Except I wouldn't bother with the #F
> vs. '() thing -- that one really doesn't matter compared to the rest.

My biggest beef with #F vs '() is that it really complicates
interactions between CL and Scheme.

Here's a page on the topic.
http://www.red-bean.com/guile/notes/emacs-lisp.html

- Daniel

[D Herring]

On 06/17/2012 12:11 AM, Alan Bawden wrote:

> Blake McBride<bl...@arahant.com> writes:

>> Macro System
>>
>> My complaint here is that the Scheme macro system is so complicated
>> and difficult to use that most of the power of lisp macros is lost. I
>> also contend that it has important limitations.
>> [...]
>
> Indeed, it's far more complicated that it needs to be. It makes easy
> problems trivial, and hard problems nearly impossible.
>
> The tragedy is that it didn't have to be that bad. There are techniques
> that could have worked just as well -- techniques that look almost exactly
> like Common Lisp-style defmacro-with-backquote macros, and that are just as
> flexible.
>
> But my side lost that argument. Oh well.

Alan, could you provide some names of/references to these techniques?

I'd like to learn about them.

Thanks,
Daniel

[Pascal Costanza]

On 17/06/2012 06:11, Alan Bawden wrote:

> When I switched from working in Scheme back to working in Common Lisp, the
> biggest plus was being able to use a decent object system again.

I'm curious: What made you switch from Scheme to Common Lisp? (Feel free
to respond privately, if you don't feel like responding in public.)

Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

The views expressed are my own, and not those of my employer.

[Pascal Costanza]

Alan was a proponent of syntactic closures in the past. You can find
references here: http://library.readscheme.org/page3.html

Another simple macro facility that is similar to defmacro, but adds
support for "manual" hygiene, is based on explicit renaming and was
described by William Clinger.

Lisp-n is technically superior to Lisp-1, because it makes most
accidental name clashes go away without the need to make the language
more complicated in any way. (Name clashes also occur outside of macro
programming, that's why you rely more on naming conventions in Scheme
than in Common Lisp to avoid them, because there is no language
mechanism that could help you there.)

[Alan Bawden]

Pascal Costanza <p...@p-cos.net> writes:

> On 17/06/2012 08:49, D Herring wrote:
>> On 06/17/2012 12:11 AM, Alan Bawden wrote:
>>> There are techniques
>>> that could have worked just as well -- techniques that look almost
>>> exactly
>>> like Common Lisp-style defmacro-with-backquote macros, and that are
>>> just as
>>> flexible.
>>>
>>> But my side lost that argument. Oh well.
>>
>> Alan, could you provide some names of/references to these techniques?
>>
>> I'd like to learn about them.
>
> Alan was a proponent of syntactic closures in the past. You can find
> references here: http://library.readscheme.org/page3.html

The syntactic closure was an interesting idea, but ultimately not the right
way to go. It was still too complicated.

> Another simple macro facility that is similar to defmacro, but adds support
> for "manual" hygiene, is based on explicit renaming and was described by
> William Clinger.

Bingo. Although it could be even simpler than Clinger's published
description. Especially if you aren't trying to support the "literal"
comparison mandated by Scheme's syntax-rules special form.

In fact, we could add explicit renaming macros to Common Lisp as an almost
completely upward compatible change -- nobody who didn't want to use them
would notice any change at all.

> Lisp-n is technically superior to Lisp-1, because it makes most accidental
> name clashes go away without the need to make the language more complicated
> in any way. (Name clashes also occur outside of macro programming, that's
> why you rely more on naming conventions in Scheme than in Common Lisp to
> avoid them, because there is no language mechanism that could help you
> there.)

It only makes "most accidental name clashes go away" because people don't
often use FLET, LABELS or MACROLET. There are actually three things that
enable us to write Common Lisp macros without namespace problems:
careful use of GENSYM, careful use of packages and the fact that the
language prohibits rebinding any of the functions or macros defined in the
COMMON-LISP package.

- Alan

[Pascal Costanza]

On 17/06/2012 22:03, Alan Bawden wrote:
> Pascal Costanza <p...@p-cos.net> writes:
>
>> On 17/06/2012 08:49, D Herring wrote:
>>> On 06/17/2012 12:11 AM, Alan Bawden wrote:
>>>> There are techniques
>>>> that could have worked just as well -- techniques that look almost
>>>> exactly
>>>> like Common Lisp-style defmacro-with-backquote macros, and that are
>>>> just as
>>>> flexible.
>>>>
>>>> But my side lost that argument. Oh well.
>>>
>>> Alan, could you provide some names of/references to these techniques?
>>>
>>> I'd like to learn about them.
>>
>> Alan was a proponent of syntactic closures in the past. You can find
>> references here: http://library.readscheme.org/page3.html
>
> The syntactic closure was an interesting idea, but ultimately not the right
> way to go. It was still too complicated.

OK, interesting to hear that from you.

>> Another simple macro facility that is similar to defmacro, but adds support
>> for "manual" hygiene, is based on explicit renaming and was described by
>> William Clinger.
>
> Bingo. Although it could be even simpler than Clinger's published
> description. Especially if you aren't trying to support the "literal"
> comparison mandated by Scheme's syntax-rules special form.

I think the literal comparison can be important. I have seen it
necessary when translating the code from
http://www.cs.indiana.edu/~dfried/ooo.pdf to a simpler macro system.
(But I also believe it's a pathological case.)

> In fact, we could add explicit renaming macros to Common Lisp as an almost
> completely upward compatible change -- nobody who didn't want to use them
> would notice any change at all.

I know. I believe that
http://www.jucs.org/doi?doi=10.3217/jucs-016-02-0271 is very close in
providing a portable implementation.

>> Lisp-n is technically superior to Lisp-1, because it makes most accidental
>> name clashes go away without the need to make the language more complicated
>> in any way. (Name clashes also occur outside of macro programming, that's
>> why you rely more on naming conventions in Scheme than in Common Lisp to
>> avoid them, because there is no language mechanism that could help you
>> there.)
>
> It only makes "most accidental name clashes go away" because people don't
> often use FLET, LABELS or MACROLET. There are actually three things that
> enable us to write Common Lisp macros without namespace problems:
> careful use of GENSYM, careful use of packages and the fact that the
> language prohibits rebinding any of the functions or macros defined in the
> COMMON-LISP package.

Exactly. And since you control the names you define in FLET, LABELS and
MACROLET, you're pretty much safe all the time.

In Scheme, I have name clashes a lot more often, because it happens
quite often that I want to reuse a name for a function or a class as a
'plain' variable (in terms of the binding). And that is independent of
macro programming. In Common Lisp, I don't have them, because it's a
Lisp-n. I regard this a much simpler solution.

But I know this is just a subjective assessment...

[Kaz Kylheku]

On 2012-06-17, Alan Bawden <al...@scooby-doo.csail.mit.edu> wrote:
> Also true. But for some reason this doesn't bother me all that much.
> Perhaps because for several years I used a Scheme implementation
> (Scheme->C) that failed to do tail recursion correctly, so I learned not to
> really depend on it even in Scheme. But I did miss it occasionally, and I
> do miss it in Common Lisp occasionally as well. But again, it doesn't keep
> me from getting work done.

http://www.kylheku.com/cgit/lisp-snippets/tree/tail-recursion.lisp

When a program critically depends on tail recursion, it is in fact a software
defect when tail recursion does not occur.

So this tail position must be carefully preserved as the program is maintained,
lest the tail call should inadvertently turn into a regular call.

How do you test for it? Say I have some big program which depends on tail
recursion in hundreds of places. What if tail recursion isn't happening in some
of them?

Ideally we would like to be able to annotate the tail calls to express the idea
"this call must be tail call; if it isn't, then please emit a diagnostic during
compilation".

With the above macros, we achieve the next best thing: tail calls which are
always tail calls regardless of their position. (If the program appears to use
the return value of a tail call, tough luck).

TAILPROG compiles down to TAGBODY and GO, via the intermediate ARGTAGS macro.
Many kinds of bodies which use LABELS can be easily converted to TAILPROG
just be replacing the LABELS symbol with TAILPROG. But TAILPROG "functions"
are not actually functions, so forget about trying to indirect upon them
with (FUNCTION ...) and application.

DEFTAIL provides tail-calling among toplevel functions. It's based on a
dispatch loop and nonlocal dynamic control transfers, so it probably
won't win any races against low-level implementations of this sort of
thing which work at the the code generation level. It does the job, though.

--
If you ever need any coding done, I'm your goto man!

[Kaz Kylheku]

On 2012-06-17, Alan Bawden <al...@scooby-doo.csail.mit.edu> wrote:

> Blake McBride <bl...@arahant.com> writes:
>
>> Problems with Scheme:
>>
>> Contradiction in separation of NIL and #f
>> [...]
>
> When you switch from programming in Common Lisp to programming in Scheme
> you quickly find that this doesn't really make all that much of a
> difference. Sure, there are a bunch of Common Lisp cliches that you have
> to learn to stop using.

But what if your code is 95% cliche? Oops. :)

Undefined return values and unspecified orders of argument evaluation leave a
bad taste in my mouth.

(The latter is for more direct compiling to C, isn't it.)

[Johnn Wei]

在 2011年3月25日星期五UTC+8上午4时13分02秒，Matthew Koichi Grimes写道：
> Hi all, I've posted a question regarding Scheme and Common Lisp to
> programmers.stackexchange.com, but so far it has gotten few answers.
> I'm hoping that the people of comp.lang.lisp might have more insight
> to offer on the subject.
>
> The question is as follows:
>
> There are no shortage of vague "Scheme vs Common Lisp" questions on
> both StackOverflow and programmers.SE, so I want to make this one more
> focused. The question is for people who have coded in both languages:
>
> While coding in Scheme, what specific elements of your Common Lisp
> coding experience did you miss most? Or, inversely, while coding in
> Common Lisp, what did you miss from coding in Scheme?
>
> I don't necessarily mean just language features. The following are all
> valid things to miss, as far as the question is concerned:
>
> * Specific libraries.
> * Specific features of development environments like SLIME,
> DrRacket, etc.
> * Features of particular implementations, like Gambit's ability to
> write blocks of C code directly into your Scheme source.
> * And of course, language features.
>
> Examples of the sort of answers I'm hoping for:
>
> * "I was trying to implement X in Common Lisp, and if I had
> Scheme's first-class continuations, I totally would've just done Y,
> but instead I had to do Z, which was more of a pain."
> * "Scripting the build process in my Scheme project got
> increasingly painful as my source tree grew and I linked in more and
> more C libraries. For my next project, I moved back to Common Lisp."
> * "I have a large existing C++ codebase, and for me, being able to
> embed C++ calls directly in my Gambit Scheme code was totally worth
> any shortcomings that Scheme may have vs Common Lisp, even including
> lack of SWIG support."
>
> So, I'm hoping for war stories, rather than general sentiments like
> "Scheme is a simpler language" etc.
>
>
> This question was originally posted here:
> http://programmers.stackexchange.com/questions/41045/scheme-vs-common-lisp-war-stories
>
> Best,
> -- Matt

Test 测试中文字符

[Blake McBride]

Very, very nice! Thanks!

Blake McBride

[Nathan]

Well, I'm certainly no expert on either language, but I've spent some time playing with CL and I'm in the process of learning Racket. So, as an armature, here's my story about why I prefer Racket. Please be aware that any or all of this may be incorrect or incomplete. My knowledge on both subjects is extremely limited, so I can only compare my understanding of one to my understanding of the other.

CL does not have a true NULL. Ask any database admin, NULL and false are not the same thing. This is an important point, NULL means "I don't know" and in CL there is no signal for this relatively fundamental idea.

CL does not have a free IDE that comes in a working state. Even if you manage to configure the infamous Emacs, which seems to come configured for nothing (why?), its key bindings are straight out of the 70's and will leave you longing for an editor as sweet as Notepad (obvious hyperbole). DrRacket on the other hand, is exactly the kind of editor that you have come to expect. It works out of the box, has good syntax highlighting, automatic indentation, step by step debugging, etc.

CL does not support true multi-threading. Racket does.

CL is a Lisp2. This is a matter of personal taste, but if you prefer (map function data) over (map 'list #'function data) you might be a Lisp1 advocate.

CL defun does nothing for scoping, which enhances use of the special forms "let" and "labels" both of which help you get Lost In Stupid Parenthesis. To contrast, Racket's "define" statement gives you scoping as you'd expect without the burden of unnecessary nesting.

CL's defmacro relies heavily on quasi-quoting, while Racket's define-syntax-rule is free of this burden. To take matters further, the greatly troublesome doubly nested quoted let form that CL employs to bind expressions to run-time generated symbols is easily replaced by a single, unquoted let in Racket. In short, simple macros are much, much simpler. I personally haven't played with complex macros, so I can't speak to that.

CL does not have pattern matching. Racket does. This feature is, IMO, almost reason enough to learn a language on it's own.

CL only supports parenthesis, while Racket supports both curly and square braces as a substitute for parenthesis anywhere. The tasteful substitution of these alternate delimiters into special forms can really help you dissect deeply nested expressions.

Racket uses JIT compilation, and so far all the code I've ported seems to run in at most 1/10th the time (I've seen from 1/50 to 1/125) CLisp required. That's not universal truth; that's just my code. Your mileage may vary.

In the end, it's really just a matter of taste. Choose the one that's most intuitive to you. I believe Racket is much more newbie friendly, but I can't speak to how it treats an expert. Ah, one final thought, If you pick Racket, you will definitely miss loop. Sorry about being one sided, CL is a very good language also

[Sam Tobin-Hochstadt]

On Tuesday, June 26, 2012 9:07:48 PM UTC-4, Nathan wrote:

[many nice things about Racket snipped]

> Ah, one final thought, If you pick Racket, you will definitely miss loop. Sorry about being one sided, CL is a very good language also

Can you give an example of some Racket code where you missed `loop`?

Sam (a Racket developer)

[Pascal J. Bourguignon]

Nathan <nbee...@gmail.com> writes:

> CL does not have a true NULL. Ask any database admin, NULL and false
> are not the same thing. This is an important point, NULL means "I
> don't know" and in CL there is no signal for this relatively
> fundamental idea.

You're wrong. CL obviously has a "I don't know" value:

:I-DONT-KNOW

> CL does not have a free IDE that comes in a working state.

You're wrong. There are several free IDE in a working state, for CL
development, whatever the meaning you have for the word "free" or "in a
working state".

> Even if you manage to configure the infamous Emacs, which seems to
> come configured for nothing (why?), its key bindings are straight out
> of the 70's and will leave you longing for an editor as sweet as
> Notepad (obvious hyperbole).

Well, if you long for Notepad, you know where to find it. Probably the
problem is that you're not smart enough to use emacs, but that's Out of
Topic. Ask again in gnu.help.emacs if you're interested.

> DrRacket on the other hand, is exactly the kind of editor
> that you have come to expect. It works out of the box, has good syntax
> highlighting, automatic indentation, step by step debugging, etc.

Good for you. But Racket is not a CL IDE, it doesn't even claim to be a
scheme IDE, so…

> CL does not support true multi-threading. Racket does.


> CL is a Lisp2.

Which is good. Thank you.

> This is a matter of personal taste, but if you prefer
> (map function data) over (map 'list #'function data) you might be a
> Lisp1 advocate.

Again, that's your problem, not that of CL.

> CL defun does nothing for scoping, which enhances use of the special
> forms "let" and "labels" both of which help you get Lost In Stupid
> Parenthesis. To contrast, Racket's "define" statement gives you
> scoping as you'd expect without the burden of unnecessary nesting.

This shows that YOU didn't understand what lisp was about.

> CL's defmacro relies heavily on quasi-quoting,

Not at all.

> while Racket's define-syntax-rule is free of this burden. To take
> matters further, the greatly troublesome doubly nested quoted let form
> that CL employs to bind expressions to run-time generated symbols is
> easily replaced by a single, unquoted let in Racket. In short, simple
> macros are much, much simpler. I personally haven't played with
> complex macros, so I can't speak to that.

Again, it's a deficiency in YOU, not in CL. You're just showing your
incompetency here.

> CL does not have pattern matching.

You're wrong. It does. There are just TOO many pattern maching
libraries in CL.

> Racket does. This feature is, IMO, almost reason enough to learn a
> language on it's own.

Whatever.



> CL only supports parenthesis,

You're wrong. CL supports any syntax you want, with reader macros.

> while Racket supports both curly and square braces as a substitute for
> parenthesis anywhere. The tasteful substitution of these alternate
> delimiters into special forms can really help you dissect deeply
> nested expressions.

But no sane CL programmer need to write parenthesis differently to

dissect deeply nested expressions.


> Racket uses JIT compilation, and so far all the code I've ported seems
> to run in at most 1/10th the time (I've seen from 1/50 to 1/125) CLisp
> required. That's not universal truth; that's just my code. Your
> mileage may vary.

Indeed. Clisp has a JITC too. All the other implementations (apart
from ABCL) compile to native code, and therefore are even faster than
Racket.

> In the end, it's really just a matter of taste.

Then say that. Instead of writing 536 words, you could just have
written:

"I don't like CL, I like Racket."

(and that's just how interesting your message is, apart from the wrong
and misleading statements).

[Manuel Giraud]

Nathan <nbee...@gmail.com> writes:

> CL does not have a true NULL. Ask any database admin, NULL and false
> are not the same thing. This is an important point, NULL means "I
> don't know" and in CL there is no signal for this relatively
> fundamental idea.

Hum? TRUE/FALSE, 1/0, t/nil doesn't really matter. It is your
interpretation of these values that matters. Obligatory CL example:

gethash key hash-table &optional default => value, present-p

present-p is true if an entry is found; otherwise, it is false.
--
Manuel Giraud

[Eli Barzilay]

"Pascal J. Bourguignon" <p...@informatimago.com> writes:
>
> [...] Probably the problem is that you're not smart enough to use
> emacs, [...]

(Was that really necessary?)

FWIW, and unrelated to CL, Scheme, or Racket -- I spend about 95% of
my waking time in an Emacs window, yet I find Emacs in its default
configuration to be almost as bad as notepad. Knowing your tools
include knowing their limitations, and as much as I like using it,
Emacs is very obviously lacking when it comes to people who don't want
to spend around a year just to learn how to use an editor. Unlike
your approach, I *do* assume that you're smart enough to know that,
which makes the above a vicious-for-no-good-reason blurb.

> Good for you. But Racket is not a CL IDE, it doesn't even claim to
> be a scheme IDE, so…

Racket is not an IDE. DrRacket, OTOH, is an IDE that supports sexpr
editing, and many of Racket's different languages are sexpr-based.
(You could even use it for CL code, but certainly for Scheme it would
work fine out of the box.)

>> CL's defmacro relies heavily on quasi-quoting,
>
> Not at all.

Sure you *can* write macros without quasiquoting, but that makes
macros impractical in the same way that explicit AST munging would be.
It therefore does rely on quasiquotes to make macros practical.
(That's not an advantage of Scheme, of course, rewrite rules could be
done in CL, and OTOH modern Scheme macro system do the same kind of
quasiquoting.)

>> CL does not have pattern matching.
>
> You're wrong. It does. There are just TOO many pattern maching
> libraries in CL.

...and *this* is a hook that just begs a reply. Scheme sucks in the
100 features that it doesn't have and CLers (specifically ones who
post on c.l.l, and more specifically ones would would post with such a
title) know very well that "... is just a quick macro definition away"
doesn't cut it. Yes, you can implement loops/oo-system/blah/blah/blah
with a quick half-assed macro that works for the few things you need,
but that's a lie since a macro that works for many people is two
orders of magnitude harder to write. And yes, some people will invest
the time and write something proper -- but that joins the other 16
implementations of the same feature, which is an obvious problem. I'm
assuming that you know that, and I agree that having the loop macro,
clos, and whatever is a huge point for CL.

Can you see how this paragraph ends when you're talking about pattern
matching in Racket (as opposed to Scheme)? (And no, destructuring
isn't close.)

>> CL only supports parenthesis,
>
> You're wrong. CL supports any syntax you want, with reader macros.

Ooh, ooh, I know -- other brackets are just a short readtable tweak
away, right?

> But no sane CL programmer need to write parenthesis differently to
> dissect deeply nested expressions.

(The convention in Racket wrt the use of square brackets is unrelated
to expression depth.)

> Indeed. Clisp has a JITC too. All the other implementations (apart
> from ABCL) compile to native code, and therefore are even faster
> than Racket.

I don't know what you call a "JIT" or "native code", but the one in
Racket compiles to machine code.

> Then say that. Instead of writing 536 words, you could just have
> written:
>
> "I don't like CL, I like Racket."
>
> (and that's just how interesting your message is, apart from the
> wrong and misleading statements).

There's a thread on c.l.l called "Scheme vs Common Lisp", and the
amount of "I don't like Scheme, I like CL" posts is as expected. One
guy who dares to say otherwise is definitely making a contribution to
the interest level. For anyone other than pjb -- I bet your house
smells of recently microwaved popcorn, right?

--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!

[Pascal J. Bourguignon]

Eli Barzilay <e...@barzilay.org> writes:

> "Pascal J. Bourguignon" <p...@informatimago.com> writes:
>>
>> [...] Probably the problem is that you're not smart enough to use
>> emacs, [...]
>
> (Was that really necessary?)
>
> FWIW, and unrelated to CL, Scheme, or Racket -- I spend about 95% of
> my waking time in an Emacs window, yet I find Emacs in its default
> configuration to be almost as bad as notepad. Knowing your tools
> include knowing their limitations, and as much as I like using it,
> Emacs is very obviously lacking when it comes to people who don't want
> to spend around a year just to learn how to use an editor. Unlike
> your approach, I *do* assume that you're smart enough to know that,
> which makes the above a vicious-for-no-good-reason blurb.

The point is that if you're not happy with the way emacs does something,
you can change it.

>>> CL's defmacro relies heavily on quasi-quoting,
>>
>> Not at all.
>
> Sure you *can* write macros without quasiquoting, but that makes
> macros impractical in the same way that explicit AST munging would be.
> It therefore does rely on quasiquotes to make macros practical.
> (That's not an advantage of Scheme, of course, rewrite rules could be
> done in CL, and OTOH modern Scheme macro system do the same kind of
> quasiquoting.)

That's not the point. The point is that if you're not happy with
quasiquote, you can write your own code generation system.

>>> CL does not have pattern matching.
>>
>> You're wrong. It does. There are just TOO many pattern maching
>> libraries in CL.
>
> ...and *this* is a hook that just begs a reply. Scheme sucks in the
> 100 features that it doesn't have and CLers (specifically ones who
> post on c.l.l, and more specifically ones would would post with such a
> title) know very well that "... is just a quick macro definition away"
> doesn't cut it. Yes, you can implement loops/oo-system/blah/blah/blah
> with a quick half-assed macro that works for the few things you need,
> but that's a lie since a macro that works for many people is two
> orders of magnitude harder to write. And yes, some people will invest
> the time and write something proper -- but that joins the other 16
> implementations of the same feature, which is an obvious problem. I'm
> assuming that you know that, and I agree that having the loop macro,
> clos, and whatever is a huge point for CL.
>
> Can you see how this paragraph ends when you're talking about pattern
> matching in Racket (as opposed to Scheme)? (And no, destructuring
> isn't close.)

I was talking about LIBRARIES, and yes, you can write CL libraries that
run conformingly on all conforming CL implementations.

Yes, I know that that's not possible in scheme, and the OP should know
it too.

>>> CL only supports parenthesis,
>>
>> You're wrong. CL supports any syntax you want, with reader macros.
>
> Ooh, ooh, I know -- other brackets are just a short readtable tweak
> away, right?

Yes, that's my point.

>> But no sane CL programmer need to write parenthesis differently to
>> dissect deeply nested expressions.
>
> (The convention in Racket wrt the use of square brackets is unrelated
> to expression depth.)

Which the OP doesn't seem to be knowing.

>> Indeed. Clisp has a JITC too. All the other implementations (apart
>> from ABCL) compile to native code, and therefore are even faster
>> than Racket.
>
> I don't know what you call a "JIT" or "native code", but the one in
> Racket compiles to machine code.

When? Just In Time Compiler compiles to native code "just in time",
ie. just before executing it (and cache the native code for the next
loop). Usual native compiler compile long before, usually in batch
mode. The OP said Racket had JIT, not a batch native code compiler.

>> Then say that. Instead of writing 536 words, you could just have
>> written:
>>
>> "I don't like CL, I like Racket."
>>
>> (and that's just how interesting your message is, apart from the
>> wrong and misleading statements).
>
> There's a thread on c.l.l called "Scheme vs Common Lisp", and the
> amount of "I don't like Scheme, I like CL" posts is as expected. One
> guy who dares to say otherwise is definitely making a contribution to
> the interest level. For anyone other than pjb -- I bet your house
> smells of recently microwaved popcorn, right?

It might have been interesting if what he said about CL was right.
There are bad spots in CL, but a newbie wouldn't know them.

[Marco Antoniotti]

On Tuesday, June 26, 2012 11:46:05 PM UTC-4, pjb wrote:
> Nathan <nbee...@gmail.com> writes:
>
> > CL does not have a true NULL. Ask any database admin, NULL and false
> > are not the same thing. This is an important point, NULL means "I
> > don't know" and in CL there is no signal for this relatively
> > fundamental idea.
>
> You're wrong. CL obviously has a "I don't know" value:
>
> :I-DONT-KNOW

It also has the

:PEOPLE-SHOULD-KNOW-BETTER

value :)

>
>
> > CL does not have a free IDE that comes in a working state.
>
> You're wrong. There are several free IDE in a working state, for CL
> development, whatever the meaning you have for the word "free" or "in a
> working state".

Racket, as good as it is, is a "single implementation" language. Therefore, comparing Racket with CL is the classic apple and oranges (or individual vs set) thingy. Now, If you take the "free" LW Personal edition you get all the stuff you get in Racket, but Nathan seems unaware of this.

>
> > CL does not have pattern matching.
>
> You're wrong. It does. There are just TOO many pattern maching
> libraries in CL.

But just one to rule them all: CL-UNIFICATION :) :) :)

Cheers
--
MA

[Stefan Mandl]

On Tue, 26 Jun 2012 18:07:48 -0700, Nathan wrote:

> [...]

> CL does not have a true NULL. Ask any database admin, NULL and false are
> not the same thing. This is an important point, NULL means "I don't
> know" and in CL there is no signal for this relatively fundamental idea.

> [...]

If think it's very good that CL does not show any signal of this
"fundamental idea":

1) NULLs in relational DBs semantically serve two distinct purposes:
a) "No value"
b) "I don't know"
This can be quite confusing if you have to work with somebody else's
schema.

2) SQL's behavior for NULL is very unsound.
a) NULL is equal to NULL in queries like "select distinct ...."
b) NULL is not equal to NULL when it appears as a value in a join column.

In CL on the other hand, there's almost always a value and NIL is a
default that turned out to be very useful in various widespread idioms.

Regards,
Stefan

[Eli Barzilay]

"Pascal J. Bourguignon" <p...@informatimago.com> writes:
>
> The point is that if you're not happy with the way emacs does
> something, you can change it.

Phew, when you miss the point you REALLY do it.

> That's not the point. The point is that if you're not happy with
> quasiquote, you can write your own code generation system.

Yes, shooting a flies and missing by several kms.

>> Can you see how this paragraph ends when you're talking about
>> pattern matching in Racket (as opposed to Scheme)? (And no,
>> destructuring isn't close.)
>
> I was talking about LIBRARIES, and yes, you can write CL libraries
> that run conformingly on all conforming CL implementations.
>
> Yes, I know that that's not possible in scheme, and the OP should
> know it too.

And it turns out that you're holding a banana.

>> Ooh, ooh, I know -- other brackets are just a short readtable tweak
>> away, right?
>
> Yes, that's my point.

Which is just as well since the fly is the neighbors' kid.

>>> Indeed. Clisp has a JITC too. All the other implementations
>>> (apart from ABCL) compile to native code, and therefore are even
>>> faster than Racket.
>>
>> I don't know what you call a "JIT" or "native code", but the one in
>> Racket compiles to machine code.
>
> When? Just In Time Compiler compiles to native code "just in time",
> ie. just before executing it (and cache the native code for the next
> loop). Usual native compiler compile long before, usually in batch
> mode. The OP said Racket had JIT, not a batch native code compiler.

JIT compilers start from pre-compiled bytecode (where static
optimizations are done), and do the last step of code generation.

http://en.wikipedia.org/wiki/Just-in-time_compilation

> It might have been interesting if what he said about CL was right.

It sure is entertaining.

> There are bad spots in CL, but a newbie wouldn't know them.

The idea that newbies are not qualified to find problems in your
<whatever> is nonsense.

[Eli Barzilay]

Kaz Kylheku <k...@kylheku.com> writes:
>
> How do you test for it? Say I have some big program which depends on
> tail recursion in hundreds of places. What if tail recursion isn't
> happening in some of them?

Testing tail calls is pretty easy, since it's a syntactically
verifiable property. (And in fact there are things that do exactly
what you want here.)

[Pascal J. Bourguignon]

Eli Barzilay <e...@barzilay.org> writes:

> Kaz Kylheku <k...@kylheku.com> writes:
>>
>> How do you test for it? Say I have some big program which depends on
>> tail recursion in hundreds of places. What if tail recursion isn't
>> happening in some of them?
>
> Testing tail calls is pretty easy, since it's a syntactically
> verifiable property. (And in fact there are things that do exactly
> what you want here.)

Not in Common Lisp.

[daniel....@excite.com]

Wow, Eli & PJB, you guys are getting really pesky...!

My 2 cents on quasiquote: Quasiquote in not just for macros. It is a super way to construct specific lists when ever you want to. Its great in regular code instead of using LIST. Makes the code much more visual. In fact it's a non s-expr way (some might even say "non-Lispy" way) of generating a specific list, when the operations are relatively simple.

Macros aren't just for quasiquote. I use quasiquote in macros only in the simplest cases. If it gets super complicated (of course it does sometimes), the s-expr way with LIST starts to become a lot more clear.

[Eli Barzilay]

daniel....@excite.com writes:
>
> My 2 cents on quasiquote: Quasiquote in not just for macros. It is
> a super way to construct specific lists when ever you want to.

Of course -- there are two properties that contribute to this. First,
it allows you to write a list (or more generally any sexpr) without
explicit constructors, which is exponentially easier than not having
quotes. Second, you get the "interpolation" feature of quasiquote as
an easy escape hatch to mix quoted data with expressions.

But macro code usually deals with exactly these issues, which is why
they're very important in that context. Without quasiquotes or some
other quotation facility (eg, patterns in Scheme rewrite rules, etc),
such code would be harder to write, read, and maintain. To see this
in action, take a simple expression like (lambda (x) (+ x 1)), and
quote it twice. With the usual lisp quote that's easy:

''(lambda (x) (+ x 1))

with a primitive kind of quote (one that is used only for symbols),
things are hard since the size of the result grows exponentially with
each level:

(list 'list
(list 'quote 'lambda)
(list 'list (list 'quote 'x))
(list 'list (list 'quote '+) (list 'quote 'x) 1))

And clearly the structure of the intended code is all gone here.

It's also a nice demonstration of why sexprs as a uniform
representation of code work so well -- without them, you'd have
something like this as the first quotation:

Function("x", Application(Identifier("+"), Identifier("x"), Integer(1)))

and the second level is

Application(Identifier("Function"),
String("x"),
Application(Identifier("Application"),
Application(Identifier("Identifier"),
String("+")),
Application(Identifier("Identifier"),
String("x")),
Application(Identifier("Integer"),
Integer(1))))

And that makes it obvious why modern syntax-manipulation systems all
have some quotation facility...

[Eli Barzilay]

"Pascal J. Bourguignon" <p...@informatimago.com> writes:

> Eli Barzilay <e...@barzilay.org> writes:
>>
>> Testing tail calls is pretty easy, since it's a syntactically
>> verifiable property. (And in fact there are things that do exactly
>> what you want here.)
>
> Not in Common Lisp.

It's easy in any language. The only thing that is needed is a
specification of tail positions for all primitive forms.

With the usual pessimistic interpretation for a lack of such
specification, it just happens to be a trivial test in the case of
portable CL. In any case, the point is that tail calls are really not
hard to use when your language has them, as the original post implied.

[Pascal J. Bourguignon]

Eli Barzilay <e...@barzilay.org> writes:

> "Pascal J. Bourguignon" <p...@informatimago.com> writes:
>
>> Eli Barzilay <e...@barzilay.org> writes:
>>>
>>> Testing tail calls is pretty easy, since it's a syntactically
>>> verifiable property. (And in fact there are things that do exactly
>>> what you want here.)
>>
>> Not in Common Lisp.
>
> It's easy in any language. The only thing that is needed is a
> specification of tail positions for all primitive forms.

But it is not syntactically verifiable in Common Lisp.

(let ((x 42)) (f 22))

F may be a tail call or may be not, depending on whether x is declared
special or not.

> With the usual pessimistic interpretation for a lack of such
> specification, it just happens to be a trivial test in the case of
> portable CL. In any case, the point is that tail calls are really not
> hard to use when your language has them, as the original post implied.

--

[Eli Barzilay]

"Pascal J. Bourguignon" <p...@informatimago.com> writes:

> Eli Barzilay <e...@barzilay.org> writes:
>
>> "Pascal J. Bourguignon" <p...@informatimago.com> writes:
>>
>>> Eli Barzilay <e...@barzilay.org> writes:
>>>>
>>>> Testing tail calls is pretty easy, since it's a syntactically
>>>> verifiable property. (And in fact there are things that do exactly
>>>> what you want here.)
>>>
>>> Not in Common Lisp.
>>
>> It's easy in any language. The only thing that is needed is a
>> specification of tail positions for all primitive forms.
>
> But it is not syntactically verifiable in Common Lisp.
>
> (let ((x 42)) (f 22))
>
> F may be a tail call or may be not, depending on whether x is
> declared special or not.

You're right, having a dynamic-scope that is not syntactically
identifiable is a major problem, but what I said is still true: you
need a specification for tail-positions, and given that in most CL
implementations the above `f' call is not guaranteed to be in a tail
position the test is becomes trivial. (In fact, it's even more
trivial for a language that doesn't talk about tail positions.)

As a side-note, I don't know if any CL implementations do so, but it
is certainly possible to have a dynamically-scoped binder and still
maintain the tail position. IOW, even with CL's sometimes-dynamic
`let' form the above could guarantee a tail call of `f'.

[Pascal J. Bourguignon]

Eli Barzilay <e...@barzilay.org> writes:

> As a side-note, I don't know if any CL implementations do so, but it
> is certainly possible to have a dynamically-scoped binder and still
> maintain the tail position. IOW, even with CL's sometimes-dynamic
> `let' form the above could guarantee a tail call of `f'.

This is harder to do: you'd need a more global analysis.

In the case of a recursive function, the global analysis is restricted
to a single function:

(defvar *level* 0)

(defun f (x)
(let ((*level* (1+ *level*)))
(if (zerop x)
*level*
(f (1- x)))))


but tail call optimization is also for non-directly-recursive functions,
and there the analysis is more complex, and may be disrupted at
run-time.

[Marco Antoniotti]

You must have dealt with XML or OCaml way too much recently :) :) :) :)

Cheers
--
MA

[Helmut Eller]

* Eli Barzilay [2012-06-28 02:36] writes:

> "Pascal J. Bourguignon" <p...@informatimago.com> writes:
>
>> Eli Barzilay <e...@barzilay.org> writes:
>>>
>>> Testing tail calls is pretty easy, since it's a syntactically
>>> verifiable property. (And in fact there are things that do exactly
>>> what you want here.)
>>
>> Not in Common Lisp.
>
> It's easy in any language. The only thing that is needed is a
> specification of tail positions for all primitive forms.

That seems pretty useless in the presence of macros, because programmers
don't look at expanded macros but at source code. Specifying what parts
of a macro are in tail position would be a lot of work and is clearly
not verifiable in general because macro expanders are Turing complete.

I would also claim that some optimizations, in particular unboxing of
numbers, makes it undesirable/impossible to preserve "proper" tail calls
in all situations.

Helmut

[daniel....@excite.com]

On Wednesday, June 27, 2012 10:36:32 PM UTC-4, Eli Barzilay wrote:

> But macro code usually deals with exactly these issues, which is why
> they're very important in that context. Without quasiquotes or some
> other quotation facility (eg, patterns in Scheme rewrite rules, etc),
> such code would be harder to write, read, and maintain.

Your example notwithstanding, I disagree, as I said: in some cases when generating deeply nested lists the interpolation functions of quasiquote become extremely difficult to sort out, and I find it easier to revert back to constructive s-expressions.

I encourage anyone who is having some difficulty learning to use CL macros to try it without quasiquote (back-quote) notation until they get the hang of the macro idea. Just use (LIST ...).

Then start using quasiquotes anywhere they want. Pretty soon they'll get the hang of both macros and quasiquotes.

[Frode V. Fjeld]

Eli Barzilay <e...@barzilay.org> writes:

> As a side-note, I don't know if any CL implementations do so, but it
> is certainly possible to have a dynamically-scoped binder and still
> maintain the tail position. IOW, even with CL's sometimes-dynamic
> `let' form the above could guarantee a tail call of `f'.

Tail calls is a very nice idea that software engineering should have
abandoned a long time ago.

--
Frode V. Fjeld

[Eli Barzilay]

"Pascal J. Bourguignon" <p...@informatimago.com> writes:

> Eli Barzilay <e...@barzilay.org> writes:
>
>> As a side-note, I don't know if any CL implementations do so, but it
>> is certainly possible to have a dynamically-scoped binder and still
>> maintain the tail position. IOW, even with CL's sometimes-dynamic
>> `let' form the above could guarantee a tail call of `f'.
>
> This is harder to do: you'd need a more global analysis.
>

> but tail call optimization is also for non-directly-recursive
> functions, and there the analysis is more complex, and may be
> disrupted at run-time.

No -- you're assuming some global optimization that gets around the
tail-context problem, but Racket uses a solution that doesn't require
that. Just try to run this in Racket:

-> (define p (make-parameter 0))
-> (define (foo f) (parameterize ((p (add1 (p)))) (f foo)))
-> (define (bar f) (parameterize ((p (add1 (p)))) (f bar)))
-> (foo bar)

Running it at the toplevel means that just like in CL, Racket is not
allowed to do some global analysis (eg, a side-thread can `set!' the
function names), yet it runs in constant space.

The way this is implemented is with something called "continuation
marks" -- which is, roughly speaking, a way to "decorate" your stack
with extra information. Since the parameter already has a value on
the current stack frame, each of these `parameterize' forms end up
changing it instead of adding frames -- and all of that is without
changing the global value of the parameter. (This is the result of
John Clements's phd work, BTW.)

[Eli Barzilay]

Absolutely! Just like the stupid idea of "anonymous functions" and
garbage collection.

[Eli Barzilay]

daniel....@excite.com writes:

> On Wednesday, June 27, 2012 10:36:32 PM UTC-4, Eli Barzilay wrote:
>> But macro code usually deals with exactly these issues, which is why
>> they're very important in that context. Without quasiquotes or some
>> other quotation facility (eg, patterns in Scheme rewrite rules, etc),
>> such code would be harder to write, read, and maintain.
>
> Your example notwithstanding, I disagree, as I said: in some cases
> when generating deeply nested lists the interpolation functions of
> quasiquote become extremely difficult to sort out, and I find it
> easier to revert back to constructive s-expressions.

See for example the two `do-primes' macro definitions here:

http://www.gigamonkeys.com/book/macros-defining-your-own.html

The first one is 7 lines long, the second uses deconstructed arguments
and gets it down to 4 lines, and then there's `do-primes-a' that
avoids backquotes and goes back up to 7 lines. That last one is
obfuscated (in the sense that it's harder to read), because the
constructed syntax is harder to read. For example, the backquote
version has:

`(do [...] ,@body)

where the resulting `do' form is clear but the latter has

(append '(do) [...] body)

where you need to know what '(do) is, how it interacts with `append',
and how `body' fits into the story -- and only then you can see what
the constructed result is supposed to be.

All of this is usually much harder to do.

> I encourage anyone who is having some difficulty learning to use CL
> macros to try it without quasiquote (back-quote) notation until they
> get the hang of the macro idea. Just use (LIST ...).

If you're serious about that rather than just flame-baiting, then how
about a demonstration: translate the `once-only' macro later down that
page to a version that doesn't use backquotes. If you do that and
keep on saying that it's easier, then you're, um, special.

[Eli Barzilay]

Marco Antoniotti <mar...@gmail.com> writes:
>
> You must have dealt with XML or OCaml way too much recently :) :) :) :)

Not really. XML suffers from having a standard way to encode things
and the questionable notion of representing only data and never
behavior. This kind of locks you into the stupid way of quotation,
and the onlyt practical way you could go around it is with some
fron-end editor that would do the encoding & decoding in a way that is
hidden from the user (but really do the full stupidity behind your
back).

With OCaml things are better -- the full syntax of constructing AST
nodes is almost never used, and instead they have a facility that is
very similar to quasiquotes. (Actually, you could say that it's
similar to `syntax-case' since pattern matching fits well with OCaml,
and you still get to write meta-level code on the RHS of matches.)

And I didn't deal with any of this recently, but it was one of the
main topics of my phd, so I spent a good number of years doing these
kind of things...

[Eli Barzilay]

Helmut Eller <eller....@gmail.com> writes:

> * Eli Barzilay [2012-06-28 02:36] writes:
>> It's easy in any language. The only thing that is needed is a
>> specification of tail positions for all primitive forms.
>
> That seems pretty useless in the presence of macros, because
> programmers don't look at expanded macros but at source code.

The feature that was referred to in the OP was a way to test that a
call is in tail position -- and the macro system should provide a way
to do this test by checking the fully expanded form. I referred to an
existing thing that does that -- one of Racket's languages has a
`return' form which is just a no-op except that it throws a syntax
error if it's not called in a tail position. The upshot of that is
exactly what Kaz wished for: if you want to rely on tail calls in some
long piece of code you could use just that.

> I would also claim that some optimizations, in particular unboxing
> of numbers, makes it undesirable/impossible to preserve "proper"
> tail calls in all situations.

(I don't know of any numeric optimizations that collide with TCO.
Examples?)

[Helmut Eller]

* Eli Barzilay [2012-06-29 15:48] writes:

> Helmut Eller <eller....@gmail.com> writes:
>
>> * Eli Barzilay [2012-06-28 02:36] writes:
>>> It's easy in any language. The only thing that is needed is a
>>> specification of tail positions for all primitive forms.
>>
>> That seems pretty useless in the presence of macros, because
>> programmers don't look at expanded macros but at source code.
>
> The feature that was referred to in the OP was a way to test that a
> call is in tail position -- and the macro system should provide a way
> to do this test by checking the fully expanded form. I referred to an
> existing thing that does that -- one of Racket's languages has a
> `return' form which is just a no-op except that it throws a syntax
> error if it's not called in a tail position. The upshot of that is
> exactly what Kaz wished for: if you want to rely on tail calls in some
> long piece of code you could use just that.

Tail position of what? How to do you say that you want to be in tail
position of a set of mutually recursive functions that make up a state
machine?

>> I would also claim that some optimizations, in particular unboxing
>> of numbers, makes it undesirable/impossible to preserve "proper"
>> tail calls in all situations.
>
> (I don't know of any numeric optimizations that collide with TCO.
> Examples?)

(defun fib (x)
(labels ((ffib (x))
(declare (double-float x))
(the double-float
(cond ((= x 0) 0d0)
((= x 1) 1d0)
(t (+ (ffib (- x 1))
(ffib (- x 2)))))))
(ffib x)))

FFIB could be a function that returns unboxed double-floats, except that
FFIB is called in tail position of FIB which makes it necessary to
return boxed floats. A simple solution is to let FFIB return unboxed
values, make the call in FIB a non-tail call and box the result before
returning from FIB; that breaks tail call optimization.

Helmut

[Eli Barzilay]

Helmut Eller <eller....@gmail.com> writes:

> * Eli Barzilay [2012-06-29 15:48] writes:
>
>> Helmut Eller <eller....@gmail.com> writes:
>>
>>> * Eli Barzilay [2012-06-28 02:36] writes:
>>>> It's easy in any language. The only thing that is needed is a
>>>> specification of tail positions for all primitive forms.
>>>
>>> That seems pretty useless in the presence of macros, because
>>> programmers don't look at expanded macros but at source code.
>>
>> The feature that was referred to in the OP was a way to test that a
>> call is in tail position -- and the macro system should provide a way
>> to do this test by checking the fully expanded form. I referred to an
>> existing thing that does that -- one of Racket's languages has a
>> `return' form which is just a no-op except that it throws a syntax
>> error if it's not called in a tail position. The upshot of that is
>> exactly what Kaz wished for: if you want to rely on tail calls in some
>> long piece of code you could use just that.
>
> Tail position of what?

Of the surrounding function.

> How to do you say that you want to be in tail position of a set of
> mutually recursive functions that make up a state machine?

(define (foo)
... (return (bar)) ...)

(define (bar)
... (return (foo)) ...)

And if `return' throws a syntax error when it's used not in a tail
context, you're guaranteed that (given tail call elimination in the
language) the resulting code is a loop.

>>> I would also claim that some optimizations, in particular unboxing
>>> of numbers, makes it undesirable/impossible to preserve "proper"
>>> tail calls in all situations.
>>
>> (I don't know of any numeric optimizations that collide with TCO.
>> Examples?)
>
> (defun fib (x)
> (labels ((ffib (x))
> (declare (double-float x))
> (the double-float
> (cond ((= x 0) 0d0)
> ((= x 1) 1d0)
> (t (+ (ffib (- x 1))
> (ffib (- x 2)))))))
> (ffib x)))
>
> FFIB could be a function that returns unboxed double-floats, except
> that FFIB is called in tail position of FIB which makes it necessary
> to return boxed floats. A simple solution is to let FFIB return
> unboxed values, make the call in FIB a non-tail call and box the
> result before returning from FIB; that breaks tail call
> optimization.

I don't see how that's different than making `ffib' itself the global
definition: you need a way to convert its result into a boxed number
when it's called in random contexts, since some might use the unboxed
number and some might need it boxed.

[Helmut Eller]

* Eli Barzilay [2012-06-29 18:08] writes:

>> Tail position of what?
>
> Of the surrounding function.
>
>
>> How to do you say that you want to be in tail position of a set of
>> mutually recursive functions that make up a state machine?
>
> (define (foo)
> ... (return (bar)) ...)
>
> (define (bar)
> ... (return (foo)) ...)
>
> And if `return' throws a syntax error when it's used not in a tail
> context, you're guaranteed that (given tail call elimination in the
> language) the resulting code is a loop.

So you are saying this return feature guarantees that (return (bar)) is
in tail position of of bar? I.e this would be a syntax error:

(define (foo)
... (return (bar)) ...)

(define (bar)

(bar)
(return (foo)))

?

>>>> I would also claim that some optimizations, in particular unboxing
>>>> of numbers, makes it undesirable/impossible to preserve "proper"
>>>> tail calls in all situations.
>>>
>>> (I don't know of any numeric optimizations that collide with TCO.
>>> Examples?)
>>
>> (defun fib (x)
>> (labels ((ffib (x))
>> (declare (double-float x))
>> (the double-float
>> (cond ((= x 0) 0d0)
>> ((= x 1) 1d0)
>> (t (+ (ffib (- x 1))
>> (ffib (- x 2)))))))
>> (ffib x)))
>>
>> FFIB could be a function that returns unboxed double-floats, except
>> that FFIB is called in tail position of FIB which makes it necessary
>> to return boxed floats. A simple solution is to let FFIB return
>> unboxed values, make the call in FIB a non-tail call and box the
>> result before returning from FIB; that breaks tail call
>> optimization.
>
> I don't see how that's different than making `ffib' itself the global
> definition: you need a way to convert its result into a boxed number
> when it's called in random contexts, since some might use the unboxed
> number and some might need it boxed.

To spell out the obvious: FFIB calls itself recursivle many times but
FIB only returns once. If FFIB returns boxed values it needs to box
many times, but if the boxing in done in FIB the boxing is done only
once.

Helmut

[Eli Barzilay]

Helmut Eller <eller....@gmail.com> writes:

> * Eli Barzilay [2012-06-29 18:08] writes:
>
>>> Tail position of what?
>>
>> Of the surrounding function.
>>
>>
>>> How to do you say that you want to be in tail position of a set of
>>> mutually recursive functions that make up a state machine?
>>
>> (define (foo)
>> ... (return (bar)) ...)
>>
>> (define (bar)
>> ... (return (foo)) ...)
>>
>> And if `return' throws a syntax error when it's used not in a tail
>> context, you're guaranteed that (given tail call elimination in the
>> language) the resulting code is a loop.
>
> So you are saying this return feature guarantees that (return (bar))
> is in tail position of of bar?

No, it's in tail position fo the `foo' function, where it appears.

> I.e this would be a syntax error:
>
> (define (foo)
> ... (return (bar)) ...)
>
> (define (bar)
> (bar)
> (return (foo)))
>
> ?

This wouldn't be an error. (If you're aiming at a way to declare
`bar' as a function can only be called in tail positions, then that's
possible too, I think, but that wasn't the original issue.)

Right, and if `ffib' is a toplevel function, then you should still be
able to making it avoid boxing of the intermediate results yet still
be usable from a context that needs the boxes.

[Helmut Eller]

* Eli Barzilay [2012-06-29 18:43] writes:

>> To spell out the obvious: FFIB calls itself recursivle many times but
>> FIB only returns once. If FFIB returns boxed values it needs to box
>> many times, but if the boxing in done in FIB the boxing is done only
>> once.
>
> Right, and if `ffib' is a toplevel function, then you should still be
> able to making it avoid boxing of the intermediate results yet still
> be usable from a context that needs the boxes.

So how does Racket does that?

Helmut

[Eli Barzilay]

I don't know. (I'm not doing that kind of work, my knowledge at that
level is just at the highleve...)

[Frode V. Fjeld]

Eli Barzilay <e...@barzilay.org> writes:

> "Frode V. Fjeld" <fro...@gmail.com> writes:
>
>> Tail calls is a very nice idea that software engineering should have
>> abandoned a long time ago.
>
> Absolutely! Just like the stupid idea of "anonymous functions" and
> garbage collection.

That other, completely unrelated ideas are good, doesn't mean that tail
calls are. You'd think that was obvious.

--
Frode V. Fjeld

[D Herring]

On 06/27/2012 02:51 PM, Eli Barzilay wrote:
> Kaz Kylheku <k...@kylheku.com> writes:
>>
>> How do you test for it? Say I have some big program which depends on
>> tail recursion in hundreds of places. What if tail recursion isn't
>> happening in some of them?
>
> Testing tail calls is pretty easy, since it's a syntactically
> verifiable property. (And in fact there are things that do exactly
> what you want here.)

Having looked at the trace functionality in Racket (do-traced in
collects/racket/tracke.rkt), I'm not seeing the simplicity of
detecting tail calls. ;)

At least some respectable lispers say to think about tail calls
semantically, not syntactically. In their eyes, the tail call isn't
an optimization; to create the extra stack frame is a wasteful
pessimization.

I somewhat agree with Kaz; IMO the tail position is a subtlety and it
does not explicitly show intent. I would like an explicit (tailcall
#'f ...) that acts like funcall, but the compiler signals if the
function is not in a tail call position. This tailcall form is more
to document intent to other humans (and to catch poorly-written
macros, like a naive TRACE) than it is to help the compiler. A
corresponding notailcall would force the extra stack frame (presumably
for debugging).

A different name would help too. Something that better evokes the
elision of a stack frame. Declarations might work; but CL does not
support them in arbitrary locations...

Maybe I just haven't written enough Scheme code to get a feel for the
culture. C++ has its share of gotchas that I've learned to live with.

- Daniel

[Eli Barzilay]

D Herring <dher...@at.tentpost.dot.com> writes:

> On 06/27/2012 02:51 PM, Eli Barzilay wrote:
>> Kaz Kylheku <k...@kylheku.com> writes:
>>>
>>> How do you test for it? Say I have some big program which depends on
>>> tail recursion in hundreds of places. What if tail recursion isn't
>>> happening in some of them?
>>
>> Testing tail calls is pretty easy, since it's a syntactically
>> verifiable property. (And in fact there are things that do exactly
>> what you want here.)
>
> Having looked at the trace functionality in Racket (do-traced in
> collects/racket/tracke.rkt), I'm not seeing the simplicity of
> detecting tail calls. ;)

Um, that's because that code has nothing to do with the detection I
talked about... But since you looked, I'm assuming that you did want
to find out more, so I'll write a more thorough explanation.
Apologies for the OT-ness. (You can safely ignore this message unless
you want to know the details.)

I'll describe the continuation marks thing that I referred to earlier,
and then I'll explain what I meant by "Testing tail calls" -- which
is, AFAICT, the same thing that Kaz talked about.

The thing is that since Racket does eliminate tail call frames, and
since it has the continuation marks facility that I mentioned earlier
(as the thing that allows `parameterize' to be used in a way that
doesn't kill tail calls), the function tracing code doesn't need to
know anything about the source code. It uses a continuation mark to
keep information about the stack of traced calls, and that makes it
possible to do the tracing in a way that doesn't interfere with tail
calls -- but you won't find code that avoids such interfering, it's
implicit in the fact that `with-continuation-mark' evaluates its body
in a tail position. This also implies that the printouts do show when
there are tail calls by the fact that you get a flat output:

-> ,r racket/trace
-> (define (fact n a) (if (zero? n) a (fact (sub1 n) (* a n))))
-> ,tr fact
-> (fact 5 1)
>(fact 5 1)
>(fact 4 5)
>(fact 3 20)
>(fact 2 60)
>(fact 1 120)
>(fact 0 120)
<120
120

The code that talks about tail-calls in that file is doing that using
these continuation marks, which essentially detects when the printout
comes from a tail call, but that uses the "runtime semantics", or in
other words, it roughly uses Racket's reflective capabilities to find
cases where the stack didn't grow... If you want to try that feature
directly, here's an example for manually tracing a function:

(define (fact n a)
(with-continuation-mark 'fact `(fact ,n ,a)
(begin
(printf "~s\n" (continuation-mark-set->list
(current-continuation-marks)
'fact))
(if (zero? n) a (fact (sub1 n) (* a n))))))

The `with-continuation-mark' form adds an annotation that maps the
symbol 'fact to a sexpr that represents the call. Then the printout
extracts all of the annotations for 'fact. If you run this code,
you'll see that there is always only one such value. Now replace the
`begin' with (λ (x y) y) to destroy the tail-position of the recursive
call, and run it again. You'll see that there are marks that
correspond to nested frames.

So yes, you could probably use this in some way to create a call that
checks that it's in tail position, and that's roughly what the trace
code does -- but it's doing that dynamically, not by looking at the
source code.

> At least some respectable lispers say to think about tail calls
> semantically, not syntactically. In their eyes, the tail call isn't
> an optimization; to create the extra stack frame is a wasteful
> pessimization.

(Yes, there was a blog post from Steele not too long ago that tries to
explain this view.)

> I somewhat agree with Kaz; IMO the tail position is a subtlety and
> it does not explicitly show intent. I would like an explicit
> (tailcall #'f ...) that acts like funcall, but the compiler signals
> if the function is not in a tail call position.

You can probably see now why the above is not what you want -- it's
completely dynamic, and you want a compiler error, which means that it
must work with only the source code, not with running it.

> This tailcall form is more to document intent to other humans

Right -- this is what Kaz talked about (at least my understanding of
it), and this is the "easy test" that I talked about. I also talked
about one language in Racket that implements a `return' form that is
doing nothing except for such a test -- (return E) is the same as E
except that it throws a compile-time error when used in a non-tail
position. (It's an interesting view that I haven't thought about,
because I never had any need for it.)

Here's how you can do this test in a minimal kind of a Lispish
language, in racket-like pseudo code. It's verbose since it should
have a case for each form, but I hope that any average programmer
would consider this easy.

(define (check expr tail?)
(match expr
[`(return ,E)
(unless tail? (error 'return "not in tail position"))]
[`(lambda ,args . ,Es)
(for ([E (butlast Es)]) (check E #f))
(check (last Es) #t)]
[`(if ,E1 ,E2 ,E3)
(check E1 #f)
(check E2 tail?)
(check E3 tail?)]
[`(let ([,x ,E1]) . ,Es)
(check E1 #f)
(for ([E (butlast Es)]) (check E #f))
(check (last Es) tail?)]
... all other core forms ...
;; assume it's a function application now
[`(f . ,args)
(for ([E expr]) (check E #f))]))

(define (verify-returns code)
(check (expand-all-except code 'return) #t))

Note that this requires that you know for each primitive form which
arguments are in a tail position wrt the form. Since CL doesn't
specify that, you can't do this in any portable way -- there was the
problem of `let' that *might* be used with a dynamic variable, but
more than that, even an `if' form might not have the above behavior
since there's nothing in the spec that requires it to do that. That's
why I said that in the case of CL the answer to the test is trivial...

(BTW, an interesting similarity to how TCE means that Scheme
implementations need to specify tail-positions in their core forms is
how Clojure deals with lists: because it allows lazy lists, the
documentation of all list functions needs to specify how much of the
input list is computed.)

> (and to catch poorly-written macros, like a naive TRACE) than it is
> to help the compiler.

I don't think that there's any simple way to do this. The above
checks only `return' forms that appear in the code, so it is a case
for a dynamic check. But now what exactly do you check? Suppose that
you have a `foo' function that needs to call itself tail-recursively.
It could use code like the manual tracing example to make sure that it
doesn't appear more than once on the stack, but then what do you do
with something like (foo (foo 0))? You can do that by looking at the
full stack trace and making sure that there are no *consecutive* `foo'
calls, but that's fragile since factoring out some code from `foo'
into a helper `bar' means that you should check that there are no
consecutive foo,bar on the stack.

> A corresponding notailcall would force the extra stack frame
> (presumably for debugging).

This one's easy to do, you just use `identity', and the expression is
no longer in tail position. BTW, I know how to do that not because I
use it for debugging (I never needed that), but because I talk about
it in my class.

> A different name would help too. Something that better evokes the
> elision of a stack frame. Declarations might work; but CL does not
> support them in arbitrary locations...

(I'd assume that CL is pretty far from the naming convention being a
problem...)

> Maybe I just haven't written enough Scheme code to get a feel for
> the culture. C++ has its share of gotchas that I've learned to live
> with.

Like I said, this is something that in all my history of writing
scheme code has never been an issue. What I can say is that it tends
to be surprising in a good way: as in, you write some code that
happens to work nicely with tail calls without even realizing it. (A
good example of that which I go over in my class is the fact that the
interpreter we write is doing TCE because it inherits it from the
language we write the interpreter in.)

[D Herring]

Hi Eli,

Thanks for the detailed post.

You explained the two things I was most interested in, as well as
providing some other interesting tidbits.

(return (call)) acts to enforce the tail call position [in schemes
that have return].

(identity (call)) acts to prevent the tail call were it might
otherwise occur.

- Daniel

[Pascal J. Bourguignon]

D Herring <dher...@at.tentpost.dot.com> writes:

> (identity (call)) acts to prevent the tail call were it might
> otherwise occur.

That's strange.

In CL, cl:identity can be open coded, and since it returns its argument,
in tail call position its argument can also be in tail call position.

In CL, (cl:identity (call)) can be compiled just like (cl:values (call)).

[Frode V. Fjeld]

"Pascal J. Bourguignon" <p...@informatimago.com> writes:

> In CL, cl:identity can be open coded, and since it returns its argument,
> in tail call position its argument can also be in tail call position.

Not if you consider multiple values, I believe? I.e. IDENTITY and VALUES
both will return exactly one value, whereas a tail call might return any
number of values, although the primary value will always be the same.

--
Frode V. Fjeld