This may not be a concern for many but when the Man calls for documentation there

are times I curse the inability of a language like C, say, to extract comments.

QxxYYodf (pingo  pongo) ;this calculates the first 37 terms of the continued
fraction pingo/pongo

What about:

calc-37-cfraction-terms (numerator denominator)

I should get out more :-)

OK then.

'Basically, avoid comments if you can. If your code needs a comment to be
understood, usually it would be better to rewrite to make it easier to
understand. Write comments and documentation strings to say what and why
something happens and not how. You must use meaningful variable and function
names. And please try to keep things simple.'

It's not a pithy but I'm thinking about selling this to Baz Lerman for a follow
up to 'sunscreen'. Do you a 50% cut of the royalties? ;-)

Best Regards,

(defmacro my-quote (thing)
  `(quote ,thing))

Now, (my-quote (comment a b c)) had better evaluate to (COMMENT A B C); the
"comment" should not be removed before expanding the macro.

But if you wait until after macro expansion, many macros will have to be
comment-aware.  Consider:

(defmacro my-prog1 (first-form &body rest-forms)
  `(let ((return-val ,first-form))
     ,@rest-forms
     return-val))

What happens to:

(my-prog1
  (comment This is a comment)
  3
  (print "foo"))

This will expand into

(let ((return-value (comment This is a comment)))
  3
  (print "foo")
  return-value)

which is clearly not what was intended.

  foo.lisp, line 100: comment doesn't accurately describe behaviour of code.

Alas, alas.

I conjecture that the programmers who will write code clear enough
not to need comments if you ask them to are also the programmers who
will keep their comments accurate if you ask them to. So saying
"don't use comments, because inaccurate comments are a disaster"
doesn't work. If accurate comments aren't an option, neither is
code that doesn't need comments. And I'd rather have sloppily
commented junk than uncommented junk; wouldn't you? (In either
case the first thing I'd do if given the chance would be to clean
up both code and comments.)

; Q3 - return the sum of all prime numbers from 1 to n
(
   defun
   q3
   (x)
   (cond
      (
         (= x 2)
         2
      )
      (
         (isPrime x) (+ (q3 (- x 1)) x)
      )
      (
         t
         (q3 (- x 1))
      )
   )
)

..BWAHahahahaha... yes, yes... i'm MAd - MAD! I tell you... MAD MaAd
MAaaaad......
...(ahem) im ok now.              :)
--
joe comunale
Queens College, NY

State one: The code is sitting there being ASCII.
Stage one: Read it.

State two: The code is lists, vectors, etc, including comments.
Stage two: Winnow it.

State three: The code is lists, vectors, etc, without
comments. Exactly the same as you get now.

Stage three: Eval it.  Or compile it, or call a macro and eval the
result, etc.  Anything along the lines of "now do what the code says"
goes here.

For code transformations, it would go:

State one: The code is sitting there being ASCII.
Stage one: Read it.

State two: The code is lists, vectors, etc, including comments.
Stage two: Transform it, managing the comments as appropriate to your
transformation.

State three: The code is some other arrangement of lists, vectors,
etc, including comments.
State three: Write it back out.

State four: The code is sitting there being ASCII, transformed in some
useful way.  Comments have not been lost.

I'm sitting on the fence wrt actually doing it.  It would make a few
things I'd like to do easier, but as you point out, it probably won't
be rewarding beyond that.  

(my-quote ;;a b c
)

to place ;;a b c in the output.  Indeed, to do so wouldn't even be
meaningful.  You wouldn't expect these comments to behave differently.

You hint another point which is valid: (comment ... ) may not be the
best syntax (FWIW, it wasn't my suggestion).  :comment "a b c" or
&comment "a b c" may be more intuitive.

* Gareth McCaughan
|
| but unfortunately compilers aren't yet intelligent enough to give
| warning messages like
|
|   foo.lisp, line 100: comment doesn't accurately describe behaviour
|   of code.
|
| Alas, alas.

I would expect that when compilers get this good you would be able to
forget all about comments and instead do

  (query #'foo "What does it do?")

and get the result

  "The function foo frobnosticates its arguments, provided that ..."

Indeed, it shouldn't be too far from the day when you can say

  (write-function 'foo "Should frobnosticate its arguments provided
                        that ...")

And right now I'd be rather happy if I had this function and its
companion write-thesis since I could then finally get out of here and
enjoy the summer.

It's like giving the web police to search your web pages and remove anything
that looks like bad words, without regard to the fact that somewhere there
might be a page of the words that are to be removed, or pages of case law
about whether certain words are ok, only to find that the program that
"winnowed" these pages removed the offensive words, in spite of quotation
and/or other specialized context.

It's further complicated by the fact that you can't look for (QUOTE x)
becuase of situations like
 (case x ((quote defun) "delay a while") ((+ - *) "more aggressive"))
where (quote defun) is not a quoted form.

And it's further complicated by the fact that even ordinary code might
be "constructed" and not passed through the reader, so all macros everywhere
would have to know whether it was "constructed for printing" (that is,
contains helpful comments that might need removing) or "constructed for
execution" (that is, has comments in awkward places removed and is safe for
execution).

And it's further complicated by the fact that the "macro expansion" phase
is itself opaque to quoting so that
 (macrolet ((foo (x) `',x))
   (foo (x ;this is a test
         y)))
quotes x but you won't know it quotes x until you macroexpand it.  But being
a macroexpander, you're suggesting that the comments would be removed before
the macro gets to see the arguments.  (The halting problem gets involved here
if you want to solve this by code-analysis before actually running the macro.)

And it's further complicated by the fact that macros can expand into other
calls that might or might not be macros.  for example,
 (macrolet ((foo (x) (bar x)) (bar (x) `',x))
   (foo x))
Here it's clear that (foo x) must not only be entered but able to wholly
complete its action in order to find out whether the item in question
is quoted becuase it's BAR, not FOO, that decides it is quoted.

And it's further complicated by the fact that macros simultaneously think
of some forms as quoted and not by dual use of the self-same datum.  Consider
 (macrolet ((foo (x) `(if x ',x)))
   (foo (car ;this is like first
         '(t nil t t))))
where this expands to
   (if (car ;this is like first
         '(t nil t t))
       '(car ;this is like first
         '(t nil t t)))
where *if* ";this is like first" is a form, then I want it to be quoted by
the quote, but at the same time, I want it NOT to be a form in the IF.
So there is a problem of overconstrained situation.

  physics is a field very heavily optimized for those who understand it, or
  to put it another way: it is not a field well known to treat beginners
  nicely and many people are unable to grasp physics for a multitude of
  reasons.  you are therefore importing a trait from a beginner-hostile
  world to another world that is not nice to you as a beginner simply
  because you are _not_ a beginner in the former and are a beginner in the
  latter.  this is a ridiculously naïve approach to solving problems.

| I submit a sample comparison test below:
|
| (cond ( ( = 0 (mod n i) ) ( + i sum ) ) )

  Lisp programmers write (cond ((= 0 (mod n i)) (+ i sum))) and don't have
  a problem with it, but if they do, they break it into two or three lines,
  as several people have shown.  moreover, modern-day programmers appear to
  think color is the answer to all problems, so those who think this is
  worth doing would colorize the condition differently from the consequent.

  since you can add color without affecting the syntax, I would suggest you
  investigate that option as a beginner rather than suggest changes that
  could only affect future code, unless you were willing to make changes to
  the way you view code.  however, once you start down the road of making
  modifications to how you view code, there's nothing to stop you from
  making your proposed syntax change locally so that you see more diverse
  parens, but the file contains normal parens.

  I view it as a serious short-coming of programming textbooks that they
  don't deal with issues of representation and presentation.  it's integral
  to Lisp whose lists have a linked list representation and a parenthesized
  presentation, whose integers are sometimes made up of "bigits" but still
  print like normal integers, and myriad other useful layerings of abstract
  idea, machine representation, and presentation forms to the human.  take
  this Y2K silliness, which should be regarded as a monumental flaw in the
  way people are taught how to deal with dates and time.

| My point is to set this up in the "kernel" and enable everyone,
| especially struggling students (like myself), to easily read LISP.

  noble goal, but your suggestion is missing the boat -- students exposed
  to your redesign would be at loss if you weren't also going to re-publish
  all textbooks using your new system.  I suggest you use colors as your
  first approximation to ease readability, and if you can't do that, use a
  paren matching mode that highlights the form the cursor/mouse is over,
  and then resort to syntactic rewrites in the display to the user.  and if
  you aren't using Emacs, now is a good time to start.  programming it is
  even done in a sort of Lisp.  (don't pick up too many habits, though.)

(quote  ;;A
        ;;B
        C
        ;;D
        E)
=> (quote C E)

So it is exactly the same as what you get now.  

Also, quote is not generally inviolate:

(defun invade-quote ( &rest l)
  (if
    (eq (car l) 'quote  )
    (do-stuff-to (cdr l))
    l))

(invade-quote 'quote "a" )

or if you prefer,

(defun invade-quote-1 (a)
  (if
    (eq (car a) 'quote  )
    (do-stuff-to (cdr a))
    a))

(invade-quote-1 '( quote "a") )

[Two more variations on quoting snipped]

However, I don't think that argument works.  Already quote is not
capable of quoting comments, regardless that read looks at them.

If the fact that there is a stage where comments can be in all sorts
of places, including inside quote, disturbs you, just think of the
post-winnowing stage as being the one you want to work with.

And comments aren't going to be mechanically reasoned about anyways,
and aren't now, so where do you see a problem?

  consider a READ function that returns data useful to the Lisp system.
  consider a READ function that returns data useful to an editor of Lisp
  code.  now, why would these two functions have to be the same?  instead
  of making comments part of the core Lisp representation, realize that
  during the writing code, some of the time, the syntactic structure of the
  code will be invalid.  structure editors are a pain, pure and simple, and
  if you had used them, you would know, but if you haven't used them, you
  only see their good sides because the pain is non-obvious.

  editing is task that is not well understood.  look at Word, with billions
  of dollars of research and development having gone into it, and it still
  is a piece of crap.  look at Emacs, with incredible amounts of time and
  work having been poured into it by sometimes brilliant people, and we
  still don't quite understand what helps people write fast and correctly,
  and many serious mistakes are yet to be made, especially as people cling
  to the stupid window-icon-mouse-pointer technology as if it were good in
  itself.

  representation-wise, however, there are some clearly advantageous lessons
  to be learned from the editors of the past: (1) navigation is the single
  most important property of a good editor.  (2) aiding and abetting
  authors with the nitty-gritty details without undue interference makes
  them _really_ happy.  (3) don't ever mess with the user's actual textual
  formatting -- if the editor gets it wrong, user frustration reaches
  record heights if the editor reverses the corrections.  getting these
  three issues right means we need to keep track of a lot of context and
  information, and it just plain doesn't work to do it with simple-minded
  data structures and semi-intelligent searching operations.

| Such a lisp system really seems appealing.  To edit a function in such a
| system you would check it out (function definition printed from lisp
| memory to editor buffer), modify it, and check it back in again (evaluate
| it).  Or you could check an entire package out.  The system could also
| handle version control on its own.  A dumped image could be considered a
| version control snapshot.

  you know, the SGML people wanted exactly the same thing a few years ago,
  and I'm not exaggerating when I say that users hate it.  the single most
  important reason users hate stuff like this that they aren't allowed to
  break the rules at any time.  and the reason for that is that if you let
  them break the rules, you need to be very, very smart to manage to get
  back in line.  I don't think it's wise to ignore these issues from the
  point of view of "structured" document editors.  feel free to repeat
  their experiment, however, but please see what they did and make sure
  that you don't repeat mistakes of the past.

1: transform everything of the form
; somehting
into
(comment something)

2: remove everything of the form
(comment something)
from the data.

But that means that you can never include external representations of
lists beginning with the symbol 'comment in your code. A simple
example (in Scheme):

(define comment  ;comments from the program.
   (lambda (message)
       (display "Comment: ") (display message) (newline)))
(comment "Doing something now") ;never even seen by eval.

While this might be undesirable, it is not different in principle from
what we already have:

(define quote ;quote Shakespeare
  (lambda ()
    (display "To be or not to be / that is the question")))
 ; doesn't work either.

When combined with literals, however, this new form becomes very
confusing. While both
(append '(quote foo bar) lis) and
(append '(function foo bar) lis) works as expected,
(append '(comment foo bar) lis) is tranformed into
(append lis) before eval even sees it!

I think this is a serious weakness with the proposal.

I suppose this could be solved be by making
; something
expand into a list whose first element is not a symbol, but a special
object without an external representation, although that would break
the symmetry with quote and function .

This concern aside, I have another doubt regarding all this. How is
this new stage, when the code is read in, but the comments not yet
sifted, to be made available to the programmer? Redefining read to
return comment-forms as well as proper code would break programs which
do not expect them. On the other hand, creating a new procedure to
return that form would be no stronger then simply adding a new
procedure (definable in terms of read-char) to the standard library,
and thus this makes a language chage seem unnessesary. Finally, giving
the programmer _no_ acces to this stage opens up the old "If a tree
falls in the woods...". =)

That is, if  ;x becomes (comment x), the problem isn't that I can't
write (foo ;x
      )
any more.  It's that I can't write (foo (comment x))
any more because your code will be confused and think I'd written
(foo ;x
)
and that I want the (comment x) removed, when perhaps I don't.

(if
  (equal
    (quote
      ;;Quote can quote comments
      "Quote cannot quote comments")
    '"Quote cannot quote comments")
  "Tom is rite"
  "Kent is rite")

If this is Vilhelm's point about the exact representation as
(comment ...) confusing people and breaking old code, that's just a
representation issue.  Would you still be unsatisfied if the symbol
you couldn't use arbitrarily was "setq" or "quote"?

If this is a point about not being able to introduce the comment
symbol itself, I remind you of the pre-winnowing state, and that this
thread started with your reply to the proposal of a non-winnowing
macro variant.  (Which I now think was probably redundant, but that's
another issue)

You make a good case as to why it shouldn't be the exact symbol
"comment", which is probably used in someone's code somewhere, but
that's just a question of representation.

is also transformed into (append lis) before eval sees it.  Is that
equally disturbing?  If not, then it's just an issue of
representation.

Again, good case as to why the representation shouldn't be "(comment
...)".

As you point out, you want read itself to behave as before, but you
also want to be able to read without winnowing.  So IMO the roster of
functions would run something like:

;;Basic functions
read-and-dont-winnow
winnow
eval-but-dont-winnow-first ;;Basically only useful for optimization.

;;Backwards-compatible functions
read, equal to (winnow (read-and-dont-winnow ... ))
eval, equal to (eval-but-dont-winnow-first (winnow ...  ))

Also, eval would have to re-winnow whenever new lexical input was
seen, which basically means when it calls a macro.  Treat eval's
cousins that compile,etc the same way.  That seems to be about all
that would be needed.

It's not clear to me that this would buy enough for it to be worth
doing. I think Tom ought to implement his suggestion (so far as it's
possible to do this with present CLs, which ought to be pretty far),
show an actual benefit it would produce, and invite criticism from others;
this isn't a debate that works well in the abstract because it's
all about whether certain possibly-confusing possibly-useful behaviour
would end up being more confusing or more useful.

;;(This object)

cannot presently be quoted.  If you don't want to consider comments
objects, then there still wouldn't be a forbidden object.

If, OT3H, you want to call comments not objects now, but objects then,
why?  Because they are represented a certain way in memory?  If,
somewhere in the bowels of read, comments were already represented as
objects and then discarded, wouldn't that invalidate your distinction?

More importantly, as I pointed out earlier, quote is not inviolate.
It is respected by eval, but user code can mutate its contents.
(Example given earlier in the thread).  More pointedly, user code can
already remove lists beginning with the symbol 'comment from quotes;
ditto other proposed syntaxes.  So whatever you're trying to avoid
with quote is already part of Lisp.

You may be assuming that the only thing that gets compiled is code
that is acquired through text input.  Some code is consed up on the
fly, and that code is capable of being considerably more complex.
Further, some code is not externalizable, but can still be compiled
in-core.  If it comes in with READ, it can end up in structure, and
structure can be compiled.  You can't write code that manipulates
these things if there isn't the possibility it can be in code, and
if there is that possibility, it needs not to be gratuitiously removed.

We are looping again.

There are no polar complex numbers right now, but if you went to add them,
there are things I'd tell you you had to make those do, too.  You can't
say you could just add them "invisibly", which is the claim you've made
which I am trying to refute.

* Tom Breton
| Such objects already exist.  They are called comments.  I apologize for
| my tone, but after saying this in several ways already, I find myself a
| bit frustrated in explaining it.  Think of the comments as comments.

  consider a list (foo bar zot), where one of the elements is present in
  one interpretation of the list and absent in another.  what determines
  whether it be present or not?  in your design, the evaluator determines
  whether the element is present.  Kent objects to that.  I have suggested
  that you use two different READ functions, one which returns the list
  with that element, and one which doesn't, because it would be necessary
  to deal with the element in code, and so it would be necessary for the
  evaluator to be able to deal with that object, not just excise it.

  incidentally, you would need a comment object that is a bit more complex:
  it needs to know the column at which it starts, the number of semi's if
  you don't do that as part of the comment string, and whether it was a
  #|...|#-style or a ;-style comment.  an editor would need to know the
  exact location on the line of every form, not just comments, however, and
  it seems awfully naïve to believe that the only thing you'd need to get
  this right would be to retain comments in the flow of the other data.

  Tom, it's a bad idea.  it has been tried, and it was garbage collected.

  we have #. to help us READ those.

I seem to be saying this a lot recently, but anyone who wants to think
about this stuff should have a look at the interlisp-d environment
which did all this.  Although it's hard, and perhaps impossible now,
you need to actually spend some time using it, to see where the
problems are. Attempts to do all this from theory will just end up
reinventing the same slightly broken wheel.