Adding syntax in lisp.

[Pascal J. Bourguignon]

Well, this would be obvious to most lispers, but since I've seen some
horrifying blog about how to add an "in" keyword to some popular
language, I'm posting here the answer I made there describing how one
adds syntactic elements to the language in lisp.

For the context, the original problem was to add an "in" keyword
allowing to search for an element in a vector, or a substring in a
string:

element in vector --> boolean
substring in string --> boolean


1) user functions are indistinguishable syntactically from lisp
operators.

Just for comparison, let's add a IN operator in Lisp.

(defun in (element sequence)
(if (stringp sequence)
(search element sequence)
(find element sequence :test (function equal))))

(defparameter *words* #("hello" "world" "foo" "bar"))
(defparameter *string* "Lisp is funnier!")

(in "hello" *words*) --> "hello" ; any non NIL value is true in lisp.
(in "foo" *words*) --> "foo"
(in "blub" *words*) --> nil

(in "Lisp" *string*) --> 0 ; still true (remember, not NIL).
(in "fun" *string*) --> 8
(in "PHP" *string*) --> nil

Since in lisp we use a uniform symbolic-expression (sexp) syntax for all
expressions, in this case we don't need to use a more sophisticated
feature than a mere function to add an operator to the language.


2) user macros are indistinguishable semantically from lisp operators:

Another level is using macros. In Common Lisp, while loops are written:

(loop while <condition> do <something>)


Assume we'd want to run some emacs lisp code that uses the following
syntax for while loops:

(while <condtion> <something>)

we could add this syntax to Lisp, still without having to rely to
heavy weaponery, just writing a simple macro:

(defmacro while (condition &body body)
`(loop while ,condition do (progn ,@body)))

(let ((i 3))
(while (plusp i)
(print i)
(decf i))
(terpri))

3
2
1
nil

In Lisp, using lisp macros we can very easily add syntactic constructs
to the language.



3) user reader macros are indistinguishable from lisp syntaxic notations
(apart of course from their syntactic aspect).


Now, as for the addition of a really new syntax to Common Lisp, let me
present a trivial and a non trivial example.


Assume we have a library using "points" represented as integers where
the bits 0-15 are the horizontal coordinate, and the bits 16-31 are
the vertical coordinate.

(defun make-point (h v)
(dpb (ldb (byte 16 0) v) (byte 16 16) (ldb (byte 16 0) h)))

(make-point 10 20) --> 1310730

Since we will use in our programs a lot of literal points, we don't
want to type (make-point 10 20) each time, and furthermore this
function would be called at run-time, which for literal points could
be avoided. Instead, we will use the syntax @(10 20) to read the
integer encoding the point at coordinates 10;20, writing this simple
function:

(defun at-reader-macro (stream ch)
"@(x y) reads a Point."
(declare (ignore ch))
(let ((coord (read stream)))
(if *read-suppress*
(values)
(values (apply (function make-point) coord)))))

and installing it into the lisp reader with:

(set-macro-character #\@ 'at-reader-macro)

@(10 20) --> 1310730



Here is a non trivial example of new syntax. In Objective-C, sending
a message called for example "displayRectIgnoringOpacity:inContext:"
to an object named "myView" written in brackets like this:

[myView displayRectIgnoringOpacity:YES inContext:myContext];

When writing Lisp code on MacOSX, we would want to use a similar
syntax to send Cocoa messages to Cocoa objects. But this is a syntax
entirely different from the symbolic-expression syntax of Lisp. We
can still do that rather trivially, and still without using bulldozers
to crush a fly. The Lisp syntax to send the same message would be:

(objc:send myview :display-rect-ignoring-opacity yes :in-context mycontext)


We can easily write a function that reads from a stream a text such as
"[myView displayRectIgnoringOpacity:YES inContext:myContext]", and
that will return a symbolic expression such as (objc:send myview
:display-rect-ignoring-opacity yes :in-context mycontext). Having
such a so called "reader macro" function named
"objcl-expression-reader-macro", we can install it in Common Lisp
with:

(set-syntax-from-char #\[ #\() ; indicates that [] are used as ().
(set-syntax-from-char #\] #\))
(set-macro-character #\[ 'objcl-expression-reader-macro nil)

and then on, type:

[myView displayRectIgnoringOpacity:YES inContext:myContext]

instead of the lisp:

(objc:send myview :display-rect-ignoring-opacity yes :in-context mycontext)


Have a look at:
https://groups.google.com/forum/?hl=en&fromgroups#!topic/comp.lang.lisp/AkU0JCnGGAA
and:
https://gitorious.org/com-informatimago/com-informatimago/trees/master/objcl
for more details.


This is done just writing simple lisp code, without invoking external
tools or any other heavy machinely.


--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

[Raffaele Ricciardi]

Thank you for this post, Pascal. Please do not forget to add it to the
"My USENET posts of interest" on your website.

On 07/28/2012 01:12 PM, Pascal J. Bourguignon wrote:>
> Assume we'd want to run some emacs lisp code that uses the following
> syntax for while loops:
>
> (while <condtion> <something>)
>
> we could add this syntax to Lisp, still without having to rely to
> heavy weaponery, just writing a simple macro:
>
> (defmacro while (condition &body body)
> `(loop while ,condition do (progn ,@body)))
>

Since it is Emacs Lisp, let's not forget the `declare' statement, to let
Emacs know how to indent the macro properly:

(defmacro while (condition &body body)

(declare (indent 1))

[Pascal J. Bourguignon]

This is not emacs lisp, this is Common Lisp, trying to run some emacs
lisp code. To add this declaration in Common Lisp, you would have to
declare it first.

(declaim (declaration indent))

(defmacro while (condition &body body)
(declare (indent 1))
`(loop while ,condition do (progn ,@body)))

But I don't know any software using it.