Macro to enable Perl-style strings

[Nepheles]

I hacked together a short macro to enable you to use Perl-style
strings in Lisp. I.e., something like "Today is $DAY" will have $DAY
replaced with the (symbol-value) of DAY. Originally, I wrote it so
that the variables were extracted from a string at compile-time which
meant only raw strings (and not variables pointing to strings) could
be used. Changing it so that they're eval'd at run-time, allowing
variables, you're forced to use a (symbol-value) call. Unfortunately,
this forbids lexical variables.

In theory, you could make it declare each variable as special, but
there's no end to the amount of code you could mess up going down that
route.

Any suggestion as to how to improve this?

(There are a few custom functions used below, but they should be
fairly self-explanatory)

(defmacro evalstr (str)
; Perl-style strings with $-prefixed words being eval'd
(let ((g-toks (gensym))
(g-vals (gensym)))
`(let* ((,g-toks (str-tok ,str nil :delim #'(lambda (char)
(values (find char '(#\Space
#\` #\' #\) #\) #\=))
t))))
(,g-vals (mapcar #'(lambda (word)
(if (eql (char word 0) #\$)
; Intern word, stripped of preceding
'$'...
(symbol-value (intern (subseq word 1
(length word))))
; ...or else just the token unchanged
word)) ,g-toks)))
(apply #'mkstr ,g-vals))))

[Pascal Costanza]

Nepheles wrote:
> I hacked together a short macro to enable you to use Perl-style
> strings in Lisp. I.e., something like "Today is $DAY" will have $DAY
> replaced with the (symbol-value) of DAY. Originally, I wrote it so
> that the variables were extracted from a string at compile-time which
> meant only raw strings (and not variables pointing to strings) could
> be used. Changing it so that they're eval'd at run-time, allowing
> variables, you're forced to use a (symbol-value) call. Unfortunately,
> this forbids lexical variables.
>
> In theory, you could make it declare each variable as special, but
> there's no end to the amount of code you could mess up going down that
> route.
>
> Any suggestion as to how to improve this?

I am not sure, but maybe PROGV can help you here. Or you could try to
put these things into macros. They can access the lexical environment,
albeit in limited ways, via environment objects.

Pascal

--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."

[Zach Beane]

neph...@myrealbox.com (Nepheles) writes:

> I hacked together a short macro to enable you to use Perl-style
> strings in Lisp. I.e., something like "Today is $DAY" will have $DAY

> replaced with the (symbol-value) of DAY. [...]

See also:

http://www.weitz.de/cl-interpol/

Zach

[Erik Naggum]

* Nepheles

| I hacked together a short macro to enable you to use Perl-style
| strings in Lisp. I.e., something like "Today is $DAY" will have $DAY

| replaced with the (symbol-value) of DAY. [...] Unfortunately, this
| forbids lexical variables.

Let the caller of the macro evaluate the expression, instead.

E.g., the macro call (evalstr "mumble $foo $bar $zot mumble") should
return (concatenate 'string "mumble " foo " " bar " " zot " mumble").

You should use READ-FROM-STRING to pick up the symbol names from the
string, not try to parse and hack things together yourself. Just give
it a :START argument of the position following the $. Then you can
use any Common Lisp expression you want after the $, not just simple
variable names. I recommend :PRESERVE-WHITESPACE T, so any whitespace
that terminates the expression is part of the string literal. This
function returns the position following what it just read.

Once you have this working, consider using a reader macro, instead, so
that, e.g., $"mumble $foo $bar $zot mumble" returns the expression at
read-time. Some languages need special syntax to prevent the name of
the variable from running into the following text. If you follow my
suggestions, you can use "foo$(values bar)zot" or any other expression
that returns the evaluated value. (Note that QUOTE does not.) But if
you're really into syntactic sugar, you could write a reader macro for
{...} that returns (values ...).

I have not looked at the pre-existing solutions for interpolating
strings, but I would be surprised if they did not do most if not all
of the above.

--
Erik Naggum | Oslo, Norway

Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.

[Adam Warner]

Nepheles,

I've quickly written this for you to demonstrate the power of character
macros. I made it dispatch off # so it won't interfere with the standard
double quote macro character.

(defun |read-#"| (in sub-char num-arg)
(declare (ignore sub-char num-arg))
(macrolet ((msos () '(make-string-output-stream))
(goss (stream) `(get-output-stream-string ,stream)))
(let ((out (msos)) escaped code var var-p)
(loop for char = (read-char in nil nil t)
until (and (not escaped) (char= char #\")) do
(cond ((char= char #\\) (setf escaped t))
(t (cond (var-p (cond ((or (upper-case-p char) escaped)
(write-char char var)
(setf escaped nil))
(t (push (intern (goss var)) code)
(setf var (msos))
(setf var-p nil)
(setf out (msos))
(write-char char out))))
((and (char= char #\$) (not escaped))
(push (goss out) code)
(setf out (msos))
(setf var (msos))
(setf var-p t))
(t (write-char char out)
(setf escaped nil))))))
(when var-p (push (intern (goss var)) code))
(push (goss out) code)
`(concatenate 'string ,@(nreverse code)))))

(set-dispatch-macro-character #\# #\" #'|read-#"|)

First let's understand how it works:

(macroexpand '#"My name is $LASTNAME, $FIRSTNAME $LASTNAME.")
=> (concatenate 'string "My name is " lastname ", " firstname " " lastname ".")

I am only recognising unescaped variable names that have uppercase
letters. Notice I am leveraging the fact that uppercase strings intern to
variables of the appropriate case for ANSI Common Lisp.

Let's put it into practice:

(let ((firstname "James") (lastname "Bond"))
#"My name is $LASTNAME, $FIRSTNAME $LASTNAME.")
=> "My name is Bond, James Bond."

I coded in escaping so you can avoid the variable name translation:

#"My name is \$LASTNAME, \$FIRSTNAME \$LASTNAME."
=> "My name is $LASTNAME, $FIRSTNAME $LASTNAME."

Or process symbol names with numbers and other non-uppercase characters:

(let ((var1 "James") (var2 "Bond")) #"My name is $VAR\2, $VAR\1 $VAR\2.")
=> "My name is Bond, James Bond."

Modify to suit. The code is dedicated in perpetuity to the public domain.

Regards,
Adam

[Adam Warner]

Nepheles,

'#"My name is $LASTNAME, $FIRSTNAME $LASTNAME."

[Adam Warner]

Nepheles,

I've quickly written this for you to demonstrate the power of character
macros. I made it dispatch off # so it won't interfere with the standard
double quote macro character.

;;Superseded post:
;;I've modified the code to reuse the string output streams
;;

(defun |read-#"| (in sub-char num-arg)
(declare (ignore sub-char num-arg))
(macrolet ((msos () '(make-string-output-stream))
(goss (stream) `(get-output-stream-string ,stream)))

(let ((out (msos)) (var (msos)) var-p escaped code)

(loop for char = (read-char in nil nil t)
until (and (not escaped) (char= char #\")) do
(cond ((char= char #\\) (setf escaped t))
(t (cond (var-p (cond ((or (upper-case-p char) escaped)
(write-char char var)
(setf escaped nil))
(t (push (intern (goss var)) code)

(setf var-p nil)

(write-char char out))))
((and (char= char #\$) (not escaped))
(push (goss out) code)

[Joe Marshall]

neph...@myrealbox.com (Nepheles) writes:

> I hacked together a short macro to enable you to use Perl-style
> strings in Lisp. I.e., something like "Today is $DAY" will have $DAY
> replaced with the (symbol-value) of DAY.

I felt rather dumb when it was pointed out to me that since " is a
terminating macro character, you can simply write things like this:

(concatenate 'string "Today is "DAY)

or

(concatenate 'string "My name is "LASTNAME", "FIRSTNAME" "LASTNAME".")

The effort involved in parsing interpolation markers (especially since
we have format) is just not worth it.