package marker and readtable

[Jeff Greif]

In an attempt to turn the lisp reader into a tokenizer which recognizes
only whitespace as a separator and treats all other characters as
constituent, I created a new readtable using copy-readtable and for
various punctuation characters, executed
(set-syntax-from-char c #\a new-readtable)

Then I attempted to (read-from-string "abc,") and suchlike. This
collapsed when I came to (read-from-string "abc:") when I got an error
from an internal symbol parsing function that no characters followed the
package marker. Setting the syntax of colon is not enough to remove the
package-marker trait from it, at least in Corman Lisp 1.1b and 1.3.

Is there any way to change the package marker character in CommonLisp to
something (that my program won't find in its input) other than colon? I
missed it in the hyperspec if it's there?

Thanks,
Jeff

[Roger Corman]

On Fri, 22 Oct 1999 05:59:45 -0700, Jeff Greif <j...@trivida.com>
wrote:

Don't use READ for this. It should be more efficient (and work
correctly) to write your own function. Your problem is that the colon
character is a constituent character of a symbol (just like #\a)
already, but the symbol parsing logic attempts to treat it as a
package marker. I imagine you don't really want symbols. Probably you
would be better off with strings, in which case you could use the
function I included here. If you really want symbols, use INTERN on
the result to make a symbol out of it. You might have to modify this
to deal with end-of-file conditions..

(defun read-non-white-as-string (&optional
(stream *standard-input*)
(whitespace-chars '(#\space #\newline #\tab)))
;; skip white space
(do ((c (read-char stream)(read-char stream)))
((not (member c whitespace-chars))(unread-char c)))
;; gather non-white characters
(do ((chars nil)
(c (read-char stream)(read-char stream)))
((member c whitespace-chars)
(return (concatenate 'string (nreverse chars))))
(push c chars)))

The general rule is, if you aren't reading lisp syntax (and you
obviously aren't) don't use READ. It usually isn't worth the hassle or
the performance overhead. READ is basically a lisp parser, with some
ability to customize it.

Roger Corman

[Jeff Greif]

Thanks for the help. I did know how to do this without using the reader (and
have since done it), but, for a relatively poor reason, was trying to use the
minimum number of lines of code. My question about the package-marker is
still unanswered. Is it possible to change which character is used as the
package marker? Youth wants to know.
Jeff

Roger Corman wrote:

> On Fri, 22 Oct 1999 05:59:45 -0700, Jeff Greif <j...@trivida.com>
> wrote:
>
> >In an attempt to turn the lisp reader into a tokenizer which recognizes
> >only whitespace as a separator and treats all other characters as
> >constituent, I created a new readtable using copy-readtable and for
> >various punctuation characters, executed
> > (set-syntax-from-char c #\a new-readtable)

> ...

> >Is there any way to change the package marker character in CommonLisp to
> >something (that my program won't find in its input) other than colon? I
> >missed it in the hyperspec if it's there?
>

> Don't use READ for this. It should be more efficient (and work
> correctly) to write your own function. Your problem is that the colon
> character is a constituent character of a symbol (just like #\a)
> already, but the symbol parsing logic attempts to treat it as a

> package marker. ....

[Erik Naggum]

* Jeff Greif

| Is it possible to change which character is used as the package marker?

no. when you let the standard tokenizer deal with a token, you have no
control over how it interprets a token it has recognized to be of its
kind. the only way to get out of this is to rewrite the tokenizer. in
practice, this is not particularly hard.

#:Erik