Anonymous packages
Alessio Stalla
effecting reader, and its potential drawbacks. I do like the CL
reader, packages, and related stuff. However, I was thinking whether
the introduction of anonymous, and potentially temporary, packages
would be a useful addition to the language, and "solve" some of the
problems a side-effecting reader has.
Consider:
(let ((*package* (make-anonymous-package :use '(:cl))))
(eval (read-from-string "(flet ((foo () ...))
(foo))")))
In this case the symbol FOO is not interned in CL-USER (or whatever
the current package is), but in a freshly-created package with no
name.
An anonymous package is a package like any other, but it's not
globally registered anywhere (a bit like an uninterned symbol): thus
import, export & c. should work as expected, as long as you hold a
reference to the anonymous package.
A symbol interned in an anonymous package is printed like an
uninterned symbol, it is therefore unreadable unless *package* is set
to the anonymous package.
If you don't keep references to the package nor to its symbols, it
will be garbage collected, and any side effects regarding symbols
introduced by the reader will go away with it.
I have in mind one use case in particular for this, but I believe
there are others, generally when you need to preprocess Lisp code
coming as a string from the outside world, and don't want to pollute
your current package with spurious symbols.
Such a facility cannot be implemented in portable CL, as packages are
specified to always have a name, and are globally registered in a
table hidden from the user. One could fake it by inventing extremely
improbable package names and by explicitly destroying unused packages
with DELETE-PACKAGE, however this is not a very clean solution.
Thoughts?
--Alessio
Stanislaw Halik
> Such a facility cannot be implemented in portable CL, as packages are
> specified to always have a name, and are globally registered in a
> table hidden from the user. One could fake it by inventing extremely
> improbable package names and by explicitly destroying unused packages
> with DELETE-PACKAGE, however this is not a very clean solution.
One can FIND-PACKAGE (format nil "FOO-~D" i) for I from 0, when it
returns NIL, make a package with that name, then do some stuff on it
with UNWIND-PROTECT DELETE-PACKAGE.
That seems to be the way ASDF is doing it.
Ron Garret
Stanislaw Halik <sth...@test123.ltd.pl> wrote:
Or use a GUID.
rg
Pascal Costanza
Another approach is to use the Java package name convention:
"com.foo.bar.baz.0123" - then you can use your own convention for
'locally' generated, but readable IDs.
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
Alessio Stalla
> > Stanislaw Halik <stha...@test123.ltd.pl> wrote:
>
> >> thus spoke Alessio Stalla <alessiosta...@gmail.com>:
>
> >>> Such a facility cannot be implemented in portable CL, as packages are
> >>> specified to always have a name, and are globally registered in a
> >>> table hidden from the user. One could fake it by inventing extremely
> >>> improbable package names and by explicitly destroying unused packages
> >>> with DELETE-PACKAGE, however this is not a very clean solution.
> >> One can FIND-PACKAGE (format nil "FOO-~D" i) for I from 0, when it
> >> returns NIL, make a package with that name, then do some stuff on it
> >> with UNWIND-PROTECT DELETE-PACKAGE.
>
> >> That seems to be the way ASDF is doing it.
>
> > Or use a GUID.
>
> Another approach is to use the Java package name convention:
> "com.foo.bar.baz.0123" - then you can use your own convention for
> 'locally' generated, but readable IDs.
These are all good advices about obtaining unused package names,
however they are "hacks" around the fact that anonymous packages do
not exist. I suppose that for practical purposes those "hacks" work
just fine, however I wonder why anonymous packages aren't in CL, given
that we can have, e.g., anonymous functions and classes, and that in
general CL distinguishes (at least conceptually) between creating a
new object and registering it under a name (e.g. defun can be
implemented as (setf (symbol-function foo) (lambda ...))). Maybe no-
one thought about it, or are there hidden dangers in them?
Alessio
Stanislaw Halik
> Maybe no- one thought about it, or are there hidden dangers in them?
For one, it would be hard to print symbols interned in anonymous packages.
Ron Garret
Stanislaw Halik <sth...@test123.ltd.pl> wrote:
No, that would be easy, no harder than printing out uninterned symbols.
The hard part would be reading them back in again.
rg
Barry Margolin
Ron Garret <rNOS...@flownet.com> wrote:
> In article <h6v88u$2qli$1...@opal.icpnet.pl>,
> Stanislaw Halik <sth...@test123.ltd.pl> wrote:
>
> > thus spoke Alessio Stalla <alessi...@gmail.com>:
> >
> > > Maybe no- one thought about it, or are there hidden dangers in them?
> >
> > For one, it would be hard to print symbols interned in anonymous packages.
>
> No, that would be easy, no harder than printing out uninterned symbols.
Uninterned symbols are easy, they all get the #: prefix, and they aren't
expected to be readable.
> The hard part would be reading them back in again.
Right, so what's the point?
Packages are not intended to be a general purpose database, they're a
clumsy module system that operates at read time. If you want something
more general, use a hash table (a package is basically just a hash table
that's used in a particular way by the reader).
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Ron Garret
Barry Margolin <bar...@alum.mit.edu> wrote:
> In article <rNOSPAMon-A6C8D...@news.albasani.net>,
> Ron Garret <rNOS...@flownet.com> wrote:
>
> > In article <h6v88u$2qli$1...@opal.icpnet.pl>,
> > Stanislaw Halik <sth...@test123.ltd.pl> wrote:
> >
> > > thus spoke Alessio Stalla <alessi...@gmail.com>:
> > >
> > > > Maybe no- one thought about it, or are there hidden dangers in them?
> > >
> > > For one, it would be hard to print symbols interned in anonymous packages.
> >
> > No, that would be easy, no harder than printing out uninterned symbols.
>
> Uninterned symbols are easy, they all get the #: prefix, and they aren't
> expected to be readable.
>
> > The hard part would be reading them back in again.
>
> Right, so what's the point?
Just trying for a bit of wry humor actually. Guess I should keep the
day job.
rg
Kaz Kylheku
> Some time ago there were some discussions on this NG about CL's side-
> effecting reader, and its potential drawbacks. I do like the CL
> reader, packages, and related stuff. However, I was thinking whether
> the introduction of anonymous, and potentially temporary, packages
> would be a useful addition to the language, and "solve" some of the
> problems a side-effecting reader has.
Paste number 72068: PKG: Finer-grained read-time package control
http://paste.lisp.org/display/72068
> An anonymous package is a package like any other, but it's not
> globally registered anywhere (a bit like an uninterned symbol): thus
PKG has support for anonymous packages in the scope of the #@ read macro.
For example
[1]> #@() '(a b c)
(G3157::A G3157::B G3157::C)
This particular #@ has an empty list of directives, so all it does is
establish an anonymous package within which the following form is read.
The anonymous package has symbols from the surrounding package, by default:
[2]> #@() '(list append cons)
(LIST APPEND CONS)
But we can suppress that by specifying an empty INHERIT list:
[3]> #@(inherit) '(list append cons)
(G3163::LIST G3163::APPEND G3163::CONS)
> import, export & c. should work as expected, as long as you hold a
> reference to the anonymous package.
PKG has a number of directives with which you get fine grained export
and import control.
For instance, suppose we want to define a macro called FOO.
Ideally, when we define this macro, the only side effect we want on
the environment is that the symbol FOO is instantiated (if necessary)
and bound to the macro. But in Lisp, the DEFMACRO form will have other
side effects: it will potentially intern a bunch of other symbols.
There is also the risk that some symbols do not intern, leading
to name capture.
Using PKG, we can solve the first problem like this:
#@(keep foo)
(defmacro foo (a b c)
`(list ,a ,b ,c))
If we evaluate the DEFMACRO form quoted, you can see what's going on:
[4]> #@(keep foo) '(defmacro foo (a b c) `(list ,a ,b ,c))
(DEFMACRO FOO (G3170::A G3170::B G3170::C)
`(LIST ,G3170::A ,G3170::B ,G3170::C))
As you can see, FOO has been propagated into the surrounding package
because of the KEEP directive, but the other symbols A, B, C have
not. The Common Lisp symbols do represent themselves.
But still have the problem that if A, B or C, had been interned
previously, they will be inherited, just like DEFMACRO and LIST.
What we can do for that is use the UNIQUE specifier.
Now that we have two directives, we have to wrap them with a list, too:
#@((keep foo)(unique a b c))
(defmacro foo (a b c)
`(list ,a ,b ,c))
[5]> '(a b c) ;; intern A, B and C
(A B C)
[6]> #@((keep foo)(unique a b c)) '(defmacro foo (a b c) `(list ,a ,b ,c))
(DEFMACRO FOO (G3197::A G3197::B G3197::C)
`(LIST ,G3197::A ,G3197::B ,G3197::C))
The UNIQUE specifier ensures that symbols A, B and C are not imported, but are
unique to the anonymous package.
The #@ syntax nests, and there are directives which have to do with the
interaction among nested forms.
#@() #@() FORM means that FORM is read in the anonymous package established by
the second #@, which is interpreted in the environment set up by the first #@.
I.e. it will inherit symbols from the outer anonymous package, and push
symbols into it, etc.
The TOP directive can be used in a nested #@ to bypass the surrounding #@
nesting and establish a direct relationship with the outer package.
A similar directive called IN-PACKAGE will establish a relationship between
the anonymous package and an arbitrary named package:
#@((in-package mymacros)
(keep foo)
(unique a b c))
(defmacro foo (a b c) `(list ,a ,b ,c))
Now FOO is interned into the MYMACROS package; A B and C are within
the anonymous package only and become homeless symbols before the #@ macro
returns to the reader.
There are also directives to bring symbols in from specific packages:
#@(from <package> import <symbol1> <symbol2> ...) FORM
#@(use <package>) ;; all exported symbols of <package>
> Such a facility cannot be implemented in portable CL, as packages are
> specified to always have a name, and are globally registered in a
> table hidden from the user. One could fake it by inventing extremely
> improbable package names and by explicitly destroying unused packages
> with DELETE-PACKAGE, however this is not a very clean solution.
I originally implemented the behavior of *implicitly* destroying the anonymous
packages in the final phase of processing each #@ read macro, but for some
reason I changed the behavior. I can't remember why! It might have been for
convenience; by not making the symbols homeless, we keep the read/print
consistency, which is better for debugging.
Kaz Kylheku
> however they are "hacks" around the fact that anonymous packages do
> not exist.
That's like saying that there is no such thing as an anonymous object
because all objects are named by a machine address, and the memory
allocator is just a hack for obtaining an unused address.
:)
Kaz Kylheku
The symbols maintain read/print consistency if you simply do not not delete
their anonymous package.
Ron Garret
Kaz Kylheku <kkyl...@gmail.com> wrote:
Not if the anonymous packages don't have names. :-)
rg
budden
My opinion is that CL package system can not be cured. It needs to be
redesigned completely.
I'm using several languages, including C, C++, (Object) Pascal, SQL
and Common Lisp. Also
I took a look at Python. Symbol concept is excellent, but CL's package
system is the worst one.
Just complaining won't help. So I tried to do some improvements, some
of them are experimental,
some are used on a daily basis now:
1. I have made hierarchical packages and per-package aliases more
portable.
2. I have made a widely portable reader hack which allows keyword::(a
b c)
to be read as (:a :b :c). And even
foo::(bar::(a _::b) to be read as
(bar::a foo::b), where _ designates "outer" package on the reader
stack.
Readtables can be associated to a package and they are also switched
while
processing package:: extended syntax. So, I could, for example,
associate special readtable with date package, and then read dates as
date::2009-08-27
Also this allows to overload readmacros, using the same character for
different
readmacros in different packages.
3. I have implemented an extension to (in-package) form.
Now I can use it as (in-package :foo :bar :bar), so temporary
namespace is created
where all non-clashing symbols from foo, bar and baz are present,
while new symbols
are interned into foo and attempt to read unqualified clashing symbol
causes an error.
Alternatively, I have made "merge-packages-and-reexport" form which
statically merges
all non-clashing external symbols from several packages to a new
package.
4. There was an experimental code which allows "symbol-readmacros"
which are like ordinary
readmacros, but are associated to symbols, not to characters. This
implements readmacro concept
in a more scalable fashion. E.g. I could write
> sql select count(*) from a table;
so that I embed sql reader in a lisp reader smoothly and without using
readmacros (there is just too small amount of them).
"Sql" here has an associated symbol-readmacro which turns sql reader
on.
5. I have experimented with triggers which work at intern time. With
them, I can trap and prohibit occasional creation of "trash" symbols,
which often happen when I reorganize my packages.
What was the result? At c.l.l, I didn't attract much attention, most
of the replies was a critics. No one wanted to help
the development. Now I use some of the code on a daily basis, but
something is still to be done and I have no resources for that.
Anyway, some things would never be as smooth as native CL features.
And my code adds complexity. The most disappointing is that
I still have no way to make things modular. I can make modules with
ease in Pascal, but not in Common Lisp. I came to a conclusion
that I would need create a new language inside a CL to feel myself
comfortable. Yes, CL package system is flexible, but
it requires a lot of manual work to shape it to what I want it to be.
This is simply a loss of productivity. And I have started
to use lisp as it seemed more productive compared to conventional
languages.
You can find my code if you look for "cl-fix" and "budden-tools" on
c.l.l archives. Anonymous package extension can be added
simply to it, as I can trap any references to a package. So, I could
implent not only an anonymous package, but also
a package with temporary lexical name.
> One could fake it by inventing extremely
improbable package names and by explicitly destroying unused packages
with DELETE-PACKAGE, however this is not a very clean solution.
I think, if you want to have practical gain, this is an optimal (and
the only available) solution.
Do you have unlimited resources? No. Do you have a power to reissue CL
standard and make implementors
follow it? No. So I think it would be best to make temporary packages
with some prefix.
E.g. API could be like this:
(defun make-package-with-prefix (prefix &rest defpackage-options)
"Creates a new package with name starting from a prefix"
(See asdf::make-temporary-package for an example).
(defmacro with-temporary-package (prefix &rest defpackage-options)
&body body)
Package with a name starting from evaluated prefix is created.
*package* is bound to a package created
and the body is evaluated. Then the package is deleted.
budden
>
> http://paste.lisp.org/display/72068
better,
but complex, undocumented and unfinished :P
Rob Warnock
+---------------
| 4. There was an experimental code which allows "symbol-readmacros"
| which are like ordinary readmacros, but are associated to symbols,
| not to characters. This implements readmacro concept in a more scalable
| fashion. E.g. I could write
| > sql select count(*) from a table;
| so that I embed sql reader in a lisp reader smoothly and without
| using readmacros (there is just too small amount of them). "Sql" here
| has an associated symbol-readmacro which turns sql reader on.
Speaking just to this one point...
Note that one can get a similar expansion of readmacro space
by defining a single readmacro that first reads a symbol then
dispatches to the corresponding extension reader. E.g., using
"#!"[1] for this, your SQL example would read:
#!sql select count(*) from a table;
Other people have previously suggested using "#[sql ... ]" for this,
since they like to use READ-DELIMITED-LIST, but I think in the
above case "#!sql ... ;" is fine, too.
-Rob
[1] Yes, "#!" is often reserved for "#!/path/to/interpreter",
but you could trivially have the "#!" readmacro distinguish
between "#!/" and "#!othersymbol".
-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Ron Garret
rp...@rpw3.org (Rob Warnock) wrote:
> budden <budde...@mail.ru> wrote:
> +---------------
> | 4. There was an experimental code which allows "symbol-readmacros"
> | which are like ordinary readmacros, but are associated to symbols,
> | not to characters. This implements readmacro concept in a more scalable
> | fashion. E.g. I could write
> | > sql select count(*) from a table;
> | so that I embed sql reader in a lisp reader smoothly and without
> | using readmacros (there is just too small amount of them). "Sql" here
> | has an associated symbol-readmacro which turns sql reader on.
> +---------------
>
> Speaking just to this one point...
>
> Note that one can get a similar expansion of readmacro space
> by defining a single readmacro that first reads a symbol then
> dispatches to the corresponding extension reader. E.g., using
> "#!"[1] for this, your SQL example would read:
>
> #!sql select count(*) from a table;
>
> Other people have previously suggested using "#[sql ... ]" for this,
> since they like to use READ-DELIMITED-LIST, but I think in the
> above case "#!sql ... ;" is fine, too.
It is further worth pointing out that if we can get everyone to agree on
which character should be used to introduce a symbol reader macro this
would help solve the more general problem of collisions in the syntax
extension space. This problem is already felt in practice. For
example, I would like to use Kaz's PKG package, but it uses the #@
dispatch reader macro, which collides with CCL's use of that reader
macro for constant NS-String objects. If instead of #@ it was e.g.
#!kaz:pkg... that kind of collision could not happen.
rg
Rob Warnock
+---------------
| rp...@rpw3.org (Rob Warnock) wrote:
| > Note that one can get a similar expansion of readmacro space
| > by defining a single readmacro that first reads a symbol then
| > dispatches to the corresponding extension reader. E.g., using
| > "#!"[1] for this, your SQL example would read:
| > #!sql select count(*) from a table;
| It is further worth pointing out that if we can get everyone to agree on
| which character should be used to introduce a symbol reader macro this
| would help solve the more general problem of collisions in the syntax
| extension space. This problem is already felt in practice. For
| example, I would like to use Kaz's PKG package, but it uses the #@
| dispatch reader macro, which collides with CCL's use of that reader
| macro for constant NS-String objects. If instead of #@ it was e.g.
| #!kaz:pkg... that kind of collision could not happen.
As I noted in footnote [1] of the previous message:
[1] Yes, "#!" is often reserved for "#!/path/to/interpreter",
but you could trivially have the "#!" readmacro distinguish
between "#!/" and "#!othersymbol".
So I suggest "#!" as the choice for *both* "#!/path/to/interpreter"
[interpreted as a comment] and "#!foo-pkg:foo".
Now we can have a nice flamefest about the pesky details: ;-} ;-}
1. Which should be the default package for FOO in "#!foo"?
2. What should be the protocol should be for "#!foo-pkg:foo"?
Should it:
(a) just call FOO-PKG:FOO with the usual readmacro protocol
[args passed on from the "#!" function]?
(b) Look up the symbol FOO-PKG:FOO in some global hashtable
and call the hash value with the usual readmacro protocol?
[In which case the KEYWORD package would be a perfectly
reasonable default to read "#!foo", since it wouldn't be
trying to define a KEYWORD:FOO function.]
(c) Either (a) or (b), but with a different protocol? That is, the
"#!" function gets called with (STREAM DISPATCH-CHAR INFIX-NUM),
but if all of the symbol reader macros start with "#!" then
we know that DISPATCH-CHAR will be #\!, so maybe we don't need
to specify that in the protocol.
(d) All of the above [relying on the registration function to
cache enough data to keep it all straight]? None of the above?
Something different altogether?
[My own preference? Document some one answer as "suggested practice",
but don't try to make it a "standard". Leave hooks for adding variations.]
-Rob
budden
And, of course, I like to do them from my CL prompt.
It is boring to enter #! every time (using weak ring
and little fingers on both hands). I believe CL
should be as convinient as command-line shell!
Currently I still use something like
fsel "select count(*) from a table"
or
> fsel `(:l select "count(*)" from a table)
But at least now I know that I can save myself
from typing innecessary characters. In extreme, this
could be made like
> s select * from table;
or
> ~ select * from table;
Also, using _any_ dispatch macro character is a
modularity problem. There are just too few possible
macro characters and too many language extensions.
I would fell into trouble when I would try to marry two
language extensions. But there are enough symbols and
they can be qualified with a package name (or short local
package alias). Having per-package readtable associations,
I now use #L syntax from iterate massively: I know
that I can always resolve the conflict if some other
library would also define #L.
vippstar
> > #!sql select count(*) from a table;
>
> I do at least hundreds sql queries a day.
> And, of course, I like to do them from my CL prompt.
> It is boring to enter #! every time (using weak ring
> and little fingers on both hands). I believe CL
> should be as convinient as command-line shell!
That's not a design problem in CL or the #! idea: it's *your* text
editting problem. Text editors are designed to solve such problems.
budden
> editting problem. Text editors are designed to solve such problems.
Functional keys are not very convinient for a touch method with ten
fingers.
Their layout depends on a concrete keyboard. Most other simple key
combinations
are occupied already in a EMACS. Also, it is not convinient to have
#!sql text at the beginning of any sql command while I browse my
history
or lisp interaction buffer.
1. It is less readable.
2. When I use history search, I need to type in entire #!sql statement
before
I can start the search itself. Should I create a special command for
sql
statements history search?
And, this is not mine problem at all: after I take time to
incorporate
sql statement reader into my environment, I would use syntax like
> ~s * from table;
or
> ~u table set foo = bar where id = :*my-id*;
and so on. It is a problem of those lisp users who would like to use #!
sql
Pascal J. Bourguignon
>> #!sql select count(*) from a table;
> I do at least hundreds sql queries a day.
> And, of course, I like to do them from my CL prompt.
> It is boring to enter #! every time (using weak ring
> and little fingers on both hands). I believe CL
> should be as convinient as command-line shell!
>
> Currently I still use something like
> fsel "select count(*) from a table"
> or
>> fsel `(:l select "count(*)" from a table)
>
> But at least now I know that I can save myself
> from typing innecessary characters. In extreme, this
> could be made like
>> s select * from table;
> or
>> ~ select * from table;
If you really type that many sql, perhaps you should do:
C/LISP[1]> (sql)
C/SQL[2]> select count(*) from table1;
...
C/SQL[3]> select * from ,(what-table);
...
...
C/SQL[n-1]> ,(compute-some-sql-query);
...
C/SQL[n]> quit;
C/LISP[n+1]> (+ 1 2)
3
...
--
__Pascal Bourguignon__
budden
This is not that elegant. Lisp is extensible in its
original idea. CL creates some obstacles to that
extensibility. I prefer to overcome that problems
instead of learning to survive with to them. And
it looks like I have already achieved my goals.
Time will tell. On the other hand, what you
suggest is like what I suggest.
> ~s count(*) from table1;
^here sql reader is initiated
with ~s symbol-readmacro. "Select"
is put on that reader's input stream.
^ here
sql reader ends reading of a
statement and returns it as a lisp code.
> C/SQL[3]> select * from ,(what-table);
This would fail here:
> select id,(select name from t2) from t1;
We need some another escape construct.
Rob Warnock
+---------------
| > #!sql select count(*) from a table;
|
| I do at least hundreds sql queries a day.
| And, of course, I like to do them from my CL prompt.
| It is boring to enter #! every time (using weak ring
| and little fingers on both hands). I believe CL
| should be as convinient as command-line shell!
O.k., fine! ;-} ;-} Then you might like this instead:
cmu> (define-symbol-macro sql (pg-query-list (read-line)))
SQL
cmu> sql select count(*) from toy
(("count") ("9"))
cmu> sql select * from toy limit 3
(("season" "medium" "title" "code")
("fall" "tape" "My Favorite Thanksgiving" "16")
("xmas" "book" "My Favorite Christmas" "2")
("xmas" "video" "The Grinch who Stole Christmas" "4"))
cmu>
And to get multi-line queries, just replace the READ-LINE above
with a function [left as an exercise for the reader] that recognizes
EOL escapes, e.g., perhaps a backslash before EOL as in Tcl:
cmu> (define-symbol-macro sql (pg-query-list (read-lines)))
SQL
cmu> sql select * \
from toy \
where season = 'xmas'
(("season" "medium" "title" "code")
("xmas" "book" "My Favorite Christmas" "2")
("xmas" "video" "The Grinch who Stole Christmas" "4")
("xmas" "video" "Home Alone" "6"))
cmu>
-Rob
p.s. This works fine in CMUCL, not sure about others.
It all depends on how the CL reader handles lookahead
and input on the same line as an initial S-expression.
Try this quick test in your CL:
cmu> (read-line) You should see ##THIS## text as a result.
"You should see ##THIS## text as a result."
NIL
cmu>
If that works, then so should the above symbol macro.
p.p.s. See <http://www.nhplace.com/kent/PS/Ambitious.html>
for a discussion of other issues with the CL reader and
line- and sexp-editing.
Nicolas Neuss
> p.s. This works fine in CMUCL, not sure about others.
> It all depends on how the CL reader handles lookahead
> and input on the same line as an initial S-expression.
> Try this quick test in your CL:
>
> cmu> (read-line) You should see ##THIS## text as a result.
>
> "You should see ##THIS## text as a result."
> NIL
> cmu>
>
> If that works, then so should the above symbol macro.
Your sql read macro is a nice idea. Unfortunately, it does not work within
the SLIME repl.
Nicolas
budden
>
> cmu> (define-symbol-macro sql (pg-query-list (read-line)))
> And to get multi-line queries, just replace the READ-LINE above
> with a function [left as an exercise for the reader] that recognizes
> EOL escapes, e.g., perhaps a backslash before EOL as in Tcl:
Not only that. I would like to pass parameters to my query
and have a way to introduce some macros into it. So, just "read-lines"
as you describe it is not enough. At least, lexical analyser is
required.
Currently I'm using some rather universal representation
of lexem sequence with a tree.
E.g.,
(let1 fld-list (id name)
(sql:select `(:l #|space-separated list|#
select #|symbols are printed verbatim|#
("," #|comma-separated list|#
,@fld-list)
from table1
where (make-where-clause #| if symbol is fbound,
function is called |#
'string #| quotes are converted to literals of
target language |#)
)
)
)
I was proud of myself when I have invented this some years ago. This
way I can use lisp syntax for
any language as I do not map entire grammar to lisp, I map only
lexical level and some text
structure nesting. In time, I can arrange some important specific
cases. E.g, so that (cond ...) emits case
statements, (when ...) emits if and so on. I'm using this approach for
a while and have found it rather practical.
But in time I found that plain SQL syntax is easier to read and
shorter to write in
many cases. SQL is just too rich language with many dialects, so
mapping of its grammar is problematic.
Lexical level representation like cited above tends to be
inconvinient. E.g., I need to write
`("," id name)` instead of just `id,name` when I list my fields. I was
unable to find a middle ground which
would satisfy me completely. Maybe I was not persistent enough. But
then I came to a disappointment to
a lisp syntax itself, with its many unnecessary paren levels. I found
that macros are not necessarily
bound to a sexp syntax (see m4, Mathematica and cpp). So, I won't win
big if I would be able to represent
my SQL queries as sexps and my stored procedures as lisp expressions.
So, I think the best way to embed SQL in other language (including
lisp)
is to use plain SQL syntax, some kind of SQL parser (maybe even call
SQL server at compile time for parsing
as embedded SQL processors for C do), parameter delimiters (:param)
and some other character for denoting
text-creation-time macros.
Anyway, now I have already a rather good approach for generating SQL
with macroexpansion,
so there is no urgent need to improve it further.
Testing your suggestion in SBCL+SLIME and Lispworks for windows:
cmu> (read-line) You should see ##THIS## text as a result
Works in lispworks only.
cmu> sql select count(*) from toy
works in neither. Lispworks treats
lispworks> foo arg1 arg2
as
> (foo arg1 arg2)
(don't remember the name of that feature, but it was discussed here
about a half year ago). So, lispwork would treat this line
as a function call.
Rob Warnock
+---------------
| rp...@rpw3.org (Rob Warnock) writes:
| > p.s. This works fine in CMUCL, not sure about others.
| > It all depends on how the CL reader handles lookahead
| > and input on the same line as an initial S-expression.
| > Try this quick test in your CL:
| > cmu> (read-line) You should see ##THIS## text as a result.
| >
| > "You should see ##THIS## text as a result."
| > NIL
| > cmu>
| > If that works, then so should the above symbol macro.
|
| Your sql read macro is a nice idea.
It was a symbol macro, actually, *not* a reader macro, see:
http://www.lispworks.com/documentation/HyperSpec/Body/m_defi_1.htm
Macro DEFINE-SYMBOL-MACRO
+---------------
| Unfortunately, it does not work within the SLIME repl.
+---------------
Does the "quick READ-LINE test" above work in SLIME? If not,
then probably SLIME either (1) does too much read-ahead or
(2) discards everything after the first form requested.
Note that the "quick READ-LINE test" above fails[1] at a
CLISP 2.3.6 REPL, suggesting that it's also going to fail
with any implementation using GNU "readline()" as a front-end
to the REPL.
-Rob
[1] Specifically, what seems to happen with CLISP is that the
whole line is buffered in the REPL's input stream [which
appears *not* to be *STANDARD-INPUT*], then when (READ-LINE)
executes it hangs waiting for *another* line to be typed
[into *STANDARD-INPUT*?], and returns *that* line as its value.
The REPL then goes back and READs the rest of the original line
for a new sexp to be evaluated:
%clisp -q
[1]> (read-line) (print "Should be a string value, *NOT* evaluated!")
Now hanging forever here... so type a line to let it continue.
"Now hanging forever here... so type a line to let it continue." ;
NIL
[2]>
"Should be a string value, *NOT* evaluated!"
"Should be a string value, *NOT* evaluated!"
[3]>
Compare with CMUCL:
cmu> (read-line) (print "Should be a string value, *NOT* evaluated!")
"(print \"Should be a string value, *NOT* evaluated!\")"
NIL
cmu>
This doesn't imply that one or the other is "wrong", just
that the symbol macro hack isn't going to work in CLISP.[2]
[2] Unless the code the symbol macro expands into does a *lot*
of poking around in the internal buffers of CLISP's REPL to
extract the rest of the line that it was typed on. Probably
not worth the effort.
Björn Lindberg
Just change the macro character of Kaz' package to something else.
Bj�rn Lindberg
Ron Garret
bj...@runa.se (Björn Lindberg) wrote:
Then no one else will be able to use my code.
rg
netsettler
...
> p.p.s. See <http://www.nhplace.com/kent/PS/Ambitious.html>
> for a discussion of other issues with the CL reader and
> line- and sexp-editing.
Actually, nothing about the technology described in that article is
specific to line or s-expression reading. It's fully general to any
user-written parser. You may have an issue with some implementations
of getting to read line-end characters (e.g., if they treat end of
line as end of stream and separately parse each line as an island unto
itself). You need to go into what we used to in call "break-all mode"
where you're just reading raw characters from the terminal and
bypassing any line-at-a-time buffering to get the right effect.
Then again, the article DOES dumb down the set of operations it offers
to ones that makes sense on a line. In reality, the original Maclisp
(www.maclisp.info/pitmanual) version of this macro was able to read
linefeeds and carriage returns and to back up into them if you were
typing multi-line input, even to do context-dependent completion, but
I reduced its scope a lot for the paper presentation just to make sure
the examples I tried were likely to run everywhere since CL has no
platform-independent display control even as powerful as Maclisp's
cursorpos.
Kaz Kylheku
I think what's needed is a way to register read macro modules, but not using
clumsy symbolic names. A disciplined read macro system should give macro
modules one-character names. Some convenient read macros should be
provided for manipulating the read table.
Suppose that CCL is registered under C and PKG is registered under P.
Then this syntax could express the wish ``process the following form
using a read-table in which the CCL macro is installed under #@ and the PKG
macro is installed under #$:
#!C@P$ <form>
Now the form can use both modules.
You see, there are two underlying problems. One is that source code dictates
read macros. The other is that the read macro modules dictate the macro. You
can't fix the first problem without editing the code, but you can address the
second problem: have a system whereby read macros do not dictate what positions
they occupy in the readtable. Since the readtable real-estate is premium,
and the potential for collisions is high, this is a configuration matter
which must be left to the user.
What if a module wants to take over more than one character? What if it
has both dispatch and regular read macros?
This could be abstracted away. The module could declare (via some API) that
its letter-name takes more than one argument. If the module registered under X
has two characters, then #!XYZ will pass Y and Z as arguments to it.
The module decides whether either or both of these two are dispatch characters,
regular macro characters, or used for some other purpose entirely. So for
instance a macro M which takes over a pair of enclosing characters might take
two arguments like this:
#!M() ... ;; M takes over parentheses
#!M[] ... ;; M uses square brackets instead
Module M will install a read macro function for ( or {, and recognize ) or ]
as the terminator, respectively.
budden
Hmmm. Then I would suggest to avoid "clumsy" symbolic names for
everything.
Let's rename "let" to #$?, "progn" to ~## and so on :)
> What if a module wants to take over more than one character? What if it
> has both dispatch and regular read macros?
Yes, this is just that problem. For "symbol-readmacros" there is a
ready solution: just
introduce the symbols you want into your package. E.g.,
there are sql:~s, sql:~e, regexp:~s and regexp:~u symbol-readmacros.
Then you can just _import_ sql:~s, sql:~e and regexp:~u to your
working package, which
is called, say, sql+regexp+my-stuff. Also you would like to import
some
functions and variables from both :sql and :regexp packages.
So, you will define reader behaviour together with package contents
which
is a compact, uniform, declarative and easy to understand definition
of your
specific sublanguage associated with a package given. No new
readmacros are required,
no new API to study!
But there are special cases like [], {}. The only thing I could do
here is
to provide sql:[] construct. This is brief and readable, but somewhat
ambigious.
It won't be very convinient to switch package to sql in the scope of
[...], as
we might want refer mostly to symbols from sql+regexp+my-stuff
package, not from sql package.
I don't know how to deal with this ambiguity in a "univrsally good"
way.
I have introduced _ package alias to my code. So,
(in-package :sql+regexp+my-staff)
(let1 table client sql:(select * from _:table)
would refer to sql+regexp+my-staff::table symbol, while
select, * and "from" are from sql package. Not very convinient,
but I couldn't make it better.
My code dated by march is here:
http://depositfiles.com/files/e12a0mx6s
Again:
portable hierarchical packages (ported to sbcl, clisp, ccl and should
work in any CL)
'keyword::(a b c) would read as (:a :b :c)
table of package/readtable associations.
sql:[select 1 from dual]
would invoke #\[ readmacro from readtable
bound to sql package
symbol-readmacros are implemented (they are called "custom-token-
parsers"), but API is
not good (see tests for examples).
Some minor improvements were made since that time. If someone
interested, I'll try to find a time
to publish newer version. Sorry, have no time to set up a project on
cl-user.net or somewhere.
Björn Lindberg
> In article <9mp4oro...@muvclx01.cadence.com>,
You are free to construct a read table to be used for reading your
code as you please. If you are not happy with Kaz' choice of macro
character, you can set up a different one in your read table. If Kaz
were to release his code as a software package, he could even provide
a function to create the reader macro under any given character into a
read table. This is the kind of control needed, the ability to tailor
a read table to the code you want to read. The largest unit of code
you have to read with the same read table is essentially one source
file. There is no need to invent a read macro package system, which in
a sense defeats the purpose of read macros to begin with.
Bj�rn Lindberg
Ron Garret
bj...@runa.se (Björn Lindberg) wrote:
> Ron Garret <rNOS...@flownet.com> writes:
>
> > In article <9mp4oro...@muvclx01.cadence.com>,
You're missing the point. Of course I can do all that, but that only
solves the problem for *me*, not anyone else. If I want to share my
code with others, then they all potentially face the same problem again,
because the character that I chose for my code may conflict with
something they chose to do in their code.
> The largest unit of code
> you have to read with the same read table is essentially one source
> file. There is no need to invent a read macro package system, which in
> a sense defeats the purpose of read macros to begin with.
And what if I want to take two modules that have a reader macro conflict
and use them as components of a single system and use ASDF to load the
combined system? This is not an academic question. I regularly run
into conflicts over the {} and [] reader macros. In fact, I have a
conflict for the {} *in my own code* where I sometimes want to use it to
write abstract associative maps, and sometimes I want to use it to write
infix expressions.
rg
budden
> into conflicts over the {} and [] reader macros.
> In fact, I have a conflict for the {} *in my own code*
> where I sometimes want to use it to write abstract
> associative maps, and sometimes I want to use it
> to write infix expressions.
_one_time_ only. My current solution is:
infix:{ switched to infix package and its readtable }
and
map:{ switched to map package and its readtable }
Maybe not very elegant, but code I referenced above allows
that. You might want to switch readtable only, not a package.
I don't know what to do... This is a design problem,
not a techical problem. I prefer to use some syntax which is
still undefined in a CL. Say, infix:::{ } could switch to infix's
readtable, but keeping a package and
map:::{ } switch to map's readtable.
But this is too verbose to satisfy me.
Also we could make two symbols: |MAP{|
and |INFIX{|, and assign a symbol-readmacro
to both of them. This way we could use ordinary
package system API to manipulate our reader.
Say, we import both symbols to our current package
and then say:
map{ a => b c => d} and infix{ 2+2}
Note we need a space to delimit map{ as
a symbol name. But it is less verbose than
map:::{ a }
Hmm, I think it is really nice :)
Kaz Kylheku
> In article <9mp7hwj...@muvclx01.cadence.com>,
> bj...@runa.se (Björn Lindberg) wrote:
>> You are free to construct a read table to be used for reading your
>> code as you please. If you are not happy with Kaz' choice of macro
>> character, you can set up a different one in your read table. If Kaz
>> were to release his code as a software package, he could even provide
>> a function to create the reader macro under any given character into a
>> read table. This is the kind of control needed, the ability to tailor
>> a read table to the code you want to read.
>
> You're missing the point. Of course I can do all that, but that only
> solves the problem for *me*, not anyone else. If I want to share my
> code with others, then they all potentially face the same problem again,
> because the character that I chose for my code may conflict with
> something they chose to do in their code.
This is only a problem if your code is mixed with their code.
The one mixing their code and your code is you.
So it's up to you to either edit their code such that every instance
of their existing dispatch character uses some other character,
or edit the little piece of dispatch character magic so that in this
mixed codebase, the read macro you are introducing goes to a different
dispatch character.
In other code where you are not mixing the two, you can use your original
dispatch character.
>> The largest unit of code
>> you have to read with the same read table is essentially one source
>> file. There is no need to invent a read macro package system, which in
>> a sense defeats the purpose of read macros to begin with.
>
> And what if I want to take two modules that have a reader macro conflict
> and use them as components of a single system and use ASDF to load the
> combined system?
Two modules cannot have a reader macro conflict, because two modules are
in different source files. When a source file is read, the reader restores
the previous value of *readtable*.
Just ensure that every damn module that uses read macros creates a new
readtable object and stores it in *readtable* at the top, rather than
manipulating the existing readable which is stored there.
This is a fundamental requirement if you are going to have any kind of peaceful
coexistence of different read macros in a big multi-module project.
If loading a file has a side effect on the read table, you are shot.
Never mind visible conflicts, the program will behave differently based on the
order in which you load the modules.
> This is not an academic question. I regularly run
> into conflicts over the {} and [] reader macros. In fact, I have a
> conflict for the {} *in my own code* where I sometimes want to use it to
> write abstract associative maps, and sometimes I want to use it to write
> infix expressions.
That's why we need some nice notation whereby you can say, for this form
(or for the remainder of this file), please install the dispatch macro
of module X, with these arguments.
E.g. like the #! I proposed in the other article.
#!X{}Y[]Z@
``Start a new readtable in *readtable*, and invoke the installer for
read macro module #\X with arguments #\{ and #\}, then the installer for
module #\Y with #\[ and #\], and finally the installer function for
macro module Z with the argument #\@. Each module declared how many
arguments it requires. They are all characters. What it does with those
characters is up to it. So for instance X installs #\{ as a macro
character (non-dispatch), and retains #\} (perhaps via a lexical closure
over the read function) as the delimiting character. The Y package
might install two dispatchers: #[ and #].
So what would happen is that you have inherited some code which has
#!X@
and you want to use your own macro package. You install that under some
other character like Y (this choice of allocating Y is up to you, in your
overall project), and then just add:
#!X@Y$
Now in this mixed source, you use #$ for dispatching the read macro you want to
introduce, and #@ refers to the existing one.
In your other source files, you can continue to use #@. Just not in this one
(at least not at the top level).
Obviously, only one read macro module can own the #@ dispatch in the top read
level of one source file.
Within individual forms, you can have fine-grained scoping, so that
at different read levels in a form, there can be different #@ macros
that are active.
So even in this mixed code, you can have their pieces use their #@, and your
embedded pieces can still use #@. Obviously, the embedded pieces will have to
be further annotated to enable that.
Ron Garret
Kaz Kylheku <kkyl...@gmail.com> wrote:
> On 2009-09-01, Ron Garret <rNOS...@flownet.com> wrote:
> > In article <9mp7hwj...@muvclx01.cadence.com>,
> > bj...@runa.se (Björn Lindberg) wrote:
> >> You are free to construct a read table to be used for reading your
> >> code as you please. If you are not happy with Kaz' choice of macro
> >> character, you can set up a different one in your read table. If Kaz
> >> were to release his code as a software package, he could even provide
> >> a function to create the reader macro under any given character into a
> >> read table. This is the kind of control needed, the ability to tailor
> >> a read table to the code you want to read.
> >
> > You're missing the point. Of course I can do all that, but that only
> > solves the problem for *me*, not anyone else. If I want to share my
> > code with others, then they all potentially face the same problem again,
> > because the character that I chose for my code may conflict with
> > something they chose to do in their code.
>
> This is only a problem if your code is mixed with their code.
No, it can be a problem even within my own code. For example, sometimes
I want to use curly braces to denote an abstract associative map using
Python's syntax:
{ a : 1, b : 2 }
and sometimes I want to use curly braces to denote infix expressions:
{ sin(x)+y }
But at any given time I have to choose one or the other.
> The one mixing their code and your code is you.
That's not true either. The collision of the #@ reader dispatch is a
counterexample. I didn't write any of the code involved in that
collision. You wrote part of it, and Clozure Associates wrote the other
part. Any yet I and anyone else who tries to use your PKG module in CCL
are the ones who experience the problem.
> >> The largest unit of code
> >> you have to read with the same read table is essentially one source
> >> file. There is no need to invent a read macro package system, which in
> >> a sense defeats the purpose of read macros to begin with.
> >
> > And what if I want to take two modules that have a reader macro conflict
> > and use them as components of a single system and use ASDF to load the
> > combined system?
>
> Two modules cannot have a reader macro conflict, because two modules are
> in different source files. When a source file is read, the reader restores
> the previous value of *readtable*.
You are assuming that the reader only ever reads code from source files.
Perhaps it has slipped your mind [1], but Lisp has this nifty feature
called the read-eval-print loop that allows code to be compiled and run
without ever being saved in a file.
BTW, I'm not trying to be snarky here. I use CCL which has a very fast
compiler, and my usual mode of operation is to recompile all my code
every single time I run it. So it often slips my mind that the rules of
the game change when you compile to fasl files. It would not be at all
surprising if someone who uses a system with a slower compiler and whose
regular mode of operation is to always compile to fasl files to make the
same mistake in reverse.
> E.g. like the #! I proposed in the other article.
>
> #!X{}Y[]Z@
Frankly, I find this such a horrible idea that I wasn't sure if you were
being serious or not. What you are proposing is a global (by which I
mean planet-wide) namespace consisting of single-character names. If it
isn't self-evident why that is a bad idea then I'm not sure what I can
say to convince you otherwise. Let's try this: how do I stake my claim
to the letter R?
rg
vippstar
<snip>
> You are assuming that the reader only ever reads code from source files.
> Perhaps it has slipped your mind [1], but Lisp has this nifty feature
> called the read-eval-print loop that allows code to be compiled and run
> without ever being saved in a file.
There's no [1] footnote in your post...
Ron Garret
<47b2be42-8498-4de1...@f33g2000vbm.googlegroups.com>,
budden <budde...@mail.ru> wrote:
> > This is not an academic question. I regularly run
> > into conflicts over the {} and [] reader macros.
> > In fact, I have a conflict for the {} *in my own code*
> > where I sometimes want to use it to write abstract
> > associative maps, and sometimes I want to use it
> > to write infix expressions.
> CL sucks here. CL is extensible, but it is extensible
> _one_time_ only. My current solution is:
>
> infix:{ switched to infix package and its readtable }
>
> and
>
> map:{ switched to map package and its readtable }
>
> Maybe not very elegant
Actually, I kinda like that.
> Also we could make two symbols: |MAP{|
> and |INFIX{|, and assign a symbol-readmacro
> to both of them. This way we could use ordinary
> package system API to manipulate our reader.
> Say, we import both symbols to our current package
> and then say:
>
> map{ a => b c => d} and infix{ 2+2}
Even better.
> Note we need a space to delimit map{ as
> a symbol name.
Not necessarily. We could just agree to globally make { a terminating
reader-macro character. That would break code that used { as a
constituent, but I suspect that's pretty rare in the current CL corpus.
I like the idea of symbol reader macros in general. It's actually not
that hard to implement in portable CL.
rg
Madhu
* Ron Garret <rNOSPAMon-61457...@news.albasani.net> :
Wrote on Tue, 01 Sep 2009 09:48:43 -0700:
| You're missing the point. Of course I can do all that, but that only
| solves the problem for *me*, not anyone else.
The only one having this problem is you.
| If I want to share my code with others, then they all potentially face
| the same problem again, because the character that I chose for my code
| may conflict with something they chose to do in their code.
Every case there is a real problem, the problem can be solved --- CL
already provides ways to address this hypothetical, which have been
mentioned and you are aware of.
All these are unacceptable to you because, it seems, they are too
flexible and you are on a mission to remove the flexibility from CL and
make it conform to BDSM patterns that you (and the hypothetical new user
audience that you are attracting) are used to.
--
Madhu
Ron Garret
<ad4bea61-0231-4bc5...@q14g2000vbi.googlegroups.com>,
vippstar <vipp...@gmail.com> wrote:
Oops, sorry about that. The footnote was the comment about not
intending to be snarky. But then I changed it to not be a footnote and
forgot to remove the footnote marker.
rg
Raffael Cavallaro
> Let's try this: how do I stake my claim
> to the letter R?
How dare you! Your name starts with "Ro" and my name starts with "Ra"
so I clearly have a more legitimate claim to the letter R on
alphabetical grounds!
Raffael (hoping that no one named Rachel wants to claim the letter R as
well...)
--
Raffael Cavallaro
budden
in a short time.
Symbol-readmacros API is not ready, but it would take about a couple
dozen loc.
Why I didn't implement it still? I tried to keep it simple. There are
several design
approaches to reader extensions.
First of all, I thought I could make per-package custom
readtables and this would be enough to use differenct existing CL
extensions
together as readtables are switched temporarily in the scope of
package:(something) construct.
But it seem to be more convinient to implement SQL reader as a parser
function, not as
a readtable. So I would need also a way to assign a reader function to
sql:<something> situation.
Then I thought I would need to parse CL "tokens", say, to read
date:2009-09-02. This way I can read list of dates as date:(2009-09-02
2009-09-03).
So, some hook at token interpretation time would be desired.
Now I see symbol-readmacros are very good.
So, there are about four possible distinct CL reader extensions. I'm
sure this is not a good idea to implement all of them. Thus reader
would
become too slow and too hard to analyse. I'm still unsure which one
(s)
to sacrify.
> > Note we need a space to delimit map{ as
> > a symbol name.
>
> Not necessarily. We could just agree to globally make { a terminating
> reader-macro character. That would break code that used { as a
> constituent, but I suspect that's pretty rare in the current CL corpus.
If you take a look at SBCL sources, you'll find symbols like
constituent[1]
in reader implementation. So I'm unsure we should make it a
terminating macro
character globally. But we can do it in some readtables.
> I like the idea of symbol reader macros in general. It's actually not
> that hard to implement in portable CL.
I don't think that is very easy, this requires some subtle and fragile
readtable
mangling. Maybe I have missed some simpler way... Anyway, I seem to
have implemented
this already (see my code referenced above).
Björn Lindberg
> In article <9mp7hwj...@muvclx01.cadence.com>,
> bj...@runa.se (Bj�rn Lindberg) wrote:
>
>> Ron Garret <rNOS...@flownet.com> writes:
>>
>> > In article <9mp4oro...@muvclx01.cadence.com>,
So write your code so that the users of it can choose macro
character. When they load their code, they have to be aware of which
read table they are using, and construct it to their needs.
>> The largest unit of code
>> you have to read with the same read table is essentially one source
>> file. There is no need to invent a read macro package system, which in
>> a sense defeats the purpose of read macros to begin with.
>
> And what if I want to take two modules that have a reader macro conflict
> and use them as components of a single system and use ASDF to load the
> combined system? This is not an academic question. I regularly run
> into conflicts over the {} and [] reader macros. In fact, I have a
> conflict for the {} *in my own code* where I sometimes want to use it to
> write abstract associative maps, and sometimes I want to use it to write
> infix expressions.
I haven't dealt much with ASDF lately, but I would be very surprised
if you could not accomplish this, e.g. by defining new file types and
overriding loading/compiling to use a particular read table. It sounds
like you are advocating for software packages to silently insert macro
characters in the global read table, which is the only way it could
work with ASDF 'transparantly'. If so, that is a horrible idea.
Bj�rn Lindberg
Ron Garret
<1413964c-661d-4782...@q7g2000yqi.googlegroups.com>,
budden <budde...@mail.ru> wrote:
> > I like the idea of symbol reader macros in general. It's actually not
> > that hard to implement in portable CL.
> I don't think that is very easy, this requires some subtle and fragile
> readtable
> mangling. Maybe I have missed some simpler way... Anyway, I seem to
> have implemented
> this already (see my code referenced above).
Well, here's how I did it:
---
(let ( (r (copy-readtable nil)) )
(defun read-symbol (stream)
(let ( (*readtable* r) )
(read-preserving-whitespace stream))))
(defun symbol-reader-macro-reader (stream char)
(unread-char char stream)
(let* ((s (read-symbol stream))
(f (get s 'symbol-reader-macro)))
(if f (funcall f stream s) s)))
(map nil (lambda (c)
(set-macro-character c 'symbol-reader-macro-reader t))
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz_")
(defun set-macro-symbol (symbol readfn)
(setf (get symbol 'symbol-reader-macro) readfn)
t)
; Example:
(defun make-special-form-reader (nargs)
(lambda (stream symbol)
(cons symbol (loop for i from 1 upto nargs collect
(if (listen stream) (read stream))))))
(set-macro-symbol 'assign
(lambda (stream symbol) `(setf ,(read stream) ,(read stream))))
(set-macro-symbol 'while
(lambda (stream symbol) `(do () ((not ,(read stream))) ,(read
stream))))
(set-macro-symbol 'display
(lambda (stream symbol) `(print ,(read stream))))
assign x 1
while (< x 10) display (incf x)
---
Actually, in my own implementation I only assign the symbol-macro-reader
to the upper-case alphabetic characters. That way to invoke the symbol
reader macro I have to begin the symbol with an upper-case character. I
find that helps keep things from getting too confusing.
rg
Russell McManus
Ron Garret <rNOS...@flownet.com> writes:
> Well, here's how I did it:
Very clever.
IMHO, this is easily the best post of the week so far on cll ...
-russ
Ron Garret
Russell McManus <russell...@yahoo.com> wrote:
> Ron Garret <rNOS...@flownet.com> writes:
>
> > Well, here's how I did it:
>
> Very clever.
Thanks.
> IMHO, this is easily the best post of the week so far on cll ...
Wow. Thanks!
rg
budden
> (defun read-symbol (stream)
> (let ( (*readtable* r) )
> (read-preserving-whitespace stream))))
But I didn't limit myself to alpha characters
and thus things became much more complex. Consider
tokens like "-045C" "1E5" or ".asdf" and you'll
get the idea of difficulties I faced. Initial goal
of my design was not to just enable symbol-readmacros,
but to intercept lookup and interning of a symbol
parsed by a reader, and thus to allow to redefine
package system behaviour in general. This way, I
needed to dig much deeper into process of reading
and this was not that simple.
It seems you're right. You're unable to assign
symbol-readmacro to the arbitrary symbol, but I
agree fully that there is no need to do so.
There are pretty many symbols starting from A..Z. I
would add some non-alpha characters as ~,! or
so, but this is more a matter of taste.
I'm still unsure if your hack is completely
compatible to all features of CL, but I can't
see any hole now. There is still something
strange:
#+assign
would invoke symbol-readmacro of assign and
I'm unsure this is right. But, again, if we
take some care, we can avoid confusion. So
the only unfinished job is to enable
symbol-readmacros to any given readtable,
not to a copy of default readtable only.
But this is rather trivial.
So, my congratulations :)
Anyway, I won't give up my code as it allows
much more than just symbol-readmacros. E.g.
I can use local package aliases, portable
hierarchical packages, portable packages
locks, forbidden symbol names and so on.
Ron Garret
<b905ad70-abbd-417e...@h3g2000yqa.googlegroups.com>,
budden <budde...@mail.ru> wrote:
> > (let ( (r (copy-readtable nil)) )
> > (defun read-symbol (stream)
> > (let ( (*readtable* r) )
> > (read-preserving-whitespace stream))))
> This was a core idea of my implementation too.
> But I didn't limit myself to alpha characters
> and thus things became much more complex. Consider
> tokens like "-045C" "1E5" or ".asdf" and you'll
> get the idea of difficulties I faced.
Those are indeed harder cases. But do you really want to be able to
define a symbol read macro on -045C? I'm pretty sure that if you don't
put symbol read macros in their own namespace somehow you're asking for
trouble.
> Initial goal
> of my design was not to just enable symbol-readmacros,
> but to intercept lookup and interning of a symbol
> parsed by a reader, and thus to allow to redefine
> package system behaviour in general. This way, I
> needed to dig much deeper into process of reading
> and this was not that simple.
Indeed. Do you know about Pascal Bourguignon's reader?
http://darcs.informatimago.com/darcs/public/lisp/common-lisp/reader.lisp
> It seems you're right. You're unable to assign
> symbol-readmacro to the arbitrary symbol, but I
> agree fully that there is no need to do so.
> There are pretty many symbols starting from A..Z. I
> would add some non-alpha characters as ~,! or
> so, but this is more a matter of taste.
And very easy to change.
> I'm still unsure if your hack is completely
> compatible to all features of CL, but I can't
> see any hole now. There is still something
> strange:
> #+assign
> would invoke symbol-readmacro of assign and
> I'm unsure this is right.
It isn't. But symbol reader macros are fraught with all manner of
peril. That's why I think it's prudent to only allow them for certain
typographically distinguished symbols.
> But, again, if we
> take some care, we can avoid confusion. So
> the only unfinished job is to enable
> symbol-readmacros to any given readtable,
> not to a copy of default readtable only.
> But this is rather trivial.
>
> So, my congratulations :)
Thanks :-)
rg
Madhu
* Russell McManus <87hbvll...@thelonious.cl-user.org> :
Wrote on Wed, 02 Sep 2009 11:59:09 -0500:
|
| IMHO, this is easily the best post of the week so far on cll ...
IMHO your post is the second best post of the week so far on CLL
--
Madhu
budden
>
> http://darcs.informatimago.com/darcs/public/lisp/common-lisp/reader.lisp
Something related to vectors. Other readers with
easier licenses can be found in open source
CL implementations. CCL's one seems to be
very easy to port to Common Lisp. But if I didn't
miss something, there is no need to rewrite a reader
entirely in order to intercept symbol parsing.
Readtable juggling helps.
>> #+assign
>> would invoke symbol-readmacro of assign and
>> I'm unsure this is right.
>It isn't.
Oops, I was wrong. Sorry. Was just too tired yesterday.
Pascal J. Bourguignon
>> Do you know about Pascal Bourguignon's reader?
>>
>> http://darcs.informatimago.com/darcs/public/lisp/common-lisp/reader.lisp
> Yes. This is GPL licensed and I found a bug in it.
> Something related to vectors.
It has been corrected.
> Other readers with
> easier licenses can be found in open source
> CL implementations. CCL's one seems to be
> very easy to port to Common Lisp. But if I didn't
> miss something, there is no need to rewrite a reader
> entirely in order to intercept symbol parsing.
Of course not. You only need to add a readtable-parse-token hook to
the readtable... I'll write a CDR about it before the end of the year.
--
__Pascal Bourguignon__
Ron Garret
Thing for symbol reader macros was this:
(let ( (r (copy-readtable nil)) )
(defun read-symbol (stream)
(let ( (*readtable* r) )
(read-preserving-whitespace stream))))
(defvar *enable-symbol-reader-macros* t)
(defun symbol-reader-macro-reader (stream char)
(unread-char char stream)
(let ( (s (read-symbol stream)) )
(if *enable-symbol-reader-macros*
(let ( (f (get s 'symbol-reader-macro)) )
(if f (funcall f stream s) s))
s)))
(defun set-macro-symbol (symbol readfn)
(setf (get symbol 'symbol-reader-macro) readfn)
t)
(map nil (lambda (c)
(set-macro-character c 'symbol-reader-macro-reader t))
"ABCDEFGHIJKLMNOPQRSTUVWXYZ")
This invokes the symbol reader macro only if the first letter of the
symbol's name is actually entered as a capital letter. I ran into too
many problems with symbol reader macros popping up where I didn't want
them in legacy code when I didn't segregate the name space this way.
There's also this related little hack:
(defvar *enable-prefix-funcall-syntax* t)
(defun prefix-funcall-syntax-reader (stream char)
(unread-char char stream)
(let ( (s (read-symbol stream)) )
(if (and *enable-prefix-funcall-syntax*
; Disable for #+foo(baz)
(not (eql (symbol-package s) (find-package :keyword)))
(eql (peek-char nil stream nil nil) #\())
(cons s (read stream))
s)))
(map nil (lambda (c) (set-macro-character c
'prefix-funcall-syntax-reader t))
"abcdefghijklmnopqrstuvwxyz")
This lets you write function calls using traditional f(x) syntax. It
requires that there be no space between the function name and the open
paren. So f(x) reads as (f x), but f (x) reads as two sexprs, f and (x).
rg