ANNOUNCE: Lexicons 2.0 (beta) released

34 views
Skip to first unread message

Ron Garret

unread,
Dec 5, 2008, 11:21:00 AM12/5/08
to
I am happy to announce a new version of my lexicons code. This version
fixes a rather serious bug in the previous version, and adds new
features, most notably semi-hygienic lexical macros and lexical classes
and methods. The implementation is also significantly cleaned up (it
consists of a single file of portable ANSI Common Lisp) and is more
seamlessly integrated into Common Lisp (lexicons are now implemented
using packages).

Code is here:

http://www.flownet.com/ron/lisp/lexicons.lisp

Documentation is here:

http://www.flownet.com/ron/lisp/lexicons.pdf

Lexicons are first-class lexical environments for Common Lisp, what in
other languages would be called modules or namespaces. They are
designed to be an adjunct, or potentially a replacement (depending on
your needs) for packages. Packages map strings on to symbols at
read-time. Lexicons map symbols onto bindings at compile-time. For
writing modular code, lexicons can be significantly more convenient than
packages. In particular, lexicons are unaffected by read-time
side-effects, and there is no need to manually maintain lists of
exported symbols.

This version has been tested only on Clozure Common Lisp, but it is
written in portable ANSI Common Lisp so it should run under any
conforming implementation.

Comments and feedback are welcome.

rg

Kaz Kylheku

unread,
Dec 5, 2008, 2:03:58 PM12/5/08
to
On 2008-12-05, Ron Garret <rNOS...@flownet.com> wrote:
> http://www.flownet.com/ron/lisp/lexicons.lisp

Not a proper DOS or Unix text file. Lines separated by carriage returns only,
no linefeeds.

Ron Garret

unread,
Dec 5, 2008, 2:35:49 PM12/5/08
to
In article <200812201...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

Sorry, that's the (old) Mac end-of-line convention. (I was editing the
file using MCL.) I've uploaded a new version that uses linefeeds
instead.

rg

budden

unread,
Dec 5, 2008, 3:31:09 PM12/5/08
to
Hi!
I can not pay much attention for lexicons now. I think they
introduce too many new constructs to lisp, and that's why I think I
won't use it. I too tried to solve package problems. I developed a
rather minimalistic approach. This approach produced some dispute
here. It is the time to give a link here: http://norvig.com/python-lisp.html
Piter Norvig says that package system is hard to use and says it is a
disadvantage of CL. I agree completely to this.
My solution is to add macro
(in-packages package &rest more-packages) which acts as following:
- when this macro is in effect, reader behaviour changes. When
reading, unqualified symbol is sought for in all
packages listed. It can be found as internal symbol in first package
and as external symbol only in the rest of the packages. If it is
found once, it is used. If it is found more than once, error is
produced. If it is not found, it it interned into package. Maybe (I
think) one more rule is needed: if this symbol is in shadowing symbols
list of the "package", it can be used without qualifier even if it is
external in some of other packages.
- you may use qualified symbols to specify exactly what you mean.
- optionally, package locks are imposed to some packages (syntax not
shown).
Additionally, read-macro of kind
#with-packages (package &rest packages) form
is added so that next form is read in scope of (with-packages) form.

This approach allows minimize both import and use-package operations
and it is relatively safe. It does not remove the need to export some
symbols manually when you want, say, have a custom + or - in your
package. Idea is borrowed from sql syntax:

select t1.id, t2.id, t1.unique_field_of_t1 from t1, t2 where
t1.id=t2.t1_id

Which proved in my practice to be fairly reliable and convinient.
I say it just in case you didn't read the topic with keywords
"symbol clashes - how to avoid them"

For this to work we need either to change implementation's "find-
package" function (which is not portable), or use portable lisp reader
written in CL (it can be done rather easily based on some of open-
source implementations, there is such a reader already, written by
Pascal J. Bourguignon, but it is under GPL and it has bugs). I managed
to substitute lisp reader completely with other lisp reader, for all
activity including compiler. So, for now all what is left to do is to
take some open-source reader and replace implementation-specific
reading primitives with CL ones. Don't know when I get time for this.

Ron Garret

unread,
Dec 5, 2008, 3:53:23 PM12/5/08
to
In article
<54b8cf3e-3bf9-47e4...@j35g2000yqh.googlegroups.com>,
budden <budd...@gmail.com> wrote:

> Hi!
> I can not pay much attention for lexicons now. I think they
> introduce too many new constructs to lisp, and that's why I think I
> won't use it.

I don't know what you consider a "new" construct, but lexicons only
introduce lexical equivalents of existing CL defining forms (defun,
defvar, defmacro, defclass and defmethod). The lexical equivalent of
each of these has the same name but preceded by an L (ldefun, ldefvar,
ldefmacro, ldefclass and ldefmethod). One could easily shadow the
existing defining forms so that programming with lexicons is no
different from programming in regular CL, but I wasn't going to do this
until I had time to do some really thorough testing.

> I too tried to solve package problems. I developed a
> rather minimalistic approach. This approach produced some dispute
> here. It is the time to give a link here: http://norvig.com/python-lisp.html
> Piter Norvig says that package system is hard to use and says it is a
> disadvantage of CL. I agree completely to this.

Which is in fact the primary motivation for lexicons.

> My solution is to add macro
> (in-packages package &rest more-packages) which acts as following:

[snip]

IMHO, the central problem with packages is that they try to resolve
namespaces at read time, when this ought to be done at compile-time.
The appropriate information for resolving namespaces is simply not
available at read-time. So the problems with packages cannot be solved
by changing the reader.

Note that this issue is in some sense the converse of the one
investigated by Kent Pitman a long time ago:

http://www.nhplace.com/kent/PS/Ambitious.html

rg

budden

unread,
Dec 5, 2008, 4:05:16 PM12/5/08
to
Well, maybe I'll take a look at it later, but it is too much.
Some packages overwrite defun too. One of them is ap5 which is very
cool.
Other is screamer which is cool to. I afraid there will be
problems...
And, say, I want to make a wrapper for defclass with standard names of
accessors, initargs, etc.
If I accept lexicons, I need to redefine ldefclass too.

Anyway, I think I'll take a closer look at them in a short just to
find what ideas are there.

Ron Garret

unread,
Dec 5, 2008, 4:19:05 PM12/5/08
to
In article
<cbe188e9-a2d4-4db3...@l42g2000yqe.googlegroups.com>,
budden <budd...@gmail.com> wrote:

> Well, maybe I'll take a look at it later, but it is too much.
> Some packages overwrite defun too. One of them is ap5 which is very
> cool.
> Other is screamer which is cool to. I afraid there will be
> problems...

It depends on what the shadowing forms in these other packages expand
into. If, for example, AP5:DEFUN expands into CL:DEFUN then you can
just muck with the package system to have it expand into LEXICONS:DEFUN
instead. It should Just Work. But this is the sort of thing I want to
actually try before unleashing it on the world.

> And, say, I want to make a wrapper for defclass with standard names of
> accessors, initargs, etc.
> If I accept lexicons, I need to redefine ldefclass too.

Nope, you'll just need to change your wrapper to expand into ldefclass
instead of defclass. All the LDEF* forms have the exact same syntax as
their corresponding non-lexical forms. Alternatively you can shadow
DEF* and just run your (unchanged) wrapper definition in the package
with the shadowed symbols.

The ultimate goal is to be able to take existing code, remove (nearly)
all the package-related stuff, move everything into the (not yet
implemented) LEXICONS package, and have it all Just Work. This may be a
pipe dream, but the fact that there is now a very close correspondence
between lexicons and packages makes me optimistic.

rg

David Golden

unread,
Dec 5, 2008, 4:30:41 PM12/5/08
to
Ron Garret wrote:


> The implementation is also significantly cleaned up (it
> consists of a single file of portable ANSI Common Lisp) and is more
> seamlessly integrated into Common Lisp (lexicons are now implemented
> using packages).
>

I'm not sure I like this, at least not the way a lexicon named "l1" now
seems to "use up" a package named l1. Haven't really played with it
enough yet to judge if it's all that much of an issue, but maybe
packages used for the implementation should be called %lex-l1 or
something.

> They are
> designed to be an adjunct, or potentially a replacement (depending on
> your needs) for packages. Packages map strings on to symbols at
> read-time. Lexicons map symbols onto bindings at compile-time. For
> writing modular code, lexicons can be significantly more convenient
> than packages.

Yeah, I'm one of the people who uses packages for stuff other than
writing modular code - they're independently useful for sorting
symbols, so it'd be nice if both lexicons and packages would work with
minimal interference with eachother...

Kenny

unread,
Dec 5, 2008, 4:33:12 PM12/5/08
to
budden wrote:
> Hi!
> I can not pay much attention for lexicons now. I think they
> introduce too many new constructs to lisp, and that's why I think I
> won't use it. I too tried to solve package problems. I developed a
> rather minimalistic approach. This approach produced some dispute
> here. It is the time to give a link here: http://norvig.com/python-lisp.html
> Piter Norvig says that package system is hard to use and says it is a
> disadvantage of CL. I agree completely to this.

Well then I do believe it is time to go shopping.

Only there is nothing hard about the package system unless your cup is
full and you want it to be like some other thing but everything is like
that.

Good lord, don't you have any application programming to do?

kxo

Ron Garret

unread,
Dec 5, 2008, 6:07:35 PM12/5/08
to
In article <ghc6i2$v5d$1...@aioe.org>,
David Golden <david....@oceanfree.net> wrote:

> Ron Garret wrote:
>
>
> > The implementation is also significantly cleaned up (it
> > consists of a single file of portable ANSI Common Lisp) and is more
> > seamlessly integrated into Common Lisp (lexicons are now implemented
> > using packages).
> >
>
> I'm not sure I like this, at least not the way a lexicon named "l1" now
> seems to "use up" a package named l1. Haven't really played with it
> enough yet to judge if it's all that much of an issue, but maybe
> packages used for the implementation should be called %lex-l1 or
> something.

That would be very easy to change.

> > They are
> > designed to be an adjunct, or potentially a replacement (depending on
> > your needs) for packages. Packages map strings on to symbols at
> > read-time. Lexicons map symbols onto bindings at compile-time. For
> > writing modular code, lexicons can be significantly more convenient
> > than packages.
>
> Yeah, I'm one of the people who uses packages for stuff other than
> writing modular code - they're independently useful for sorting
> symbols, so it'd be nice if both lexicons and packages would work with
> minimal interference with eachother...

Lexicons have always been orthogonal to (and therefore compatible with)
packages from the beginning. It's just that now packages are used in
the implementation whereas before they weren't.

Out of curiosity, what kind of "symbol sorting" are you using packages
for?

rg

David Golden

unread,
Dec 5, 2008, 8:57:07 PM12/5/08
to
Ron Garret wrote:


> That would be very easy to change.
>

I assumed as much, but you did ask for feedback... :-)



> Out of curiosity, what kind of "symbol sorting" are you using packages
> for?
>

If you're actually programming with symbols as pseudo-objects
(hey, they are first-class in lisp), then it's just handy to be able to
stash them into separate packages so that ones with the same name have
separate identities but are easy to refer to.

You have symbols with the same symbol-name. You want them to be
different objects still, and you don't want the emacs lisp "solution"
(um, make them have different names...). I mean, not much more to it.

Personally, I've used it for an (unreleased and half-assed) glsl and c
translators usable in one image, in combination, that used e.g.
glsl:main and c:main. Probably more stuff.

Mostly convenience for humans writing/reading code, not for the
computer - Nothing you _couldn't_ do with a different implementation
strategy, without packages (or even first-class symbols), just a naming
convention or a bunch of hashtables you look up different things in in
different contexts. But you wind up implementing half a package system
anyway, shrug. OTOH, people wouldn't go around calling your
(non-performance-critical, simple, short, works) code "old
fashioned"...

Leslie P. Polzer

unread,
Dec 6, 2008, 12:49:33 PM12/6/08
to
I have a minor gripe with the function MAKE-LEXICON:

It modifies some global object (i.e. the list of lexicons).

This is not consistent with the usual semantics of MAKE-* functions
and should probably be called DEFLEXICON instead.

Leslie

Ron Garret

unread,
Dec 6, 2008, 5:04:31 PM12/6/08
to
In article
<c92308a2-7b39-4f65...@y1g2000pra.googlegroups.com>,

I agree with you that this is a bit of a wart in the design, but there's
a reason for it. The design of the lexicons API was intended to mirror
the API for packages as much as possible while still achieving the
design goals. Accordingly, MAKE-LEXICON is intended to be the
counterpart to MAKE-PACKAGE, which also modifies a global. DEFLEXICON,
if it were to exist, would be the counterpart to DEFPACKAGE. But the
whole point of lexicons is to get rid of all the complexity that makes
DEFPACKAGE necessary, so DEFLEXICON would in some sense be an oxymoron.

In an ideal world, MAKE-LEXICON would not modify a global. It would
also allow you to make anonymous lexicons, and multiple lexicons with
the same name. But it is not possible to make anonymous packages, nor
is it possible to make multiple packages with the same name, since both
of these would undermine the whole point of having packages. (Note that
these features would not necessarily undermine the utility of lexicons,
which is IMHO another indication that lexicons are the Right Way to
achieve modularity.)

I did it this way because I thought that it would make package fans feel
warmer and fuzzier about lexicons to know that the binding of a symbol S
in lexicon L was invariably the symbol L::S. It would be easy enough to
abandon this invariant and just use uninterned symbols for the bindings
instead. But personally, although I am no fan of packages, the current
design has kind of grown on me. It's far from clear that enforcing the
global uniqueness of lexicon names is a bad idea, even if it may not be
logically necessary.

rg

David Golden

unread,
Dec 6, 2008, 11:15:48 PM12/6/08
to
Ron Garret wrote:


> I did it this way because I thought that it would make package fans
> feel warmer and fuzzier about lexicons to know that the binding of a
> symbol S
> in lexicon L was invariably the symbol L::S.

Hmm. I guess that does sound handy. Perhaps my clash complaint is
adequately addressed by the user just adopting some explicit naming
convention for lexicons, rather than lexicon code doing any
name-munging. i.e. a lexicon named =lyekka= with an implementation
package named =lyekka= isn't going to clash with a package named
lyekka.

Slobodan Blazeski

unread,
Dec 7, 2008, 7:30:05 AM12/7/08
to
May somebody please explain me with simple words why cl packages are
hard to use, 'couse I really don't get it?
The best is probably to hear from somebody who just learned lisp or
somebody who teaches lisp (do such people exist anymore?) and his/her
students has problems with packages.

bobi

Pascal J. Bourguignon

unread,
Dec 7, 2008, 8:34:15 AM12/7/08
to
Slobodan Blazeski <slobodan...@gmail.com> writes:

> May somebody please explain me with simple words why cl packages are
> hard to use, 'couse I really don't get it?

Well, I'd say you're the best placed to explain with simple words why CL
packages are hard to use, because for us they are not hard to use.

But if you cannot explaint it with simple words, you may also use
complicated words, we've got good dictionaries.


> The best is probably to hear from somebody who just learned lisp or
> somebody who teaches lisp (do such people exist anymore?) and his/her
> students has problems with packages.

I don't find CL packages hard to use. On the contrary, they're a very
simple concept, and at the same time they're very powerful.


Let's start without packages, and try to solve some simple practical
problems.

Assume we create two symbols with the same name, "ABC", and "bind" them
to different values:

[11]> (list (list (make-symbol "ABC") 1) (list (make-symbol "ABC") 2))
((#:ABC 1) (#:ABC 2))


Then we can find a symbol from a name, for example when we read the
string "ABC" from the user, we wan to find what symbol it corresponds
to, and retrieve its value:

[12]> (let ((syms (list (list (make-symbol "ABC") 1) (list (make-symbol "ABC") 2))))
(second (find "ABC" syms :key (function first) :test (function string=))))
1

But how could we find instead the other symbol named "ABC", with value
2? The solution proposed is to put the symbols in different
lists, named "packages", having different names. So now, we can find a
symbol named "ABC" in a current 'package' refered to by *pack*, or a
symbol named "ABC" qualified by a package name such as "YOUR::ABC".

[14]> (let ((packs (list (list "MINE" (list (make-symbol "ABC") 1))
(list "YOUR" (list (make-symbol "ABC") 2))))
(*pack* "MINE"))
(list (second (find "ABC" (rest (find *pack* packs
:key (function first) :test (function string=)))
:key (function first) :test (function string=)))
(second (find "ABC" (rest (find "YOUR" packs
:key (function first) :test (function string=)))
:key (function first) :test (function string=)))))
(1 2)


CL:IMPORT puts a symbol on the list of symbols of a package.
CL:FIND-SYMBOL finds a symbol in the list of symbols of a package.
CL:FIND-PACKAGE finds a package in the list of packages.

CL:INTERN does both CL:FIND-SYMBOL, and if not found, CL:MAKE-SYMBOL and CL:IMPORT.


With CL:USE-PACKAGE, there's a little technicality, in that the symbols
from the used packages are not really "interned" or "imported" into the
using package, but it works most of the time exactly the same.

Each package maintains also a list of "exported" symbols, and this is
used only to signal an error when you try to read a qualified symbol
that is not exported with only one colon.

SYMB is exported from PACK SYMB is not exported from PACK
PACK:SYMB ok error
PACK::SYMB ok ok

--
__Pascal Bourguignon__

Kenny

unread,
Dec 7, 2008, 12:42:34 PM12/7/08
to

Ron Garret

unread,
Dec 7, 2008, 12:50:07 PM12/7/08
to
In article
<2c967dc5-a2f7-482d...@j11g2000yqg.googlegroups.com>,
Slobodan Blazeski <slobodan...@gmail.com> wrote:

Packages aren't "hard to use", they just do the Wrong Thing ;-)
Packages operate at read-time, but most programmer's intuitions about
modularity relate to global bindings, which is a compile-time concept.

The biggest practical problem with packages, and the central motivation
for lexicons, is that exported symbol lists have to be maintained
manually. This violates the DRY principle, and makes maintenance more
difficult than it needs to be, rather like the burden imposed by C on
the programmer to maintain the .c file and the .h files in parallel.
Lexicons let you write modular code without manually maintaining an
export list.

A secondary problem with packages is that because they do what they do
at read time, merely parsing the program has potential side-effects
(interning symbols). If you attempt to parse a program in an
environment where your packages have not been set up properly, these
side effects can end up messing up your environment to the point where
recovery is very difficult because you suddenly have a zillion symbol
interned in a package where they should not be interned. Lexicons avoid
this problem because they operate at compile time, not read time. They
also have a deferred binding mechanism that allows recovery from
unresolved bindings cause by missing libraries to be done at run-time,
so if you don't have your libraries set up properly you can recover by
simply importing the right library. You don't have to recompile the
client code.

rg

Brian Adkins

unread,
Dec 7, 2008, 12:57:47 PM12/7/08
to

Ron Garret

unread,
Dec 7, 2008, 1:08:31 PM12/7/08
to
In article <8763lwr...@informatimago.com>,

p...@informatimago.com (Pascal J. Bourguignon) wrote:

> I don't find CL packages hard to use. On the contrary, they're a very
> simple concept, and at the same time they're very powerful.

That's true, but...

> Let's start without packages, and try to solve some simple practical
> problems.
>
> Assume we create two symbols with the same name, "ABC", and "bind" them
> to different values:
>
> [11]> (list (list (make-symbol "ABC") 1) (list (make-symbol "ABC") 2))
> ((#:ABC 1) (#:ABC 2))

With all due respect, this is not the kind of "practical problem" that
most programmers deal with on a day-to-day basis, particularly not when
they are starting out. Using MAKE-SYMBOL is a pretty advanced concept,
and using it to make multiple symbols with the same name is a pretty
esoteric technique. In fact, the whole concept of an uninterned symbol
is pretty advanced. Peter Seibel's book, for example, doesn't even
mention them until chapter 21, and then only in passing. MAKE-SYMBOL is
not mentioned at all.

The vast majority of symbols that beginning and work-a-day Lispers deal
with are interned by the reader, and most non-advanced Lisp programmers
think about modular programming in terms of exporting functions,
classes, macros, and methods, not symbols. If you doubt this, do a
Google search in c.l.l. for "How do I export a function?" and see how
many hits you get.

rg

Ron Garret

unread,
Dec 7, 2008, 1:10:57 PM12/7/08
to
In article <m2d4g3d...@gmail.com>,
Brian Adkins <lojic...@gmail.com> wrote:

Ah what the hell.

http://www.flownet.com/ron/packages.pdf

rg

Pascal Costanza

unread,
Dec 7, 2008, 1:28:23 PM12/7/08
to
Ron Garret wrote:
> In article
> <2c967dc5-a2f7-482d...@j11g2000yqg.googlegroups.com>,
> Slobodan Blazeski <slobodan...@gmail.com> wrote:
>
>> May somebody please explain me with simple words why cl packages are
>> hard to use, 'couse I really don't get it?
>> The best is probably to hear from somebody who just learned lisp or
>> somebody who teaches lisp (do such people exist anymore?) and his/her
>> students has problems with packages.
>>
>> bobi
>
> Packages aren't "hard to use", they just do the Wrong Thing ;-)
> Packages operate at read-time, but most programmer's intuitions about
> modularity relate to global bindings, which is a compile-time concept.

And therein lies the answer: Only people who stick to their intuition
and try to use packages as if they were about bindings will find it hard
to use packages. You have to adjust your expectations, and then packages
become very easy. (Some people don't need this kind of adjustment,
because the view packages provide comes natural to them.)

IMHO, packages, i.e. managing the mapping from strings to symbols, solve
the problem better than modules, i.e. managing the mapping from
identifiers to concepts. So I wouldn't agree that they do the wrong
thing, to the contrary, I think they do the right thing. But it's good
that you put a smiley there... ;)


Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Ron Garret

unread,
Dec 7, 2008, 1:54:08 PM12/7/08
to
In article <6q2iu8F...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

> IMHO, packages, i.e. managing the mapping from strings to symbols, solve
> the problem better than modules, i.e. managing the mapping from
> identifiers to concepts.

Why?

rg

Pascal Costanza

unread,
Dec 7, 2008, 2:23:11 PM12/7/08
to

Because they can make distinctions that you cannot make with module
systems. For example, if you had only a module system, a class would not
be able to inherit from two different classes with slots of the same
name and be able to keep them as two different slots (unless you come up
with funky renaming mechanisms, like in the R6RS module system, for
example).

With a CL-style package system, the choice is always unambiguous: If you
use the same symbol as somebody else, you definitely mean the same
concept. If you want to denote a different concept, you have to use a
different symbol. It's much harder to make such distinctions with module
systems.

[Maybe having both packages and modules, like your lexicons, in the same
system is beneficial in that it provides the functionalities of both
worlds - but I cannot tell because I haven't tried this.]

David Golden

unread,
Dec 7, 2008, 3:05:17 PM12/7/08
to
Ron Garret wrote:


> I did it this way because I thought that it would make package fans
> feel warmer and fuzzier about lexicons to know that the binding of a
> symbol S
> in lexicon L was invariably the symbol L::S.


No, wait. This isn't good.

CL-USER> (defpackage :a)
#<PACKAGE "A">
CL-USER> (defpackage :b)
#<PACKAGE "B">
CL-USER> (make-lexicon :=l=)
#<Lexicon =L=>
CL-USER> (in-lexicon :=l=)
#<Lexicon =L=>
CL-USER> (ldefun a::x () "one")
=L=::X
CL-USER> (ldefun b::x () "two")
STYLE-WARNING: redefining =L=::X in DEFUN
=L=::X
CL-USER> (a::x)
"two"

a::x and b::x are different symbols supporting
different bindings absent lexicons, IMO lexicons should
just allow them to be lexical, not merge them, if lexicons
are truly on another axis to packages.


Kaz Kylheku

unread,
Dec 7, 2008, 3:24:33 PM12/7/08
to
On 2008-12-07, Pascal J. Bourguignon <p...@informatimago.com> wrote:
> Slobodan Blazeski <slobodan...@gmail.com> writes:
>
>> May somebody please explain me with simple words why cl packages are
>> hard to use, 'couse I really don't get it?
>
> Well, I'd say you're the best placed to explain with simple words why CL
> packages are hard to use, because for us they are not hard to use.
>
> But if you cannot explaint it with simple words, you may also use
> complicated words, we've got good dictionaries.

I.e. we have lexicons to deal with hard-to-use packages of verbiage. :)

Kaz Kylheku

unread,
Dec 7, 2008, 3:54:41 PM12/7/08
to

Because we have symbols as a first-class data type. So we need packages to work
everywhere, including in data:

(setf foo::y 42)

(quote foo::x)

Lis packages have to work they way they do because in general, you do not know
what symbols in a form are evaluated normally, which symbols are evaluated
according to special rules, and which ones not at all.

Packaging symbols provides the right separation of concerns, for instance in OO
programming.

We can define a class in which the slots belong to different packages. We can
write methods that are named by symbols in different packages, yet specialize
on the same object.

This is very nice; given some existing class, I can extend that class in my own
module, using identifiers from my namespace.

(in 'my-namespace)

(defclass my-class (other-namespace:base)
((my-slot :accessor my-slot ...)))

All of the slots of other-namespace:base are named using other-namespace
symbols, and so are the existing generic functions.

I can make my own slots and generic functions in my own namespace.

This does not cause a problem when we are referencing slots:

(slot-value instance 'my-slot)
(slot-value instance 'other-namespace:foo)

The alternative solution would be to make namespacing a part of slot lookup
(i.e. the mapping of the symbol to the concept). If symbols don't have
namespaces, you have to invent qualified identifiers to deal with this:

Suppose that you inherit from two base classes A and B that both define a slot
FOO. Compound identifier would have to be introduced to solve the
disambiguation problem:

(slot-value instance '(a foo)) ;; the foo from base A

(slot-value instance '(b foo)) ;; the foo from base B

Also keep in mind that slot-value is a function, not a special operator, so it
cannot pull a namespace context from lexical scope. Suppose that in some scope
you want you want (slot-value instance 'foo) to behave like (slot-value
instance '(a foo)). You need now need a dynamically scoped mapping of
unqualified identifiers to qualified ones. It smells like big-time suckage!

Kaz Kylheku

unread,
Dec 7, 2008, 4:23:26 PM12/7/08
to
On 2008-12-07, Ron Garret <rNOS...@flownet.com> wrote:
> In article
><2c967dc5-a2f7-482d...@j11g2000yqg.googlegroups.com>,
> Slobodan Blazeski <slobodan...@gmail.com> wrote:
>
>> May somebody please explain me with simple words why cl packages are
>> hard to use, 'couse I really don't get it?
>> The best is probably to hear from somebody who just learned lisp or
>> somebody who teaches lisp (do such people exist anymore?) and his/her
>> students has problems with packages.
>>
>> bobi
>
> Packages aren't "hard to use", they just do the Wrong Thing ;-)
> Packages operate at read-time, but most programmer's intuitions about
> modularity relate to global bindings, which is a compile-time concept.
>
> The biggest practical problem with packages, and the central motivation
> for lexicons, is that exported symbol lists have to be maintained
> manually.

Exporting symbols is braindamaged. The solution is: don't manage exported lists.

What is the purpose of exporting symbols?

One purpose is documentary: by exporting certain symbols, you are asserting
that they are yoru public interface.

But exported symbols are not just documentary. What does the machine actually
let us do with exported symbols?

It lets us apply USE-PACKAGE, (or the :USE clause in DEFPACKAGE). This is
called package inheritance: all of the exported symbols of another package
become visible in this one.

USE-PACKAGE is a braindamaged concept borrowed from other languages.

The proper engineering solution is to maintain explicit import lists: pick
every single identifier from a foreign package that you wish to use locally as
an unqualified name, and explicitly install it into your current scope.

Can you come up with a hack that lets me choose exactly which identifiers I
pull in from some namespace (regardless of what that namespace says it
exports), without having to maintain import lists?

That would be useful. But I don't see how; thep roblem implicitly calls for
list maintenance. I mean, you have to write down the symbols that you want.
That's a list.

Now as far as export lists go, the difficulty of maintaining them isn't due to
any fundamental property of Lisp packages. The automation of export lists could
be added to Lisp packages without changing the design.

Like with import lists, the programmer ultimately has to indicate to the
machine that he wants certain symbols exported. And that constitutes a list!
Your complaint is that you have to write down this list in some place (like
your DEFPACKAGE) in addition to all the other definitions elsewhere that assign
meaning to those symbols (like DEFMETHOD-s, DEFUN-s, DEFCLASS-es, etc).

One way to solve that might be to have some kind of special state in a package
so that you could do this:

;; top level
(in-package 'foo)

(public)

(defun foo ...)

(private)

(defun bar ...)

PUBLIC and PRIVATE would set and reset some flag, and defining constructs could
use it to export or not export the name which they are defining.

The problem with this is that a package definition is often put into a separate
file which is loaded by everyone. That file has to do the complete job of
establishing what is exported and what isn't. You cannot leave it to a bunch of
load-time side effects.

With the above mechanism, if module A depends on B, B has to be loaded first,
so that all of these effects in its toplevel forms are elaborated before
anything is done with A.

In other words, mantaining export lists is simply a consequence of applying ad
ependency inversion to decouple module dependencies.

> This violates the DRY principle, and makes maintenance more
> difficult than it needs to be, rather like the burden imposed by C on
> the programmer to maintain the .c file and the .h files in parallel.

If you do not maintain an interface and implementation separately (either as
separate files or as sections of the same file, as is the case with some
langauges), it means that in before processing module B that depends on A, the
language implementation has to process all of A, so that it could pull out the
interface information that is implicitly distributed throughout B.

C header files, crude as the mechanism is, do allow us to compile some foo.c
that uses bar.h, without having access bar.c, or any representation of it.

In order to have an efficient interface/implementation mechanim (i.e. not like
that of C) you'd need some kind of special support. I.e. there wuold have to be
a way of processing a file, similar to COMPILE-FILE, which would digest all of
the interface information and spit it into some kind of efficient form: perhaps
some special interface file: like a fasl, but with no code. Or maybe a special
section of a fasl.

When a module B uses A, the compiler would pull out A's digested interface info
(refreshing it first, if the source of A is newer). In this way, B could be
compiled without doing a complete compilation of A.

You know, you could develop an annotation system for Lisp modules which would
allow such processing. Code is data right?

Let's say we have this syntax:

(module foo)

:public

(defun x ...)

:private

(defun y ...)

You could cob together a Lisp build system which will munge source code and
generate the defpackage definitions, with export lists and all.

The above would result in there being a (DEFPACKAGE FOO ...) somewhere
which exports the symbol X.

Ron Garret

unread,
Dec 7, 2008, 4:29:55 PM12/7/08
to
In article <6q2m50F...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

> Ron Garret wrote:
> > In article <6q2iu8F...@mid.individual.net>,
> > Pascal Costanza <p...@p-cos.net> wrote:
> >
> >> IMHO, packages, i.e. managing the mapping from strings to symbols, solve
> >> the problem better than modules, i.e. managing the mapping from
> >> identifiers to concepts.
> >
> > Why?
>
> Because they can make distinctions that you cannot make with module
> systems. For example, if you had only a module system, a class would not
> be able to inherit from two different classes with slots of the same
> name and be able to keep them as two different slots (unless you come up
> with funky renaming mechanisms, like in the R6RS module system, for
> example).

That's true, but how often do you actually do multiple inheritance from
two different classes that were separately developed?

BTW, because the current implementation of lexicons is tightly coupled
with packages it is fairly straightforward to "fix" this. I put "fix"
in scare quotes because it's not at all clear to me that this is
actually a problem, but if it really is then you can just have LDEFCLASS
lexify the slot names along with the superclasses, e.g.:

(defmacro ldefclass (name superclasses slots &rest stuff &environment
env)
`(progn
(lexify ,name)
(defclass ,(lbind name (env-lexicon env))
,(mapcar (lambda (c) (or (find-cell (env-lexicon env) c)
(error "There is no class named ~S in
the current lexical environment." c)))
superclasses)
,(mapcar (lambda (s) (lbind s (env-lexicon env))) slots)
,@stuff)))

(defmacro lslot-value (instance slot-name &optional lexicon &environment
env)
`(slot-value ,instance ,(ref-form slot-name (or (find-lexicon lexicon)
(env-lexicon env)))))

> With a CL-style package system, the choice is always unambiguous: If you
> use the same symbol as somebody else, you definitely mean the same
> concept. If you want to denote a different concept, you have to use a
> different symbol. It's much harder to make such distinctions with module
> systems.

That is not quite a fair argument. It's true, but only because CL uses
symbols as slot names. That stacks the deck in favor of packages. One
could easily envision a minor variation on CLOS that used abstract slot
identifier objects instead of symbols as slot names, and had lexically
scoped bindings of identifiers onto abstract slot identifiers. This
would provide the exact same functionality but with the slot-identifier
bindings taking place at compile time rather than read time.

But there's a more serious critique of your argument as well: It's true
that "If you use the same symbol as somebody else, you definitely mean
the same concept." However, because *package* is a global resource
which can be manipulated in various ways, then unless you fully qualify
every single symbol in your code (which no one ever does) you have to
carefully manage *package* in order to insure that all the references to
FOO that are supposed to be the same are in fact the same, and
vice-versa. This is certainly doable, but I would claim that it's a
non-trivial burden on the programmer, particularly an inexperienced one.

rg

Ron Garret

unread,
Dec 7, 2008, 5:07:09 PM12/7/08
to
In article <ghha9u$fmh$1...@aioe.org>,
David Golden <david....@oceanfree.net> wrote:

The design intention was that for global bindings one would use either
packages or lexicons but not both, particularly now that lexicons are
closely coupled with packages. Don't forget that every lexicon now has
a package associated with it, and that the binding of a symbol in a
lexicon is the symbol with the same name interned in the lexicon's
package. So binding P1::FOO and P2::FOO into the same lexicon under the
current design would require two different symbols named FOO to be
interned in the same package, which is not possible.

I could "fix" this (fix in scare quotes because it's not clear that this
is actually a problem) by munging symbol names to include their
packages, but I fear this would undermine one of the principal design
goals of lexicons which is to keep things simple and appeal to
programmers who are used to e.g. Python.

rg

Ron Garret

unread,
Dec 7, 2008, 5:20:22 PM12/7/08
to
In article <200812230...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

> Can you come up with a hack that lets me choose exactly which identifiers I
> pull in from some namespace (regardless of what that namespace says it
> exports), without having to maintain import lists?
>
> That would be useful. But I don't see how; thep roblem implicitly calls for
> list maintenance. I mean, you have to write down the symbols that you want.
> That's a list.

Yes, but it does not follow that those lists need to be generated by
manually writing down all of their members. For example, I might want
to be able to say, "I want to use class C1" and automatically get all of
the symbols (slot names, associated methods, etc.) associated with that
class without having to manually list them all.

> Now as far as export lists go, the difficulty of maintaining them isn't due
> to
> any fundamental property of Lisp packages. The automation of export lists
> could
> be added to Lisp packages without changing the design.

Really? How? In particular, how do you distinguish between those
symbols in the package that actually have functionality associated with
them versus those that got interned there as a side-effect of some other
process (like someone mistyping something into an interactive session)?

> One way to solve that might be to have some kind of special state in a
> package
> so that you could do this:
>
> ;; top level
> (in-package 'foo)
>
> (public)
>
> (defun foo ...)
>
> (private)
>
> (defun bar ...)
>
> PUBLIC and PRIVATE would set and reset some flag, and defining constructs
> could
> use it to export or not export the name which they are defining.

How would this work if I'm doing interactive development?

How would it know to export my slot names and accessor function names?

What if I defined a macro that defined other things, how would those
things get exported?

> > This violates the DRY principle, and makes maintenance more
> > difficult than it needs to be, rather like the burden imposed by C on
> > the programmer to maintain the .c file and the .h files in parallel.
>
> If you do not maintain an interface and implementation separately (either as
> separate files or as sections of the same file, as is the case with some
> langauges), it means that in before processing module B that depends on A,
> the
> language implementation has to process all of A, so that it could pull out
> the
> interface information that is implicitly distributed throughout B.

Yes. So?


> C header files, crude as the mechanism is, do allow us to compile some foo.c
> that uses bar.h, without having access bar.c, or any representation of it.

Yes. So the programmer has to do a lot of work to make the compiler's
work easier. I thought computers were supposed to make our lives
easier, not vice-versa.


> In order to have an efficient interface/implementation mechanim (i.e. not
> like
> that of C) you'd need some kind of special support. I.e. there wuold have to
> be
> a way of processing a file, similar to COMPILE-FILE, which would digest all
> of
> the interface information and spit it into some kind of efficient form:
> perhaps
> some special interface file: like a fasl, but with no code. Or maybe a
> special
> section of a fasl.

Not necessarily. Lexicons are implemented so that bindings can be
resolved at run-time if needed. The bindings are cached so the overhead
is only incurred once (and it's only a hash-table lookup, so it's pretty
minimal overhead anyway).


> When a module B uses A, the compiler would pull out A's digested interface
> info
> (refreshing it first, if the source of A is newer). In this way, B could be
> compiled without doing a complete compilation of A.
>
> You know, you could develop an annotation system for Lisp modules which would
> allow such processing. Code is data right?
>
> Let's say we have this syntax:
>
> (module foo)
>
> :public
>
> (defun x ...)
>
> :private
>
> (defun y ...)
>
> You could cob together a Lisp build system which will munge source code and
> generate the defpackage definitions, with export lists and all.
>
> The above would result in there being a (DEFPACKAGE FOO ...) somewhere
> which exports the symbol X.

The problem is that in the presence of macros there's no way to know
what all is being defined without actually running the code.

rg

David Golden

unread,
Dec 7, 2008, 6:48:17 PM12/7/08
to
Ron Garret wrote:

> In article <ghha9u$fmh$1...@aioe.org>,


> I could "fix" this (fix in scare quotes because it's not clear that
> this is actually a problem) by munging symbol names to include their
> packages,

Or perhaps it'd be easier to do it in the package name, so that

a:b --> l.a:b

?

The existing hierarchical.package.support in several popular
implementations would handle that easily, though you maybe don't want
to rely on that being present, the subset needed would probably be
straightforward to include locally.

> but I fear this would undermine one of the principal design
> goals of lexicons which is to keep things simple and appeal to
> programmers who are used to e.g. Python.

Shrug. Well, I'm used to common lisp, and used to package*symbol, not
symbol, being unique. It just seems an avoidable limitation. Means
e.g. ldefun etc. aren't transparently substitutable for defun etc.

Kaz Kylheku

unread,
Dec 7, 2008, 7:19:47 PM12/7/08
to
On 2008-12-07, Ron Garret <rNOS...@flownet.com> wrote:
> In article <200812230...@gmail.com>,
> Kaz Kylheku <kkyl...@gmail.com> wrote:
>> In order to have an efficient interface/implementation mechanim (i.e. not
>> like
>> that of C) you'd need some kind of special support. I.e. there wuold have to
>> be
>> a way of processing a file, similar to COMPILE-FILE, which would digest all
>> of
>> the interface information and spit it into some kind of efficient form:
>> perhaps
>> some special interface file: like a fasl, but with no code. Or maybe a
>> special
>> section of a fasl.
>
> Not necessarily. Lexicons are implemented so that bindings can be
> resolved at run-time if needed. The bindings are cached so the overhead
> is only incurred once (and it's only a hash-table lookup, so it's pretty
> minimal overhead anyway).

Compilers need to know about bindings. For instance, an inline function cannot
be procedurally integrated by a compiler, if it's called through a binding that
is resolved at run-time.

it's nice to be able to defer bindings to run time, but you can't use that
to solve problems at the cost of interfering with compilation.

>> When a module B uses A, the compiler would pull out A's digested interface
>> info
>> (refreshing it first, if the source of A is newer). In this way, B could be
>> compiled without doing a complete compilation of A.
>>
>> You know, you could develop an annotation system for Lisp modules which would
>> allow such processing. Code is data right?
>>
>> Let's say we have this syntax:
>>
>> (module foo)
>>
>> :public
>>
>> (defun x ...)
>>
>> :private
>>
>> (defun y ...)
>>
>> You could cob together a Lisp build system which will munge source code and
>> generate the defpackage definitions, with export lists and all.
>>
>> The above would result in there being a (DEFPACKAGE FOO ...) somewhere
>> which exports the symbol X.
>
> The problem is that in the presence of macros there's no way to know
> what all is being defined without actually running the code.

But you have that problem regardless. For instance, Lexicons provides
alternatives to some some well-known defining macros, but not all.

The build system for the above modules could hack it with some simple guesses.
here are some heuristics that would work ``99%'' of the time:

1. Any symbol that is at the immediate level of a toplevel compound form, and
that is interned in the module's package, is being defined.

Example:

(module :goop
(:uses :blorg))

:public

(define-blorg zorch (squiggle ..))


Here, the package in effect is GOOP. (DEFINE-BLORG ...) is a toplevel
form, with two symbols at the highest nesting level: DEFINE-BLORG
and ZORCH. DEFINE-BLORG comes from the BLORG package. ZORCH is interned
in GOOP. It's in a :PUBLIC section, so we add it to the list of
exported symbols. We know nothing about what DEFINE-BLORG does,
only that it's a form appearing in the :EXPORT section.
The symbol SQUIGGLE is ignored by this analysis, even though it's
in the BLORG package.

2. A toplevel form is understood properly, with regard for PROGN, EVAL-WHEN
and the rest:

:public

(eval-when (... situations ...)
(define-blorg zorch ...)) ;; zorch exported


You know, maybe even having an explicit export construct would help. Experience
in the Linux kernel, for instance, tells me that it's not much of a bother to
do:

void some_function(...)
{
}
EXPORT_SYMBOL(some_function)

In spite of the repetition, there is value in the exporting being locate near
the function rather than in some list somewhere else. If you decide to delete
the function or move it elsewhere, all you have to do is extend the scope of
your edit over one additional line.

So for the cases where the simple heuristic doesn't work, the programmer could
use an explicit export directive.

Also, there could be an API into the system that would allow the programmer to
teach it where the defined symbols are in particular forms.

Suppose you have a

(DEFINE-ZARK (:OP (STRANGE-ZARK) ...))

The name STRANGE-ZARK is embedded in the list. Using some interface, you could
associate the symbol DEFINE-ZARK with a ``name extractor'' hook. The module
system, seeing that DEFINE-ZARK appears in a :PUBLIC section, would then run
this test: (GET-NAME-EXTRACTOR-P 'DEFINE-ZARK). If this returns a function, it
is invoked like this (FUNCALL EXTRACTOR '(DEFINE-ZARK (:OP (STRANGE-ZARK)
...))) and it returns a list of names, namely (STRANGE-ZARK). These are then
added to the list of exported symbols.

The entire project is scanned this way and ther results are automatically
compiled into a repository of generated files containing DEFPACKAGE forms.

The build system arranges for everything to be loaded in the right order,
and the (MODULE ...) construct in each file expands to some forms like
(IN-PACKAGE) and whatnot.

The defining of name extractors would be in the (MODULE ...) syntax, of course.

;;; module ZARK: provides ZARK:DEFINE-ZARK, with an
;;; associated name extractor.

(module :zark
(:name-extractors
(define-zark ((form)
(second (second form)))))
...)

:public

(defmacro define-zark ...)

Ron Garret

unread,
Dec 8, 2008, 2:35:06 AM12/8/08
to

There's no difference in this respect between lexicons and packages. In
both cases, if you know the binding at compile time the compiler can use
it. If you don't, it can't.

> >> When a module B uses A, the compiler would pull out A's digested interface
> >> info
> >> (refreshing it first, if the source of A is newer). In this way, B could
> >> be
> >> compiled without doing a complete compilation of A.
> >>
> >> You know, you could develop an annotation system for Lisp modules which
> >> would
> >> allow such processing. Code is data right?
> >>
> >> Let's say we have this syntax:
> >>
> >> (module foo)
> >>
> >> :public
> >>
> >> (defun x ...)
> >>
> >> :private
> >>
> >> (defun y ...)
> >>
> >> You could cob together a Lisp build system which will munge source code
> >> and
> >> generate the defpackage definitions, with export lists and all.
> >>
> >> The above would result in there being a (DEFPACKAGE FOO ...) somewhere
> >> which exports the symbol X.
> >
> > The problem is that in the presence of macros there's no way to know
> > what all is being defined without actually running the code.
>
> But you have that problem regardless. For instance, Lexicons provides
> alternatives to some some well-known defining macros, but not all.

What's missing?

rg

Ron Garret

unread,
Dec 8, 2008, 2:37:07 AM12/8/08
to
In article <ghhnc2$qsm$1...@aioe.org>,
David Golden <david....@oceanfree.net> wrote:

> Shrug. Well, I'm used to common lisp, and used to package*symbol, not
> symbol, being unique. It just seems an avoidable limitation. Means
> e.g. ldefun etc. aren't transparently substitutable for defun etc.

They are (or at least they should be) as long as you are not defining
symbols with an explicit package qualifier. I suspect this is the case
for the vast majority of actual Lisp code.

rg

Helmut Eller

unread,
Dec 8, 2008, 3:09:46 AM12/8/08
to
* Ron Garret [2008-12-05 17:21+0100] writes:

> Comments and feedback are welcome.

Are lexicons supposed to work with compiled code or only with
interpreted code?

For instance I tried this:

CL-USER> (load "lexicons.lisp")
#P"/home/helmut/lisp/lexicons/lexicons.lisp" CL-USER> (make-lexicon :l1)
#<Lexicon L1> CL-USER> (ldefun foo () (bar))
;Compiler warnings :
; In ROOT::FOO: Undefined function BAR
ROOT::FOO
CL-USER> (compile 'foo)
FOO NIL NIL CL-USER> (ldefun bar () (print 'bar))
ROOT::BAR CL-USER> (foo)
> Special operator or global macro-function BAR can't be FUNCALLed or APPLYed
[Condition of type CCL::CALL-SPECIAL-OPERATOR-OR-MACRO]


Helmut.

PS: lexicons.lisp can't be compiled because the function ENV-LEXICON is
needed during compilation.

Pascal J. Bourguignon

unread,
Dec 8, 2008, 4:16:51 AM12/8/08
to
Ron Garret <rNOS...@flownet.com> writes:

Sorry, I didn't explain well enough what I was doing and what problems
were being solved. I took the point of view of the implementor or
language designer, and starting from the simple notion of symbol, how
do we implement a reader and some system to be able to have different
symbols with the same name. That is, how do we defined different
namespaces.

Introducing named packages that can contain a set of symbol, we can
qualify our symbols with the name of some package, and therefore we
can access two symbols with the same name by prefixing them with the
package name. Q&D but effective namespaces.

Actually since we didn't consider anything else than symbols, symbol
names and package names to build the notion of package, this is
totally orthogonal to the rest of the language, and can therefore be
applied very effectively and modularly to any namespace problem.


For example, in CL you can very easily add methods to be applied to
pre-defined classes (either built-in or coming from libraries) without
any risk of method name clash, because method names are symbols, and
symbols in different packages can be different symbols.


To contrast with the situation in Ruby, where you can effectively add
a method to a predefined class:

(class Array
(def find(element , options = {:key => (function :identity,0), :test (function :==,1),
:start => 0, :end => nil})
...
end)
end)

but unfortunately, this is done in the namespace of the Array class,
and since Ruby is not Lisp, Greenspuning happends everywhere and some
other library will defined a find method too. Name collision, breakage.

So you have to use modules, but then you still cannot include modules,
since when you do so, the methods defined in the module latest
included override any method previously defined in the class or
included from another module.

Therefore you cannot define methods, but you have to define functions
in modules if you want to use namespace separation in Ruby. Lame.


--
__Pascal Bourguignon__

Didier Verna

unread,
Dec 8, 2008, 8:34:01 AM12/8/08
to
Pascal Costanza wrote:

> Because they can make distinctions that you cannot make with module
> systems. For example, if you had only a module system, a class would
> not be able to inherit from two different classes with slots of the
> same name and be able to keep them as two different slots (unless you
> come up with funky renaming mechanisms, like in the R6RS module
> system, for example).

I'm not sure I follow you.

#include <iostream>

struct foo1
{
foo1 () { slot = 1; };
int slot;
};

struct foo2
{
foo2 () { slot = 2; };
int slot;
};

struct mystruct : public foo1, foo2
{
mystruct () { slot = 0; };
int slot;
};


int main (int argc, char *argv[])
{
mystruct ms;
std::cout << ms.slot << ms.foo2::slot << ms.foo1::slot << std::endl;
}

./test => 021

--
Resistance is futile. You will be jazzimilated.

Scientific site: http://www.lrde.epita.fr/~didier
Music (Jazz) site: http://www.didierverna.com

EPITA/LRDE, 14-16 rue Voltaire, 94276 Le Kremlin-Bicętre, France
Tel. +33 (0)1 44 08 01 85 Fax. +33 (0)1 53 14 59 22

Ron Garret

unread,
Dec 8, 2008, 11:46:04 AM12/8/08
to
In article <m2vdtvn...@gmail.com>,
Helmut Eller <eller....@gmail.com> wrote:

> * Ron Garret [2008-12-05 17:21+0100] writes:
>
> > Comments and feedback are welcome.
>
> Are lexicons supposed to work with compiled code or only with
> interpreted code?
>
> For instance I tried this:
>
> CL-USER> (load "lexicons.lisp")
> #P"/home/helmut/lisp/lexicons/lexicons.lisp"
> CL-USER> (make-lexicon :l1)
> #<Lexicon L1>
> CL-USER> (ldefun foo () (bar))
> ;Compiler warnings :
> ; In ROOT::FOO: Undefined function BAR
> ROOT::FOO
> CL-USER> (compile 'foo)
> FOO
> NIL
> NIL
> CL-USER> (ldefun bar () (print 'bar))
> ROOT::BAR
> CL-USER> (foo)
> > Special operator or global macro-function BAR can't be FUNCALLed or APPLYed
> [Condition of type CCL::CALL-SPECIAL-OPERATOR-OR-MACRO]

This is a known limitation of the current implementation, and is
described in the documentation in section 3.2. In brief, you can't
LDEFUN something after you've refer to it because by then it has already
been compiled as a call to an unknown function whereas it needs to be
compiled as a deferred lexical reference. To fix this requires a code
walker.

> PS: lexicons.lisp can't be compiled because the function ENV-LEXICON is
> needed during compilation.

They are supposed to work compiled, so this is a bug. (I use CCL which
compiles everything by default, so I hardly ever use compile-file.) I
don't see offhand what's causing it, and the quick fix I tried didn't
work, but you can work around this problem by loading the file before
compiling it.

rg

Helmut Eller

unread,
Dec 8, 2008, 1:55:34 PM12/8/08
to
* Ron Garret [2008-12-08 17:46+0100] writes:

> This is a known limitation of the current implementation, and is
> described in the documentation in section 3.2. In brief, you can't
> LDEFUN something after you've refer to it because by then it has already
> been compiled as a call to an unknown function whereas it needs to be
> compiled as a deferred lexical reference. To fix this requires a code
> walker.

In section 3.3 you are talking about mutually recursive functions and I
naively expected that LDEFUN was supposed to handle that. Apparently
that refers to something else.

Well, I guess something like DEFDEFERRED would be easy to add. A code
walker would still be good to avoid calls to undefined functions.

Helmut.

Kaz Kylheku

unread,
Dec 8, 2008, 3:09:30 PM12/8/08
to
On 2008-12-08, Didier Verna <did...@lrde.epita.fr> wrote:
> Pascal Costanza wrote:
>
>> Because they can make distinctions that you cannot make with module
>> systems. For example, if you had only a module system, a class would
>> not be able to inherit from two different classes with slots of the
>> same name and be able to keep them as two different slots (unless you
>> come up with funky renaming mechanisms, like in the R6RS module
>> system, for example).
>
> I'm not sure I follow you.

Hi Didier, allow someone with more C++ kung fu.

> #include <iostream>
>
> struct foo1
> {
> foo1 () { slot = 1; };
> int slot;
> };
>
> struct foo2
> {
> foo2 () { slot = 2; };
> int slot;
> };
>
> struct mystruct : public foo1, foo2

Note that public doesn't apply to foo2, which is inherited privately. Syntax
bites!

> {
> mystruct () { slot = 0; };
> int slot;
> };

Your test case is flawed. Although you have demonstrated that multiple classes
in the hiearchy can have a separate definition for slot, what is missing is
this:

1. The observation that this is a semantic feature of C++ which makes it
ipossible to override "slot". In other words, C++ fails to provide virtual data
members, instead giving us something that resembles C structs.

2. The observation that C++ does provide overrideable functions, for which the
example would behave quite differently.

So, let's whip out my pet example:

// two completely different meanings of draw()

class lottery { public: virtual void draw(); };
class graphic { public: virtual void draw(); };

class graphic_lottery : public lottery, public graphic
{ public: void draw(); };

Because draw is a virtual function everywhere, with the same type signature,
it's considered the same function.

Lexically, the name graphic_lotter::draw still shadows the base class
definition. But semantically, draw overrides both functions!

If you pass the object to some module that only knows about lotteries, and
which treats it via a lottery & reference, and that module calls draw, it will
call the override! Likewise, of course, if you pass it to some graphic subystem
that treats is as a graphic, the call to draw also calls the override.

So Pascal's basic thesis is correct: if you have a class system with
overrideable members, and you only have a module system for global bindings,
you get clashes like this.

C++ doesn't effectively have a module system at the class level, because it
treats "lottery::draw" and "graphic::draw" as the same item under inheritance.
C++ namespaces can't fix the problem either because they don't work inside
classes. (Equivalent as Pascal's observation).

Kaz Kylheku

unread,
Dec 8, 2008, 3:29:38 PM12/8/08
to

I think I will stick to Lisp's packaging system.

Handling it at the token -> symbol conversion level is the smart design.

It's one of those good ideas in computing, like abstracting files at the
operating system level. It's good to be able to use the same function to write
to a socket or file.

Handling identifier namespaces at the symbol -> reference level is wrong.
It's like having separate functions for writing to sockets and files.

Now you need a way to handle namespacing in global bindings, a way to handle
namespacing in class slot lookup, and who knows what else. Any data structure
that supports symbol lookup needs to now be equipped with namespace rules. And
that includes user-defined data structures that haven't even been written yet.

There is a kind of Fourier effect in programming. If you are working in the
right Fourier transform space, you implement something once and it fixes a
problem everywhere. If you work in a different space, you not only replicate
effort but create future effort.

I like this analogy: a simple spike in the time domain is a big mess in the
frequency domain requiring infinite bandwidth.

Ron Garret

unread,
Dec 8, 2008, 5:56:22 PM12/8/08
to
In article <m28wqqo...@gmail.com>,
Helmut Eller <eller....@gmail.com> wrote:

> * Ron Garret [2008-12-08 17:46+0100] writes:
>
> > This is a known limitation of the current implementation, and is
> > described in the documentation in section 3.2. In brief, you can't
> > LDEFUN something after you've refer to it because by then it has already
> > been compiled as a call to an unknown function whereas it needs to be
> > compiled as a deferred lexical reference. To fix this requires a code
> > walker.
>
> In section 3.3 you are talking about mutually recursive functions and I
> naively expected that LDEFUN was supposed to handle that.

It is supposed to. And it does, it's just a bit awkward at the moment.
Tokens have to be lexified in order to be handled properly (otherwise
they get CL's normal semantics). But they *don't* have to be bound.

> Well, I guess something like DEFDEFERRED would be easy to add.

It's already there. It's called LEXIFY.

> A code
> walker would still be good to avoid calls to undefined functions.

Yep.

rg

Ron Garret

unread,
Dec 8, 2008, 6:12:19 PM12/8/08
to
In article <200812081...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

> On 2008-12-08, Helmut Eller <eller....@gmail.com> wrote:
> > * Ron Garret [2008-12-08 17:46+0100] writes:
> >
> >> This is a known limitation of the current implementation, and is
> >> described in the documentation in section 3.2. In brief, you can't
> >> LDEFUN something after you've refer to it because by then it has already
> >> been compiled as a call to an unknown function whereas it needs to be
> >> compiled as a deferred lexical reference. To fix this requires a code
> >> walker.
> >
> > In section 3.3 you are talking about mutually recursive functions and I
> > naively expected that LDEFUN was supposed to handle that. Apparently
> > that refers to something else.
> >
> > Well, I guess something like DEFDEFERRED would be easy to add. A code
> > walker would still be good to avoid calls to undefined functions.
>
> I think I will stick to Lisp's packaging system.
>
> Handling it at the token -> symbol conversion level is the smart design.

It is indeed, but that's not what CL does. CL translates *program text*
directly into symbols. There are no "tokens." That is precisely the
problem.

rg

Kaz Kylheku

unread,
Dec 8, 2008, 6:53:29 PM12/8/08
to

ANSI CL:

2.3 Interpretation of Tokens

2.3.1 Numbers as Tokens

2.3.2 Constructing Numbers from Tokens

2.3.3 The Consing Dot

2.3.4 Symbols as Tokens

2.3.5 Valid Patterns for Tokens

2.3.6 Package System Consistency Rules

I usually know what I'm token about. :)

Ron Garret

unread,
Dec 8, 2008, 9:50:56 PM12/8/08
to
In article <200812240...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

OK, let me rephrase: the problem is that tokens as defined in section
2.3 are not first-class data structures.

The whole point of Lisp -- the main thing that differentiates it from
other languages -- is that program semantics are defined on a
first-class intermediate data type (forms) rather than directly on
program text. This has a number of advantages which I hope I don't have
to enumerate here. But the interpretation of *symbols* is defined
directly on program text, not on an intermediate first-class data
structure (which is what I thought you meant by the word "token", but I
guess I'll have to find some other term since token is taken).

rg

Didier Verna

unread,
Dec 9, 2008, 9:28:20 AM12/9/08
to
Kaz Kylheku <kkyl...@gmail.com> wrote:

> Your test case is flawed. Although you have demonstrated that multiple
> classes in the hiearchy can have a separate definition for slot, what
> is missing is this:
>
> 1. The observation that this is a semantic feature of C++ which makes

> it impossible to override "slot". In other words, C++ fails to provide


> virtual data members, instead giving us something that resembles C
> structs.

Pascal didn't seem to refer to slot overriding; only having different
slots with the same name kept separate. But I agree that virtual data
members is missing from C++. Not sure this has anything to do with the
original discussion though.

> 2. The observation that C++ does provide overrideable functions, for
> which the example would behave quite differently.

The situation is even more complex than that:

- virtual functions can be overridden provided that the return value
type is covariant and the arguments types are INVARIANT (not even just
contravariant as would only be required to guarantee type safety),
- non-virtual functions can be overloaded (same name but different
prototypes will be considered as different functions),

- and (and this is were the fun begins): virtual functions not complying
with the covariance/invariance rule will be considered different
(overloadable) ones; so for instance

virtual void lottery::draw (int wireframep)

and later

virtual void graphic_lottery::draw () // oops, forgot the argument

are different functions (you don't even get a warning). Then, if you

lottery *mylot = new graphic_lottery;
mylot->draw (1);

you miss the overridden method.


> So, let's whip out my pet example:
>
> // two completely different meanings of draw()
>
> class lottery { public: virtual void draw(); };
> class graphic { public: virtual void draw(); };
>
> class graphic_lottery : public lottery, public graphic
> { public: void draw(); };
>
> Because draw is a virtual function everywhere, with the same type
> signature, it's considered the same function.
>
> Lexically, the name graphic_lotter::draw still shadows the base class
> definition. But semantically, draw overrides both functions!

Well, yeah, that's the definition of virtual methods.

> If you pass the object to some module that only knows about lotteries,
> and which treats it via a lottery & reference, and that module calls
> draw, it will call the override!

Well, yeah, that's the definition of virtual methods :-) But on the
other hand, if draw is a virtual method and your module doesn't want to
use the override, it will go

lot_or_subclass->lottery::draw ();

instead of just

lot_or_subclass->draw ();

So you still have a separation between different implementations of the
same virtual method.

I still fail to see how all this is relevant to either Pascal's
argument, or to the original point, which was that CL packages operate
at read time instead of at a later stage.

--
Resistance is futile. You will be jazzimilated.

EPITA/LRDE, 14-16 rue Voltaire, 94276 Le Kremlin-Bic�tre, France

Kaz Kylheku

unread,
Dec 9, 2008, 5:24:37 PM12/9/08
to
On 2008-12-09, Didier Verna <did...@lrde.epita.fr> wrote:
> Kaz Kylheku <kkyl...@gmail.com> wrote:
>
>> Your test case is flawed. Although you have demonstrated that multiple
>> classes in the hiearchy can have a separate definition for slot, what
>> is missing is this:
>>
>> 1. The observation that this is a semantic feature of C++ which makes
>> it impossible to override "slot". In other words, C++ fails to provide
>> virtual data members, instead giving us something that resembles C
>> structs.
>
> Pascal didn't seem to refer to slot overriding; only having different
> slots with the same name kept separate.

Which C++ does by making them a different slot, but which it doesn't
do for functions.

The inconsistency itself perfectly illustrate's Pascal's point.

Namespacing rules are shoehorned into reference semantics.

> But I agree that virtual data
> members is missing from C++. Not sure this has anything to do with the
> original discussion though.

It has everything to do with it, because of C++ had virtual
data members, you'd have a problem under inheritance.

Well, the point is that this is a stupid definition. C++ has
a jumble of confused lookup rules. Sometimes two class members
which have the same name are considered to be defining the
same thing, sometimes they are completely unrelated.

>> If you pass the object to some module that only knows about lotteries,
>> and which treats it via a lottery & reference, and that module calls
>> draw, it will call the override!
>
> Well, yeah, that's the definition of virtual methods :-) But on the
> other hand, if draw is a virtual method and your module doesn't want to
> use the override, it will go
>
> lot_or_subclass->lottery::draw ();

That's no good, because you have defeated virtual dispatch.

The code here must not assume that it has a lottery
object.

> So you still have a separation between different implementations of the
> same virtual method.

But if I access them that way, they cease to be virtual.

There is still the fact that my graphic_lottery::draw overrides
both lottery drawing and graphical drawing. That's the function
that is reached by virtual dispatch on either base class.

> I still fail to see how all this is relevant to either Pascal's
> argument, or to the original point, which was that CL packages operate
> at read time instead of at a later stage.

The point is that with symbol packaging, we would have LOTTERY::DRAW
and GRAPHIC::DRAW generic functions. They are different symbols.

For the multiply-derived class, we could independently define methods for
these, so we could customize the drawing and the lottery picking.

(And no, this doesn't really have anything to with method and class separation,
because the same reasoning applies to slots).

C++ doesn't have symbol packaging, so namespace issues are entangled into a
confusing mess of lookup rules that involve semantics of type matching and
whatnot.

These two instances of draw /are/ considered the same symbol because of type
signatures and return value of coinheritance. These two are not the same symbol
because they are data members in different classes. Etc.

C++ illustrate's Pascal's points very well:

P> IMHO, packages, i.e. managing the mapping from strings to symbols, solve
P> the problem better than modules, i.e. managing the mapping from
P> identifiers to concepts.
P> [ ... ]
P>Because they can make distinctions that you cannot make with module systems.
P>For example, if you had only a module system, a class would not be able to
P>inherit from two different classes with slots of the same name and be able to
P>keep them as two different slots (unless you come up with funky renaming
P>mechanisms, like in the R6RS module system, for example).

The clash between the unrelated draw virtual functions shows how you are not
able to inherit from two classes and keep two different draw functions. They
are considered the same, and the proof is that you can write a single draw that
overrides both. There is in fact only one draw symbol, which is why those
functions can be connected together through that name. Scope resolution
doesn't help because it defeats virtual dispatch.

This is the consequence of handling namespace issues at the module (i.e. in
this case class) level. The consequence is that class_a::foo and class_b::foo
are different things, only not really.

The symbols GRAPHIC::DRAW and LOTTERY::DRAW would never be connected
together, except by some devious mechanism that looks at SYMBOL-NAME
equivalence.

D Herring

unread,
Dec 9, 2008, 6:50:28 PM12/9/08
to
Ron Garret wrote:
...

> The whole point of Lisp -- the main thing that differentiates it from
> other languages -- is that program semantics are defined on a
> first-class intermediate data type (forms) rather than directly on
> program text. This has a number of advantages which I hope I don't have
> to enumerate here. But the interpretation of *symbols* is defined
> directly on program text, not on an intermediate first-class data
> structure (which is what I thought you meant by the word "token", but I
> guess I'll have to find some other term since token is taken).

Clojure introduced a useful view of how this could work. IIRC, it
uses text->symbol->resolution instead of interning everything straight
away.

Evaluation
http://clojure.org/evaluation?responseToken=d25a462a7b80271f95464457123e5d63

Differences with Lisps
http://clojure.org/lisps?responseToken=d9edfe17ab0592cbb7f873eb6326aa34
...
# Keywords are not Symbols
# Symbols are not storage locations (see Var)
# nil is not a Symbol
...
# syntax-quote does symbol resolution, so `x is not the same as 'x.
...


- Daniel

Pascal Costanza

unread,
Dec 11, 2008, 9:47:19 AM12/11/08
to
Ron Garret wrote:
> In article <6q2m50F...@mid.individual.net>,
> Pascal Costanza <p...@p-cos.net> wrote:
>
>> Ron Garret wrote:
>>> In article <6q2iu8F...@mid.individual.net>,
>>> Pascal Costanza <p...@p-cos.net> wrote:
>>>
>>>> IMHO, packages, i.e. managing the mapping from strings to symbols, solve
>>>> the problem better than modules, i.e. managing the mapping from
>>>> identifiers to concepts.
>>> Why?
>> Because they can make distinctions that you cannot make with module
>> systems. For example, if you had only a module system, a class would not
>> be able to inherit from two different classes with slots of the same
>> name and be able to keep them as two different slots (unless you come up
>> with funky renaming mechanisms, like in the R6RS module system, for
>> example).
>
> That's true, but how often do you actually do multiple inheritance from
> two different classes that were separately developed?

The question is not how often this happens, but what you can do if and
when it happens. I am aware of situations in Java, for example, where
this can lead to a situation where you have to restart to write
considerable portions of your software from scratch. It's good that this
danger doesn't exist in Common Lisp.

>> With a CL-style package system, the choice is always unambiguous: If you
>> use the same symbol as somebody else, you definitely mean the same
>> concept. If you want to denote a different concept, you have to use a
>> different symbol. It's much harder to make such distinctions with module
>> systems.
>
> That is not quite a fair argument. It's true, but only because CL uses
> symbols as slot names. That stacks the deck in favor of packages. One
> could easily envision a minor variation on CLOS that used abstract slot
> identifier objects instead of symbols as slot names, and had lexically
> scoped bindings of identifiers onto abstract slot identifiers. This
> would provide the exact same functionality but with the slot-identifier
> bindings taking place at compile time rather than read time.

...module figuring out all the fine details. ;)

> But there's a more serious critique of your argument as well: It's true
> that "If you use the same symbol as somebody else, you definitely mean
> the same concept." However, because *package* is a global resource
> which can be manipulated in various ways, then unless you fully qualify
> every single symbol in your code (which no one ever does) you have to
> carefully manage *package* in order to insure that all the references to
> FOO that are supposed to be the same are in fact the same, and
> vice-versa. This is certainly doable, but I would claim that it's a
> non-trivial burden on the programmer, particularly an inexperienced one.

Indeed, that part of CL's package system is not ideal. I would prefer
something like in Modula-2's or Oberon's module system, where you are
required to import each and every identifier, or use fully qualified
names. Especially Oberon's approach is extremely good, IMHO. That would
be a great improvement for CL's packages.

But that's not a reason to throw the baby out with the bathwater...


Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/

Pascal Costanza

unread,
Dec 11, 2008, 9:53:21 AM12/11/08
to
Didier Verna wrote:
> Pascal Costanza wrote:
>
>> Because they can make distinctions that you cannot make with module
>> systems. For example, if you had only a module system, a class would
>> not be able to inherit from two different classes with slots of the
>> same name and be able to keep them as two different slots (unless you
>> come up with funky renaming mechanisms, like in the R6RS module
>> system, for example).
>
> I'm not sure I follow you.
>
> #include <iostream>
>
> struct foo1
> {
> foo1 () { slot = 1; };
> int slot;
> };
>
> struct foo2
> {
> foo2 () { slot = 2; };
> int slot;
> };
>
> struct mystruct : public foo1, foo2
> {
> mystruct () { slot = 0; };
> int slot;
> };
>
>
> int main (int argc, char *argv[])
> {
> mystruct ms;
> std::cout << ms.slot << ms.foo2::slot << ms.foo1::slot << std::endl;
> }
>
> ./test => 021

Right. I wasn't precise enough.

You wouldn't be able to keep the slots separate without support from the
language construct that you're dealing with (here structs). In C,
structs are forced to come up with a semantics for dealing with
nameclashes, and as Kaz pointed out, the resolutions are different
depending on whether you deal with slots or methods.

For every new language construct with similar characteristics, you would
have to do this exercise again and again, with the danger that different
language constructs may have slightly different semantics. For example,
Common Lisp may have resulted in a language where defstruct, defclass
and define-condition have different resolution mechanisms for such
nameclashes, and if I wanted to add a new kind of data structure (using
macros, etc.), I would have to think about this again.

Packages solve this ones, and it's orthogonal to other constructs in the
language. I don't have to think about such things anymore. I think
that's a big advantage.

Kaz Kylheku

unread,
Dec 11, 2008, 12:49:53 PM12/11/08
to
On 2008-12-07, Ron Garret <rNOS...@flownet.com> wrote:
> In article <6q2m50F...@mid.individual.net>,
> Pascal Costanza <p...@p-cos.net> wrote:
>
>> Ron Garret wrote:
>> > In article <6q2iu8F...@mid.individual.net>,
>> > Pascal Costanza <p...@p-cos.net> wrote:
>> >
>> >> IMHO, packages, i.e. managing the mapping from strings to symbols, solve
>> >> the problem better than modules, i.e. managing the mapping from
>> >> identifiers to concepts.
>> >
>> > Why?
>>
>> Because they can make distinctions that you cannot make with module
>> systems. For example, if you had only a module system, a class would not
>> be able to inherit from two different classes with slots of the same
>> name and be able to keep them as two different slots (unless you come up
>> with funky renaming mechanisms, like in the R6RS module system, for
>> example).
>
> That's true, but how often do you actually do multiple inheritance from
> two different classes that were separately developed?

It's not a problem only for multiple inheritance (base-base clash) but also for
single inheritance (base-derived clash). And it doesn't involve just
immediate superclasses.

Remember that in CLOS, slots of the same name override each other (a damn
useful behavior). If the superclass has a slot FOO, and the subclass has a slot
FOO, there is only one slot FOO. You can change the :allocation of a slot
under inheritance, too. Perhaps in the base class, assigning to a slot FOO only
changs FOO in that instance. The derived one spcifies :ALLOCATION :CLASS, so
changing FOO mutates it in every instance of that class. Or vice versa.

Think this is only a potential problem for someone writing a new derived
class? Think again.

Suppose that I have a base class which is widely inherited already in a large
program. I would like to add a slot to this class, so I pick the name FOO.
Now I have to search the entire class hierarchy subclasses from my class
to see if someone is already using that slot name.

Solution: name the slot using the private symbol MYPACKAGE::FOO.

Kaz Kylheku

unread,
Dec 11, 2008, 1:11:40 PM12/11/08
to
On 2008-12-07, Ron Garret <rNOS...@flownet.com> wrote:

Yeah, if Lisp used dumb strings for naming things, that would stack
the deck in the favor of related dumb hacks like modules, wouldn't it.

> One
> could easily envision a minor variation on CLOS that used abstract slot
> identifier objects instead of symbols as slot names, and had lexically

Although any object can be exploited for its EQ equality in order
to identify, not all objects in Lisp have useful interning semantics
under the reader.

> scoped bindings of identifiers onto abstract slot identifiers.

This already exists in the form of the widely used WITH-SLOTS macro.

(with-slots ((f foo) (b bar))) obj
;; now we have lexical F and B representing
;; slots FOO and BAR of obj.
)

But the above relies on the interning of FOO and BAR.

If slots were named by any objects whatsoever, how would you actually write the
code to map some lexical identifier to the abstract slot?

> But there's a more serious critique of your argument as well: It's true
> that "If you use the same symbol as somebody else, you definitely mean
> the same concept." However, because *package* is a global resource
> which can be manipulated in various ways, then unless you fully qualify

COMPILE-FILE and LOAD take care to save and restore *package*.

> every single symbol in your code (which no one ever does) you have to
> carefully manage *package* in order to insure that all the references to

Only read-time or compile-time manipulation of *package* can change
where your code is interning symbols.

In your module of code, just write the (in-package ...) top-level form
somewhere near the top. Then things are read against the package.

It's conceivable that some nasty macro could change the value or contents
of *package* (i.e. nasty because the documented purpose of that macro
is not related to doing anything with packages).

Oh well!

Since Lisp gives us full power of the language at macro-expansion time, no
scoping system with exposed data structures is immune to such interference.

Ron Garret

unread,
Dec 11, 2008, 3:32:18 PM12/11/08
to
In article <200812270...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

> Suppose that I have a base class which is widely inherited already in a large
> program. I would like to add a slot to this class, so I pick the name FOO.
> Now I have to search the entire class hierarchy subclasses from my class
> to see if someone is already using that slot name.

No you don't. All you have to do is call CLASS-SLOTS.

> Solution: name the slot using the private symbol MYPACKAGE::FOO.

There is no guarantee that this slot does not already exist unless you
have complete control over the contents of MYPACKAGE. By extension, for
this solution to be effective, every Lisp programmer in the world who
ever collaborates with someone else needs their own private package (and
probably a different private package for each project they work on).
And because the package namespace is global you need a planet-wide
mechanism for managing package names. So you can't use a package named
MYPACKAGE and expect that to work because sooner or later you'll collide
with someone else's MYPACKAGE. It would have to be
com.gmail.kkylheku.mypackage or something like that. To really be sure,
every package anyone uses anywhere in the world would need to have a
GUID.

There is effectively no difference between that and just choosing a
naming convention that forces all programmers to only use symbols with a
unique prefix in a totally flat namespace, particularly since you are an
advocate of explicit importation of every symbol you want to use from
another package. Instead of

(import com.gmail.kkylheku.mypackage::foo *package*)

(slot-value thing 'foo)

you would do:

(symbol-macrolet ((foo 'com.gmail.kkylheku.mypackage-foo))
...
(slot-value thing foo) ; Note foo is unquoted here
...
)

Now you have the exact same effect without packages, just a single, flat
global namespace. Instead of having "foo" mapped onto
com.gmail.kkylheku.mypackage::foo by the reader, you have foo mapped
into com.gmail.kkylheku.mypackage-foo by the compiler. But the net
effect is exactly the same in both cases, both in terms of avoiding name
clashes, and in terms of how much typing you have to do.

BTW, I was going to mention this earlier but I'll do it now: since you
believe that USE-PACKAGE is a Bad Thing, how exactly am I supposed to
make my own package and still be able to use Common Lisp? Do I have to
explicitly import all of the 700 or so symbols in the common-lisp
package into my package every time? That seems like quite a burden.

rg

Ron Garret

unread,
Dec 11, 2008, 3:52:33 PM12/11/08
to
In article <200812270...@gmail.com>,
Kaz Kylheku <kkyl...@gmail.com> wrote:

> > That is not quite a fair argument. It's true, but only because CL uses
> > symbols as slot names. That stacks the deck in favor of packages.
>
> Yeah, if Lisp used dumb strings for naming things, that would stack
> the deck in the favor of related dumb hacks like modules, wouldn't it.

Just calling something dumb doesn't make it so.

>
> > One
> > could easily envision a minor variation on CLOS that used abstract slot
> > identifier objects instead of symbols as slot names, and had lexically
>
> Although any object can be exploited for its EQ equality in order
> to identify, not all objects in Lisp have useful interning semantics
> under the reader.

So what? Why is it so important that the reader do the work?

Note by the way that the Lisp reader can produce some surprising
results, e.g.:

P2[19]> (eq 'baz (read-from-string (format nil "~S" 'baz)))
T
P2[20]> (eq 'foo (read-from-string (format nil "~S" 'foo)))
NIL

How this can happen is left as an exercise for the reader (no pun
intended).

>
> > scoped bindings of identifiers onto abstract slot identifiers.
>
> This already exists in the form of the widely used WITH-SLOTS macro.
>
> (with-slots ((f foo) (b bar))) obj
> ;; now we have lexical F and B representing
> ;; slots FOO and BAR of obj.
> )
>
> But the above relies on the interning of FOO and BAR.
>
> If slots were named by any objects whatsoever, how would you actually write
> the
> code to map some lexical identifier to the abstract slot?

(let ((identifier (find-slot-identifier class slot-description)))
(slot-value instance identifier))

You can actually do this in standard CL by using uninterned symbols for
slot names, and using e.g.

(defun find-slot-identifier (class description)
(find description (class-slots class) :test 'string-equal :key
'slot-name))

or something like that.

> > But there's a more serious critique of your argument as well: It's true
> > that "If you use the same symbol as somebody else, you definitely mean
> > the same concept." However, because *package* is a global resource
> > which can be manipulated in various ways, then unless you fully qualify
>
> COMPILE-FILE and LOAD take care to save and restore *package*.
>
> > every single symbol in your code (which no one ever does) you have to
> > carefully manage *package* in order to insure that all the references to
>
> Only read-time or compile-time manipulation of *package* can change
> where your code is interning symbols.
>
> In your module of code, just write the (in-package ...) top-level form
> somewhere near the top. Then things are read against the package.
>
> It's conceivable that some nasty macro could change the value or contents
> of *package* (i.e. nasty because the documented purpose of that macro
> is not related to doing anything with packages).
>
> Oh well!
>
> Since Lisp gives us full power of the language at macro-expansion time, no
> scoping system with exposed data structures is immune to such interference.


You think too much like a C++ programmer. In Lisp it is not necessarily
the case that all code is contained in files and compiled all at once
before the program is run. In fact, code can be and often is compiled
at run-time, at which time the reader may be used to parse not only code
but data as well. Run-time manipulation of *package* (and *readtable*
for that matter) are not "nasty", they are standard (albeit advanced)
coding techniques in Lisp.

rg

Ron Garret

unread,
Dec 11, 2008, 4:01:09 PM12/11/08
to
In article <6qcnfpF...@mid.individual.net>,
Pascal Costanza <p...@p-cos.net> wrote:

> Ron Garret wrote:
> > In article <6q2m50F...@mid.individual.net>,
> > Pascal Costanza <p...@p-cos.net> wrote:
> >
> >> Ron Garret wrote:
> >>> In article <6q2iu8F...@mid.individual.net>,
> >>> Pascal Costanza <p...@p-cos.net> wrote:
> >>>
> >>>> IMHO, packages, i.e. managing the mapping from strings to symbols, solve
> >>>> the problem better than modules, i.e. managing the mapping from
> >>>> identifiers to concepts.
> >>> Why?
> >> Because they can make distinctions that you cannot make with module
> >> systems. For example, if you had only a module system, a class would not
> >> be able to inherit from two different classes with slots of the same
> >> name and be able to keep them as two different slots (unless you come up
> >> with funky renaming mechanisms, like in the R6RS module system, for
> >> example).
> >
> > That's true, but how often do you actually do multiple inheritance from
> > two different classes that were separately developed?
>
> The question is not how often this happens, but what you can do if and
> when it happens. I am aware of situations in Java, for example, where
> this can lead to a situation where you have to restart to write
> considerable portions of your software from scratch. It's good that this
> danger doesn't exist in Common Lisp.

The same thing can happen in CL. Unless you can guarantee that package
names are globally unique there's always the possibility of wanting to
combine two code bases that use the same package name. Such name
clashes (for packages called "UTILITIES" for example) are not unheard of.