Symbol pollution non-issue.

[Kaz Kylheku]

You know I was just thinking about this whole non-issue that seems to irk some
people that if you misspell some symbol in your REPL, it gets interned and then
stays there forever until you manually scrub it away. I know that Clojure
obsesses over this also: it has a way by which such symbols are treated as
garbage.

What if packages simply used weak pointers for hanging on to symbols? So, come
garbage collection time, if the only reference to a symbol is via its
registration in the package system, it disappears.

If you call (intern "FOO" some-package), ignore the return value, and then the
next time you call (intern "FOO" some-package), you may get a different object.
However, sucha program has no way to tell that it got a different object; it
has no reference to the previously returned symbol to be able to make a
comparison, and if it does retain that reference, then doing so ensures that
the same symbol is returned.

It's a change in behavior for a program which interns a symbol, discards the
reference and then performs a FIND-SYMBOL to check whether the symbol is still
in the package. It may or may not be.

It seems as if DEFPACKAGE breaks because DEFPACKAGE is used to introduce a
bunch of symbols. The weakness could be set up on a per symbol basis. For
instance exported symbols could be normal references, protected from lapsing
under gc.

Shadowing definitions pose another problem. We sometimes make visible one
package inside another package, but then shadow some selectged symbols.
The shadowing symbols are often internal, and so not protected from lapsing by
the above rule. Solution: the symbols which are in the shadow symbol list of a
package are also protected from lapsing.

The principle is to separate "semantic garbage" symbols in a package,
from ones that are deliberate installed there by programmer intent.

Thoughts?

[Barry Margolin]

In article <201202062...@kylheku.com>,

Kaz Kylheku <k...@kylheku.com> wrote:

> What if packages simply used weak pointers for hanging on to symbols? So,
> come
> garbage collection time, if the only reference to a symbol is via its
> registration in the package system, it disappears.

(defvar foo 3)
(gc)
(print foo)
;;; Error: Unbound symbol FOO

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[Kaz Kylheku]

On 2012-02-06, Barry Margolin <bar...@alum.mit.edu> wrote:
> In article <201202062...@kylheku.com>,
> Kaz Kylheku <k...@kylheku.com> wrote:
>
>> What if packages simply used weak pointers for hanging on to symbols? So,
>> come
>> garbage collection time, if the only reference to a symbol is via its
>> registration in the package system, it disappears.
>
> (defvar foo 3)
> (gc)
> (print foo)
> ;;; Error: Unbound symbol FOO

This wouldn't be a problem because the existence of a variable binding would
create a reference to the symbol FOO.

(I realize that some represenations of dynamic variables don't work that way,
but they would have to for this proposal. I.e. there has to be some association
which binds FOO to a location, making it boundp, and that association holds a
reference to FOO.)

[Kaz Kylheku]

On 2012-02-06, Kaz Kylheku <k...@kylheku.com> wrote:
> On 2012-02-06, Barry Margolin <bar...@alum.mit.edu> wrote:
>> In article <201202062...@kylheku.com>,
>> Kaz Kylheku <k...@kylheku.com> wrote:
>>
>>> What if packages simply used weak pointers for hanging on to symbols? So,
>>> come
>>> garbage collection time, if the only reference to a symbol is via its
>>> registration in the package system, it disappears.
>>
>> (defvar foo 3)
>> (gc)
>> (print foo)
>> ;;; Error: Unbound symbol FOO
>
> This wouldn't be a problem because the existence of a variable binding would
> create a reference to the symbol FOO.

So in other words:

> (defvar foo 3)
> (gc)
> foo
3
> (makunbound 'foo)
> (gc)
> (find-symbol "FOO")
NIL
> foo
;; error: unbound symbol

[Barry Margolin]

In article <20120206...@kylheku.com>,

Kaz Kylheku <k...@kylheku.com> wrote:

> On 2012-02-06, Barry Margolin <bar...@alum.mit.edu> wrote:
> > In article <201202062...@kylheku.com>,
> > Kaz Kylheku <k...@kylheku.com> wrote:
> >
> >> What if packages simply used weak pointers for hanging on to symbols? So,
> >> come
> >> garbage collection time, if the only reference to a symbol is via its
> >> registration in the package system, it disappears.
> >
> > (defvar foo 3)
> > (gc)
> > (print foo)
> > ;;; Error: Unbound symbol FOO
>
> This wouldn't be a problem because the existence of a variable binding would
> create a reference to the symbol FOO.

No it wouldn't. A variable binding is a reference FROM the symbol, not
TO the symbol.

> (I realize that some represenations of dynamic variables don't work that way,
> but they would have to for this proposal. I.e. there has to be some
> association
> which binds FOO to a location, making it boundp, and that association holds a
> reference to FOO.)

If bindings were maintained in a symbol table, it would work that way.
But the more common representation is just a pointer from the symbol's
value cell.

A dynamic LET binding of a special variable will usually contain a
reference to the symbol, but toplevel bindings usually don't.

[Barry Margolin]

MACLISP had a feature called "GCTWA": Garbage Collect Truly Worthless
Atoms. Symbols with no value or function bindings, and which were not
referenced in any data structures other than the obarray, would be
garbage collected.

[Kaz Kylheku]

On 2012-02-06, Barry Margolin <bar...@alum.mit.edu> wrote:
> In article <20120206...@kylheku.com>,
> Kaz Kylheku <k...@kylheku.com> wrote:
>
>> On 2012-02-06, Barry Margolin <bar...@alum.mit.edu> wrote:
>> > In article <201202062...@kylheku.com>,
>> > Kaz Kylheku <k...@kylheku.com> wrote:
>> >
>> >> What if packages simply used weak pointers for hanging on to symbols? So,
>> >> come
>> >> garbage collection time, if the only reference to a symbol is via its
>> >> registration in the package system, it disappears.
>> >
>> > (defvar foo 3)
>> > (gc)
>> > (print foo)
>> > ;;; Error: Unbound symbol FOO
>>
>> This wouldn't be a problem because the existence of a variable binding would
>> create a reference to the symbol FOO.
>
> No it wouldn't. A variable binding is a reference FROM the symbol, not
> TO the symbol.

In the abstract, a binding is some kind of association between a symbol and a
storage place. This association is semantically important, regardless of how
the arrows go and must prevent the symbol from being semantic garbage.

(For instance in simple textbook Lisp interpreters a common strategy is to use
an assoc list to represent an environment. A binding is found using (assoc sym
env). That representation of environments "naturally" protects symbols, but
the list has to be traversed to find variables.

>> (I realize that some represenations of dynamic variables don't work that way,
>> but they would have to for this proposal. I.e. there has to be some
>> association
>> which binds FOO to a location, making it boundp, and that association holds a
>> reference to FOO.)
>
> If bindings were maintained in a symbol table, it would work that way.
> But the more common representation is just a pointer from the symbol's
> value cell.

Perfectly fine; this falls into the same logic as exported symbols being
protected. Basically various attributes somehow protect a symbol:
- presence in the shadowing symbols list
- presence in the list of exported symbols
- function or variable binding
- anything else: definition of the symbol as a class, type, etc.

Clearly, disappearing variables are a complete nonstarter, everyone must agree.

[Pascal J. Bourguignon]

Barry Margolin <bar...@alum.mit.edu> writes:

>> >> Kaz Kylheku <k...@kylheku.com> wrote:
>> >>
>> >>> What if packages simply used weak pointers for hanging on to symbols?
>> >>> So,
>> >>> come
>> >>> garbage collection time, if the only reference to a symbol is via its
>> >>> registration in the package system, it disappears.
>> >>

> MACLISP had a feature called "GCTWA": Garbage Collect Truly Worthless
> Atoms. Symbols with no value or function bindings, and which were not
> referenced in any data structures other than the obarray, would be
> garbage collected.

Indeed. In the case of Common Lisp symbols, they have a value slot, a
function slot and a property list.

If the value slot is bound (even to NIL), or the function slot is bound,
or the property list is not NIL (it cannot be "unbound"), then one could
consider that symbol useless.

However, in Common Lisp, FIND-SYMBOL let you retrieve at least one bit
of information, and collecting "useless" symbols destroy that
information.

'seen-pjb
(find-symbol "SEEN-PJB") ; --> SEEN-PJB
(gc)
(find-symbol "SEEN-PJB") ; -- NIL

would be bad.


On the other hand, if we considered that all the symbols already exist,
this would break the package system, where importing symbols from other
packages or "using" other packages creates collision errors. One would
have to unintern (would that be effective), or "shadow" all the imported
or used symbols before.


But as the title says, it's a non-issue. You just reboot a lisp image
and load a compiled library and those symbols don't exist anymore.

--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

[RG]

In article <201202062...@kylheku.com>,
Kaz Kylheku <k...@kylheku.com> wrote:

> You know I was just thinking about this whole non-issue

It's not a non-issue. The mere fact that this discussion is happening
means that, by definition, it is an issue. See:

http://www.merriam-webster.com/dictionary/issue

"is·sue noun - a matter that is in dispute between two or more parties"

> that seems to irk some
> people that if you misspell some symbol in your REPL, it gets interned and
> then stays there forever until you manually scrub it away.

The problem is not that it stays forever, the problem is that it can
cause extraneous naming conflicts even if it doesn't stay forever. The
most common screw case is this one:

? (make-package :lib)
#<Package "LIB">
? (make-package :app)
#<Package "APP">
? (in-package :lib)
#<Package "LIB">
? (defun libfn () t)
LIBFN
? (in-package :app)
#<Package "APP">
? (defun foo () (libfn))
;Compiler warnings :
; In FOO: Undefined function LIBFN
FOO
? (import 'lib::libfn)
- Error: Importing LIB::LIBFN to #<Package "APP"> would conflict
- with symbol LIBFN .


But that is far from the only screw case.

Note that the mere act of parsing the code (not the subsequent
unsuccessful attempt to compile it) is what left the system in a state
where it is no longer possible to use the library without manual
corrective action.

rg

[Barry Margolin]

In article <87wr7zv...@kuiper.lan.informatimago.com>,

"Pascal J. Bourguignon" <p...@informatimago.com> wrote:

> However, in Common Lisp, FIND-SYMBOL let you retrieve at least one bit
> of information, and collecting "useless" symbols destroy that
> information.
>
> 'seen-pjb
> (find-symbol "SEEN-PJB") ; --> SEEN-PJB
> (gc)
> (find-symbol "SEEN-PJB") ; -- NIL
>
> would be bad.

Why? If you're not actually using the symbol for anything, what could it
break? After all, if you store the result of find-symbol anywhere, that
will prevent it from being GC'ed. If you don't, how would you know that
it ever existed and then disappeared? The user's memory doesn't count;
lots of things get seen by the user, but later get GC'ed.

The only thing I can imagine that would be affected by this would be a
program that uses a package as a replacement for a hash table mapping
strings to booleans. Screw them, they should use an actual hash table.

[Pascal J. Bourguignon]

Barry Margolin <bar...@alum.mit.edu> writes:

> In article <87wr7zv...@kuiper.lan.informatimago.com>,
> "Pascal J. Bourguignon" <p...@informatimago.com> wrote:
>
>> However, in Common Lisp, FIND-SYMBOL let you retrieve at least one bit
>> of information, and collecting "useless" symbols destroy that
>> information.
>>
>> 'seen-pjb
>> (find-symbol "SEEN-PJB") ; --> SEEN-PJB
>> (gc)
>> (find-symbol "SEEN-PJB") ; -- NIL
>>
>> would be bad.
>
> Why? If you're not actually using the symbol for anything, what could it
> break?

I just explained what it could break!
Programs such as:

(defun encode (message)
(loop
:for ch :across message
:for i :from 0
:do (intern (format nil "MSG[~D]=~C" i ch))))

(defun decode ()
(let ((buf (make-array 80 :element-type 'character
:adjustable t
:fill-pointer 0
:initial-element #\?)))
(do-symbols (s)
(let ((ss (string s)))
(when (and (< 4 (length ss)) (string= ss "MSG[" :end1 4))
(multiple-value-bind (i p) (parse-integer ss :start 4
:junk-allowed t)
(when (<= (length buf) i)
(setf buf (adjust-array buf (1+ i)
:element-type (array-element-type buf)
:fill-pointer (fill-pointer buf)
:initial-element #\?)))
(when (<= (fill-pointer buf) i)
(setf (fill-pointer buf) (1+ i)))
(setf (aref buf i) (aref ss (+ p 2)))))))
buf))


CL-USER> (encode "Hello World!")
NIL
CL-USER> (decode)
"Hello World!"

> Screw them, they should use an actual hash table.

What about obfuscation?



But it is a non-issue, because changing this, would change the language,
and have possibly wide ramifications.

[RG]

In article <87obtbu...@kuiper.lan.informatimago.com>,

"Pascal J. Bourguignon" <p...@informatimago.com> wrote:

> Barry Margolin <bar...@alum.mit.edu> writes:
>
> > In article <87wr7zv...@kuiper.lan.informatimago.com>,
> > "Pascal J. Bourguignon" <p...@informatimago.com> wrote:
> >
> >> However, in Common Lisp, FIND-SYMBOL let you retrieve at least one bit
> >> of information, and collecting "useless" symbols destroy that
> >> information.
> >>
> >> 'seen-pjb
> >> (find-symbol "SEEN-PJB") ; --> SEEN-PJB
> >> (gc)
> >> (find-symbol "SEEN-PJB") ; -- NIL
> >>
> >> would be bad.
> >
> > Why? If you're not actually using the symbol for anything, what could it
> > break?
>
> I just explained what it could break!
> Programs such as:

You do realize that if you advanced this as a serious example anywhere
but c.l.l. you would be a laughingstock, right?

> But it is a non-issue, because changing this, would change the language,
> and have possibly wide ramifications.

Nope, can't have any of that evil changey stuff around here, no sir.

rg

[Antony]

I use
(defvar foo )
and leave it intentionally unbound for stuff that I expect to bind in
the context of a thread (as in http processing). This allows me catch
some issues.
We can not throw away such a symbol.

But I think defvar introduced symbols are not a good example of what you
are suggesting to fix anyway. This is because defvar also marks
something as 'special', whether bound or not.
That makes it more (?) than just a symbol.

That brings a question to my mind - how would someone check if something
is special var.

-Antony

[Kaz Kylheku]

On 2012-02-07, RG <rNOS...@flownet.com> wrote:
> In article <87obtbu...@kuiper.lan.informatimago.com>,
> "Pascal J. Bourguignon" <p...@informatimago.com> wrote:
>> I just explained what it could break!
>> Programs such as:
>
> You do realize that if you advanced this as a serious example anywhere
> but c.l.l. you would be a laughingstock, right?

Only because of all the parentheses.

Here it is also a laughingstock because of what it's doing.

[Kaz Kylheku]

Correct. So the throwing away or not cannot be naively predicated on having a
value or not. But if you defvar something, that symbol has to be remembered
somewhere so that it can be pervasively special, and that can protect it.

> That brings a question to my mind - how would someone check if something
> is special var.

(special-variable-p 'sym)

[Tamas Papp]

I don't see the need for manual correction as a nuisance, especially
in this case. Any decent environment should offer the obvious
restarts, and resolving the conflict just takes a few keystrokes in
SLIME (BTW, SBCL also gives a style-warning about the DEFUN FOO using
the undefined function LIBFM).

I prefer deciding these issues manually when they occur instead of
some dwim-style cleverness that would save a few keystrokes in 99% of
the cases, then, in the remaining 1%, fail silently and cause trouble
later.

Best,

Tamas

[Tim Bradshaw]

RG <rNOS...@flownet.com> wrote:

> It's not a non-issue. The mere fact that this discussion is happening
> means that, by definition, it is an issue. See:

It's an issue in the sense that people worry about it. Would it be an
issue if they just stopped worrying? I don't know the answer to that, but
I suspect not.

[Antony]

Didn't find it in CLHS
Says undefined function in ccl (also sbcl)
Then I tried in clisp, it works

Interesting.

-Antony

[RG]

In article
<208471263350311915...@reader443.eternal-september.org>,

http://en.wikipedia.org/wiki/Dr._Strangelove

[Pascal J. Bourguignon]

RG <rNOS...@flownet.com> writes:

> In article <87obtbu...@kuiper.lan.informatimago.com>,
> "Pascal J. Bourguignon" <p...@informatimago.com> wrote:
>
>> Barry Margolin <bar...@alum.mit.edu> writes:
>>
>> > In article <87wr7zv...@kuiper.lan.informatimago.com>,
>> > "Pascal J. Bourguignon" <p...@informatimago.com> wrote:
>> >
>> >> However, in Common Lisp, FIND-SYMBOL let you retrieve at least one bit
>> >> of information, and collecting "useless" symbols destroy that
>> >> information.
>> >>
>> >> 'seen-pjb
>> >> (find-symbol "SEEN-PJB") ; --> SEEN-PJB
>> >> (gc)
>> >> (find-symbol "SEEN-PJB") ; -- NIL
>> >>
>> >> would be bad.
>> >
>> > Why? If you're not actually using the symbol for anything, what could it
>> > break?
>>
>> I just explained what it could break!
>> Programs such as:
>
> You do realize that if you advanced this as a serious example anywhere
> but c.l.l. you would be a laughingstock, right?

This is a conforming program.

>> But it is a non-issue, because changing this, would change the language,
>> and have possibly wide ramifications.
>
> Nope, can't have any of that evil changey stuff around here, no sir.

[RG]

In article <jgr3nq$nc4$1...@dont-email.me>, Tamas Papp <tkp...@gmail.com>
wrote:

For that toy example it's not so bad. Consider this case:

(defun foo () (f1) (f2) (f3) ... (fn))

where each of the f's are defined in different package.

Or, worse:

(defun foo (x y z) ...)

(use-package :snoz) ; which happens to export symbols named X, Y and Z

Now you have to manually resolve a conflict that may or may not actually
be a conflict (but almost certainly isn't).

> (BTW, SBCL also gives a style-warning about the DEFUN FOO using
> the undefined function LIBFM).

Yes, but by the time you get those warnings the damage has already been
done.

> I prefer deciding these issues manually when they occur instead of
> some dwim-style cleverness that would save a few keystrokes in 99% of
> the cases, then, in the remaining 1%, fail silently and cause trouble
> later.

But packages *are* "dwim-style cleverness" it's just that you have to
decide the w-i-m part before your code is parsed. When you type "foo"
what you really mean is (intern "foo" *package*) because:

(defun foo (x y z) (f (g x) (h y z)))

looks a lot nicer than:

(cl:defun com.foo.joe::foo (com.foo.joe::x com.foo.joe::y com.foo.joe::z)
(com.baz.lib::f (com.baz.lib::g com.foo.joe::x) (com.foo.lib::h
com.foo.joe::y com.foo.joe::z)))

I've actually seen people write CL code that looks like the second
example, BTW.

rg

[Pascal J. Bourguignon]

(defun special-variable-p (sym)
(eval `(let ((,sym 42)) (declare (ignorable ,sym)) (boundp ',sym))))

(defvar *special*)
(let ((lexical 42))
(declare (ignorable lexical))
(list (special-variable-p '*special*)
(special-variable-p 'lexical)
(special-variable-p 'unknown)))
--> (T NIL NIL)

[Barry Margolin]

In article <201202070...@kylheku.com>,

Also, if you have functions that bind it, the references from those
functions should protect it. Although that's not totally safe -- what
if a GC happens between the DEFVAR and DEFUN.

[Barry Margolin]

In article
<208471263350311915...@reader443.eternal-september.org>,
Tim Bradshaw <t...@tfeb.org> wrote:

I looked up "nonissue" in the dictionary. It says "a topic of little or
no importance". It's not simply the opposite of an issue, it's a
statement about how significant it is.

[Kaz Kylheku]

On 2012-02-07, Antony <remove+spam...@gmail.com> wrote:
> On 2/7/2012 12:05 AM, Kaz Kylheku wrote:
>> On 2012-02-07, Antony<remove+spam...@gmail.com> wrote:
>>> That brings a question to my mind - how would someone check if something
>>> is special var.
>>
>> (special-variable-p 'sym)
> Didn't find it in CLHS
> Says undefined function in ccl (also sbcl)
> Then I tried in clisp, it works

Damn, did I teach you a CLISP-ism? Sorry.

[Kaz Kylheku]

On 2012-02-07, Pascal J. Bourguignon <p...@informatimago.com> wrote:

> RG <rNOS...@flownet.com> writes:
>> You do realize that if you advanced this as a serious example anywhere
>> but c.l.l. you would be a laughingstock, right?
>
> This is a conforming program.

Yes, well x = a//*haha*/b; was statement in a conforming program in C90.

Then // became a comment delimiter.

The chaos was unprecedented. An entire civilization perished.

>> Nope, can't have any of that evil changey stuff around here, no sir.

Ahem.

[Tim Bradshaw]

Barry Margolin <bar...@alum.mit.edu> wrote:

> I looked up "nonissue" in the dictionary. It says "a topic of little or
> no importance". It's not simply the opposite of an issue, it's a
> statement about how significant it is.

Yes, and I suspect that it is indeed a topic of little or no importance. I
don't know it is not important, but I suspect it may be, people making
rather spurious comparisons with nuclear weapons notwithstanding (that has
to be some kind of generalised Godwinisation I think).

I can't remember the time it last caused me pain (to which I know the
standard response is "but you *know* CL, think of the poor starving
children, do you want them do die you heartless brute? &c &c").

[Sam Steingold]

> * Kaz Kylheku <x...@xlyurxh.pbz> [2012-02-07 17:03:08 +0000]:

no, CLOCC/PORT has it portably.

--
Sam Steingold (http://sds.podval.org/) on Ubuntu 11.10 (oneiric) X 11.0.11004000
http://pmw.org.il http://www.PetitionOnline.com/tap12009/
http://memri.org http://openvotingconsortium.org http://dhimmi.com
If a train station is a place where a train stops, what's a workstation?

[Kaz Kylheku]

On 2012-02-07, Barry Margolin <bar...@alum.mit.edu> wrote:
> I looked up "nonissue" in the dictionary. It says "a topic of little or
> no importance". It's not simply the opposite of an issue, it's a
> statement about how significant it is.

I just used the word because I suspected it would spark a lively debate.

[RG]

In article <87fwemv...@kuiper.lan.informatimago.com>,

"Pascal J. Bourguignon" <p...@informatimago.com> wrote:

> RG <rNOS...@flownet.com> writes:
>
> > In article <87obtbu...@kuiper.lan.informatimago.com>,
> > "Pascal J. Bourguignon" <p...@informatimago.com> wrote:
> >
> >> Barry Margolin <bar...@alum.mit.edu> writes:
> >>
> >> > In article <87wr7zv...@kuiper.lan.informatimago.com>,
> >> > "Pascal J. Bourguignon" <p...@informatimago.com> wrote:
> >> >
> >> >> However, in Common Lisp, FIND-SYMBOL let you retrieve at least one bit
> >> >> of information, and collecting "useless" symbols destroy that
> >> >> information.
> >> >>
> >> >> 'seen-pjb
> >> >> (find-symbol "SEEN-PJB") ; --> SEEN-PJB
> >> >> (gc)
> >> >> (find-symbol "SEEN-PJB") ; -- NIL
> >> >>
> >> >> would be bad.
> >> >
> >> > Why? If you're not actually using the symbol for anything, what could it
> >> > break?
> >>
> >> I just explained what it could break!
> >> Programs such as:
> >
> > You do realize that if you advanced this as a serious example anywhere
> > but c.l.l. you would be a laughingstock, right?
>
> This is a conforming program.

You have completely missed the point. You would not be a laughingstock
because your program is non-conforming, you would be a laughingstock
because your program is stupid.

rg

[RG]

In article
<675780956350327169...@reader443.eternal-september.org>,

Well, we keep going around in circles on this. I'm glad it doesn't
cause you pain, but it drives me crazy. Resolving namespaces at parse
time is a Bad Idea. It violates at least two fundamental principles of
good software engineering practice: global state is evil, and early
binding is evil. And given that Lisp is the origin of both late binding
and functional programming, there is considerable irony in the fact that
this is even controversial, never mind that I seem to be a lone voice in
the wilderness.

rg

[Tim Bradshaw]

On 2012-02-07 19:21:31 +0000, RG said:

> Well, we keep going around in circles on this. I'm glad it doesn't
> cause you pain, but it drives me crazy. Resolving namespaces at parse
> time is a Bad Idea. It violates at least two fundamental principles of
> good software engineering practice: global state is evil, and early
> binding is evil. And given that Lisp is the origin of both late binding
> and functional programming, there is considerable irony in the fact that
> this is even controversial, never mind that I seem to be a lone voice in
> the wilderness.

That sounds to me like evidence that it is, in fact, not a problem for
almost everyone.

[RG]

In article <jgs1it$hqv$1...@dont-email.me>, Tim Bradshaw <t...@tfeb.org>
wrote:

"Almost everyone" and "almost everyone on c.l.l." are two very different
things. And there's a pretty strong selection bias at work here.

rg

[Tim Bradshaw]

On 2012-02-07 21:32:52 +0000, RG said:

> "Almost everyone" and "almost everyone on c.l.l." are two very different
> things. And there's a pretty strong selection bias at work here.

True. As (apocryphal) evidence I can add that no one I've been
involved in teaching Lisp has worried about this (but that is not a
good sample for lots of reasons).

[Barry Margolin]

In article <rNOSPAMon-8889E...@news.albasani.net>,

RG <rNOS...@flownet.com> wrote:

> Well, we keep going around in circles on this. I'm glad it doesn't
> cause you pain, but it drives me crazy. Resolving namespaces at parse
> time is a Bad Idea. It violates at least two fundamental principles of
> good software engineering practice: global state is evil, and early
> binding is evil. And given that Lisp is the origin of both late binding
> and functional programming, there is considerable irony in the fact that
> this is even controversial, never mind that I seem to be a lone voice in
> the wilderness.

We've known all along that the package system is a very poor system. It
was intentionally put in Chapter 11 of CLTL and the ANSI spec, because
Chapter 11 is a reference to "bankrupt" in the US.

The reason we couldn't just adopt something like the module systems used
by many other languages is because symbols in Lisp are not like names in
other languages. We have the problem that symbols are first-class
objects, they're not just used to name variables and functions.

[Raffael Cavallaro]

On 2012-02-07 19:21:31 +0000, RG said:

> And given that Lisp is the origin of both late binding
> and functional programming, there is considerable irony in the fact that
> this is even controversial, never mind that I seem to be a lone voice in
> the wilderness.

I for one agree with you in both theory and practice - it should never
have been designed this way, and it is a daily annoyance mitigated only
by the fact that I have learned to reflexively choose the desired
restart.

warmest regards,

Ralph

--
Raffael Cavallaro

[RG]

In article <jgt2kn$hmu$1...@dont-email.me>,
Raffael Cavallaro

Thanks, Ralph, I appreciate that.

rg

[RG]

In article <barmar-99766A....@news.eternal-september.org>,

Barry Margolin <bar...@alum.mit.edu> wrote:

> In article <rNOSPAMon-8889E...@news.albasani.net>,
> RG <rNOS...@flownet.com> wrote:
>
> > Well, we keep going around in circles on this. I'm glad it doesn't
> > cause you pain, but it drives me crazy. Resolving namespaces at parse
> > time is a Bad Idea. It violates at least two fundamental principles of
> > good software engineering practice: global state is evil, and early
> > binding is evil. And given that Lisp is the origin of both late binding
> > and functional programming, there is considerable irony in the fact that
> > this is even controversial, never mind that I seem to be a lone voice in
> > the wilderness.
>
> We've known all along that the package system is a very poor system.

Who is "we"? Most people here seem to be big fans.

> It
> was intentionally put in Chapter 11 of CLTL and the ANSI spec, because
> Chapter 11 is a reference to "bankrupt" in the US.

Ha! I thought so.

https://groups.google.com/d/msg/comp.lang.lisp/wE7yOt434nw/xMc48dZD_mwJ

> The reason we couldn't just adopt something like the module systems used
> by many other languages is because symbols in Lisp are not like names in
> other languages. We have the problem that symbols are first-class
> objects, they're not just used to name variables and functions.

Yes, that's true, but that turns out not to be a show-stopper to fixing
the problem.

http://rondam.blogspot.com/2010/02/new-and-improved-lexicons-now-50-lexie
r.html

Of course, the first step to solving any problem is admitting you have a
problem.

rg

[Tim Bradshaw]

RG <rNOS...@flownet.com> wrote:

> Who is "we"? Most people here seem to be big fans.

For what it's worth, I am not a big fan of it. I just don't think it gets
in my way, and since I am entirely driven by pragmatism, it is therefore
not high on my list of worries. If I cared at all about clean I would be
using another language.

[Barry Margolin]

In article <rNOSPAMon-E1504...@news.albasani.net>,

RG <rNOS...@flownet.com> wrote:

> In article <barmar-99766A....@news.eternal-september.org>,
> Barry Margolin <bar...@alum.mit.edu> wrote:
>
> > In article <rNOSPAMon-8889E...@news.albasani.net>,
> > RG <rNOS...@flownet.com> wrote:
> >
> > > Well, we keep going around in circles on this. I'm glad it doesn't
> > > cause you pain, but it drives me crazy. Resolving namespaces at parse
> > > time is a Bad Idea. It violates at least two fundamental principles of
> > > good software engineering practice: global state is evil, and early
> > > binding is evil. And given that Lisp is the origin of both late binding
> > > and functional programming, there is considerable irony in the fact that
> > > this is even controversial, never mind that I seem to be a lone voice in
> > > the wilderness.
> >
> > We've known all along that the package system is a very poor system.
>
> Who is "we"? Most people here seem to be big fans.

The CL designers.

It was an ugly compromise, an incremental improvement on the Lisp
Machine package system.

[Don Geddis]

Barry Margolin <bar...@alum.mit.edu> wrote on Wed, 08 Feb 2012:
> We've known all along that the package system is a very poor system. It
> was intentionally put in Chapter 11 of CLTL and the ANSI spec, because
> Chapter 11 is a reference to "bankrupt" in the US.

That's really funny! Never knew the joke before. Great story from the
good old days.
_______________________________________________________________________________
Don Geddis http://don.geddis.org/ d...@geddis.org
"Apparently, it was not as big of a surprise as we had anticipated."
-- Cylon Warrior to Imperious Leader

[Waldek Hebisch]

Well, I would like symbols to have life independent of any package.
Packages then would be mapping from symbols to variables. I would
prefer Lisp-1 so I wrote variables, but of course you get Lisp-2
if you have parallel mapping from symbols to functions. Also,
I am not a fan of property lists, but if you want to avoid
global conflicts of property lists you need third mapping.

I believe that such Lisp would be better than Common Lisp. But
given attitude in Lisp comunity I do not expect that somebody
will implement this for me, and I do not care that much to
do this myself.

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[RG]

In article <jguqnk$gkc$2...@z-news.wcss.wroc.pl>,

That's right. That's one of the beauties of having first-class global
lexical environments: you can have as many of them as you need.

> I believe that such Lisp would be better than Common Lisp. But
> given attitude in Lisp comunity I do not expect that somebody
> will implement this for me, and I do not care that much to
> do this myself.

It has been done for you:

http://rondam.blogspot.com/2010/02/new-and-improved-lexicons-now-50-lexie
r.html

rg

[Marco Antoniotti]

It depends on your definition of "variable". What would be the concrete syntax of a variable in your scheme?

Cheers
--
MA

[RG]

In article
<22086599.700.1328879040495.JavaMail.geo-discussion-forums@vbnm4>,

Huh? Why would it be different from any other Lisp?

rg

[Marco Antoniotti]

Exaclty. The question was not directed at you.

MA

[RG]

In article
<5127445.686.1329115059663.JavaMail.geo-discussion-forums@yns25>,

Ah.

rg

[Waldek Hebisch]

Why do you ask syntax? You can make it _look_ like current syntax,
but semantics would be different: "foo" would be read as a symbol
and cause compiler/interpreter to access correspondig variable
in current package, "foo:bar" would be read as compund expression
containing two symbols "foo" and "bar" say (package-deref 'foo 'bar)
and only at compile/interpret time bar would be looked up in
the package foo.

Of course purists would object that the proposal introduces
infix syntax into Lisp. If you want prefix syntax (: foo bar)
or (package-deref foo bar) could be used (however the second
one is IMHO too heavy, the first one probaly could work).

--
Waldek Hebisch
heb...@math.uni.wroc.pl

[Kaz Kylheku]

On 2012-02-08, Waldek Hebisch <heb...@math.uni.wroc.pl> wrote:
> Well, I would like symbols to have life independent of any package.
> Packages then would be mapping from symbols to variables. I would
> prefer Lisp-1 so I wrote variables, but of course you get Lisp-2
> if you have parallel mapping from symbols to functions. Also,

That's inadequate because not everything that needs to be named
by a symbol and put into a package is a function or variable.

Your package system needs to support additional namespaces like
classes.

Plus it has to be user-extensible to new spaces. Someone might want to
invent a language with, say, some global named rules, and want those to be
isolated into packages.

> I believe that such Lisp would be better than Common Lisp.

I believe you drank too much cough syrup this morning.

A symbol should be enough to resolve the question of identity. Under your
system, a symbol is no longer enough to identify. It requires a package.

Now everything becomes complicated. For instance FIND-CLASS cannot just
take a single argument, because a symbol only identifies a class within
some package. It has to be (find-class symbol package).
Or worse; (find-class symbol) --- package is implicit somehow based
on where the call occurs, ugh.

No thanks.

[RG]

In article <201202170...@kylheku.com>,

Kaz Kylheku <k...@kylheku.com> wrote:

> On 2012-02-08, Waldek Hebisch <heb...@math.uni.wroc.pl> wrote:
> > Well, I would like symbols to have life independent of any package.
> > Packages then would be mapping from symbols to variables. I would
> > prefer Lisp-1 so I wrote variables, but of course you get Lisp-2
> > if you have parallel mapping from symbols to functions. Also,
>
> That's inadequate because not everything that needs to be named
> by a symbol and put into a package is a function or variable.
>
> Your package system needs to support additional namespaces like
> classes.

That's not a problem. Just add more top-level environments. See the
lexicons implementation for an existence proof.

> Plus it has to be user-extensible to new spaces.

See above. Because lexicons are first-class objects you can create as
many of them as you want.

> > I believe that such Lisp would be better than Common Lisp.
>
> I believe you drank too much cough syrup this morning.
>
> A symbol should be enough to resolve the question of identity. Under your
> system, a symbol is no longer enough to identify. It requires a package.
>
> Now everything becomes complicated. For instance FIND-CLASS cannot just
> take a single argument, because a symbol only identifies a class within
> some package. It has to be (find-class symbol package).

This is true of Common Lisp as well, it's just that the ambiguity arises
at read-time, and is resolved by the read-time value of the global
variable *package*. The same solution can be used at compile time or
run-time.

> Or worse; (find-class symbol) --- package is implicit somehow based
> on where the call occurs, ugh.

In CL, package is implicit depending on how symbol was constructed, but
a typical scenario is:

(find-class 'symbol)

where the symbol is constructed by the reader and package is implicit
depending on the value of *package* when that form was read. How you
can find that any less distasteful is a mystery to me.

There is one (and only one) situation in which symbols with packages are
actually necessary in CL (which is to say, in which it is not
straightforward to provide equivalent functionality using compile-time
constructs). I doubt most CL programmers even know what that situation
is.

rg

[Kaz Kylheku]

This violates the principle of: do not move into run-time what is
adequately and trivially solved at read time.

(How far do you take this silliness? Why don't we look for closing
parentheses at run time also.)

>> Or worse; (find-class symbol) --- package is implicit somehow based
>> on where the call occurs, ugh.
>
> In CL, package is implicit depending on how symbol was constructed, but
> a typical scenario is:
>
> (find-class 'symbol)
>
> where the symbol is constructed by the reader and package is implicit
> depending on the value of *package* when that form was read.

I.e. package is no longer implicit at the time the find-class is being run.
The symbol is done. We don't have to /think/ about the package as it relates
to FIND-CLASS (because it doesn't, in any way).

> How you can find that any less distasteful is a mystery to me.

You can find that less distasteful precisely because read-time is a simple
processing phase that you can forget about once it is done, and just think
about the resulting symbols (which are exactly like symbols in a Lisp that
has no package system.)

Never do at run time what can be perfectly taken care of at read-time.
Whenever static can completely "pwn" dynamic, we should support static.
Compiling beats interpreting; macros kick the crap out of fexprs.
(OTOH, dynamic typing kills static, so screw that.)

These little package annoyances exist for those who insist on programming
at the REPL. Programmers who put code into files and build their programs
don't see any of this "oh oh, I interned a symbol and now there will be
a clash."

Which is why I have never been bitten by these problems; I generally do not
not use REPLs. Even when experimenting with code, I put it into small
files and load those.

Any signficant programming should never be done outside of the context
of a version control system anyway. Just being able to do a diff between
experiment and the last working version is darn useful.

You will never achieve a REPL in which you cannot screw up so badly that
it's not easier to just kill the image and start with a fresh one.
Packages are not really at the root of this problem. It's the REPL itself.

Whenever I use a REPL for anything more complicated than evaluating a few
expressions, I get bitten by the fact that it's a write-only black hole
way sooner than I can get bitten by any package-related issue.

[Pascal J. Bourguignon]

Kaz Kylheku <k...@kylheku.com> writes:

> Whenever I use a REPL for anything more complicated than evaluating a few
> expressions, I get bitten by the fact that it's a write-only black hole
> way sooner than I can get bitten by any package-related issue.

Quoting Zach quoting Rob Warnock: Use the Lisp environment to make your
life easier!!!

http://www.informatimago.com/develop/lisp/small-cl-pgms/ibcl/

--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

[RG]

In article <201202171...@kylheku.com>,

We have an honest difference of opinion about what constitutes the
problem being "solved." IMHO a parser that mutates global state is not
a solution.

> >> Or worse; (find-class symbol) --- package is implicit somehow based
> >> on where the call occurs, ugh.
> >
> > In CL, package is implicit depending on how symbol was constructed, but
> > a typical scenario is:
> >
> > (find-class 'symbol)
> >
> > where the symbol is constructed by the reader and package is implicit
> > depending on the value of *package* when that form was read.
>
> I.e. package is no longer implicit at the time the find-class is being run.

Yeah? So? It is no longer implicit at run time when using lexicons
either. In this respect the two approaches are equivalent.

> > How you can find that any less distasteful is a mystery to me.
>
> You can find that less distasteful precisely because read-time is a simple
> processing phase that you can forget about once it is done

But you can't forget about it. That's the whole point. The are common
error situations where the reader mutates global state in ways that you
don't want.

> These little package annoyances exist for those who insist on programming
> at the REPL. Programmers who put code into files and build their programs
> don't see any of this "oh oh, I interned a symbol and now there will be
> a clash."

First, that's not true. Even if you put your code into files you can
still get bitten by the reader's mutating global state.

And second, one of the oft-touted features of Lisp is that it supports
interactive exploratory development. A "solution" to a problem that
discards one of the biggest (for some people) advantages of using the
language in the first place is not a solution, IMHO.

> Which is why I have never been bitten by these problems; I generally do not
> not use REPLs. Even when experimenting with code, I put it into small
> files and load those.

Well, bully for you. I use the REPL. I like the REPL. I get a lot of
leverage from the REPL. IMHO the REPL is one of the things that makes
Lisp better than, say, Java. And I don't think I'm the only one who
feels this way.

> Any signficant programming should never be done outside of the context
> of a version control system anyway. Just being able to do a diff between
> experiment and the last working version is darn useful.
>
> You will never achieve a REPL in which you cannot screw up so badly that
> it's not easier to just kill the image and start with a fresh one.
> Packages are not really at the root of this problem. It's the REPL itself.

Ah. Well. It seems that our disagreement runs even deeper than I
thought.

> Whenever I use a REPL for anything more complicated than evaluating a few
> expressions, I get bitten by the fact that it's a write-only black hole
> way sooner than I can get bitten by any package-related issue.

Maybe you just don't know how to use the REPL properly.

BTW, note that "a REPL" is not synonymous with "a REPL that receives its
input from GNU readline." I agree with you that such a REPL is not good
for much more than casual noodling around. But that is not the only
sort of REPL there is.

Maybe you have just never experienced a proper REPL.

rg

[Tim Bradshaw]

On 2012-02-17 23:20:43 +0000, Kaz Kylheku said:

> You can find that less distasteful precisely because read-time is a simple
> processing phase that you can forget about once it is done, and just think
> about the resulting symbols (which are exactly like symbols in a Lisp that
> has no package system.)

Actually, this is something I was thinking about. I don't want to
defend the package system[*], but, for those of us who learned Lisp on
implementations without any kind of namespace management, it is better
than that, and it's also conceptually compatible with what those
implementations did.

--tim

[*] on the other hand I think there are a lot of things which would be
ahead of it if I had to make a list of things-to-fix, such as a
standard sockets API for instance (with probable resulting need for a
properly-sorted I/O API) and so on.

[Tim Bradshaw]

On 2012-02-18 00:15:41 +0000, RG said:

> Well, bully for you. I use the REPL. I like the REPL. I get a lot of
> leverage from the REPL. IMHO the REPL is one of the things that makes
> Lisp better than, say, Java. And I don't think I'm the only one who
> feels this way.

strongly agree with that. I live in a REPL for weeks.

[Pascal Costanza]

On 17/02/2012 20:21, RG wrote:

> There is one (and only one) situation in which symbols with packages are
> actually necessary in CL (which is to say, in which it is not
> straightforward to provide equivalent functionality using compile-time
> constructs). I doubt most CL programmers even know what that situation
> is.

Language constructs are never "necessary" - the only thing that is
"necessary" is Turing equivalence (and even that is not always the case).

I can think of at least two situations where the Common Lisp package
system is /useful/:

1) With the Common Lisp package system, it's easy to make the correct
distinctions between slots and slot accessors in classes that happen to
have the same name: In Common Lisp this is extremely easy - they are the
same slot exactly when they are named by the same symbol. In all other
languages that provide some form of multiple inheritance, this is
substantially harder and can cause real headaches.

2) The Common Lisp package system virtually removes hygiene issues from
its macro system. The package system gives you a good way to precisely
specify which symbols are known outside of a macro and which are not.
Without the package system, you have to start introducing constructs for
ensuring hygiene into the macro system (either low-level constructs, or
"automatic" hygiene).

In both cases, the underlying language constructs (classes, macros) can
remain relatively simple with the package system and just rely on symbol
identity, but have to come up with their own name distinction mechanisms
without the package system. This translates directly to other language
extensions that you may want to add as libraries to the language: With
the package system, you don't have to worry too much about naming
ambiguities, but without the package system, you have to start designing
additional features to deal with them for each and every extension.

If you argue for dropping the package system, and replacing it with
something else, you need to provide an answer here...


Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
The views expressed are my own, and not those of my employer.

[RG]

In article <9q9vlf...@mid.individual.net>,

Pascal Costanza <p...@p-cos.net> wrote:

> If you argue for dropping the package system, and replacing it with
> something else, you need to provide an answer here...

I have NEVER argued for DROPPING the package system.

rg

[Pascal Costanza]

So what are you arguing for?

[RG]

In article <9qa52a...@mid.individual.net>,

Pascal Costanza <p...@p-cos.net> wrote:

> On 18/02/2012 18:09, RG wrote:
> > In article<9q9vlf...@mid.individual.net>,
> > Pascal Costanza<p...@p-cos.net> wrote:
> >
> >> If you argue for dropping the package system, and replacing it with
> >> something else, you need to provide an answer here...
> >
> > I have NEVER argued for DROPPING the package system.
>
> So what are you arguing for?

http://rondam.blogspot.com/2010/02/new-and-improved-lexicons-now-50-lexie
r.html

rg

[Pascal Costanza]

I see. I wasn't aware (anymore) that you're suggesting a mapping between
your lexicons to packages. Considering what you seem to think is the
major problem with packages, this may still be overkill.

A problem in the way how you word your concerns is that you describe the
package system as fundamentally flawed, but the package system is
actually not the source of the problem you have. It's rather a detail of
how the read algorithm in Common Lisp works.

What you suggest under the link above is essentially a different read
algorithm, where symbols that cannot be found are not automatically
interned, but rather patched up in such a way that they find their
actual definition later on during evaluation. (This reminds me of the
delayed finalization of CLOS classes, which ensures that superclasses
don't have to be fully defined when a defclass form is evaluated.)

I guess this way of patching up / delaying symbol interning can probably
be provided without the need to build a whole package system replacement
around it. This could even be a compatible extension of Common Lisp
(which doesn't really specify which read algorithm to use in a
read-eval-print-loop anyway, as far as I can tell).

It's confusing that you criticize the package system for something that
it is ultimately not responsible for. What you should be criticizing is
Section 2.3.4 of the HyperSpec.

[Antony]

On 2/18/2012 7:50 AM, Pascal Costanza wrote:
>
> 2) The Common Lisp package system virtually removes hygiene issues from
> its macro system. The package system gives you a good way to precisely
> specify which symbols are known outside of a macro and which are not.

This may be a bit off the thread.
I usually don't write anaphoric macros (not intentionally anyway :) )
But I have one macro in my unit test system that goes like this

(in-package :unit-test-framework-pkg)
...
(defmacro verify-error (error-class test-form &body body)
`(or (handler-case
(progn
(,@test-form)
nil)
(,error-class (e)
(progn
,@body
(incf *passed-count*)
(test-message 5 "caught expected error ~A" e)
t)))
(error "expected error of type ~A not thrown" ',error-class)))

Most places I call it without a body
Sometimes I call it with a body like

(in-package :something)
...
(verify-error pkga::error1
(some-test-code-that-will-fail)
(let* ((message (pkga::error1-message unit-test-framework-pkg::e)))
(is (equal "expected message" message)))))

Is this reasonable.

I ask because all the noise about the specialty of a anaphoric macro and
it's description in the books seem to miss out that the variables can be
safely in different packages.

Even the variable capture stuff that is explained does not usually point
this out (in the places I have read it). Or may be I missed it, cause I
had to make this up when I needed it.

-Antony

[RG]

In article <9qac48...@mid.individual.net>,

Pascal Costanza <p...@p-cos.net> wrote:

> On 18/02/2012 19:07, RG wrote:
> > In article<9qa52a...@mid.individual.net>,
> > Pascal Costanza<p...@p-cos.net> wrote:
> >
> >> On 18/02/2012 18:09, RG wrote:
> >>> In article<9q9vlf...@mid.individual.net>,
> >>> Pascal Costanza<p...@p-cos.net> wrote:
> >>>
> >>>> If you argue for dropping the package system, and replacing it with
> >>>> something else, you need to provide an answer here...
> >>>
> >>> I have NEVER argued for DROPPING the package system.
> >>
> >> So what are you arguing for?
> >
> > http://rondam.blogspot.com/2010/02/new-and-improved-lexicons-now-50-lexier.h
> > tml
>
> I see. I wasn't aware (anymore) that you're suggesting a mapping between
> your lexicons to packages. Considering what you seem to think is the
> major problem with packages, this may still be overkill.
>
> A problem in the way how you word your concerns is that you describe the
> package system as fundamentally flawed, but the package system is
> actually not the source of the problem you have. It's rather a detail of
> how the read algorithm in Common Lisp works.

I guess it depends on your definition of "fundamentally flawed." The
whole point of having symbols in the first place is that they are
automatically constructed when reading Lisp source code. You can
construct a legal Lisp program without using the reader (except to
bootstrap the process) by doing something like this:

(let ((cl-package (find-package "COMMON-LISP"))
(my-package (or (find-package "foo" (make-package "foo")))))
(list (intern "DEFUN" cl-package) (intern "my-fun" my-package)
(list (intern "arg1" my-package) (intern "arg2" my-package) ...

So yes, it is technically true that my concern is (to a large extent)
about "a detail of how the read algorithm in Common Lisp works." But it
is IMHO a rather important detail.

> What you suggest under the link above is essentially a different read
> algorithm, where symbols that cannot be found are not automatically
> interned, but rather patched up in such a way that they find their
> actual definition later on during evaluation.

No. Lexicons use the standard Lisp reader algorithm.

Lexicons are not "better packages." They are fundamentally different
from packages. They map symbols (or to be strictly technically correct,
identifiers, which in CL happen to be symbols) onto bindings. Packages
map strings onto symbols. Lexicons are first-class global lexical
environments. Packages are hash tables from strings onto symbols.
Lexicons do what they do at macroexpand/compile time. Packages (mostly)
do what they do at read time. They are as different as two
computational concepts can be. If you do not see that then you do not
understand what lexicons are.

> (This reminds me of the
> delayed finalization of CLOS classes, which ensures that superclasses
> don't have to be fully defined when a defclass form is evaluated.)

Yes, delayed binding of lexical references is very similar to many other
forms of delayed binding in Lisp, and has many of the same advantages.

> I guess this way of patching up / delaying symbol interning can probably
> be provided without the need to build a whole package system replacement
> around it. This could even be a compatible extension of Common Lisp
> (which doesn't really specify which read algorithm to use in a
> read-eval-print-loop anyway, as far as I can tell).

Lexicons ARE a compatible extension to Common Lisp, since they are
implemented in (mostly) portable Common Lisp. (That this is possible is
a sterling credit to the overall design of the language.)

> It's confusing that you criticize the package system for something that
> it is ultimately not responsible for. What you should be criticizing is
> Section 2.3.4 of the HyperSpec.

The idea of the package system being "responsible" for anything strikes
me as a little bizarre.

What I am criticizing is one aspect of Common Lisp's design. It is an
aspect that involves elements of the reader, the package system, and
indeed the whole concept of symbols and symbolic programming (though I
don't often highlight that. I have learned over the years that one must
pick one's battles.)

The real fundamental problem, as succinctly as I can state it, is this:
state in general is bad software engineering practice, to be avoided
whenever possible (it's not always possible). GLOBAL state is even
worse and it is to be even more assiduously avoided. And global state
that is mutated by the language parser is a Really Bad Idea. It leads
to real problems of the sort that sends newbies to C.L.L. with the same
questions year in and year out, and makes the Idiots Guide to CL
Packages such a perennial best seller on the page rank circuit. It also
leads to experienced Lisp programmers arguing over whether they should
be writing :FOO or #:FOO or "FOO" in their defpackage forms.

Whether that is a criticism of packages or the reader or something else
is in the eye of the beholder, and in any case quite beside the point.
The point is: there's a problem. Reasonable people can disagree over
how serious this problem is or what (if anything) should be done about
it. But that there is a problem is undeniable. (Note that "there is no
problem" and "there is no problem FOR ME" are not equivalent statements.)

rg

[Pascal Costanza]

On 18/02/2012 21:38, RG wrote:

> Whether that is a criticism of packages or the reader or something else
> is in the eye of the beholder, and in any case quite beside the point.
> The point is: there's a problem. Reasonable people can disagree over
> how serious this problem is or what (if anything) should be done about
> it. But that there is a problem is undeniable. (Note that "there is no
> problem" and "there is no problem FOR ME" are not equivalent statements.)

Yes, there is a problem. However, the concept of mapping strings to
symbols is in general more expressive, and hence superior to mapping
symbols to bindings. Any suggestion for solving the problem that favors
the latter over the former is a loser.

Lisp, and especially Common Lisp, is about making complex problems
easier to solve, not about making easy problems easy to solve. Packages
are in line with that.

[RG]

In article <jhovgs$dkf$1...@speranza.aioe.org>,

Yes. But it's not anaphoric :-) It's only anaphoric if the macro binds
something that is then referenced in the macro invocation.

An anaphoric macro invocation looks like:

(aif (some-computation) (do-something-with it))

The apparently unbound reference to IT is (presumably) bound by AIF.

This actually points out another problem with packages: symbols with the
same name conflict with each other whether or not they have any bindings
that would actually conflict.

So if you want to say:

(aif (something) (f it))

as opposed to:

(anaphoric:aif (something) (f anaphoric:it))

and you also want to define a function called IT, you can't do that
safely using packages. You can do it safely using lexicons, because
conflicts are resolved separately in each namespace.

rg

[RG]

In article <9qakho...@mid.individual.net>,

Pascal Costanza <p...@p-cos.net> wrote:

> On 18/02/2012 21:38, RG wrote:
>
> > Whether that is a criticism of packages or the reader or something else
> > is in the eye of the beholder, and in any case quite beside the point.
> > The point is: there's a problem. Reasonable people can disagree over
> > how serious this problem is or what (if anything) should be done about
> > it. But that there is a problem is undeniable. (Note that "there is no
> > problem" and "there is no problem FOR ME" are not equivalent statements.)
>
> Yes, there is a problem. However, the concept of mapping strings to
> symbols is in general more expressive,

No, it isn't. I have made no commitment about what kind of bindings are
associated with identifiers via lexicons, and indeed one of the features
of lexicons is that one need not make any such commitment because you
can create new lexicons and hence new kinds of bindings at will. So one
possible mode of deploying lexicons is where every identifier has a
single kind of binding (per lexicon), and that binding is a symbol.
That design is isomorphic to CL's current design and hence cannot
possibly be any less expressive than CL. (It can, however, be
implemented without packages!)

> and hence superior to mapping symbols to bindings.

Even if it were true that symbols and packages were more expressive than
identifiers and lexicons (they aren't, but let's pretend for the sake of
argument) this would still not follow. Expressiveness is not the only
thing that people care about in their programming languages. Other
things people care about include readability, debuggability,
changeability, and conformance with whatever they consider to be the
principles of good software engineering, and their intuitive
expectations and prior knowledge. When all that is taken into account,
it is far from clear that packages would be a win *even if* they
provided superior expressiveness (which they don't).

> Any suggestion for solving the problem that favors
> the latter over the former is a loser.

That depends on what you mean by "favors" and "loser". CL is the only
extant language with symbols and packages. Even Scheme doesn't have
symbols. (Scheme symbols aren't really symbols at all, they are just
identifiers.) For a CL programmer to say that anything that deprecates
packages is a loser is sort of like an Amish person saying that anything
that deprecates horse-drawn buggies is a loser. By the Amish quality
metric, it's true. But reasonable people can and do disagree.

> Lisp, and especially Common Lisp, is about making complex problems
> easier to solve, not about making easy problems easy to solve. Packages
> are in line with that.

Again, this depends on what you mean by "complex problem." There is
nothing that I would consider a "complex problem" today for which either
packages or lexicons provide a significant lever. This is a purely
academic issue as far as I can tell.

rg

[Pascal Costanza]

On 18/02/2012 23:35, RG wrote:
> In article<9qakho...@mid.individual.net>,
> Pascal Costanza<p...@p-cos.net> wrote:
>
>> On 18/02/2012 21:38, RG wrote:
>>
>>> Whether that is a criticism of packages or the reader or something else
>>> is in the eye of the beholder, and in any case quite beside the point.
>>> The point is: there's a problem. Reasonable people can disagree over
>>> how serious this problem is or what (if anything) should be done about
>>> it. But that there is a problem is undeniable. (Note that "there is no
>>> problem" and "there is no problem FOR ME" are not equivalent statements.)
>>
>> Yes, there is a problem. However, the concept of mapping strings to
>> symbols is in general more expressive,
>
> No, it isn't.

Yes, it is. See my posting from this afternoon.

>> and hence superior to mapping symbols to bindings.
>
> Even if it were true that symbols and packages were more expressive than
> identifiers and lexicons (they aren't, but let's pretend for the sake of
> argument) this would still not follow. Expressiveness is not the only
> thing that people care about in their programming languages. Other
> things people care about include readability, debuggability,
> changeability, and conformance with whatever they consider to be the
> principles of good software engineering, and their intuitive
> expectations and prior knowledge. When all that is taken into account,
> it is far from clear that packages would be a win *even if* they
> provided superior expressiveness (which they don't).

It's perfectly clear that packages are a win, even when taking all these
issues into account. The problems are acknowledged, but the benefits
outweigh them.

>> Any suggestion for solving the problem that favors
>> the latter over the former is a loser.
>
> That depends on what you mean by "favors" and "loser". CL is the only
> extant language with symbols and packages. Even Scheme doesn't have
> symbols. (Scheme symbols aren't really symbols at all, they are just
> identifiers.) For a CL programmer to say that anything that deprecates
> packages is a loser is sort of like an Amish person saying that anything
> that deprecates horse-drawn buggies is a loser. By the Amish quality
> metric, it's true. But reasonable people can and do disagree.

Sure, they disagree. But they are wrong.

>> Lisp, and especially Common Lisp, is about making complex problems
>> easier to solve, not about making easy problems easy to solve. Packages
>> are in line with that.
>
> Again, this depends on what you mean by "complex problem." There is
> nothing that I would consider a "complex problem" today for which either
> packages or lexicons provide a significant lever. This is a purely
> academic issue as far as I can tell.

Naming conflicts are a real issue in other languages. This is not just
an academic problem.

[Pascal Costanza]

On 18/02/2012 20:51, Antony wrote:
> On 2/18/2012 7:50 AM, Pascal Costanza wrote:
>>
>> 2) The Common Lisp package system virtually removes hygiene issues from
>> its macro system. The package system gives you a good way to precisely
>> specify which symbols are known outside of a macro and which are not.
>
> This may be a bit off the thread.
> I usually don't write anaphoric macros (not intentionally anyway :) )
> But I have one macro in my unit test system that goes like this

[snip]

> Is this reasonable?

It's better to do it like this, roughly:

(defmacro verify-error ((error-class error-var)) test-form

&body body)
`(or (handler-case
(progn
(,@test-form)
nil)

(,error-class (,error-var)

(progn
,@body
(incf *passed-count*)
(test-message 5 "caught expected error ~A" e)
t)))
(error "expected error of type ~A not thrown" ',error-class)))

Now you can say this:

(in-package :something)
...
(verify-error (pkga::error1 err)
(some-test-code-that-will-fail)
(let* ((message (pkga::error1-message err)))

(is (equal "expected message" message)))))

If you really want to go anaphoric, use a more distinctive name than
just 'e, and make it possible to replace the default name with one's own
name (by making 'error-var &optional, for example). Otherwise, you can
get into trouble when you want to nest uses of your macros.

[RG]

In article <9qaoqm...@mid.individual.net>,

Pascal Costanza <p...@p-cos.net> wrote:

> On 18/02/2012 23:35, RG wrote:
> > In article<9qakho...@mid.individual.net>,
> > Pascal Costanza<p...@p-cos.net> wrote:
> >
> >> On 18/02/2012 21:38, RG wrote:
> >>
> >>> Whether that is a criticism of packages or the reader or something else
> >>> is in the eye of the beholder, and in any case quite beside the point.
> >>> The point is: there's a problem. Reasonable people can disagree over
> >>> how serious this problem is or what (if anything) should be done about
> >>> it. But that there is a problem is undeniable. (Note that "there is no
> >>> problem" and "there is no problem FOR ME" are not equivalent statements.)
> >>
> >> Yes, there is a problem. However, the concept of mapping strings to
> >> symbols is in general more expressive,
> >
> > No, it isn't.
>
> Yes, it is. See my posting from this afternoon.

"This afternoon" is not a well defined concept on the internet.

But even if I could figure out which post you meant it wouldn't matter.
If you claim to have discovered an even prime greater than 2 it is not
necessary for me to examine the details of your argument to know that
you are wrong. I have already explained to you how lexicons can be made
isomorphic to packages, so it cannot possibly be the case that packages
are more expressive than lexicons. Anything you can do with a package,
I can do with a lexicon.

There are two things that people commonly think you need packages for,
and that is macros and slot value names. These people are wrong.
Coincidentally (or is that ironically?) the person who solved the
problem of how to do hygienic macros in Common Lisp without packages is
also named Pascal Costanza.

rg

[Kaz Kylheku]

I'm afraid that I don't have the confidence that whatever is working
in the REPL's image will still work if I suddenly do a clean rebuild of the
system from whatever I have on disk (even if I reloaded all those materials
into the REPL).

How can you be sure that something you typed in three days ago isn't what is
making some test cases work?

I do all serious work from the basic principle of having a clean build
from properly versioned sources, all the time.

[RG]

In article <201202182...@kylheku.com>,

Kaz Kylheku <k...@kylheku.com> wrote:

> On 2012-02-18, Tim Bradshaw <t...@tfeb.org> wrote:
> > On 2012-02-18 00:15:41 +0000, RG said:
> >
> >> Well, bully for you. I use the REPL. I like the REPL. I get a lot of
> >> leverage from the REPL. IMHO the REPL is one of the things that makes
> >> Lisp better than, say, Java. And I don't think I'm the only one who
> >> feels this way.
> >
> > strongly agree with that. I live in a REPL for weeks.
>
> I'm afraid that I don't have the confidence that whatever is working
> in the REPL's image will still work if I suddenly do a clean rebuild of the
> system from whatever I have on disk (even if I reloaded all those materials
> into the REPL).

Neither do I. The REPL is tremendously useful nonetheless.

> How can you be sure that something you typed in three days ago isn't what is
> making some test cases work?

You can't be sure. The REPL is tremendously useful nonetheless.

> I do all serious work from the basic principle of having a clean build
> from properly versioned sources, all the time.

Bully for you. That does not change the fact that some people find
value in the REPL and manage to make it suit their needs despite its
limitations and lack of theoretical purity.

BTW, nowadays most serious work involves interacting with systems and/or
datasets which you do not control, and managing large quantities of
state outside of the source code of your application. So unless you are
working on a (by modern standards) trivial application, you can't be
sure of anything even if you do start from a clean build before each
run. And yet, people somehow manage to build things that work.

rg

[Kaz Kylheku]

On 2012-02-19, RG <rNOS...@flownet.com> wrote:
> In article <201202182...@kylheku.com>,
> Kaz Kylheku <k...@kylheku.com> wrote:
>
>> On 2012-02-18, Tim Bradshaw <t...@tfeb.org> wrote:
>> > On 2012-02-18 00:15:41 +0000, RG said:
>> >
>> >> Well, bully for you. I use the REPL. I like the REPL. I get a lot of
>> >> leverage from the REPL. IMHO the REPL is one of the things that makes
>> >> Lisp better than, say, Java. And I don't think I'm the only one who
>> >> feels this way.
>> >
>> > strongly agree with that. I live in a REPL for weeks.
>>
>> I'm afraid that I don't have the confidence that whatever is working
>> in the REPL's image will still work if I suddenly do a clean rebuild of the
>> system from whatever I have on disk (even if I reloaded all those materials
>> into the REPL).
>
> Neither do I. The REPL is tremendously useful nonetheless.
>
>> How can you be sure that something you typed in three days ago isn't what is
>> making some test cases work?
>
> You can't be sure. The REPL is tremendously useful nonetheless.

I'm not saying the REPL isn't useful, but rather that it's still tremendously
useful even if you kill the image and start a new one once in a while.

Staying in the REPL for two weeks is mildly obsessive.

I want to regularly take stock of what I have, build the image cleanly, and
/then/ pop into the (tremendously useful) REPL and see how some things are
behaving. (Languages in which you can't do that are crippled in that regard:
a point that you made earlier about Java, etc.)

If I soil something with a package misuse, who cares. Kill it, and re-start.
I'm not living in the age of 30-minute-reboot Lisp machine.

>> I do all serious work from the basic principle of having a clean build
>> from properly versioned sources, all the time.
>
> Bully for you. That does not change the fact that some people find
> value in the REPL and manage to make it suit their needs despite its
> limitations and lack of theoretical purity.
>
> BTW, nowadays most serious work involves interacting with systems and/or
> datasets which you do not control, and managing large quantities of
> state outside of the source code of your application. So unless you are
> working on a (by modern standards) trivial application, you can't be
> sure of anything even if you do start from a clean build before each
> run. And yet, people somehow manage to build things that work.

Those that do will more often than not have a test bed stuffed with specific
test data that lets them execute repeatable test cases, and can be reset to a
known initial state before every cycle with a new load.

People manage to build a lot of stuff that doesn't work, too.

[RG]

In article <20120218...@kylheku.com>,

Kaz Kylheku <k...@kylheku.com> wrote:

> On 2012-02-19, RG <rNOS...@flownet.com> wrote:
> > In article <201202182...@kylheku.com>,
> > Kaz Kylheku <k...@kylheku.com> wrote:
> >
> >> On 2012-02-18, Tim Bradshaw <t...@tfeb.org> wrote:
> >> > On 2012-02-18 00:15:41 +0000, RG said:
> >> >
> >> >> Well, bully for you. I use the REPL. I like the REPL. I get a lot of
> >> >> leverage from the REPL. IMHO the REPL is one of the things that makes
> >> >> Lisp better than, say, Java. And I don't think I'm the only one who
> >> >> feels this way.
> >> >
> >> > strongly agree with that. I live in a REPL for weeks.
> >>
> >> I'm afraid that I don't have the confidence that whatever is working
> >> in the REPL's image will still work if I suddenly do a clean rebuild of
> >> the
> >> system from whatever I have on disk (even if I reloaded all those
> >> materials
> >> into the REPL).
> >
> > Neither do I. The REPL is tremendously useful nonetheless.
> >
> >> How can you be sure that something you typed in three days ago isn't what
> >> is
> >> making some test cases work?
> >
> > You can't be sure. The REPL is tremendously useful nonetheless.
>
> I'm not saying the REPL isn't useful

Oh? You sure fooled me:

> I generally do not not use REPLs. Even when experimenting with
> code, I put it into small files and load those.

If the REPL is useful, why don't you use it?

BTW, it's not just the REPL where package problems show up. I find
having to maintain export lists extremely annoying. (It violates the
DRY principle.) Having name collisions among symbols with no bindings
that are only used as identifiers is also annoying. None of this is
catastrophic of course, but why make your life harder than it needs to
be?

> If I soil something with a package misuse, who cares. Kill it, and re-start.
> I'm not living in the age of 30-minute-reboot Lisp machine.

You are also apparently not working on a project that takes 30 minutes
to warm up its cache of stored state. Such applications are not
uncommon nowadays. The project I'm currently working on takes about 2-3
minutes to restart on a small test dataset.

rg

[Tamas Papp]

On Sun, 19 Feb 2012 06:21:17 +0000, Kaz Kylheku wrote:

> I'm not saying the REPL isn't useful, but rather that it's still
> tremendously useful even if you kill the image and start a new one once
> in a while.
>
> Staying in the REPL for two weeks is mildly obsessive.
>
> I want to regularly take stock of what I have, build the image cleanly,
> and /then/ pop into the (tremendously useful) REPL and see how some
> things are behaving. (Languages in which you can't do that are crippled
> in that regard: a point that you made earlier about Java, etc.)
>
> If I soil something with a package misuse, who cares. Kill it, and
> re-start. I'm not living in the age of 30-minute-reboot Lisp machine.

Restarting is fairly simple and relatively costless per se, but that's
not the issue. For me the issue is reconstructing the state I had
before I killed the image, which can be relatively costly. I can of
course deal with this by saving that state externally when possible,
but dealing with that imposes an extra development burden, which I
would prefer to postpone to a later stage in a project.

Of course there are other reasons for restarting the image (for me the
most common is memory corruption error when [mis]using foreign
libraries), but the fewer of these reasons, the better. So I am
watching attempts to fix packages with interest, even though my
development style is similar to the one you described above: I prefer
to work in files and I rarely go to the REPL directly, I only use it
via SLIME.

Best,

Tamas

[Tim Bradshaw]

Kaz Kylheku <k...@kylheku.com> wrote:

>
> How can you be sure that something you typed in three days ago isn't what is
> making some test cases work?

That's easy: If I'm developing software I want to keep: I have a big
powerful computer and I have a background job which cranks up a
noninteractive Lisp, does a cold build of the system and runs all its tests
every few minutes.

(This isn't meant to sound as sarcastic as it does sound, sorry)

[RG]

In article
<1967145657351358570...@reader443.eternal-september.org>
,

My development process very rarely involves typing directly at the REPL.
Usually I have one REPL window and an editor window (sometimes a stack
of editor windows). I'll edit something in the editor and hit ENTER,
which causes the current sexpr to be sent to the listener (I use CCL).
The listener window keeps a transcript of everything I've done, and the
editor window always has the latest version of the code. When I'm
working this way, the compile-run overhead can be very close to zero,
and it is not unusual for me to do multiple code iterations in under a
minute despite the fact that I'm not a very fast typer.

When I have something working to the point where I think it's ready to
commit, I re-evaluate the whole editor window and check to make sure it
is still working. Then -- and only then -- do I fire up another Lisp
image and re-load everything. It's pretty rare for something to fail at
that point, and it's usually a macro that ended up being defined after
it's invoked, which is usually obvious and fast to fix.

I've been working this way since 1985, when CCL stood for Coral Common
Lisp instead of Clozure Common Lisp. It was then and remains to this
day by far the best development environment I have ever used, bar none.

rg

[Antony]

On 2/19/2012 9:45 AM, RG wrote:
> of editor windows). I'll edit something in the editor and hit ENTER,
> which causes the current sexpr to be sent to the listener (I use CCL).
> The listener window keeps a transcript of everything I've done, and the
> editor window always has the latest version of the code.

I use slime and have bound F6 to do the following in lisp mode
indent buffer
save buffer
slime compile load buffer
(otherwise I have F6 bound to
indent buffer
save buffer
)

So the differences are
instead of enter I press F6
instead of last s-exp it's the whole file compiled and loaded

Although I don't understand what package issues you are complaining
about, would they be less if you follow the whole file approach (sounds
simplistic, but I am asking anyway).

As you and others have said, start up is costly for a large project. At
the moment I am doing that more often than I'd like.

From a newbie POV, 'how to' do interactive development is crucial to
Lisp productivity (as it is also now a days to some extent for other
languages). But there aren't enough pointers on that, probably for good
reasons.

-Antony