But how do I finally fix package system?
budden
I'm now testing some of my improvements to package system,
but I see I didn't get rid of "trash" symbols. If I forget to import
library:fun into my :application package and refer to it,
I create a trash symbol application:fun. When I refactor my code,
I can create a dozen trash symbols with one "compile-file".
How to save oneself from that?
I have my own (implementation-dependent) method of protecting from
trash
symbols.
I write:
(setf *forbidden-symbol-names* '((#:let1 :common-lisp)
(#:in-readtable :editor-hints.named-
readtables)
(#:string-or :budden-tools)
)
)
And now, when I try to intern "STRING-OR" to :cl-user, I'll get a
error.
This is very nice tool and it helps me a lot. Unfortunately, I still
have no
portable code. Some code can be easily written in spirit of Ron
Garret's implementation of symbol-readmacros,
http://groups.google.com/group/comp.lang.lisp/msg/85f1862e8dd13070
but it will not cover all possible symbols.
And, what could I do if I have already created trash symbols?
I think some interactive "package cleaner tool" would be useful.
How could it look like?
Say, one might call
(package-cleaner :package-name)
Then, package-cleaner would look for all potentially trash symbols.
Potential trash symbol is a symbol which name equals to the name
of some symbols from other package and which is unbound, funbound,
etc.
On the second stage, one might take a decision on each of the
symbols found: unintern it, shadow or something.
Third stage is a reverse engineering of defpackage form.
Fourth stage is a production of special form which remembers user's
decision
on some symbols. E.g. form like:
(defpackage :application ...)
(non-trash-symbol-names #.x #.y #.i)
(trash-symbol-names #.let1 #.appendf)
These forms could be appended to a package.lisp of the package.
If implementation allows hooking on intern, then error can
occur while trying to create trash symbols.
I think I could implement this rather easily with wish.
Any additions, ideas, critics?
Also, I have difficulties integrating per-package package aliases
(from Tim Bradshaw's hierarchical packages) to IDEs. Things should
be Ok in a EMACS, but I have a trouble doing this for Lispworks and
this would likely fail in an Allegro. So, I can see no portable
solution
for that and it is a pity. It looks like one would better have per-
lisp-image
table of aliases and use rename-package to set them up.
Pascal J. Bourguignon
> And, what could I do if I have already created trash symbols?
> I think some interactive "package cleaner tool" would be useful.
> How could it look like?
Like this:
C/P2[261]> (defpackage :p1 (:use :cl) (:export :f))
#<PACKAGE P1>
C/P2[262]> (defun p1:f () '(p1::f p1::in p1::p1))
F
C/P2[263]> (defpackage :p3 (:use :cl))
#<PACKAGE P3>
C/P2[264]> (in-package :p3)
#<PACKAGE P3>
C/P3[265]> (defun f () '(f in p3))
F
C/P3[266]> (use-package :p1)
*** - (USE-PACKAGE (#<PACKAGE P1>) #<PACKAGE P3>): 1 name conflicts remain
Which symbol with name "F" should be accessible in #<PACKAGE P3>?
The following restarts are available:
P1 :R1 #<PACKAGE P1>
P3 :R2 #<PACKAGE P3>
ABORT :R3 ABORT
C/Break 1 P3[267]> :r1
T
C/P3[268]> (f)
(F P1::IN P1::P1)
C/P3[269]>
Oops, it already exists in all CL implementations...
> I think I could implement this rather easily with wish.
> Any additions, ideas, critics?
(defun force-use-package (package)
(loop
(handler-bind ((package-error (lambda (err) (invoke-restart (first (compute-restarts err))))))
(use-package package)
(return-from force-use-package t))))
--
__Pascal Bourguignon__ http://www.informatimago.com/
In a World without Walls and Fences,
who needs Windows and Gates?
budden
Trash symbols can be created without a error (it I forgot
to import something).
Pascal J. Bourguignon
>> Oops, it already exists in all CL implementations...
> No. It allows only to process clashes one by one.
You've not tried my force-use-package function.
> Trash symbols can be created without a error (it I forgot
> to import something).
You need a CL implementation called MrsMirza.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Litter box not here.
You must have moved it again.
I'll poop in the sink.
budden
> > No. It allows only to process clashes one by one.
>
> You've not tried my force-use-package function.
but package system errors can occur in many situations.
Do you mean you suggest a general solution to all
package-related problems? I still don't understand
how it can help. Maybe I need some time to think.
Pascal J. Bourguignon
Well, my point is that the existing package facilities are adequate,
and that you're trying to solve a non-problem.
You can define a different kind of package system if you want, but I
fail to see the point really.
Ron Garret
<d6843539-25b8-4f1f...@21g2000yqj.googlegroups.com>,
budden <budde...@mail.ru> wrote:
> Hi list!
>
> I'm now testing some of my improvements to package system,
> but I see I didn't get rid of "trash" symbols. If I forget to import
> library:fun into my :application package and refer to it,
> I create a trash symbol application:fun.
You might want to take a look at:
http://www.flownet.com/ron/lisp/lexicons.pdf
There's an implementation at:
http://www.flownet.com/ron/lisp/lexicons.lisp
but if you decide you want to use it please contact me privately. It's
not quite ready for general use.
rg
Lars Rune Nøstdal
> Hi list!
>
> I'm now testing some of my improvements to package system,
> but I see I didn't get rid of "trash" symbols. If I forget to import
> library:fun into my :application package and refer to it,
> I create a trash symbol application:fun. When I refactor my code,
> I can create a dozen trash symbols with one "compile-file".
> How to save oneself from that?
> I have my own (implementation-dependent) method of protecting from
> trash
> symbols.
>
> I write:
> (setf *forbidden-symbol-names* '((#:let1 :common-lisp)
> (#:in-readtable :editor-hints.named-
> readtables)
> (#:string-or :budden-tools)
> )
> )
>
> And now, when I try to intern "STRING-OR" to :cl-user, I'll get a
> error.
> This is very nice tool and it helps me a lot. Unfortunately, I still
> have no
> portable code. Some code can be easily written in spirit of Ron
I use :SERIAL T and export _everything_ as a last (ASDF :file entry)
operation; even all the thrash. Then at the target (depending) package
(s) I use DO-EXTERNAL-SYMBOLS (referring to the source packages) +
SHADOWING-IMPORT. It is sloppy by default (which is great), the
default makes sense 99% of the time, and _when_ I want to be explicit
about something I can and my solution propagates or is inherited to
sub-packages so I don't have to repeat conflict resolution over and
over again.
> If I forget to import library:fun into my :application package and refer
> to it, I create a trash symbol application:fun. When I refactor my code,
> I can create a dozen trash symbols with one "compile-file".
> How to save oneself from that?
Again I simply repeat the D-E-S + S-I process and things fall into
place by themselves:
http://en.wikipedia.org/wiki/Window_blind
Of course this means that the user can no longer deduce what the API
is by just looking at which symbols are external, but I do not care
about that at all. As you sort of mention; I look for symbols _bound_
to "interesting stuff".
WFM
Captain Obvious
b> (setf *forbidden-symbol-names* '((#:let1 :common-lisp)
b> (#:in-readtable :editor-hints.named-
b> readtables)
b> (#:string-or :budden-tools)
b> )
b> )
b> And now, when I try to intern "STRING-OR" to :cl-user, I'll get a
b> error.
So you make a list of symbols which you should not forget to include in a
list
of symbols?
Like, you're making a list of all mistakes you can make to avoid
making them.
Sorry, but it is just retarded.
Learn how to use existing tools before you make your own.
Most CL implementations already signal a warning if you reference
undefined function. You should read those warnings.
It is useful anyway.
b> And, what could I do if I have already created trash symbols?
b> I think some interactive "package cleaner tool" would be useful.
b> How could it look like?
After you've read warnings and fixed errors by adding missing
imports to your defpackage definition, you just evaluate defpackage,
and when your implementaiton is doing imports, it will interactively
ask you what to do with those "trash symbols", offering to replace
them with correct ones. If you're sure that defpackage is right,
you can make an automatic reaction via handler-bind to automatically
replace a trash symbol with a good one.
I doubt you can do anything better than that -- with such approach,
you do not need to specially address "trash symbols" at all, you
just need to write code or your applciation, and CL will do the rest.
b> On the second stage, one might take a decision on each of the
b> symbols found: unintern it, shadow or something.
b> Third stage is a reverse engineering of defpackage form.
b> Fourth stage is a production of special form which remembers user's
b> decision
b> on some symbols.
b> E.g. form like:
b> (defpackage :application ...)
b> (non-trash-symbol-names #.x #.y #.i)
b> (trash-symbol-names #.let1 #.appendf)
Um, so instead of just fixing your code (defpackage) you want some
interactive tool to write it for you in a very messy way? Wow, that's great
idea.
budden
I have read about your effort before. I think CL is complex enough
already. Adding new concepts would only increase its complexity.
For me, it may violate "keep it simple" principle.
What I dislike in your "lexicons"? I suppose your tool would
require a great work to integrate with IDEs. I use M-. In EMACS
to get to definition of function, variable, type of package.
This is extremely useful. Will it work with lexicons? Also, you
name your macros with ldef... But it looks like some versions
of lisp (at least, Lispworks) prefer definitions that
start from DEF forms.
budden
similar thing, but I use just an import, not a shadowing-import.
I have it packaged in a macro which is like defpackage,
but accepts additional clauses. E.g.
(defpackage-autoimport
:editor-budden-tools
(:auto-import-from
:editor :budden-tools
:budden :alexandria :cllib)
(:auto-import-first-clashing t)
(:use :cl)
(:export #:goto-xy)
)
This form will create package and import all non-clashing symbols
from packages listed in :auto-import-from clause. If some
symbols clash, warning would be issued and symbol from leftmost
package would be imported.
I hope to publish the source soon, but this does not help
if I have already messed things up.
I don't know if shadowing-import is better here. I
have to think about it. Thanks.
budden
> interactive tool to write it for you in a very messy way? Wow, that's great
> idea.
forgot
to import that package to some of packages where the symbol was used.
CL creates new symbol silently and it is hard to find where this
happend.
Undefined function warnings do not always help, consider funcall and
apply.
Captain Obvious
??>> interactive tool to write it for you in a very messy way? Wow, that's
??>> great idea.
b> Problem is related to refactoring. I moved symbol to a new package and
b> forgot
b> to import that package to some of packages where the symbol was used.
b> CL creates new symbol silently and it is hard to find where this
b> happend.
b> Undefined function warnings do not always help, consider funcall and
b> apply.
Anyway, when you refactor things you test stuff from time to time, no?
Sooner or later you'll find that you forgot something in defpackage.
Then you fix defpackage and CL will fix those trash symbols accordingly.
This way a problem with symbols is just like any other problem
with refactoring. If you'll make some logic error, you'll find about it at
some
time, fix your code and recompile, right? You do not need any fancy
tool to fix logic errors for you. Likewise, to fix symbol error you fix
defpackage which imports symbols.
Barry Margolin
<7e0bd073-6699-4814...@21g2000yqj.googlegroups.com>,
budden <budde...@mail.ru> wrote:
Forgetting to import something IS an error.
--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
Kaz Kylheku
> Hi list!
>
> I'm now testing some of my improvements to package system,
> but I see I didn't get rid of "trash" symbols. If I forget to import
> library:fun into my :application package and refer to it,
> I create a trash symbol application:fun.
This is not a trash symbol; it holds a reference. It's only a trash
symbol if the reference itself is not retained, so that the only
reference to it is from the package where it is interned. This is
``semantic garbage''.
Suppose you have defined a function which makes that bad reference;
then the symbol isn't garbage.
> When I refactor my code,
> I can create a dozen trash symbols with one "compile-file".
> How to save oneself from that?
I solved these kinds of (non-)problems in the PKG read macro. Look for
it on the Lisp paste server.
Under the scope of the PKG read syntax, you can have all local symbols
automatically anonymized.
Ron Garret
<a8f15205-dc2f-466e...@o28g2000yqh.googlegroups.com>,
budden <budde...@mail.ru> wrote:
Yeah, that's all part of the not-quite-ready-for-general-use bit.
(BTW, you should quote the relevant parts of the messages you're
replying to so that it's easier for people to know what you're talking
about.)
My plan is to some day rename all the LDEF... forms so that they shadow
the regular defining forms in the lexicons package. My goal is to be
able to take a large corpus of open-source CL code, get rid of all the
package code (which is easy because it's usually in a separate file),
load the rest of the code into the lexicons package and have it all Just
Work. I'm pretty sure it will be (mostly) possible, but at the moment
there is one dangling technical problem that I have not been able to
work out. That's at least in part because I'm not putting a lot of
effort into it. If someone were interested in using lexicons that could
motivate me to go ahead and tie up these loose ends.
rg
Pascal J. Bourguignon
When refactoring, a cross-reference could be used:
http://common-lisp.net/project/slime/doc/html/Cross_002dreference.html
Personnaly, I just use grep. If the output of grep is unwildly, it's
a sign there's a problem in the sources, so edit them until grep is as
useful as a cross reference.
budden
> with refactoring. If you'll make some logic error, you'll find about it at
> some time, fix your code and recompile, right?
very precise information about my export/import error and it does it
very quickly. CL compiles an order of magnitude slower and
information
it gives is not as preciese. So, moving library elements between
modules
is very easy with Delphi, and it is rather hard in CL. I see this as
a disadvantage of CL and looking for a ways to fix it. I need about
30 seconds to fix one namespace/export/import error in a Delphi, and
it took me up to an hour per symbol in CL, until I invented my
(unportable)
*forbidden-symbol-names*. No it takes about 5-10 minutes as I need
to rebuild my lisp image. So, Delphi is still much better.
budden
a CL system event, not a programming event.
budden
Do you mean http://paste.lisp.org/display/72068#5 ?
I tried this approach, and it does not interplay well with editor.
Completion breaks,
so I'd prefer to import as much as I can, and use prefixes if I can't.
I think
I'll also introduce my own short (image-global) package nicknames, but
this is
not done yet. Per-package package nicknames are implemented, but they
can break
editor too. And, as you remember we now have symbol-readmacros, which
were implemented very gracefully by someone here (Ron Garrett?), so
no
need to use magical characters anymore. Something like "With-package"
would go well.
> This is not a trash symbol; it holds a reference.
I mean "trash" in a sence of "something unwanted", not as a "garbage"
in
terms of garbage collection. Sorry, even Russian sometimes can't
understand
my Russian, it is hard to expect that English-speaking people would
understand
my English easily :)
budden
I'd like to split to half dozen of smaller ones. So, it'd be nice to
draw common cross-reference graph somehow and then split it.
But, I think something like Lars's approach would work well for the
beginning, and then one could carefully remove unwanted exports.
*forbidden-symbol-names* would help there. I'll publish code for it
as soon as I'll make it more portable. BTW, it can be easily
introduced into your reader. The only thing one need is an intern
hook. Your reader allows that, but SBCL, ECL and CLISP native readers
do not.
Pascal J. Bourguignon
Eventually, you'll be able to program in Delphi with a Common Lisp
implementation. I'm not sure there's any gain in that. Do you have
any problem with Delphi? Why don't you just keep using it?
In Common Lisp, we don't usually import or export symbols.
We just use packages.
If two packages export symbols with the same name, then you have to
choose which symbol to import when you use both packages in the same
(third) package. You can do it interactively by choosing the right
restart when you use USE-PACKAGE interactively, or you can declare it
with the :SHADOWING-IMPORT-FROM option of DEFPACKAGE.
If you move a colliding symbol from one package to another, then you
just have to update the :SHADOWING-IMPORT-FROM option.
I don't know how Delphi manages its packages. Modula-2 does it
differently too, and at first I too tried to bend Common Lisp to the
Modula-2 way. But this is futile. If you want to program in Common
Lisp, then learn the Common Lisp way. Do not try to implement Delphi
in CL.
budden
> export symbols. We just use packages.
wrote me that he prefers to export symbols
one by one.
> If two packages export symbols with the same name, then
> you have to
> choose which symbol to import when you use both packages in the same
> (third) package. You can do it interactively by choosing the right
> restart when you use USE-PACKAGE interactively, or you can declare it
> with the :SHADOWING-IMPORT-FROM option of DEFPACKAGE.
Yes, I know it, but it is boring. I'm using cl-utilities, alexandria
and cllib. Clashes are multiple, and it is rather hard to eliminate
either library at the moment.
> I don't know how Delphi manages its packages.
Very simply. It looks like "use-package" and "shadowing-import" which
is done automatically. First symbol found in a used packages
(according
to their order in :use clause) is shadowing-imported. Important, that
Delphi restarts at each compilation and no trash symbols are created.
Also important that Delphi does not allow interning of arbitrary
symbols.
(not (equalp code data)) in Delphi, so when Delphi expects a
function or variable, there should already exist function or
variable with that name. If not, Delphi stops and shows the place.
CL issues a warning, I still have to trace back to find a place where
the symbol was interned. Then I have to remove the symbol, then I have
to add it to export clause. You see, I have much more work to do.
> If you want to program in Common Lisp, then learn the Common Lisp
> way. Do not try to implement Delphi in CL.
I'm not trying to do so. But it is obvious that package system is
a great obstacle to my productivity when I code in CL, compared
to Delphi. Partly it is inevitable due to interactive nature
of CL, but I'd like to make it as easy as possible.
Captain Obvious
??>> with refactoring. If you'll make some logic error, you'll find about
??>> it at some time, fix your code and recompile, right?
b> Most of my time I work not with CL, but with Delphi.
So it is a reason not to learn how to do stuff in CL?
b> Delphi shows me very precise information about my export/import error
b> and it does it very quickly. CL compiles an order of magnitude slower
Have you tried implementations besides SBCL? They say other ones
are faster.
Then, you do not need to compile everything.
Then, you can make an image with all the libs which you do not change,
and compile only your app, which presumable takes much less time.
b> a disadvantage of CL and looking for a ways to fix it. I need about
b> 30 seconds to fix one namespace/export/import error in a Delphi, and
b> it took me up to an hour per symbol in CL, until I invented my
b> (unportable)
b> *forbidden-symbol-names*.
What? Hour per symbol???
Definitely you're doing it wrong. Even if you restart you lisp, does it take
hour
to load stuff?
If you follow procedure I've described, you can fit in 30 seconds.
As soon as you've found you forgot import, add it to defpackage
and reload files.
budden
everything, with exception of one-run, throwaway machine-generated
code
sometimes.
> Then, you can make an image with all the libs which you do not change,
> and compile only your app, which presumable takes much less time.
Unfortunately, my "core" library is the place which I refactor.
Reloading everything takes 2-4 minutes as there are many interaction
with database, etc. Many of my definitions are macro calls which
are displayed as "top-level form 40" or something like this.
It is hard to find a source of problem even if you know that
you have warning at top-level form 40.
> What? Hour per symbol???
Maybe not a hour, but it is very painful compared to Delphi.
> Definitely you're doing it wrong. Even if you restart you lisp, does it take
> hour to load stuff?
No, but I'm unable to find all references to "trash" symbol from the
first
effort. E.g., asdf introduces many mess to error messages, it is
really painful
to find any error in files loaded with asdf. Maybe I'm really doing
something wrong.
> If you follow procedure I've described, you can fit in 30 seconds.
> As soon as you've found you forgot import, add it to defpackage
> and reload files.
There may be many situations: when I split my initial :cl-user
package to an :cl-user, :library and :application, I should
1. Move definition to a new file.
2. Compile it
3. Export it from "library"
4. Import it to an "application"
5. Make sure all my application's packages refer to
that library.
When refactoring, I prefer to move several symbols at a time.
Also, I prefer to split it to several libraries. So, things
get much more complicated. Maybe I should review my procedures.
Anyway, *forbidden-symbol-names* help a lot as I need no more
browse all messages to find first reference to undefined symbol.
Also I think some functions should be helpful:
1. find-all-symbols "NAME" - list all interned symbols with
that name and packages where they are accessible.
2. symbol-status (from SLIME) - which helps to make difference
between states of accessible symbol: if it is "domestic", imported or
comes from use-package. Find-symbol does not provide this information
in a complete way.
3. remove-symbol - make sure symbol is removed from all packages
or remains only as internal symbol in its home package.
These functions are simple to implement, but they add rather
useful bits of information which are inconvinient to get from
package system API directly.
Captain Obvious
??>> and compile only your app, which presumable takes much less time.
b> Unfortunately, my "core" library is the place which I refactor.
You do not refactor cl-utilities and alexandria, do you?
??>> Definitely you're doing it wrong. Even if you restart you lisp, does
??>> it take hour to load stuff?
b> No, but I'm unable to find all references to "trash" symbol from the
b> first
b> effort.
You do not need to find all references, you just need to import it
appropriately.
That is if your problem is with importing.
If you have wrong name, grep will do.
b>Maybe I should review my procedures.
Definitely you should. I can say that my procedures are not terribly
effective when I'm lazy and believe that stuff will just work.
But instead of wasting hours on it, it is better to think about how
to do it.
I believe solution won't be complex and won't require new complex
utils.
b> Also I think some functions should be helpful:
b> 1. find-all-symbols "NAME" - list all interned symbols with
b> that name and packages where they are accessible.
Hmm, have you seen APROPOS function? I think it does just that.
You should not assume that CL authors are stupid -- a lot of
functionality is already there, you just need to learn about it.
Also available in SLIME, btw.
b> 2. symbol-status (from SLIME) - which helps to make difference
b> between states of accessible symbol: if it is "domestic", imported or
b> comes from use-package.
C-c C-d d - describe symbol in SLIME. It says what package symbol
comes from. Usually it is enough.
b> 3. remove-symbol - make sure symbol is removed from all packages
b> or remains only as internal symbol in its home package.
Why would you want to do this in runtime? Fix your defpackage definitions.
budden
process of fixing this.
> You do not need to find all references, you just need to import it
> appropriately.
This is not that simple. When I split libraries, I first have to
resolve dependencies between parts of my library. In a Delphi,
compiler helps a lot. There may be only acyclyc references
between "exported" items, but may be also cyclic references
between "internal" items. This is not the case in CL. Well,
now I see where the root of problem is: due to dynamic
nature of Lisp, I can have code running while not all the libraries
are loaded. This is the principal difference between CL and Delphi.
And I would say this is more likely to be advantage of CL. But
this causes some difficulties too. E.g., asdf systems can have
only acyclic dependencies as they are loaded in some sequence, not
simultaneously. So, it turns out inexpectedly that Delphi's
module system is initially more flexible than that of CL.
So, it is harder to resolve references between
modules in CL. My symbol-related problems is only the secondary
manifestation of the problem.
Now, I think we could stop discussing that for a while. I have
to think a bit.
> But instead of wasting hours on it, it is better to think about how
> to do it.
In fact, I have already solved that with *forbidden-symbol-names*.
This is rather easy and general approach. But while I'm promoting
this,
you say my issue is non-issue :)
> b> Also I think some functions should be helpful:
> b> 1. find-all-symbols "NAME" - list all interned symbols with
> b> that name and packages where they are accessible.
>
> Hmm, have you seen APROPOS function? I think it does just that.
No it does not. (apropos "pr")
> b> 3. remove-symbol - make sure symbol is removed from all packages
> b> or remains only as internal symbol in its home package.
>
> Why would you want to do this in runtime? Fix your defpackage definitions.
Just fixing defpackage definitions might cause new conflicts when I
try
to resolve them. I prefer to fix them in advance - this is just more
convinient. Also, I want to avoid reloading of all my staff after I
have
fixed one error. E.g., *forbidden-symbol-names* breaks my load process
with
cerror. I can fix what I need, change and reload defpackages and then
continue loading.
Captain Obvious
b>>> or remains only as internal symbol in its home package.
??>> Why would you want to do this in runtime? Fix your defpackage
??>> definitions.
b> Just fixing defpackage definitions might cause new conflicts when I
b> try
b> to resolve them.
How exactly? You just replace "trash" symbols with imported ones.
b> Also, I want to avoid reloading of all my staff after I have
b> fixed one error.
You do not need to reload all your stuff -- just reload stuff in a package.
ASDF should handle this if you've wrote correct asd files. It takes only
few seconds to reload a couple of files.
Give it a try, really.
b> E.g., *forbidden-symbol-names* breaks my load process with
b> cerror. I can fix what I need, change and reload defpackages and then
b> continue loading.
But you need to populate that list manually, no? I think that defeats
the purpose -- instead of fixing it once, you need to do it twice.
budden
I have
(defpackage :a)
(defpackage :b (:use :a))
there is a::fun (fbound) and b::fun (trash).
Then I change a:
(defpackage :a (:export :fun))
When I try eval this definition I have name clash
in b. This is the simplest case. More complex case
is when I evaluate asdf:oos. THis is necessary when,
say, a::fun is indeed a macro. So, asdf shows me
"error between functions", not a cerror. So, I can't
fix situation interactively. And I have manually remove b::fun.
Or something like this. Anyway, all this is too complex.
I prefer:
1. Clean up what is wrong.
2. Change defpackage
3. Re-run them
4. Be sure all is Ok.
> ASDF should handle this if you've wrote correct
> asd files. It takes only
> few seconds to reload a couple of files.
I try to do it, of course, but my systems are
rather heavy and it takes about a minute or two to load biggest
of them.
> But you need to populate that list manually, no? I think that defeats
> the purpose -- instead of fixing it once, you need to do it twice.
Yes, first of all I have something hardly decodable messages from
asdf.
I'm grepping compiler warnings and see there is something wrong with
some symbol. I remove trash symbol, put it on *forbidden-symbols-
list*
and then reload everything. Loading breaks when the symbol is about
to
be interned. It is a bit easier to find a error at this point.
I fix the situation (maybe interactively) and either continue or
restart loading. Two or three iterations suffice usually.
Pascal J. Bourguignon
>
>> �b> Also I think some functions should be helpful:
>> �b> 1. find-all-symbols "NAME" - list all interned symbols with
>> �b> that name and packages where they are accessible.
>>
>> Hmm, have you seen APROPOS function? I think it does just that.
> No it does not. (apropos "pr")
So in what way doesn't APROPOS return here all the symbols whose name
is case insensibly "pr" and indicating in what package they are?
C/USER[31]> (setf custom:*apropos-matcher* (lambda (pattern) (lambda (symbol-name) (string-equal pattern symbol-name))))
#<FUNCTION :LAMBDA (PATTERN) (LAMBDA (SYMBOL-NAME) (STRING-EQUAL PATTERN SYMBOL-NAME))>
C/USER[32]> (apropos "pr")
C/USER[33]> (apropos "print")
PRINT function
:PRINT constant
C/USER[34]>
if that's what you mean...
Or if you want the home package instead of the package where they're
accessible from:
C/USER[37]> (map nil (lambda (s) (format t "~A:~A~%" (package-name (symbol-package s)) (symbol-name s))) (apropos-list "print"))
COMMON-LISP:PRINT
KEYWORD:PRINT
NIL
C/USER[38]>
or whatever.
budden
I am not interested in a "print" at all. That's
why apropos does not fit the goal.
Pillsy
Try FIND-ALL-SYMBOLS.
Cheers,
Pillsy
Pascal J. Bourguignon
>> How exactly? You just replace "trash" symbols with imported ones.
> But trash symbol already exists. E.g.
> I have
> (defpackage :a)
> (defpackage :b (:use :a))
>
> there is a::fun (fbound) and b::fun (trash).
>
> Then I change a:
> (defpackage :a (:export :fun))
> When I try eval this definition I have name clash
> in b. This is the simplest case. More complex case
> is when I evaluate asdf:oos. THis is necessary when,
> say, a::fun is indeed a macro. So, asdf shows me
> "error between functions", not a cerror. So, I can't
> fix situation interactively. And I have manually remove b::fun.
> Or something like this. Anyway, all this is too complex.
> I prefer:
> 1. Clean up what is wrong.
> 2. Change defpackage
> 3. Re-run them
> 4. Be sure all is Ok.
Right. Perhaps you didn't notice CL:DELETE-PACKAGE.
When you reload changed packages, you could delete them first, so that
you don't have name clashes with old symbols.
Or you can just reboot a new lisp image before reloading your changed
packages.
Pascal J. Bourguignon
budden <budde...@mail.ru> writes:
Again:
C/USER[44]> (setf custom:*apropos-matcher* (lambda (pattern) (lambda (symbol-name) (string-equal pattern symbol-name))))
#<FUNCTION :LAMBDA (PATTERN) (LAMBDA (SYMBOL-NAME) (STRING-EQUAL PATTERN SYMBOL-NAME))>
C/USER[45]> (apropos "pr")
C/USER[46]> 'pr
PR
C/USER[47]> (apropos "pr")
PR
C/USER[48]>
budden
Oops, rather strange how I have overlooked it :) Thanks.
Sometimes it is desired to list all accessible symbols,
sometimes only symbols at their home packages. E.g.,
I might want that a:fun from my example would be listed
once or twice. E.g.
((:a :a:fun :domestic :external)
(:b a:fun :inherited))
And I have written such a function already.
budden
Sometimes it helps, sometimes not.
1. Package I want to delete might be used by other
package already.
2. It can have important data bound to its symbols.
Pascal J. Bourguignon
>> CL:DELETE-PACKAGE
> Sometimes it helps, sometimes not.
> 1. Package I want to delete might be used by other
> package already.
Then obviously delete them first! If you don't, you WILL get name clashes!
> 2. It can have important data bound to its symbols.
Sometime between 50 and 70 years ago, a smart researcher invented
something called "file" IIRC. That might be useful here.
Captain Obvious
b> But trash symbol already exists. E.g.
b> I have
b> (defpackage :a)
b> (defpackage :b (:use :a))
b> there is a::fun (fbound) and b::fun (trash).
b> Then I change a:
b> (defpackage :a (:export :fun))
b> When I try eval this definition I have name clash
b> in b.
And you can resolve this clash with three keystrokes
in SLIME -- 0, 1, <enter>.
b> This is the simplest case. More complex case is when I evaluate
b> asdf:oos.
It is not that hard to evalutate defpackage you've just edited.
b> THis is necessary when, say, a::fun is indeed a macro.
It is always necessasy. Code will still hold a reference to b::fun
until you reload code.
b> So, asdf shows me
b> "error between functions", not a cerror.
Have you tried it?
I've tried and defpackage works under asdf just
like it works interactively -- it breaks into debugger
and you can use RESOLVE-CONFLICT restart.
Compiler handles (and aborts compilation) only compiler-errors.
defpackage does not generate compiler-error, so it goes
to debugger.
And note that it is behaviour of a compiler, not asdf.
If you have something different, probably you've broken
it somehow, but I don't see how.
b> So, I can't fix situation interactively. And I have manually remove
b> b::fun. Or something like this. Anyway, all this is too complex.
b> I prefer:
b> 1. Clean up what is wrong.
b> 2. Change defpackage
b> 3. Re-run them
b> 4. Be sure all is Ok.
1. Change defpackage
2. Re-run them
3. Clean up
Seems easier, no?
b> Yes, first of all I have something hardly decodable messages from
b> asdf.
Error messages have nothing do do with ASDF, it is SBCL's compiler.
b> I'm grepping compiler warnings and see there is something wrong with
b> some symbol. I remove trash symbol, put it on *forbidden-symbols-
b> list*
b> and then reload everything. Loading breaks when the symbol is about
b> to
b> be interned. It is a bit easier to find a error at this point.
b> I fix the situation (maybe interactively) and either continue or
b> restart loading. Two or three iterations suffice usually.
Wow, so you use forbidden-symbols to find where error is?
Have you tried compiling in SLIME?
E.g. you find what file error is in, compile it via SLIME and SLIME
shows you a list of errors/warnings, and if you press enter on one of them,
it jumps to location.
budden
> in SLIME -- 0, 1, <enter>.
definition, then
I need (generally) to reload everything again to make sure that
imports are Ok now. E.g. I might need to import the symbol
to some another package too. Or maybe I have duplicated
symbol definition by an omission and now I have two non-trash,
equivalent definitions of my function/data in different packages.
> It is not that hard to evalutate defpackage you've just edited.
It is not always easy as, generally, it requires reloading all my
code.
> It is always necessasy. Code will still hold a reference to b::fun
> until you reload code.
Generally speaking, no. It depends on CL implementation, and optimize
declarations. It is one of the strongest advantages of CL that very
often
(in most cases) it allows to recompile just one function instead of
recompiling all dependencies.
> b> So, asdf shows me
> b> "error between functions", not a cerror.
>
> Have you tried it?
In my previous effort I have developed a macro to automatically
import all unclashing symbols and I use it rather intensively.
Maybe this is my fault that I don't signal erros manually in a
way which would make sure I break into debugger.
> 1. Change defpackage
> 2. Re-run them
> 3. Clean up
>
> Seems easier, no?
This is not always possible and I have already given examples of
that.
> Error messages have nothing do do with ASDF, it is SBCL's compiler.
But ASDF wraps them and this is not very convinient to manage them.
> Wow, so you use forbidden-symbols to find where error is?
I just use them to recover error at earlier stage. E.g. there is
a::foo function, and b::bar macro which refers to foo in its
macroexpansion. I forgot to export a::foo from a and b uses a. Or,
I forgot to import a::foo to b. So, when I compile
definition of b::bar, I have no warning from SBCL.
Trash b::foo is interned silently. I have a warning only when I
compile some function which uses bar. Then I have to use
introspection or grep to find where trash b::foo was interned.
Instead of that, I put "FOO" to forbidden-symbol-names and
get a reader error at the place I'm trying to intern (trash) b::foo.
First of all, it is an early diagnostics and I save time and
effort to find the reference. Second, I avoid creation of trash symbol
so there's no need to delete it. Yes, I could do all that manually
(look at warnings carefully, find bar definition manually,etc), but
computers were invented to make our labour easier, werent they?
Tim Bradshaw
Bourguignon) said:
> Well, my point is that the existing package facilities are adequate,
> and that you're trying to solve a non-problem.
I would like to vote for this to be projected by ENORMOUS LASERS onto
the moon at all times please. The amount of time people have spent
trying to "fix" the package system could, at reasonable contract rates,
have funded this (if not a manned mission to Mars).
Yes, the package system has its deficiencies, but so does everything.
If you think you need to fix it before writing your program, you almost
certainly are trying to avoid facing the awful truth that you have
nothing useful you want to write. Better not to waste time fixing it
but face that truth instead: you will be better for it.
budden
> If you think you need to fix it before writing your program, you almost
> certainly are trying to avoid facing the awful truth that you have
> nothing useful you want to write. Better not to waste time fixing it
> but face that truth instead: you will be better for it.
were working on condiut packages? I'm using CL on a daily
basis for my work and currenty I have about 500k of
library and application code in lisp and lisp-based DSLs.
Captain Obvious
??>> in SLIME -- 0, 1, <enter>.
b> No, I can't. I need to resolve it, then I need to fix package
b> definition,
You should press this keystrokes TO resolve clashes
AFTER you have fixed defpackage AFTER you have find
what the problem is.
I'll remind you that it was in context of discussion that you
need some functions to resolve clashes. No, you do not,
SBCL developers have already made it for you. Just evaluate
defpackage and press three goddamn keystrokes. I can't believe
that your manual resolution process is faster than that -- if you're
going to call some function, you need more keystrokes.
I'm sure you can resolve any "trash symbols" this way -- that is,
when you've forgor to import something.
If you have some general fuck up, perhaps you can't resolve
it this way, but you can't resolve general fuckup in general
way by definition.
b> E.g. I might need to import the symbol
b> to some another package too.
We were discussing very specific part -- fixing one "trash symbol".
Of course you need test your software more after that, but you will
need this in any case, so it is out of scope.
??>> It is not that hard to evalutate defpackage you've just edited.
b> It is not always easy as, generally, it requires reloading all my
b> code.
Ok, somehow it is not a problem for me. If you have dependencies
set up correctly in ASDF then simply calling (asdf:oos 'asdf:load-op
:whatever)
will do, and it takes only some seconds.
??>> It is always necessasy. Code will still hold a reference to b::fun
??>> until you reload code.
b> Generally speaking, no. It depends on CL implementation, and optimize
b> declarations. It is one of the strongest advantages of CL that very
b> often
b> (in most cases) it allows to recompile just one function instead of
b> recompiling all dependencies.
Did I say you need to recompile all dependencies?
??>> Error messages have nothing do do with ASDF, it is SBCL's compiler.
b> But ASDF wraps them and this is not very convinient to manage them.
ASDF does not do anything with errors at all.
Check source code here: http://common-lisp.net/project/asdf/asdf.lisp
You can see only handful of handler-case and handler-binds, and they are
not relevant to discussion -- they deal with internal stuff. So no way it
can
wrap anything.
SBCL compiler deals with errors. A relevant chapter from documentation:
http://www.sbcl.org/manual/Errors-During-Macroexpansion.html
----
The compiler handles errors that happen during macroexpansion, turning them
into compiler errors.
If you want to debug the error (to debug a macro), you can set
*break-on-signals* to error.
----
SImple, eh? Just set it goddamn variable, and it will break just in the
place where you've
got an error, so you can see backtrace etc.
??>> Wow, so you use forbidden-symbols to find where error is?
b> I just use them to recover error at earlier stage. E.g. there is
b> a::foo function, and b::bar macro which refers to foo in its
b> macroexpansion. I forgot to export a::foo from a and b uses a. Or,
b> I forgot to import a::foo to b. So, when I compile
b> definition of b::bar, I have no warning from SBCL.
b> Trash b::foo is interned silently. I have a warning only when I
b> compile some function which uses bar. Then I have to use
b> introspection or grep to find where trash b::foo was interned.
Well, yeah, just press M-. and it will show you where does it
come from.
b> Instead of that, I put "FOO" to forbidden-symbol-names and
b> get a reader error at the place I'm trying to intern (trash) b::foo.
Holy shit. So instead of using introspection or grep and fixing
error in place, you choose to
1) edit some code
2) and recompile whole thing
just to find where error is?
It looks like you're not in hurry...
I take it it is due to your Delphi experience you see recompiling
as the only way.
b> First of all, it is an early diagnostics and I save time and
b> effort to find the reference.
Save time??? How exactly editing something and recompiling
is faster than pressing M-.?
b> Second, I avoid creation of trash symbol so there's no need to delete
b> it. Yes, I could do all that manually (look at warnings carefully, find
b> bar definition manually,etc), but computers were invented to make our
b> labour easier, werent they?
I've explained above -- you only fix your code, SBCL automatically
fixes live packages for you, making your labour easier.
budden
> You should press this keystrokes TO resolve clashes
> AFTER you have fixed defpackage AFTER you have find
> what the problem is.
Again, this depends on situation. I prefer to fix error first, and
only then I fix defpackage form as this approach proved to be more
general.
> No, you do not, SBCL developers have already
> made it for you. Just evaluate defpackage and
> press three goddamn keystrokes.
There are other implementations beyond SBCL. And
even in SBCL, it depends on situation. This is not
that easy even with the single trash symbol - if
the package which it is located in is used by
some other package.
> If you have some general fuck up, perhaps you can't resolve
> it this way, but you can't resolve general fuckup in general
> way by definition.
True :)
> We were discussing very specific part -- fixing one
"trash symbol".
In this case, maybe your approach works well.
> b> (in most cases) it allows to recompile just one function instead of
> b> recompiling all dependencies.
>
> Did I say you need to recompile all dependencies?
I erred, I mean reload all.
> ASDF does not do anything with errors at all.
I see ASDF erred on ASDF::OP... or smth like this.
Then I need to do some keystokes to inspect real
condition object.
> If you want to debug the error (to debug a macro),
> you can set *break-on-signals* to error.
Thanks, that might be useful.
> Well, yeah, just press M-. and it will show you
> where does it come from.
Do you mean it would show where the symbol was interned?
As far as I remember, it shows only where the function/
variable/class of that symbol is defined. If we have
(defun foo () (trash-symbol))
We won't get to this point with M-., or am I wrong?
Sorry, I'm not at SLIME prompt now, but I was always
sure SLIME wont' get us to intern point.
> I take it it is due to your Delphi experience
> you see recompiling as the only way.
No :)
> I've explained above -- you only fix your code, SBCL automatically
> fixes live packages for you, making your labour easier.
Well, thank you for you desire to help. I think due to that help
I found the (evident) difference between CL and Delphi which
relates to circular references. When expressed in CL terms,
Delphi compile-load-run loop is like this:
1. Evaluate all defpackage forms. Create all types,
all variables (initialized with garbage) and all functions
(with empty bodies). This corresponds to interface part
compilation.
2. Evaluate all defun forms. This corresponds to
implementation part compilation, linking and
loading of an application into a memory. During
this stage, no function in any application can
be called (no macros and eval-when), and no new
symbol can be exported to either package. Reader
can never access internal symbols of any package
but *package*.
3. Initialize all variables with their values and
do any bootstrapping. This corresponds to
unit's begin..end, or initialization clause.
4. Run entry point function.
What do we see?
1. Lisp is more flexible as Delphi's model can be
easily emulated with lisp's one as far as we can
forbid some operations but not vice versa.
Namely, we should be able to forbid references
to unimported symbols and interning of symbols
at some period of time.
2. Trash symbol problem is impossible with Delphi.
This comes at a cost: the only place we can export
something is in the defpackage form and we can't
re-evaluate defpackage form.
3. *forbidden-symbol-names* is a partial
emulation of Delphi's useful limitations
in CL.
4. More limitations could be useful and
some of them are implemented as package
locks.
5. Someone have reported here that
creating namespaces in advance could be
useful in CL too and this approach was
used in some large projects. But asdf
does not support that.
budden
It looks like problems would occur only when
some code defined is being evaluated at
compile/load time. First of all, this
refers to macros. So, some kind of
pseudo-declaration form would be
useful.
E.g. in package.lisp:
(defpackage :my-package
(:export #:fun #:macro #:var))
(in-package :my-package)
(declare-macro #:macro
#:internal-macro)
(declare-compile-time-function
#:internal-fun)
and then in foo.lisp:
(eval-when (:compile-toplevel ...)
(internal-fun ...))
where
(defmacro declare-macro (&rest symbols)
(eval-when (:compile-toplevel :load-toplevel :execute)
(dolist (s symbols)
(eval
`(defmacro ,s (&rest ignore)
(error "Macro ~A is not defined yet" ,s)
))))
and
(defmacro declare-compile-time-function (&rest symbols)
(eval-when
(dolist (s symbols)
(eval
`(defun ,s (&rest ignore)
(error "Function ~A is not defined yet" ,s)
))))
Declare-compile-time-function seem to be
less useful. It only serves to reset all
functions which are called at compile time
and helps to ensure we won't call old code
at new compilation.
I didn't think about variables and generics
yet. I prefer using defparameters when
they are appropriate. Also there are
macros like defvar-resettable somewhere,
but I still didn't use them.
Downside of solutions suggested are
warnings of function/macro being redefined
in another place. Can they be inhibited?