Interference from LispWorks packages re sql-odbc and cl-http
Jeff
in two different contexts. I do not know the proper way to resolve the
issue.
First, when compiling Paul Meurer's sql-odbc, I got "Defining
(function/macro) XX visible in package YY" errors in package SQL (and
later ODBC). I noticed LW loading its SQL patches prior to the error.
I worked around the problem by modifying sql-odbc to use package PMSQL
instead of SQL.
Second, when compiling CL-HTTP, I got
.
; MINIPROC:WITHOUT-SCHEDULING
**++++ Error between functions:
Defining macro WITHOUT-SCHEDULING visible from package CLIM-SYS.
.
Without-scheduling is defined in LW's CLIM-SYS (visible only after
(require "CLIM")). After egreps, princ's, etc, the problem does not
seem to me to be a conflict within cl-http itself. I had not issued a
(require "CLIM") prior to (load "lw/start") in cl-http. My workaround
here is to (setf *handle-warn-on-redefinition* nil) prior to (load
"lw/start"). This gets cl-http to a (seemingly) functional state.
Why are functions/macros visible from packages I have not required? LW
Professional does not even include a license for SQL (or
KnowledgeWorks or Prolog), yet the patches for it load. (I get a
licensing dialog if I try to (require "sql"), so SQL is apparently
present.) How do I block LW's "native" packages when I'm trying to use
my own packages of the same name? Is my cl-http workaround correct? It
seems to me that it is not because the LW CLIM stuff must be hanging
around "polluting" the space prior to compiling cl-http.
Finally, after the cl-http workaround, I get the following
compile-time messages (one of each type shown here):
Warning: Using DEFPACKAGE to add to #<PACKAGE HTML2>.
;;;*** Warning in (SUBFUNCTION HTTP::EXECUTE-FORM
HTTP::PARSE-LOCATION-HEADER): URL::NO-SCHEME-FOUND is not a valid
type.
Warning: (METHOD PARSE-SHTML-TEMPLATE (T)) defined more than once in
c:\lisp\projects\cl-http-70-23\server\shtml.lisp.
;;;*** Warning in HTTP::IMAGE-LINE: Syntactic warning for form
HTTP::*IMAGE-LINE-URL*:
HTTP::*IMAGE-LINE-URL* assumed special.
Is this normal for cl-http? Ignore the warnings? Or should I clean
them up and submit my newbie changes? (ha!)
Sebastien Saint-Sevin
>In LispWorks for Windows 4.2.7 Professional I have run into a problem
>in two different contexts. I do not know the proper way to resolve the
>issue.
>
>First, when compiling Paul Meurer's sql-odbc, I got "Defining
>(function/macro) XX visible in package YY" errors in package SQL (and
>later ODBC). I noticed LW loading its SQL patches prior to the error.
>I worked around the problem by modifying sql-odbc to use package PMSQL
>instead of SQL.
>
Regarding your first point:
Add the following line to your siteinit.lisp file :
(setf lw:*handle-warn-on-redefinition* :warn)
and have a look at section "5.1.10.1 Controlling definition in system
packages" in the release notes book.
Sebastien.
Arthur Lemmens
Jeff wrote:
> First, when compiling Paul Meurer's sql-odbc, I got "Defining
> (function/macro) XX visible in package YY" errors in package SQL (and
> later ODBC).
Yes, I noticed the same thing with SQL-ODBC and with other stuff that
redefines symbols in one of Lispworks' predefined packages. (This only
happens in Lispworks 4.2, by the way. Version 4.1 gave no warnings.)
I solved this with:
(setq hcl:*packages-for-warn-on-redefinition*
(remove-if (lambda (package-name)
(member package-name '("SQL"
;; Add other package names here.
)
:test #'string-equal))
hcl:*packages-for-warn-on-redefinition*))
This should solve some of your CL-HTTP problems as well (if you add
"CLIM-SYS" to the list of packages above).
I think this is a better solution than your trick of just setting
*HANDLE-WARN-ON-REDEFINITION* to nil (that would also eliminate warnings
you _do_ want).
Arthur Lemmens
Tim Bradshaw
> Yes, I noticed the same thing with SQL-ODBC and with other stuff that
> redefines symbols in one of Lispworks' predefined packages. (This only
> happens in Lispworks 4.2, by the way. Version 4.1 gave no warnings.)
> I solved this with:
> [...]
> I think this is a better solution than your trick of just setting
> *HANDLE-WARN-ON-REDEFINITION* to nil (that would also eliminate warnings
> you _do_ want).
It would probably be a better solution still if these systems were
written such that they didn't go smashing the definitions of symbols
in implementation-defined packages. There can be reasons for doing
this (I do it occasionally to work around implementation bugs), but
not often. Such redefinitions should almost certainly be extracted
into implementation-dependent parts of the system, since they are by
definition implementation-dependent.
I *suspect* that what is going on here is that various completely
unintentional redefinitions are happening because some package is
using some other package which happens to export some symbol. This is
something that needs to be worked around by not using those
packages...
--tim
Arthur Lemmens
Tim Bradshaw wrote:
> It would probably be a better solution still if these systems were
> written such that they didn't go smashing the definitions of symbols
> in implementation-defined packages.
Normally, yes. But Paul Meurer's SQL-ODBC is an independent implementation
of (part of) Lispworks' CommonSQL library. In that case, I think it's
quite reasonable to use the same package names as CommonSQL does. The
other case where this happened to me, is in my own partial CLIM
implementation. Here, too, the package names are defined by the
specification and I need the possibility to redefine anything I want
in those packages. In both cases, I'm not interested in whatever is
made available by the implementation; I explicitly want to use my own
versions.
> I *suspect* that what is going on here is that various completely
> unintentional redefinitions are happening because some package is
> using some other package which happens to export some symbol.
I don't think that's the case for SQL-ODBC; the redefinitions there are
completely intentional (don't know about CL-HTTP, though; never used it).
Arthur Lemmens
Tim Bradshaw
> Normally, yes. But Paul Meurer's SQL-ODBC is an independent implementation
> of (part of) Lispworks' CommonSQL library. In that case, I think it's
> quite reasonable to use the same package names as CommonSQL does. The
> other case where this happened to me, is in my own partial CLIM
> implementation. Here, too, the package names are defined by the
> specification and I need the possibility to redefine anything I want
> in those packages. In both cases, I'm not interested in whatever is
> made available by the implementation; I explicitly want to use my own
> versions.
Well, I think in the case that you're choosing deliberately clashing
package names, then you (the system that is choosing these names)
should be doing whatever it takes to cope - if you detect that you're
being loaded on top of a system which already has those packages, and
they've been defined by someone other than you, then you need to deal
with whatever stuff you have to do to make those packages go away, or
make them safe for redefinitions, or whatever. Don't leave it up to
the user - it's not their problem and they shouldn't have to know.
But I think the only cases where it's reasonable to choose the same
package names as an existing are where you are emulating it, or
emulating a strict subset of it. If you are doing something which is
not a subset (including the whole thing as a subset), then, if you've
chosen the same names, there's no way for a system that relies on your
system and a system that relies on the other system to coexist in the
same image.
Even then, it's such a tiny amount of work for a client to specify
which package it wants to use, that it seems hardly worth the possible
pain.
Partly, of course, this is all an artifact of the lack of a package
registry and the superficially unstructured names of packages. But I
think it turns out that the DNS is an adequate package registry - if
you name your packages <domain-prefix>.<private-part> then you're
always safe - so our packages get called COM.CLEY.<something>, and I'm
fairly sure that no-one else is going to choose names which clash with
us. Of course, all this is nicked from Java. If you use Franz-style
relative package names (which work for at least Allegro, CMUCL and
LispWorks with some small hacks) you can even avoid having to change
source files in some cases. I find that domain-prefixed names are
enough though.
--tim
Jeff Caldwell
people's replies:
Tim Bradshaw wrote:
>I *suspect* that what is going on here is that various completely
>unintentional redefinitions are happening because some package is
>using some other package which happens to export some symbol.
After a fresh load of LispWorks, (find-package 'sql) => nil, so the
package itself is not loaded. Perhaps the LispWorks image contains a
list of package names that, when seen, are assumed 'native' and
automatically loaded. What is the best way, or code available, to walk
through the symbol tree in an image and display all the symbols/lists/etc?
Arthur Lemmons wrote:
>(setq hcl:*packages-for-warn-on-redefinition*
> (remove-if #'(lambda (package-name)
>
(member package-name '("SQL"
> "ODBC"
>
"CLIM-SYS"
>
)
>
:test #'string-equal))
>
hcl:*packages-for-warn-on-redefinition*))
That works just fine for cl-http. I haven't had time to test sql but I
am certain the same principle applies.
Sebastien Saint-Sevin wrote:
>Add the following line to your siteinit.lisp file :
>(setf lw:*handle-warn-on-redefinition* :warn)
This works as well.
Tim Bradshaw wrote:
> * Arthur Lemmens wrote:
>>Normally, yes. But Paul Meurer's SQL-ODBC is an independent implementation
>>of (part of) Lispworks' CommonSQL library. In that case, I think it's
>>quite reasonable to use the same package names as CommonSQL does.
...
> Well, I think in the case that you're choosing deliberately clashing
> package names, then you (the system that is choosing these names)
> should be doing whatever it takes to cope...
...
> I think it turns out that the DNS is an adequate package registry - if
> you name your packages <domain-prefix>.<private-part> then you're
> always safe - so our packages get called COM.CLEY.<something>...
I wonder why I don't have pretty much complete control over packages
that are initially loaded in an image. Particularly, it makes sense to
have control over a package such as SQL, for which it is clear in a
given application whether the package is needed or not. From a bemused
standpoint, there is mild irony that I'm getting name-space collisions
from a package for which I did not pay.
I really like the DNS-style package naming convention.
Finally, thanks to a person who emailed me privately pointing me to
ftp://ftp.ai.mit.edu/pub/cl-http/devo/sources-70-156a-pre.tar.gz
instead of 70.23, the release version which I have been using. He
assures me the developer's release is "actually quite stable" and that
it contains his correction to the problem I experienced. I am
downloading but have not yet tried that version.
Arthur Lemmens
Jeff Caldwell wrote:
> After a fresh load of LispWorks, (find-package 'sql) => nil, so the
> package itself is not loaded. Perhaps the LispWorks image contains a
> list of package names that, when seen, are assumed 'native' and
> automatically loaded.
I suppose that packages like SQL and CLIM are only defined after you
execute something like (REQUIRE "CLIM"). I agree with you that it's
confusing that Lispworks signals a warning or error when you try to
define stuff in Lispworks' 'own' packages when those packages don't
even exist.
> What is the best way, or code available, to walk through the symbol
> tree in an image and display all the symbols/lists/etc?
To display all symbols, you can try something like:
(loop for package in (list-all-packages)
do (loop for symbol being the present-symbol of package
do (format t "~A~%" symbol)))
I don't think there's a way to walk through all lists (and you probably
wouldn't want to, unless you were a garbage collector).
> there is mild irony that I'm getting name-space collisions from a
> package for which I did not pay.
Indeed ;-)
Arthur Lemmens
(I'm sending a cc to the Lispworks mailing list: lisp...@xanalys.com).
sv0f
<alem...@xs4all.nl> wrote:
>Jeff Caldwell wrote:
>> What is the best way, or code available, to walk through the symbol
>> tree in an image and display all the symbols/lists/etc?
>
>To display all symbols, you can try something like:
>
> (loop for package in (list-all-packages)
> do (loop for symbol being the present-symbol of package
> do (format t "~A~%" symbol)))
See also the DO-SYMBOLS macro. You'll still need an
outer loop that iterates over all packages, however.
Fred Gilham
> See also the DO-SYMBOLS macro. You'll still need an
> outer loop that iterates over all packages, however.
The naive old-fashioned way to do it:
(dolist (package (list-all-packages))
(do-symbols (symbol package)
(print symbol)))
--
Fred Gilham gil...@csl.sri.com
And then [Clinton] turned to Hunter Thompson, of all people, and said
with wholehearted fervor, "We're going to put one hundred thousand new
police officers on the street."
I was up all night persuading Hunter that this was not a personal
threat. -- P. J. O'Rourke
Erik Naggum
| The naive old-fashioned way to do it:
|
| (dolist (package (list-all-packages))
| (do-symbols (symbol package)
| (print symbol)))
Is there something about `do-all-symbols´ that I have missed?
--
Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
Now showing on CNN: Harry Potter and the Search for Weapons of Mass Destruction
Jeff Caldwell
This post confirms final resolution of the subject problem.
(from xanalys.com's mail list)
...
> The bug here is that the LispWorks patch loader was
> called when a user
> module was REQUIREd. Thanks for reporting it, we
> have fixed it for the
> next release.
My workaround to avoid the patches getting loaded, while continuing to
use the package names as defined in the sql-odbc-09 release, is to
rename the patch directories. Starting with:
c:\Program Files\Xanalys\LispWorks\lib\4-2-0-0\patches\
I renamed the sub-directories sql (to sql-rename) and odbc (to
odbc-rename). I then do not experience the problem.
...
>> If I rename
>> the packages LSQL and LODBC, the patches are not
> > loaded nor do I get the MATCH error.
>
> I assume you mean here that you renamed the modules
> (thus avoiding a
> name clash which triggered the patch loader), rather
> than renaming
> packages.
I believe you saying you do not think I used (rename-package ...). That
is correct, I did not. What I did was modify the sql-odbc-09 source
code, prior to compile or load. I changed e.g. (defpackage "ODBC" ... to
(defpackage "LODBC" ... That also avoided the bug (as does the
directory-rename workaround presented above). Obviously, the
directory-rename workaround is more portable so far as the use of the
sql-odbc-09 module is concerned.
I would like to thank everyone who helped me with this problem. In
particular, Arthur Lemmens responded to several private emails and sent
working code and code examples.