In one library, I used *load-pathname* / *load-truename* to find
additional files to load, relative to the location where the library was
found. This worked quite well for a couple of years. Alas, ASDF seems to
have changed the meaning of these two variables (or something) - they
now refer to some automatically created cache directory, where the
resources I'm looking for are, of course, not available. This happened
to me with the new release of ECL that was released two days ago, which
includes a newer version of ASDF.
So: What's the recommended way to locate such resources? (In my
particular case, it's .lisp files in a subfolder, to be loaded and
evaluated by the library - they cannot be declared as part of the system
definition, but have to be found in that folder.)
Any hints are appreciated.
Thanks,
Pascal
--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
> Hi,
>
> In one library, I used *load-pathname* / *load-truename* to find
> additional files to load, relative to the location where the library
> was found. This worked quite well for a couple of years. Alas, ASDF
> seems to have changed the meaning of these two variables (or
> something) - they now refer to some automatically created cache
> directory, where the resources I'm looking for are, of course, not
> available. This happened to me with the new release of ECL that was
> released two days ago, which includes a newer version of ASDF.
>
> So: What's the recommended way to locate such resources? (In my
> particular case, it's .lisp files in a subfolder, to be loaded and
> evaluated by the library - they cannot be declared as part of the
> system definition, but have to be found in that folder.)
ASDF:SYSTEM-RELATIVE-PATHNAME might help.
Zach
Time to use another defsystem + logical pathnames?
>
> So: What's the recommended way to locate such resources? (In my
> particular case, it's .lisp files in a subfolder, to be loaded and
> evaluated by the library - they cannot be declared as part of the
> system definition, but have to be found in that folder.)
I think that, if ASDF copies things so that the *LOAD-x* variables
don't correspond to the original source location, then it should
provide some way of things knowing their original location, so they can
find other bits of themselves - not just other bits of code, but data
and so on. The obvious approach would be to bind some variables like
*ORIGINAL-LOAD-PATHNAME* etc.
(so I guess this is a missing feature in ASDF.)
Yes, but someone needs to write it first. ;)
In the mean time, I believe Zach gave the correct answer.
- Daniel
> Hi,
>
> In one library, I used *load-pathname* / *load-truename* to find
> additional files to load, relative to the location where the library was
> found. This worked quite well for a couple of years. Alas, ASDF seems to
> have changed the meaning of these two variables (or something) - they
> now refer to some automatically created cache directory, where the
> resources I'm looking for are, of course, not available. This happened
> to me with the new release of ECL that was released two days ago, which
> includes a newer version of ASDF.
>
> So: What's the recommended way to locate such resources? (In my
> particular case, it's .lisp files in a subfolder, to be loaded and
> evaluated by the library - they cannot be declared as part of the system
> definition, but have to be found in that folder.)
>
> Any hints are appreciated.
>
> Thanks,
> Pascal
For example,
(truename
(asdf:system-definition-pathname
(asdf:find-system :closer-mop)))
yields #P"/usr/share/common-lisp/source/closer-mop/closer-mop.asd" here.
You can then merge that with the pathname of your resource file relative
to the .asd
--
Stelian Ionescu a.k.a. fe[nl]ix
Quidquid latine dictum sit, altum videtur.
http://common-lisp.net/project/iolib
> Time to use another defsystem + logical pathnames?
>
IMHO, ASDF is good enough for delivering and using off-the-shelf libraries
but is less convenient for development than, for example, LispWorks
defsystem.
FWIW, I define the following hack to allow logical pathnames within ASDF.
;;; Allow any component accept logical pathname as the :pathname initarg
;;; CAUTION: Underscore '_' is treated differently from the "usual"
pathname.
;;; It is not allowed in the name part of a logical pathname!
(defparameter *use-logical-pathnames* t
"True if logical pathname is allowed as the :pathname option of any
component")
(defmethod reinitialize-instance :after ((component asdf:component) &key
pathname)
(when (and pathname *use-logical-pathnames*)
(setf (slot-value component 'asdf::relative-pathname)
(translate-logical-pathname pathname))))
---
Sincerely,
Dmitriy Ivanov
lisp.ystok.ru
Thanks a lot indeed, this is exactly what I need!
Best,
Maybe. Last time I checked logical pathnames, I found them hard to
understand, and I didn't quite understand what problem they solved. Is
there a good resource to read about them? Does Peter Seibel describe
them in his book?
> Last time I checked logical pathnames, I found them hard to
> understand, and I didn't quite understand what problem they solved. Is
> there a good resource to read about them? Does Peter Seibel describe
> them in his book?
I've been researching logical pathnames recently. There is no logical
pathname info in Practical Common Lisp, but CLTL2 has the best info I've
seen so far. It explains the rationale and possible use cases fairly
well.
Zach
> On 07/03/2010 17:37, jos...@corporate-world.lisp.de wrote:
> > On 7 Mrz., 12:55, Pascal Costanza<p...@p-cos.net> wrote:
> >> Hi,
> >>
> >> In one library, I used *load-pathname* / *load-truename* to find
> >> additional files to load, relative to the location where the library was
> >> found. This worked quite well for a couple of years. Alas, ASDF seems to
> >> have changed the meaning of these two variables (or something) - they
> >> now refer to some automatically created cache directory, where the
> >> resources I'm looking for are, of course, not available. This happened
> >> to me with the new release of ECL that was released two days ago, which
> >> includes a newer version of ASDF.
> >>
> >> So: What's the recommended way to locate such resources? (In my
> >> particular case, it's .lisp files in a subfolder, to be loaded and
> >> evaluated by the library - they cannot be declared as part of the system
> >> definition, but have to be found in that folder.)
> >>
> >> Any hints are appreciated.
> >
> > Time to use another defsystem + logical pathnames?
>
> Maybe. Last time I checked logical pathnames, I found them hard to
> understand, and I didn't quite understand what problem they solved. Is
> there a good resource to read about them? Does Peter Seibel describe
> them in his book?
Logical pathnames are location and operating system independent
IDs for files and directories. Similar to URNs in the web area -
but more primitive.
Let's say you have a drawing editor called DRED
and you have a logical pathname host DRED.
DRED uses a bunch of directories:
dred:src; for the sources
dred:bin; for binaries
dred:fonts; for special fonts
dred:forms; for various graphical forms
dred:forms;icons; for various icons
dred:examples; for various examples
and more...
For the simple case one loads a file that sets the logical
pathname translations. The trivial version is something like
this:
(setf (logical-pathname-translations "DRED")
`(("dred:**;*.*" ,(compute-your-pathname-here))))
Where (compute-your-pathname-here) returns something like:
"/Lisp/software/dred/**/*.*" on a Mac
or
"rj-vlm:>software>dred>**>*.*" on a Lisp Machine
or a pathname under Windows ...
On a Lisp Machine one has a registry directory for
logical hosts and systems. Whenever I first use
a logical host, it loads the corresponding
translation via the registry (the sys;site; directory,
itself a logical pathname directory).
So once that is set up, you can refer to files
of your software everywhere the same way.
dred:src;color;chooser.lisp stays the same.
Wherever you work, (ed "dred:src;color;chooser.lisp") will
edit that file. Independent where it is in the directory
or the network (if your machine can access remote files).
Now the system definition should then be done with logical pathnames:
(my-defsystem dred ()
(:serial "dred:src;package"
"dred:src:macros"
"dred:src;drawing"
...
))
The defsystem then should call LOAD and COMPILE-SYSTEM
with the logical pathnames, so that the source locations
also will be recorded with logical pathnames.
Then you can load the fasls and press M-. on a symbol
and the editor will load the source - independent of
where you are running the code.
If you have some additional lisp files to load and you
know that they are in the contrib subdirectory of DRED, then
do for example
(map nil 'load (directory "dred:contrib;*.lisp")).
Given that you have a logical pathname and a translation,
then locating any piece of your software is easy.
You don't need that when
* you don't share software with other people
* you have only one computer, one file system, one OS
* you want to compute pathnames on your own
Remaining problems:
* there are restrictions to logical pathnames on various Lisps
* the interpretation of logical pathnames can be slightly different
For the better implementations it works okay, though.
Not sure if ASDF supports this or if it makes
sense for it - I mostly don't use ASDF.
Actually, as soon as you use a character outside of [-A-Z0-9] for any
component in the logical pathnames, you lose entirely the advantages of
using them, since then their behavior becomes totally implementation
dependant.
Check "19.3.1 Syntax of Logical Pathname Namestrings":
word---one or more uppercase letters, digits, and hyphens.
> (setf (logical-pathname-translations "DRED")
> `(("dred:**;*.*" ,(compute-your-pathname-here))))
Implementation dependant. Anything can happen.
> dred:src;color;chooser.lisp stays the same.
Implementation dependant. Anything can happen.
> Wherever you work, (ed "dred:src;color;chooser.lisp") will
> edit that file.
You bet! Implementation dependant. Anything can happen.
> Now the system definition should then be done with logical pathnames:
>
> (my-defsystem dred ()
> (:serial "dred:src;package"
> "dred:src:macros"
> "dred:src;drawing"
> ...
> ))
Implementation dependant. Anything can happen.
> If you have some additional lisp files to load and you
> know that they are in the contrib subdirectory of DRED, then
> do for example
> (map nil 'load (directory "dred:contrib;*.lisp")).
Implementation dependant. Anything can happen.
I don't know what you people have against uppercase letters. They're
perfectly nice letters.
--
__Pascal Bourguignon__
> Rainer Joswig <jos...@lisp.de> writes:
> > Logical pathnames are location and operating system independent
> > IDs for files and directories. Similar to URNs in the web area -
> > but more primitive.
> >
> > Let's say you have a drawing editor called DRED
> > and you have a logical pathname host DRED.
> >
> > DRED uses a bunch of directories:
> >
> > dred:src; for the sources
> > dred:bin; for binaries
> > dred:fonts; for special fonts
> > dred:forms; for various graphical forms
> > dred:forms;icons; for various icons
> > dred:examples; for various examples
>
> Actually, as soon as you use a character outside of [-A-Z0-9] for any
> component in the logical pathnames, you lose entirely the advantages of
> using them, since then their behavior becomes totally implementation
> dependant.
>
> Check "19.3.1 Syntax of Logical Pathname Namestrings":
>
> word---one or more uppercase letters, digits, and hyphens.
19.3.1.1.7
When parsing words and wildcard-words,
lowercase letters are translated to uppercase.
All right. So logical pathnames are case insensitive. Good to know.
Thanks.
--
__Pascal Bourguignon__
> Rainer Joswig <jos...@lisp.de> writes:
>> 19.3.1.1.7
>>
>> When parsing words and wildcard-words,
>> lowercase letters are translated to uppercase.
>
> All right. So logical pathnames are case insensitive. Good to know.
> Thanks.
Last time I checked, Allegro CL gets this wrong. In fact, their
documentation on logical pathnames reeks of "this part of the standard
seemed stupid so we made it easy to make incompatible use of them." It
dismays me.
Zach
Yes, it is indeed Allegro CL behavior that made me think it was case
sensitive.
--
__Pascal Bourguignon__
OK, thanks. I will take a look at it...
Lisp Machine:
Command: "http:server;url.lisp"
"http:server;url.lisp"
Command: (pathname *)
#P"HTTP:SERVER;URL.LISP"
Command: (translate-logical-pathname *)
#P"RJNXP:>rel-8-3>sys>http>server>url.lisp"
Btw., the Lisp Machine file system usually is case
insensitive, case preserving.
Just like the default of HFS+ on the Mac.
LispWorks on my Mac:
CL-USER 228 > "http:server;url.lisp"
"http:server;url.lisp"
CL-USER 229 > (pathname *)
#P"HTTP:SERVER;URL.LISP"
CL-USER 230 > (translate-logical-pathname *)
#P"/Lisp/cl-http/cl-http-380/server/url.lisp"
Parsing the string to a logical pathname object, converts
the characters to uppercase.
Translating the logical pathname to a physical
pathname may do a lot of things. It can
convert the characters to the preferred case
for example. Lowercase, probably. It can shorten
the name for filesystems with limited name
lengths, it can shorten the extension for filesystems
with limited extension lengths, and more...
If the extension is "LISP" and the file system
only has 3 character long extensions, it may
also abbreviate it for the physical pathname
to the preferred extension "lsp" .
To me, this seems exactly the wrong way around. I don't want to set up
anything. What I do know is that my library distribution contains a
subfolder "tests" with lisp source files to be loaded and evaluated by
the library. ASDF gives me a path to the location of the system
definition, and then I can just refer to that subfolder. It seems to me
that with logical pathnames, I would have to require users to configure
where they put the library, which seems a lot less reliable to me.
In other words, I still don't quite understand what problem logical
pathnames solve. But I will read the chapter in CLtL2...
> Not sure if ASDF supports this or if it makes
> sense for it - I mostly don't use ASDF.
Which system definition facility does this better?
I think you missed the part where I mentioned that the logical
pathname can, in the primitive case, be automagically configured when
loading the software or when loading the library.
> In other words, I still don't quite understand what problem logical
> pathnames solve.
Machine, OS and host independent pathnames.
In your Lisp software you then never have to use physical
pathnames. On every machine you can do
(compile-file "dred:src;rectangle.lisp") . No need
to remember where the file actually is on this
particular machine.
What is so difficult
to understand? DOS/Windows/Mac/Unix/VMS/... all have different
pathname semantics. All have different file system layouts
(user home directories, fonts, applications, libraries, tmp
directories, help files, ...).
Logical pathname give you a independent way to specify
pathnames and a resolution mechanism to find the actual
physical pathname on the Lisp/OS/Filesystem combination
you are using.
> But I will read the chapter in CLtL2...
>
> > Not sure if ASDF supports this or if it makes
> > sense for it - I mostly don't use ASDF.
>
> Which system definition facility does this better?
Don't know. I can't compare it. I'm using platform
specific defsystems or mk-defsystem.
>
>
> Pascal
> To me, this seems exactly the wrong way around. I don't want to set up
> anything. What I do know is that my library distribution contains a
> subfolder "tests" with lisp source files to be loaded and evaluated by
> the library.
Then you should indeed only use physical pathnames.
> ASDF gives me a path to the location of the system
> definition, and then I can just refer to that subfolder. It seems to
> me that with logical pathnames, I would have to require users to
> configure where they put the library, which seems a lot less reliable
> to me.
It would actually be impossible, in general. (Unless you declare a
logical pathname translation for each of the physical pathnames).
> In other words, I still don't quite understand what problem logical
> pathnames solve. But I will read the chapter in CLtL2...
The problem it solves is:
Access to some files with portable and easy to write pathnames.
You could access some files portably using the various pathname
functions including make-pathname and merge-pathnames, but their
portable use is complex.
You could use the easy to write physical pathnames or namestrings, but
they wouldn't be portable.
If you want both, you want logical pathnames.
Note that accessing ALL the files present in a file system is hard to do
using logical pathnames. For this, you would have to define a mapping
from logical pathnames to each of the physical pathnames.
If you want to write a program that must deal with an existing file
system, then you must use physical pathnames.
On the other hand, if you want to access files portably, using
pathnames, without caring about their exact physical name, or accepting
restrictions on their physical names, then logical pathnames will do
nicely.
--
__Pascal Bourguignon__
This may be the case (pun intended :-). It is hard to get the :local
mappings of pathnames correct for the various styles of filenames
(e.g. case sensitive on Unix, and case-insensitive-preserving-original
on Mac and Windows). However:
> In fact, their
> documentation on logical pathnames reeks of "this part of the standard
> seemed stupid so we made it easy to make incompatible use of them." It
> dismays me.
Please show me such documentation - I'd like to see where you find
this attitude anywhere in our docs. There are certainly places where
we allow extensions to logical pathname parsing; for example, the spec
doesn't allow the inclusion of underscores in a logical pathname, and
so we allow an extension, if a variable is set, which allows a
namestring with an underscore in it to be parsed as a logical
pathname. But in reading that documentation I get no sense of "the
standard seemed stupid" - it does seem very restrictive to disallow
one character that occurs quite often in Unix and C names, but if we
allow users to set a variable and use that character, do you consider
that thumbing our nose at the spec?
Duane
> On Mar 13, 11:12 am, Zach Beane <x...@xach.com> wrote:
>> p...@informatimago.com (Pascal J. Bourguignon) writes:
>>
>> > Rainer Joswig <jos...@lisp.de> writes:
>> >> 19.3.1.1.7
>>
>> >> When parsing words and wildcard-words,
>> >> lowercase letters are translated to uppercase.
>>
>> > All right. So logical pathnames are case insensitive. Good to know.
>> > Thanks.
>>
>> Last time I checked, Allegro CL gets this wrong.
>
> This may be the case (pun intended :-). It is hard to get the :local
> mappings of pathnames correct for the various styles of filenames
> (e.g. case sensitive on Unix, and case-insensitive-preserving-original
> on Mac and Windows).
It seems like this is a problem only if you want some translations to be
automagic. If you are explicit with every translation, you don't have to
worry local case conventions, right?
> However:
>
>> In fact, their
>> documentation on logical pathnames reeks of "this part of the standard
>> seemed stupid so we made it easy to make incompatible use of them." It
>> dismays me.
>
> Please show me such documentation - I'd like to see where you find
> this attitude anywhere in our docs. There are certainly places where
> we allow extensions to logical pathname parsing; for example, the spec
> doesn't allow the inclusion of underscores in a logical pathname, and
> so we allow an extension, if a variable is set, which allows a
> namestring with an underscore in it to be parsed as a logical
> pathname. But in reading that documentation I get no sense of "the
> standard seemed stupid" - it does seem very restrictive to disallow
> one character that occurs quite often in Unix and C names, but if we
> allow users to set a variable and use that character, do you consider
> that thumbing our nose at the spec?
I do. It defeats the purpose of a precise and complete specification of
the syntax of logical pathnames. If a character appears in a physical
pathname that isn't allowed in a logical pathname, it seems to me that
the right reaction is to create a translation, not allow non-standard
implementation-specific syntax to creep into the pathname literals in
the source.
Many have found this restriction onerous. Because actual work is
typically done on just a few platforms, forbidding characters such as
an underscore (_), which in fact causes no problems on any popular
platform, contributes nothing to actual portability and adds
additional development rules whose purpose (to non-Lisp programmers
and to many Lisp programmers) is obscure.
In particular, I find "contributes nothing to actual portability"
extremely obnoxious, as breaking this restriction introduces real
problems with actual portability to implementations that choose to take
the spec at face value and implement it as written.
Zach
Oh, OK, we are chewing the scenery foaming at the mouth over the top
abusing the English language, sure, Franz is paving the way to hell and
damnation for mankind and the ridiculously long red light at the corner
of Ocean and Rte 35 is the end of civilization as we know it.
Damn, it's fun talking this way. Al Sharpton move over!
hth, hk
What if one is distributing an application and ASDF is long gone (I'd
rather say "dead and buried")? lpns then are a lispy config file.
>
> In other words, I still don't quite understand what problem logical
> pathnames solve. But I will read the chapter in CLtL2...
ISTR the ability to specify multiple possibilities was quite handy for
running different ways without touching the source. On VMS with the more
powerful logical names one specified a list of places to look for X, so
a developer (or other experimenter) could put a substitute file early in
the chain and have that be in effect for a run. I do not recall lpns
going that far, but with entire directories yes.
kt
>
>> Not sure if ASDF supports this or if it makes
>> sense for it - I mostly don't use ASDF.
>
> Which system definition facility does this better?
>
I like a list of compile-file statements over ASDF.
kt
The "automagic" part is precisely what should be intended in the
spirit of logical pathnames. Otherwise, why would you need logical
pathnames over just pathnames? You could just name all of your
pathnames using regular pathnames, and do all of the translations to
account for operating system differences programmatically.
> > However:
>
> >> In fact, their
> >> documentation on logical pathnames reeks of "this part of the standard
> >> seemed stupid so we made it easy to make incompatible use of them." It
> >> dismays me.
>
> > Please show me such documentation - I'd like to see where you find
> > this attitude anywhere in our docs. There are certainly places where
> > we allow extensions to logical pathname parsing; for example, the spec
> > doesn't allow the inclusion of underscores in a logical pathname, and
> > so we allow an extension, if a variable is set, which allows a
> > namestring with an underscore in it to be parsed as a logical
> > pathname. But in reading that documentation I get no sense of "the
> > standard seemed stupid" - it does seem very restrictive to disallow
> > one character that occurs quite often in Unix and C names, but if we
> > allow users to set a variable and use that character, do you consider
> > that thumbing our nose at the spec?
>
> I do. It defeats the purpose of a precise and complete specification of
> the syntax of logical pathnames. If a character appears in a physical
> pathname that isn't allowed in a logical pathname, it seems to me that
> the right reaction is to create a translation, not allow non-standard
> implementation-specific syntax to creep into the pathname literals in
> the source.
It may seem that way to you, but many of our customers that use
logical pathnames don't like that restriction at all, and do indeed
use such a feature, to avoid having to mess with such translations.
> Many have found this restriction onerous. Because actual work is
> typically done on just a few platforms, forbidding characters such as
> an underscore (_), which in fact causes no problems on any popular
> platform, contributes nothing to actual portability and adds
> additional development rules whose purpose (to non-Lisp programmers
> and to many Lisp programmers) is obscure.
>
> In particular, I find "contributes nothing to actual portability"
> extremely obnoxious, as breaking this restriction introduces real
> problems with actual portability to implementations that choose to take
> the spec at face value and implement it as written.
Why should that be the case? If you want to be portable between CL
implementations, don't use underscores, and don't turn on the feature
that allows the nonportable code. It is exactly like any other
extension to a CL implementation. If your argument is that a user who
takes advantage of this extension is tied to one lisp, then that's the
same kind of decision that a user must make about any other such
extension. If you think the extension is a crock (and that's
perfectly OK) then don't use it.
Duane
> On Mar 13, 6:15 pm, Zach Beane <x...@xach.com> wrote:
>> Duane Rettig <du...@franz.com> writes:
>> > On Mar 13, 11:12 am, Zach Beane <x...@xach.com> wrote:
>> >> p...@informatimago.com (Pascal J. Bourguignon) writes:
>>
>> >> > Rainer Joswig <jos...@lisp.de> writes:
>> >> >> 19.3.1.1.7
>>
>> >> >> When parsing words and wildcard-words,
>> >> >> lowercase letters are translated to uppercase.
>>
>> >> > All right. So logical pathnames are case insensitive. Good to know.
>> >> > Thanks.
>>
>> >> Last time I checked, Allegro CL gets this wrong.
>>
>> > This may be the case (pun intended :-). It is hard to get the :local
>> > mappings of pathnames correct for the various styles of filenames
>> > (e.g. case sensitive on Unix, and case-insensitive-preserving-original
>> > on Mac and Windows).
>>
>> It seems like this is a problem only if you want some translations to be
>> automagic. If you are explicit with every translation, you don't have to
>> worry local case conventions, right?
>
> The "automagic" part is precisely what should be intended in the
> spirit of logical pathnames. Otherwise, why would you need logical
> pathnames over just pathnames?
My impression is that they're to provide an application with a (ahem)
logical virtual filesystem that is independent of the physical pathnames
actually involved, and the primary benefit is to be able to rearrange or
override the physical layout by changing the translations rather than
the pathname-using code.
> You could just name all of your pathnames using regular pathnames, and
> do all of the translations to account for operating system differences
> programmatically.
You also use logical pathnames and generate translations
programmatically.
Because people use features specific to Allegro LPNs without realizing
it.
> If you want to be portable between CL implementations, don't use
> underscores, and don't turn on the feature that allows the nonportable
> code.
That would work if I was the original author.
> It is exactly like any other extension to a CL implementation.
It would be a little like Allegro's CL:STRING-CAPITALIZE returning
"Don't" when passed "don't", and having someone rely on that reasonable
behavior without realizing it's contrary to the specification.
> If your argument is that a user who takes advantage of this extension
> is tied to one lisp, then that's the same kind of decision that a user
> must make about any other such extension. If you think the extension
> is a crock (and that's perfectly OK) then don't use it.
People use it without realizing the portability issue.
Zach
>>> In particular, I find "contributes nothing to actual portability"
>>> extremely obnoxious, as breaking this restriction introduces real
>>> problems with actual portability to implementations that choose to take
>>> the spec at face value and implement it as written.
>>
>> Why should that be the case?
>
> Because people use features specific to Allegro LPNs without realizing
> it.
On rereading the documentation, I see I mixed up the mixed-case issue
with the additional-allowed-characters issue, and I can see now someone
would be quite unlikely to use forbidden-by-the-standard characters
accidentally. And the mixed case issue can be easily solved with
exhaustive translations. Sorry for the noise.
Zach