Scott Schwartz <"schwartz+@usenet "@bio.cse.psu.edu> writes:
> Right, so why not have a package, which is standard, but only > available on a subset of systems you support (the POSIX ones). Right
Because that's not what "standard" means.
> now your Common Lisp code has lots of conditionals to handle > implementation differences, so that's obviously not a problem. The > improvement I'm proposing is simply that you will have standard names > for, and access to, things that exist on 90% of the computers (99% of > the users!) where Common Lisp does run, so you have an easier time and > fewer implementation specific conditionals.
It would be a worthy project but perhaps not a terribly worthwhile project. As compared to, say, a decent standard for extensible streams, or a meta-object protocol, or multithreading[*], saving the user four minutes per supported implementation to look up a function name is not really compelling.
Daniel Barlow <d...@telent.net> writes: > > now your Common Lisp code has lots of conditionals to handle > > implementation differences, so that's obviously not a problem. The > > improvement I'm proposing is simply that you will have standard names > > for, and access to, things that exist on 90% of the computers (99% of > > the users!) where Common Lisp does run, so you have an easier time and > > fewer implementation specific conditionals.
> It would be a worthy project but perhaps not a terribly worthwhile > project. As compared to, say, a decent standard for extensible > streams, or a meta-object protocol, or multithreading[*], ...
How worthwhile it is depends on what type of problems you are trying to solve. For the problems I want to solve (which involve writing various types of servers for POSIX based systems), having "standard" POSIX bindings would be a help (i.e. I wouldn't have to keep writing my own). On the other hand none of the items in your list would be of any help to me. However, I don't dismiss them as not being terribly worthwhile because of this.
Erik Naggum <e...@naggum.no> writes: > * Scott Schwartz > | It's just that I'm willing to settle for 90% portable.
> In that case, Common Lisp is not known to comply.
> | I don't understand why all the unix lisp implementors cannot agree on > | what to name the lisp interface to the standard function for running > | another program on unix.
> Neither can I.
> | While you are happy to nit-pick this into oblivion, perl and python > | have posix packages and get the job done with no fuss at all.
> What truly amazes me is that you do not understand that Perl and > Python are /only/ implemented on POSIX systems and have no > specification at all.
I don't know about Python, but I'm reasonably confident that Perl runs on non POSIX systems. And, as Scott seems to fail to realise, using the POSIX module is usually the counsel of someone who doesn't realise that there's a Better Way To Do It available either in the core libraries or on CPAN. The POSIX interfaces are painfully C-like, and the module itself is bloody enormous.
It's also the case that, in general, Perl appears to be moving away from having system dependent functions as part of the core language, instead we're seeing a steady exodus of such functionality into support modules. It's already happened with the DBM interface, which got moved into the Tie::* system some time ago, it's in the process of happening with the Socket related functions, and from what Larry has said in his design docs for Perl, that trend will only continue.
-- Piers
"It is a truth universally acknowledged that a language in possession of a rich syntax must be in need of a rewrite." -- Jane Austen?
* Scott Schwartz <schwartz+@usenet @bio.cse.psu.edu>:
> Does all that sound bitter? I'd really like to be able to use lisp (any > flavor) to do some real programming, but I've never seen an implementation > that suits my needs. Most don't even come close.
Which implementations, which can't use the `port' package from clocc, do you have in mind? The last time I looked clocc was working with ACL, clisp, cmucl and gcl. I'd be surprised to find that it was difficult to add support for other Common Lisp implementations.
,---- | davep@hagbard:~$ clocc | [1]> (require :port-shell #p"clocc:src;port;shell") | ;; Loading file /usr/local/src/clocc/src/port/shell.fas ... | ;; Loading of file /usr/local/src/clocc/src/port/shell.fas is finished. | t | [2]> (port:run-prog "ls") | DbaMGR.zip Nautilus archive develop etc lib nng_list.txt temp | Mail News bin docs html music spool work | 0 | [3]> `----
> Somehow their goals are totally at odds with those of a typical unix > programmer. (Siskind's Stalin is the right idea, but it's too much of a > research prototype, and anyway that's Scheme, which I know you hate.)
The above solution has worked for me more than once. What sort of "typical programmer" does that make me?
-- Dave Pearson: | lbdb.el - LBDB interface. http://www.davep.org/ | sawfish.el - Sawfish mode. Emacs: | uptimes.el - Record emacs uptimes. http://www.davep.org/emacs/ | quickurl.el - Recall lists of URLs.
> Chris Beggy <chr...@kippona.com> writes: > > Consider:
> > (run-program "<your perl script>")
> > run-program's syntax is dependent on your implementation.
> Why can't you just say > (POSIX::system "perl program")
> Doesn't lisp have standard POSIX bindings? How could that be > possible?
The predominant platform that runs on most of the planet's computers has no POSIX entry points, and most of the programmers targetting that platform don't give a damn about it.
POSIX is just another language that needs to be targetted at the underlying host platform. Some of those targettings are quite suboptimal. (Does Cygwin have copy-on-write fork() yet, for instance?) A good POSIX implementation requires a POSIX-friendly kernel. This is not even true of some Unix-like operating systems; for example, the threading model in the Linux kernel causes problems for an implementation of POSIX threads.
Much of POSIX is useful only to people writing in C. For example printf() is POSIX (by way of inheritance from ANSI C). You have FORMAT in Lisp, so you don't need that stupidity. You have garbage collection, so malloc() is of no use. Many interfaces are idiotic and so require thick wrapping, like any place where you have to specify a fixed-size buffer for a string (e.g. readlink). Writing a binding for a function like readlink() requires you to write a whole bunch of code to get intelligent behavior that just returns a string. E.g. I use this implementation, and target that instead of targetting readlink directly:
for (;;) { int result = readlink(path, str, size); if (result == -1) goto bail; if (result < size) { str[result] = 0; break; } if (size * 2 < size) goto bail; size *= 2; if ((temp = realloc(str, size)) == 0) goto bail; str = temp; }
return str;
bail: free(str); return 0;
}
Quite likely, no symbolic link will need 256 bytes of space, but you never know. So this resizing logic is required to do it right every time.
All decent Lisp implementations have a way to glue to C. But you have already expressed your view elsethread that foreign function interfaces ``demote'' the outside functions and are resource-intensive.
I think that the foreign function languages offered by Lisp implementations are more convenient than what implementations of other languages offer. They leverage Lisp macros to create powerful and useful mini-languages in which you can declare any C interface, and even annotate it with semantics, like direction of data transfer of pointer parameters, and memory management responsibility. When targeting the above impl_readlink in CLISP, I can tell it that the returned data came from malloc, so it automatically takes care of freeing it. No programming is required other than annotating the interface.
There can be various overheads in FFI calls, depending on implementation. So what; use the interface intelligently, then. For instance, if you are wrapping a graphics library written in C, don't call put_pixel(x,y) a million times from Lisp to create an image. Expose something less primitive that operates on entire raster lines or image regions, and call that.
* Kaz Kylheku | Quite likely, no symbolic link will need 256 bytes of space, but you | never know. So this resizing logic is required to do it right every time.
There is a system limit on the maximal length of symbolic links. You are supposed to query the system for this value and allocate that much space. Given the inimitable programmer-friendliness of C and Posix, the idea of supplying a function that simply returned an allocated string would be impossible.
-- Erik Naggum, Oslo, Norway
Act from reason, and failure makes you rethink and study harder. Act from faith, and failure makes you blame someone and push harder.
> Matthew Danish <mdan...@andrew.cmu.edu> writes: > > And what is wrong with FFI's? They work, and are pretty easy to use for > > most tasks.
> My objection is to the name "foreign".
That is irrational. Do you also object to ``negative number'' on the grounds that it discourages subtraction of a larger quantity from a smaller one? A FFI does what it does, regardless of how you spell it.
> Some people want lisp to be a > universal shell that hides you from the real system. They call the > real system's library functions "foreign", because they want to demote > and discourage them.
That's crock. It takes someone a lot of work to develop a good FFI; why would they discourage its use?
Maybe you live in a place where ``foreign'' is a dirty word or something?
As in, ``those goddamned foreign functions are taking away jobs that should be done in the Lisp image''. ;)
> So, a really good FFI is fine. Just call it a NFI, and encourage it's > use. Then when someone asks "Using Lisp to Call another program in
Note that the term ``native'' has connotations of supremacy of the C compilers's calling conventions. It's very inappropriate in the context of a Lisp system that already generates native code. There is a connotation behind the term ``native'' that an inefficient interpreted system is gaining access to the processor (as in ``native'' code versus interpreted code). The term foreign captures the idea that there are different calling conventions and differences in data representation. The two realms are mutually foreign.
> linux?", you just say, "Use the NFI to call the standard system() > routine, just like in perl."
This would be poor advice on two counts. Firstly, the Lisp you are using may provide a way to run external programs as an extension, so you'd be expending unnecessary effort to set up the FFI. Secondly, the system() function calls the command interpreter, as in /bin/sh -c "your command here". There can be, firstly, security issues there. Secondly, you need to shell-escape the command argument list that you want to run, so that the interpreter won't break tokens on your whitespace, or interpret meta-characters. A call like (run-program "vi" "<file>"), if implemented over system(), has to translate the string to, for instance, "vi '<file>'" to protect the < > characters from serving as redirection operators. For running external programs, you really want a combination of fork(), exec*() and wait*(). That extension for running programs may already provide a straightforward interface to these three.
Wrong. To get useful POSIX, there is the Interix product which Microsoft owns now. They probably bought it to bury it, because it commoditizes their OS.
> Having a posix package, whereever possible, would not limit Common > Lisp. It would make it much more useful, though.
Even C doesn't have a POSIX package; the ANSI C standard does not mention POSIX anywhere. ISO C++ doesn't mention POSIX anywhere. Implementations of these languages have easy access to platform functions, because those functions are defined using C headers.
Everyone else is a second-class citizen who must target the C. Lisp is no worse off than any other non-C, non-C++ language. Lisp is better off because of the ability to write sub-languages to do the job, so you don't have to leave the language.
I predict that over the next few years, there will emerge a common FFI syntax for all Lisps, and it will be de-facto standard (standard meaning working with many implementations, not in the way you use the word to denote the features of one-implementation-wonder languages like perl). There is the UFFI project, the obvious candidate. It will be easier to use C functions from Lisp than from C++, because there will be header file parsing that will not barf on C++ keywords and incompatible namespace rules.
By the way, what POSIX are you talking about anyway? POSIX is a moving target, being bloated all the time. Do you mean 1990, or 200x POSIX or something in between?
k...@ashi.footprints.net (Kaz Kylheku) writes: > POSIX is just another language that needs to be targetted at the > underlying host platform. Some of those targettings are quite > suboptimal. (Does Cygwin have copy-on-write fork() yet, for instance?) > A good POSIX implementation requires a POSIX-friendly kernel. This is > not even true of some Unix-like operating systems; for example, the > threading model in the Linux kernel causes problems for an > implementation of POSIX threads.
I agree that not all platforms support the POSIX semantics. However, that isn't any reason not to provide an interface for the subset that they do provide.
> Much of POSIX is useful only to people writing in C.
Perhaps so, but there is also much that can be used from any language. The last two program I wrote used the following POSIX/Unix functions: accept, bind, close, connect, dup2, execlp, fcntl, getsockopt, gettimeofday, getuid, inet_aton, ioctl, kill, listen, open, pipe, poll, read, recv, send, seteuid, setsockopt, sigaction, socket, wait, and write. I used these all using the "foreign function interface" of the particular implementation I chose. I accept having to do this as the cost of doing business, but it would reduce the cost if they were already available in some (semi-)standard library.
k...@ashi.footprints.net (Kaz Kylheku) writes: > Everyone else is a second-class citizen who must target the C. Lisp is > no worse off than any other non-C, non-C++ language. Lisp is better > off because of the ability to write sub-languages to do the job, so > you don't have to leave the language.
There is a non-C, non-C++ language with multiple implementations that does have a standard Posix interface: SML. The details are at http://cm.bell-labs.com/cm/cs/what/smlnj/doc/basis/pages/posix-chapte... Granted there aren't that many SML implementations and the spec. doesn't cover the whole of POSIX but IMHO it is a step in the right direction.
k...@ashi.footprints.net (Kaz Kylheku) writes: > Wrong. To get useful POSIX, there is the Interix product which > Microsoft owns now. They probably bought it to bury it, because it > commoditizes their OS.
The old Interix product is now part of Microsoft's "Windows Services for UNIX" product.
k...@ashi.footprints.net (Kaz Kylheku) writes: > ... the ANSI C standard does not > mention POSIX anywhere.
of course it does. even the old one:
ansi x3.159-1989, rationale at pp. 87: "the comittee anticipates that IEEE 1003 will wish to standardize the kill function in the POSIX specification." :)
> ... POSIX is a moving > target, being bloated all the time. Do you mean 1990, or 200x POSIX or > something in between?
professionals who have any interest in working with posix refer to the recent open group standard IEEE Std 1003.1-2001 (ISBN 0-7381-3010-9). anything else is history. if you search around, you may still be able to find a pdf copy of the open group final draft. if not, order the CD set.
oz --- given enough bugs, all eyeballs are shallow.
> * Kaz Kylheku > | Quite likely, no symbolic link will need 256 bytes of space, but you > | never know. So this resizing logic is required to do it right every time.
> There is a system limit on the maximal length of symbolic links. You are > supposed to query the system for this value and allocate that much space.
I've encountered Unix boxes where the calls to find these maxima all returned 2**31, which was quite a bit beyond what malloc() was prepared to believe. I, too, ended up with a few grow-the-buffer-and-retry loops.
Scott Schwartz <"schwartz+@usenet "@bio.cse.psu.edu> writes:
> My objection is to the name "foreign". Some people want lisp to be a > universal shell that hides you from the real system. They call the > real system's library functions "foreign", because they want to demote > and discourage them. In contrast Java implementors call the same > functions "native", because they *are*, and getting access to them is > necessary and desirable.
How about `aboriginal', `primeval', or `autochthonous'?
Joe Marshall <j...@ccs.neu.edu> writes: > Scott Schwartz <"schwartz+@usenet "@bio.cse.psu.edu> writes:
> > My objection is to the name "foreign". Some people want lisp to be a > > universal shell that hides you from the real system. They call the > > real system's library functions "foreign", because they want to demote > > and discourage them. In contrast Java implementors call the same > > functions "native", because they *are*, and getting access to them is > > necessary and desirable.
> How about `aboriginal', `primeval', or `autochthonous'?
Joe Marshall <j...@ccs.neu.edu> writes: > Scott Schwartz <"schwartz+@usenet "@bio.cse.psu.edu> writes:
>> My objection is to the name "foreign". Some people want lisp to be a >> universal shell that hides you from the real system. They call the >> real system's library functions "foreign", because they want to demote >> and discourage them. In contrast Java implementors call the same >> functions "native", because they *are*, and getting access to them is >> necessary and desirable.
> How about `aboriginal', `primeval', or `autochthonous'?
Splendid! I'll go with autochthonous, since I'm feeling Jungian today.
In article <8gk7jk3hau....@galapagos.cse.psu.edu>, Scott Schwartz <"schwartz+@usenet "@bio.cse.psu.edu> wrote:
>My objection is to the name "foreign". Some people want lisp to be a >universal shell that hides you from the real system. They call the >real system's library functions "foreign", because they want to demote >and discourage them. In contrast Java implementors call the same >functions "native", because they *are*, and getting access to them is >necessary and desirable.
The analogy that I think inspired the term is "foreign language". If you normally speak English, French is a foreign language, and vice versa. No superiority is implied, they're just different.
So a FFI is a way for a program written in Lisp to call a function written in C, as it's a foreign language from its perspective.
-- Barry Margolin, bar...@genuity.net Genuity, Woburn, MA *** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups. Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Scott Schwartz <"schwartz+@usenet "@bio.cse.psu.edu> writes:
> ... lisp implementors usually demote native functions to > something you access via a "foreign function interface". That's all > very toy-like, albeit a very big and resource intensive toy.
> Does all that sound bitter? I'd really like to be able to use lisp > (any flavor) to do some real programming, but I've never seen an > implementation that suits my needs. Most don't even come close. > Somehow their goals are totally at odds with those of a typical unix > programmer. (Siskind's Stalin is the right idea, but it's too much of > a research prototype, and anyway that's Scheme, which I know you > hate.)
For those who don't like to deal with "foreign function interfaces", we just announced in this newsgroup our brand new Operating System Interface module. I think it goes a long way toward doing what you want, even though it's not specifically Posix oriented (it does gather as many common Unix system functionalities as practical, and even tries to include MS in the fold, though not all of the functionality is appropriate in Windows systems). We're also about to announce the newest version of our opensource FTP server, which uses this interface and which now contains no FFI calls.
> The analogy that I think inspired the term is "foreign language". > If you normally speak English, French is a foreign language, and > vice versa. No superiority is implied, they're just different.
> So a FFI is a way for a program written in Lisp to call a function > written in C, as it's a foreign language from its perspective.
Well, life is passing us by. My daughter takes French, and she got into the French Honor Society. There was an awards ceremony. The awards ceremony was not about foreign languages. It was about `world languages'.
Personally I think we should keep calling them foreign function calls, but if we need to change the name, call them `primitive' function calls --- and yes, I am considering denotation here. :-)
-- Fred Gilham gil...@csl.sri.com || "If I thought there was anything at all in your arguments, I should have to be not only a theist, but an Episcopalian to boot," he said, after one interchange, reckoning that since Episcopalianism was, in his book, that than which nothing could be worse, this was an effective reductio ad absurdum. - J. R. Lucas
* Scott Schwartz wrote: > My objection is to the name "foreign". Some people want lisp to be a > universal shell that hides you from the real system. They call the > real system's library functions "foreign", because they want to demote > and discourage them.
So when I, say, describe French as a `foreign language' for English speakers I am demoting and discouraging it? Right, of course!
* Scott Schwartz wrote: > Right, so why not have a package, which is standard, but only > available on a subset of systems you support (the POSIX ones). Right > now your Common Lisp code has lots of conditionals to handle > implementation differences, so that's obviously not a problem. The > improvement I'm proposing is simply that you will have standard names > for, and access to, things that exist on 90% of the computers (99% of > the users!) where Common Lisp does run, so you have an easier time and > fewer implementation specific conditionals.
Might I suggest that rather than sitting around complaining about how you want something and causing everyone to dislike you, a better approach might be to try and do something towards making it exist? I think Posix bindings would be useful since I pretty much only use Posix systems (well, I use windows as well, which is Posix in about the sense that any MS system conforms to any standard, namely not at all). However as there's a lot of work to do to make them suitable for CL: many things need to have wrappers which provide some kind of sensible error behaviour (such as signalling an error, not silently returning some `can't happen' value), many other things need to be changed to, say, handle pathnames as well as strings, and so on. It's a good lot of work, but the result would be useful - even without an implementation, a document on `suggested Posix bindings for CL' would be useful.
On 10 Nov 2002 22:40:25 -0500, Scott Schwartz <"schwartz+@usenet
"@bio.cse.psu.edu> wrote: > My objection is to the name "foreign". Some people want lisp to be a > universal shell that hides you from the real system. They call the > real system's library functions "foreign", because they want to demote > and discourage them. In contrast Java implementors call the same > functions "native", because they *are*, and getting access to them is > necessary and desirable.
Funny you should mention that. The real system's library functions may be called "foreign", but the Lisp syntax treats them as first class citizens.
Tim Bradshaw <t...@cley.com> writes: > Might I suggest that rather than sitting around complaining about how > you want something and causing everyone to dislike you, a better > approach might be to try and do something towards making it exist?
What makes you think I haven't? My name is on some of the code in scm, for example. (There used to be more, but it got subducted over the years. That's scheme not common lisp, though, so maybe it doesn't count.)
In any event, the fact that a lot of people here seem to think that it wouldn't be of value dissuades me from getting involved.
Then, too, there were a lot of messages around here lately wondering why python was more popular than lisp. Don't shoot the messenger for answering your question.
* Scott Schwartz wrote: > What makes you think I haven't? My name is on some of the code in > scm, for example. (There used to be more, but it got subducted over > the years. That's scheme not common lisp, though, so maybe it doesn't > count.)
Well, as far as I know you haven't done stuff on Posix bindings for CL, which is what I was referring to. I didn't mean to claim you hadn't done stuff on other things (which would be arrogant of me, to put it mildly, and insulting), and I'm sorry if I gave that impression.
> In any event, the fact that a lot of people here seem to think that it > wouldn't be of value dissuades me from getting involved.
I think bindings to posix systems would be of value. Great value in fact. I think it's a lot of work though, but I'd like to see it done.
> Then, too, there were a lot of messages around here lately wondering > why python was more popular than lisp. Don't shoot the messenger for > answering your question.
