general Interface CL <-> interactive Programm ?

4 views
Skip to first unread message

Norman Werner

unread,
May 10, 2003, 2:53:10 PM5/10/03
to
Hello,

I was wondering whether there exists an general Mechanism to connect
lisp to another interactive Programm (let's say: gnuplot, a shell,
another lisp-process , spice ...) in a plattform-independent way(ansi-cl).

Up to now I only got the idea to misuse the fileinterface through
fifos:

Norman@welt> cat fifo |gnuplot| backfifo &
Norman@welt> lisp
lisp> (with-open-file (send "fifo" :direction :output)
lisp> (write "plot sin(x)" :stream send))

But this seems to be rather ineffective, insecure and not-a-good-idea.

Any other suggestions?

norman
--
Liebe Leute ... in letzter Zeit erhalte ich ständig excel und word dateien .
ICH KANN DIE NICHT ÖFFNEN.
Also vorher lieber z.B. nach pdf konvertieren ...tausend Dank

Norman Werner (Zi. 307)
J. G. Nathusius ring 7 (WH6)
39106 Magdeburg
norman...@student.uni-magdeburg.de

Pascal Bourguignon

unread,
May 10, 2003, 4:19:07 PM5/10/03
to
Norman Werner <norman...@student.uni-magdeburg.de> writes:

> Hello,
>
> I was wondering whether there exists an general Mechanism to connect
> lisp to another interactive Programm (let's say: gnuplot, a shell,
> another lisp-process , spice ...) in a plattform-independent way(ansi-cl).

That's not the same thing, being platfom independent and being ANSI
(standard) Common-Lisp. So you have two answers:

- No, there is no IPC provision in the standard Common-Lisp.
(however, on unix systems, you could use a named pipe and
the standard stream I/O, or merely files to implement an IPC).

- Yes, you could implement an API on a range of implementations on
various platforms, either using the FFIs or the various socket
packages or extensions that are provided by the implementations.

Have a look at clocc/clocc/src/port/*.lisp for example.
(http://clocc.sourceforge.net).


> Up to now I only got the idea to misuse the fileinterface through
> fifos:
>
> Norman@welt> cat fifo |gnuplot| backfifo &
> Norman@welt> lisp
> lisp> (with-open-file (send "fifo" :direction :output)
> lisp> (write "plot sin(x)" :stream send))
>
> But this seems to be rather ineffective, insecure and not-a-good-idea.

It's as efficient as sockets. Only a shared memory IPC could be
slightly more efficient. It's not particularly insecure, only that it
bases its security on the file system access rights. Perhaps the
problem is that people usually lack imagination and don't know how to
set the access rights on their files and directories?


> Any other suggestions?

See above.


--
__Pascal_Bourguignon__ http://www.informatimago.com/
----------------------------------------------------------------------
Do not adjust your mind, there is a fault in reality.

Matthew Danish

unread,
May 10, 2003, 4:35:24 PM5/10/03
to
On Sat, May 10, 2003 at 08:53:10PM +0200, Norman Werner wrote:
> Hello,
>
> I was wondering whether there exists an general Mechanism to connect
> lisp to another interactive Programm (let's say: gnuplot, a shell,
> another lisp-process , spice ...) in a plattform-independent way(ansi-cl).
>
> Up to now I only got the idea to misuse the fileinterface through
> fifos:
>
> Norman@welt> cat fifo |gnuplot| backfifo &
> Norman@welt> lisp
> lisp> (with-open-file (send "fifo" :direction :output)
> lisp> (write "plot sin(x)" :stream send))
>
> But this seems to be rather ineffective, insecure and not-a-good-idea.

Nor is it portable. But neither is running a program/IPC. ANSI CL doesn't
cover that, it is left to implementations. All implementations I know of
provide some function to do what you want. The CLOCC PORT package attempts to
unify the many interfaces, to some degree of success, you may want to
investigate that. There is a GNUPlot interface somewhere as well, I think.

> Any other suggestions?

You probably don't want to use WRITE like that. You will end up with:

"plot sin(x)"

rather than

plot sin(x)

going to gnuplot.

I advise using FORMAT here, since you probably will end up needing it anyhow.
ie.

(format send "plot sin(x)~%")

or maybe

(format send "plot sin(~A)~%" variable-name)

See the Hyperspec for all formatting options.

--
; Matthew Danish <mda...@andrew.cmu.edu>
; OpenPGP public key: C24B6010 on keyring.debian.org
; Signed or encrypted mail welcome.
; "There is no dark side of the moon really; matter of fact, it's all dark."

Kent M Pitman

unread,
May 10, 2003, 9:01:24 PM5/10/03
to
Norman Werner <norman...@student.uni-magdeburg.de> writes:

> I was wondering whether there exists an general Mechanism to connect
> lisp to another interactive Programm (let's say: gnuplot, a shell,
> another lisp-process , spice ...) in a plattform-independent way(ansi-cl).

Since these things you are connecting to are platform-dependent,
that's a kind of odd request.

On the Lisp Machine, for example, which is lisp down to the hardware,
what would it mean to connect to gnuplot? Do you suppose that if gnuplot
existed there, it would be invoked by shell command, there being no shell?
It would be a Lisp function with keyword arguments.

And even among PC and Mac, the variety of shells (sh/csh/bash vs DOS vs
whatever Apple has) is broad.

There are legacy operating systems like VAX VMS, ITS DDT, MIT Multics,
and there could be (if we're lucky) future operating systems that are
different yet again than the dreck that passes for operating systems
these days. I would _hope_ that the first requirement of such a system
is that if you did a callout to another process, you didn't do it by
writing commands onto a stream that had to be parsed before they could
be executed, but instead you passed real objects to real functions.

Your problem is ill-defined.

If you relax the platform-independence part, you have half a hope
of winning.

Norman Werner

unread,
May 11, 2003, 4:00:13 PM5/11/03
to
Kent M Pitman <pit...@world.std.com> writes:

> Date: Sun, 11 May 2003 10:27:50 +0200
> From: Norman Werner <norman...@student.uni-magdeburg.de>
> To: Kent M Pitman <pit...@world.std.com>
> Subject: Re: general Interface CL <-> interactive Programm ?
>
> Aargh,
> the last letter should have gone to the newsgroup
>
> Norman
>
> In case you want to re-post it, I've included your text below.
>
> Btw, fyi, webster.com says:
>
> Main Entry: dreck
> Variant(s): also drek /'drek/
> Function: noun
> Etymology: Yiddish drek & German Dreck, from Middle High German drec;
> akin to Old English threax rubbish
> Date: 1922
> : TRASH, RUBBISH
>
> So I hope it means the same for us as for you. :) Sounds like it
> does, but one never knows.

It does
> From: Norman Werner <norman...@student.uni-magdeburg.de>
> Subject: Re: general Interface CL <-> interactive Programm ?
> To: Kent M Pitman <pit...@world.std.com>
> Date: Sun, 11 May 2003 10:25:26 +0200


>
> Kent M Pitman <pit...@world.std.com> writes:
>
>>
>> Since these things you are connecting to are platform-dependent,
>> that's a kind of odd request.
>>
>> On the Lisp Machine, for example, which is lisp down to the hardware,
>> what would it mean to connect to gnuplot? Do you suppose that if gnuplot
>> existed there, it would be invoked by shell command, there being no shell?
>> It would be a Lisp function with keyword arguments.
>>
>> And even among PC and Mac, the variety of shells (sh/csh/bash vs DOS vs
>> whatever Apple has) is broad.
>>
>> There are legacy operating systems like VAX VMS, ITS DDT, MIT Multics,
>> and there could be (if we're lucky) future operating systems that are
>> different yet again than the dreck that passes for operating systems
>> these days.
>

> You are saying "dreck" ? ... That's funny - At least for me being
> german.

>> I would _hope_ that the first requirement of such a system
>> is that if you did a callout to another process, you didn't do it by
>> writing commands onto a stream that had to be parsed before they could
>> be executed, but instead you passed real objects to real functions.
>>
>> Your problem is ill-defined.
>>
>> If you relax the platform-independence part, you have half a hope
>> of winning.
>

> Ok - with platform-independent meaning "runs on toasters and
> space-shuttles" the question is odd. But with platform-independent
> meaning "runs on all major lisp-implementations" ... Oops the question
> is odd too - since a package with support for different lisps will do.
>
>
> Norman
> ----------
>
Norman

Pascal Bourguignon

unread,
May 11, 2003, 4:09:34 PM5/11/03
to

Kent M Pitman <pit...@world.std.com> writes:

> Norman Werner <norman...@student.uni-magdeburg.de> writes:
>
> > I was wondering whether there exists an general Mechanism to connect
> > lisp to another interactive Programm (let's say: gnuplot, a shell,
> > another lisp-process , spice ...) in a plattform-independent way(ansi-cl).
>
> Since these things you are connecting to are platform-dependent,
> that's a kind of odd request.
>
> On the Lisp Machine, for example, which is lisp down to the hardware,
> what would it mean to connect to gnuplot? Do you suppose that if gnuplot
> existed there, it would be invoked by shell command, there being no shell?
> It would be a Lisp function with keyword arguments.

Lisp Machines had processes so there was a way in the LISP of Lisp
Machines to fork a process and to communicate with another process (I
hope so at least). The point is that there's no such facility in
Common Lisp. Now, on what architecture where a Common Lisp
implementation exists is there no processes and no IPC?

That's exactly beacuse it's platform dependend that a common API
should have been standardized! ^W^W^W be standardized now!


> And even among PC and Mac, the variety of shells (sh/csh/bash vs DOS vs
> whatever Apple has) is broad.
>
> There are legacy operating systems like VAX VMS, ITS DDT, MIT Multics,
> and there could be (if we're lucky) future operating systems that are
> different yet again than the dreck that passes for operating systems
> these days. I would _hope_ that the first requirement of such a system
> is that if you did a callout to another process, you didn't do it by
> writing commands onto a stream that had to be parsed before they could
> be executed, but instead you passed real objects to real functions.
>
> Your problem is ill-defined.

No. If you have such hope, you should push for such an API. Java with
its RMI specifies such an API of remote objects and messages.


> If you relax the platform-independence part, you have half a hope
> of winning.

You cannot deny the power of the unix way in particular with respect
to the stream of bytes paradigm. It may seem rustic, but it works and
goes a long way. I would argue that it will be more resilient than
any object API. Otherwise, they're equivalent: one can be implemented
on the other, so you just have to standardize one API and let people
implement the other if they prefer it. We're not discussing that,
we're discussing the fact that none is standardized.

Kent M Pitman

unread,
May 11, 2003, 4:48:57 PM5/11/03
to
Pascal Bourguignon <sp...@thalassa.informatimago.com> writes:

> You cannot deny the power of the unix way in particular with respect
> to the stream of bytes paradigm.

I can deny that it is implementation-independent.

Christopher C. Stacy

unread,
May 12, 2003, 12:35:12 AM5/12/03
to
>>>>> On 11 May 2003 22:09:34 +0200, Pascal Bourguignon ("Pascal") writes:

Pascal> Lisp Machines had processes so there was a way in the LISP of
Pascal> Lisp Machines to fork a process and to communicate with
Pascal> another process (I hope so at least)

Lisp Machines did not have "processes" in the sense that
the word is used today on operating systems such as Unix.
The Lisp Machine had what were called "stack groups" and
a multiprocessing scheduler that could switch between them.
We called them "processes", but the word had a different
meaning than today. There was only a single address space
on the machine, and all of these "processes" (which were
much more like what today are called "threads") lived
in that single address space.

There was no "inter-process" communication because there were no
seperate processes; communication between programs was accomplished
the same way that it is done in Lisp implementations today -- by
direct reference to the objects shared between two programs.

The "process" primitives like PROCESS-RUN-FUNCTION that you find
in most Common Lisp implementations today are patterned after
the functions of the same name on the Lisp Machine, and they
do the same thing.

Pascal> The point is that there's no such facility in Common Lisp.
Pascal> Now, on what architecture where a Common Lisp implementation
Pascal> exists is there no processes and no IPC?

Besides the Lisp Machine? I don't know, and neither do you.
And neither did X3J13, nor did they know what operating systems
in the year 2013 would have for multiprocessing abstractions,
nor did they wish to exclude all the systems that don't have
multiprocessing at all. That's why it's not in the language.

(If you're really hung up on this and missing the point:
one guess offhand would be the "embedded" implementations
of Common Lisp that are used for on-board robot control.
I don't think they have processes or pipes.)

Pascal> No. If you have such hope, you should push for such an API.
Pascal> Java with its RMI specifies such an API of remote objects
Pascal> and messages.

Something like RMI would seem to be a good idea. In fact, I believe
RMI has been implemented in Lisp at least twice already.

Pascal> You cannot deny the power of the unix way in particular with
Pascal> respect to the stream of bytes paradigm. It may seem rustic,
Pascal> but it works and goes a long way.

Not all operating systems support Unix interprocess communication.
But you can open pipes from most Lisps that run on Unix as well as
sockets (at least TCP sockets). If you're on Windows, you can write
foreign function calls to open named pipes.

What you are asking for is implementation dependant.

If you want to invent a standard and write the library that
implements your favorite method of IPC, and is portable across
the operating systems that you like, nobody is stopping you.

What's the problem?

Tim Bradshaw

unread,
May 12, 2003, 5:20:59 AM5/12/03
to
* Pascal Bourguignon wrote:

> No. If you have such hope, you should push for such an API. Java with
> its RMI specifies such an API of remote objects and messages.

So does CORBA. And, oh look, there is a standard CL CORBA binding.

--yim

Marco Antoniotti

unread,
May 12, 2003, 12:01:27 PM5/12/03
to

Christopher C. Stacy wrote:

> Pascal> No. If you have such hope, you should push for such an API.
> Pascal> Java with its RMI specifies such an API of remote objects
> Pascal> and messages.
>
> Something like RMI would seem to be a good idea. In fact, I believe
> RMI has been implemented in Lisp at least twice already.

Are you referring, e.g. to CMUCL WIRE package?

Cheers

--
Marco Antoniotti

Christopher C. Stacy

unread,
May 12, 2003, 12:24:04 PM5/12/03
to
>>>>> On Mon, 12 May 2003 12:01:27 -0400, Marco Antoniotti ("Marco") writes:

Marco> Christopher C. Stacy wrote:

Pascal> No. If you have such hope, you should push for such an API.
Pascal> Java with its RMI specifies such an API of remote objects
Pascal> and messages.
>>
>> Something like RMI would seem to be a good idea. In fact, I believe
>> RMI has been implemented in Lisp at least twice already.

Marco> Are you referring, e.g. to CMUCL WIRE package?

No, I think there was a project at BBN that implemented
RMI so they could talk to Java graphics pacakge, and
there was also one at MIT for some other project.

I don't know anything else about them, just saw
them going by...

Pascal Costanza

unread,
May 12, 2003, 4:27:58 PM5/12/03
to
In article <uissg8...@dtpq.com>,

I know about two approaches:

- Jacol uses sockets to communicate between Java and Common Lisp
applications. http://jacol.sourceforge.net/

- lijos implements Java ObjectStreams in Common Lisp. http://pointnclick.com/lijos/

I don't know anything about the quality of these libraries since I
haven't tried them by myself.


Pascal

Pascal Bourguignon

unread,
May 13, 2003, 12:37:47 AM5/13/03
to

cst...@dtpq.com (Christopher C. Stacy) writes:

> Pascal> The point is that there's no such facility in Common Lisp.
> Pascal> Now, on what architecture where a Common Lisp implementation
> Pascal> exists is there no processes and no IPC?
>
> Besides the Lisp Machine? I don't know, and neither do you.
> And neither did X3J13, nor did they know what operating systems
> in the year 2013 would have for multiprocessing abstractions,
> nor did they wish to exclude all the systems that don't have
> multiprocessing at all. That's why it's not in the language.
>
> (If you're really hung up on this and missing the point:
> one guess offhand would be the "embedded" implementations
> of Common Lisp that are used for on-board robot control.
> I don't think they have processes or pipes.)

[...]

> What you are asking for is implementation dependant.

Well, let's say "implementation-class dependant". There are a range
of implementations having feature X, and a range of implementations
having feature Y, etc. It would be nice to have standard APIs to
access feature X or feature Y.


> If you want to invent a standard and write the library that
> implements your favorite method of IPC, and is portable across
> the operating systems that you like, nobody is stopping you.

You're right, nobody's stopping me.

> What's the problem?

Well, I don't have the resources for example to implement a POSIX
package or a standard SOCKET package on all the free Common-Lisp
implementation, much less the commercial ones.

Come to think, the situation would be much easier if there was only a
common, standardized FFI API. Then people could implement such
packages in a portable way, and some of them could become de-facto
standards.

But then, a FFI would not be enough to implement threads or
co-routines in a Common-Lisp, because of the interactions with the
garbage-collector. So I think that a Common-Lisp standardized API for
the threads would be useful nonetheless.

Christopher C. Stacy

unread,
May 13, 2003, 1:01:59 AM5/13/03
to
>>>>> On 13 May 2003 06:37:47 +0200, Pascal Bourguignon ("Pascal") writes:

Pascal> cst...@dtpq.com (Christopher C. Stacy) writes:

Pascal> The point is that there's no such facility in Common Lisp.
Pascal> Now, on what architecture where a Common Lisp implementation
Pascal> exists is there no processes and no IPC?
>>
>> Besides the Lisp Machine? I don't know, and neither do you.
>> And neither did X3J13, nor did they know what operating systems
>> in the year 2013 would have for multiprocessing abstractions,
>> nor did they wish to exclude all the systems that don't have
>> multiprocessing at all. That's why it's not in the language.
>>
>> (If you're really hung up on this and missing the point:
>> one guess offhand would be the "embedded" implementations
>> of Common Lisp that are used for on-board robot control.
>> I don't think they have processes or pipes.)

Pascal> [...]

>> What you are asking for is implementation dependant.

Pascal> Well, let's say "implementation-class dependant". There are a range
Pascal> of implementations having feature X, and a range of implementations
Pascal> having feature Y, etc. It would be nice to have standard APIs to
Pascal> access feature X or feature Y.



>> If you want to invent a standard and write the library that
>> implements your favorite method of IPC, and is portable across
>> the operating systems that you like, nobody is stopping you.

Pascal> You're right, nobody's stopping me.

>> What's the problem?

Pascal> Well, I don't have the resources for example to implement a POSIX
Pascal> package or a standard SOCKET package on all the free Common-Lisp
Pascal> implementation, much less the commercial ones.

But maybe you can get together and pool your money to fund the
development of such an interface with the other customers who
want that. With the money, you afford to to do it yourselves,
perhaps using UFFI, or you could pay the vendors to hire
someone to do it.

Other than that, yeah, it would be nice if I had a Mercedes Benz, too.

Pascal> Come to think, the situation would be much easier if there was only a
Pascal> common, standardized FFI API. Then people could implement such
Pascal> packages in a portable way, and some of them could become de-facto
Pascal> standards.

That's what UFFI is supposed to be, and people are doing that, I thought.

Pascal> But then, a FFI would not be enough to implement threads or
Pascal> co-routines in a Common-Lisp, because of the interactions with the
Pascal> garbage-collector. So I think that a Common-Lisp standardized API for
Pascal> the threads would be useful nonetheless.

Yes, native threads require serious involvement with the compiler,
but some sort of generic multiprocessing functional interface veneer
could be done. Isn't that called "ACL-COMPAT"?

Daniel Barlow

unread,
May 13, 2003, 6:31:33 AM5/13/03
to

ACL-COMPAT is (obviously) more a clone of the ACL MP interface than a
"generic" interface, and as such it still contains things like
without-scheduling and process-wait, which could be difficult to
implement sensibly on some systems.

(Although Allegro themselves have that rather neat 'gate' thing, which
looks like it would do an efficient 'put a process on a queue' wait
behind the scenes so it probably works out quite well in their
implementation, I don't see any obvious signs that the acl-compat
ports have implemented that bit yet)

OK, this is really all just a thinly disguised plug: after taking
apart the clocc port MP layer the other day, I put together a proposal
of my own. It can be summarized as "CLIM-SYS minus the bits that
assume a Lisp-scheduled implementation, plus Posix/Java-style
condition variables". I don't think I announced it on the newsgroup
yet, so comments on

http://alu.cliki.net/com-metacircles-clim-sys-mp

(here or on that page) very welcome. Something like this will
probably be in the forthcoming SBCL native threads implementation.


-dan

--

http://www.cliki.net/ - Link farm for free CL-on-Unix resources

Christophe Rhodes

unread,
May 13, 2003, 11:32:50 AM5/13/03
to
Daniel Barlow <d...@telent.net> writes:

> OK, this is really all just a thinly disguised plug: after taking
> apart the clocc port MP layer the other day, I put together a proposal
> of my own. It can be summarized as "CLIM-SYS minus the bits that
> assume a Lisp-scheduled implementation, plus Posix/Java-style
> condition variables". I don't think I announced it on the newsgroup
> yet, so comments on
>
> http://alu.cliki.net/com-metacircles-clim-sys-mp
>
> (here or on that page) very welcome. Something like this will
> probably be in the forthcoming SBCL native threads implementation.

One of the problems with having different names for everything from
the rest of the world is naming collisions... sadly, MAKE-CONDITION is
unlikely to be a great name for something that makes a condition
object for use with CONDITION-WAIT and CONDITION-NOTIFY :-(

Christophe
--
http://www-jcsu.jesus.cam.ac.uk/~csr21/ +44 1223 510 299/+44 7729 383 757
(set-pprint-dispatch 'number (lambda (s o) (declare (special b)) (format s b)))
(defvar b "~&Just another Lisp hacker~%") (pprint #36rJesusCollegeCambridge)

Reply all
Reply to author
Forward
0 new messages