GNU Extension Language Plans
Eric Bouck
Richard Stallman, GNU Project
[Please redistribute widely]
Many software packages need an extension language to make it easier
for users to add to and modify the package.
In a previous message I explained why Tcl is inadequate as an
extension language, and stated some design goals for the extension
language for the GNU system, but I did not choose a specific
alternative.
At the time, I had not come to a conclusion about what to do. I knew
what sort of place I wanted to go, but not precisely where or how to
get there.
Since then, I've learned a lot more about the topic. I've read about
scsh, Rush and Python, and talked with people working on using Scheme
as an extension and scripting language. Now I have formulated a
specific plan for providing extensibility in the GNU system.
Who chooses which language?
Ousterhour, the author of Tcl, responded to my previous message by
citing a "Law" that users choose the language they prefer, and
suggested that we each implement our favorite languages, then sit back
and watch as the users make them succeed or fail.
Unfortunately, extension languages are the one case where users
*cannot* choose the language they use. They have to use the language
supported by the application or tool they want to extend. For
example, if you wanted to extend PDP-10 Emacs, you had to use TECO.
If you want to extend GNU Emacs, you have to use Lisp.
When users simply want "to write a program to do X or Y," they can use
any language the system supports. There's no reason for system
designers to try to decide which language is best. We can instead
provide as many languages as possible, to give each user the widest
possible choice. In the GNU system, I would like to support every
language people want to use--provided someone will implement them.
With the methods generally used today, we cannot easily provide many
languages for extending any particular utility or application package.
Supporting an extension language means a lot of work for the developer
of the package. Supporting two languages is twice as much work,
supposing the two fit together at all. In practice, the developer has
to choose a language--and then all users of the package are stuck with
that one. For example, when I wrote GNU Emacs, I had to decide which
language to support. I had no way to let the users decide.
When a developer chooses Tcl, that has two consequences for the
users of the package:
* They can use Tcl if they wish. That's fine with me.
* They can't use any other language. That I consider a problem.
Sometimes developers choose a language because they like it. But not
always. Sun recently announced a campaign to "make Tcl the universal
scripting language." This is a campaign to convince all the
developers who *don't* prefer Tcl that they really have no choice.
The idea is that each one of us will believe that Sun will inevitably
convince everyone else to use Tcl, and each of us will feel compelled
to follow where we believe the rest are going.
That campaign is what led me to decide that I needed to speak to the
community about the issue. By announcing on the net that GNU software
packages won't use Tcl, I hope to show programmers that not everyone
is going to jump on the Tcl bandwagon--so they don't have to feel
compelled to do so. If developers choose to support Tcl, it should be
because they want to, not because Sun convinces them they have no
choice.
Design goals for GNU
When you write a program, or when you modify a GNU program, I think
you should be the one who decides what to implement. I can't tell you
what language to support, and I wouldn't want to try.
But I am the leader of one particular project, the GNU project. So I
make the decision about which packages to include in the GNU operating
system, and which design goals to aim for in developing the GNU
system.
These are the design goals I've decided on concerning extension
languages in the GNU system:
* As far as possible, all GNU packages should support the same
extension languages, so that a user can learn one language (any one of
those we support) and use it in any package--including Emacs.
* The languages we support should not be limited to special, weak
"scripting languages". They should be designed to be good for writing
large programs as well as small ones.
My judgement is that Tcl can't satisfy this goal. (Ousterhout seems
to agree that Tcl doesn't serve this goal. He thinks that doesn't
constitute a problem--I think it does.) That's why I've decided not
to use Tcl as the main system-wide extension language of the GNU
system.
* It is important to support a Lisp-like language, because they
provide certain special kinds of power, such as representing programs
as data in a structured way that can be decoded without parsing.
** It is desirable to support Scheme, because it is simple and clean.
** It is desirable to support Emacs Lisp, for compatibility with Emacs
and the code already written for Emacs.
* It is important to support a more usual programming language syntax
for users who find Lisp syntax too strange.
* It would be good to support Tcl as well, if that is easy to do.
The GNU extension language plan
Here is the plan for achieving the design goals stated above.
* Step 1. The base language should be modified Scheme, with these features:
** Case-sensitive symbol names.
** No distinction between #f and (), for the sake of supporting Lisp
as well as Scheme.
** Convenient fast exception handling, and catch and throw.
** Extra slots in a symbol, to better support
translating other Lisp dialects into Scheme.
** Multiple obarrays.
** Flexible string manipulation functions.
** Access to all or most of the Unix system calls.
** Convenient facilities for forking pipelines,
making redirections, and so on.
** Two interfaces for call-outs to C code.
One allows the C code to work on arbitrary Scheme data.
The other passes strings only, and is compatible with Tcl
C callouts provided the C function does not try to call
the Tcl interpreter.
** Cheap built-in dynamic variables (as well as Scheme's lexical variables).
** Support for forwarding a dynamic variable's value
into a C variable.
** A way for applications to define additional Scheme data types
for application-specific purposes.
** A place in a function to record an interactive argument reading spec.
** An optional reader feature to convert nil to #f and t to #t,
for the sake of supporting Lisp as well as Scheme.
** An interface to the library version of expect.
** Backtrace and debugging facilities.
All of these things are either straightforward or have already been
done in Scheme systems; the task is to put them together. We are
going to start with SCM, add some of these features to it, and write
the rest in Scheme, using existing implementations where possible.
* Step 2. Other languages should be implemented on top of Scheme.
** Rush is a cleaned-up version of the Tcl language, which runs far
faster than Tcl itself, by means of translation into Scheme. Some
kludgy but necessary Tcl constructs don't work in Rush, and Tcl
aficionadoes may be unhappy about this; but Rush provides cleaner ways
to get the same results, so users who write extensions should like it
better. Developers looking for an extension language are likely to
prefer Rush to Tcl if they are not already attached to Tcl.
Here are a couple of examples supplied by Adam Sah:
*** To pass an array argument without copying it, in Tcl you must use
upvar or make the array a global variable. In Rush, you can simply
declare the argument "pass by reference".
*** To extract values from a list and pass them as separate arguments
to a function, in Tcl you must construct a function call expression
using that list, and then evaluate it. This can cause trouble if the
other arguments contain text that includes any special Tcl syntax. In
Rush, the apply function handles this simply and reliably.
*** Rush eliminates the need for the "expr" command by allowing infix
mathematical expressions and statements. For example, the Tcl
computation `"set a [expr $b*$c]' can be written as `a = b*c' in
Rush. (The Tcl syntax works also.)
Some references:
[SBD94] Adam Sah, Jon Blow and Brian Dennis. "An Introduction to the Rush
Language." Proc. Tcl'94 Workshop. June, 1994.
ftp://ginsberg.cs.berkeley.edu:pub/papers/asah/rush-tcl94.*
[SB94] Adam Sah and Jon Blow. "A New Architecture for the Implementation of
Scripting Languages." Proc. USENIX Symp. on Very High Level Languages.
October, 1994. to appear.
ftp://ginsberg.cs.berkeley.edu:pub/papers/asah/rush-vhll94.*
** It appears that Emacs Lisp can be implemented efficiently by
translation into modified Scheme (the modifications are important).
** Python appears suitable for such an implementation, as far as I can
tell from a quick look. By "suitable" I mean that mostly the same
language could be implemented--minor changes in semantics would be ok.
(It would be useful for someone to check this carefully.)
** A C-like language syntax can certainly be implemented this way.
* Distribution conditions.
We will permit use of the modified Scheme interpreter in proprietary
programs, so as to compete effectively with alternative extensibility
packages.
Translators from other languages to modified Scheme will not be part
of any application; each individual user will decide when to use one
of these. Therefore, there is no special reason not to use the GPL as
the distribution terms for translators. So we will encourage
developers of translators to use the GPL as distribution terms.
Conclusion
Until today, users have not been able to choose which extension
language to use. They have always been compelled to use whichever
language is supposed by the tool they wish to extend. And that has
meant many different languages for different tools.
Adopting Tcl as the universal scripting language offers the
possibility of eliminating the incompatibility--users would be able to
extend everything with just one language. But they wouldn't be able
to choose which language. They would be compelled to use Tcl and
nothing else.
By making modified Scheme the universal extension language, we can
give users a choice of which language to write extensions in. We can
implement other languages, including modified Tcl (Rush), a Python
variant, and a C-like language, through translation into Scheme, so
that each user can choose the language to use. Even users who choose
modified Tcl will benefit from this decision--they will be happy with
the speedup they get from an implementation that translates into
Scheme.
Only Scheme, or something close to Scheme, can serve this purpose.
Tcl won't do the job. You can't implement Scheme or Python or Emacs
Lisp with reasonable performance on top of Tcl. But modified Scheme
can support them all, and many others.
The universal extension language should be modified Scheme.
Request for Volunteers
If you understand Scheme implementation well, and you want to
contribute a substantial amount of time to this project, please send
mail to Tom Lord, lo...@gnu.ai.mit.edu.
If you expect to have time later but don't have time now, please send
mail when you do have time to work. Participation in a small way is
probably not useful until after the package is released.
Aaron Watters
> GNU Extension Language Plans
> Richard Stallman, GNU Project
>* Step 1. The base language should be modified Scheme, with these features:...
Fine. Please extend Scheme (as I knew it) with a flexible
object-encapsulation/inheritence system and convenient, well
designed, portable interfaces to common os functionalities
and libraries -- for a good example of how to do this you might do this
look at the Python code. If you want people to USE the language
without fear of illegality, maybe you could use a copyright which
protects your rights without restricting the USE of the language
-- like the one that applies to Python.
In the mean time, since I want a good scripting/extension language
without scary copyright restrictions and with good interfaces to
just about everything I could possibly want NOW, I'll burrow on
ahead using Python.
Aaron Watters
Department of Computer and Information Sciences
New Jersey Institute of Technology
University Heights
Newark, NJ 07102
phone (201)596-2666
fax (201)596-5777
home phone (908)545-3367
email: aa...@vienna.njit.edu
PS:
Personally, I've always found Scheme a little irritating --
ever since I read the standard text on the subject which mentions
arrays somewhere around page 400, in a footnote, without telling
you how to use one. (Do I detect MIT/NIH? Naw.)
Josef Dalcolmo
Steven D. Majewski <sd...@virginia.edu> wrote:
>
>Python Workshop! ( Since Guido has already accused me of trying to
>turn Python into Scheme, I thought I might try to reverse the
>process. :-)
I like the idea. I certainly prefer to program in python over programming in
Scheme being a human myself, but for the task of passing scripts between
programs and machine generated programs Scheme seems simpler.
I wonder what that is going to mean for the tk interface: my guess would be
that STk would replace tkinter, since according to RMS the underlying Scheme
the real interface to the application is - right?
- Josef
Barry Margolin
>Personally, I've always found Scheme a little irritating --
>ever since I read the standard text on the subject which mentions
>arrays somewhere around page 400, in a footnote, without telling
>you how to use one. (Do I detect MIT/NIH? Naw.)
The standard reference for Scheme is the "Revised**4 Report on the
Algorithmic Language Scheme", which is only 55 pages long. I suspect
you're referring the "Structure and Interpretation of Computer Programs".
As the name implies, this is a text on programming, not on any particular
programming language. This book is oriented towards teaching about
advanced programming structures such as closures, streams, and
continuations, which are unique to Scheme and similar languages; mundane
features like arrays do admittedly get little coverage. The course assumes
that the student has some prior, basic programming skills, so already knows
how to use arrays.
--
Barry Margolin
BBN Internet Services Corp.
bar...@near.net
Steven D. Majewski
On Wed, 19 Oct 1994, Eric Bouck forwarded RMS on Tcl,Scheme,Python,etc.
One particular point concerning implementing Python on top of Scheme:
>
> ** Python appears suitable for such an implementation, as far as I can
> tell from a quick look. By "suitable" I mean that mostly the same
> language could be implemented--minor changes in semantics would be ok.
> (It would be useful for someone to check this carefully.)
>
Actually, this was already on my topic list for discussion at the
Python Workshop! ( Since Guido has already accused me of trying to
turn Python into Scheme, I thought I might try to reverse the
process. :-)
-- Steve Majewski (804-982-0831) <sd...@Virginia.EDU> --
-- UVA Department of Molecular Physiology and Biological Physics --
-- Box 449 Health Science Center Charlottesville,VA 22908 --
[ "Cheese is more macho?" ]
Aaron Watters
>In article <1994Oct20.1...@njitgw.njit.edu> aa...@vienna.njit.edu (Aaron Watters) writes:
>>ever since I read the standard text on the subject which mentions
>>arrays somewhere around page 400, in a footnote, without telling
>
>The standard reference for Scheme is the "Revised**4 Report on the
>Algorithmic Language Scheme", which is only 55 pages long. I suspect
>you're referring the "Structure and Interpretation of Computer Programs".
You got it!
>As the name implies, this is a text on programming, not on any particular
>programming language. This book is oriented towards teaching about
>advanced programming structures such as closures, streams, and
>continuations, which are unique to Scheme and similar languages; mundane
>features like arrays do admittedly get little coverage....
Quite right. As we all know, all uses of arrays are mundane and
trivial. Advanced programmers never use arrays. There are no
interesting algorithms that use arrays in interesting ways. Arrays
never come up in good classes about "advanced" programming.
In fact: didn't Turing show that all we *really* need is two stacks
of bits? Hmmm....
This gives me a really good idea for my own extension language!
[which I'll make freely copiable, but I'll restrict it from use in
any activity which makes any money in any way using a really complex
copyright: I'll take the gpl as a starting point (this'll be the
really fun part -- hey, maybe I'll start a really important movement
or revolution by forcing other people to not make money using my
program!!!).]
-a.
Ps: the real reason this "text on programming" doesn't
talk about arrays is there is no
good way to "do" arrays in functional programming, even though
arrays are the single most useful structures in real programming;
hence my irritation.
Stefan Monnier
Aaron Watters <aa...@vienna.njit.edu> wrote:
> Quite right. As we all know, all uses of arrays are mundane and
> trivial. Advanced programmers never use arrays. There are no
Arrays are simple in concept. SICP is supposed to teach other more
"advanced" concepts. Doesn't mean that arrays are less used. They are
just assumed to be known !
> In fact: didn't Turing show that all we *really* need is two stacks
> of bits? Hmmm....
Yup ! Unbounded stacks !
And those stacks are easier to implement with lisp lists than with
arrays (and their fixed size)
> Ps: the real reason this "text on programming" doesn't
> talk about arrays is there is no
> good way to "do" arrays in functional programming, even though
There is, but it's still recent technology.
> arrays are the single most useful structures in real programming;
> hence my irritation.
And arrays are the single most obvious reason why most programs either
crash on big data sets or (if you're lucky) complain because it's
bigger than some arbitrary internal limit !
Stefan
Aaron Watters
some points of interest, but in retrospect they were mean spirited.
[I'm trying to put into practice various pieces of advice I recieved
privately, like "don't be such a moron".]
Scheme is a beautiful little language and could be a great extension/
scripting language if it had standard portable interfaces to a large
number of libraries, and if it had native object support with
inheritance -- I understand some mutant strains do...
[Python certainly does.]
I think the GPL and its variants should be changed to something
less restrictive and simpler -- in one case I know of a developer did some
work using GNU stuff and ended ripping a lot of it out in order
to avoid the bother of complying with the terms. He says next
time he'll license proprietary source with binary distribution
rights (such things exist, and can be very nice). Clearly, in this
case, the GPL didn't encourage the use of freely copiable software.
Next time I see him I'll recommend python and it's associated tools,
since he can use them however he pleases, as long as he credits the
source. A gnu-scheme would fair better in the world if it had
a copyright like the ones on TCL and python.
No more comments on books. Sorry,sorry,sorry.
Scott McLoughlin
> good way to "do" arrays in functional programming, even though
> arrays are the single most useful structures in real programming;
> hence my irritation.
Howdy,
Exactly why the universal scripting language should be
an embedded APL ;-)
=============================================
Scott McLoughlin
Conscious Computing
=============================================
Gareth Newfield
Language", or an "Universal Extension Language" is unecessary and
counterproductive. Actually the very concept sounds like something Microsoft
would dream up, but yes I know the concept predates Microsoft by many years.
Gareth
Joseph H Allen
Aaron Watters <aa...@vienna.njit.edu> wrote:
>In article <EBOUCK.94O...@dsdbqvarsa.er.usgs.gov> ebo...@usgs.gov writes:
> > GNU Extension Language Plans
>> Richard Stallman, GNU Project
>>....
>>* Step 1. The base language should be modified Scheme, with these features:...
>
>Fine. Please extend Scheme (as I knew it) with a flexible
>object-encapsulation/inheritence system and convenient, well
>designed, portable interfaces to common os functionalities
>and libraries -- for a good example of how to do this you might do this
>look at the Python code.
Actually I would say that an object system is almost essential for a good
extension langauge. You want to be able to add new types to the language
which correspond to the elements of whatever system the language is embedded
in. Without this, you have to add tons of keywords to manage data objects
which are not really part of the language.
Python can do this, although it's operator overloading syntax is a little
awkward and it might be too slow. Some of the modern functional languages
might be fast enough. Haskel comes to mind, but it has a horrendous syntax.
Another concern is that you probably want the source-code inputs to these
languages to be event driven- for X and multi-user applications. With
scheme you could do this with a preprocessor (just collect input until you
have balanced parenthasis). Python's parser could probably be made event
driven as well, since it's table driven.
--
/* jha...@world.std.com (192.74.137.5) */ /* Joseph H. Allen */
int a[1817];main(z,p,q,r){for(p=80;q+p-80;p-=2*a[p])for(z=9;z--;)q=3&(r=time(0)
+r*57)/7,q=q?q-1?q-2?1-p%79?-1:0:p%79-77?1:0:p<1659?79:0:p>158?-79:0,q?!a[p+q*2
]?a[p+=a[p+=q]=q]=q:0:0;for(;q++-1817;)printf(q%79?"%c":"%c\n"," #"[!a[q-1]]);}
John Ousterhout
I'd like to respond to an error in Richard Stallman's latest posting.
Stallman said:
Sun recently announced a campaign to "make Tcl the universal
scripting language." This is a campaign to convince all the
developers who *don't* prefer Tcl that they really have no choice.
The idea is that each one of us will believe that Sun will inevitably
convince everyone else to use Tcl, and each of us will feel compelled
to follow where we believe the rest are going.
Please understand that this "campaign" exists only in Stallman's mind.
As far as I know there has never been *any* official Sun announcement
about Tcl. There is no campaign at Sun to stamp out Tcl competitors;
Tcl and Tk aren't even official Sun products right now, nor has Sun
made a commitment to turn them into products (yet). If anyone has
concrete evidence to back up Stallman's accusations, please post it
so we can all see it.
The only information I recall from Sun about Tcl is a couple of job
listings and one or two personal messages from me, which were intended
to keep the Tcl community informed about what my group is doing and
to solicit input. I believe that these messages were posted on
comp.lang.tcl only; if the goal had been to browbeat people who dislike
Tcl, the messages would have been posted where Tcl haters would see them.
Of course, I hope that someday Tcl and Tk will become Sun products (and
also products at many other companies), but I think that this will
*increase* the alternatives available to people, not decrease them.
I also hope that Tcl and Tk will become a universal scripting language
for the Internet, but I hope to do this by making Tcl and Tk so attractive
that people *want* to use them, not by somehow preventing people from
using alternatives. I don't see how such a negative campaign could work
anyhow.
If there has been a negative campaign, I daresay it has come from
Stallman. After all, who posted a message that used half-truths to
try to convince people not to use a particular system, without even
providing a viable alternative?
Fortunately, I found the rest of Stallman's message, where he began
the process of designing a new scripting language, more encouraging.
I just hope that he and those who work with him can focus on the
positive process of developing what they think is a better scripting
language, rather than a negative process of accusation and
misinformation.
The saddest thing about all of this is that the UNIX community
continues to bicker about silly details such as whether a programming
system can be considered to have linked lists if it doesn't have a
garbage collector too, while Microsoft steadily increases its market
share and makes us irrelevant. If you're looking for a company that
really knows how to squash its competition, you should look farther
north.
Tom Lord
Excessive exegesis is the greatest vice of internet discourse. I
would much rather use my posting minutes to work on the project than
to argue about the sources of its inspiration. However, I feel I have
a responsibility to clear up one point for the record, and to extend
an offer.
John Ousterhout quoted RMS and replied:
Sun recently announced a campaign to "make Tcl the
universal scripting language." This is a campaign to
convince all the developers who *don't* prefer Tcl that
they really have no choice. The idea is that each one of
us will believe that Sun will inevitably convince everyone
else to use Tcl, and each of us will feel compelled to
follow where we believe the rest are going.
Please understand that this "campaign" exists only in Stallman's mind.
[...]
If anyone has concrete evidence to back up Stallman's accusations,
please post it so we can all see it.
[...]
The only information I recall from Sun about Tcl is a couple of job
listings and one or two personal messages from me, which were intended
to keep the Tcl community informed about what my group is doing and
to solicit input.
Was the campaign only in RMS' mind? In one ``personal message'' to
the Tcl community, Dr. Ousterhout wrote:
I'm enclosing below my "official blurb" on what is happening in my new
group at Sun. [....]
Here's the blurb:
The Tcl/Tk project that I'm heading at Sun has the long-term goal
of making Tcl and Tk into a universal scripting language for the
Internet.
One of the job postings was written this way:
_________________________
SUN LABS Tcl/Tk Project
_________________________
Sun Micorsystems Laboratories, Inc. is embarking on a new project
directed by Dr. John Ousterhout. Our goal is to make Tcl/Tk the
universal scripting language.
By all means, Dr. Ousterhout is forever free to clarify his meaning or
even simply change the stated goals of his project -- but RMS' posts
are very much responsive to a reasonable interpretation of what
Dr. Ousterhout wrote in the past.
But the news isn't all contentious. Dr. Ousterhout explains the
positive emphasis of his campaign this way:
I also hope that Tcl and Tk will become a universal scripting
language for the Internet, but I hope to do this by making Tcl
and Tk so attractive that people *want* to use them, not by
somehow preventing people from using alternatives.
In that case, Dr. Ousterhout, you should find the GNU extension
language plans very exciting, and i hope we can find ways to cooperate
more directly.
For example, a program that supports the GNU extension language will
be programmable using Rush or a Rush-like language, provided only that
a suitable translator has been written. (Rush is, in fact, already
written as a translator to Scheme).
Rush is semanticly and syntactictly very close to Tcl; the differences
are not likely to be noticed by many programmers. The performance of
Rush is generally superior to Tcl. It is quite plausible to view Rush
(or a Rush-like language) as a direction in which Tcl can evolve
smoothly. Upward compatability is a delicate matter, but I am sure
that, working together, we could handle it with the greatest
gentleness towards existing users.
I am at your disposal to discuss the possibility of cooperation off
line. (Our offices are within minutes of each other).
-t
Raul Deluth Miller
. Ps: the real reason this "text on programming" doesn't talk about
. arrays is there is no good way to "do" arrays in functional
. programming, even though arrays are the single most useful
. structures in real programming; hence my irritation.
watserv1.waterloo.edu: languages/apl/j
Raul D. Miller n =: p*q NB. 9<##:##:n [.large prime p, q
<rock...@nova.umd.edu> y =: n&|&(*&x)^:e 1 NB. -.1 e.e e.&factors<:p,q [.e<n
NB. public e, n, y
x -: n&|&(*&y)^:d 1 NB. 1=(d*e)+.p*&<:q
Robert Willard
: (sigh... here we go again)
With all due respect (and IMHO much is due to both parties involved), I
would suggest that such discussions be moved to a new newsgroup. Perhaps
comp.lang.politics would be appropriate.
Mark C. Allman
A recent note from <rjw...@bcstec.ca.boeing.com (Robert Willard)> speaks volumes:
>With all due respect (and IMHO much is due to both parties involved), I
>would suggest that such discussions be moved to a new newsgroup. Perhaps
>comp.lang.politics would be appropriate.
Is arguing ALL you people have to do???? Please quit cluttering up everyone's
mailbox.
-- Mark C. Allman
-- Grad. Student, Physics, Univ. of Houston (Stat. Mech.)
-- gh...@ghost.neosoft.com