My theory is that scripting languages were borne out of the "computer
phobia" phenomenon.  They were built for people who were "afraid to
touch" the computer, other than to run the application (some people
wouldn't even do that; they would use a mechanical word-processor, for
example, with all of its breakdowns, rather than use one on a computer,
simply because "it is a _computer_".  The scripting languages tended to be
"dumbed down" to meet (not these people's expectations, but) these people's
fears.  Well, that computer phobia phenomenon has died down quite a bit,
but the scripting languages live on, mainly due to their usefulness as
an end-around to the compile/link/load/go process.  It's interesting to
watch a dumbed-down scripting language grow ad hoc to meet the challenge
of a group of people whose fear shrinks, and who want ever more features.
As the power of scripting languages approaches that of general purpose
languages, it does so from the direction of ad hoc design, and thus tends
to turn the stomachs of those who have always used gp languages for gp
problems.

  The fact that people here get flamed for being idiots and people in the Perl
  culture are flamed for being smart should be a pretty good indicator.

--
Erik Naggum, Oslo, Norway

Eventually, of course, users will want to do things that the originator
of the scripting language never thought of, or the task (and the machine
feature set) at hand will change, and then you need a general-purpose
programming language, and a seamless way of moving between the
task-specific and fully-expressive modes. If this looks like Greenspun's
Law yet again, that's because it is, but there's an important twist.
Those limited, task-specific sublanguages are crucial to actually
getting the job done -- you don't want' people starting from scratch all
the time just because it's more elegant. So every CL program that solves
a real-world problem ends up containing a buggy, incomplete version of
some domain-specific scripting language...

[two "scripting languages" that should be reviewed for their place
in history: hypercard and newtonscript...]

Scripting is a kind of high-level programming where you depend on
applications already written to perform complex functions, and the
script drives those applications, or glues them together.  In modern
terms, you could easily argue that nearly all programming is essentially
this.  We depend (and *should* depend) on libraries and OS calls to do
nearly everything that is not specifically new function.  So even at
level that you'd *never* call "scripting", you're doing a lot of exactly
what "scripting" is.  

On the other side, "scripting" languages these days are, more often than
not, general purpose, even if they don't make as good general purpose
languages as others.  Because there are always things that a scripter
thinks of to do that the original author of the scriptable software
didn't, and having a Turing-complete semi-general purpose scripting
language makes just about every add on possible without having to
redevelop large pieces of the software in question.

I use applescript a lot, and it's a classic example of a modern
scripting language.  But it is turing complete, and I've certainly
written hacks in it that don't depend on any other software (other than
the applescript interpreter).  Is it a scripting language or a
programming language?  Well, I'd call it a scripting language because
its *strength* is driving other apps.  It is a programming language, and
I've used worse, but it's significantly less powerful than Lisp, and
lacks some essential features that would be required to compete with
java/C/etc. as a general purpose language (e.g. I can create a hash
table object, but it doesn't actually do O(1) lookups and k is large,
and I have to work around arbitrary interpreter limitations before
there's all that much data in it.).

OTOH, I script apps in AS a lot, so I'm very familiar with it and have
an IDE open all the time.  Thus I find myself writing small general
purpose programs in it fairly often.  For anything that that doesn't
depend on driving other software, and which requires a large data
structure, or would take more than a day or so to program, I use a
different language.

Anyway, I think it's a somewhat useful distinction, but it's loose and
becoming looser.  I also consider it a "kind of" relationship.
Scripting is a kind of programming, and a scripting language is a kind
of programming language.

Michael

btw, since a operating system tend to be a glue between the user and
some hardware which do most of the critical work, maybe a operating
system is simply a collection of scripts in some sense?

mvh,

| The principal lesson of Emacs is that a language for extensions should
| not be a mere "extension language". It should be a real programming
| language, designed for writing and maintaining substantial programs.
| Because people will want to do that!

   http://www.vanderburg.org/Tcl/war/0000.html

Duane Rettig continues:

| In cases where PHP has been determined to be inappropriate, what
| language should be used? There is considerable choice here; few
| languages are as bad as PHP for doing serious development work.

   http://www.ukuug.org/events/linux2002/papers/html/php/index.html

-dan

--

* Michael Sullivan
| I think there's a distinction, but (a) it's not hard and fast, and (b)
| scripting is a subset of programming -- they are not completely different
| things.

  This thread puzzles me greatly, so I have to ask a pretty stupid question
  because I realize that I may have an invalid assumption as to what people
  mean by something I have taken for granted for years.

  If I say that there is a distinction between A and B, I intend to communicate
  that the set of A things and the set of B things are not identical, as they
  would be if there were no distinction between A and B, but this does not
  preclude one from being a subset of the other.  Quite the contrary.  A subset
  relation is distinctive.  I read several responses to this question to mean
  that others would say that there is a distinction between A and B only if no
  A is a B and no B is an A and that a subset relation is /not/ a distinction.
  I take the many responses that people have posted about this to mean that a
  distinction exists between A and B if they are at the same hierarchical level
  in some model or other and that subset relations are a different kind of
  animal altogether.  Since I have no other word for this subset relation than
  distinction, I wonder how this works.

  For instance, one might say that there is a distinction between a computer
  and a personal computer in that a computer is not restricted to one-person
  use, but a personal computer is.  Some people here would, I presume, answer
  "no, there is no distinction, personal computers are a subset of computers",
  which does not communicate "there is no distinction" = "they are the same",
  but "there is no distinction" = "subsets are a different kind of not-sameness
  than distinctions".  In this model of usage, there would be a distinction
  between a server and a personal computer, because both are refinements of the
  concept "computer" at some notion of the "same level", but there would not be
  a /distinction/ between a computer as the superclass and the server as the
  subclass.

  Since we have discussed hierarchical classifications and ontologies lately,
  as well, I wonder if there is not something deeper going on in here that has
  some influence on how people approach classification.  It may be useful to
  know that I had first approached classification in libraries before I found
  books about computers in the libary whose math section I had devoured, and
  that I was taught the basic ten classes in Dewey before I was ten myself.
  The reason I brought up Dewey's just recently was that the 5th Norwegian
  edition had been completed and announced with some fanfare in circles I had
  not worked with for several years; yet it occurs me that much of my thinking
  about concepts and ontologies has indeed been shaped by Dewey's in ways I
  have not been fully aware.  For instance, a simple concept like an automobile
  could have one classification and the particular kind of automobile known as
  sports utility vehicles another, subordinate to automobiles, but people who
  object to distinction would presumably never classify anything as automobiles
  but would have to classify non-SUVs in a new, separate class and would be
  unhappy to find that there was a category for SUVs but that both other kinds
  of automobiles and automobiles in general had a class that appeared both to
  subsume SUVs and not.  The same problem comes up with comp.lang.lisp and
  Common Lisp and Scheme.  Some people have strongly objected to requests not
  to discuss Scheme matters in the "general Lisp newsgroup" even though Scheme
  has its own specialized newsgroup, because, presumably, it should be allowed
  since the general category subsumes the more specific.  It occurs to me that
  the objections have been over fundamental differences in concept-formation
  and hierarchical organization of knowledge rather than over "is Scheme a
  Lisp?".

  As I started to wonder about this, I also realized that some of my failure to
  understand objections to classifications that have appeared absolutely
  natural to me may not be obvious without a similar background, and that the
  reason for what I see as ignorance and wheel-reinvention in the Semantic Web
  community may come from different schools of thought.  I do not, however,
  believe that I have been "brainwashed" by Dewey's classification but rather
  that I found it so amazingly natural that I internalized it effortlessly, but
  I have also not been obsessing about it and "thinking in Dewey numbers" when
  I read books or articles.  It has therefore not been conscious when I have
  organized my own 1350-volume library roughly along Dewey lines by having
  sections of my apartment dedicated to their own areas.  This has likewise not
  been conscious when I have objected to discussing some topic X in a newsgroup
  for super-X when X had its own newsgroup.

  From another point of view, that two things have similarities and might be
  the same for some purposes, does not mean that they are not distinct for
  other purposes, nor vice versa.  Therefore, "A and B are distinct" does not
  mean that A and B are not the same in /some/ respects, any more than "A and B
  are the same" does not mean that they are not distinct in /some/ respects,
  for the obvious reason that no two things are identical.  Likewise, I would
  tend to discuss some specialized class of things in their more general
  setting without having a strong urge to point out that the more general
  meaning is intended or that I did not mean the specialized class only, but I
  do object to statements that attempt to make attributes stick to the broader
  class when the observations have only been about a narrower class or about
  individuals whose classification is all but certain, mostly because such
  statements are noxious pollutants in the information flow; they are the seeds
  of many false yet unchecked conclusions.  This applies to statements about
  Scheme that are imputed to Lisp in general, to statements about the self-
  destructive black ghetto culture imputed to blacks in general, to statements
  about the most dangerous president of the United States ever imputed to the
  American public, to statements about Nigerian §419 advance fee scams and
  African businesses in general, and other similar rather obvious instances.

  Subsets and supersets appear to pose a philosophical problem of some sort.  I
  have obviously not solved the problem, since I am only beginning to become
  aware of it, but I wonder if most people are equally unaware of how they deal
  with classification.  The discussion on the use of Dewey in particular showed
  that there are implicit assumptions at work at some level and that I have
  just been "lucky" to have been exposed to the Dewey model and perhaps other
  models that are very similar to it.  This model obviously works sufficiently
  well that I have not (hitherto) noticed any fault in it.  I can only imagine
  that most other people have the same observations for their models.  Worse, I
  have not found any serious discussion of this topic in the philosophy I have
  read other than the exposition of systems of concept-formation that I already
  accepted.  I may even have "chosen" those philosophy texts because they fit
  some early, pre-conceptual model that made sense.  Others I have talked to
  over the years have also reported that they picked their philosophies and
  even (non-inherited) religions based on reading lots of material and slowly
  focusing on things that made more sense than others.  I have long known that
  what makes sense to people is a part of their personality but fairly recently
  (a couple years) became aware that how a person forms expectations and deals
  with frustrations thereof is also a personality trait.  I do not exclude the
  possibility that concept-formation and classification are also personality
  traits or similarly deeply ingrained and unquestioned, as (1) it appears to
  be hard to parameterize (no philosophers I know of have described and used
  more than one model, but I would appreciate evidence of counter-instances),
  (2) many deeply rooted disagreements and misunderstandings appear to be
  explainable in terms of differences in classification models or systems, (3)
  most discussions about ontology breaks down with people choosing a small set
  of diverging ways.

  To wrap this up, it appears that a simple question such as asking if there
  are distinctions or a distinction between scripting languages and programming
  languages implicitly requests the responder to reveal parts of his scheme for
  classification or theories of concept-formation, but since this is such a
  deepl ingrained issue, disagreements and confusions result when the nature of
  distinction and sameness are not made explicit.  The question might therefore
  more productively be posed to learn how people have classified languages in
  either category, and the to see whether any of the four possible categories
  (A) non-scripting, non-programming languages, (B) scripting, non-programming
  languages, (C) non-scripting programming languages, (D) scripting programming
  languages, are empty and under which conditions each language would be put in
  each category.  For instance, I would put HTML in category A, but I wonder
  what those who consider it a programming would do with the scripting aspect.

  I apologize for the fuzzyiness of the preceding, I realize that I may be
  groping in the dark for the proverbial black cat in the coal cellar that may
  or may not be there.

whether or not it is productive is subject to the vagaries in conception
you mention.  let's just say it's nice to be entertained by diversity in
expression (for the most part) and that trying to stay patient w/ those
still growing their understanding is a challenge often (but not always!)
well-rewarded.

  This just goes to show how differently we conceptualize.  I do not see a
  runtime system as an application in itself.  If, however, the application
  merely runs inside the Lisp system which is also an application from the
  operating system's point of view, the question becomes whether the Lisp
  application does the same as the LIsp system.  E.g., Emacs Lisp usually only
  enhance the operation of Emacs and relies heavily on the Emacs "machine" to
  do its work.  Therefore, Emacs Lisp is a scripting language with programming
  language capabilities, but its programs could not run without Emacs running
  first, and Emacs is a stand-alone application.  A Common Lisp system may only
  provide the environment for a Common Lisp program but would not do anything
  useful by itself.  In a sense, it has the same capacity as a shell to launch
  applications to run "in" the system, but the same way that the interactive
  loop acts as a job control language the way the shell does for Unix programs,
  the applications launched from the Common Lisp REPL are as distinct from the
  REPL as Unix programs are from the shell.

  To refine the distinction I see, a programming language has to assume control
  over the execution of the application and terminate execution when it
  returns, whereas a scripting language is subject to the control exercised by
  another application, which does not necessarily terminate when the script
  terminates.

  Now, I believe there is a distinction from some deep-rooted sense of there
  being a difference in the amount of machinery needed to support a language in
  addition to the notion of an intention of the language to be compiled into
  native machine code.  Another point of view is the intended efficiency of the
  resulting code.  Scripting languages are not expected to be compiled into
  native code that assumes full control over the machine for an extended period
  of time and only calls upon the support system, but rather the reverse: a
  script is run only occasionally as part of another application that has such
  full control.

  However, I am sure there are people who consider Common Lisp to be a "glue
  language" when they observe that low-level functionality is implemented in C
  and the higher-level application is written in Common Lisp.

--
Erik Naggum, Oslo, Norway

We had a term in test engineering - the best test machine was the one that
was "gorilla-proof".  It's not that we thought of operators on the
manufacturing floor as gorillas, but when we would get a tester back from
there to be repaired, it certainly seemed as if gorillas had been loosed
on the poor tester.  This applied both to the physical tester and to the
software as well.  Gorilla-proofing involves overbuilding the testers
with interlocks, safety devices, and warnings that are designed to focus
the operator's attention strictly to the task at hand.  Similarly, a
tester scripting language restricts the programming (where appropriate)
to those operations related to the test.

I have noticed that "sets" (as in set-theory) and "classes" (i.e. a
set of attributes/attribute-values) form some sort of parallel
notations that are sometimes hard for me to keep straight when I need
to use both.

e.g: (progressing from attributes to set-membership)

A is a name for things that are either red *or* blue.
=>
x is an A -> (x is red) *or* (x is blue)
=>
Any A is in the set of red things *or* in the set of blue things.
=>
The set of red things *and* the set of blue things (union) will
contain all  A.

My intuition (i.e. unthinking normal self) will vacillate from one way
of classifying to the other, sometimes mixing both with horrible
results. Note the switch of conjunction from *or* (logical)  to *and*
(set-theoretical union).

Now, what does "Lisp is a scripting language and a programming
language" mean? Is it an impossible (always false) statement, or does
it mean something ? If it means something, what does it mean? Lisp has
the "scripting" attribute ? Lisp is a member of the "scripting" class
? Some instance (implementation) of lisp can be used for scripting?

The last one might translate into:

A) some members of the lisp class of languages have the scripting
attribute.

B)some languages with lisp-like attributes exist in the class of
scripting languages.

( C and D follows with all attributes or all class-membership)

  [ other good examples elided ... ]

I think that in communicating distinctions/discriminations/knowledge, it is
always important to include the context for which the communication is
taking place.  In the case of making distinctions, where you've demonstrated
that we all tend to fall short of being clear on these, we should be
including the _category_ for which we are trying to distinguish two things,
rather than assuming that everyone here obviously knows on what levels the
distinctions are being made.

"A scripting language generally has these properties:

* Source code is present at run time in production system.

* Explicit compilation or linking phases are not needed.

* Variables, functions, and methods typically do not require type
declarations. There are automated conversions or equivalence between types.

* The ability to map external components to the semantics of the
built-in types and/or some component-based protocol (ie. SWIG, OCX, COM)

* Very powerful built-in types - typically a dictionary, a set, a
literal type that can act as either numerics or strings.

* The ability to arbitrarily combine types into heterogeneous structures

* Very powerful iteration semantics map, filter, lambda, etc. (missing
in VB, Tcl, ... )

* You can generate, load, and interpret source code at run time through
an eval function. (Perl, Python, Tcl, shell scripts; missing in VB

* Language interpreter can be embedded within another application,
allowing users to automate application operation, to provide customized
handling of application events, or to define new application functions
using the scripting language."

Can you say "Greenspun"?

However, there is also a continuing debate on the wiki about the
usefulness of this definition...

I don't think you can find a reasonable definition or classification of
scripting languages vs. programming languages. We're dealing with social
phenomena here, and they typically resist certain kinds of "fixed"
definitions.

In Ian Hacking's terminology, scripting/programming langauges are
"interactive kinds" (as opposed to "indifferent kinds", see "The Social
Construction of What?", http://makeashorterlink.com/?R308537D1). Here is
a very abridged version: As soon as you define "scripting language",
programmers will change their own behavior, they will welcome or reject
the definition, and language designers will change their languages based
on that definition, either to support or to attack the definition; so
sooner or later the definition is not valid anymore and needs to be
changed itself.

Hacking calls this interaction between a definition and its
"application" (that changes the definition) "classificatory looping".

This may be frustrating to people who are trained in mathematics or
natural sciences (that mainly deal with indifferent kinds). However, I
am convinced that we have to live with the fact that computer science is
essentially a social science, regardless that it also uses _some_
methodologies borrowed from mathematics.

Pascal

I rather see it like this. Mainstream programming languages tend to do
two things: (a) They involve some kind of static type system and other
ways to deliberately restrict their expressive power, in order to make
them "safer", more "reliable" and "more efficient". (b) They are
inclined to be used for creating components, i.e. more or less
monolotihic building blocks.

So there's where scripting languages come into play: (a) They are mainly
used for glueing components, and not for developing complete
applications/components. (b) They lack static typing. Obviously people
can live without the "safety", "reliability" and "efficiency"
purportedly provided by static typing, because "it's just the glue".

Again, let me stress that I try to describe the social aspects of these
terms and not their technical merits. And again, I am not 100% sure
because I might have oversimplified things here.

In order to do some kind of reality check: How well is Common Lisp
suited for component-oriented software? My impression is that Common
Lisp (and Scheme) don't provide good means for developing components. Of
course, you can implement CORBA or SOAP wrappers for your applications,
but I guess these things don't feel natural in a Lisp environment. Am I
right? How easy is it to develop, for example, COM components?

Pascal

Many of our customers use CL precisely as a glue, to be able to
salvage many non-homogenous components from the bit bucket and
instead to allow them to work together with our product being the
glue.  I doubt that any of these customers would call CL a scripting
language, although some do use it that way.  But it can definitely be
used a glue, or as a control or dispatching program for legacy code.

-write a specification
-describe a design
-produce instructions for a computer program
-specify test procedures and run them
-specify rigorous domain specific vocabulary
-communicate ideas to other people
-clarify my thinking to myself

I think there are great benefits in viewing Lisp as a language, instead of the
limited "programming" language classification.

Wade

Consider:

CL-USER(1): (defvar *work* nil)
*WORK*
CL-USER(2): (compile (defun foo () (push 'did-some *work*)))
FOO
NIL
NIL
CL-USER(3): (compile (defun bar () (push 'did-some-more *work*)))
BAR
NIL
NIL
CL-USER(4): (progn
              (foo)
              (bar)
              (reverse *work*))
(DID-SOME DID-SOME-MORE)
CL-USER(5):

Foo and bar are compiled functions (programs - unrelated complex
applications, if you will, for some value of complex), and the progn
is the script. So Lisp is a scripting language (among many other things).

It's interesting how many people answered a different question from the
one I actually asked.  The question many people are answering in this
thread is: "Is there a distinction between scrtiping languages and
programming languages (and if so what is it)?"  But the question I posed
was "Is there a USEFUL distinction?"  I think it's pretty clear that there
*are* distinctions (if nothing else, the fact that people make the
distinction is itself a distinction).  But whether any of the distinctions
have practical utility is an entirely different question, one to which I
join Tim in answering with a resounding "NO!".

The fact that I found gross mistakes in the FAQ, and met not one shred
of support for my position from anyone, is an attitude indicator that
cannot be easily dismissed.

In comp.lang.lisp, you *would* find support in advocating correct
programming, rather than opposition in the form of claims that it's
the user's fault for supplying certain kinds of data to the program,
which don't conform to your religious beliefs.

The comp.unix.shell FAQ is *still* broken. Casually scanning through
it, I come upon 2.6:

  for f in *.foo; do
    base=`basename $f .foo`
    mv $f $base.bar
  done

It should really be:

  for f in *.foo ; do
    base="$(basename -- "$f" .foo)"
    mv -- "$f" "$base.bar"
  done

The backtick syntax is obsolescent; the modern POSIX equivalent is $(
... ). You need to quote the argument to basename, as well as the
expansion, because there could be spaces in the expansion of $f. And
of course $f could expand to something that looks like a command line
option to basename or mv.

[...]