Programming language evaluation criteria

Jim Howatt

unread,

Dec 12, 1992, 10:18:18 AM12/12/92

to

I'm teaching a comparative programming languages class next quarter using
Sebesta's second edition as the primary text. In the first chapter he lists
several language selection/evaluation criteria; but the list "feels"
uncomfortably incomplete and subjective. In view of the flame wars that we
often see among proponents of different languages, I'd like to present my
students with a set of reasonably objective criteria for selecting a
language for a given task.

I've already been thru a half-dozen programming language texts and many
articles, but think that there must be more criteria to consider, especially
from practitioners who've had to evaluate and select languages for real
projects.

I'd appreciate any insight and pointers to insight that you may have.

Thanks in advance for your help. As always, if there's sufficient interest,
I'll be more than happy to post a summary of responses.

Dr. James W. (Jim) Howatt
Department of Computer Science
139 Engineering and Computing Center
St. Cloud State University
St. Cloud, MN 56301-4498
email: jho...@eeyore.StCloud.MSUS.edu

John Leslie

unread,

Dec 12, 1992, 8:12:34 PM12/12/92

to

jho...@eeyore.stcloud.msus.edu (Jim Howatt) writes:

> I'm teaching a comparative programming languages class next quarter using
> Sebesta's second edition as the primary text.

I've never read Sebesta, first or second; and frankly I'm not about to
drop things to read it. If Jim needs responses particular to it, he'd
best post some specifics.

> In the first chapter he lists several language selection/evaluation
> criteria; but the list "feels" uncomfortably incomplete and subjective.

In the real world, the list of criteria has one item -- politics.

> In view of the flame wars that we often see among proponents of
> different languages, I'd like to present my students with a set of
> reasonably objective criteria for selecting a language for a given task.

Why?

> I've already been thru a half-dozen programming language texts and many
> articles, but think that there must be more criteria to consider, especially
> from practitioners who've had to evaluate and select languages for real
> projects.

I've seen literally hundreds of criteria, most often generated after
the fact to justify a (usually reasonable) choice already made. I think
such a list is nearly useless, unless it has stood the fire of flame-wars
on the Net. I _would_ welcome some submissions, open to flaming, because
we _might_ be ready to reach consensus on a few criteria...

> I'd appreciate any insight and pointers to insight that you may have.

I really think you're barking up the wrong tree. The first thing a
course on comparative languages should teach is that most algorithms can
be programmed in any of them. The point _shouldn't_ be that Algorithm A
is best programmed in Language L; rather students should get a feeling
for tradeoffs -- Language Q may not be ideal for Algorithm A, but the
penalty is in the neighborhood of H hours, which might be made up elsewhere
in the project.

Students should leave your course with a feeling that most times the
savings from choosing the right language are less than the time they'd
spend fighting the political battle to use it.

John Leslie <jo...@jlc.mv.com>

PS: I've set followup to comp.edu, where I think the discussion belongs.

Jacob Gore

unread,

Dec 13, 1992, 1:35:58 PM12/13/92

to

If it's pragmatics you want in language selection, here's the list:

1. See what languages the people who fund your project or pay your
salary want you to use.

2. See which of those have reliable implementations available and
affordable.

3. See which of those your teammates know or are willing to learn.

4. See which of those your teammates want to use.

5. Use it (learn it, if necessary).

Having got this out of the way, I explain to the students in my
comparative language class that the reason they need the class is not
to learn a bunch of languages, but to discover a variety of problem
solving approaches. They do that by doing exercises in ML (a
math-looking strongly-typed functional language), Scheme (an
assembly-language-looking dynamically-typed functional language where
one can execute data structures one builds), and Objective-C (a
dynamically typed but strongly typable traditional/object-oriented
hybrid language -- my first choice would have been Eiffel, then
Smalltalk, but I have neither handy). I give assignments that are
easy to do if you let yourself flow with the new (to most of the
students) paradigm, but suffer greatly if you try to do things the old
way. (If time permits, we also do Prolog.)

Later, when they are forced to use whatever language filters through
the first four lines of the above list, they will (hopefully) be able
to reason along the following lines: "Well, this problem would have
been a two-liner in ML, but the language I have to use is K&R C with
no vararg library. Still, I can try using recursive functions without
side-effects and that will simplify the data structure needed, so
here's how I can design this........"

Learning a language for the sake of learning a language is about as
stimulating as learning all the legal chess moves. Picking a language
within one paradigm is like picking the shape and color of the pieces
and the board: the choice will indeed make it easier or harder to
play...but it is all insignificant if you are a bad strategist.
Computer science education should be about becoming a good strategist.
Exposure to other paradigms is like playing other strategy games: it
is usually not directly translatable into chess moves, but it provides
enlightening insights.

Sorry for the unsolicited preaching, Jim, but I'm a big believer in
the "teach concepts, illustrate with specifics" approach (as opposed
to "teach marketable skills, leave concepts for grad school" approach,
whose proponents always want me to teach some bandwagon language. As
David Gries puts it (don't know if he quotes someone else): "program
INTO the language, not IN it." I do not for a moment think that
you're in the bandwagon language camp (if you were, you probably
wouldn't be thinking hard about language selection criteria), but the
comparative language course is such a wonderful opportunity to make
students reevaluate and enrich their bag of programming techniques...

Jacob
(a horrible chess player, actually:-)

Jacob Gore, Eastern NM U. Ja...@BlackBox.ENMU.Edu | Ja...@Gore.Com
Member of the League for Programming Freedom (LPF)--l...@uunet.uu.net

George C. Harrison, Norfolk State University

unread,

Dec 16, 1992, 4:30:40 AM12/16/92

to

In article <1992Dec12....@msus1.msus.edu>, jho...@eeyore.stcloud.msus.edu (Jim Howatt) writes:
> I'm teaching a comparative programming languages class next quarter using
> Sebesta's second edition as the primary text. In the first chapter he lists
> several language selection/evaluation criteria; but the list "feels"
> uncomfortably incomplete and subjective. In view of the flame wars that we
> often see among proponents of different languages, I'd like to present my
> students with a set of reasonably objective criteria for selecting a
> language for a given task.

..

> Dr. James W. (Jim) Howatt
> Department of Computer Science
> 139 Engineering and Computing Center
> St. Cloud State University

Objective criteria often have little to do with an implementation choice;
often the reasons for selecting the language are subjective. Anyway, here's my
2 cents:

1. Is the language appropriate to the application?
2. Availability of compilers/support tools...?
3. Cost of #2...?
4. Ease of learning...?
5. Current experience of programmers...?
6. Does the language support your disign style: Object Oriented, Top-Down,
Abstract Data Types, etc. (and mixes thereof)?
7. Does the customer DEMAND that a certain language be used?
8. Maintainability support of the code...?
9. Speed and Size of final executable (very compiler-dependent)...?
10. Do the language's constructs fit the design?

There are many others, but the bottom line is that the "quality" of a language
is often application-experience dependent. Perhaps some criteria build around
the "style" of the COnstructive COst MOdel (COCOMO) in software engineering
might be an interesting way of evaluating a language.

Scott Mcgregor

unread,

Dec 18, 1992, 2:08:04 AM12/18/92

to

One little remembered TECHNICAL reason for selecting or rejecting a
programming language is how easy it is to make a coding mistake in the
language which you don't discover (this is in part related to how
forgiving the compiler is). You can even break that down to a measure
of the probability of making a mistake and the probability of catching
it. The cost of detecting defects, and of fixing them are both
relatively high, so languages which limit or prevent many defects, or
which ensure their recognition can be a business advantage. The above
could be a justification for languages like Prograph which make
impossible a whole set of syntactic errors.

One little remembered BUSINESS reason for selecting or rejecting a
programming language is the availability of trained programmers in
that language, the cost and time of training an existing programmer in
that new language and the opportunity cost of doing the training
instead of using a language they already knew. I think this accounts
for the continued use of COBOL and FORTRAN in many settings.

--

Scott L. McGregor mcgr...@netcom.com
President tel: 408-985-1824
Prescient Software, Inc. fax: 408-985-1936
3494 Yuba Avenue
San Jose, CA 95117-2967

Prescient Software sells Merge Ahead, the tool for Merging Text or Code and
offers consulting & training in project management and design for usability.