Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Oldie (or Oldbie?) Seeks Help in New World!

25 views
Skip to first unread message

aptpupil

unread,
Jul 14, 2001, 7:02:39 PM7/14/01
to
Imagine that I am a Rip Van Winkle. I was raised on the workstation
Unixes of 1980s academia, did a fair amount of C programming, then
did a lot of Lisp programming (but not AI and only a little CLOS) for
many years, particularly on Symbolics (+R.I.P.+).

Then I went to sleep for a long time. (Or into a coma for half a
decade...or into the Peace Corps...or to Walden Pond like Thoreau
to recover from burnout...or something like that.)

Now I wake up. (Or return, as the case may be.)

The Internet and the Web has exploded. Communications has become
the hot area, rather than computers. Optical networking is hot,
object orientation is [c]old. Client-server is out,
"web programming" and "application server" seems to be in.
Lucid is gone (+R.I.P.+) but Franz is still alive, for some inexplicable
reason. (Pray, what?) Dylan lost out to Java. Smalltalk is
still just that.

Richard Gabriel's Worse-is-Better vs. The-Right-Thing
philosophical analysis is but a distant memory, although the
effects of the triumph of Worse-is-Better are pervasive
and detectable as background cosmic radiation by anyone with
Oldie L-band radar.

In the bookstore, I am amazed at the miles and miles of computer
books: Java this, Java that, HTML, Dynamic HTML, XHTML, XSLT, XML,
XML and This, XML and That (e.g., Java, UML, Data Warehousing,
you name It and there is an "XML and It" book),
Visual Basic, ColdFusion, Delphi, Perl, PHP, Python, Oracle,
Oracle, Oracle, Oracle, Oracle, O'Reilly books, Microsoft books,
Cisco books, Teach Yourself Beginning Professional Inside
Implementing XML on Apache Samba Basket Weaving in 24 Hours
For Dummies Unleashed Distilled By Example, etc., etc., etc.,
ad infinitum.

Help! The mind boggles in this Brave New World.

More to the point, I find myself in the Real World(tm), having
to replace an antiquated collection of programs written in Clipper,
running
on Novell. (Apparently, Clipper was the popular database and Novell
the popular networking system of the 1980s PC era, out there on the
Fruited Plain, outside of the twin ivory towers of Academia and High
Tech).
This legacy system is not some fancy, brainy AI-ish planning/scheduling
thing
that always seems to be what gets trotted out as examples of Lisp
success in the Real World, but a bread-and-butter "mission-critical"
"customer information" system.

(Think of, say, ... the billing, accounting, and operations programs
of your friendly electric or gas or water company, Timbuktu Utilities.)

But there is an unusual twist: Instead of having to work for
Pointy Hair Boss, I find that I *am* da Boss! But I have
programmers and an "IT Manager" who have known nothing but
Microsoft, Clipper, FoxPro, Visual Basic and such all their lives.
Naturally, they want to jump onto the Java, Oracle, and NT
bandwagons. Even just this is a big step forward for them.

I can probably handle getting Timbuktu U. to move from Novell to
Unix (GNU Linux or FreeBSD) instead of NT, at least on the servers.
But I have no database experience to be able to tell Oracle from Eve.
And beyond the question of what database to use (Oracle or MySQL or
PostgreSQL? relational or OODB?), I am at a loss as to what the
Big Picture, the overall software architecture and approach, should be.

For example, is Oracle and all the fancy CASE tools around it---Oracle
Designer, Oracle JDeveloper (J=Java, presumably), Oracle
Forms Developer, et al---the way to go, as the IT Manager wants?
Is there any Lispy alternative to this seductive morass of
(database-oriented?) "visual process modeling", "rapid application
development", "automatic code generating" tools? Or are they
actually pretty good and powerful, and I am just being overly
skeptical or cynical?

Bandwagons being what they are, I can understand IT people's
great desire to jump on them and eventually get "certified" on
them because they make one more valuable in today's Microsoft-,
Oracle-, and Java-oriented job market.

Should we bother with UML and Rational? Or is that just a waste
of time and a potential brain damager, as some posts in an
earlier UML thread seemed to suggest. [I am just starting to
read this and other comp.* newsgroups; I just woke up, remember.]

I suppose there are worse choices than Oracle and Java, but is
it possible to do better? In particular, is there a role for
Lisp and if so where exactly? Is it possible to do worse by
going Lisp?

If one had free hand to set technical direction (no Pointy Hair
heads above, but maybe below), but limited time, budget, and
average non-Lisp programmers to start with (but who might be
open to new horizons if they could be persuaded it was in their
self-interst), what would the ideal software architecture
and approach be?

----------

Mainstream (non-Lisp) languages and tools for the average
corporate drone programmers; Lisp only for the few, highly skilled,
anointed ones. Is this a valid, prescriptive dichotomy; or just
an artifact, a self-reinforcing coincidence, and a myth?

----------

So far, since awaking from my long sleep, I have read:
CLOS vs. Java papers in Franz's web site
In http://www.paulgraham.com/ :
"Beating the Averages"
"Lisp for Web-Based Applications"

The language comparisons are all well and good but not really
news to me. I kinda knew that before. Paul Graham's articles
are very helpful and more of what I am looking for. Are there
any other writings in this vein?

(Interestingly, Graham doesn't say anything about whether there
is any database in his system. Is there? Where does all
that data go---the user-built store web sites and the
data associated with the customers of the each store?)

In other words, The Big Picture and Where Lisp Fits In. How best
to position Lisp among the various software blobs and layers in
today's complicated world---distributed database, web server,
app server, GUI, security, network directory, VPN, and what not.
What do you do in Lisp and what do you do in the database or
elsewhere. How do you do the GUI. I assume we can forget about
CLIM since web programming is the fashion today, rather than writing
"client" programs with their own GUI, right?

Are there any other recent writings that, like Paul Graham's, gives
insight into---a decoded view of---a more palatable introduction
to---some
piece of the non-Lisp world from a Lisp perspective and sensibility.
Say, databases and database programming. I leafed through some books
on Oracle CASE tools and on UML and felt like throwing up. I couldn't
help but think, "Hmmm, feels like a whole other country". And not a
pretty one at that.

Lastly, is there any open source, "industrial strength", high
performance,
general purpose Lisp environment today? Or is Allegro CL pretty much
THE choice (as I assume it is) for "mission critical" deployment
and richness of development environment. Put another way, is there any
free Lisp today that can complete the quintessential sentence,
"No one ever got fired for choosing ..."? (Assuming, of course, that
there is *any* Lisp that can.)

Thanks for any suggestions/comments!

Wade Humeniuk

unread,
Jul 14, 2001, 8:19:16 PM7/14/01
to
>
> Thanks for any suggestions/comments!

Lisp still is better than all those other things.

With a free hand I would simply.

Decide to use Lisp.
Stay on course.
Commit to using Lisp.
Stay on course.
Stop listening to the nay-sayers.
Stay on course.
_Hire_ some Lisp Developers!! (I am looking for work!)
...
Pick a commercial Lisp development system. LispWorks or Franz ACL or maybe
even Open Genera (it sounds good).
...
Pick A Platform. Something robust.
...
Pick a Database.
...

Wirte Code. Write Code. Write Code.
...
Deploy, Deploy, Deploy.
...

Succeed!

Wade


Kent M Pitman

unread,
Jul 14, 2001, 8:27:32 PM7/14/01
to
aptpupil <aptpu...@yahoo.com> writes:

> Lastly, is there any open source, "industrial strength", high
> performance, general purpose Lisp environment today? Or is Allegro
> CL pretty much THE choice (as I assume it is) for "mission critical"
> deployment and richness of development environment.

Perhaps while you were asleep, Mr. Wrinkle, you missed the arrival of
Harlequin/Xanalys on the scene. While it's certainly true that
Franz/Allegro is a fine product, it's far from true that it's the only
choice for commercially-supported "mission critical" deployment.

Likewise MCL might have slipped in while you were snoozing. It also is
not open source but is, I'm told by all who've used it, an excellent
option as well if you have a Mac around.

> Put another way, is there any
> free Lisp today that can complete the quintessential sentence,
> "No one ever got fired for choosing ..."? (Assuming, of course, that
> there is *any* Lisp that can.)

I don't understand why choosing Allegro would keep you from getting fired,
nor why choosing LispWorks or MCL would get you fired. I thought people
were mostly fired for out-of-band things like inattention to duty,
insubordination, lying, cheating, or stealing. I thought people mostly got
laid off for failing to produce product, or for no reason at all (e.g.,
another arm of the company lost money but someone wants to "make up for it"
by deleting your arm of the company).

Products can be and are routinely produced by competent and attentive
people in any of these implementations. Most companies don't lay you off
for having produced product, though some do. Nothing is a guarantee.

(And there are open-source Lisps out of which people are making commercial
products.)

Btw, nothing says you have to deploy in the same environment you develop in.
On a number of occasions since the "fall" of Symbolics, I've done my
development work on Genera and deployed elsewhere. Likewise, I've done
development on a PC and deployed on a Unix or vice versa.
The qustion of what makes a good development environment is well-separated
from what makes a good deployment environment. Not developing in the same
environment you deliver in does create some additional QA burden, but it
may still be worth the trade.

> Thanks for any suggestions/comments!

Sure.

Kaz Kylheku

unread,
Jul 14, 2001, 8:56:45 PM7/14/01
to
In article <3B50CF80...@yahoo.com>, aptpupil wrote:
>But there is an unusual twist: Instead of having to work for
>Pointy Hair Boss, I find that I *am* da Boss! But I have
>programmers and an "IT Manager" who have known nothing but
>Microsoft, Clipper, FoxPro, Visual Basic and such all their lives.

I'd fire these people, and replace them with actual developers that know
some real programming languages like C, C++, Lisp, and--more
importantly--have some credible computing background.

>Naturally, they want to jump onto the Java, Oracle, and NT
>bandwagons. Even just this is a big step forward for them.

A FoxPro and Visual Basic programmer will have to undergo a brain
transplant to properly handle any kind of real development in an
intelligent programming language that takes into account the development
of computing beyond 1965. Since you are the boss, get rid of these
people, fast. Let them learn Java on someone else's time, and project.

>Bandwagons being what they are, I can understand IT people's
>great desire to jump on them and eventually get "certified" on
>them because they make one more valuable in today's Microsoft-,
>Oracle-, and Java-oriented job market.

Since you have been away for a long time, you need to learn
what some of these acronyms stand for, starting with
MCSE == Minesweeper Consultant and Solitaire Expert.

``IT people'' nowadays refers to people whose job it is to reboot a
server once in a while. These certifications that these people undergo
make no sense for software developers; they are basically system
administrator traininig in a particular vendor technology. That training
is largely a cash grab. Each year it becomes obsolete, when
some irrelevant factoids change, and the minions must learn the
new factoids to pass the new exam which costs $$$.

>Should we bother with UML and Rational? Or is that just a waste

Good luck trying explaining UML to some FoxPro and VB programmers.

For UML to make sense, one first has to know the first few things about
decomposing a system into classes. UML notation is fine, if a little too
slanted toward supporting every feature of C++. But do not be suckered
into buying the actual Rational process, or any related expensive-as-hell
software that does nothing.

Also forget about trying to generate code from a UML model, or trying to
maintain it coherent. When the drawings no longer represent the system,
file them away for posterity.

>of time and a potential brain damager, as some posts in an
>earlier UML thread seemed to suggest.

UML is a brain-damager if all you do is mentally translate from C++
and then draw the corresponding pictures. :) Use a subset of the
notation that makes sense, and use it in an actual design process.

In UML class diagrams, one can codify language-specific artifacts, such as
interfaces or abstract base classes. These could be used as concepts the
design of a CLOS program, but they would not be directly realized as code,
but rather they would denote cohesive collections of generic functions.

>Lastly, is there any open source, "industrial strength", high
>performance,
>general purpose Lisp environment today?

There is CMU Common Lisp (CMUCL), CLISP and GNU Common Lisp (GCL).
Why not investigate all three?

cbbr...@hex.net

unread,
Jul 14, 2001, 11:05:59 PM7/14/01
to
"Wade Humeniuk" <hume...@cadvision.com> writes:
> > Thanks for any suggestions/comments!
>
> Lisp still is better than all those other things.

Perhaps, but it may be difficult to enunciate _why_ to superiors,
which _is_ of some importance.

> With a free hand I would simply.
>
> Decide to use Lisp.
> Stay on course.
> Commit to using Lisp.
> Stay on course.
> Stop listening to the nay-sayers.
> Stay on course.
> _Hire_ some Lisp Developers!! (I am looking for work!)

> ....


> Pick a commercial Lisp development system. LispWorks or Franz ACL
> or maybe even Open Genera (it sounds good).

Is Open Genera still actively sold? I thought that the marketing
efforts seemed pretty non-existent, a little while ago, and what with
Compaq/Digital deep-sixing the Alpha platform, that would seem to put
it on its death bed...

> ....


> Pick A Platform. Something robust.

> ....
> Pick a Database.

Methinks that this (databases) is an area in which lies considerable
'discussability,' as well as likely a fair bit of controversy.

There's the "relational" way, with SQL, popular names like Oracle,
Sybase, Informix, and such. In some respects, SQL is pretty ugly;
it's not particularly elegant.

There are "network/object" databases, including the one that
integrates into Franz ACL; they _may_ have some aspects of "beauty,"
but two major demerits:

-> The Lisp-specific options are _definitely_ not portable; you
_instantly_ ally yourself to a sole vendor;

-> The "ODBMS" systems wind up taking a sort of
"network/hierarchical" approach (some shades of the COBOL Codasyl
stuff of yesteryear), where _any_ attempt to query is likely to be
_entirely_ specific to your application, and requires writing code
to get at _anything_. In contrast, the 'tabular' way of SQL
allows some more "algebraic/declarative" access to data.

Integrating with other enterprise systems is almost certainly going to
be easier with an SQL system, though data access from the Lisp side
will require some custom effort to build accessor/modifier functions.

The above obviously represents my own biased opinions; hopefully
bouncing some violent agreement/disagreement off of it might provide
some further perspective that would be of value.

Aside from all that, I find it marginally surprising/disappointing
that there haven't been any visible attempts to attach lower level DB
functionality.

I would _think_ that throwing in something like DBM or (better, as it
supports several data models, with hashing as well as B-trees)
Berkeley DB as a data store, and then building up data accessors on
top of this would be _very_ interesting.

There has been discussion by the PostgreSQL folks of replacing their
underlying data storage engine with Berkeley DB to improve some of the
characteristics of the RDBMS, and the folks that built MySQL finally
got "real transaction" support by virtue of replacing their own
storage engine with Berkeley DB. The point of this comment is that
Berkeley DB is clearly flexible/powerful enough to be used as a
(perhaps hidden) engine on top of which more sophisticated things can
be built; the same could readily be true for the way a CL system might
"abuse" it to attain desired DBMS functionality.

> ....


>
> Wirte Code. Write Code. Write Code.

> ....
> Deploy, Deploy, Deploy.
> ....
>
> Succeed!

--
(concatenate 'string "aa454" "@freenet.carleton.ca")
http://www.ntlug.org/~cbbrowne/unix.html
"Of course 5 years from now that will be different, but 5 years from
now everyone will be running free GNU on their 200 MIPS, 64M
SPARCstation-5." -- Andrew Tanenbaum, 1992.

raj

unread,
Jul 14, 2001, 11:41:22 PM7/14/01
to
>I have programmers and an "IT Manager" who have known nothing but
>Microsoft, Clipper, FoxPro, Visual Basic and such all their lives.
>Naturally, they want to jump onto the Java, Oracle, and NT
>bandwagons. Even just this is a big step forward for them.

Trying to introduce these mediocrities to Lisp sounds like a great way
to go over budget and have your project fail.

>Is there any Lispy alternative to this seductive morass of
>(database-oriented?) "visual process modeling", "rapid application
>development", "automatic code generating" tools?

Look at :
Uncommon SQL
"UncommonSQL is a database integration kit for CL, based on another CL
database library, MaiSQL. It provides a CommonSQL compatible
interface with both a functional SQL syntax, and a CLOS integrated
Object-to-Relational mapping. That's right, throw perfectly good CLOS
objects into an RDBMS. It is distributed under an MIT/X like
license
http://alpha.onshored.com/lisp-software/

Persistent Lisp.
http://sunsite.berkeley.edu/Dienst/UI/2.0/Describe/ncstrl.ucb/CSD-88-401

A Common LISP Hypermedia Server
http://www.ai.mit.edu/projects/iiip/doc/cl-http/home-page.html
http://www.ai.mit.edu/projects/iiip/doc/cl-http/server.html

>Should we bother with UML and Rational?

UML : Yes.
( On the other hand , I do not know of any UML products that support
Lisp.)
Rational : No.
(There are cheaper and better vendors.)

>Say, databases and database programming. I leafed through some books
>on Oracle CASE tools and on UML and felt like throwing up. I couldn't
>help but think, "Hmmm, feels like a whole other country". And not a
>pretty one at that.


Why not check out Mnesia and Erlang ?
http://www.erlang.org/faq/t1.html
Mnesia is a heavy duty real-time distributed database.

"Erlang makes a good starting ground for playing
with XML and FP. Those with Scheme or Lisp experience should find it
comfortable enough, and it's "non-parenthesized" enough not to turn
off others. "
http://www.erlang.org/faq/t1.html
http://www.xml.com/pub/a/2001/02/14/functional.html?page=3
In addition, DB3 sounds interesting:
"There is a large project going on implementing an object-oriented
database in Erlang. The database, which is called DB3, is main-memory
based"
http://www.ericsson.com/cslab/projects/erlang/sqlerlang/node85.html

>(Interestingly, Graham doesn't say anything about whether there
>is any database in his system. Is there? Where does all
>that data go---the user-built store web sites and the
>data associated with the customers of the each store?)

From a review of his ViaWeb product ( as it was called before Yahoo
bought it and added a Oracle backend and renamed it Yahoo Store )

"ViaWeb is not necessarily the panacea for online stores. For one,
the biggest store they have currently online is around 10,000 items.
Because you cannot connect ViaWeb's service to any "heavy-duty"
databases (such as SQL or Oracle) you're product database is limited.
So immediately, ViaWeb is not an option for those who want to build
large online stores."
http://www.meep.com/magazine/biz/ecom/viawebreview.html

raj

unread,
Jul 14, 2001, 11:49:49 PM7/14/01
to
On Sun, 15 Jul 2001 03:05:59 GMT, cbbr...@hex.net wrote:

> -> The Lisp-specific options are _definitely_ not portable; you
> _instantly_ ally yourself to a sole vendor;
>

>Integrating with other enterprise systems is almost certainly going to
>be easier with an SQL system, though data access from the Lisp side
>will require some custom effort to build accessor/modifier functions.

If Erlang is being considered:
Check:
http://www.ericsson.com/cslab/projects/erlang/sqlerlang/orc.html
"This report describes SQL Erlang, an interface between the
programming language Erlang and the query language SQL. SQL Erlang
provides the programmer with both a conventional call-level interface
and the programming language embedded SQL.

SQL Erlang has been implemented for the database manager
ORACLE in a UNIX environment."

Daniel Barlow

unread,
Jul 14, 2001, 9:44:58 PM7/14/01
to
aptpupil <aptpu...@yahoo.com> writes:

> More to the point, I find myself in the Real World(tm), having
> to replace an antiquated collection of programs written in Clipper,
> running
> on Novell. (Apparently, Clipper was the popular database and Novell
> the popular networking system of the 1980s PC era, out there on the
> Fruited Plain, outside of the twin ivory towers of Academia and High
> Tech).

You don't say what's actually wrong with it. Presumably you have some
reason for needing to replace it?

What does it do anyway? What should it do?

-dan

--

http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources

Friedrich Dominicus

unread,
Jul 15, 2001, 3:14:40 AM7/15/01
to
aptpupil <aptpu...@yahoo.com> writes:

> Help! The mind boggles in this Brave New World.
>
> More to the point, I find myself in the Real World(tm), having
> to replace an antiquated collection of programs written in Clipper,
> running
> on Novell.

AFAIK does clipper and dbase are tight related. Now there are still
packages out there which can be used for clipper programming. Sorry no
URL at hand.

>
> But there is an unusual twist: Instead of having to work for
> Pointy Hair Boss, I find that I *am* da Boss! But I have
> programmers and an "IT Manager" who have known nothing but
> Microsoft, Clipper, FoxPro, Visual Basic and such all their lives.
> Naturally, they want to jump onto the Java, Oracle, and NT
> bandwagons. Even just this is a big step forward for them.

Do you think that someone never been exposed to
- oo programming
- a new operating system
- and a fully fledged Database

can handle that transition and "re-write" a software package not
involved with anything of that before?

Now it's nice that they are open to new things, but I would be a bit
cautious to just jump on the newest hype...

>
> I can probably handle getting Timbuktu U. to move from Novell to
> Unix (GNU Linux or FreeBSD) instead of NT, at least on the servers.
> But I have no database experience to be able to tell Oracle from Eve.
> And beyond the question of what database to use (Oracle or MySQL or
> PostgreSQL? relational or OODB?),

AFAIK clipper is somewhat relational, so I would stick to it.

> I am at a loss as to what the
> Big Picture, the overall software architecture and approach, should
> be.

I would be too in your situation, but there are some people which have
worked in that area. Though about hiring one? and /or ask for
coaching?

>
> For example, is Oracle and all the fancy CASE tools around it---Oracle
> Designer, Oracle JDeveloper (J=Java, presumably), Oracle
> Forms Developer, et al---the way to go, as the IT Manager wants?
> Is there any Lispy alternative to this seductive morass of
> (database-oriented?) "visual process modeling", "rapid application
> development", "automatic code generating" tools? Or are they
> actually pretty good and powerful, and I am just being overly
> skeptical or cynical?

Oracle is the best known Database, the can not do all things
wrong. Anyway regarding what you posted it seems that using Oracle is
a kind of overkill.

>
> Bandwagons being what they are, I can understand IT people's
> great desire to jump on them and eventually get "certified" on
> them because they make one more valuable in today's Microsoft-,
> Oracle-, and Java-oriented job market.

Do you really thing all the good ones jump on nay bandwagon?

>
> Should we bother with UML and Rational? Or is that just a waste
> of time and a potential brain damager, as some posts in an
> earlier UML thread seemed to suggest. [I am just starting to
> read this and other comp.* newsgroups; I just woke up, remember.]

I do not like it but others obviously did. If you are thinking of
using Common Lisp I do think you would limit all the things Common
Lisp has to offer. BTW Symbolics is not fully dead, you can even get
their OS running on DEC (now Intels?) Alpha Processor but as I
understand all the things you had found in the old OS can be found in
the new one too.

>
> I suppose there are worse choices than Oracle and Java, but is
> it possible to do better?

What do you expect as anser here. One can work with it and there are
probably areas in which the combination is the right one. I doubt it
is in you case but who am I do judge that?


>In particular, is there a role for
> Lisp and if so where exactly?

Now there is always room to take Lisp ....


>Is it possible to do worse by
> going Lisp?

Consider you situation nobody knows either
Java
nor Lisp

You know Lisp and have worked with it. Now at least on of you knows
what Lisp can do.

>
> If one had free hand to set technical direction (no Pointy Hair
> heads above, but maybe below), but limited time, budget, and
> average non-Lisp programmers to start with (but who might be
> open to new horizons if they could be persuaded it was in their
> self-interst), what would the ideal software architecture
> and approach be?

A lot here would probably suggest go with Lisp. Now if you look into a
more modern hype wave called XP I would say, what's the deal with
it. Lisp was always there for incremental software development. A lot
of Lisp lovers are looking for a chance to work with Lisp.

After two years of learning Common Lisp I would conclude that it has
been a pitty that I do not started that adventure a year earlier. Lisp
offers all the things I do think are helpful for programmers. But it's
definitly different from the programming I did before. But even that
was and is not mainstream programmin anyway. (it was OO with Eiffel).

Now for me programming can just work in one way. Write a small kernel
which does what you want and extend it. Lisp is one of the languaes in
which that is easy and "suppored" style. As it is e.g in Smalltalk,
Python, Ruby etc.

>
> ----------
>
> Mainstream (non-Lisp) languages and tools for the average
> corporate drone programmers; Lisp only for the few, highly skilled,
> anointed ones. Is this a valid, prescriptive dichotomy; or just
> an artifact, a self-reinforcing coincidence, and a myth?

It's redicioulous. The injuring categoration of Programmers is simply
inacceptable.

>
> ----------
>
> So far, since awaking from my long sleep, I have read:
> CLOS vs. Java papers in Franz's web site
> In http://www.paulgraham.com/ :
> "Beating the Averages"
> "Lisp for Web-Based Applications"
>
> The language comparisons are all well and good but not really
> news to me. I kinda knew that before. Paul Graham's articles
> are very helpful and more of what I am looking for. Are there
> any other writings in this vein?
>
> (Interestingly, Graham doesn't say anything about whether there
> is any database in his system. Is there? Where does all
> that data go---the user-built store web sites and the
> data associated with the customers of the each store?)

All the commercial Common Lisps and even the free one can deal with
Databases. And if you insist you can use OpenGenera and ther Statice
database too.

>
> In other words, The Big Picture and Where Lisp Fits In. How best
> to position Lisp among the various software blobs and layers in
> today's complicated world---distributed database, web server,
> app server, GUI, security, network directory, VPN, and what not.
> What do you do in Lisp and what do you do in the database or
> elsewhere. How do you do the GUI. I assume we can forget about
> CLIM since web programming is the fashion today, rather than writing
> "client" programs with their own GUI, right?

Why do you care? What do you want to do with it? That's important, why
should you care about the Internet while you don't need it? Does you
old software deal with it? Why should the new software.


>
> Lastly, is there any open source, "industrial strength", high
> performance,
> general purpose Lisp environment today? Or is Allegro CL pretty much
> THE choice (as I assume it is) for "mission critical" deployment
> and richness of development environment. Put another way, is there any
> free Lisp today that can complete the quintessential sentence,
> "No one ever got fired for choosing ..."? (Assuming, of course, that
> there is *any* Lisp that can.)

Check it our yourself, Visit www.alu.org Download the free Lisp and
see if the suite you need. I have choosen LispWorks from Xanalys and
I'm quite happy with it. There is too digitook with MCL which runs
under Mac OX X too. I do think anyway that Franz is the largest of the
vendors. You may check out earlier posts in this Group on what people
like or dislike about Franz.

Regards
Friedrich

Friedrich Dominicus

unread,
Jul 15, 2001, 3:19:12 AM7/15/01
to
cbbr...@hex.net writes:

>
> Is Open Genera still actively sold?

Yes.

>I thought that the marketing
> efforts seemed pretty non-existent, a little while ago, and what with
> Compaq/Digital deep-sixing the Alpha platform, that would seem to put
> it on its death bed...


>
> > ....
> > Pick A Platform. Something robust.
> > ....
> > Pick a Database.
>
> Methinks that this (databases) is an area in which lies considerable
> 'discussability,' as well as likely a fair bit of controversy.

Definitly.


>
> There's the "relational" way, with SQL, popular names like Oracle,
> Sybase, Informix, and such. In some respects, SQL is pretty ugly;
> it's not particularly elegant.

On the other hand it's more or less standard and easy to use. I do not
think it's ugly. It does it's job and if you are somewhat serious into
Databases you'll use it anyway. I don't have used it till now but
LispWorks Common SQL seems to be available in a "free" from to. So
maybe that is what you're looking for?

Regards
Friedrich

Friedrich Dominicus

unread,
Jul 15, 2001, 3:22:45 AM7/15/01
to
Daniel Barlow <d...@telent.net> writes:

>
> You don't say what's actually wrong with it. Presumably you have some
> reason for needing to replace it?
>
> What does it do anyway? What should it do?

Oh I was overhasty. I posted before I read this. IMHO you are fully
right. It depends solly on what his goals are. If it is a transition
from Netware to something else (why? to what?) he may be better search
for Clipper on the OS on it's mind.

Regards
Friedrich

Goldhammer

unread,
Jul 15, 2001, 3:37:31 AM7/15/01
to
On Sat, 14 Jul 2001 23:02:39 GMT, aptpupil <aptpu...@yahoo.com> wrote:


>Should we bother with UML and Rational? Or is that just a waste
>of time and a potential brain damager, as some posts in an
>earlier UML thread seemed to suggest. [I am just starting to
>read this and other comp.* newsgroups; I just woke up, remember.]


The last ten years have given rise to a
bewildering amound of meta-programming
gobblygook: UML, design patterns, extreme
programming, etc., etc.


--
Don't think you are. Know you are.

Goldhammer

unread,
Jul 15, 2001, 6:11:01 AM7/15/01
to
On Sat, 14 Jul 2001 23:02:39 GMT, aptpupil <aptpu...@yahoo.com> wrote:


>Should we bother with UML and Rational? Or is that just a waste
>of time and a potential brain damager, as some posts in an
>earlier UML thread seemed to suggest. [I am just starting to
>read this and other comp.* newsgroups; I just woke up, remember.]

The last ten years have given rise to a

bewildering amount of meta-programming

Greg Menke

unread,
Jul 15, 2001, 9:38:15 AM7/15/01
to

> >that always seems to be what gets trotted out as examples of Lisp
> >success in the Real World, but a bread-and-butter "mission-critical"
> >"customer information" system.
> >
> >(Think of, say, ... the billing, accounting, and operations programs
> >of your friendly electric or gas or water company, Timbuktu Utilities.)
>

> For a similar project, I'm planning to use:
> a) Lispworks Enterprise for win32 (www.xanalys.com)
> b) Interbase (www.webinterbase.com) as a relational db.
>
> Accounting is a rather well known territory so I guess you can't go wrong, no
> matter what tools you chose. Good luck.

I worked on a pretty complex accounting/finance system from design to
delivery for about 4 years, we used Visual Basic on Access (it was
some time ago- the best we could realistically use was VB. The
alternatives were Powerbuilder or clipper). Our clients did not want
to use C/C++.

After walking through that particular valley of shadow, I think Lisp
could be a very good match for this type of system. One half of it
involved a lot of financial calculations, we were pretty much forced
into floating point, which forced us to hack a fixed point subsystem
into C to get the math done at adequate levels of accuracy. It led to
some prodigiously difficult to maintain code. I would love to hear
about how rationals work in this sort of context, I suspect they would
be exactly what we were needing. The other half of the system
involved heavy accounting stuff, and this was were VB really took its
toll. Though the SQL integration isn't too bad, the language's
inflexibility led to lots of hard work and indirect approaches to data
processing, again making the system needlessly hard to maintain. Lisp
would have made these tasks LOTS easier. To be fair, C++ would also,
but I'm pretty sure Lisp would be easier to manage given the
complicated datasets and involved algorithms.


> >But there is an unusual twist: Instead of having to work for
> >Pointy Hair Boss, I find that I *am* da Boss! But I have
> >programmers and an "IT Manager" who have known nothing but
> >Microsoft, Clipper, FoxPro, Visual Basic and such all their lives.
> >Naturally, they want to jump onto the Java, Oracle, and NT
> >bandwagons. Even just this is a big step forward for them.

Lisp would be a big technical and cultural leap for many. Back in the
early 90's, nearly all the systems we wrote used clipper & many of the
people who wrote them learned on dbase, leading to some quite horrible
examples. VB was a pretty reasonable step forward after CA destroyed
Clipper & there were always the Foxpro people. I think you'd have a
hard time convincing run of the mill programmers that just grind out
smaller systems using the tools & techniques they've always used, but
I think in shops like this theres always a few who might be interested
in something completely different. I think it would be a mistake to
try and impose a revolutionary step on everyone.

Oracle is always an easy choice, its buzzword compliance forming no
small part of that. If your budgets can stand it, theres often no
compelling reason to fight against it given the other battles always
looming, but you can always push Postgres when clients are nervous
about cost. I imagine most applications where Oracle is chosen could
provide equal performance with MUCH simpler & cheaper database
systems; honestly even Access can do quite well in a number of
applications.

I think one of the biggest reasons people might avoid different
approaches is "risk" due to a perceived difficulty in getting
personnel. But, finding people to support a fielded app is a
nightmare no matter what language & platform you choose. If it is
true that Lisp people tend to be more skilled (or maybe experienced)
on average, then it might make finding functional bodies to hurl into
the breach a little easier in the long run. Its fairly easy to find a
nitwit with current certifications on the Latest Technology who will
make a mess of your system, then leave for a better paying job.
Regardless of the language & platform, its lots harder to find someone
who can maintain anything well.

One good use for Rational Rose and UML is to provide a dumping ground
for people who are unable to make functional contributions to real
projects- its gets them out of the way & you can often get some budget
help from clients because buzzword stuff impresses their management.
So, form some kind of "UML Direction Committee" or something, put the
dead weight on it, and they'll enthuse away, occasionally providing
recommendations and procedures everyone can easily ignore. Those
involved in trying to get stuff done will thank you.

Gregm

DJC

unread,
Jul 15, 2001, 10:59:16 AM7/15/01
to
On Sat, 14 Jul 2001 23:02:39 GMT, aptpupil <aptpu...@yahoo.com>
wrote:

>Imagine that I am a Rip Van Winkle. I was raised on the workstation


>Unixes of 1980s academia, did a fair amount of C programming, then
>did a lot of Lisp programming (but not AI and only a little CLOS) for
>many years, particularly on Symbolics (+R.I.P.+).

>But there is an unusual twist: Instead of having to work for


>Pointy Hair Boss, I find that I *am* da Boss! But I have
>programmers and an "IT Manager" who have known nothing but
>Microsoft, Clipper, FoxPro, Visual Basic and such all their lives.
>Naturally, they want to jump onto the Java, Oracle, and NT
>bandwagons. Even just this is a big step forward for them.

Perhaps it is not the technical issues that you should be pondering?
It sounds as if you are an academic/technician who has suddenly
acquired a management role. I am sure you can research and find your
own resolution of the technical choices but what of the management
problems?

How far do you *need* to change the system you have. Are you prepared
to replace your existing "IT people". Do you want to get involved in
the technical details; and even if you do is that what you are
supposed to be doing in your present position? Are your IT people
suggesting change because they want to update skills, or because
change is unavoidable and they favour something that, if not familiar,
is at least mainstream. What will they do with those new marketable
skills -- move on?

More questions than answers, but possibly relevant questions.

djc
--
David...@dcs.warwick.ac.uk
Human Computing Lab 2, Department of Computer Science.
University of Warwick, UK - CV4 7AL +44 (0)24 765 73797

Paolo Amoroso

unread,
Jul 15, 2001, 11:07:14 AM7/15/01
to
On Sat, 14 Jul 2001 23:02:39 GMT, aptpupil <aptpu...@yahoo.com> wrote:

> Lucid is gone (+R.I.P.+) but Franz is still alive, for some inexplicable
> reason. (Pray, what?) Dylan lost out to Java. Smalltalk is

Maybe Franz is alive because there's still demand for Lisp tools, but Lucid
is not because it diverted too many resources on C++ tools. Indeed, Lucid's
LCL product (now Liquid Common Lisp) is still marketed by Xanalys (formerly
Harlequin).


> Richard Gabriel's Worse-is-Better vs. The-Right-Thing
> philosophical analysis is but a distant memory, although the

There are some recent developments you may be already aware of:

http://www.dreamsongs.com/WorseIsBetter.html


> PostgreSQL? relational or OODB?), I am at a loss as to what the
> Big Picture, the overall software architecture and approach, should be.

From what frame of reference are you looking at the big picture? From the
buzzword/mainstream/bandwagon one?


> Is there any Lispy alternative to this seductive morass of
> (database-oriented?) "visual process modeling", "rapid application
> development", "automatic code generating" tools? Or are they

Hasn't Lisp always offered most--if not all--of that?


> news to me. I kinda knew that before. Paul Graham's articles
> are very helpful and more of what I am looking for. Are there
> any other writings in this vein?

You may check:

Basic Lisp Techniques
http://www.franz.com/resources/educational_resources/cooper.book.pdf

I also suggest that you contact Franz's sales department and purchase
copies of all the proceedings of Lisp events sponsored by them that are
available.


> elsewhere. How do you do the GUI. I assume we can forget about
> CLIM since web programming is the fashion today, rather than writing

In case you decide that you want a CLIM-based GUI anyway, besides the
commercial vendors' offerings (there are at least 3) you may consider Free
CLIM:

http://www.mikemac.com/mikemac/McCLIM/index.html
http://www2.cons.org:8000/mailman/listinfo/free-clim

The implementation is not complete yet, but work on it has been increasing
with the help of new developers.


> Lastly, is there any open source, "industrial strength", high
> performance,
> general purpose Lisp environment today? Or is Allegro CL pretty much

As others have pointed out, other commercial Lisp systems are available
besides ACL. As for open-source ones, both CMU CL and CLISP have been used
for industrial applications, but I can't comment on them.


Finally, here are a couple of other useful resources:

CLiki
http://ww.telent.net/cliki

Lispweb mailing list
http://www.red-bean.com/mailman/listinfo/lispweb

If you finally decide to use Lisp for your project, I would appreciate it
if you could let us know. Likewise, if you decide not to use Lisp, it would
be interesting to know the reasons behind your decision.


Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/

Kurt B. Kaiser

unread,
Jul 15, 2001, 11:30:19 AM7/15/01
to
aptpupil <aptpu...@yahoo.com> writes:

Absolutely! Fire all those incompetents, grab the hariest free tools you can
find, and recode everything from scratch! Enjoy!

> This legacy system is not some fancy, brainy AI-ish planning/scheduling
> thing
> that always seems to be what gets trotted out as examples of Lisp
> success in the Real World, but a bread-and-butter "mission-critical"
> "customer information" system.

Why don't you just buy the damn thing?

Good thing for you you're the "da" boss.

KBK

Wade Humeniuk

unread,
Jul 15, 2001, 11:37:51 AM7/15/01
to
As you can see from the responses to your post there are many viewpoints,
historical prespectives and preferences.

From your post I am sure that _you_ think Lisp can do the job but just need
justification for using it. There are lots of tools to make apps, like
Powerbuilder and such. The main problem with these is that they provide a
limited way to solving a problem. If your solution goes ouside the box the
tool becomes a millstone around your neck.

Lisp can do the job.
If a new thing comes along, Lisp can adapt to do the job.

Wade

Don Geddis

unread,
Jul 15, 2001, 1:37:01 PM7/15/01
to
aptpupil <aptpu...@yahoo.com> writes:
> Lastly, is there any open source, "industrial strength", high performance,
> general purpose Lisp environment today? Or is Allegro CL pretty much THE
> choice (as I assume it is) for "mission critical" deployment and richness of
> development environment.

Franz's ACL works fine, as you suggest. But if you need the "open-source"
part, you can use CMUCL. We've successfully built and deployed a commercial
system on top of CMUCL (and ACL too, for that matter) at GoTo.

-- Don
_______________________________________________________________________________
Don Geddis www.goto.com ged...@goto.com
Vice President of Research and Development Phone 650-403-2220
GoTo.com, 1820 Gateway Drive, Suite 360, San Mateo, CA 94404 Fax 650-403-2201

Tim Moore

unread,
Jul 15, 2001, 9:07:47 PM7/15/01
to
On Sat, 14 Jul 2001, aptpupil wrote:
>
> If one had free hand to set technical direction (no Pointy Hair
> heads above, but maybe below), but limited time, budget, and
> average non-Lisp programmers to start with (but who might be
> open to new horizons if they could be persuaded it was in their
> self-interst), what would the ideal software architecture
> and approach be?

Hire an intern who took a Lisp course at school, write the prototype with
him while the others are learning the buzzword-enabled technologies, and
then fire them and turn the prototype into the real system.

>
> Mainstream (non-Lisp) languages and tools for the average
> corporate drone programmers; Lisp only for the few, highly skilled,
> anointed ones. Is this a valid, prescriptive dichotomy; or just
> an artifact, a self-reinforcing coincidence, and a myth?

Obviously it's a myth, though possibly a helpful one for Lisp programmers
looking for work.

> In other words, The Big Picture and Where Lisp Fits In. How best
> to position Lisp among the various software blobs and layers in
> today's complicated world---distributed database, web server,
> app server, GUI, security, network directory, VPN, and what not.
> What do you do in Lisp and what do you do in the database or
> elsewhere. How do you do the GUI. I assume we can forget about
> CLIM since web programming is the fashion today, rather than writing
> "client" programs with their own GUI, right?

There has been some work in using CLIM for html presentation. I think
cl-http has some example code.

> Are there any other recent writings that, like Paul Graham's, gives
> insight into---a decoded view of---a more palatable introduction
> to---some
> piece of the non-Lisp world from a Lisp perspective and sensibility.
> Say, databases and database programming. I leafed through some books
> on Oracle CASE tools and on UML and felt like throwing up. I couldn't
> help but think, "Hmmm, feels like a whole other country". And not a
> pretty one at that.

Much like nursing an endangered species back from the brink, you have to
rely on surrogate mothers for your Lisp nurishment in the IT and web
space. Books on Smalltalk and Extreme Programming should give you a lot
of good ideas; just replace "SmallTalk" with "Lisp" and "double dispatch"
with "multiple inheritance" :-) The book "San Francisco Design Patterns"
looks like a very good overview of objects, methods and patterns
encountered in IT. Phillip Greenspun's books on web publishing are great
and are written by an actual Lisp sympathizer.

For straight up Lisp pages, both Franz' site and the Cliki
(http://ww.telent.net/cliki) have good stories and code.

Frank Gleeson

unread,
Jul 15, 2001, 11:45:35 PM7/15/01
to
My you are so cynical. That is a great way to use UML however.
It's a shame tools like UML are percieved as so peripheral to *real*
development. I think as long as companies like Rational own these
tools and charge such exorbitant per seat prices they are doomed to
the ashheap of computing history.

aptpupil

unread,
Jul 16, 2001, 12:49:37 AM7/16/01
to
> Integrating with other enterprise systems is almost certainly going to
> be easier with an SQL system, though data access from the Lisp side
> will require some custom effort to build accessor/modifier functions.

I am still trying to figure out this whole area (of databases) and
am not sure I even know the right questions to ask, so this is just
a question out of the blue, in the context of SQL databases:

Is that a big deal? showstopper? difficult? pain in the behind?
I assume that the custom functions being referred to here
serve to enclose, are some kind of wrapper over, the functions or macros
provided in a particular Lisp's ODBC library (e.g., Allegro ODBC)?
And I assume that those library functions or macros are essentially
a one-to-one (argument-for-argument) Lispification of SQL statements?

> Aside from all that, I find it marginally surprising/disappointing
> that there haven't been any visible attempts to attach lower level DB
> functionality.

Are you referring to Lisp in the context of object-oriented
(as opposed to SQL) databases?
By "lower level", I assume you are desiring the ability to
issue more SQL-like queries/updates...?

> I would _think_ that throwing in something like DBM or (better, as it
> supports several data models, with hashing as well as B-trees)
> Berkeley DB as a data store, and then building up data accessors on
> top of this would be _very_ interesting.
>
> There has been discussion by the PostgreSQL folks of replacing their
> underlying data storage engine with Berkeley DB to improve some of the
> characteristics of the RDBMS, and the folks that built MySQL finally
> got "real transaction" support by virtue of replacing their own
> storage engine with Berkeley DB. The point of this comment is that
> Berkeley DB is clearly flexible/powerful enough to be used as a
> (perhaps hidden) engine on top of which more sophisticated things can
> be built; the same could readily be true for the way a CL system might
> "abuse" it to attain desired DBMS functionality.

Interesting. I wonder whether this (MySQL being able to "do
transactions") is still in the future, or already present in the
current version. And whether this addresses and neutralizes
the arguments in "Why Not MySQL" [circa May 2000]...
http://openacs.org/philosophy/why-not-mysql.html

aptpupil

unread,
Jul 16, 2001, 1:07:58 AM7/16/01
to
> Perhaps while you were asleep, Mr. Wrinkle, you missed the arrival of
> Harlequin/Xanalys on the scene. While it's certainly true that
> Franz/Allegro is a fine product, it's far from true that it's the only
> choice for commercially-supported "mission critical" deployment.

Yes, I did. Thanks for the mention; will consider LispWorks!
I left the scene (so to speak) around the time that Harlequin
was releasing Dylan beta. I am glad to know what became of
Harlequin ... although I not so sure about the wisdom of
choosing company names with "anal" as a prominent substring...

aptpupil

unread,
Jul 16, 2001, 2:02:15 AM7/16/01
to
> You don't say what's actually wrong with it. Presumably you
> have some reason for needing to replace it?

Undocumented, original programmer left, DOS-based,
buggy, brittle, hard to debug, hard to fix, hard to extend.
No one wants to work on it any more. Who would.
The language [Clipper] looks like a precursor to C, and
the "GUI" programming is...80x24 char.
Much of current effort is now on understanding and documenting it
(the spaghetti of a bazillion program files and tables).

> What does it do anyway?

Using the example of the electric/gas/water company
"Timbuktu Utilities" that I mentioned, it interfaces to the
meter readers, does billing, collection, and generates
reports. Various report data then goes, manually,
eventually, into spreadsheets, QuickBooks, yet more paper
forms and whatnot.

The programs and databases are physically distributed (across
great distances). The primary method of interaction and
synchronization is...via daily transportation of floppy disk.
Remote debugging is via dialup and pcAnywhere.
Remote teller (of which there are quite a few) operation is
via dialup and pcAnywhere. Big phone bill.

> What should it do?

The metering, billing, and collection programs (and the
processes they assume) need to be improved, made more adaptible.
Need to add operations and planning---GIS, facilities
management, interface to substation and distribution
system automation...

aptpupil

unread,
Jul 16, 2001, 2:42:10 AM7/16/01
to
> For a similar project, I'm planning to use:
> a) Lispworks Enterprise for win32 (www.xanalys.com)
> b) Interbase (www.webinterbase.com) as a relational db.
>
> Accounting is a rather well known territory so I guess you can't go wrong, no
> matter what tools you chose. Good luck.

At the moment, I understand that the company is doing its "accounting",
such as it is, in QuickBooks. (And with very long time lag. This is
no Cisco, able to "close its books" at the drop of a hat and know
its financial status in almost real-time.)
But there is a problem due to the magnitude of the amounts
involved. (Think of a currency like Japanese yen.) Apparently,
QuikBooks can't handle the very large amounts that come up.
So the workaround, as I understand it, has been to enter
proportionally reduced amounts (say X/10 instead of X)
and to interpret the QuickBooks results appropriately.

(Hmmm, if I understand that workaround correctly, it sounds as if
its correctness depends on all financial calculations
satisfying some kind of linearity property. I sure hope that
is the case with stuff like, you know, depreciation calculations
and such.)

> Then stay away from Oracle: you need a db that is easy to mantain, like
> interbase o Caché (www.intersystems.com).
> PS Take a look at a book called "Basic Lisp techniques" at franz.com (it's
> available as a pdf file): it comes with a db related sample project.

I have always had a gut feeling that Oracle is overkill
and overly complex for the situation at hand, but have not
(yet) been able to be more specific and persuasive (without
being dictatorial). In other words, which features are not
really needed and/or come at too high a complexity cost.

Fortunately, Oracle is free for development purposes, so there
is the chance to try it out, as well as other alternatives.

aptpupil

unread,
Jul 16, 2001, 3:53:38 AM7/16/01
to
> More questions than answers, but possibly relevant questions.

Perceptive observations, and very good questions.
Now, can we have the answers please??? :)

> How far do you *need* to change the system you have.

Need is a subjective, beholder's-eye variable, of course.
I think everyone is agreed that the system needs to be replaced.

> Are you prepared to replace your existing "IT people".

I think everyone has potential, though in varying
amounts. The existing people are crucial because they
are familiar with (and in the process of understanding
and documenting) the existing system. What will be needed
is a few more people...of the right kind...at the appropriate time
[Brooks' Law and all that].

> Do you want to get involved in the technical details;
> and even if you do is that what you are
> supposed to be doing in your present position?

Yes; I believe I have to, at least initially.
This will have to be played by ear though, and re-evaluated
over time.

> Are your IT people suggesting change because they want to
> update skills, or because change is unavoidable and they
> favour something that, if not familiar, is at least mainstream.

Yes and yes. "Favour" may be too strong a word though,
as an alternative (to the default mainstream) has not yet
been proposed or even just demonstrated.

> What will they do with those new marketable skills -- move on?

I could be dreaming, but I think that if this is done
right the motivation to remain should be greater than the
motivation to leave, at least for a significant amount of
time. And when one might want to move on, that should be
fine too. The alternative is stagnation, of both individual
and organization.

But I understand the point.

aptpupil

unread,
Jul 16, 2001, 5:18:43 AM7/16/01
to
> There are some recent developments you may be already aware of:
> http://www.dreamsongs.com/WorseIsBetter.html

No, I wasn't aware of this. Thanks. What?! After more than a
decade, rpg can't make up his mind, and this issue still isn't
decided??!!
Oh well, Life is Interesting. And as we can probably all predict,
this isn't going to be decided anytime soon, if ever.

> > PostgreSQL? relational or OODB?), I am at a loss as to what the
> > Big Picture, the overall software architecture and approach, should be.
>
> From what frame of reference are you looking at the big picture? From the
> buzzword/mainstream/bandwagon one?

I am (we are) basically trying to figure out what the pieces are --
might be -- in this new (to me) world of distributed programs.
I think the question I was trying to ask, obviously not very well,
is not so much whether or not to use Lisp, but...Where, For What, and
How
Exactly.

One article (but non-Lisp-oriented) that was helpful in this regard:
"Front-End, Back-End, All Across the Net"
http://www.extremetech.com/article/0,3396,s%253D1694%2526a%253D6509,00.asp

I'm sure I/we will eventually figure this all out just by
starting and prototyping, and reading bits of information
here and there. But I was just hoping that someone who has been
down this road before, with Lisp, might have already spelled it
all out, or at least illuminated the paths, in more glorious detail
[than has Paul Graham].

> > Is there any Lispy alternative to this seductive morass of
> > (database-oriented?) "visual process modeling", "rapid application
> > development", "automatic code generating" tools? Or are they
>
> Hasn't Lisp always offered most--if not all--of that?

In general and in theory, yes.

But Oracle Designer might be valuable and well suited to its
particular domain (database design/programming), with no
ready-made Lisp equivalent. Designing the tables and
automatically generating the SQL statements, as I understand
it. We'll have to check these out. Again, I think
the underlying question here is, what do you do in Lisp (or
Java, etc.) and what do you do in SQL.

As for UML and Rational or similar, maybe they are just
glorified drawing tools, but maybe there is something
there? If they are essentially process and OO modeling
tools (respectively), is there not some value in that.
We'll have to see (UML at least). The point that conventional
OO modeling can't model generic functions sounds like a
serious problem, though I wonder how often in our applications the
need for dispatching on more than one argument will arise...

> If you finally decide to use Lisp for your project, I would appreciate it
> if you could let us know. Likewise, if you decide not to use Lisp, it would
> be interesting to know the reasons behind your decision.

Will do.

aptpupil

unread,
Jul 16, 2001, 5:28:14 AM7/16/01
to
> Why don't you just buy the damn thing?

The usual reasons.

It doesn't exist (in for-sale and/or customizable form).

Or if it ever did, it would be unaffordable.

aptpupil

unread,
Jul 16, 2001, 5:40:33 AM7/16/01
to
> Hire an intern who took a Lisp course at school, write the prototype with
> him while the others are learning the buzzword-enabled technologies, and
> then fire them and turn the prototype into the real system.

Yes, been thinking along these lines...except for the part
about firing...

> Much like nursing an endangered species back from the brink, you have to
> rely on surrogate mothers for your Lisp nurishment in the IT and web
> space. Books on Smalltalk and Extreme Programming should give you a lot

I like the analogy, but also find it disturbing. :)

> of good ideas; just replace "SmallTalk" with "Lisp" and "double dispatch"
> with "multiple inheritance" :-) The book "San Francisco Design Patterns"
> looks like a very good overview of objects, methods and patterns
> encountered in IT. Phillip Greenspun's books on web publishing are great
> and are written by an actual Lisp sympathizer.
>

Oh great. So IBM software work, after the Taligent debacle,
amounted to something after all, huh.

Piotr Kuchta

unread,
Jul 16, 2001, 6:17:49 AM7/16/01
to

Friedrich Dominicus <fr...@q-software-solutions.com> wrote:

> > What does it do anyway? What should it do?
> Oh I was overhasty. I posted before I read this. IMHO you are fully
> right. It depends solly on what his goals are. If it is a transition
> from Netware to something else (why? to what?) he may be better search
> for Clipper on the OS on it's mind.

from:
http://www.harbour-project.org/

"Harbour is a free software compiler for the xBase superset language often
referred to as Clipper (the language that is implemented by the compiler
CA-Clipper).
Harbour is a cross platform compiler and is known to compile and run on
MS-DOS, MS-Windows, OS/2 and GNU/Linux."

I can't say anything about it, I didn't use it.

Regards,
Piotr


Greg Menke

unread,
Jul 16, 2001, 8:01:24 AM7/16/01
to

lol... I don't think UML is useless in the same way TQM or SEI is
useless. Which is an oversimplification I suppose, but I think most
orgs use TQM and SEI as a blunt instrument more than a tool for
actually improving. But as far as UML is concerned, I've yet to see
how it significantly improves on the plain old systems analysis
techniques and notations from 15 years ago. There were nifty case
tools letting you draw pretty pictures & generate code back then also.
I think UML <can> be a useful tool, just as long as its not forced
down peoples throats. Frequently, if management types spent the time
and money actually learning the requirements, problems and technical
details of the systems they're managing that they spend with UML and
other buzzword "technologies", many problems could be solved with much
less fuss and bother.

In the right hands I think UML could be useful, but the same applies
to a 3 ring binder and notepad. It really depends on the people
staffing a project, quality people will set up the systems they need
which may or may not include UML. People who can't manage & develop a
project won't suddenly start turning out sucessful systems just
because they use UML- it won't even increase their odds of success.

Gregm

cbbr...@hex.net

unread,
Jul 16, 2001, 9:22:48 AM7/16/01
to
aptpupil <aptpu...@yahoo.com> writes:
> > Integrating with other enterprise systems is almost certainly going to
> > be easier with an SQL system, though data access from the Lisp side
> > will require some custom effort to build accessor/modifier functions.

> I am still trying to figure out this whole area (of databases) and
> am not sure I even know the right questions to ask, so this is just
> a question out of the blue, in the context of SQL databases:

> Is that a big deal? showstopper? difficult? pain in the behind? I
> assume that the custom functions being referred to here serve to
> enclose, are some kind of wrapper over, the functions or macros
> provided in a particular Lisp's ODBC library (e.g., Allegro ODBC)?
> And I assume that those library functions or macros are essentially
> a one-to-one (argument-for-argument) Lispification of SQL
> statements?

No, _not_ a good assumption.

The area to look at as a parallel for which there's more easily
available literature is the Java database access stuff.

a) There's "basic JDBC," where the system is basically doing SQL/ODBC
with a Java flavor, which is not too far from a one-to-one mapping.

But that's a bit of a waste of time; people tend to move towards...

b) Building persistence frameworks on top of this, where by attaching
some extra code to objects (could be CLOS, for Lisp; the Java
literature obviously references Java classes...), objects can load and
save themselves, building SQL queries as needed to do so.

As for the "pain in the behind" side of things, I've built some CL
code that provides a "sorta-ODBC-like-layer;" that's not overly
difficult, although making it efficient may be a Bit of Work. In the
long run, you probably want objects to be smart enough to make
themselves persistent, from whence extensive further complications
come, and _that_ is the "pain in the behind," particularly when there
aren't many common conventions for how to do this as it's out of scope
both of CL standards as well as of things done with more than one CL
implementation.

>> Aside from all that, I find it marginally surprising/disappointing
>> that there haven't been any visible attempts to attach lower level DB
>> functionality.

> Are you referring to Lisp in the context of object-oriented (as
> opposed to SQL) databases? By "lower level", I assume you are
> desiring the ability to issue more SQL-like queries/updates...?

Nope. I'm thinking more primitive than that; stuff like Unix DBM.
DBM provides the ability to store keyed values into a data table on
disk, with pretty much the same sort of functionality as you get with
CL hash tables, with the difference that the data is automagically
heading to disk.

In Perl, it is downright easy to attach an "associative list" to a
disk-based DBM file (actually, this can be done for most DBM-like
database schemes), which, if we head it over to the Lisp world, lets
you do things like:

(setf (get-dbm-hash "this-key" *DBM-REFERENCE*) 27)
which would hit the data structure on disk just the same way that
you'd modify an in-memory reference with...
(setf (get-hash "this-key" *HASH-TABLE-REFERENCE*) 27)

I will up-front agree that this is a primitive way of doing things,
compared to the much "smarter" queries you can throw at an SQL
database, and does not admit to the notion of "data dictionary" that
SQL DBMSes offer.

The point is that having this would allow constructing interesting
Lisp-oriented extensions atop the primitives.

>> I would _think_ that throwing in something like DBM or (better, as it
>> supports several data models, with hashing as well as B-trees)
>> Berkeley DB as a data store, and then building up data accessors on
>> top of this would be _very_ interesting.

>> There has been discussion by the PostgreSQL folks of replacing
>> their underlying data storage engine with Berkeley DB to improve
>> some of the characteristics of the RDBMS, and the folks that built
>> MySQL finally got "real transaction" support by virtue of replacing
>> their own storage engine with Berkeley DB. The point of this
>> comment is that Berkeley DB is clearly flexible/powerful enough to
>> be used as a (perhaps hidden) engine on top of which more
>> sophisticated things can be built; the same could readily be true
>> for the way a CL system might "abuse" it to attain desired DBMS
>> functionality.

> Interesting. I wonder whether this (MySQL being able to "do
> transactions") is still in the future, or already present in the
> current version. And whether this addresses and neutralizes the
> arguments in "Why Not MySQL" [circa May 2000]...
> http://openacs.org/philosophy/why-not-mysql.html

This particular feature is _precisely_ what moves MySQL towards
neutralizing those arguments, yes.

--
(reverse (concatenate 'string "gro.mca@" "enworbbc"))
http://vip.hex.net/~cbbrowne/internet.html
"Robot: Mechanical apparatus designed to do the work of a man."
-- Encyclopedia Galactica

aptpupil

unread,
Jul 16, 2001, 11:14:59 AM7/16/01
to
> _Hire_ some Lisp Developers!! (I am looking for work!)

Oh I would, and from this ng or similar venue, in a heartbeat.
Unfortunately, Timbuktu is halfway around the world, in...Nepal.
Or it might as well be, as far as most people here are concerned.

The local ("Nepalese") Lisp talent, if any, will have to be
investigated...

Bulent Murtezaoglu

unread,
Jul 16, 2001, 11:51:46 AM7/16/01
to
>>>>> "Fernando" == Fernando <f...@wanadoo.es> writes:
[...]
Fernando> It _is_ a horrible name, but it has nothing to do with
Fernando> anal inside. ;-D Was it xanalys.com? Or xanalysis.com,
Fernando> xanalist.com, xanalisp???? Looks like a great way to
Fernando> lose traffic...

Actually, I just checked xanalisp.com and it is available. Xanayls
could get it and set up an http redirect before Franz beats them to it!

Or maybe native speakers of English have a better time with it and those
of us from .es (and .tr for me) are the exception?

cheers,

BM

Wade Humeniuk

unread,
Jul 16, 2001, 12:23:24 PM7/16/01
to

"aptpupil" <aptpu...@yahoo.com> wrote in message
news:3B5304E1...@yahoo.com...

Yikes!

You could still parcel the work out over the internet and have developers
work remotely. Though India is of course close and there are lots of
programmers there.

The rarified atmosphere should be conducive to programming in Lisp.

Wade

Tim Bradshaw

unread,
Jul 16, 2001, 1:14:59 PM7/16/01
to
* Kurt B Kaiser wrote:

> Why don't you just buy the damn thing?

Do you know how much such things typically cost? And then how much
you spend to get them bespoken for you? 7 figure sums (dollars or
pounds...) is probably about right, 6 would be cheap, 8 not unusual.

--tim

Paolo Amoroso

unread,
Jul 16, 2001, 2:48:28 PM7/16/01
to
On 16 Jul 2001 01:07:47 GMT, Tim Moore <mo...@herschel.bricoworks.com>
wrote:

> There has been some work in using CLIM for html presentation. I think
> cl-http has some example code.

Another approach is described in this paper:

Adapting CLIM Applications for Use on the WWW
ftp://ftp.ai.sri.com/pub/papers/paley-luv95.ps.Z

Paolo Amoroso

unread,
Jul 16, 2001, 2:48:29 PM7/16/01
to
On Mon, 16 Jul 2001 06:42:10 GMT, aptpupil <aptpu...@yahoo.com> wrote:

> involved. (Think of a currency like Japanese yen.) Apparently,
> QuikBooks can't handle the very large amounts that come up.

Lisp's bignums look ideal for this task.

Thomas F. Burdick

unread,
Jul 16, 2001, 3:12:37 PM7/16/01
to
Fernando <f...@wanadoo.es> writes:

> On 16 Jul 2001 08:01:24 -0400, Greg Menke <gregm...@mindspring.com> wrote:
>
>
> >I think UML <can> be a useful tool, just as long as its not forced
> >down peoples throats.
>

> Even with CLOS? It looks like UML is too Java/C++ centric and not generic
> enough for CLOS, or maybe I don't know enough uml... :-?

UML -> CLOS is always possible, and easy. Going the other way, on the
other hand, is not. In fact, I'd think UML -> CLOS would be a good
idea, because it would allow you to use UML if possible, and if you
run into a subset of the problem that needs the full power of CLOS,
it's available to you (at the cost of having to mark part of your UML
mess as "incorrect, see actual code"). Of course, this is just
speculation. My mental pictures of object-oriented systems are
radically different than what every UML drawing I've seen looks like;
so I've never used it (and have fortunately never been forced to).

Kaz Kylheku

unread,
Jul 16, 2001, 3:13:11 PM7/16/01
to
In article <3B529D71...@yahoo.com>, aptpupil wrote:
>> More questions than answers, but possibly relevant questions.
>
>Perceptive observations, and very good questions.
>Now, can we have the answers please??? :)
>
>> How far do you *need* to change the system you have.
>
>Need is a subjective, beholder's-eye variable, of course.
>I think everyone is agreed that the system needs to be replaced.
>
>> Are you prepared to replace your existing "IT people".
>
>I think everyone has potential, though in varying
>amounts. The existing people are crucial because they
>are familiar with (and in the process of understanding
>and documenting) the existing system.

That means that they are indispensable *now*, but upon completion
of the documentation, they will no longer be.

I would just extract the functional requirements from the current users
of the existing software. Construct a set of use cases and proceed
from there.

Quite likely, all you need to know is how the data is stored in the
existing system so that it can be migrated. You probably don't need a
detailed description of every procedure operating on that data.

Don't allow these VB-pushing imbeciles :) to extend their stay by producing
useless documentation---such as function-by-function paraphrasing of
their code or something like that.

cbbr...@hex.net

unread,
Jul 16, 2001, 3:26:39 PM7/16/01
to
Paolo Amoroso <amo...@mclink.it> writes:
> On Mon, 16 Jul 2001 06:42:10 GMT, aptpupil <aptpu...@yahoo.com> wrote:
>> involved. (Think of a currency like Japanese yen.) Apparently,
>> QuikBooks can't handle the very large amounts that come up.

> Lisp's bignums look ideal for this task.

I'd think it likely that rationals would be a little better still, as
they can cope with fractional currencies not limited to dollars and
cents.

One problem to work around would be in the area of storage, if
interfacing with something like Oracle, as Oracle (and other SQL
systems) _tends_ to prefer fixed size fields for numeric values,
whilst bignums are most definitely _not_ of fixed size :-(.

Probably not a crucially daunting issue, but nonetheless one to think
about...
--
(concatenate 'string "cbbrowne" "@ntlug.org")
http://vip.hyperusa.com/~cbbrowne/sap.html
"When you have eliminated the impossible, whatever remains, however
improbable, must be the truth." -- Sir Arthur Conan Doyle (1859-1930),
English author. Sherlock Holmes, in The Sign of Four, ch. 6 (1889).
[...but see the Holmesian Fallacy, due to Bob Frankston...
<http://www.frankston.com/public/Essays/Holmesian%20Fallacy.asp>]

Thomas F. Burdick

unread,
Jul 16, 2001, 3:33:32 PM7/16/01
to
k...@ashi.footprints.net (Kaz Kylheku) writes:

> Don't allow these VB-pushing imbeciles :) to extend their stay by producing
> useless documentation---such as function-by-function paraphrasing of
> their code or something like that.

While a fn-by-fn paraphrasing of code is not sufficient documentation,
it *can* be an important part of documenting a system, if the overall
system is also described, and the code you're paraphrasing is opaque
(this will allow someone maintaining the code to find the relevant
functions without being forced to puzzle through all the code). It
is, of course, of questionable value if the code is easy to read.

Speaking from experience,
Tom

Marcin Tustin

unread,
Jul 16, 2001, 3:01:13 PM7/16/01
to

Bile Warning: If you are allergic to unwarranted bile, not meant
sincerely, do not read.

Wade Humeniuk <hume...@cadvision.com> wrote in message
news:9iv4cm$2tv$1...@news3.cadvision.com...

> You could still parcel the work out over the internet and have developers
> work remotely. Though India is of course close and there are lots of
> programmers there.
>
> The rarified atmosphere should be conducive to programming in Lisp.

I think that's "Nepal" in Texas - aren't you close to CA, home of US IT?
Or is it infested with java programmers, perl hackers, DreamWeaver jockeys,
and the 3 last remaining guys who insist on using smalltalk, because they
invented it?


Marcin Tustin

unread,
Jul 16, 2001, 2:48:44 PM7/16/01
to

Greg Menke <gregm...@mindspring.com> wrote in message
news:m3hewdc...@europa.mindspring.com...

>
> But as far as UML is concerned, I've yet to see
> how it significantly improves on the plain old systems analysis
> techniques and notations from 15 years ago.

S'object-oriented, i'ntit? Seriously, however great it may be, it aint
great when skilled artists (Completed 1 year of my MEng) such as myself are
expected to design software in UML without touching a prototype, without
even a compiler.


Marcin Tustin

unread,
Jul 16, 2001, 2:51:49 PM7/16/01
to

Fernando <f...@wanadoo.es> wrote in message
news:78q5lt4q8u3v1gsdd...@4ax.com...

> On 16 Jul 2001 08:01:24 -0400, Greg Menke <gregm...@mindspring.com>
wrote:
>
>
> >I think UML <can> be a useful tool, just as long as its not forced
> >down peoples throats.
>
> Even with CLOS? It looks like UML is too Java/C++ centric and not generic
> enough for CLOS, or maybe I don't know enough uml... :-?

I actually don't see what you couldn't model in UML. Although
polymorphism exists on many types, rather than just one, this is, AFAI can
see the same as having the function on multiple objects, overriden many
times with different signatures. Of course, you could also model generic
functions as functors...


DJC

unread,
Jul 16, 2001, 5:33:08 PM7/16/01
to
On Mon, 16 Jul 2001 19:13:11 GMT, k...@ashi.footprints.net (Kaz
Kylheku) wrote:

>In article <3B529D71...@yahoo.com>, aptpupil wrote:

>>I think everyone has potential, though in varying
>>amounts. The existing people are crucial because they
>>are familiar with (and in the process of understanding
>>and documenting) the existing system.
>
>That means that they are indispensable *now*, but upon completion
>of the documentation, they will no longer be.
>
>I would just extract the functional requirements from the current users
>of the existing software. Construct a set of use cases and proceed
>from there.

In my experience, when replacing a long-lived system the code and the
pragmatic processes around it become inseparable. What the users do
works around the software, and the software in turn probably
accomodate proceedures that were 'untouchable' when the system was
first specified.. The users will tell you it has to be done that way
because that is how the system works, but part of that system is
precisely what you are about to change (because, technically at least,
it dosn't workany more)

>Quite likely, all you need to know is how the data is stored in the
>existing system so that it can be migrated. You probably don't need a
>detailed description of every procedure operating on that data.
>
>Don't allow these VB-pushing imbeciles :) to extend their stay by producing
>useless documentation---such as function-by-function paraphrasing of
>their code or something like that.

But day to day the business has to continue to function, and they
probably know more than they can tell. You may not need their coding
skills, nor even the ability to maintain the system, you will need
thier tacit knowledge of how the whole complex system really
functions.

DJC

--
David...@dcs.warwick.ac.uk
Human Computing Lab 2, Department of Computer Science.
University of Warwick, UK - CV4 7AL +44 (0)24 765 73797

Kaz Kylheku

unread,
Jul 16, 2001, 8:52:06 PM7/16/01
to
In article <3b534f70...@news.blueyonder.co.uk>, DJC wrote:
>On Mon, 16 Jul 2001 19:13:11 GMT, k...@ashi.footprints.net (Kaz
>Kylheku) wrote:
>
>>In article <3B529D71...@yahoo.com>, aptpupil wrote:
>
>>>I think everyone has potential, though in varying
>>>amounts. The existing people are crucial because they
>>>are familiar with (and in the process of understanding
>>>and documenting) the existing system.
>>
>>That means that they are indispensable *now*, but upon completion
>>of the documentation, they will no longer be.
>>
>>I would just extract the functional requirements from the current users
>>of the existing software. Construct a set of use cases and proceed
>>from there.
>
>In my experience, when replacing a long-lived system the code and the
>pragmatic processes around it become inseparable. What the users do
>works around the software, and the software in turn probably

That's because often, they are *working around* the software. ;)

>But day to day the business has to continue to function, and they
>probably know more than they can tell. You may not need their coding
>skills, nor even the ability to maintain the system, you will need
>thier tacit knowledge of how the whole complex system really
>functions.

Which assumes they have it. More typically, whoever really knew the
system longer works there. ;)

Greg Menke

unread,
Jul 16, 2001, 11:24:10 PM7/16/01
to

> > Lisp's bignums look ideal for this task.
>
> I'd think it likely that rationals would be a little better still, as
> they can cope with fractional currencies not limited to dollars and
> cents.
>
> One problem to work around would be in the area of storage, if
> interfacing with something like Oracle, as Oracle (and other SQL
> systems) _tends_ to prefer fixed size fields for numeric values,
> whilst bignums are most definitely _not_ of fixed size :-(.
>

Considering the pain and expense we had to endure to get sufficient
accuracy in the financial arithmetic, I would have been quite content
with a 10-fold increase in storage overhead along with the additional
server costs to go with it. If we did have something like Lisp, or at
least something that did rationals available as a "math" subsystem, we
would have worked something out- even if it ended up as messy as
storing rationals as blobs & using a glue layer to read/write them.

Greg

cbbr...@hex.net

unread,
Jul 16, 2001, 11:31:57 PM7/16/01
to

One thing I'd point out is that [barring using some of the
"object-relational" extensions in PostgreSQL] you _won't_ be able to
do SQL queries looking like:
select total(amount) from some_table where vendor = 'AOL';

You can put just about anything in a VARCHAR field, if you "box" it
suitably [in the APL sense of boxing]; the _problem_ is that by using
a data type that the SQL engine doesn't know anything about, _all_
processing has to take place inside the Lisp engine.

That may not be prohibitive; as you suggest, having 'good math' is
worth a lot...


--
(concatenate 'string "cbbrowne" "@ntlug.org")

http://vip.hex.net/~cbbrowne/internet.html
"Put simply, the antitrust laws in this country are basically a joke,
protecting us just enough to not have to re-name our park service the
Phillip Morris National Park Service."
-- Courtney Love, Salon.com, June 14, 2000

Gavin E. Mendel-Gleason

unread,
Jul 16, 2001, 3:53:27 PM7/16/01
to

Why not just store them as strings. If you keep metadata about what types are stored in which columns, you can
safely coerce the values back and forth.

Blobs are painful.

Gavin

--
Gavin E. Mendel-Gleason
(let ((e-mail-address "PGIU...@VGIHKRR.TKZ"))(loop with new-string = (make-string (length e-mail-address))
for count from 0 to (1- (length e-mail-address)) for char-code = (char-code (aref e-mail-address count))
for new-char-code = (if (and (> char-code 64)(< char-code 123))(+ (mod (+ 13 char-code) 52) 65) char-code)
do (setf (aref new-string count) (code-char new-char-code)) finally (return new-string)))

Greg Menke

unread,
Jul 16, 2001, 11:52:50 PM7/16/01
to

> >
> >Perceptive observations, and very good questions.
> >Now, can we have the answers please??? :)
> >
> >> How far do you *need* to change the system you have.
> >
> >Need is a subjective, beholder's-eye variable, of course.
> >I think everyone is agreed that the system needs to be replaced.
> >
> >> Are you prepared to replace your existing "IT people".
> >
> >I think everyone has potential, though in varying
> >amounts. The existing people are crucial because they
> >are familiar with (and in the process of understanding
> >and documenting) the existing system.
>
> That means that they are indispensable *now*, but upon completion
> of the documentation, they will no longer be.
>
> I would just extract the functional requirements from the current users
> of the existing software. Construct a set of use cases and proceed
> from there.

Of course its not always that easy in practice. What he probably has
is a highly kludged, organic system with ill-defined modules (if
indeed there are any to begin with), written over many years by many
people. The database itself will have some parts in a reasonable
Normal Form, the rest will end up storing all kinds of magically
computed totals, intermediate results and lots of flag variables &
counters all over the place. Transitions of state between accounting
events are one-way and you better hope the tape backup of the system
works in case something blows up midway. In Clipper, database tables
are essentially global resources, there are no result sets, no
transactions and nothing to prevent you from corrupting indexes- it
tends to breed mess.

The normal way of working on the system is principally by
putting out various fires and with great dexterity, trying to avoid
starting others; then spending huge amounts of time tracking down very
subtle bugs and interactions discovered by users that don't understand
how the system really works- only the few screens they work with.

The problem is magnified because there is no comprehensive
documentation of any part of the system; presumably docs of some scope
were written here and there, but they're pretty well out of date and
they don't match up well. The existing programmer staff has probably
aquired a tolerable rule-of-thumb understanding of whats going on
(after having worked on the system for several years), enough for some
improvements here and there, yet large areas of the system remain
unknown because either they aren't used and no-one dares touch them,
or they've "always worked" to the extent no problems could be traced
into them.

There is no "just" when faced with a system like this. You can't
recompile it with something "mostly" compatible, you can't document it
at any reasonable cost, you don't know all the requirements it meets-
because they're buried way down in highly disparate parts of the
system. Pretty much the only thing you can do is start with the
basics; do the requirements analysis in detail, then the design-
incrementally, cyclically, whatever. Use cases certainly- anything to
capture knowledge, but thats only a small part of whats necessary.
Its helpful to salvage what data you can, and that can form the basis
for a hopefully substantial part of the new database- but a new
database is likely whats necessary, along with lots of work to make
the new system do what the old one did. It could easy take a couple
years just discovering all the little things the old system did &
trying to figure out how to implement them without making a mess.

On one legacy system redevopment I worked on, we spent 6 months
designing the throwaway data conversion utilities to haul over the 15
years of mainframe accounting data, and another 2 months or so getting
it right- and we STILL needed kludge tables to manage the results
computed at different times by the evolving algorithms used by the old
system as it matured. Its quite involved work, and people who learn
the system this way over several (many) years ARE indispensable. You
could remove all layers of management above the people maintaining the
system and it will pretty much run fine. Remove the veteran
programmers and the system will start to fall apart no matter how many
new people you hire.

Gregm

Kurt B. Kaiser

unread,
Jul 17, 2001, 1:54:30 AM7/17/01
to
aptpupil <aptpu...@yahoo.com> writes:

I might suggest that you consider changing your requirements to match what's
available. Do you really think you can design/develop/integrate something like
this more cheaply than you can buy it, customized if absolutely necessary?
Haven't there been a lot of implementations of exactly this kind of thing
recently?

Would you mind telling us how much "unaffordable" is in dollars?

Regards, KBK

Kurt B. Kaiser

unread,
Jul 17, 2001, 1:58:24 AM7/17/01
to
Tim Bradshaw <t...@cley.com> writes:

Is that because they are ripping you off, or does it really cost a lot to
implement? If the latter, does it make sense to do *one* from scratch,
including the learning curve??

KBK

Kurt B. Kaiser

unread,
Jul 17, 2001, 2:08:42 AM7/17/01
to
aptpupil <aptpu...@yahoo.com> writes:

> > You don't say what's actually wrong with it. Presumably you
> > have some reason for needing to replace it?
>
> Undocumented, original programmer left, DOS-based,
> buggy, brittle, hard to debug, hard to fix, hard to extend.
> No one wants to work on it any more. Who would.
> The language [Clipper] looks like a precursor to C, and
> the "GUI" programming is...80x24 char.
> Much of current effort is now on understanding and documenting it
> (the spaghetti of a bazillion program files and tables).


>
> > What does it do anyway?
>

> Using the example of the electric/gas/water company
> "Timbuktu Utilities" that I mentioned, it interfaces to the
> meter readers, does billing, collection, and generates
> reports. Various report data then goes, manually,
> eventually, into spreadsheets, QuickBooks, yet more paper
> forms and whatnot.
>
> The programs and databases are physically distributed (across
> great distances). The primary method of interaction and
> synchronization is...via daily transportation of floppy disk.
> Remote debugging is via dialup and pcAnywhere.
> Remote teller (of which there are quite a few) operation is
> via dialup and pcAnywhere. Big phone bill.
>
> > What should it do?
>
> The metering, billing, and collection programs (and the
> processes they assume) need to be improved, made more adaptible.
> Need to add operations and planning---GIS, facilities
> management, interface to substation and distribution
> system automation...

Good grief. The Augean stables.

Not what I'd call a "customer information" system. Forget my earlier comments.

Regards (and good luck :), KBK

Software Scavenger

unread,
Jul 17, 2001, 3:23:44 AM7/17/01
to
k...@ashi.footprints.net (Kaz Kylheku) wrote in message news:<W_L47.713097$166.14...@news1.rdc1.bc.home.com>...

> Which assumes they have it. More typically, whoever really knew the
> system longer works there. ;)

There's a good reason for that. Some programmers are 100 or even 1000
times more productive than others. And the same programmer can be
that much more productive from knowing the system they're working on
than before they learn it. Even if the typical situation involves
much smaller ratios, the overall ratio can still be very big from
combining those two factors. That means some programmers are
accomplishing many times more work than others. But they aren't
getting paid according to how much work they accomplish. It's typical
for a 1000% or even 100,000% increase in programmer productivity to
result in a small token gain in salary. In other words, the most
productive programmers are routinely underpaid by their employers,
while the least productive are overpaid. And then the employers
wonder where all the productive programmers went.

But there might be some better ways for a programmer to fight back
than to quit the job. In a job where dozens of average programmers
use a low-productivity programming language to develop a system
extremely wastefully, the programmer could rewrite the system in Lisp
in his spare time. Then, when the work to clean up the
low-productivity version seems overwhelming even to the employer, the
programmer can bring out the Lisp version, demonstrate how much more
robust and bug-free it is, and offer to let the employer use it in
exchange for a vice-president level job where the programmer would
hire a team of Lisp programmers to continue to develop and maintain
the Lisp version. It has to be a vice-president level job with
generous stock options because at levels below that the programmer
would still be underpaid. But there would not necessarily be any need
to spend any time as a manager. The new vice president could delegate
all such work to a secretary, and spend all his time doing Lisp
development with his new Lisp team.

Erik Naggum

unread,
Jul 17, 2001, 4:40:35 AM7/17/01
to
* Thomas F. Burdick

> While a fn-by-fn paraphrasing of code is not sufficient documentation, it
> *can* be an important part of documenting a system, if the overall system
> is also described, and the code you're paraphrasing is opaque (this will
> allow someone maintaining the code to find the relevant functions without
> being forced to puzzle through all the code). It is, of course, of
> questionable value if the code is easy to read.

I think this should be in the code, very close to the function body,
possibly _in_ the functionb ody. Documentation strings in Common Lisp
should have this purpose and should be well supported. (We currently
lack the ability to extract documentation for a system with a few simple
functions (they are easy to write, though).) However, taking such "code
documentation" out of the code is often a recipe for disaster: You soon
lose the connection between the actual and the documented code. Both
comments and documentation strings serve a purpose. I think it is a good
idea to comment on things only in close proximity to that commented on.

> Speaking from experience,

Experience is what you get in the absence of what you wanted or expected.

#:Erik
--
Travel is a meat thing.

Erik Naggum

unread,
Jul 17, 2001, 4:50:30 AM7/17/01
to
* cubic...@mailandnews.com (Software Scavenger)

> But there might be some better ways for a programmer to fight back than
> to quit the job. In a job where dozens of average programmers use a
> low-productivity programming language to develop a system extremely
> wastefully, the programmer could rewrite the system in Lisp in his spare
> time. Then, when the work to clean up the low-productivity version seems
> overwhelming even to the employer, the programmer can bring out the Lisp
> version, demonstrate how much more robust and bug-free it is, and offer
> to let the employer use it in exchange for a vice-president level job
> where the programmer would hire a team of Lisp programmers to continue to
> develop and maintain the Lisp version.

A brilliant solution to the problem. This would also get rid of the VPs
who are rabidly anti-Lisp simply because they fear that they would not be
able to maintain the software with "drones off the street" and who prefer
to hire cheap labor instead of good labor.

Tim Bradshaw

unread,
Jul 17, 2001, 6:01:50 AM7/17/01
to

It's mostly because software costs a lot. Consider something like a
telco billing system. Maybe it takes 10 people a year to write this
(including management and presales, probably a conservative
estimate). At a charge-out rate of 1,000/day (maybe a bit high, but
balance it with the 10), 200 worked days/year, that's (* 10 200 1,000)
or 2,000,000 in development costs.

You might sell 5 of these things, so naively that looks like 400,000
each. But a telco will definitely not buy such a system off-the-shelf:
the way they bill is central to their operations and one of the ways
they distinguish themselves. So you need to bespeak it per customer,
maybe this takes a 3 people 3 months (2 programmers, 1 requirements
person), say 150 days at 1000/day again thats 150,000 on the cost or
550,000.

The billing system is how the telco makes money: if the system is down
it has no revenue stream. So they want extremely plausible 24x7
support for this thing. You probably charge this as a support
contract rather than up front. How you cost this is a bit flexible
(probably scales with the system).

Risk premium. Maybe you sell 5, maybe you sell 1, maybe you sell none
at all. Double the cost for this? Say 1,100,000 + support. So you
need to sell 2 to break even.

Insurance: if it fouls up badly they will sue you. Don't know how
much this is. If your a big company you probably self-insure, if you
aren't they'll want you to have insurance so you don't just go
bankrupt.

Licenses for constituent parts. Oracle is not cheap for systems like
this, other stuff you use. Don't know how much this is, say 100,000 per
system sold. 1,200,000 + support.

These are obviously back-of-the-news-article calculations, but you see
how it works. This stuff is expensive!

As to whether it makes sense to do one internally. I think it can.
Buying one will involve a whole lot of requirements capture and
misunderstanding by the company who are trying to sell it to you,
which can be really painful. And you don't own the result!

--tim


Tim Bradshaw

unread,
Jul 17, 2001, 6:35:16 AM7/17/01
to
* Software Scavenger wrote:
> But there might be some better ways for a programmer to fight back
> than to quit the job. In a job where dozens of average programmers
> use a low-productivity programming language to develop a system
> extremely wastefully, the programmer could rewrite the system in Lisp
> in his spare time. Then, when the work to clean up the
> low-productivity version seems overwhelming even to the employer, the
> programmer can bring out the Lisp version, demonstrate how much more
> robust and bug-free it is, [...]

An approach I've tried (not successfully, people didn't bite, though I
think it's a good idea) is the secret-reimplementation-as-prototype
technique.

The scenario is that you have a big, rotting, system which needs
something doing to it fairly urgently before it sinks into
unmaintainability. Unfortunately no-one really knows what the current
system does, in the usual way.

People are jumping up and down to do a new
C++-Java-XML-CORBA-UML-distributed-peer-to-peer-client-server-frobnomatic
good-buzzwords-on-your-CV thing. But everyone (or at least the
management) can see that no one knows what is actually needed and just
leaping in will result in disaster. Various attempts have been made
to tie down the old system with UML to no effect, and anyway you don't
want another old system, you want something else.

At this point you take the managers aside and suggest a rapid
throwaway prototype which can be used to explore ideas for the new
system. It will be done in a very flexible language, very fast, and
will be intended purely as a playground to see what a new system
should look like. Maybe it takes a couple of people a couple of
months to do. Maybe it will also be a way of making sure the couple
of really good people you have on the old system don't leave through
sheer frustration at having to work so hard on something that is
obviously doomed.

After getting agreement, you pick the best people (who will not be
averse to using Lisp for a prototype, that's kind of a definition of a
good programmer, even a good C++ programmer), and you go off and write
your prototype. You write it pretty carefully and well (you have good
programmers, so you can!)

Two months in a couple of things happen. The client starts asking for
changes to the old system which you probably can't make in realistic
time, if at all. The old system is also suffering from reliability
issues and rot because the good programmers are on the new system and
they really relied on these people (even though they didn't realise
how much). Things look fairly bad.

At this point you do two things. Firstly you make the changes to the
prototype system corresponding to those needed for the old system in a
day or so. Secondly you go to the managers and suggest that, as a
stopgap, they could deploy the prototype - anything has to be better
than the current situation. By this time they've spent the last week,
solidly on the phone to the client firefighting and they'd agree to
anything that lets them get some sleep.

So you deploy the prototype. It works, you get promoted to
assistant-deputy-vice-King, marry the King's daughter, succeed to the
throne and live happily ever after.

That's my theory anyway.

--tim

Tim Bradshaw

unread,
Jul 17, 2001, 6:12:02 AM7/17/01
to
* Erik Naggum wrote:

> I think this should be in the code, very close to the function body,
> possibly _in_ the functionb ody. Documentation strings in Common Lisp
> should have this purpose and should be well supported. (We currently
> lack the ability to extract documentation for a system with a few simple
> functions (they are easy to write, though).) However, taking such "code
> documentation" out of the code is often a recipe for disaster: You soon
> lose the connection between the actual and the documented code. Both
> comments and documentation strings serve a purpose. I think it is a good
> idea to comment on things only in close proximity to that commented on.

This is a really good point. I've looked at a sufficient number of
`detailed design documents' which had drifted so far from the code as
to be completely useless to never want to do that again. The
situation is also unstable - once the thing starts drifting it becomes
sufficiently painful to change it to be in-line with the code that it
rapidly discourages any change at all and it rots. And it's often
kept in Word with no version control (hello?) so it's all just
nightmarish. Detailed documentation needs to be glued to the code
with strong glue.

I think it's important that you can *extract* the documentation
mechanically, so test people can look at the documentation while not
looking at the code, so they don't end up thinking the documentation
is correct when it's wrong because they can see the code is correct.
Lisp should be really good at this, of course!

--tim

DJC

unread,
Jul 17, 2001, 7:48:09 AM7/17/01
to
On 17 Jul 2001 11:01:50 +0100, Tim Bradshaw <t...@cley.com> wrote:


>[...]. So you need to bespeak it per customer,

the evolution of language! :-)

Greg Menke

unread,
Jul 17, 2001, 7:50:27 AM7/17/01
to

> Why not just store them as strings. If you keep metadata about what types are stored in which columns, you can
> safely coerce the values back and forth.
>
> Blobs are painful.

Strings would probably be the way to do it- though it kind of sucks to
not be able to directly query against them. Even so, we ran into the
rounding problems mostly on the financial side of the system where
queries were pretty much limited to a record id, so it wouldn't have
been a big loss there.

Gregm

Paolo Amoroso

unread,
Jul 17, 2001, 8:27:27 AM7/17/01
to
On Tue, 17 Jul 2001 08:40:35 GMT, Erik Naggum <er...@naggum.net> wrote:

> * Thomas F. Burdick
> > While a fn-by-fn paraphrasing of code is not sufficient documentation, it
> > *can* be an important part of documenting a system, if the overall system

[...]


> I think this should be in the code, very close to the function body,
> possibly _in_ the functionb ody. Documentation strings in Common Lisp

Do you mean something like the way comint.el in Emacs is documented?

cbbr...@hex.net

unread,
Jul 17, 2001, 8:59:48 AM7/17/01
to
Paolo Amoroso <amo...@mclink.it> writes:
> On Tue, 17 Jul 2001 08:40:35 GMT, Erik Naggum <er...@naggum.net> wrote:
>
> > * Thomas F. Burdick
> > > While a fn-by-fn paraphrasing of code is not sufficient documentation, it
> > > *can* be an important part of documenting a system, if the overall system
> [...]
> > I think this should be in the code, very close to the function body,
> > possibly _in_ the functionb ody. Documentation strings in Common Lisp

> Do you mean something like the way comint.el in Emacs is documented?

That seems like a not-too-horrible example of this; there's perhaps a
little _too_ much stuff going on in comint.el, but in suggesting where
docs might go, I'd certainly be easy to convince that putting them in
comint.el is likely to be better than sticking them in
/usr/doc/emacs/emacs-base/comint.txt

The further the documentation gets away from the code, the more likely
it is that changes resulting from production problems will cause the
documentation to rot.

It may be _very marginally_ easier to enforce "keeping up docs" if
working in C/C++, where you have to do a "big bang compile" to
generate something to push out to production, as compared to more
dynamic [e.g. - not quite so much nests of chunks of binary-compiled
stuff] languages where it might actually be source code that gets
pushed to production.

But I'd certainly agree with the notion that documentation that is in
the source code will have a better "shelf life" than documentation
that sits some place quite independent of the source code...
--
(concatenate 'string "aa454" "@freenet.carleton.ca")
http://vip.hex.net/~cbbrowne/lsf.html
0 7 * * * echo "...Linux is just a fad" | mail bi...@microsoft.com \
-s "And remember..."

aptpupil

unread,
Jul 17, 2001, 12:44:01 PM7/17/01
to
This description is dreadfully, depressingly accurate.
I read it with a sinking feeling.

With one minor exception: The system was written over about
10 years mostly by one person (X) with small contributions
by his series of underlings, only one of whom remains.
Unfortunately, X never mentored and developed his charges.
Whenever anyone wanted to expand their understanding beyond
their particular limited sphere, he would just say,
basically, "Go read the code."

> The problem is magnified because there is no comprehensive
> documentation of any part of the system; presumably docs of some scope
> were written here and there, but they're pretty well out of date and
> they don't match up well.

In response to management's desire for the system to be
documented, X produced: 1) table information (a list of all the
tables and their columns---names, types, cryptic oneliner descriptions;
and 2) user screen information (printouts of the many screens
with some flowchart-like, user-guide-type information).
Needless to say, (1) is out of date and no user is known
to have ever read (2).

When not fighting fires (e.g., index file corruptions, mysterious
bugs) and doing must-do changes or extensions, the focus now is on
properly and appropriately documenting the system. The main tool
in this effort is...Excel spreadsheets, into which information
about the roughly 100 tables, 200 column names, and few
dozen programs are being entered. The first level of
discovery---which programs are called by which user menu
items; which programs use which tables; and an approximate
(i.e., not necessarily standards quality) description
of each column---is essentially done.

I don't suppose there are any silver bullets/tools to this
process. How about even just some plastic ones...?

The Lisp "reimplementation-as-prototype" approach described
by Tim Bradshaw in a nearby subthread might work. Am I
hoping it will, but this could be a delusion. In any case,
it crucially depends (as does any mainstream approach) on
understanding the existing processes, programs, and data
"well enough"...

Thomas F. Burdick

unread,
Jul 17, 2001, 1:18:37 PM7/17/01
to
Tim Bradshaw <t...@cley.com> writes:

I was thinking of a literate programming system for the
tightly-tied-to-the-code documentation. One could certainly build an
LP system based on docstrings. The point is to regenerate the
documentation on each build, keeping the pretty docs up-to-date with
the code, and stuck next to eachother in the source. It also allows
the code and docs to be ordered in whatever way is easiest to use as
documentation, not in whatever restricted order the compiler requires
(this can be important for languages with more severe restrictions
than Lisp)

* Erik Naggum wrote:
> * Thomas F. Burdick

> > Speaking from experience,
>
> Experience is what you get in the absence of what you wanted or expected.

No, experience means you've done it at least once before, whether well
or poorly. Period. In this context, obviously it means that the code
being discussed was less than what one would want (or it would not
have needed a concerted effort to make it understandable -- the
difficult bits would be commented, and the rest would be at least
somewhat readable). However, given that, it was overall a positive
experience. It involved not having to reimplement the whole system,
but still having a maintainable code base.

Larry Loen

unread,
Jul 17, 2001, 2:04:39 PM7/17/01
to
cbbr...@hex.net wrote:

>
> But I'd certainly agree with the notion that documentation that is in
> the source code will have a better "shelf life" than documentation
> that sits some place quite independent of the source code...
> --

Perhaps, but in my experience, not better enough. The crux of this problem is not technology, but management of human beings.

There is, as of yet, no real substitute for the original designer and/or the one that did the major re-write five years later.

I fairly recently had the experience of looking at some very important commercial code (a page fault handler) that I wrote in the late '70s. Since it
was performance and function critical, I could identify few surviving lines of code that I personally wrote, at least none on any remotely interesting
paths. Things do change.

However, I could see that virtually all of my "block" comments were pretty much to-the-byte intact and had suffered almost no change in over twenty
years, even though I had long moved on to other assignments.

Now, some of this is because a lot of the fundamental design decisions I made were never really reversed and even fewer of the more arbitrary ones
that never changed. But some of it was that, despite the fact I was a short walk away at all times doing other jobs, I suspect there is a marked
reluctance for people to interfere with the words from the originator -- a reluctance to lose original intent, even if it is a bit of a mismatch with
the current code. Or, maybe they were too damn busy adding function, who knows? Regardless, the prose I wrote seems to have been immortal.

Why is this relevant? Any tool I know of that generates design documentation -- especially prose descriptions of "what was wanted" instead of "what
was coded" -- is going to preferentially live in block style comments ahead of some bunch of code.

So, if folks are relucant to change what the originator wrote, for whatever reasons, it is doomed to go out of date. Short of harnessing the original
designers to their chairs, I don't have an easy anwer.


Larry


brl...@my-deja.com

unread,
Jul 17, 2001, 2:38:00 PM7/17/01
to
aptpupil <aptpu...@yahoo.com> writes:

> Unfortunately, X never mentored and developed his charges.
> Whenever anyone wanted to expand their understanding beyond
> their particular limited sphere, he would just say,
> basically, "Go read the code."

Has this been tried? Is it hard to find the corresponding code for any
given program? Is it difficult to find the part of the code that
corresponds to some particular functionality?

Documentation wants to be obsolete.

--
Bruce R. Lewis http://brl.sourceforge.net/
I rarely read mail sent to brl...@my-deja.com

aptpupil

unread,
Jul 17, 2001, 2:47:40 PM7/17/01
to
> I was thinking of a literate programming system for the
> tightly-tied-to-the-code documentation. One could certainly build an
> LP system based on docstrings. The point is to regenerate the
> documentation on each build, keeping the pretty docs up-to-date with
> the code, and stuck next to eachother in the source.

Good gawd! 2001 already and we are still thinking - talking endlessly
about coulda - woulda - shoulda. What's the deal?! Are we still
going to be wishing for a (non-hack, non-DoItYouself) literate
programming system for Lisp in 2010? 2020? 2030?

Assuming that, as of this late date, there is still no standardized
or at least widely used or de facto LP system for Common Lisp,
what does this show?

1. It is low in priority.

2. The emphasis in Lisp (the language and the community) is not
so much in practical, real world software engineering,
but in its traditional esoteric researchy ivory tower niche.
(Implying that it is unlikely that Lisp will ever escape that,
as it is missing the opportunity to meet unmet needs in
software engineering, or at least, *this* particular need.)

3. No one can agree on exactly what an LP should do and produce, and
there is a lot of art (including aesthetics) and multiple purposes in
documentation. For example, witness the balancing act between...
;;; Top or function level comments
(defun quux ()
;; Mid level comments
(cond ()
(zork ; and line level comments

4. ???

Greg Menke

unread,
Jul 17, 2001, 3:12:03 PM7/17/01
to

> This description is dreadfully, depressingly accurate.
> I read it with a sinking feeling.
>
> With one minor exception: The system was written over about
> 10 years mostly by one person (X) with small contributions
> by his series of underlings, only one of whom remains.
> Unfortunately, X never mentored and developed his charges.
> Whenever anyone wanted to expand their understanding beyond
> their particular limited sphere, he would just say,
> basically, "Go read the code."

heh. I've been there, done that- on both sides. Its a tough
situation, not hopeless just tough.

>
> > The problem is magnified because there is no comprehensive
> > documentation of any part of the system; presumably docs of some scope
> > were written here and there, but they're pretty well out of date and
> > they don't match up well.
>
> In response to management's desire for the system to be
> documented, X produced: 1) table information (a list of all the
> tables and their columns---names, types, cryptic oneliner descriptions;
> and 2) user screen information (printouts of the many screens
> with some flowchart-like, user-guide-type information).
> Needless to say, (1) is out of date and no user is known
> to have ever read (2).

Pretty much typical. The one-liner descriptions of the tables and
fields will be somewhat useful- even if they are sort of up to date.
Sometimes a little information can do a lot.



> When not fighting fires (e.g., index file corruptions, mysterious
> bugs) and doing must-do changes or extensions, the focus now is on
> properly and appropriately documenting the system. The main tool
> in this effort is...Excel spreadsheets, into which information
> about the roughly 100 tables, 200 column names, and few
> dozen programs are being entered. The first level of
> discovery---which programs are called by which user menu
> items; which programs use which tables; and an approximate
> (i.e., not necessarily standards quality) description
> of each column---is essentially done.
>
> I don't suppose there are any silver bullets/tools to this
> process. How about even just some plastic ones...?

I think this is a good beginning. If you have any users that have a
detailed understanding of the system, from front to back- or at least
thoroughly understand how their parts works, they are the who you must
get hold of. It will likely require long conferences where you hash
out what the data is & how you get it & how you're supposed to process
it. They are the ones who can tell you what the data means.
Technological prowess is secondary. I think you'll find they are the
best resource you have for requirements & identifying the shortcomings
of the old system which you now have the opportunity to overcome.



> The Lisp "reimplementation-as-prototype" approach described
> by Tim Bradshaw in a nearby subthread might work. Am I
> hoping it will, but this could be a delusion. In any case,
> it crucially depends (as does any mainstream approach) on
> understanding the existing processes, programs, and data
> "well enough"...

I've never been in a position to try this approach, but I think it
might be worth attempting, particularly if self-contained parts of the
system can be reimplemented that way. We've had good luck with
creating prototypes for critical subsystems, then fleshing them out
into the full system- its been very helpful to sanity test designs,
because you always learn more requirements as you try to actually
implement something.

A full-blown redevlopment of a system like the one you describe is a
BIG job- particularly when the target is moving due to various
"critical" enhancements, bug fixes, etc.. I think its important to
get as comprehensive a map of the database as possible. If you end up
with a significantly flawed database, you'll end up with at least as
big a mess after years of work.

Its of course impossible to get a complete map of your existing system
because theres lots of funky stuff buried down inside that makes no
sense even when you do track it down, and parts of the database are
holding state which the program uses as it processes; ie- some parts
of the database never have stable state, so there is no way of
representing the data even if you wanted to because its an
intermediate, undefined product of the labyrinthine code. These are
the parts you'll go crazy over.

I think you should resist the temptation to implement a
"do-everything" system, and keep a very strict control over
requirments creep. The tendency will be to add more to fix possibly
egregious shortcomings- but you have to keep the system do-able. It
would be far better to implmement something that takes care of the
essentials, but leave room for enhancement.

Greg Menke

Thomas F. Burdick

unread,
Jul 17, 2001, 3:32:25 PM7/17/01
to
aptpupil <aptpu...@yahoo.com> writes:

> > I was thinking of a literate programming system for the
> > tightly-tied-to-the-code documentation. One could certainly build an
> > LP system based on docstrings. The point is to regenerate the
> > documentation on each build, keeping the pretty docs up-to-date with
> > the code, and stuck next to eachother in the source.
>
> Good gawd! 2001 already and we are still thinking - talking endlessly
> about coulda - woulda - shoulda. What's the deal?! Are we still
> going to be wishing for a (non-hack, non-DoItYouself) literate
> programming system for Lisp in 2010? 2020? 2030?
>
> Assuming that, as of this late date, there is still no standardized
> or at least widely used or de facto LP system for Common Lisp,
> what does this show?

It shows that most programmers don't use LP, in Lisp or in any
language. My LP system of choice (noweb) works perfectly well with
Lisp. However, I never use it with Lisp; I don't feel like I'm
missing anything, either. I guess the ability to explore the system
in other ways satisfies the same need that LP wishes to address.

> 1. It is low in priority.

I think so.

> 2. The emphasis in Lisp (the language and the community) is not
> so much in practical, real world software engineering,
> but in its traditional esoteric researchy ivory tower niche.
> (Implying that it is unlikely that Lisp will ever escape that,
> as it is missing the opportunity to meet unmet needs in
> software engineering, or at least, *this* particular need.)

In fact, just the opposite. All of my purely-academic impulses tell
me to use LP for all my lisp projects. Thinking practically, however,
it would be a waste of time -- unlike C where LP is a vitally
important tool to understanding large systems, I don't get the same
value per energy expended using LP with Lisp.

> 3. No one can agree on exactly what an LP should do and produce, and
> there is a lot of art (including aesthetics) and multiple purposes in
> documentation. For example, witness the balancing act between...
> ;;; Top or function level comments
> (defun quux ()
> ;; Mid level comments
> (cond ()
> (zork ; and line level comments

I don't think an LP system should just extract comments. In fact, I
don't think it should extract them at all. They should stay in the
code, and docstrings should probably become part of the documentation.

Tim Bradshaw

unread,
Jul 17, 2001, 3:18:38 PM7/17/01
to
* aptpupiltx wrote:

> Assuming that, as of this late date, there is still no standardized
> or at least widely used or de facto LP system for Common Lisp,
> what does this show?

> 1. It is low in priority.

Yes. Lack of LP systems doesn't seem to have held back other
languages, and I should think that for languages with LP systems they
are not widely used commercially.


> 2. The emphasis in Lisp (the language and the community) is not
> so much in practical, real world software engineering,
> but in its traditional esoteric researchy ivory tower niche.
> (Implying that it is unlikely that Lisp will ever escape that,
> as it is missing the opportunity to meet unmet needs in
> software engineering, or at least, *this* particular need.)

If it was such an urgent need it would be more widely used in other
languages. No.

> 3. No one can agree on exactly what an LP should do and produce, and
> there is a lot of art (including aesthetics) and multiple purposes in
> documentation. For example, witness the balancing act between...
> ;;; Top or function level comments
> (defun quux ()
> ;; Mid level comments
> (cond ()
> (zork ; and line level comments

Certainly this.

--tim

Tim Moore

unread,
Jul 17, 2001, 3:39:08 PM7/17/01
to
On Tue, 17 Jul 2001, aptpupil wrote:

> > I was thinking of a literate programming system for the
> > tightly-tied-to-the-code documentation. One could certainly build an
>

> Good gawd! 2001 already and we are still thinking - talking endlessly
> about coulda - woulda - shoulda. What's the deal?! Are we still
> going to be wishing for a (non-hack, non-DoItYouself) literate
> programming system for Lisp in 2010? 2020? 2030?

I'm sure plenty of people have done this over the years; I saw one such a
system in 1987.

> Assuming that, as of this late date, there is still no standardized
> or at least widely used or de facto LP system for Common Lisp,
> what does this show?
>
> 1. It is low in priority.

Yes.

>
> 2. The emphasis in Lisp (the language and the community) is not
> so much in practical, real world software engineering,
> but in its traditional esoteric researchy ivory tower niche.

Or, the emphasis in Literate Programming is not so much in practical, real


world software engineering, but in its traditional esoteric researchy
ivory tower niche.

> (Implying that it is unlikely that Lisp will ever escape that,
> as it is missing the opportunity to meet unmet needs in
> software engineering, or at least, *this* particular need.)

I'm not sure you why think that Literate Programming is the sine qua non
of software engineering. It's never been clear to me that beautiful,
automatically generated program documentation is a substitute for powerful
editors, browsers and navigation tools. Now that we have HTML and web
browsers the lines may be blurred somewhat, but a web browser is still,
IMHO, a crappy way to navigate a program.

Tim


Tim Bradshaw

unread,
Jul 17, 2001, 4:15:37 PM7/17/01
to
> In fact, just the opposite. All of my purely-academic impulses tell
> me to use LP for all my lisp projects. Thinking practically, however,
> it would be a waste of time -- unlike C where LP is a vitally
> important tool to understanding large systems, I don't get the same
> value per energy expended using LP with Lisp.

Actually, has LP ever been seriously used *outside* academia?

--tim

Erik Naggum

unread,
Jul 17, 2001, 4:21:47 PM7/17/01
to
* aptpupil <aptpu...@yahoo.com>

> Assuming that, as of this late date, there is still no standardized or at
> least widely used or de facto LP system for Common Lisp, what does this
> show?

That it is a really, really, _really_ bad idea.

Daniel Barlow

unread,
Jul 17, 2001, 6:16:26 PM7/17/01
to
t...@apocalypse.OCF.Berkeley.EDU (Thomas F. Burdick) writes:

> I don't think an LP system should just extract comments. In fact, I
> don't think it should extract them at all. They should stay in the
> code, and docstrings should probably become part of the documentation.

I'm playing with something in the general LP/javadoc space with
db-sockets. Some random observations:

- there are two distinct possibilities for what you want the system
to do: should it make pretty docs from existing source code, or
should it make pretty docs only from source code that was written
with automatic doc extraction in mind? The latter is, obviously,
much simpler

- docstrings and API documentation serve different purposes: a
docstring is a summary or aide-memoire for the user, whereas API
docs are presumed to be a little more comprehensive. Ruining the
normal use of docstrings (browsing the code in an interactive
development environment) to get pretty HTML is a bad idea.

- comments and API docs are even more different. It's rare that a
comment makes that much sense at all apart from the code it's
attached to.

- the aim is not to replace interactively browsing the source in a
decent development environment. The aim is to create overview docs
that you can put up on the web so that users can decide whether
it's worth installing and investigating the package in the first
place. People can print them out and read them on the train.
Google can find them and index them so that web searches for "foo
in lisp" turn up your packages. &c.

That last point is important.

The current state for db-sockets documentation is that

1) it processes the files in the order that defsystem would

2) it prints arglists and doc strings for functions, methods, classes,
slots, and packages. It also prints any multiline comment introduced
by #||, so that the programmer can add more detailed explanations
of functions than he would want to put in a doc string, or conceptual
overviews of files or sections.

(It achieves this last feat by replacing the dispatch macro for #|
when it reads the source files. I don't know that this is necessarily
portable, but it works in all the implementation I have tried it in)

3) it does heuristic HTML markup (e.g. attempting to create links to
symbols when it detects words in UPPERCASE)

4) the implementation is sucky. Among other sins, it uses READ and
interns all manner of rubbish in the current package.

5) making the output pretty is still on the TODO list.

You can see how it works by getting db-sockets from any cCLan archive,
and see the typical output (temporarily) at
http://ww.telent.net/db-sockets.html. One day soon I'll be changing
the format to look more like the layout of the LISA docs, or the
reference section of AMOP

(LISA docs: http://lisa.sourceforge.net/ref-guide.html)


-dan

--

http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources

Greg Menke

unread,
Jul 17, 2001, 6:26:30 PM7/17/01
to

> > Unfortunately, X never mentored and developed his charges.
> > Whenever anyone wanted to expand their understanding beyond
> > their particular limited sphere, he would just say,
> > basically, "Go read the code."
>
> Has this been tried? Is it hard to find the corresponding code for any
> given program? Is it difficult to find the part of the code that
> corresponds to some particular functionality?
>
> Documentation wants to be obsolete.

I'm sure it has, and the code will range from straghtforward to
indecipherable. To the extent it follows a coherent design, "read the
code" isn't so bad, but large parts of system are so hacked up and so
dependent on dispersed global state, its pretty much impossible to
understand without understanding the whole subsystem- if not most of
the whole system itself. Often the problem is actually trying to find
the corresponding code- its spread all over the place, both in source
and "temporally", meaning, different parts of the system affect each
other via ill-specified global variables and data stuffed away in
"temporary" tables.

Documentation can become obsolete, but only when people let it get
that way. 3-ring binders full of boilerplate docs meant to satisfy
some TQM "mandate" are intrinsically useless- the important docs are
those distilled from the development of the system, and those are
critical. You can get lots of that in well-commented, well-designed
code- but if the code falls short, then it doesn't serve very well as
documentation.

Gregm

Reini Urban

unread,
Jul 24, 2001, 9:31:15 PM7/24/01
to
Tim Bradshaw <t...@cley.com> wrote:
<a good fairy tail>

: So you deploy the prototype. It works, you get promoted to


: assistant-deputy-vice-King, marry the King's daughter, succeed to the
: throne and live happily ever after.

and then you wake up.

it might happen, but who knows realistic war stories with this kind of happy
end? I only know it the other way round.

e.g.
what happened to gabriel?
what happened to will deakens prototype app?
what will happen to viaweb now c++ folks have to maintain it.
what will happen to/with autodesk in the next years?

you can build such a prototype.
you cannot convice management to invest in this.
you are allowed to rewrite in c++ then.

good story, sad end.
--
Reini Urban

Reini Urban

unread,
Jul 24, 2001, 9:44:47 PM7/24/01
to
Wade Humeniuk <hume...@cadvision.com> wrote:
: Though India is of course close and there are lots of
: programmers there. The rarified atmosphere should be
: conducive to programming in Lisp.

but there are no indians doing lisp. it least in india.
they are doing java/c++ and most likely c# soon.

looks like you'll have to meet in the middle somewhere.
asp/vb/c++ vs lisp/functional/declarative => python/java/c#
--
Reini Urban

Steve Wart

unread,
Jul 24, 2001, 10:56:55 PM7/24/01
to
"Reini Urban" <rur...@x-ray.at> wrote in message
news:9jl8af$ats$4...@fstgss02.tu-graz.ac.at...

>
> but there are no indians doing lisp. it least in india.


According to 1998 figures there are over 1 billion people in India.

How can you say that *none* of them are doing Lisp, when India has some of
the highest standards in the world for teaching English and Mathematics (and
computation theory, I presume).

How can you defend this outrageous statement?

--
Steve (steve at wart dot ca ICQ 50919689)

raj

unread,
Jul 25, 2001, 9:12:11 AM7/25/01
to
On 25 Jul 2001 01:44:47 GMT, Reini Urban <rur...@x-ray.at> wrote:

>Wade Humeniuk <hume...@cadvision.com> wrote:
>: Though India is of course close and there are lots of
>: programmers there. The rarified atmosphere should be
>: conducive to programming in Lisp.
>
>but there are no indians doing lisp.

Check:
http://www.cs.rice.edu/~shriram/LispM/ for a picture of an honorary
Indian Lisp machine:
"Our newest effort commemorated India's fiftieth anniversary as a
free state (August 15, 1997). We draped the machine in many fine
fabrics,papered it with currency bills, historic speeches and
suchlike, and adorned it with various Indian trinkets. "
( Shriram is involved with the Dr Scheme project)

Also, I hear that there is a Gnu-India Lisp project afoot.
I am afraid that I have no details


Reini Urban

unread,
Jul 29, 2001, 8:41:31 AM7/29/01
to
raj <isra...@optushome.com.au> wrote:

good to hear. I'm very interested. I worked together with some indian lisp
programmers (living in the states) some years ago.

what I knew from india *in the public* (not form indians on the US, who are
doing a lot of lisp, not also at rice) was nothing about lisp at all.
there was one autolisp guy but doesn't count.
having some numbers wouldn't be bad.
--
Reini Urban
http://xarch.tu-graz.ac.at/acadwiki/AutoLispFaq

0 new messages