Ruby performance woes
Scott Burson
http://www.regdeveloper.co.uk/2008/04/21/bray_ruby_rails/
You'd think there would be some way for Lisp to fill this gap.
-- Scott
Sean T Allen
a lot of the people who got pulled into the rails orbit came from php,
i doubt you'd move them over to lisp. given the emphasis in the ruby
community on beautiful looking code, i can also see problems for lisp
there. you'd need one of those wham-bam overwhelming flashy demos to
make progress with the first group ( like how activerecord db
introspection and scaffolding wow'ed em ). the 2nd well i don't know
how you'd get them over the whole 'all those ()s just look ugly'.
a side by side fancy comparision of advanced techniques wouldnt hurt either.
ruby is def. way out in front on the community being newbie friendly
compared to lisp.
-says the guy who threw his hands up in disgust a few months ago and
decided to only use ruby as a shell and perl script replacement-
Jeff
Scary.
Tim Bradshaw
> You'd think there would be some way for Lisp to fill this gap.
It is almost always the case that "slow language implementatons" are
in fact "language implementations in which a huge amount of *really*
shit code has been written". Lisp implementatons were, of course, the
first for which this mistake was made, but I think they have long been
eclipsed by Java, and probably soon by Ruby.
--tim
Pascal J. Bourguignon
Well, lisp is a meta-programming programming language. Could the
problems with Ruby be that it hasn't been implemented in Lisp?
After all, all these P* languages, Php, Perls, Python, Ruby, Java,
they're all nice domain specific programming language (even if some
users have strange syntactic tastes). The shame is that they've not
been developed in lisp. Then they'd benefit of the lisp compiler and
run-time compiler.
Lisp advocates shouldn't try to convince the programmers using these
programming languages, but the programmers who design and implement
these programming languages.
;-)
--
__Pascal Bourguignon__
Rainer Joswig
I THINK THE LAST THREE CHARACTERS ARE THE MOST IMPORTANT
OF THAT MESSAGE ... OOPS ... CAPS LOCK ... READing too
much old code lately ...
Christophe
Hi all,
I am not sure that "Java have eclipsed Common Lisp".
When I see the money generate by Java for Sun, I am really not sure.
It's always the confusion between Quality and Quantity
Quality for Lisp, of course :)
Concerning Ruby, it's just a Smalltalk without image/repository, IDE
and Speed.
Just compared Smalltalk + Seaside vs Ruby on Rails.
Best Regards
Christophe
Duane Rettig
Or perhaps become implementors of those languages themselves:
http://common-lisp.net/project/clpython/
> ;-)
>
> --
> __Pascal Bourguignon__
--
Duane Rettig du...@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
ddd
<p...@informatimago.com> wrote:
> After all, all these P* languages, Php, Perls, Python, Ruby, Java,
> they're all nice domain specific programming language (even if some
> users have strange syntactic tastes). The shame is that they've not
> been developed in lisp. Then they'd benefit of the lisp compiler and
> run-time compiler.
>
Php in Scheme:
Ken Tilton
http://gitorious.org/projects/hunchncells
hth, kenny
--
http://smuglispweeny.blogspot.com/
http://www.theoryyalgebra.com/
"I've never read the rulebook. My job is to catch the ball."
-- Catcher Josh Bard after making a great catch on a foul ball
and then sliding into the dugout, which by the rules allowed the
runners to advance one base costing his pitcher a possible shutout
because there was a runner on third base.
"My sig is longer than most of my articles."
-- Kenny Tilton
Tim Bradshaw
<christophe.allegr...@birdtechnology.net> wrote:
> I am not sure that "Java have eclipsed Common Lisp".
>
I'm guessing you misunderstood what I was saying: Java has eclipse CL
in the "language most people think is slow when actually it's the
grotesquely awful code people write in it" field, this being distinct
from any other one.
Apologies if you did understand me.
Tim Bradshaw
wrote:
> After all, all these P* languages, Php, Perls, Python, Ruby, Java,
> they're all nice domain specific programming language
The domain of Perl being "world domination".
Jason Nielsen
>
> Or perhaps become implementors of those languages themselves:
>
> http://common-lisp.net/project/clpython/
>
Too bad it only runs on Allegro as I doubt this project will gain any
traction among the Pythonista relying on a commercial lisp as its "vm".
If a common lisp system was the "jvm" of today I'd be a happy man... an
open source common lisp system of course ;-)!
Jason
Willem Broekema
> >http://common-lisp.net/project/clpython/
>
> Too bad it only runs on Allegro
CLPython works completely on LispWorks as well.
The website is outdated, sorry about that. The porting was done in the
last weeks, to have something to brag about at the ECLM last weekend.
Support for CMUCL or SBCL is next, then it's time for a 1.0 version.
- Willem
Jason Nielsen
Well I stand corrected by the man himself ;-)! Just out of curiosity I'm
assuming you have dropped the reliance on Allegro's version of yacc.
What of the environments stuff? If there is nothing major stopping
porting I'd be happy to help out with the leg work of adding a few #+sbcl
(sb-pcl:.... etc. to get this working on sbcl and cmucl.
Jason
Brian Adkins
> Scott Burson wrote:
> > Just came across this...
>
> > http://www.regdeveloper.co.uk/2008/04/21/bray_ruby_rails/
>
> > You'd think there would be some way for Lisp to fill this gap.
>
> http://gitorious.org/projects/hunchncells
"Yet another web-framework. This one uses cells and XMLHttpRequest to
create dynamically updating models on the server that update their
state on the client.
The dynamically updating bit isn't working yet but hopefully we'll
have it working by the time anyone notices this."
Too late.
Brian Adkins
There's not much of a gap to fill. Sure, Ruby is dog slow, but the
number of customers it takes to saturate a reasonable dedicated server
running Ruby on Rails is substantial enough to make the cost of the
server a *non-issue* for a very large set of web apps.
Now, if you can get an equivalent, or superior, development
productivity (I'm referring to web apps only) with a Lispy solution
along with faster speed, that would be fantastic, but given the track
record, I think we'll see dozens of mediocre (at best) web frameworks
spring up rather than one good one that provides the productivity
close to Rails. Python had a diffusion of frameworks also, but Django
seems to have taken enough of a lead to gather a community.
I had high hopes for Arc ( http://arclanguage.com/item?id=6245 ), and
those may be realized someday; however, it still lacks many important
features and seems to have stalled out.
Ken Tilton
Not for my killfile. But thanks for noticing!
Willem Broekema
> Well I stand corrected by the man himself ;-)! Just out of curiosity I'm
> assuming you have dropped the reliance on Allegro's version of yacc.
Yep, a slightly extended version of cl-yacc can be used in its place.
It is included in the source.
> What of the environments stuff?
There is a fallback mechanism for keeping state, which uses symbol-
macrolet.
> If there is nothing major stopping porting I'd be happy to help
> out with the leg work of adding a few #+sbcl
> (sb-pcl:.... etc. to get this working on sbcl and cmucl.
There is a dependency on Closer to Mop, in order to abstract way from
the different names for the MOP package. Give the porting a try if you
have time, that would be appreciated. :)
- Willem
Marco Antoniotti
> On Apr 23, 1:11 am, Jason Nielsen <j...@cs.sfu.ca> wrote:
>
> > Well I stand corrected by the man himself ;-)! Just out of curiosity I'm
> > assuming you have dropped the reliance on Allegro's version of yacc.
>
> Yep, a slightly extended version of cl-yacc can be used in its place.
> It is included in the source.
What are the extensions and when will it be folded back in CL-YACC
main trunk?
Cheers
--
Marco
Christophe
I am curious, but if my memory is good, Lisp is build with an
interpreter and compiler.
And, it's a better way than just a VM.
To finish, what is the interest of use an open source Lisp less good
compared to Allegro ?
CLPython is an interesting project because it works with good Lisp,
commercial, of course.
(translation_in_real_world "Open Source"
> "the summun of economical hypocrisy." :)
Best Regards,
Christophe
Willem Broekema
> What are the extensions and when will it be folded back in CL-YACC
> main trunk?
Setting precedence level at production level, in addition to at token
level. That allows unary plus/minus (x = +3) to have a different
precedence level than binary (x = 1 + 2). That change has been mailed
to the author, but he remains silent. The same happened with another
small fix to make things work on Allegro.
As soon as CL-Yacc integrates it, the copy in CLPython will be removed
of course.
- Willem
Brian Adkins
No need to get all sensitive.
I just thought it was odd that your proposal for a web framework to
"fill the gap" was a framework whose reason for existence seems to be
the ability to "create dynamically updating models on the server that
update their state on the client", and that's the part that's not
working which begs the question ...
Andy Chambers
Actually it is working now. I'll go and update that bit of the
description now.
--
Andy
Robert Uhl
>
> given the emphasis in the ruby community on beautiful looking code, i
> can also see problems for lisp there.
Have you _seen_ Ruby code? It looks like what one would expect if Perl
caught syphilis. It's painfully ugly. I'll grant that Lisp parens
aren't pretty either, but once they disappear Lisp is awfully easy on
the eyes.
--
Robert Uhl <http://public.xdi.org/=ruhl>
But never forget that the original Dilbert strips were nature studies.
--Shmuel Metz
Didier Verna
> Have you _seen_ Ruby code? It looks like what one would expect if Perl
> caught syphilis. It's painfully ugly. I'll grant that Lisp parens
> aren't pretty either, but once they disappear Lisp is awfully easy on
> the eyes.
I guess the uglyness in Ruby also disappear for people programming
Ruby all day long. The same goes for offside-rule based languages like
Haskell or Python. You can't really make an argument out of that. You
have to get used to a language before starting to complain about it.
But I agree; Ruby code is ugly ;-)
--
5th European Lisp Workshop at ECOOP 2008, July 7: http://elw.bknr.net/2008/
Didier Verna, did...@lrde.epita.fr, http://www.lrde.epita.fr/~didier
EPITA / LRDE, 14-16 rue Voltaire Tel.+33 (0)1 44 08 01 85
94276 Le Kremlin-Bicętre, France Fax.+33 (0)1 53 14 59 22 did...@xemacs.org
Sean T Allen
> Sean T Allen <se...@monkeysnatchbanana.com> writes:
>>
>> given the emphasis in the ruby community on beautiful looking code, i
>> can also see problems for lisp there.
>
> Have you _seen_ Ruby code? It looks like what one would expect if Perl
> caught syphilis. It's painfully ugly. I'll grant that Lisp parens
> aren't pretty either, but once they disappear Lisp is awfully easy on
> the eyes.
I've programmed a ton of it. The people who use it think that it is
very beautiful.
Your asthetic aside, it would be an issue as very few people find lisp and ruby
asthetically pleasing to look at. Usually its one or the other.
Case in point, you. Or all the people who mention how ugly lisp is when
it comes
up on the ruby mailing lists.
Jeff
> > can also see problems for lisp there.
>
> Have you _seen_ Ruby code? It looks like what one would expect if Perl
> caught syphilis. It's painfully ugly.
Why Ruby is an acceptable Perl - http://artfulcode.net/articles/why-ruby-acceptable-perl/
Sohail Somani
> Sean T Allen <se...@monkeysnatchbanana.com> writes:
>> given the emphasis in the ruby community on beautiful looking code, i
>> can also see problems for lisp there.
>
> Have you _seen_ Ruby code? It looks like what one would expect if Perl
> caught syphilis. It's painfully ugly. I'll grant that Lisp parens
> aren't pretty either, but once they disappear Lisp is awfully easy on
> the eyes.
Python is the same way. Been doing a lot of it the past few days and it
makes my eyes bleed. In fact, to generate XML, I took a cue from the
Lisp libraries and abused tuples as s-expressions :-) Might make a post
about it one of these days.
Sohail Somani
> Robert Uhl <eadm...@NOSPAMgmail.com> wrote:
>
>> Have you _seen_ Ruby code? It looks like what one would expect if Perl
>> caught syphilis. It's painfully ugly. I'll grant that Lisp parens
>> aren't pretty either, but once they disappear Lisp is awfully easy on
>> the eyes.
>
> I guess the uglyness in Ruby also disappear for people programming
> Ruby all day long. The same goes for offside-rule based languages like
> Haskell or Python. You can't really make an argument out of that. You
> have to get used to a language before starting to complain about it.
>
> But I agree; Ruby code is ugly ;-)
I think the ugliness disappears only if you don't know of any alternatives.
I also agree: Ruby code is ugly ;-)
Nathaniel Calloway
> Sean T Allen <se...@monkeysnatchbanana.com> writes:
>>
>> given the emphasis in the ruby community on beautiful looking code, i
>> can also see problems for lisp there.
>
> Have you _seen_ Ruby code? It looks like what one would expect if Perl
> caught syphilis. It's painfully ugly. I'll grant that Lisp parens
> aren't pretty either, but once they disappear Lisp is awfully easy on
> the eyes.
Every programming language is beautiful in a different way. Whereas
lisp is a regimented anitomical study (its about the beauty of a
simple curve), ruby is... well... a Jackson Pollock.
-Nat
EL
> Now, if you can get an equivalent, or superior, development
> productivity (I'm referring to web apps only) with a Lispy solution
> along with faster speed, that would be fantastic,
Hop looks good: http://hop.inria.fr/
Although it is in Scheme, not CL
> but given the track
> record, I think we'll see dozens of mediocre (at best) web frameworks
> spring up rather than one good one that provides the productivity
> close to Rails.
This is true for other languages as well...
Although I must say that Rails would be to big for me. How much of it do
you really need - in contrast to what you must learn to get your stuff done?
When I looked at Rails first, I watched one of their tutorials. I
thought "Fine, if you want to develop a wiki this might be the way to
go. But what if you want to develop something *new*? Something that is
not there already? This is probably complicated."
And guess what? This guy at the cited page mentions exactly this,
confirming my very first impression (fourth to last paragraph):
"According to Bray, the prescriptive nature of the Rails framework suits
most web front-ended applications' database batch needs when it comes to
create, repeat, update and delete. The downside? If you need your
database calls to do something a little different, you're on your own."
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
--
Eckhard
Sean T Allen
> Just came across this...
>
> http://www.regdeveloper.co.uk/2008/04/21/bray_ruby_rails/
>
> You'd think there would be some way for Lisp to fill this gap.
i've been thinking about this after going a pass at checking out cl-python.
the rubinius folks are working on a vm for ruby and a class library where
everything is in ruby. if someone did a cl-ruby it could leverage that work
by the rubinius people.
there is a lot of interest in the ruby community in improving the performance.
there are numerous attempts at it. running on .net, on java, new
interpreter for
the standard ruby, rubinius... if a cl-ruby implementation could show obvious
speed or other gains over rubinius and use all those pure ruby libraries.. then
headway might actually be made.
Jason Nielsen
> i've been thinking about this after going a pass at checking out cl-python.
>
> the rubinius folks are working on a vm for ruby and a class library where
> everything is in ruby. if someone did a cl-ruby it could leverage that work
> by the rubinius people.
>
> there is a lot of interest in the ruby community in improving the
> performance.
> there are numerous attempts at it. running on .net, on java, new interpreter
> for
> the standard ruby, rubinius... if a cl-ruby implementation could show obvious
> speed or other gains over rubinius and use all those pure ruby libraries..
> then
> headway might actually be made.
>
The path of least resistance is to take the output of parse_tree
(http://parsetree.rubyforge.org/) which outputs sexp's and write a
translator from the this "ast" to common lisp. I am sure that a
performance boost will be had with little effort! Lisp compiler magicians
have been figuring out how to squeeze performance out of a dynamically
typed language for... say a few years. If you are sufficiently motivated
you could likely do some type inference from your sexp ast tree to common
lisp and decorate with type info... and wizz-bang easily 20-50 fold
performance improvement! What you don't know won't hurt you but what you
do know makes you realize how painfully unknowledgeable you are...
re-educate and repeat until sufficiently disheartened by the recursiveness
of it all ;-).
Jason
Robert Uhl
Lisp is ugly when one is starting out, esp. traditional all-caps Lisp:
(SETF (CAR FOO) BAR) is sinfully, shamefully, irredeemably ugly. As one
reads better Lisp, writes better Lisp and gets used to it, that turns
into (setf (first foo) bar) and then into something like 'setf "first
foo" bar' which is very easy to read.
AFAICT, Ruby starts out ugly and stays ugly. Sigils are just _wrong_.
I thought that even when I loved Perl and didn't know any better.
--
Robert Uhl <http://public.xdi.org/=ruhl>
In every civilisation, software will advance to such a level that to the
average manager, a desktop environment looks like a game of memory. And
they always cheat. --Pim van Riezen
Mark Wooding
> Lisp is ugly when one is starting out, esp. traditional all-caps Lisp:
> (SETF (CAR FOO) BAR) is sinfully, shamefully, irredeemably ugly. As one
> reads better Lisp, writes better Lisp and gets used to it, that turns
> into (setf (first foo) bar) and then into something like 'setf "first
> foo" bar' which is very easy to read.
I agree about the unpleasantness of uppercase Lisp. I used to set
*PRINT-CASE* to :DOWNCASE in my various .lisprcs, but I gave up partly
because I realized that uppercase responses disambiguated the
implementation's output from my input in transcripts, but mostly because
far too many add-on systems broke in unpleasant ways, usually as a
result of saying things like (intern (format nil "FROB-~A" some-symbol))
and then trying to use FROB-MUMBLE.
But I really can't force myself to prefer FIRST, SECOND, THIRD and so on
over CAR, CADR and CADDR. Then again, it seems to me that there's a
kind of poetry in the old names, many of them invented by people used to
cramming meaning into the six significant characters allowed them by
assemblers and similar tools of the time. I'll take NCONC over Scheme's
prosaic APPEND! any day.
-- [mdw]
Peter Jones
> AFAICT, Ruby starts out ugly and stays ugly. Sigils are just _wrong_.
> I thought that even when I loved Perl and didn't know any better.
Ruby only uses sigils to denote the scope of a variable as opposed to
Perl using them to denote the type of a variable.
The instance variable notation '@x' looks very strange at first, but
then becomes natural. The only sigil you're likely to see in good
Ruby code is '@' (regularly scoped variables don't have any special
notation).
It makes me think about the first time I saw `(,@foo).
--
Peter Jones [pjones at domain below]
pmade inc. - http://pmade.com
Rob Warnock
+---------------
| I agree about the unpleasantness of uppercase Lisp. I used to set
| *PRINT-CASE* to :DOWNCASE in my various .lisprcs, but I gave up partly
| because I realized that uppercase responses disambiguated the
| implementation's output from my input in transcripts, but mostly because
| far too many add-on systems broke in unpleasant ways, usually as a
| result of saying things like (intern (format nil "FROB-~A" some-symbol))
| and then trying to use FROB-MUMBLE.
You might try leaving *PRINT-CASE* alone and instead try changing
(READTABLE-CASE *READTABLE*) from :UPCASE to :INVERT, which both
preserves any CamelCase you might need to READ from data files and
also downcases all-uppercase symbols when printing [yes, even with
*PRINT-CASE* remaining the default :UPCASE, see CLHS 22.1.3.3.2
"Effect of Readtable Case on the Lisp Printer"]. This gives the
following behavior:
> (loop for sym in '(foo FooBar FOO)
collect (list sym (symbol-name sym)))
((FOO "FOO") (FOOBAR "FOOBAR") (FOO "FOO"))
> (setf (readtable-case *readtable*) :invert)
:invert
> (loop for sym in '(foo FooBar FOO)
collect (list sym (symbol-name sym)))
((foo "FOO") (FooBar "FooBar") (FOO "foo"))
>
Of course, it messes up your FROB example as written:
> (intern (format nil "FROB-~A" 'quux))
FROB-quux
:internal
> (symbol-name *)
"FROB-quux"
>
But that can be fixed [compatibly with the defaults!] this way:
> (intern (format nil "FROB-~A" (symbol-name 'quux)))
frob-quux
:internal
> (symbol-name *)
"FROB-QUUX"
>
This is arguably more nearly correct, since you shouldn't really
be concatenating a string and a symbol in the first place, but
rather two strings [which is what the "fix" does].
But I'll grant you that :INVERT *does* lose the feature you mentioned
"that uppercase responses disambiguate the implementation's output
from input in transcripts":
cmu> (apropos 'quux)
frob-quux
quux
FROB-quux
>
So you still may prefer to stay with the standard defaults.
-Rob
p.s. But it's really, *really* helpful when dealing
with external CamelCase input [such as EDIF files]... ;-} ;-}
-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Pascal Bourguignon
> [...]
> Lisp is ugly when one is starting out, esp. traditional all-caps Lisp:
> (SETF (CAR FOO) BAR) is sinfully, shamefully, irredeemably ugly. As one
> reads better Lisp, writes better Lisp and gets used to it, that turns
> into (setf (first foo) bar) and then into something like 'setf "first
> foo" bar' which is very easy to read.
'setf "first foo" bar' is easier to read than
(setf (first foo) bar) ???
--
__Pascal Bourguignon__ http://www.informatimago.com/
Litter box not here.
You must have moved it again.
I'll poop in the sink.
Pascal Bourguignon
> But I really can't force myself to prefer FIRST, SECOND, THIRD and so on
> over CAR, CADR and CADDR. Then again, it seems to me that there's a
> kind of poetry in the old names, many of them invented by people used to
> cramming meaning into the six significant characters allowed them by
> assemblers and similar tools of the time. I'll take NCONC over Scheme's
> prosaic APPEND! any day.
LISP 1.5 allowed symbol names of any length. Only, since it was a
36-bit word with 6-bit character machine, you losed less space when
your symbol names were of length 6k instead of 6k+m. Moreover, these
words were kept in a list, so they costed k+1 conses. Hence the
goodness of symbols 6-character long. But it was allowed to have
longer names.
Robert Maas, http://tinyurl.com/uh3t
> Just came across this...
> http://www.regdeveloper.co.uk/2008/04/21/bray_ruby_rails/
> You'd think there would be some way for Lisp to fill this gap.
Looking at the Web page:
Rails offers a clean, prescribed and predictable framework that - like
PHP - get some of their advantage from choosing to do less.
Why is this an "either-or" situation? Why can't a language provide
a subset of functionality that's super-easy to use, with choice of
icon-based drag-and-drop flowcharting or templates for nested
textual suntax, but have all that integrated into a completely
general purpose programming language such as Common Lisp? Why not
LHP (Lisp instead of Perl Hypertext Preprocessor)? Or from the
other jargon starting point, why not LSP (Lisp instead of Java
Server Pages)? OK, I see why we can't use LSP because it's
ambiguous with LSP as alternative to LISP as file-extension for
Lisp source files on MS-DOS/Windows systems that require
three-character file-extensions (thus JPG instead of the more
correct JPEG), nevermind MacLisp and Stanford/UCI lisp long ago.
Hey, LHP can be honestly pronounced "lithp" :-) We have a winner!
<http://www.acronymattic.com/results.aspx?q=LHP>
Lead Hollow Point ***
(hey, we can advertise that LHP is a powerful weapon in the fight
against low-quality over-budget delayed-release software)
Lightweight High Performance **
(perfect term for the agile development you get with Lisp H P!)
Life Has Purpose *
(ah, the zen meaning to Lisp H P)
Links to Hacking Pages *
(yeah!)
According to Bray, the prescriptive nature of the Rails framework
suits most web front-ended applications' database batch needs when it
comes to create, repeat, update and delete. The downside? If you need
your database calls to do something a little different, you're on your
own.
Um, it seems to me that in Lisp it would be easy to have a spectrum
(continuum) from a very limited Rails-like framework for that very
limited set of functionality through more general usefulness all
the way to the full expression of Common Lisp (within an explicitly
defined playpen that offers full access control thus avoids the
pitfalls of JavaScript and MS-Windows vulnerable to trojans such as
pop-up ads and spambots). S-expression syntax is more compact than
XML, so I see no reason not to allow both serverside LHP and
clientside LispScript to cooperate by exchanging s-expression
objects as needed, with graceful scale-down of function whenever
the client isn't Lisp-equipped.
Speed of development is the number-one reason that CTOs at big banks
and airlines are calling Bray in to advise on Ruby. "They've heard it
gives them applications in months rather than years. That's why Ruby
came out of nowhere," he said. "You can get something done faster in
Ruby once you've got it built, you can do updates and maintenance
faster than you can in PHP."
But with Lisp, we get applications in hours or days or at worst
weeks, followed by upgrades to install new capability in hours or
days. Over the course of one year a Lisp project can grow to
include just about everything anyone can think of during that year,
without needing major re-design, and refactoring of a major
component takes only days, so refactoring (of data structures, of
algorithms, and of program syntax) can be considered a normal part
of agile development instead of a major hassle to be avoided.
(For example, almost all my function definitions start out as a
PROG with SETQs, and are later refactored into a combination of DO
and/or LOOP and/or MAPsomething and/or LET/LET* only if that seems
to actually help in some way without messing it up. If the code
alternates between binding new variables and calling code that
modifies existing structure without returning a value, then I can
either change to LET* with dummy bindings to handle the
non-value-returning statements, or I can just leave it a PROG which
really doesn't bind any unnecessary values.)
Other languages could learn a thing or two from Rails and Bray
predicts frameworks would become more "Rails-like" - meaning the
future does not belong to Ruby on Rails alone..
AFAIK I don't have access to Rails, so I don't know whether it's
similar to anything I've used or to anything I'm proposing, but:
For a year or two I've been seriously thinking about implementing a
way to write software without using any syntax whatsoever beyond
keying in literal values (strings and integers only) and
making up names for local variables and for the functions being
defined. You just fill out a form telling what you want to do next:
enter this literal value (and assign it to a GENSYMmed label or to
a local variable by any name you want), or call this function
(which you don't even have to know the name of, you look in a
CookBook/Matrix database to find the function that does what you
want, and then you can call it anything you want, or accept the any
of the available default names provided by the library) and then
fill in the spaces regarding which literals or variables are going
to be parameters to this function. Behind the scenes, it builds
Common Lisp code, but the application programmer never needs to
know that. But after development of an application is finished (or
more likely, has reached a good stopping point for first beta
release), the system can express the application's algorithms in
Lisp or Java or Perl or PHP, or maybe even C or C++ or Flaming
Thunder if the algorithm doesn't exceed the easy limits of those
languages, whatever the customer wants within reason.
(For example, if the application needs to build lots of data
structures with multiple links to the same sub-object, it's easy in
Lisp or Java but not so easy in C++ and virtually impossible in C,
while if it needs lots of circular structures then it absolutely
can't be done in C++ without memory leak, requires Lisp or Java only.)
So does anybody already have something like Lisp Hypertext Preprocessor
(or Lisp Server Pages), or syntax-free application programming?
OT: Oh boy, I was up way too late last night, and posted an article
that shows on Google like this:
Date: Fri, 26 Apr 2008 01:29:59 -0700
Local: Sat, Apr 26 2008 4:29 am
Yeah, sleepily I changed the numeric date but failed to change the day-of-week.
There comes a time when you **really** need to go to bed! Well at least
I'm awake now and noticed the mistake and fixed it for *this* article!
(Note for dummies: I'm on Pacific time but Google is on Eastern time,
each daylight-savings [gag] time, so the 3-hour difference is correct.)
VOT: I'd make a more trustworthy President than Hillary. If I was
half asleep and said I was ducking hostile fire as I got off the
helocopter, I'd realize my mistake the next day and volunteer a
correction, rather than spend weeks avoiding facing my mistake and
need to be dragged kicking and screaming to finally admit I might
have mis-spoke.
EOT: Hillary can now, in full honesty, claim she was subjected to
being dragged kicking and screaming ... :-)
Explanation of abbreviations: It's adapted from from a Babylon Five
skit, where two guys were playing a game of "I Spy", while hiding
in the cargo hold of a starship, and they saw something that starts
with the letter B (boxes), and something that stars with the letter
M (more boxes), and something that starts with the letter E (even
more boxes).
So if you haven't figured it out yet: [|Very|Extremely] Off Topic.
EOM.
Robert Maas, http://tinyurl.com/uh3t
> a lot of the people who got pulled into the rails orbit came from
> php, i doubt you'd move them over to lisp. given the emphasis in
> the ruby community on beautiful looking code, i can also see
> problems for lisp there.
That's a straw man. Nobody really needs to look at **code** nowadays!
See for example my proposal for syntax-free (syntaxless, non-syntax)
application-programming.
> you'd need one of those wham-bam overwhelming flashy demos to
> make progress with the first group ( like how activerecord db
> introspection and scaffolding wow'ed em ).
What's so hard about doing that in LIsp, assuming you already have
the API for connecting to the RDBS and issuing SQL commands and
getting back objects containing the resultset and success-status.
> the 2nd well i don't know how you'd get them over the whole 'all
> those ()s just look ugly'.
The straw man again! Consider my proposed no-syntax (except literal
values, and even they aren't the usual syntax in the case of
strings):
Nested radio buttons:
( ) String literal:
( ) Short string, all on one line (text field)
( ) Long string on multiple lines (text area)
( ) Load string from contents of file
( ) Integer literal
( ) Short/medium size integer (at most 75 digits, text field)
( ) Long integer (text area, newlines ignored)
( ) Load integer from text file using radix
For the above, radix: (*) decimal ( ) hexadecimal ( ) other ____
( ) Load integer from binary file, little end first
( ) Load integer from binary file, big end first
( ) Call function/method
( ) Search CookBook/Matrix for suitable function in library
( ) Search your personal library for your function
( ) Enter standard name in text field
( ) Enter your personally-defined name in text field
SUBMIT
Suppose the user has previously entered the string "Hello test",
which is just test data for the algorithm, and the string " ".
which is a vital literal constant of the algorithm.
Suppose now the user says he/she wants to search the CookBook/Matrix,
a future version of a fully database-implmented search engine,
vaguely modeled like this crude incomplete preliminary WebPage:
<http://www.rawbw.com/~rem/HelloPlus/CookBook/Matrix.html>
The user finds this method:
<http://www.rawbw.com/~rem/HelloPlus/CookBook/Matrix.html#Arr>
Array indexing: Given an array already allocated, of dimension k, and
integer indexes i1, i2, ... ik, reference the element at that position
within the array:
which applies to special cases of vectors (one-dimensional arrays)
and strings (vectors of characters). So the user realizes he/she
first needs to set up the integer 0 as an index into the string,
thus the user backtracks to set up that literal integer, then gets
back into array indexing and actually selects the function for
retrieving an element from a vector. The system then asks what
string to use and what integer index to use, offering a menu of
just that literal strings "Hello test" and " ", and just that
literal integer 0 since that's all the user has set up so-far.
So internally the system is doing (aref " " 0) or equivalently (elt
" " 0), either of which returns #\Space, which is presented to the
user as a diagram:
+---------------+
| Character |
| Name: Space |
| Unicode: 0040 |
+---------------+
The user can, if desired, select to show such characters in some
other more compact notation such as ' ' (C sytax) or #\Space
(Common Lisp syntax) or U+0040 (unicode notation) or  (HTML
syntax) etc.
Next the user selects function from searching library again, and
this time finds:
Find character in string: Given string str, and character ch, find
first instance of ch as element of string:
The system prompts for string (user selects the test data string
this time) and the character (the space character obtained just
previously).
Internally the system is doing (position #\Space "Hello test")
The result is the integer 5.
Next the user selects function from searching library again, and
this time finds:
Select sub-string: Given string str, and integers ix1,ix2 which are
indexes within the string, return a newly-allocated string containing
just the characters between those two positions.
The user selects that test-data string again, and the indices 0 and
5 from earlier, and gets back the string "Hello", which is
displayed in a diagram I won't bother to draw right now.
Eventually the user has completed building the steps needed to
break a string into two sub-strings according to some delimiter
such as #\Space. Then the user selects the option of completing the
function definition (not offered earlier when I showed the nested
radio buttons because there weren't any steps to incorporate yet),
tells which local variables are given as parameters (the test-data
assignments for these parameters are then discarded except as test
data for an automated test script), and which variables are to be
returned as values, and picks a name for this user-defined
function, and now or later can write a documentation string for it,
but the system might provide default documentation that at least
vaguely gives some idea what's going on, for example:
;Given a string, and a character:
;(algorithm missing)
;Return two sub-strings of the original string.
The user need fill in only the middle part:
;Split the string where that delimiter occurs, generating two
; sub-strings (before and after that delimiter).
Of course the system would automatically enter the user-defined
function in the private addendum to the CookBook/Matrix, in the
section deailng with Strings and Characters, and optionally the
user could submit the function-definition for public beta testing,
to discover any bugs in the algorithm, or to discover problems with
the documentation, or to suggest alternate code already available
elsewhere that is more flexible/general, or to get glowing
recommendations that the algorithm be formally published for
general use by adding it to the public CookBook/Matrix, etc.
You get the idea how this would work?
> ruby is def. way out in front on the community being newbie friendly
> compared to lisp.
But not compared to my proposed no-syntax software-development system?
Hey, anybody reading this thread like my ideas (both the CookBook/Matrix
I already started last year, and the newer no-syntax system that
would make heavy use of a DB-converted version of the CookBook/Matrix)?
Anybody want to encourage me to finish the CookBook/Matrix prior to
start of work on the no-syntax system? Anybody want to actually
*help* me finish the CookBook/Matrix, like by submitting new
sections to include, whereby I just do a quick inspection of what
you submit to make sure the format agrees with my standards, then
copy&paste it into the C/M? Anybody want to set up a Wiki whereby
lots of people can submit new material and proofread each other's
submissions, so that by the time I take a look (maybe once per
week) it's already in good condition and ready to encorporate in
the C/M? I would then replace each copied section by a stub that
simply indicated it's already been included in the C/M, with URL to
find it. So each section in the Wiki would have a quick list of
links to stuff already in the C/M, and then full text of anything
new that I haven't yet included. I have essentially zero available
disk space on my shell account, so I can't set up a Wiki here.
Does Yahoo! Geocities or some other free Web hosting site with
gigabytes of available Web-accessible disk space have Wiki already
supported so that it would be easy to set up a WIki for C/M there?
Robert Maas, http://tinyurl.com/uh3t
> http://common-lisp.net/project/clpython/
So you're proposing we fully engage the enemy in an arms race
<http://en.wikipedia.org/wiki/Evolutionary_arms_race>
or "red queen's race"
<http://en.wikipedia.org/wiki/Red_Queen's_race>
<http://en.wikipedia.org/wiki/Red_Queen>
whereby every time they implement a new programming language/syntax
we emulate the whole thing in Lisp, and many of their potential
users realize that our emulation is better than the original, so
they go bankrupt for lack of sales of their inferior system?
That would eliminate most competitors to Lisp after-the-fact, but
not prevent newcomers from making up brand-new languages/syntaxes
and thereby forcing us into yet another round of the race just to
keep up with them.
What if we *also* pre-empt new programming languages by
implementing my idea for a syntax-free software-development system,
whereby newbies won't be attracted to *any* of those newfangled
languages/syntaxes because they don't need it because our system
avoids the hassle of learning *any* programming-language syntax in
the first place, destroying most of the arguments the newcomers use
to advertise their wares?
An ounce of prevention is worth a pound of Red queen's race??
OT: Russ "the moose" Syracruse used to have (on KYA 1260) a
"musical mixmaster", whereby he'd play two songs at the same time,
which somehow harmonized together. My favorite was "Surfing Bird"
and "Papa Oom Mow Mow", which I have on tape. Anybody with a
working reel-to-reel tape recorder want to hear it?
Anyway, maybe what I did just above is like "cliche mixmaster"?
Robert Maas, http://tinyurl.com/uh3t
> I'll grant that Lisp parens aren't pretty either, ...
IMO, among the various alternative pairs of nesting characters,
parens are, easiest on the eyes. Just look and compare:
(eukaryote
(animal (chordate ...) (arthropod ...) (jellyfish ...) (sponge ...))
(plant (moss ...) (fern ...) (conifer ...) (angiosperm ...))
(fungus (yeast ...) (mushroom ...))
)
[eukaryote
[animal [chordate ...] [arthropod ...] [jellyfish ...] [sponge ...]]
[plant [moss ...] [fern ...] [conifer ...] [angiosperm ...]]
[fungus [yeast ...] [mushroom ...]]
]
<eukaryote
<animal <chordate ...> <arthropod ...> <jellyfish ...> <sponge ...>>
<plant <moss ...> <fern ...> <conifer ...> <angiosperm ...>>
<fungus <yeast ...> <mushroom ...>>
>
{eukaryote
{animal {chordate ...} {arthropod ...} {jellyfish ...} {sponge ...}}
{plant {moss ...} {fern ...} {conifer ...} {angiosperm ...}}
{fungus {yeast ...} {mushroom ...}}
}
Practically, square brackets would be best because you don't have
to press the shift key to get them, but my eyes still prefer round
parens. Wanna take a poll just on the nesting-eyecandy?
And before anybody suggests the stupid idea of using indentation
instead of nesting characters to indicate nesting level within
deeply nested structures, thereby making it impossible to put more
than one list within a larger list on a single line of text,
thereby forcing the above to be spread out like this:
eukaryote
animal
chordate
.
.
.
arthropod
.
.
.
jellyfish
.
.
.
sponge
.
.
.
plant
moss
.
.
.
fern
.
.
.
conifer
.
.
.
angiosperm
.
.
.
fungus
yeast
.
.
.
mushroom
.
.
.
and maybe even programmers would be required to draw the tree
structure explicitly, like this:
eukaryote
|-animal
| |-chordate
| | |-.
| | |-.
| | `-.
| |-arthropod
| | |-.
| | |-.
| | `-.
etc.
That's a nice format for *presenting* cladograms, and it's nice for
the Linux/Windows directory-hierarchy browser, and the Java class
browser, and I wish Google Groups would go back to showing threads
that way, all because it's *generated* by the computer, not
anything you'd have to type, but that's not the way I want to be
required to key in my software syntax, and not even the way I want
to view most of my software structure. I like prettyprinted
(indented only as needed at user-selected linebreaks) parens-nested
expressions!
Duane Rettig
http://tinyurl.com/uh3t) writes:
>> From: Duane Rettig <du...@franz.com>
>> http://common-lisp.net/project/clpython/
>
> So you're proposing we fully engage the enemy in an arms race
It's clear that you don't buy into the Lisp idea that it is the
programmers programming language. With your statement above (unless
it's just a joke), where do you draw the line between a
domian-specific language created out of lisp for specific tasks, which
lisp is eminently suited for, and "the enemy"? If you take this
aversion to other languages to the extreme, then every DSL becomes
"the enemy", and indeed every application could be considered the
enemy, because Lisp is an extensible language and thus one can think
of every program being an extension - indeed, every functin written is
is an extension, and to the extent that it solves a specific problem,
it becomes part of a domain-specific language.
Of course, I'm talking about the other extreme way of thinking here,
but it's to make a point: the danger of an "us against them" thinking
is that it creates a tendency to narrow the possibilities for what can
be done in Lisp. And as far as I'm concerned, the Lisp philosophy is
completely opposite of that straightjacket - it tends to be more
inclusive than exclusive.
So yes, I'm proposing that we fully engage the enemy, and make "them"
our friends.
--
Duane Rettig du...@franz.com Franz Inc. http://www.franz.com/
555 12th St., Suite 1450 http://www.555citycenter.com/
Oakland, Ca. 94607 Phone: (510) 452-2000; Fax: (510) 452-0182
Rob Warnock
+---------------
| Why not LHP (Lisp instead of Perl Hypertext Preprocessor)?
Why not, indeed? See:
http://rpw3.org/hacks/lisp/minimal.lhp
http://rpw3.org/hacks/lisp/appsrv-demo.lhp
I've been using that filetype since 2000 or so for my "appsrv"
infrastructure [which does dynamic loading/compiling/caching of
Lisp source files which use HTOUT to emit HTML (though CL-WHO
or HTMLGEN should also work as well)].
+---------------
| Or from the other jargon starting point, why not LSP (Lisp instead
| of Java Server Pages)? OK, I see why we can't use LSP because it's
| ambiguous with LSP as alternative to LISP as file-extension for
| Lisp source files on MS-DOS/Windows systems that require
| three-character file-extensions...
+---------------
Well, the reason I avoided it myself is that when I was developing
"appsrv" there was already a Lisp-based server from some SRI folks
that was using[1] that filetype, namely, "Lisp Server Pages":
http://www.cliki.net/Lisp%20Server%20Pages
http://sourceforge.net/projects/lsp
Plus, the acronym LSP implies (correctly) that the pages are coded
in a style similar to ASP or JSP, that is, as native HTML with escapes
into code with some sort of "<% ... %>" brackets. I wanted to avoid
that association, since my pages would be "all Lisp" [see first two
URLs above].
So I went with "Lisp-Handled Pages" (LHP) instead.
Note: I certainly have no "lock" on LHP, any more than SRI has a "lock"
on LSP, see [2]; anyone is free to develop anything they want that uses
those filetypes. I *do* think it would be nice if the ones named LHP
used "pure Lisp" and the ones named LSP used "<% ... %>" escapes, just
to avoid confusion. [And, yes, I'm aware that by this convention PHP
should really be called "PSP". Foolish consistency, hobgoblins, etc.]
-Rob
[1] Hmmm... <http://sourceforge.net/forum/forum.php?forum_id=94130>
says that the project was suspended as of 2001-06-24, though there
seems to have been at least one more release on 2001-07-06 [to run
under LispWorks on Windows].
[2] There's an example of coding up a lightweight LSP processor here:
http://lemonodor.com/archives/000128.html
http://lemonodor.com/archives/misc-gems/lsp/lsp.lisp.txt
Sean T Allen
Maas, http://tinyurl.com/uh3t) said:
>> From: Sean T Allen <s...@monkeysnatchbanana.com>
>> a lot of the people who got pulled into the rails orbit came from
>> php, i doubt you'd move them over to lisp. given the emphasis in
>> the ruby community on beautiful looking code, i can also see
>> problems for lisp there.
>
> That's a straw man. Nobody really needs to look at **code** nowadays!
> See for example my proposal for syntax-free (syntaxless, non-syntax)
> application-programming.
You can call that point and others I made straw men all you want but it
doesnt change the realities of how a large numbers of people in the
ruby community view lisp. Go read the ruby mailing lists for a while,
it will come up.
And so now you idea for how lisp could benefit from ruby performance
woes is not only should you attempt to move people off their langauge
and platform of choice ( because in the end rails is the driver behind
ruby adoption ) but they should also learn a new syntax free style that
is a proposal?
Yeah, I call not very likely where that is concerned.
You can cry straw man all you want but you need to knock down straw men
in a fashion that convinces people who believe in that straw man not
just call it a straw man.
>
>> you'd need one of those wham-bam overwhelming flashy demos to
>> make progress with the first group ( like how activerecord db
>> introspection and scaffolding wow'ed em ).
>
> What's so hard about doing that in LIsp, assuming you already have
> the API for connecting to the RDBS and issuing SQL commands and
> getting back objects containing the resultset and success-status.
And where is that demo exactly? O right it isnt hard. That is a bit
different than,
go to this website, watch the snazzy demo for what is basically a sales
pitch feature.
Rails did an amazing job selling itself. If people want to take serious
advantage of performance
issues with ruby, they need to sell them on it. Whiz bang. Damn thats
cool, I should give it a try.
Rails make simple development easy, it isnt until you do something
complicated that it gets in
to be a pain.
For simple little web apps, rails is still my choice for simplest way
to get from point A to point B.
>
>
>> ruby is def. way out in front on the community being newbie friendly
>> compared to lisp.
>
> But not compared to my proposed no-syntax software-development system?
actually no, i mean in terms of someone coming to this forum and asking
newbie questions vs someone asking on the ruby talk mailing lists. the
tone is very different. ruby talk has a much friendlier tone to
answers, a helpful tone. the community aspect is very big in the ruby
world and for lack of better terms in comparision to one another, the
ruby community comes off as 'warm and fuzzy' compared to lisp's 'cold
and prickly'.
Robert Uhl
> Robert Uhl <eadm...@NOSPAMgmail.com> writes:
>> [...]
>> Lisp is ugly when one is starting out, esp. traditional all-caps Lisp:
>> (SETF (CAR FOO) BAR) is sinfully, shamefully, irredeemably ugly. As one
>> reads better Lisp, writes better Lisp and gets used to it, that turns
>> into (setf (first foo) bar) and then into something like 'setf "first
>> foo" bar' which is very easy to read.
>
> 'setf "first foo" bar' is easier to read than
> (setf (first foo) bar) ???
What I was trying to get at is that the parentheses recede into the
background--they're still there but a lot less noticeable. Also, quote
chars (not foot/inch markers, which I had to use due to ASCII
constraints) look kinda like little raised parens.
What I was saying is the standard Lisper argument: that after using Lisp
for awhile the parentheses don't really matter anymore. They're there,
just not prominent.
--
Robert Uhl <http://public.xdi.org/=ruhl>
I believe that many who find that nothing happens when they sit down, or
kneel down, to a book of devotion, would find that the heart sings
unbidden while they are working their way through a tough bit of
theology with a pipe in their teeth and a pencil in their hand. --C.S. Lewis
George Neuner
(Robert Maas, http://tinyurl.com/uh3t) wrote:
>and maybe even programmers would be required to draw the tree
>structure explicitly, like this:
>
>eukaryote
>|-animal
>| |-chordate
>| | |-.
>| | |-.
>| | `-.
>| |-arthropod
>| | |-.
>| | |-.
>| | `-.
>etc.
>
>That's a nice format for *presenting* cladograms, and it's nice for
>the Linux/Windows directory-hierarchy browser, and the Java class
>browser, and I wish Google Groups would go back to showing threads
>that way, all because it's *generated* by the computer, not
>anything you'd have to type, but that's not the way I want to be
>required to key in my software syntax, and not even the way I want
>to view most of my software structure. I like prettyprinted
>(indented only as needed at user-selected linebreaks) parens-nested
>expressions!
I had occasion to do some ladder programming for PLCs in a former
life. There is a trivial example at
http://en.wikipedia.org/wiki/Programmable_logic_controller - imagine
coding logic for a much more complex set of conditions.
I found it to be a particularly inefficient way to program,
particularly because the target was actually an 8 or 16-bit
microcontroller rather than an analog board for which a logic diagram
is appropriate. But the only alternative at the time was assembler
and many useful ladder programming constructs like latches, counters,
timers, etc. were implemented as assembler macros that were hard to
identify when expanded in the assembly listing and (timers in
particular) difficult to use directly. In effect, any non-trivial
coding really had to be done via ladders.
It's possible that drawing tree diagrams might work much better for
inputting data ... but my experience with ladder programming left such
a bad taste that I would probably be loathe to try it.
George
--
for email reply remove "/" from address
Robert Maas, http://tinyurl.com/uh3t
> > So you're proposing we fully engage the enemy in an arms race
> It's clear that you don't buy into the Lisp idea that it is the
> programmers programming language.
Huh? What gives you that idea??
<OT diversion> Jay Leno says some economist says gasoline prices
are "flirting" with $4.00 per gallon, but Jay says we're already
way past that, we're already getting screwed! </OT diversion>
> With your statement above (unless it's just a joke), where do you
> draw the line between a domian-specific language created out of
> lisp for specific tasks, which lisp is eminently suited for, and
> "the enemy"?
There's a key difference: The former starts with a package in CL
that implements a domain-specific language. "The enemy" acts like
CL is worthless, and instead hand-codes the entire domain-specific
language in C, and then advertises that CL is crap and their new
language is much better, and only later does some Lisper emulate
the new language inside Lisp like should have been done in the
first place.
> If you take this aversion to other languages to the extreme, then
> every DSL becomes "the enemy", and indeed every application could
> be considered the enemy,
Nope. Only if they hand-code that DSL in C (or assembly language)
and compete with Lisp and draw potential Lisp customers away from
Lisp and lock them into that other language because it doesn't
interoperate with Lisp.
> because Lisp is an extensible language and thus one can think of
> every program being an extension
Only if that program is written in Lisp, not if it's written in C
or assembly language.
> indeed, every functin written is is an extension, and to the
> extent that it solves a specific problem, it becomes part of a
> domain-specific language.
That's an interesting way of looking at it. But the idea of a DSL
is that the syntax is changed via macros. That's the difference
between bottom-up tool-building (defining functions on top of
what's already defined) which merely adds functionality within the
existing syntax rules, and defining macros, or even a new parser,
to actually change the sytax to violate the usual rules.
I would categorize three levels:
-1- Just adding functions, DEFSTRUCTs/CLOSclasses/genericFunctions.
-2- Also defining macros to override the eval-all-elements-of-tail rule,
and also moderately changing reader macros falls in here.
-3- Also writing your own parser so that you don't even need to
follow the rule of s-expressions in any "way shape or form".
I would consider -1- *not* creating a DSL, merely a domain-specific
library. I would consider -2- a weak form of DSL in the Lisp sense.
I would consider -3- a strong form of DSL that covers emulating
some *other* language independently developed from Lisp as well as
novel languages first invented *in* Lisp. CGOL (V.Pratt) and
RLISP/Reduce (T.Hearn) each falls into this category in a weak
sense since they essentially emulate Algol syntax for essentially
Lisp semantics. In a sense, Reduce and MacSyma are true examples of
DSL at level 3, because they *both* provide a true DSL (symbolic
algebra) rather than a general-purpose language and also involve
the new parser for a totally non-sexpr syntax. Unfortunately I
can't give a better example of level-3 DSL because those are the
only two I'm directly familiar with.
My own MRPP3 (later renamed "POX") was a domain-specific language
for text-formatting on XGP (Xerox Graphics Printer) circa 1975, but
that's such a crufty thing that I am not comfortable including it
in this discussion as any sort of good example. (And I have a vague
memory that Hans Moravec or Bill Gosper or Mike Farmwald, I don't
remember which, proved it Turing complete by actually emulating a
Turing machine in it, a *very* crufty hackish emulation if I
recall.) Of course later TeX and HTML did similar things.
> Of course, I'm talking about the other extreme way of thinking
> here, but it's to make a point: the danger of an "us against them"
> thinking is that it creates a tendency to narrow the possibilities
> for what can be done in Lisp.
I don't think so. Remember my obverved scenerio was that first
somebody totally ignores Lisp, handcoding the DSL in C or assembly
language, at great effort and expense, when coding in Lisp would
have been a lot easier. But their reason was to make $money$ and to
lock their customers away from Lisp. At that point IMO they were
our enemy. It's only later that people on *our* side of the war
pulled some of their rug out from under them by emulating their
language in Lisp so that their customers could switch over to our
emulation.
> And as far as I'm concerned, the Lisp philosophy is completely
> opposite of that straightjacket
What straightjacket?? The only straightjacket is during the time
span when customers of that newfangled DSL are locked away from
Lisp. The Lisp philosopy, of emulating all DSLs in Lisp, is indeed
opposed to that straightjacket, and eventually liberates those
jacketed customers.
> it tends to be more inclusive than exclusive.
Yes.
> So yes, I'm proposing that we fully engage the enemy, and make
> "them" our friends.
How do we convince them *not* to implement a new DSL in C or
assembly language but instead implement each new DSL in Lisp??
Wouldn't helping them to get started with their new DSL in Lisp, so
that they are our "friends" from the start of their new-DSL
project, be better than *not* helping them, thereby driving them to
ignore Lisp and use C or assembly language, so that they are our
enemy from the time they release their first new-DSL until years
later when we finally get around to emulating their DSL in Lisp??
Here's an idea I came up with just now: Suppose we create some
public forum for brainstorming new DSLs, for example a newsgroup
comp.programming.dsl or alt.design.dsl. We plant spies in all the
newsgroups and other public forums for special topics such as
genetics or medical diagnosis etc. where it's likely anybody might
wish to create a new DSL. Then whenever our spy detects such an
innovation, the thread is dragged (by cross-post) over to the
DSL-design forum where the new DSL is formally defined by BNF and
then immediately emulated in Lisp and put up as a CGI demo. Further
discussion of the new DSL is then all in relation to the
already-established BNF and the parse tree generated by the
corresponding already-implemented Lisp-based parser. Basically the
BNF directly yields the parser, which translates DSL-syntax to
parse-tree, which is presented as s-expressions for human viewing.
Then the semantics are all expressed in terms of the parse tree
(via s-expressions) rather than the actual DSL syntax. (Anyone who
expresses semantics in terms of the actual DSL syntax will find
his/her expression immediately auto-translated to being expressed
in terms of parse tree i.e. s-expressions.)
Anyone who might have been tempted to implement the new DSL in C or
assembly language, or even in Java, would find it "already done" in
Lisp, hence not worth playing catch-up by starting from scratch in
any other base language.
Duane Rettig
http://tinyurl.com/uh3t) writes:
>> >> http://common-lisp.net/project/clpython/
>> > So you're proposing we fully engage the enemy in an arms race
>> From: Duane Rettig <du...@franz.com>
>> It's clear that you don't buy into the Lisp idea that it is the
>> programmers programming language.
>
> Huh? What gives you that idea??
I have to admit that I misunderstood the context of your phrase
"engage the enemy". Rereading your original post more carefully
cleared it up for me.
> <OT diversion> Jay Leno says some economist says gasoline prices
> are "flirting" with $4.00 per gallon, but Jay says we're already
> way past that, we're already getting screwed! </OT diversion>
>
>> With your statement above (unless it's just a joke), where do you
>> draw the line between a domian-specific language created out of
>> lisp for specific tasks, which lisp is eminently suited for, and
>> "the enemy"?
>
> There's a key difference: The former starts with a package in CL
> that implements a domain-specific language. "The enemy" acts like
> CL is worthless, and instead hand-codes the entire domain-specific
> language in C, and then advertises that CL is crap and their new
> language is much better, and only later does some Lisper emulate
> the new language inside Lisp like should have been done in the
> first place.
This sometimes occurs, and it almost always makes headlines here.
However, because Lisp (including CL) is a universal progenitor, and
adding a little animism to this already thick analogy, it stands to
reason that Lisp will spawn some children that will grow to hate it.
This strength is one of the major reasons why Lisp will never be
popular; every program (taking my extreme cases below) is a new
language, and people who get used to that language are pulled away
from the language that made their new one possible.
>> If you take this aversion to other languages to the extreme, then
>> every DSL becomes "the enemy", and indeed every application could
>> be considered the enemy,
>
> Nope. Only if they hand-code that DSL in C (or assembly language)
> and compete with Lisp and draw potential Lisp customers away from
> Lisp and lock them into that other language because it doesn't
> interoperate with Lisp.
It is a natural tendency to program in one's favorite DSL exclusively,
regardless of whether it was or is coded in Lisp, and regardless of
whether it interoperates with Lisp. I first became aware of this
phenomenon with an analogous situation (though not the same as a
different language): emacs key bindings. If a person has an elaborate
set of key bindings in emacs, or if he gets used to an emacs/lisp
interface such as slime, he becomes increasingly dependent on these
bindings or on his setup in order to operate, and whenever faced with
a new system setup, the first thing he _must_ do is to set up his key
bindings before he can be productive; he finds that he cannot operate
efficiently without them. Of course, this doesn't always happen; it
depends very much on how dependent the person becomes on his setup.
But in any extensible language or highly configurable system, the
potential is always there.
>> because Lisp is an extensible language and thus one can think of
>> every program being an extension
>
> Only if that program is written in Lisp, not if it's written in C
> or assembly language.
Look again at the link I provided; you'll find that clpython is
written entirely in CL. In fact, those portions of the python library
that are written in C must be rewritten in CL before they can be used
in CLpython.
>> indeed, every function written is is an extension, and to the
>> extent that it solves a specific problem, it becomes part of a
>> domain-specific language.
>
> That's an interesting way of looking at it. But the idea of a DSL
> is that the syntax is changed via macros.
This depends on your definition of syntax. Some say that Lisp has no
syntax. So how do you change something that's not there? If you
consider the "syntax" to be all-the-parens, then yes, you may have to
deal with the question of recreating the syntax of the target
language, but many such implementations don't do that - they either
define their DSL to have a paren-based syntax, or, in the case of an
existing language, they may just punt and provide the DSL with lisp
paren style. Allegro Prolog is like that; it is a marriage of the
lisp style (and it has the full power of the CL macro facility for it)
and yet it is unmistakably Prolog in nature.
> That's the difference
> between bottom-up tool-building (defining functions on top of
> what's already defined) which merely adds functionality within the
> existing syntax rules, and defining macros, or even a new parser,
> to actually change the sytax to violate the usual rules.
>
> I would categorize three levels:
> -1- Just adding functions, DEFSTRUCTs/CLOSclasses/genericFunctions.
> -2- Also defining macros to override the eval-all-elements-of-tail rule,
> and also moderately changing reader macros falls in here.
> -3- Also writing your own parser so that you don't even need to
> follow the rule of s-expressions in any "way shape or form".
These three levels are not all there are, but yes, they can be said to
categorize a DSL into degrees-of-recognizability wrt Lisp. But even
within these levels, there are degrees of variance which
determine/allow the distance the final language will be from Lisp.
> I would consider -1- *not* creating a DSL, merely a domain-specific
> library.
Having come from backgrounds in many Algol-style languages and also
Forth, I will not quibble with the idea that a Lisp function or a
Forth word is equivalent to a Basic subroutine. However, with the
meta-programming available with <builds does> words and with special
Lisp functions that happen to be known as macros, the extent of your
claim for such programming to be "merely ... libraries" is arguable,
and is mostly in the eye of the beholder.
I would consider -2- a weak form of DSL in the Lisp sense.
Your #2 seems so specific; there are many other uses for macros,
reader macros, and even object orientation that I would tend to place
at various levels of this category. Also, between #2 and #3 is the
less-explored world of code-walking, where the forms are lisp-like but
the rewriting is done on a whole-program basis with rules that need
not be lisp-like at all. You may consider these to be weak re DSL,
but if you interact with them their behavior might surprise you.
Also, there is a lot that can be done within #2 with reader macros
that encroaches on #3 - the lisp reader is extermely powerful. For
example, in the older Franz Lisp (a MacLisp variant) I was able to use
infox splicing reader macros and lexicon lookup to implement an
English Grammar parser. This allowed me to parse a sentence like
John's going to Mary's house.
properly. I was able to port the grammar to Allegro CL, but not using
portable CL code, since it didn't have the infix splicing macros.
> I would consider -3- a strong form of DSL that covers emulating
> some *other* language independently developed from Lisp as well as
> novel languages first invented *in* Lisp. CGOL (V.Pratt) and
> RLISP/Reduce (T.Hearn) each falls into this category in a weak
> sense since they essentially emulate Algol syntax for essentially
> Lisp semantics. In a sense, Reduce and MacSyma are true examples of
> DSL at level 3, because they *both* provide a true DSL (symbolic
> algebra) rather than a general-purpose language and also involve
> the new parser for a totally non-sexpr syntax. Unfortunately I
> can't give a better example of level-3 DSL because those are the
> only two I'm directly familiar with.
No contention here; we've approached a turing argument, and it is less
interesting to me, because such things can be done in any decent
language.
In reality, though, as I said above, the people who use these DSLs
tend to be locked into them by force of habit. It is not Lisp that is
the straighjacket, but the tendencies of the users. That is why Lisp
users will always be in the minority.
>> it tends to be more inclusive than exclusive.
>
> Yes.
>
>> So yes, I'm proposing that we fully engage the enemy, and make
>> "them" our friends.
>
> How do we convince them *not* to implement a new DSL in C or
> assembly language but instead implement each new DSL in Lisp??
There it is again; us against them. Who is "we", and who are "they"?
The minute you consider an "enemy", you have a war.
[The answer is: "we" don't convince them, we demonstrate by doing, and
"they" can respond however they want to]
> Wouldn't helping them to get started with their new DSL in Lisp, so
> that they are our "friends" from the start of their new-DSL
> project, be better than *not* helping them, thereby driving them to
> ignore Lisp and use C or assembly language, so that they are our
> enemy from the time they release their first new-DSL until years
> later when we finally get around to emulating their DSL in Lisp??
Absolutely. How is this different from precisely what Willem Broekema
has done with CL-Python, which was the original impetus for our part
of this thread?
> Here's an idea I came up with just now: Suppose we create some
> public forum for brainstorming new DSLs, for example a newsgroup
> comp.programming.dsl or alt.design.dsl. We plant spies in all the
> newsgroups and other public forums for special topics such as
> genetics or medical diagnosis etc. where it's likely anybody might
> wish to create a new DSL. Then whenever our spy detects such an
> innovation, the thread is dragged (by cross-post) over to the
> DSL-design forum where the new DSL is formally defined by BNF and
> then immediately emulated in Lisp and put up as a CGI demo. Further
> discussion of the new DSL is then all in relation to the
> already-established BNF and the parse tree generated by the
> corresponding already-implemented Lisp-based parser. Basically the
> BNF directly yields the parser, which translates DSL-syntax to
> parse-tree, which is presented as s-expressions for human viewing.
> Then the semantics are all expressed in terms of the parse tree
> (via s-expressions) rather than the actual DSL syntax. (Anyone who
> expresses semantics in terms of the actual DSL syntax will find
> his/her expression immediately auto-translated to being expressed
> in terms of parse tree i.e. s-expressions.)
Spies? Detection? Man, you really are entrenched in this war theme.
How do you determine that the "enemy" has been killed?
But I'll give the standard answer that we tend to give on c.l.l. to
anyone who says things like "Here what I think the Lisp Community
should do...", and that is: OK, you have an idea for what should be
done. When will I be able to read about what you've done to make it
happen?
Christophe
Hi all,
When I open a dictionary, I can see that all words of a language are
define in the same language.
It's exactly the same thing for Lisp.
Try to redefine with Java for Java, instruction like class, extend, of
just if is very difficult, impossible ?.
This is explain that Lisp is a language, Java is just a syntax same as
C.
Best Regards,
Christophe
Robert Maas, http://tinyurl.com/uh3t
Linkname: here
URL: http://rpw3.org/hacks/lisp/minimal.lisp
Lynx won't let me see it, but will let me download it to the hard
disk on my personal account on my ISP:
2 -rw------- 1 rem user 743 May 1 23:34 tmp-minimal.lisp
Hmm, you seem to have extended the Common Lisp syntax whereby a
keyword in the first position in a list is a HTML-like tag whereas
a normal symbol is the name of a function (or special operator or
macro) as usual. That's a neat idea. But this is something
completely different from what I was asking about, which would be
mostly HTML syntax with embedded scriptlets, like PHP and JSP. What
you've done is similar to a *different* idea I've been thinking
about from time to time.
Now here's my take on all this: PHP and JSP and what I inquired
about in this thread are a way of having a graceful transition
between people who know *only* how to code HTML and learning how to
enhance the Web pages with a little bit of active content. By
comparison, CGI using Perl or other language is fullfledged coding
the Web Page in some language other than HTML, where in effect the
entire CGI application is one huge analogy to a scriptlet. Your
idea (similar to what I suggested elsewhere) goes in a different
direction, using the s-expression parse-tree to intermix two modes:
- Ordinary Lisp code, as if CGI.
- S-expressions used as a directly alternate expression of SGML or XML.
My similar idea would have used strings rather than keyword symbols
in the first position within a list, but the basic idea is the
same.
So the bad thing is that you have usurped the filetype "LHP" to
mean the fully s-expression alternate to SGML or XML, so that
nobody can use that to represent what I proposed in this thread,
something analagous to PHP or JSP. Maybe it would have been better
if you had used "LML", to be directly analagous in name with SGML
and XML. Is it too late to make that change? (Do you have a user
base of more than a hundred different Web authors?)
Now I have one actual nitpick: You don't show a version number. So
if somebody uses your idea but extends to new keywords as tags,
they won't be able to differentiate their syntax from yours so that
you can share the same filetype and the Web server can use the
version number to know how to emulate the source correctly.
What do you think about including a version number somehow?
(I don't have a specific proposal, although perhaps binding the
lexical variable version at the very top would have that implied
meaning as a side effect? Perhaps the version "number" should
actually be Java-style package (reversed FQDN) plus particulars?)
> http://rpw3.org/hacks/lisp/appsrv-demo.lhp
Linkname: here
URL: http://rpw3.org/hacks/lisp/appsrv-demo.lisp
4 -rw------- 1 rem user 3533 May 1 23:58 tmp-appsrv-demo.lisp
Oops, I didn't need to download it, because it was already inline.
(That's what I get for listening to Jay Leno at the same time.)
> I've been using that filetype since 2000 or so for my "appsrv"
> infrastructure [which does dynamic loading/compiling/caching of
> Lisp source files which use HTOUT to emit HTML (though CL-WHO
> or HTMLGEN should also work as well)].
Hmm, I don't recall seeing you mention it in the newsgroup before.
Did you post and I missed it, or is this the first time you've
explicitly mentionned this kind of server-dynamic Web page?
> ... when I was developing "appsrv" there was already a Lisp-based
> server from some SRI folks that was using[1] that filetype, namely,
> "Lisp Server Pages":
> http://www.cliki.net/Lisp%20Server%20Pages
> http://sourceforge.net/projects/lsp
XLSP is a lisp based dynamic XML management facility. It is currently
usable with Allegro CL and LispWorks implementations. It's
implementation is in two parts: XMLP for general XML processing, and
XLSP for managing interactions with web servers.
Typo: "It's" (contraction of "it is") should read "Its" (possessive)
It's not clear what is going on there. It sounds like it's merely a
package for parsing and otherwise dealing with XML, which has not
much to do with PHP/JSP-like scriptlets or anything else like
what we've been talking about. Too bad there's no link to any
introduction or tutorial or other way to know what they're really
talking about.
> Plus, the acronym LSP implies (correctly) that the pages are coded
> in a style similar to ASP or JSP, that is, as native HTML with escapes
> into code with some sort of "<% ... %>" brackets. I wanted to avoid
> that association, since my pages would be "all Lisp" [see first two
> URLs above].
I'll take your word for the first part of that because I can't
figure out from their Web pages what they are really doing.
By the way, I've never had access to a Web server that can run ASP,
so I never had any curiosity to learn what ASP source is like.
But now I decided I needed to know, so I did a Google search,
and found a set of examples:
<http://www.w3schools.com/asp/asp_examples.asp>
<http://www.w3schools.com/asp/showasp.asp?filename=demo_variable>
<http://www.w3schools.com/asp/showcode.asp?source=demo_variable>
<%
dim name
name="Donald Duck"
response.write("My name is: " & name)
%>
Hmm, no delimiter to break apart statements, just newline.
So I guess it's impossible to have really long statements, and
impossible to nest statements? I already don't like it.
Oh well. My major question was answered with those samples, indeed
it's like JSP and PHP in syntax, not like your all-sexpr syntax.
> So I went with "Lisp-Handled Pages" (LHP) instead.
OK, that's not parallel to PHP (Perl Hypertext Preprocessor).
I guess PHP needs to be renamed PSP to be analagous to JSP and ASP.
> Note: I certainly have no "lock" on LHP, any more than SRI has a "lock"
> on LSP, see [2]; anyone is free to develop anything they want that uses
> those filetypes. I *do* think it would be nice if the ones named LHP
> used "pure Lisp" and the ones named LSP used "<% ... %>" escapes, just
> to avoid confusion. [And, yes, I'm aware that by this convention PHP
> should really be called "PSP". Foolish consistency, hobgoblins, etc.]
Yeah, fat chance of our PHP->PSP name change ever being effected.
I'm too sleepy to go on, so I'll stop my reply now and plan to look
in detail at the lemonodor stuff another day.
Robert Maas, http://tinyurl.com/uh3t
> [2] There's an example of coding up a lightweight LSP processor here:
> http://lemonodor.com/archives/000128.html
> It's generic lisp and should run in any lisp that AllegroServe supports.
I'm guessing that it can't be run directly from the Apache server on my ISP?
PHPINFO says this (tiny excerpt of 18 pages of output):
System FreeBSD shell.rawbw.com 4.10-STABLE FreeBSD 4.10-STABLE #0: Thu Feb i386
Build Date Dec 15 2004 23:06:51
Configure Command './configure' '--enable-versioning'
'--with-system-regex' '--disable-debug' '--with-mysql' '--with-gdbm'
'--with-ftp' '--with-zlib' '--with-imap=/usr/local'
'--with-ldap=/usr/local' '--with-openssl=/usr' '--with-gd=/usr/local'
'--enable-sockets'
SERVER_SOFTWARE Apache/1.3.26 (Unix) mod_macro/1.1.1
If that's not enough info, do this to see the whole 18 pages:
<http://www.rawbw.com/~rem/HelloPlus/phpinfo.php>
With PHP, I needed to set up a .HTACCESS file which enabled PHP on
my personal account. I assume that even after I download the
software (for either lemonodor's HTML+LispScriptlets or your
FullySexprWebpage, to convert either syntax to fully HTML output)
to my private directory, there's nothing I can put in my .HTACCESS
file to cause that some/all-Lisp-to-fully-HTML rendering code to be
invoked whenever the server sees anything .LSP or.LHP respectively
requested from my account? (Correct me if there *is* a way to do
that with our present Apache configuration!)
On the other hand, it would seem that if I set up a separate CGI
hook to simply pass control to the rendering code, giving it a
parameter of the syntax that needs rendering, that ought to work?
For example:
#! /bin/sh
/usr/local/bin/lisp -eval '(progn (load "lhp.x86f" :verbose nil) (lhp:render-page "page1.lhp") (quit))'
> http://lemonodor.com/archives/misc-gems/lsp/lsp.lisp.txt
;;; To publish an LSP page, call the PUBLISH-LSP function:
;;; PUBLISH-LSP (&key path file server) [function]
When exactly should that function be called??
Just write a Web page using the HTML+Lisp syntax, then call that
function from any random Lisp coreimage that is running from the
same user account??
Why does each separate LHP/LSP Web page need to be individually published?
Why isn't it like PHP where you set up .HTACCESS after which
*every* Web page with filetype .PHP in the public_html tree is available?
;;; <%= ... %> is an expression tag, and the effect of this tag is to
;;; evaluate the contents as if they were wrapped with the Franz
;;; net.html.generator:html macro. For example,
;;; <%= (:h1 "header") "hi" (:princ-safe (generate-footer)) %>
That looks like very similar to the syntax *you* are using, where
any normal HTML tag is replaced by a keyword symbol, except for the
extra wrapper around the very outside. So if one of your pages is
prefixed by <%= and suffixed by %> would it then be usable by their
system, as long as the page used only the :KEYWORD tags in common
between your and their sexpr-markup-language syntax?
;;; In my first attempt to do this, I tried to construct forms instead
;;; of strings. That just made it trickier to separate forms across
;;; <% ... %> tags (see the dotimes example above). Just because it's
;;; bad that other languages are often *limited* to manipulating code
;;; as strings doesn't mean there aren't times where it's appropriate.
Yeah, I experienced that first with PHP, where it's common to go
into and out of a scriptlet to mix output generated by scriptlets
with output verbatim in the PHP source. It's sort of the inverse of
the way Perl supports labeled blocks of verbatim text. Anyway, it
was really weird seeing a Perl-like script split into multiple
parts in a totally un-syntax-like way with medium/large blocks of
verbatim output right in the middle. But eventually I realized the
value of such a hack for avoiding explicit Perl/PHP code that
generated large blocks of text in pieces with a separate call to
printf or whatever for each individual piece. Of course when
writing CGI/Lisp pages I never had that problem, because with
FORMAT and the tilde-newline directive it's easy to divorce the
formatting of source from the formatting of generated output, all
within a single huge format string within a single call to FORMAT,
no need to make a separate call to FORMAT or PRINC for each piece
of a long batch of output.
So actually it occurs to me to ask, what's the advantage of being
able to split Lisp syntax across HTML/Lisp boundaries by going out
of and back into a scriptlet in the middle of a Lisp form? Is it
primarily to allow graceful transition from fully-HTML Web page
through tiny bits of scriptlet in mostly-HTML page to eventually
nearly-all one huge LHP s-expression with single wrapper tag around
the outside? The main advantage seems to be that if there's a huge
block of pure HTML source that was *always* emitted because it was
part of a regular HTML page, but now you need to emit it only
conditionally (when flag is true), then it's easy to write something like:
<% (when flag %>
..hugeBlockOfHtmlFromOriginalPage...
<% ) %>
or something like that?? I know that sort of thing works in PHP:
<?php if (stristr($qs, "date") === false) { ?>
You didn't ask about 'date' this time. <?php
} else { ?>
In California, current date+time given by this associative array:
<?php $today = getdate(); print_r($today);
} ?>
That's an excerpt from:
<http://www.rawbw.com/~rem/HelloPlus/hellos.html#php2>
Linkname: WebPage with embedded PHP
Method: GET
Enctype: application/x-www-form-urlencoded
Action: http://www.rawbw.com/~rem/HelloPlus/h2.php
Linkname: See source
URL: http://www.rawbw.com/~rem/cgi-bin/show-h2-php.cgi
Robert Maas, http://tinyurl.com/uh3t
> >> php, i doubt you'd move them over to lisp. given the emphasis in
> >> the ruby community on beautiful looking code, i can also see
> >> problems for lisp there.
> > That's a straw man. Nobody really needs to look at **code** nowadays!
> > See for example my proposal for syntax-free (syntaxless, non-syntax)
> > application-programming.
> From: Sean T Allen <s...@monkeysnatchbanana.com>
> You can call that point and others I made straw men all you want
> but it doesnt change the realities of how a large numbers of people
> in the ruby community view lisp.
Why do we really care how they view Lisp? What if we set up a Web
service that let newbies program (design and implement software)
without regard to any programming-language syntax? So as they are
learning how to design algorithms, and how to make effective use of
already-existing software methods (functions), the syntax
distinction between various programming languages doesn't arise
because none of the syntax is ever seen by the newbie. At some
point the newbie has advanced to the level of designing algorithms
that use features of Lisp that aren't directly available in any
other language, or which look grotesque or inscrutable in other
languages but not really bad in Lisp. When the newbie has reached
that point, our service offers to translate the newbie's algorithms
to any of several programming language, but *only* those languages
where the algorithms can be expressed directly. The newbie chooses
Ruby, or Java, and sees a grotesque concoction. The newbie gets
desperate and tries Lisp, and sees something with a lot of parens
but actually quite succinct by comparison with the other langauges.
So what if we infiltrate the Ruby discussion forums with
advertisements for our no-syntax software-development service, so
that absolute newbies who haven't yet gotten used to Ruby's syntax
will like our service better than having to learn Ruby, so they
will never learn Ruby, and will never get locked into Ruby's syntax
as "what I'm used to already".
> Go read the ruby mailing lists for a while, it will come up.
Unless there's a Web-accessible way to view those forums, it's not
a option. Any e-mail address I post in any public place gets
flooded with spam to where the address is useless. The last time I
tried to join a mailing list was several years ago when I joined
the procmail discussion forum for a few weeks, during which my
mailbox was totally swamped with a mix of spam and procmail
discussion and I never did find time to disentangle the two even
after I unsubscribed to stop the additional flood.
> And so now you idea for how lisp could benefit from ruby
> performance woes is not only should you attempt to move people off
> their langauge and platform of choice ( because in the end rails is
> the driver behind ruby adoption ) but they should also learn a new
> syntax free style that is a proposal?
No, only woo away the newbies, so they get no new converts, and as
the old converts die of old age or decide they like something else
better than Ruby the number of people using Ruby declines to zero.
If we had the no-context programming service up ten years ago,
there might never have been anybody deciding to use Ruby or Java or
C# or VB or Python etc.
> You can cry straw man all you want but you need to knock down
> straw men in a fashion that convinces people who believe in that
> straw man not just call it a straw man.
Agreed. Would you like to work with me to design and implement
a no-syntax (*) software-development service?
* (The only syntax would be for *data*:
- integers, in whatever base the customer wants
- strings, given as highlighted/boxed text rather than
quotemark-delimited inline strings
- ordered lists, presented just about any way the customer wants
that we can implement easily
- higher-level objects designed by the programmer, typically some kind
of "intent wrapper" around a restricted set of nested lists
When I say "no syntax", I mean no syntax for algorithms.
There's still syntax for *data*.)
Note: The list of data types above is *only* a very very
preliminary off-the-cuff idea I composed just now. I haven't yet
found even one person to brainstorm with regarding the design we
might implement, and it isn't worth my energy to try to design
the whole thing without "input" from other people.
> For simple little web apps, rails is still my choice for simplest
> way to get from point A to point B.
OK, taking a look at: <http://www.rubyonrails.org/>
To go live, all you need to add is a database
and a web server.
What kind of database is needed? If MySQL or other standard RDBS is
not available/feasible, is there a pure-Ruby implmentation of a
RDBS, something like CL-SQL?
By the way:
% whereis ruby
(nothing comes back; is there another name I should be looking for on Unix?)
it's a great fit for practically any type of web application
It isn't a great fit on my Unix shell ISP where it's not even available.
By comparison, PHP *is* available. Given circumstances here, PHP
would be my choice for newbie framework for Web site with dynamic
responsive content including handling submitted HTML forms.
A lot of free Web-hosting servers also provide PHP, with MySQL, and
PHP would be my choice for newbies there too.
Do you know of any free Web-hosting services that provide Rails?
> >> ruby is def. way out in front on the community being newbie friendly
> >> compared to lisp.
> > But not compared to my proposed no-syntax software-development system?
> ... i mean in terms of someone coming to this forum and asking
> newbie questions vs someone asking on the ruby talk mailing lists.
> the tone is very different. ruby talk has a much friendlier tone to
> answers, a helpful tone. the community aspect is very big in the
> ruby world and for lack of better terms in comparision to one
> another, the ruby community comes off as 'warm and fuzzy' compared
> to lisp's 'cold and prickly'.
We're harsh towards people asking us to do their homework for them
and refusing to do even part of the work themselves. We're harsh
toward people trolling in obvious ways. But I was under the
impression that sincere newbie questions are handled nicely here.
Please explain more about the difference in warmth/fuzziness.
Maybe we can improve our style when responding to sincere newbie questions.
I would like to see a direct comparison in response to "the same"
question in both forums. Of course any question about syntax of a
particular language can't possibly be asked the same in both
forums. Perhaps a question of the form "I have this kind of data,
and I need to get advice as to a good format for expressing that
data in machine-readable form, so that I can then perform this kind
of analysis upon that data" might allow parallel replies in both
forums as a basis for warm/fuzzy comparisons? Do you know of any
recent questions like that in either forum?
Robert Maas, http://tinyurl.com/uh3t
> I have to admit that I misunderstood the context of your phrase
> "engage the enemy". Rereading your original post more carefully
> cleared it up for me.
Correction accepted.
> > There's a key difference: The former starts with a package in CL
> > that implements a domain-specific language. "The enemy" acts like
> > CL is worthless, and instead hand-codes the entire domain-specific
> > language in C, and then advertises that CL is crap and their new
> > language is much better, and only later does some Lisper emulate
> > the new language inside Lisp like should have been done in the
> > first place.
> This sometimes occurs, and it almost always makes headlines here.
> However, because Lisp (including CL) is a universal progenitor, and
> adding a little animism to this already thick analogy, it stands to
> reason that Lisp will spawn some children that will grow to hate it.
Hmm, so you believe somebody who actually likes the ideas in Lisp,
nevertheless abandons it totally and implements a new programming
language entirely using C to build it instead of implementing it in
Lisp? Why would somebody do that? Is it because there are free C
compilers on all systems but free Lisp is missing on some systems
and the new-language designer isn't willing to pay the license fee
to use Lisp as the basis for compiling and/or emulating the new
language? Or is it because the new-language designer isn't aware
that Lisp can write binary output files just the same as any other
language, so Lisp can generate C-compatible libraries and
applications and/or directly generate executables just the same as
C can except it's easier to write the compiler etc. in Lisp?
> This strength is one of the major reasons why Lisp will never be
> popular; every program (taking my extreme cases below) is a new
> language, and people who get used to that language are pulled
> away from the language that made their new one possible.
The strength of inspiring new ideas is *not* IMO a reason for Lisp
failing in popularity here, because it would make sense to do all
the work for the language right within Lisp rather than going back
to C as the implementation language. But a perception that Lisp
isn't available on all platforms at an affordable price, or that
Lisp isn't capable of being used to generate libraries, might be a
reason. I just don't understand the connection you assert between
Lisp's strength at spawning new ideas and Lisp being abandoned in
this context.
> It is a natural tendency to program in one's favorite DSL
> exclusively, regardless of whether it was or is coded in Lisp, and
> regardless of whether it interoperates with Lisp.
I agree, but this is irrelevant to the reason why the *implementor*
of the new DSL chooses to hand-code it all in C rather than take
the easy route of writing it in portable ANSI Common Lisp.
> If a person has an elaborate set of key bindings in emacs, or if
> he gets used to an emacs/lisp interface such as slime, he becomes
> increasingly dependent on these bindings or on his setup in order
> to operate, and whenever faced with a new system setup, the first
> thing he _must_ do is to set up his key bindings before he can be
> productive; he finds that he cannot operate efficiently without
> them.
But he's still running *in* elisp, not in some re-implementation of
EMACS by means of C, or even worse a handcoded implementation of
his key bindings in C without using any form of EMACS whatsoever.
So I don't see this as a good analogy.
> But in any extensible language or highly configurable system, the
> potential is always there.
If somebody wants to set up his personal function bindings for his
personal DSL within Common Lisp, I see no problem. When he tries to
port his code for others to use, of course he'll have to either set
up his private key bindings as a dependency for his application, or
re-code his application to avoid use of his private function
bindings. (Of course with Lisp it's easy to automatically scan
source code to detect usages of nonstandard functions, making it
easy to eliminate such dependencies.)
> Look again at the link I provided; you'll find that clpython is
> written entirely in CL. In fact, those portions of the python
> library that are written in C must be rewritten in CL before they
> can be used in CLpython.
Here's my read on that: Python itself is somebody re-inventing the
wheel rather than simply emulating a DSL from within Lisp. But
CL-python is somebody on *our* side emulating phthon to pull the
rug out from under the wheel re-inventor. Do you agree?
> Some say that Lisp has no syntax.
Those people are wrong. Forth is the language with no syntax. Lisp
has a very simple syntax, essentially: A few kinds of atomic tokens
treated as first-class objects (strings, symbols, numbers), and one
non-atomic rule for expressing lists by putting parens around
expressions of objects. The only precedence is at the tokenizer
level, that inside the expression of a string parens are regular
characters whereas at the regular parse level parens are used to
nest lists. Lisp has no precedence beyond the tokenizer, unlike
most other languages. But very simple syntax (one production rule
beyond tokenizer the rules) is not the same as *no* syntax.
Of course Common Lisp has a few other production rules, whereby
vectors and some other objects with composite parts can be
recursively expressed in a very similar way to how lists are
expressed. But Lisp per se (common to all Lisps from 1.5 onward)
has only that one production rule beyond the tokenizer, i.e. only
one recursive production rules. All the tokenizer rules are leaves
or dead-end branches. Only parens lisps are recursion loops.
> So how do you change something that's not there?
Your premise is false, so your question is nonsense.
In fact Lisp macros don't change the syntax at all, they change
only the semantics. The set of nested parenthesized lists are the
same with or without macros. It's just that the semantics of some
forms, which without macros defined would evaluate all except the
first element recursively then apply the first element to that list
of evaluated arguments, would instead not evaluate the arguments at
all before applying the macro to the whole form to transform it
into another form. Both the pre-expansion and post-expansion forms
are of the same general internal structure, which can be expressed
using exactly the same general syntax. (If we have loops in the
resultant structure, then we have to extend the syntax slightly to
support *print-circle*, or allow a theoretical infinite
s-expression that can never be completely printed, but that kind of
result is quite rare when expanding macros.)
> If you consider the "syntax" to be all-the-parens,
Well the syntax is the rule that parens around representations of
objects separated by whitespace is a representation of a object of
type LIST.
> then yes, you may have to deal with the question of recreating
> the syntax of the target language, but many such implementations
> don't do that - they either define their DSL to have a paren-based
> syntax, or, in the case of an existing language, they may just punt
> and provide the DSL with lisp paren style.
Well, somewhere in this thread I distinguished between:
- function-only DSLs, not even macros are needed
- macro-only DSLs where the syntax is exactly the same as Lisp,
only the semantics is different
- readtable+macro DSLs where the syntax is such a small extension
from Lisp syntax that readtables (and macros) are sufficient
- new-parser DSLs where the syntax is sufficiently different from
Lisp syntax that a whole new parser needs to be written,
possibly by just specifying something like BNF to a general
parser, possibly by hand-coding a parser
Implementing *any* of those kinds of DSLs is relatively easy in
Lisp, compared to trying to hand-code it all in assembly language
or C or Forth or even Java.
> > I would categorize three levels:
[snipped]
Ah, it wasn't *just* somewher in this thread, it was in the very
message you were replying to. I overlooked the case of readtable
modifications, so there are actually four not three levels of DSLs
implementable using Lisp, as I correctly listed above.
> These three [now 4] levels are not all there are, but yes, they
> can be said to categorize a DSL into degrees-of-recognizability wrt
> Lisp. But even within these levels, there are degrees of variance
> which determine/allow the distance the final language will be from
> Lisp.
Of course. For example, you can define just a few macros, and have
everything else be regular Lisp. Or you can redefine *all* the
functions and macros you would ever need for your DSL, and never
call even one native Lisp function. Both of those would be at the
second level of just macros, no readtable mods and no new parser.
Or you can add a new parser which is used only some of the time,
such as to parse complicated mathematical expressions, but write
the rest of your code in native Lisp. Or you can write a new parser
that carries the full weight of the new DSL, with no way to excape
back to native Lisp code. You can use the parser to generate code
that is interpreted or JIT-compiled within Lisp, or you can use the
parser plus a compiler to generate a native-CPU executable. All of
those variations are possible within the single level of
new-parser.
Given those variations of how much of Lisp you keep and how much of
the new DSL outright replaces it, the distinction of four levels of
DSL (functions, macros, readtables, parser) may be "academic".
Still IMO it's a good starting point for making sure we "cover all
bases" when discussing this topic, that we don't overlook something
major.
> > I would consider -1- *not* creating a DSL, merely a domain-specific
> > library.
> Having come from backgrounds in many Algol-style languages and
> also Forth, I will not quibble with the idea that a Lisp function
> or a Forth word is equivalent to a Basic subroutine. However, with
> the meta-programming available with <builds does> words and with
> special Lisp functions that happen to be known as macros, the
> extent of your claim for such programming to be "merely ...
> libraries" is arguable, and is mostly in the eye of the beholder.
The point was that if you don't ever define even one new macro in
implementing the new DSL, then it's "just a library". The fact that
the capability to define new macros was available and *could* have
been used is IMO moot if such never occurred in building this
particular new DSL. Now the point about variation *within* a single
level of DSL creation is of course valid. It's possible to define
new functions with such completeness of coverage of the domain that
a DSL application never need call even one native LIsp function,
can get by calling *nothing* except the DSL-defined functions. So
if you glance at source code in the DSL, you'd see the familiar
s-expression notation, yet not recognize even one familiar function
name, and it would sure not look like Lisp. Maybe some of the
people who like Lisp's functionality but don't like the arcane
names such as CAR CDR CONS LAMBDA etc. could simply import-rename
all the usual fuctions with names they like better LEFT RIGHT PAIR
ANONFUN for example.
Actually I have an even better, which requires me extend another
dimension to my classification of levels of DSL. In that
classification (three or four levels) I assumed that EVAL worked
the same at all levels. But it occurs to me that completely
replacing EVAL might be a different direction to go. A few decades
ago there was work on two new polymorphic languages, ECL (EClectic
Language) and PPL (Polymorphic Programming Language), and I vaguely
recall one or the other used production rules to define cases of
functions applied to different types of arguments, similar in some
ways to generic function in Common Lisp. (I have the manuals for
each language, but they're all in storage where they're
inaccessible to me currently.) I tried a Google search, but ECL was
the only one where I could find a code sample, and it was like
Algol 60, not like production rules, so it must have been PPL that
had the production-rule syntax, something vaguely like this:
factorial 0 := 1
factorial n := (factorial n-1) * n
where in that example the first matching rule masks any later rules.
(I'm not sure PPL did that first-match-mask, it's been many years)
The bad thing about such a language is that you have to write a
whole new parser for it, and it's not obvious what the abstract
parse tree should be for any given production rule. So my new idea
today is to use s-expression syntax for production rules, so the
Lisp parser (READ) and printer work just fine, but EVAL needs to be
completely replaced. Production rules could be used not just for
expression generic functions which take different types of
parameters, and special-case overrides such as 0 overriding n
above, but also for automatic destructuring, like this:
((factorial 0) 1)
((factorial n) (* n (factorial (- n 1))))
((copy-tree (a . b)) (cons (copy-list a) (copy-list b)))
((copy-tree c) c)
((copy-list (el . tl)) (cons el (copy-list tl)))
((copy-list a) a)
((copy-alist ((k . v) . tl)) (cons (cons k v) tl))
((copy-alist a) a)
((reduce f a b . tl) (reduce (cons (funcall f a b) tl)))
((reduce f a) a)
(Extra whitespace added to make righthand sides of productions line up.)
I hope I didn't make any typos there. I don't have a system for
doing those examples to see if they actually work. The best I could
do was test the syntax by feeding that input to (loop (print (read)))
> there are many other uses for macros, reader macros,
I'm treating regular macros and "readtable macros" as two
completely different concepts.
If you mean that some macros are used to make DSLs whereas other
macros are used just to enhance Common Lisp, then I agree, except
that in an important sense the enhance-CL macros do really extend
CL to be in a weak sense a new language beyond regular CL. Even the
built-in macros such as WHILE-OPEN-FILE extend the base CL to make
it easier to program certain classes of applications, such as
processing the contents of files. In a sense CL really is a general
purpose base language (similar to Scheme in its simplicity), plus
several domain-speciic extensions to make it easy to handle certain
kinds of data-processing, such as parsing strings, processing data
in files, formatting reports, embedding languages, etc. These
domains are much more general then DSL usually refers to, but still
there's the idea that certain types of semi-specialized software
applications ar made much more easier with these language additions
than they'd be without them. Both functions and macros in ANSI CL
provide these semi-specialized D/P processing aids (which are in
general not present from Scheme). Java has even more of these
semi-specialized-domain add-ons included as part of the
vendor-supplied J2SE, such as XML, GUI, applets, beans, RMI,
security, sound/MIDI, and CORBA. All the Java add-ons are at the
level of function (method) definition, because Java (at least
through version 1.4) doesn't allow users to define new macros.
> Also, between #2 and #3 is the less-explored world of
> code-walking, where the forms are lisp-like but the rewriting is
> done on a whole-program basis with rules that need not be lisp-like
> at all.
This sounds similar to my different direction I proposed above,
such as the production-rule semantics as an alternative to generic
functions within the same-old-EVAL semantics.
> You may consider these to be weak re DSL, but if you interact
> with them their behavior might surprise you.
The nice thing about defining new semantics to replace EVAL is that
EMACS can still edit such sourcefiles in Lisp Mode without problem!
Actually, as Turing et al proved long ago, even software written in
regular Lisp/Algol procedural style, no changes whatsoever to
either syntax or semantics, can still surprise the author. So this
issue is rather irrelevant to the task of defining the distinction
between a DSL and the original language it derived from. In a
sense, if you define just one new function that has application
only within a limited problem domain, then Lisp plus that one new
function is a DSL if you call that new function all over the place
and hardly ever directly call any of the built-in functions in
Lisp. But surely if you change the syntax by means of extensive
reader macros, and use those new cases all the time and never again
write built-in s-expressions again, or if you write a new parser
and never again call READ, that would count as a fullfledged DSL.
Where to draw a "line" between one function and a whole new parser
is pretty much a matter of personal opinion on any given day.
Perhaps we ought not to debate the precise location of the "line",
and instead just understand the five (at latest count) ways to
extend Lisp toward a DSL
(define functions, define macros, change the readtable including
defining new reader macros, write a whole new parser, walk the
code in a different way from how EVAL does it)
and strive to use these methodologies, instead of hanc-coded C, to
implement all new DSLs, and strive to enlighten others to please
use these methodologies instead of starting from scratch using C or
assembly language.
I think it's neat that one bright person, acting alone, was able to
conceive a whole new programming-language syntax, give it a fancy
name "Flaming Thunder", and hand-code a compiler for it in assembly
language, such that it's so small (70566 bytes the last time I
downloaded it) that I could install it on my private directory
(which has very little free space), and that it could satisfy my
criterion for CGI-Hello-plus-3-steps, only the 7th language to
achieve that (after PHP, Perl, C, C++, Common Lisp, and Java). But
I really doubt that Flaming Thunder will ever achieve half the
capabilities of Common Lisp or Java, nor the wide use of the other
four languages, no matter how bright its author is and how much
personal work he puts into it. Sooner or later the other six
languages will accomplish some software benchmark which the author
of Flaming Thunder won't be able to duplicate, at least that's my
ultimate prediction as to its fate. But for the moment, Flaming
Thunder is above Python and C# and many other highly touted
languages in being able to demonstrate live HTML-form-CGI-decoding.
(I'm sure JSP can do the same, but JSP isn't available here and
nobody has volunteered to set up a demo of it. I suspect ASP could
too, but likewise nobody has shown any interest in setting up a
demo of it either. DItto Fortran. I don't have any idea whether
Python or Cobol could do it.)
> Also, there is a lot that can be done within #2 with reader
> macros that encroaches on #3 - the lisp reader is extermely
> powerful.
Yeah, I overlooked readtable modifications, reader macros, in my
original list. I've never done any readtable modifications myself
because it's so tricky to get right it's not worth the effort for
my purposes.
> > ... In a sense, Reduce and MacSyma are true examples of
> > DSL at level 3, because they *both* provide a true DSL (symbolic
> > algebra) rather than a general-purpose language and also involve
> > the new parser for a totally non-sexpr syntax. ...
> No contention here; we've approached a turing argument, and it is
> less interesting to me, because such things can be done in any
> decent language.
Unfortunately most software engineers use C, which isn't a decent
language at all, and where implementing a new parser requires
either using a third-party storage-allocation package or
re-inventing storage allocation from scratch (with only malloc and
friends, and defstruct, to help). Both Reduce and MacSyma used Lisp
as their base language (SL/PSL and MacLisp respectively), which
gave them a big advantage compared to stuff written in C.
I agree that *any* Turing-complete language can theoretically be
used to implement a new parser for a new DSL, but we ought to keep
everyone in the know that Lisp is the best language in which to
write such a new parser, that it's silly to write a new parser in C
unless except just to "prove a point" like how manly you are what a
good hacker you are etc. As a practical measure, everyone should
program new parsers in Lisp (or possibly Java). How do we get
everyone to understand that? Is it that Lisp isn't a part of a new
operating system as delivered from MicroSoft whereas Visual C++ *is*?
Is MicroSoft, pushing Visual C++ and Visual Basic at the expense
of Lisp, our real enemy? And all the programmers who implement
new parsers in C instead of Lisp merely innocent victims of MicroSoft?
> In reality, though, as I said above, the people who use these
> DSLs tend to be locked into them by force of habit. It is not Lisp
> that is the straighjacket, but the tendencies of the users.
And as I pointed out in response, the reason why they are using a
DSL handcoded in C instead of written in Lisp comes first. I don't
mind that they use their DSL. I only mind that their DSL isn't
written in Lisp, so it can't be extended easily to take advantage
of features that Lisp provides already but which would need to be
written from scratch in C.
> > How do we convince them *not* to implement a new DSL in C or
> > assembly language but instead implement each new DSL in Lisp??
> There it is again; us against them. Who is "we", and who are "they"?
They are people who choose to hand-code everything in C, instead of
taking advantage o Lisp's more powerful built-in capabilities,
which then cause their users/victims to be locked into a
non-Lisp-based DSL.
We are the people who code virtually every new application in Lisp,
because we realize we're a lot more productive that way. We code in
other languages only after considering Lisp first and having a
really good reason not to use Lisp for that particular application.
(Did you look at the section of my hello-plus document where I list
a few reasons somebody might want to ever code an application in
some language other than Lisp? Any additions/changes to suggest?)
> The minute you consider an "enemy", you have a war.
If MicroSoft is our ultimate enemy, we may indeed have a war
already. Millions of computers running versions of MicroSoft
WIndows have been infected with spambots which have been flooding
my mailboxes with hundreds of spam per day. When will MicroSoft
pay damage compensation to all the victims of all the spam they've enabled??
> The answer is: "we" don't convince them, we demonstrate by doing,
> and "they" can respond however they want to
It still may be helpful to educate them as to how much more
cost-effective it is to code new applications *our* way. But how do
we enlighten them? What do most people on the InterNet appreciate?
The Web, of course!! But about the Web do they most appreciate?
Search engines such as Google? What else? I think we need to figure
out what Web services most people appreciate, then provide better
versions of such services, as well as brand-new services they might
also like, using Lisp, and make it clear to customers that Lisp is
what's making it so much better, so that Lisp will become a
well-liked buzzword in popular culture instead of just an academic
reference as most people consider Lisp to be. I don't think we'll
be able to provide a general-purpose search engine better than
Google already provides. Can you think of some other Web service
where we might be able to easily out-do existing services and
thereby put up a showpiece for Lisp?
On a related topic: How easy is it to use MySQL from Common Lisp?
How easy is it to use a Common-Lisp implementation of a relational
database instead? Is the latter sufficiently reliable to be used
instead of MySQL for data we really really don't want corrupted by
race conditions or other problems? A relational database is a
nearly essential component of most large applications that deal
with large amounts of organized data. Currently I have in mind
several new applications I might like to implement, each of which
would require a relational database to make it work decently.
> > Wouldn't helping them to get started with their new DSL in Lisp, so
> > that they are our "friends" from the start of their new-DSL
> > project, be better than *not* helping them, thereby driving them to
> > ignore Lisp and use C or assembly language, so that they are our
> > enemy from the time they release their first new-DSL until years
> > later when we finally get around to emulating their DSL in Lisp??
> Absolutely. How is this different from precisely what Willem Broekema
> has done with CL-Python, which was the original impetus for our part
> of this thread?
Huh? I was under the impression that Python came first, then
CL-Python came later. I would have wanted to avoid regular Python
in the first place and have CL-Python be the very first (and only)
implementation of Python syntax. Do I misunderstand the history of
Python? I tried to find that info online but the WikiPedia history
section makes no mention of what language was used to first code
Python, and the source distribution on the Python site is a TAR
which reuires downloading the whole thing before I could look at
even one line of code. I tried to look locally just now.
There's an executable here: /usr/local/bin/python
But /usr/local/src/ is read-protected so I can't see Python's source.
2 drwx------ 64 root wheel 2048 Oct 27 2007 /usr/local/src/
> But I'll give the standard answer that we tend to give on c.l.l.
> to anyone who says things like "Here what I think the Lisp
> Community should do...", and that is: OK, you have an idea for what
> should be done. When will I be able to read about what you've done
> to make it happen?
Just me by myself, without even one other serious user to beta-test
my Web services, there's not much that can be done. Have you tried
any of my existing free Web services, and given me suggestions how
to make them better? Have you looked at any of my proposals for new
Web services and offered any feedback of a constructive nature?
On the other hand, do you know of any other free Web services that
were coded in Lisp (by you, or by anyone else) that I ought to try,
so that I can beta-test them and thereby aid the community effort?
Robert Maas, http://tinyurl.com/uh3t
> When I open a dictionary, I can see that all words of a language
> are define in the same language.
That results in a lot of circular definitions.
> It's exactly the same thing for Lisp.
That wouldn't be reasonable. There must be some base language, a
subset of Lisp, that is defined in terms of something prior to
Lisp, and then all the rest of Lisp can be defined in terms of
that. That avoids circular definitions.
By the way, did you ever see the PBS series "Mechanical Universe"?
In particular do you remember the "function" animation that
resembled a hand-crank food grinder, where parameters come in at
the top and get converted to the value of the function? Perhaps
such an animation could be used in a tutorial for the base part of
a programming language in lieu of a textual specification. Instead
of a black-box food-grinder, perhaps something more like an
animation of biochemical catalysis, or a robot working at an
assembly line. For example, malloc could be represented by a
sheet-metal cutter who is loaned a measuring stick, and proceeds to
cut a piece of metal of that size from the master supply, and
returns the borrowed measuring stick and also passes over the cut
piece of metal. Free could be another worker at an adjacent station
who takes a piece of metal and returns it back to the main supply.
CONS then calls malloc to get a block of metal big enough to hold
two pointers, and the attaches the two parameters to the two halves
of that newly-allocated metal. Garbage collector would be a worker
who looks under the floor boards for any pieces of metal that have
fallen down because they are no longer attached to anything, and
ferries all those pieces of metal back to free. There would be two
versions of garbage collector, one with a magic stopwatch that
makes all the other workers free while he sweeps everyone to see if
any loose stuff can be induced to fall down, and one without a
magic stopwatch who doesn't stop anybody, just collects stuff that
has already fallen all the way down, ignoring stuff that is loose
but hasn't yet fallen all the way down.
samantha
> Just came across this...
>
> http://www.regdeveloper.co.uk/2008/04/21/bray_ruby_rails/
>
> You'd think there would be some way for Lisp to fill this gap.
>
This may be a dumb idea but it keeps recurring so I'll get it out in
the open for some debugging. Is it at all reasonable to take say a
Ruby program and translate it into an equivalent Lisp program that is
then compiled? Are you likely to gain much performance wise? The
same question could apply to other languages such as Python. I think
the answer is that it is easy enough to write Python or Ruby in Lisp
but that extracting the actual useful semantic intent sufficiently to
do more than a brute-force rewrite from Ruby or whatever to Lisp is
really hard. But the idea still nags at me. So what do you guys
think?
- samantha
samantha
> > ...CTOs at big banks and airlines are calling Bray in to advise on Ruby.
>
> Scary.
Yeah. Sort of like those stars of my long ago youth who now appear on
late night TV doing commercials for god-knows-what.
Ruby on Rails is a cool little hack. But I've done object
persistence, object databases, transparent data persistence stuff for
most of my career. Compared to the full domain that is in need of
addressing Rails is a toy off in one corner. It is a corner where a
lot of small apps live and it is nice they have a better and simpler
cookie cutter to churn these things out with. But it is just not the
cat's meow it is made out to be.
- samantha
Brian Adkins
"not the cat's meow" is vague. I'm curious about the deficiencies you
see in Ruby/Rails; would you care to elaborate?
Pascal J. Bourguignon
>> From: Christophe <christophe.allegr...@birdtechnology.net>
>> When I open a dictionary, I can see that all words of a language
>> are define in the same language.
>
> That results in a lot of circular definitions.
>
>> It's exactly the same thing for Lisp.
>
> That wouldn't be reasonable. There must be some base language, a
> subset of Lisp, that is defined in terms of something prior to
> Lisp, and then all the rest of Lisp can be defined in terms of
> that. That avoids circular definitions.
There's no problem with circular definitions. You just need a
bootstrap procedure.
Of course, the bootstrap procedure is greatly simplified if you can
define your language in a reduced subset of that language. For lisp,
you can reduce it to just lambda. (and function application). Of
course, defining your language in a reduced subset is greatly
simplified if you can use a subset large enough to define it easily
For lisp, that means AIM-8, which is not too big, but still large
enough to be able to implement that language on one sheet of paper.
Then you can implement that sheet of paper in assembler by hand,
<bootstrap> and the rest is history.
--
__Pascal Bourguignon__
Maciej Katafiasz
> You might try leaving *PRINT-CASE* alone and instead try changing
> (READTABLE-CASE *READTABLE*) from :UPCASE to :INVERT, which both
> preserves any CamelCase you might need to READ from data files and also
> downcases all-uppercase symbols when printing
This is a bit late in the discussion, but: After you mentioned that
previously, I had been running with :INVERT for a long time, but finally
gave up when CFFI-GROVEL broke for the nth time in a way I completely
didn't understand. As much as I loved the neat semantics of :INVERT, too
much code broke with it (to the point that I became "the guy who runs
with strange READTABLE-CASE" on #lisp). One of the most popular offences
was (:use :CL). I've been running with *PRINT-CASE* set to :DOWNCASE
since then, and it seems to break less.
Cheers,
Maciej
Robert Maas, http://tinyurl.com/uh3t
> > That results in a lot of circular definitions.
> >> It's exactly the same thing for Lisp.
> > That wouldn't be reasonable. There must be some base language, a
> > subset of Lisp, that is defined in terms of something prior to
> > Lisp, and then all the rest of Lisp can be defined in terms of
> > that. That avoids circular definitions.
> There's no problem with circular definitions. You just need a
> bootstrap procedure.
You've contradicted yourself. The whole point of a bootstrap
procedure is to *avoid* having an actual circular definition.
Syntactically it appears to be a circular definition, but actually
the key term of the apparent circularity is defined not in terms of
itself but in terms of the previous bootstrap-round of its
syntactic twin.
Here's a simplified scenerio.
- You use an already known language AKL to define a new term T1, and
write code C1 in that already-installed language to compile T1 to
machine langauge. We denote any such compilation result by R1.
- You use the new term T1 to express the act of compiling T1 to
machine language. We denote that expression involving T1 by T0+T1+T2
where T0 and T2 denote the additional parts of the expression
that don't mention T1 hence already compile just fine in AKL
hence aren't relevant for this discussion.
- You use the already-known-language, plus C1, to compile T0+T1+T2,
resulting in machine-language program R0+R1+R2, which can compile T1.
- You use R0+R1+R2 to actually compile T0+T1+T2 to something that
is functionally identical to R0+R1+R2, thus producing a compiler
that can compile itself (per functionality). We'll denote that
second-order compilation output as RR0+RR1+RR2.
You look at what was defined in terms of what, and you see that:
RR0 was defined in terms of T0 (the data) and AKL+R0+R1+R2 (the compiler).
RR1 was defined in terms of T1 (the data) and AKL+R0+R1+R2 (the compiler).
RR2 was defined in terms of T2 (the data) and AKL+R0+R1+R2 (the compiler).
R0 was defined in terms of T0 (the data) and AKL (the compiler).
R1 was defined in terms of T1 (the data) and AKL (the compiler).
R2 was defined in terms of T2 (the data) and AKL (the compiler).
Thus the semantics of T1 the second time around, namely RR1, are
defined not in terms of itself but in terms of R1, which are in
turn defined not in terms of itself but in base terms from AKL.
Now eventually with many iterations of compilation, you may reach a
fixed point where the current version of a compiler, being given
the current source code for the compiler, will in fact build an
identical core image for that compiler. Thus it may appear that the source
code for that compiler is all you need to define the semantics of
that compiler. Ain't so. Just try to compile the compiler without
the compiler already around. You need either the compiler itself
already around, which begs the question
(you already have the compiler, so you don't *need* to create it
again, and re-compiling the compiler doesn't give you anything you
didn't already have),
or the just-previous of the compiler, whereupon the current
compiler is defined not in terms of its own source alone, but its
own source *plus* the previous version of compiler. For example, if
it takes five bootstrap rounds to reach the fixed point, then
(denoting source by S and compiler versions by C1,C2,C3,C4,C5)
we have essentially:
AKL+T1+S -> C1
C1+S -> C2
C2+S -> C3
C3+S -> C4
C4+S -> C5
C5+S -> C5 (dunzell, doesn't produce anything we didn't already have)
So omitting the dunzell, there's no circular definition.
As for natural-language dictionaries that are written in the same
language: In general they don't provide actual definitions of
words in the mathematical sense, especially they don't provide a
rigorous way to define all the words in the language. At best they
provide an aid to learning some new words when you already know
most of the words and at least one word that expresses *each*
concept. If you look up a word you don't know, you follow various
paths until you find a definition that involves *only* words you
already know. For example, denoting known words by K<number> and
unknown words by U<number>, you might backtrack like this:
U1 defined in terms of (K1 K2 K3 U2 K4 K5 U3 K6 K7 U4)
U2 defined in terms of (K1 K5 U3 K8)
U3 defined in terms of (K3 K4 U4 K7 K9)
U4 defined in terms of (K2 K6 K8 K9)
U4 ultimately defined in terms of (K2 K6 K8 K9)
U3 ultimately defined in terms of (K3 K4 (K2 K6 K8 K9) K7 K9)
U2 ultimately defined in terms of (K1 K5 (K3 K4 (K2 K6 K8 K9) K7 K9) K8)
U1 ultimately defined in terms of (K1 K2 K3 (K1 K5 #1=(K3 K4 #2=(K2 K6 K8 K9) K7 K9) K8) K4 K5 #1# K6 K7 #2#)
> Of course, the bootstrap procedure is greatly simplified if you
> can define your language in a reduced subset of that language.
Yeah. I outlined something like that using an enhanced version of
LAP plus SYSLISP to allow a smooth transition from AKL through
+LAP+SYSLISP to +LAP+SYSLISP+BaseLisp to +LAP+SYSLISP+FullLisp.
<http://groups.google.com/group/comp.programming/msg/c39295233746b158>
= Message-ID: <rem-2008...@yahoo.com>
| Subject: Building Lisp from scratch (was: Program compression)
> For lisp, you can reduce it to just lambda. (and function application).
Unfortunately that's not quite sufficient to allow true
bootstrapping, where you have more than just a chalkboard handwave
or a "proven" mathematical theorem, but you have an actual compiler
that can compile itself. To accomplish that, at a minimum you need
some primitive to open/close disk files and to write binary data to
a binary output file. (And it would be nice to be able to specify
the value of each byte in something other than decimal, although
that's not necessary at the very start, since you could include the
definition of PARSE-HEXADECIMAL within the build-up of a function
library, except you don't have strings in the first place, so I
guess you also need to include an improved version of READ that
understands string syntax, and somehow fake the string intentional
datatype as some sort of S-expression involving symbols and
numbers.) Thus (open-binary-file "foo.exe")
(write-byte (parse-hexadecimal "C0"))
(write-byte (parse-hexadecimal "FF"))
(write-byte (parse-hexadecimal "58"))
(write-byte (parse-hexadecimal "27"))
(close-binary-file)
(Note: Only one binary file can be open at a time in this
hypothetical bootstrap SubLisp environment, so you don't need to
specify the stream.)
But really, LAP and SYSLISP seem like a better starting point for
the binary part of compiling the next version of the compiler.
(See that other article cited above for raw-draft ideas.)
> Of course, defining your language in a reduced subset is greatly
> simplified if you can use a subset large enough to define it easily
> For lisp, that means AIM-8, which is not too big, but still large
> enough to be able to implement that language on one sheet of paper.
I got curious and did a Google search for
AIM-8
got lots of false matches, tried searching for
AIM-8 Lisp
Bingo, found it, first time I ever saw this seminal paper by
McCarthy, see URL and my notes which follow later below.
> Then you can implement that sheet of paper in assembler by hand,
> <bootstrap> and the rest is history.
Yes, you absolutely need that assembly code to build the executable
that you can then use to emulate the L-calculus and thence feed in
McCarthy's source code for defining a Lisp interpretor. But then
you also need what I said above, some Lisp primitives for binary
file I/O, so that you can actually compile an executable from
S-expression source, and ideally also with LAP and SYSLISP to make
the process scrutable (the opposite of inscrutable chicken
scratches).
<http://www.informatimago.com/develop/lisp/small-cl-pgms/aim-8/index.html>
->
<http://www.informatimago.com/develop/lisp/small-cl-pgms/aim-8/aim-8.html>
The S-expressions are formed according to the following recursive rules.
1. The atomic symbols p_1 p_2 etc are S-expressions.
2. A null expression is also admitted.
3. If e is an S-expression so is (e).
4. If e_1 and (e_2) are S-expressions so is (e_1,e_2).
OK, except 3 is redundant if 4 is present and we
presume that (e_1,e_2) denotes the dotted-pair (e_1 . e_2)
while (e) is shorthand for (e . NIL) which would be
(e,) in here. Thus a proper list of three elements
could be represented equally as (e_1,(e_2,(e_3,))) or
as (e_1,(e_2,(e_3))) which in modern Lisp notation
would be represented as (e_1 . (e_2 . (e_3 . NIL))) or
(e_1 . (e_2 . (e_3))) or (e_1 . (e_2 e_3)) or (e_1 e_2 e_3).
... Some examples of S-expressions are;
AB
(AB,A)
So-far so-good, but ...
(AB,A,,C,)
That's not a valid S-expression per the above recursive rules.
There's no rule that says you can put more than two
sub-expressions directly inside a single level of
parens.
((AB,C),A,(BC,(B,B)))
Also not a valid S-expression per the above resursive rules.
All the inner S-expressions are valid, but the
outermost level of expression again uses three
sub-expressions directly inside the same parens.
Am I wrong, or didn't anybody notice that logical gaffe?
There are three elementary predicates:
1. null[e]
null[e] is true if and only if S-expression e is the
null expression . (We shall use square brackets and semi-colons
for writing functions of S-expressions since parentheses and
commas have been pre-empted. When writing about functions in
general we may continue to use parentheses and commas.)
Aha (guessing what comes ahead) ... prior to defining
the "universal function" (EVAL), we need separate
notational conventions for S-expressions vs.
S-functions. But after EVAL is defined, we can just use
an S-expresssion and say "the value of" or "==EVAL=>"
to convert the S-expression (lambda (arg) ...) into the
corresponding S-function (technically a "partial function"
as McCarthy pointed out earlier).
2. atom[e]
atom[e] is true if and only if the S-expression is an atomic symbol.
Note that in practice the ATOM predicate was changed
such that NIL (the empty expression) would also satisfy
the predicate, thus making ATOM the antonym of PAIRP or
CONSP.
3. p_1=p_2
p_1=p_2 is defined only when p_1 and p_2 are both atomic
symbols in which case it is true if and only if they are the
same symbol. This predicate expresses the distinguishability
of the symbols.
Why does he use an infix operator, instead of
maintaining consistency with the rest of his treatise
by saying eq[p_1,p_2] ? I'd say this is a stylistic gaffe.
4. first[e]
first[e] is defined for S-expressions which are
neither null nor atomic. If e has the form (e_1,e_2) where e_1
is an expression, then first[e]=e_1. If e has the form (e_1)
wehre e_1 is an S-expression again we have first[e]=e_1.
That's fine, but ...
5. rest[e]
rest[e] is also defined for S-expressions which are
neither null nor atomic. If e has the form (e_1,e_2) where e_1
is an S-expression, then rest[e]=(e_2). If e has the form (e_1)
wehre e_1 is an S-expression we have rest[e]=.
At this point, that's a totally silly name for that
function!! We have a pair of parens, with two
sub-expressions, with a comma between them. That's all
we have!! We don't (yet) have any sort of convention
that these expresssions are CDR-chained to form linked
lists, whereupon "rest" would mean the rest of the list
after the first CONS cell is skipped/omitted/ignored.
The appropriate name would have been "second" at this point.
Yeah, that conflicts with use of that word for dealing
with linked lists, but this paper is supposed to be a
theoretical essay, not a software manual, so who cares??
Clarity of exposition should overrule compatibility of
a software module which hasn't yet been defined.
("SECOND" as a synonym for CADR wasn't standardized until
Common Lisp, more than twenty years after this paper, right?)
Alternately he could have used LEFT and RIGHT without problem.
6. combine[e_1;e_2]
combine[e_1;e_2] is defined when e_2 is not atomic.
When e_2 has the form (e_3), then combine[e_1;e_2]=(e_1;e_3)
When e_2 is we have combine[e_1;e_2]=(e_1).
Some examples are:
combine[A;]=(A)
combine[(A,B);(B,C)]=((A,B),B,C)
Huh??? This makes no sense at all. Semicolon isn't a
valid part of S-expressions. No explanation of using
semicolon as a meta-character for expressing
syntactical manipulations has been defined so-far in
this paper. This is just inscrutable magic that
(e_1;e_3) would have any meaning whatsoever, and that
the meaning of (e_1;) would in fact be (e_1). Is it
possible that all uses of semicolon here are
typographic mistakes which should be comma instead??
I suspect the person who converted PDF to ASCII made a
goof here and needs to go back and proofread the entire
article to fix such mistakes.
If so, then a key definition is missing, that (a) is an
abbreviation for (a,), as I mentionned earlier, where
clause 3 of the recursive definition of S-expresssion
would therefore be redundant.
The funcntions first, rest and combine are related by the relations
first[combine[e_1;e_2]]=e_1
rest[combine[e_1;e_2]]=e_2
combine[first[e];rest[e]]=e
whenever all the quantities involved are defined.
Yes, the prime tautology of CONS trees.
<ot>That that is is. That that is not is not.
That that is is not that that is not.
That that is not is not that that is.
(The prime tautology of existence,
and the prime tautology of predicates,
per The Great Time-Machine Hoax by Keith Laumer.)</ot>
Pascal J. Bourguignon
>> >> When I open a dictionary, I can see that all words of a language
>> >> are define [sic] in the same language.
>> > That results in a lot of circular definitions.
>> >> It's exactly the same thing for Lisp.
>> > That wouldn't be reasonable. There must be some base language, a
>> > subset of Lisp, that is defined in terms of something prior to
>> > Lisp, and then all the rest of Lisp can be defined in terms of
>> > that. That avoids circular definitions.
>> From: p...@informatimago.com (Pascal J. Bourguignon)
>> There's no problem with circular definitions. You just need a
>> bootstrap procedure.
>
> You've contradicted yourself. The whole point of a bootstrap
> procedure is to *avoid* having an actual circular definition.
The whole point of a bootstrap procedure is to *implement* an actual
circular definition. The point of bootstrapping a compiler is exactly
to reach the fixed point.
This is how I define "bootstrap" for language compilers.
>> Of course, defining your language in a reduced subset is greatly
>> simplified if you can use a subset large enough to define it easily
>> For lisp, that means AIM-8, which is not too big, but still large
>> enough to be able to implement that language on one sheet of paper.
>
> I got curious and did a Google search for
> AIM-8
> got lots of false matches, tried searching for
> AIM-8 Lisp
> Bingo, found it, first time I ever saw this seminal paper by
> McCarthy, see URL and my notes which follow later below.
>
>> Then you can implement that sheet of paper in assembler by hand,
>> <bootstrap> and the rest is history.
>
> Yes, you absolutely need that assembly code to build the executable
> that you can then use to emulate the L-calculus and thence feed in
> McCarthy's source code for defining a Lisp interpretor. But then
> you also need what I said above, some Lisp primitives for binary
> file I/O, so that you can actually compile an executable from
> S-expression source, and ideally also with LAP and SYSLISP to make
> the process scrutable (the opposite of inscrutable chicken
> scratches).
No, you don't absolutely need that assembly. You could as well ask an
electronician to build hardware that would directly execute the lisp
code.
By the same token, you could have this electronician build hardware
that would directly execute lambda calculus, with the addition of a
few functions with strange side effects. Who said I/O had to be done
in "binary"? You can do I/O directly with Church numerals.
And think about it, in your computer, I/O is not done in "binary". It
is done in transistors and electrons.
It just happens that once you have a computer it's faster and cheaper
to implement your lambda calculus virtual machine, or your lisp,
programming this computer than soldering electronic components. Just
one student could do it in a few months, while it would have been a
major project if it would have been done with tubes.
But all this is rather irrelevant. The only thing that matters, it's
the fixed-point that John McCarthy invented, the omega of the
bootstrap process, but the alpha of lisp!
> <http://www.informatimago.com/develop/lisp/small-cl-pgms/aim-8/index.html>
> ->
> <http://www.informatimago.com/develop/lisp/small-cl-pgms/aim-8/aim-8.html>
>
> The S-expressions are formed according to the following recursive rules.
> 1. The atomic symbols p_1 p_2 etc are S-expressions.
> 2. A null expression is also admitted.
> 3. If e is an S-expression so is (e).
> 4. If e_1 and (e_2) are S-expressions so is (e_1,e_2).
>
> OK, except 3 is redundant if 4 is present and we
> presume that (e_1,e_2) denotes the dotted-pair (e_1 . e_2)
> while (e) is shorthand for (e . NIL) which would be
> (e,) in here. Thus a proper list of three elements
> could be represented equally as (e_1,(e_2,(e_3,))) or
> as (e_1,(e_2,(e_3))) which in modern Lisp notation
> would be represented as (e_1 . (e_2 . (e_3 . NIL))) or
> (e_1 . (e_2 . (e_3))) or (e_1 . (e_2 e_3)) or (e_1 e_2 e_3).
You presume wrongly. Read the definition of combine and you should be
able to understand that (a,b,c) is a list of three elements, and (a,b)
a list of two. And since combine is not defined with a second
argument being an atom, it's a CONS function used only to build lists.
(defun combine (a b)
(check-type b cons)
(cons a b))
> ... Some examples of S-expressions are;
> AB
> (AB,A)
> So-far so-good, but ...
> (AB,A,,C,)
> That's not a valid S-expression per the above recursive rules.
50 years ago, there was no personnal computer. Papers were written
with a pen, by hand, on sheets of paper, and given to a secretary who
would type it on a typewriter. Good luck if the secretary could
decipher your handwritten squiggles, and good luck if she could
transcribe it character by character with 100% accuracy. This would
explain why there may be sometimes typoes in old papers (there are
still typoes in recent papers too).
But in this case, I don't think it's a typo. The glyph used for the
"null expression" was also used in languages and grammars to denote
the empty word. Nowadays, we usually use 'e' to denote the empty
word. I think McCarthy was thinking of NIL as the empty word of
languages, and thus could as well write (A,⋀,B) than (A,,B) for #+cl(A
NIL B). And given that the unicode character ⋀ wasn't available on
the typewriter they had at the time, and that he would have had to put
there a space and write with a pen the glyph, it seems very probable
that he choose to use the empty word to represent the "null
expression" in that context.
> Am I wrong, or didn't anybody notice that logical gaffe?
Therefore I think you are wrong.
But even if you were right, don't doubt everybody with more than two
neurons will have noticed such a "gaffe".
That's the difference between a mathematical or computer science
_paper_ and a computer _program_. The former is interpreted by
wetware and can easily contain typoes or even logical errors, that are
automatically corrected by the wetware, while the later must be more
formally correct to be accepted by the machine (be it implemented in
software or in silicium), but cannot (yet?) be automatically corrected
at the logical level.
And any error would have been noticied (and have been noticed) very
fast, given that Russel implemented this lisp very soon.
> 4. first[e]
> first[e] is defined for S-expressions which are
> neither null nor atomic. If e has the form (e_1,e_2) where e_1
> is an expression, then first[e]=e_1. If e has the form (e_1)
> wehre e_1 is an S-expression again we have first[e]=e_1.
>
> That's fine, but ...
>
> 5. rest[e]
> rest[e] is also defined for S-expressions which are
> neither null nor atomic. If e has the form (e_1,e_2) where e_1
> is an S-expression, then rest[e]=(e_2). If e has the form (e_1)
> wehre e_1 is an S-expression we have rest[e]=.
>
> At this point, that's a totally silly name for that
> function!! We have a pair of parens, with two
> sub-expressions, with a comma between them. That's all
> we have!! We don't (yet) have any sort of convention
> that these expresssions are CDR-chained to form linked
> lists, whereupon "rest" would mean the rest of the list
> after the first CONS cell is skipped/omitted/ignored.
> The appropriate name would have been "second" at this point.
> Yeah, that conflicts with use of that word for dealing
> with linked lists, but this paper is supposed to be a
> theoretical essay, not a software manual, so who cares??
> Clarity of exposition should overrule compatibility of
> a software module which hasn't yet been defined.
That's why it's first and rest. And read it again, it's exactly that.
> ("SECOND" as a synonym for CADR wasn't standardized until
> Common Lisp, more than twenty years after this paper, right?)
Of course, CADR wasn't standardized before the standard. But CADR et
al. were already present in LISP 1.5 in 1963 (and I'd bet they were
already there in LISP 1 in 1960).
> Alternately he could have used LEFT and RIGHT without problem.
>
> 6. combine[e_1;e_2]
> combine[e_1;e_2] is defined when e_2 is not atomic.
> When e_2 has the form (e_3), then combine[e_1;e_2]=(e_1;e_3)
> When e_2 is we have combine[e_1;e_2]=(e_1).
> Some examples are:
> combine[A;]=(A)
> combine[(A,B);(B,C)]=((A,B),B,C)
>
> Huh??? This makes no sense at all. Semicolon isn't a
> valid part of S-expressions. No explanation of using
> semicolon as a meta-character for expressing
> syntactical manipulations has been defined so-far in
> this paper.
Yes of course. Read again:
1. null[e]
null[e] is true if and only if S-expression e is the
null expression ⋀. (We shall use square brackets and semi-colons
for writing functions of S-expressions since parentheses and
commas have been pre-empted. When writing about functions in
general we may continue to use parentheses and commas.)
> This is just inscrutable magic that
> (e_1;e_3) would have any meaning whatsoever, and that
> the meaning of (e_1;) would in fact be (e_1).
Unfortunately for you, the form (x;y) doesn't appear in the paper.
It's either (x,y) or [x;y]. It looks like you didn't read it, and you
are just copy-and-pasting random sections and transcribing some
incoherent misconceptions of yours.
> Is it
> possible that all uses of semicolon here are
> typographic mistakes which should be comma instead??
> I suspect the person who converted PDF to ASCII made a
> goof here and needs to go back and proofread the entire
> article to fix such mistakes.
Perhaps you could read the PDF if you don't trust the converter.
I would be very happy if you could signal me any correction.
(And notice that it is not converted to ASCII, but to UNICODE).
In addition to the PDF, you could also read this:
http://www-formal.stanford.edu/jmc/history/lisp/lisp.html
--
__Pascal Bourguignon__ http://www.informatimago.com/
COMPONENT EQUIVALENCY NOTICE: The subatomic particles (electrons,
protons, etc.) comprising this product are exactly the same in every
measurable respect as those used in the products of other
manufacturers, and no claim to the contrary may legitimately be
expressed or implied.
Robert Maas, http://tinyurl.com/uh3t
Why this response is so belated:
<http://groups.google.com/group/misc.misc/msg/cea714440e591dd2>
= <news:rem-2008...@yahoo.com>
> From: p...@informatimago.com (Pascal J. Bourguignon)
> >> There's no problem with circular definitions. You just need a
> >> bootstrap procedure.
> > You've contradicted yourself. The whole point of a bootstrap
> > procedure is to *avoid* having an actual circular definition.
> The whole point of a bootstrap procedure is to *implement* an
> actual circular definition. The point of bootstrapping a compiler
> is exactly to reach the fixed point.
Can we agree on this re-wording: The point of bootstrapping (a
compiler) is to provide a non-circular entry point into what would
otherwise be a circular definition (a "chicken-egg" conundrum). At
the entry point to the fixed-loop, there are two ways to reach that
point, one of which is to enter via the bootstrap path, which
typically comes from an earlier fixed-loop defined by an earlier
version (of the compiler), and the other way to reach that point is
to go once around the fixed-loop from itself. Without the bootstrap
entry point to that fixed-loop, there's no way to ever have the
fixed-loop in the first place.
(... AIM-8 ...)
> >> Then you can implement that sheet of paper in assembler by hand,
> >> <bootstrap> and the rest is history.
> > Yes, you absolutely need that assembly code to build the executable
> > that you can then use to emulate the L-calculus and thence feed in
> > McCarthy's source code for defining a Lisp interpretor. But then
> > you also need what I said above, some Lisp primitives for binary
> > file I/O, so that you can actually compile an executable from
> > S-expression source, and ideally also with LAP and SYSLISP to make
> > the process scrutable (the opposite of inscrutable chicken
> > scratches).
> No, you don't absolutely need that assembly. You could as well
> ask an electronician to build hardware that would directly execute
> the lisp code.
Yeah, if you had a thousand times as much money and nothing better
to do with it. I stand technically corrected but feasibly correct.
> ... And think about it, in your computer, I/O is not done in
> "binary". It is done in transistors and electrons.
The intentional type of all that TTL or cMOS etc. technology is
binary. Whenever the hardware fails to correctly implement that
intentional type, the computer stops working.
> It just happens that once you have a computer it's faster and
> cheaper to implement your lambda calculus virtual machine, or your
> lisp, programming this computer than soldering electronic
> components. Just one student could do it in a few months, while
> it would have been a major project if it would have been done with
> tubes.
Tubes??? Did the IBM 704 use tubes??? I just assumed it used
discrete transistors (and resistors, capacitors, etc.) on either
wire-wrap or printed-circuit boards, just like the IBM 1620 did (I
actually looked at the innerds once when it was being repaired or
demonstrated or somesuch), and the IBM 1401 too probably.
> But all this is rather irrelevant. The only thing that matters,
> it's the fixed-point that John McCarthy invented, the omega of the
> bootstrap process, but the alpha of lisp!
Hmm, you have a way with words there.
But still if we use C every time we build Lisp, the C lovers will
continue claiming that C is essential and everyone needs to use C
and Lisp is garbage because it can't even compile itself.
At the very least we need a version of LAP (coded in terms of
WRITE-BYTE of course) that allows Lisp to convert from a reasonable
representation of machine-language code to corresponding bytes of
data written to a file, such that an executable can be specified
directly from within the Lisp language and thus directly built from
such specification. So we might have "cheated" by using C to
compile the current version of CMUCL etc., thus requiring C to
exist every time we upgrade CMUCL to a new version, but in the
future CMUCL or whatever could directly assemble+compile its own
source (LAP+LISP) to its own executable and we'd never need to use
C again. Note, to distinguish this version of LAP from the usual
one (in MacLisp for example) which could *only* assemble the bodies
of function definitions, we might call this BOOTLAP or LOWLAP.
So has anybody actually written an assembler in Common Lisp,
something that can convert expressions of the form
((opcode ac mem)
(opcode ac mem)
(opcode ac mem)
(opcode ac mem)
(opcode ac mem))
or whatever the latest pattern is for Intel CPUs, either to an
array of bytes or directly to a binary output stream for a file
being written?
> Read the definition of combine and you should be able to
> understand that (a,b,c) is a list of three elements, and (a,b) a
> list of two. And since combine is not defined with a second
> argument being an atom, it's a CONS function used only to build
> lists.
The first mention of combine (except in the erratum added at the top) is here:
6. combine[e_1;e_2]
combine[e_1;e_2] is defined when e_2 is not atomic.
When e_2 has the form (e_3), then combine[e_1;e_2]=(e_1;e_3)
When e_2 is we have combine[e_1;e_2]=(e_1).
The first mention of the semicolon notation is the same place.
Thus that "definition" of combine uses a notational convention that
hasn't been defined previously. Now I can guess that
combine[e_1;e_2] means apply the operator combine to the two
parameters e_1 and e_2. But I have no idea whatsoever what the
notation (e_1;e_3) means. Where is *that* use of semicolon defined???
Also that "definition" of combine tries to say what combine does
when the second parameter is empty or single-element, but doesn't
say what combine does when the second parameter is a list of more
than one element. Can you find that part of the definition anywhere
in McCarthy's paper?
> (defun combine (a b)
> (check-type b cons)
> (cons a b))
Yeah, if Lisp is already implemented, *then* you can define combine
in terms of Lisp. But the major point of McCarthy's paper was to
define a subset of Lisp the very first time, when it didn't already
exist (and then to show that Lisp as defined this very first time
contained a universal function, namely APPLY, which can be used to
define a universal interpretor, namely EVAL).
(Yeah, I know you wrote that DEFUN to explain to *me* in 2008 what
McCarthy must have meant in that ancient paper he wrote. But I
can't find anywhere in that paper where he clearly expresses what
you claim he must have meant. So show me where McCarthy in that
paper defined combine in the general case where the second
parameter was a list with two or more elements! I don't see it.)
> 50 years ago, there was no personnal computer. Papers were
> written with a pen, by hand, on sheets of paper, and given to a
> secretary who would type it on a typewriter. Good luck if the
> secretary could decipher your handwritten squiggles, and good luck
> if she could transcribe it character by character with 100%
> accuracy.
Yeah, tell me!!! In 1966-67 I had finished my undergraduate
research project in differential algebra, wherein I demonstrated by
construction (determinant of almost-diagonal matrix) and consequent
proof (inversion of Newton's equations relating two standard types
of symmetric functions, namely the
all-combination-no-repeats set and the
single-powers-of-each-symbol set;
for example, with three symbols X,Y,Z,
the all-combinations functions are X+Y+Z XY+XZ+YZ XYZ
and the single-powers functions are X+Y+Z XX+YY+ZZ XXX+YYY+ZZZ)
that the nth matrix of my sequence (of size n by n) yielded a
differential polynomial whose principle differential ideal had
exponent n
(which means that for every differential polynomial d such that
there exists some k such that d^k is in the ideal, then in fact
d^n for that fixed n is in the ideal, taking higher powers never
helps you get something into the ideal that didn't already have
some lower power in the ideal).
This had previously been known only for n=1 (trivial) and n=2
(major result by Ritt that my undergraduate research advisor (*)
wisely presumed could be extended to larger n hence offering me
the task),
but I established it for all positive integers n.
* (A.P.Hillman, top-five Putnam winner long before, my advisor for
the first year of the undergraduate research project. Then he
moved to Albuquerque and David Mead took over as research
advisor. It was David Mead who showed me a book with Newton's
formulas relating symmetric functions, which was a key to
*proving* that my already-obtained result was in fact correct
for all n. He apparently recognized that the expresion for the
case of n=3 somewhat resembled Newton's relations, which hadn't
been noticed in the case n=2 because that case is so simple that
it could look like almost anything, not just Newton's relations.)
So next I needed to write up my paper to publish. First I did it by
hand. Then I needed to find somebody who could type it up. I was
now in graduate school at University of Oregon, and the math
department secretary was willing to do it. It came back months
later, in mid-1968, half on an IBM standard and the other half on
an IBM Selectric, which had grossly different typefaces making it
look like two independent works that had been accidently stapled
together, with hundreds of gross mistakes. Even after I manually
compared handwritten version with typed version to fix all the
mistakes I could find without any help from anyone, it was totally
unsuitable for making photocopies to send to journals.
That prompted me in 1969-70, out of graduate school (nervous
breakdown for about eight or ten major reasons, including being
re-classified from 2-F student deferrment to 1-A available for
draft to the Vietnam War) to use the IBM 1130 with CalComp plotter
with FORTRAN IV to write my first document-formatting program,
called Maas Research-Paper Plotting Program (MRPPP), then majorly
re-write/refactor it to do exponents and subscripts much better
(MRPP2), a lot of work with Hollerith cards and keypunch but I did
it!! It took a half hour to draw each page of text, and the CalComp
pen-locator mechanism had a lot of springiness that caused loopy
overshoots that produced characteristically weird looking text, but
it was 100% correct and in a way more legible than what the math
secretary at U.Ore. had done the year before.
Later, in 1995, Stanford A.I. got a Xerox Graphics Printer, and I
re-wrote the program in PDP-10 assembly language, adding lots of
wonderful new features including overlays
(which replaced the crufty up/down shift mechanism from MRPP2,
which AFAIK is *still* the cruddy way that Johnny-come-lately TeX
does math layout in running text)
and text-substitution macros that take internal-substitution
textual parameters
(very similar to how C does them, including allowing nesting of
another macro expansion inside the parameter passed to another
macro or even the same macro),
yielding MRPP3, which Whit Diffie insisted on renaming, so it came
to be known as "POX". With MRPP3 I finally could produce a truly
beautiful (and error-free) rendering of my differential algebra
paper. Bill Gosper used MRPP3/POX, with a large set of really good
macros he wrote himself, to produce some even more beautiful papers
reporting his results in continued fractions and summation of
hypergeometric series. Once there was a conference where everyone
submitted their papers in whatever form they had them, which were
then photocopied directly into the booklet of papers that was
distributed to participants. Bill showed me his copy of the
booklet. It was amusing looking through that booklet, seeing the
wide variety of crappy, and sometimes halfway decent, attempts to
lay out math on paper, some manually typed, some handprinted, some
using a standard Unix or PDP-10 utility such as RUNOFF or TROFF
etc., some using some SRI or BBN etc. text-layout technology that
I'd never seen before, with Bill Gosper's MRPP3/POX paper being by
far the most legible/beautiful/amazing paper of the lot! Can you
believe a continued fraction that fills half the page, running
diagonally all the way across, with total perfection in layout,
even better layout than what Springer-Verlag typically did in
textbooks (although lacking the high-quality typefont due to the
200 dots-per-inch resolution of the XGP and the non-kerning
fonts)??
Yeah, tell me about the pain of trying to get a very complex math
paper with matrices and deeply nested expressions inside summations
correctly typed up using pre-computer technology or even early
computer software prior to MRPP2/MRPP3. Even TeX which came later
probably couldn't do my paper correctly.
> But in this case, I don't think it's a typo. The glyph used for
> the "null expression" was also used in languages and grammars to
> denote the empty word. Nowadays, we usually use 'e' to denote the
> empty word. I think McCarthy was thinking of NIL as the empty word
> of languages, and thus could as well write (A,<E2>,B) than (A,,B)
> for #+cl(A NIL B).
Well (A,<E2>,B) would need some explaining, especially since
Latin-1 (iso-8859-1) hadn't been invented yet. Perhaps <null>
instead of <E2> would have made been easier to explain/define back
then.
I don't like using 'e' in McCarthy's context at all. Seeing an
obvious missing place between two words where an apparent
zero-length word was sitting there between two space characters was
enough to let me know something invisible or at least non-ASCII was
supposed to be there between the two space characters.
> And given that the unicode character <E2> wasn't available on the
> typewriter they had at the time, and that he would have had to put
> there a space and write with a pen the glyph, it seems very
> probable that he choose to use the empty word to represent the
> "null expression" in that context.
Did the typewriter have an underscore character available for such
use? In printed IBM manuals circa 1964 I used to see two different
glyphs for the space character, the lower-case b with diagonal
strike-through, and maybe also underscore with both edges bending
upward (or maybe the latter I saw somewhere else). It would make
sense to adapt the latter (serif underscore) to mean null
expression in McCarthy's context, making do with plain underscore
where serif underscore wasn't available, with instruction to the
publisher to replace plain underscore by serif underscore.
Anyway, the missing null-expression (serif underscore or empty box
or whatever) doesn't bother me as much as the use of semicolon in
places where that use hasn't been defined and I actually don't know
what it's supposed to mean.
> That's the difference between a mathematical or computer science
> _paper_ and a computer _program_. The former is interpreted by
> wetware and can easily contain typoes or even logical errors, that
> are automatically corrected by the wetware, while the later must be
> more formally correct to be accepted by the machine (be it
> implemented in software or in silicium), but cannot (yet?) be
> automatically corrected at the logical level.
Contrarywise, with computer programs it's much much easier to fix
all the gross typos because each of them causes a syntax error or
runtime exception (even ERROR 0006 IN LINE 274 from Fortran on IBM
1620 was better than no syntax error detected at all), so by the
time an algorithm gets published it's *correct*. By comparison,
typos in natural language (even computer/math/science jargon) can
slip by proofreaders because they know what the author really means
and are too exhausted to make a note of the mistake, then later
some *other* reader seeing the published paper is totally confused
because because he/she can't guess what the correction should have
been. That's the case here. I don't see any place where use of
semicolon within s-expression notation has been defined or even
*can* reasonably be defined.
> And any error would have been noticied (and have been noticed)
> very fast, given that Russel implemented this lisp very soon.
Didn't Russell ever talk with McCarthy face to face, and therefore
be privy to ideas that McCarthy expressed in multiple forms at
different times, whereupon Russell built up a sense of what
McCarthy really meant, such that the formal paper was *not*
necessary to tell Russell what he needed to implement, so Russell
could DWIM most times there was a typo on McCarthy's paper, and
personaly ask McCarthy to clarify (or just re-engineer from the
math needs) the few remaining times?
> > 5. rest[e]
> > ...
> > At this point, that's a totally silly name for that
> > function!!
Given your explanation that an expression can only be a *proper*
LIST of unlimited length, *never* a dotted pair with (non-null)
atom in CDR, I accept that rest makes sense. Thanks for explanation
(elsewhere) in your reply.
> > 6. combine[e_1;e_2]
> > combine[e_1;e_2] is defined when e_2 is not atomic.
> > When e_2 has the form (e_3), then combine[e_1;e_2]=(e_1;e_3)
> > ...
> > Huh??? This makes no sense at all. Semicolon isn't a
> > valid part of S-expressions. No explanation of using
> > semicolon as a meta-character for expressing
> > syntactical manipulations has been defined so-far in
> > this paper.
Ah, here's where I pointed out the semicolon problem in my original critique.
> Yes of course. Read again:
> (We shall use square brackets and semi-colons for writing
> functions of S-expressions since parentheses and commas have
> been pre-empted. ...)
You're missing my point.
I'm not complaining about the use of semicolon in
combine[e_1;e_2]
That's defined as function of S-expression taking two arguments.
I'm complaining about use of semicolon in
(e_1;e_3)
That makes no sense at all. It's neither notation for a function of
S-expressions nor an S-expression by itself.
S-expressions use commas, not semicolons.
Functions of S-expressions use square brackets, not parens.
It's gotta be a typo or scano,
maybe adaptive-OCR which correctly scanned a semicolon earlier on
that same line hence judged from context/adaption that an ambiguous
character later in the same line was probably also a semicolon,
or somebody (you?) made a mistake "fixing" a correct scan of (e_1,e_3).
> Unfortunately for you, the form (x;y) doesn't appear in the paper.
> It's either (x,y) or [x;y].
So you're calling me a liar? Here's an exact copy&paste of what
shows on-screen in that Web page, no editing whatsoever of these
lines of text:
6. combine[e_1;e_2]
combine[e_1;e_2] is defined when e_2 is not atomic.
When e_2 has the form (e_3), then combine[e_1;e_2]=(e_1;e_3)
When e_2 is we have combine[e_1;e_2]=(e_1).
Some examples are:
combine[A;]=(A)
combine[(A,B);(B,C)]=((A,B),B,C)
The funcntions first, rest and combine are related by the
relations
first[combine[e_1;e_2]]=e_1
rest[combine[e_1;e_2]]=e_2
combine[first[e];rest[e]]=e
whenever all the quantities involved are defined.
I see one (1) semicolon where you and I both agree a comma should
appear. All the other semicolons in that section are where they
belong, separating arguments inside square brackets.
(Also one misspelling: "funcntions". Is that typo in the hardcopy
that was scanned, or did the OCR studder?)
Now I'm going to print the entire WebPage to disk file, and 'more'
it, to show if any non-USASCII characters appear ... that section
is identical to what's on screen, no non-USASCII characters
whatsoever (as 'more' would show by <xx> notation) there.
At this point I got diverted and looked at the source of your Web page:
The top of your WebPage source shows:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
but doesn't show any charset declaration.
I know that your Apache server defaults to UTF-8 (some other
servers default to Latin-1, so your Web page isn't portable).
Let me ask the HTML-validation services what they think:
<http://validator.w3.org/>
Validate by URI
Validate a document online:
Address: http://www.informatimago.com/develop/lisp/small-cl-pgms/aim-8/aim-8.html
->
This page is not Valid HTML 4.01 Strict!
Result: Failed validation, 21 Errors
Encoding: utf-8
Doctype: HTML 4.01 Strict
Root Element: HTML
1. Error Line 41, Column 9: document type does not allow element
"BIG" here.
<PRE><BIG>
2. Error Line 42, Column 3: document type does not allow element "HR"
here; missing one of "MAP", "BUTTON" start-tag.
<HR>
<http://www.htmlhelp.com/tools/validator/>
URL: http://www.informatimago.com/develop/lisp/small-cl-pgms/aim-8/aim-8.html
->
* Last modified: Sun, 06 May 2007 17:45:26 GMT
* Character encoding: UTF-8
* Level of HTML: HTML 4.01 Strict
(same list of errors)
So I guess your WebPage has bigger problems than just failure to
declare an appropriate charset thus defaulting to UTF-8 on your
server and Latin-1 if you copy the page to another server.
Maybe you ought not declare your document as 4.01 Strict if you're
not going to strictly follow HTML 4.01 syntax??
> > Is it
> > possible that all uses of semicolon here are
> > typographic mistakes which should be comma instead??
> > I suspect the person who converted PDF to ASCII made a
> > goof here and needs to go back and proofread the entire
> > article to fix such mistakes.
> Perhaps you could read the PDF if you don't trust the converter.
> I would be very happy if you could signal me any correction.
AFAIK there doesn't exist any software for Macintosh System 7.5.5
to convert PDF to legible plain text.
> (And notice that it is not converted to ASCII, but to UNICODE).
It claims to be delivered in UTF-8, and indeed when I view it with
'more' I see what appear on first glance to be valid UTF-8 sequences:
"1<AF><B6>" and "T<AF><B6>" should be interchanged.
subsq=l[[x;y;z];[null[z]<AF><B6>;atom[z]<AF><B6>
[y=z<AF><B6>x;l<AF><B6>z];first[z]=QUOTE<AF><B6>z;l<AF><B6>
combine[subsq[x;y;first[z]];subsq[x;y;rest[x]]]]]
but I also see some non-USASCII codes which are *not* valid UTF-8,
such as:
tions has important advantages. Devices such as G<F6>del number-
l-notation. If <B0> is a functional expression and x_1,...,xn are
variables which may occur in <B0>, then l[[x_1,...,xn],] denotes
the function of n variables that maps x_1,...,xn into <B0>. For
f=l[[x_1,...,xn],<B0>] where the expression <B0> may contain the
That seems to be Latin-1 text that has been pasted into the
document without conversion to UTF-8 representation to match the
rest of the document and the default rendering on the Web. Let me
pass these snippets of 'more'-rendered text through my
'more'-unrendering function to recover the actual bytes of data
delivered by your Apache server, and then analyze the resultant
bytes according to UTF-8 and Latin-1 standards ...
(princ (nch-latin1-to-charstr-brac #xF6)) => {o"}
(princ (latin1-bytes-to-bigstr-with-braces (str-more-unrender-bytes "G<F6>del")))
=> G{o"}del (looks correct, so my assumption of Latin-1 was correct)
(mapcar #'(lambda (by) (format nil "<~2,'0X>" by))
(one-unicode-to-utf8bytes #xF6))
("<C3>" "<B6>")
(princ (nch-latin1-to-charstr-brac #xB0)) => {deg}
(princ (latin1-bytes-to-bigstr-with-braces (str-more-unrender-bytes "If <B0> is a functional expression")))
=> "If {deg} is a functional expression" (is degree symbol correct here?)
(mapcar #'(lambda (by) (format nil "<~2,'0X>" by))
(one-unicode-to-utf8bytes #xB0))
("<C2>" "<B0>")
<F6> in Latin-1 definitely needs to be converted to <C3><B6> in UTF-8.
<B0> in Latin-1 needs to be converted to maybe <C2><B0> in UTF-8.
By comparison, the places where there are two-byte non-USASCII
combos, which to me look (on visual inspection) like correct UTF-8:
(setq bytes (str-more-unrender-bytes "<AF><B6>"))
(princ (latin1-bytes-to-bigstr-with-braces bytes))
[Unknown Latin-1 character: ¯ {AF}]
<http://www.ramsch.org/martin/uni/fmi-hp/iso8859-1.html>
[Unknown Latin-1 character: ¶ {B6}]
<http://www.ramsch.org/martin/uni/fmi-hp/iso8859-1.html>atom[z]{AF}{B6}
macron accent / ¯ --> / ¯ --> /
paragraph sign 6 ¶ --> 6 ¶ --> 6
That pair makes no sense as Latin-1, so let's try UTF-8 next:
(setq ucodes (utf8-bytes-to-unicode-nums bytes))
Error in function UTF8BYTES-TO-ONE-UNICODE: First byte is 10xxxxxx
(mapcar #'(lambda (byte) (format nil "~8B" byte)) bytes)
("10101111" "10110110")
That's not valid UTF-8 either!! UTF-8 is supposed to be like this:
(text copied from RFC 2279, parens at right added by me)
0000 0000-0000 007F 0xxxxxxx (US-ASCII)
0000 0080-0000 07FF 110xxxxx 10xxxxxx (Includes the rest of Latin-1)
0000 0800-0000 FFFF 1110xxxx 10xxxxxx 10xxxxxx (Includes most of the rest)
0001 0000-001F FFFF 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
0020 0000-03FF FFFF 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
0400 0000-7FFF FFFF 1111110x 10xxxxxx ... 10xxxxxx
Last update : 2007-05-06 19:44:30 by : Pascal Bourguignon
Valid HTML 4.01!
Pascal didn't do a good job of creating a UTF-8 document.
(Or he took on maintenance of a legacy document without realizing
it was already trashed, or he somehow trashed a legacy document by
editing with the wrong text editor or somesuch.)
Parts were left as valid Latin-1 bytes, not converted to UTF-8 at
all, and other parts were converted to total garbage, namely
10xxxxx bytes without a leading 110xxxxx byte as required by UTF-8.
Does anybody know what character should be there in place of the
two bytes #b10101111 #b10110110 ?
Except for the US-ASCII bulk of text, none of the WebPage is valid
UTF-8, at least how it's delivered by the Apache server to me.
I assume somebody scanned the hardcopy to produce a PDF, then
somebody ran some program to convert the PDF to UniCode file,
then somebody (probably Pascal Bourguignon) edited the UniCode
file in an attempt to produce a UTF-8 WebPage. Is that correct?
Do you have access to the original UniCode file (converted from PDF
but not yet edited to be HTML)? Or do you have access to the PDF
plus access to the PDF-to-UniCode program to re-generate the
UniCode non-HTML file? If either, please 'more' the file on a VT100
terminal (the default you get with MS-Windows TELNET to a Unix
shell account) and see how the mathematician's name G{o"}del shows:
G<F6>del (correct Latin-1)
G<C3><B6>del (correct UTF-8)
Anything else??
My reply is getting long, and I'm really tired, and this a good
stopping point, so I'll suspend my reply now. Whether I'll have any
more to say later, in response to later parts of your article I'm
responding to, I don't yet know, so this could be a <split> in my
reply or the <veryEnd> of my reply.
Pascal J. Bourguignon
>> Date: Wed, 25 Jun 2008 23:09:35 +0200
> Why this response is so belated:
> <http://groups.google.com/group/misc.misc/msg/cea714440e591dd2>
> = <news:rem-2008...@yahoo.com>
>> From: p...@informatimago.com (Pascal J. Bourguignon)
>> >> There's no problem with circular definitions. You just need a
>> >> bootstrap procedure.
>> > You've contradicted yourself. The whole point of a bootstrap
>> > procedure is to *avoid* having an actual circular definition.
>> The whole point of a bootstrap procedure is to *implement* an
>> actual circular definition. The point of bootstrapping a compiler
>> is exactly to reach the fixed point.
>
> Can we agree on this re-wording: The point of bootstrapping (a
> compiler) is to provide a non-circular entry point into what would
> otherwise be a circular definition (a "chicken-egg" conundrum). At
> the entry point to the fixed-loop, there are two ways to reach that
> point, one of which is to enter via the bootstrap path, which
> typically comes from an earlier fixed-loop defined by an earlier
> version (of the compiler), and the other way to reach that point is
> to go once around the fixed-loop from itself. Without the bootstrap
> entry point to that fixed-loop, there's no way to ever have the
> fixed-loop in the first place.
Yes.
> (... AIM-8 ...)
>
>> >> Then you can implement that sheet of paper in assembler by hand,
>> >> <bootstrap> and the rest is history.
>> > Yes, you absolutely need that assembly code to build the executable
>> > that you can then use to emulate the L-calculus and thence feed in
>> > McCarthy's source code for defining a Lisp interpretor. But then
>> > you also need what I said above, some Lisp primitives for binary
>> > file I/O, so that you can actually compile an executable from
>> > S-expression source, and ideally also with LAP and SYSLISP to make
>> > the process scrutable (the opposite of inscrutable chicken
>> > scratches).
>> No, you don't absolutely need that assembly. You could as well
>> ask an electronician to build hardware that would directly execute
>> the lisp code.
>
> Yeah, if you had a thousand times as much money and nothing better
> to do with it. I stand technically corrected but feasibly correct.
>
>> ... And think about it, in your computer, I/O is not done in
>> "binary". It is done in transistors and electrons.
>
> The intentional type of all that TTL or cMOS etc. technology is
> binary. Whenever the hardware fails to correctly implement that
> intentional type, the computer stops working.
For example, the soviets didn't line binary because it was invented by
capitalist dogs. So they built a computer that worked in ternary.
+12V, ~0V, -12V. http://www.computer-museum.ru/english/setun.htm
Now, if you think about it, building lambda calculus hardware is not
so hard. After all there are only four notions to implement and some
microcode to manage the memory.
>> It just happens that once you have a computer it's faster and
>> cheaper to implement your lambda calculus virtual machine, or your
>> lisp, programming this computer than soldering electronic
>> components. Just one student could do it in a few months, while
>> it would have been a major project if it would have been done with
>> tubes.
>
> Tubes??? Did the IBM 704 use tubes??? I just assumed it used
> discrete transistors (and resistors, capacitors, etc.) on either
> wire-wrap or printed-circuit boards, just like the IBM 1620 did (I
> actually looked at the innerds once when it was being repaired or
> demonstrated or somesuch), and the IBM 1401 too probably.
No. The original 704 on which McCarthy and his team started
implementing LISP was a tube computer. Later they had a 709 and then
a 7090 which were transistor based.
> [...]
> So has anybody actually written an assembler in Common Lisp,
> something that can convert expressions of the form
Of course. Almost all lisp implementations have a LAP. Well, let's
discard ecls and gcl, since their LAP is C itself.
SBCL has VOPs http://sbcl-internals.cliki.net/VOP
OpenMCL (MCL) has a nice LAP, and uses almost no C code.
clisp has a nice virtual machine. The LAP API is not public, but it's
there and we've got the source.
And again, this is something that exist in LISP since day 1, so to
speak, at least in LISP 1.5:
* LAP IS THE ASSEMBLER. ONE ARG IS LISTING. IT IS LIST OF INSTRUC-
* TIONS, NON-ATOMIC OR NIL. THE ATOMIC SYMBOLS ARE LOCATION SYMBOLS
* SECOND ARG IS START OF SYMBIL TABLE WHICH IS AN A-LIST.
* THE FIRST ITEM IS ORG AS FOLLOWS-
* NIL= IN BPS
* ATOM= AT SYMBOLIC LOCATION
* NUM= ATHIS NUMBER
* (NAME TYPE NUM) = IN BPS, AND PUT TXL ON PROP LIST OF NAME
* WITH FLAG TYPE AND NUM (B DEC. OF TXL.
* INSTRUCTION FORMAT IS (OP ADDR TAG DEC)
* FIELD FORMAT IS AS FOLLOWS-
* TEMP SYMBOL
* NUMBER
* SYM SUBR OR FSUBR
* (E NAME) FOR IMMEDIATE AS IN TXL FILTER
* (QUOTE NAME) FOR IMTE IN DEC OF WORD ON QTLST
* POINTER TO COMMON WORD.MAKES ONE IF NONE ALREADY
* SUM OF ANY OF ABOVE
* LAP IS IDENTITY FUNCTION
* LAP DOES NOT USE IX1. IX2,4 ARE SCARTCH
* ERRORS IN LAP AS FOLLOWS-
* *L 1* UNABLE TO EVALUATE ORIGIN
* *L 2* OUT OF BPS DISCOVERED AFTER PASS 1
* *L 3* UNDEFINED SYMBOL
* *L 4* FIELD WAS RECURSIVE
*
>> Read the definition of combine and you should be able to
>> understand that (a,b,c) is a list of three elements, and (a,b) a
>> list of two. And since combine is not defined with a second
>> argument being an atom, it's a CONS function used only to build
>> lists.
>
> The first mention of combine (except in the erratum added at the top) is here:
> 6. combine[e_1;e_2]
> combine[e_1;e_2] is defined when e_2 is not atomic.
> When e_2 has the form (e_3), then combine[e_1;e_2]=(e_1;e_3)
> When e_2 is we have combine[e_1;e_2]=(e_1).
> The first mention of the semicolon notation is the same place.
> Thus that "definition" of combine uses a notational convention that
> hasn't been defined previously. Now I can guess that
> combine[e_1;e_2] means apply the operator combine to the two
> parameters e_1 and e_2. But I have no idea whatsoever what the
> notation (e_1;e_3) means. Where is *that* use of semicolon defined???
The point 6 defines it. It's pattern matching. The notation is
defined at the same time as the notion, with some natural pattern
matching implemented in the wetware of the mathematician who reads the
paper.
This is exactly how it is done in prolog. You define new operators by
matching some compound form and associating it with some other
compound form that is rewritten with substitution of the unified
variables.
> Also that "definition" of combine tries to say what combine does
> when the second parameter is empty or single-element, but doesn't
> say what combine does when the second parameter is a list of more
> than one element. Can you find that part of the definition anywhere
> in McCarthy's paper?
Yes, the second clause says it. You missed the unicode character for
NIL:
> When e_2 is we have combine[e_1;e_2]=(e_1).
^
|
+--- here was the symbol for NIL.
I don't know. Perhaps using more linefeeds would make the texts you
edit clearer and let you better read them?
>> (defun combine (a b)
>> (check-type b cons)
>> (cons a b))
>
> Yeah, if Lisp is already implemented, *then* you can define combine
> in terms of Lisp. But the major point of McCarthy's paper was to
> define a subset of Lisp the very first time, when it didn't already
> exist (and then to show that Lisp as defined this very first time
> contained a universal function, namely APPLY, which can be used to
> define a universal interpretor, namely EVAL).
>
> (Yeah, I know you wrote that DEFUN to explain to *me* in 2008 what
> McCarthy must have meant in that ancient paper he wrote. But I
> can't find anywhere in that paper where he clearly expresses what
> you claim he must have meant. So show me where McCarthy in that
> paper defined combine in the general case where the second
> parameter was a list with two or more elements! I don't see it.)
There's no need for e₃ to be a S-expression. It can be formally a
sequence of S-expression. What matters is that (e₃) be a S-expression,
and by rule 4, we know that we can have lits of any number of
elements:
4. If e₁ and (e₂) are S-expressions so is (e₁,e₂).
(e₃) = (e₄,e₅) <=> e₃ = e₄,e₅
This is formally true, without any condition on what e₃ is. Itcan be a
S-expression or not it doesn't matter. Actually it is not a
S-expression. But for the following we only consider whether (e₃) is
a S-expression.
e₁ and (e₃) are S-expressions => (e₁,e₃) is a S-expression
<=> (e₁,e₄,e₅) is a S-expression.
and so on. You can do the recursion yourself...
[0] 6. combine[e₁;e₂]
[1] combine[e₁;e₂] is defined when e₂ is not atomic.
[2] When e₂ has the form (e₃), then combine[e₁;e₂]=(e₁;e₃)
There's a typo here, there must be a comma in the last parentheses:
[2'] When e₂ has the form (e₃), then combine[e₁;e₂]=(e₁,e₃)
[3] When e₂ is ⋀ we have combine[e₁;e₂]=(e₁).
[0] translates to (defun combine (e1 e2)
[1] translates to (check-type e2 cons) ; given that (not atom) === cons
(cond
[2'] translates to ((not (null e2)) (cons e1 e2))
[3] translates to ((null e2) (cons e1 nil))))
and this cond is exactly what (CONS e1 e2) does.
> Did the typewriter have an underscore character available for such
> use?
If you read the AIM-8 paper scanned into PDF or PS, you will see that
these symbols were added manually (as well as all the square brackets
and the arrows).
> Didn't Russell ever talk with McCarthy face to face, and therefore
> be privy to ideas that McCarthy expressed in multiple forms at
> different times, whereupon Russell built up a sense of what
> McCarthy really meant, such that the formal paper was *not*
> necessary to tell Russell what he needed to implement, so Russell
> could DWIM most times there was a typo on McCarthy's paper, and
> personaly ask McCarthy to clarify (or just re-engineer from the
> math needs) the few remaining times?
Of course. But intelligence is usually enough to make sense of papers
with typoes, and even with gross logical errors.
> Ah, here's where I pointed out the semicolon problem in my original critique.
>
>> Yes of course. Read again:
>> (We shall use square brackets and semi-colons for writing
>> functions of S-expressions since parentheses and commas have
>> been pre-empted. ...)
>
> You're missing my point.
Yes, indeed I did. The semicolon in the parentheses was a typo of
mine, it doesn't appear in the PDF or PS of the paper. I'll have to
correct my web page.
> I'm not complaining about the use of semicolon in
> combine[e_1;e_2]
> That's defined as function of S-expression taking two arguments.
> I'm complaining about use of semicolon in
> (e_1;e_3)
> That makes no sense at all. It's neither notation for a function of
> S-expressions nor an S-expression by itself.
> S-expressions use commas, not semicolons.
Yes.
> Functions of S-expressions use square brackets, not parens.
> It's gotta be a typo or scano,
> maybe adaptive-OCR which correctly scanned a semicolon earlier on
> that same line hence judged from context/adaption that an ambiguous
> character later in the same line was probably also a semicolon,
> or somebody (you?) made a mistake "fixing" a correct scan of (e_1,e_3).
I did.
>> Unfortunately for you, the form (x;y) doesn't appear in the paper.
>> It's either (x,y) or [x;y].
>
> So you're calling me a liar?
Sorry. I must have been watching the PDF when I wrote that sentence.
> This page is not Valid HTML 4.01 Strict!
> [...]
> So I guess your WebPage has bigger problems than just failure to
> declare an appropriate charset thus defaulting to UTF-8 on your
> server and Latin-1 if you copy the page to another server.
> Maybe you ought not declare your document as 4.01 Strict if you're
> not going to strictly follow HTML 4.01 syntax??
I intend to strictly follow it one day.
> [...]
> but I also see some non-USASCII codes which are *not* valid UTF-8,
> such as:
>
> tions has important advantages. Devices such as G<F6>del number-
No, it must be your how your software renders it. The document
contains two bytes betwee the 'G' and the 'd'. Notice that unicode
is a superset of ISO-8859-1. The codes from 0 to 255 encode the same
characters in unicode as in iso-8859-1.
> Pascal didn't do a good job of creating a UTF-8 document.
It's emacs who did the job of encoding the unicode text into utf-8.
And I thrust emacs to do a good job here.
> I assume somebody scanned the hardcopy to produce a PDF, then
> somebody ran some program to convert the PDF to UniCode file,
> then somebody (probably Pascal Bourguignon) edited the UniCode
> file in an attempt to produce a UTF-8 WebPage. Is that correct?
No. I read the PDF and typed in the unicode in emacs.
> Do you have access to the original UniCode file (converted from PDF
> but not yet edited to be HTML)?
Yes, it is linked form the web page.
Or do you have access to the PDF
> plus access to the PDF-to-UniCode program to re-generate the
> UniCode non-HTML file? If either, please 'more' the file on a VT100
> terminal (the default you get with MS-Windows TELNET to a Unix
> shell account) and see how the mathematician's name G{o"}del shows:
> G<F6>del (correct Latin-1)
> G<C3><B6>del (correct UTF-8)
> Anything else??
The source is ok.
od -t x1 -a aim-8.html
...
v a n t a g e s . sp sp D e v i c
0013700 65 73 20 73 75 63 68 20 61 73 20 47 c3 b6 64 65
e s sp s u c h sp a s sp G C 6 d e
0013720 6c 20 6e 75 6d 62 65 72 2d 0a 69 6e 67 20 61 72
l sp n u m b e r - nl i n g sp a r
...
and when I fetch it from the web server it's bit for bit identical.
So I'd say the problem is how your software handles it. Notice that
the encoding is specified in the html header as a meta content-type.
[pjb@hubble aim-8]$ head -20 aim-8.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<HTML>
<HEAD>
<link rel="icon" href="/favicon.ico" type="image/x-icon">
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
<link rel="stylesheet" href="default.css" type="text/css">
<TITLE>RECURSIVE FUNCTIONS OF SYMBOLIC EXPRESSIONS
AND THEIR COMPUTATION BY MACHINE</TITLE>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=utf-8">
<META HTTP-EQUIV="Description"
NAME="description" CONTENT="AIM-8">
<META NAME="author" CONTENT="John McCarthy">
<META NAME="keywords" CONTENT="LISP, AIM-8, AI Memo-8, John McCarthy,recursive function, symbolic expression, computation, turing machine, lambda calculus, computer history, programming language history">
</HEAD>
--
__Pascal Bourguignon__ http://www.informatimago.com/
NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.
Vassil Nikolov
On Wed, 09 Jul 2008 12:20:28 -0700, jaycx2.3....@spamgourmet.com.remove (Robert Maas, http://tinyurl.com/uh3t) said:
| ...
| But still if we use C every time we build Lisp, the C lovers will
| continue claiming that C is essential and everyone needs to use C
| and Lisp is garbage because it can't even compile itself.
As an aside, it seems that a good candidate for the intermediate
step on the shortest reasonable path from the bare metal to Lisp is
Forth. (Well, this is a red herring, rather...)
| ...
| I don't like using 'e' in McCarthy's context at all.
For what it's worth, the _LISP 1.5 Programmer's Manual_ uses a
straight epsilon (if that's the name for it) for an expression (both
in M-expressions and in S-expressions).
| ...
| underscore with both edges bending
| upward (or maybe the latter I saw somewhere else).
Algol (?) contexts.
---Vassil.
--
Peius melius est. ---Ricardus Gabriel.
Juanjo
wrote:
> Of course. Almost all lisp implementations have a LAP. Well, let's
ECL uses both C and a special purpose bytecodes which are processed by
a threaded interpreter written in C. So in that respect it is not too
different from CLISP.
Juanjo
Bob Felts
> http://www.computer-museum.ru/english/setun.htm
This doesn't lead to an English link and I've only had a few months of
Russian language instruction, and that was long ago. Was this the right
URL?
Pascal J. Bourguignon
It was. But today it redirects somewhere else.
In these situations, use web.archive.org:
http://web.archive.org/web/20070310124240/http://www.computer-museum.ru/english/setun.htm
--
__Pascal Bourguignon__ http://www.informatimago.com/
Our enemies are innovative and resourceful, and so are we. They never
stop thinking about new ways to harm our country and our people, and
neither do we. -- Georges W. Bush
Bob Felts
> wr...@stablecross.com (Bob Felts) writes:
>
> > Pascal J. Bourguignon <p...@informatimago.com> wrote:
> >
> >> http://www.computer-museum.ru/english/setun.htm
> >
> > This doesn't lead to an English link and I've only had a few months of
> > Russian language instruction, and that was long ago. Was this the right
> > URL?
>
> It was. But today it redirects somewhere else.
>
> In these situations, use web.archive.org:
> http://web.archive.org/web/20070310124240/http://www.computer-museum.ru/
> english/setun.htm
Spasiba, tovarisch.
Robert Maas, http://tinyurl.com/uh3t
> > compiler) is to provide a non-circular entry point into what would
> > otherwise be a circular definition (a "chicken-egg" conundrum). At
> > the entry point to the fixed-loop, there are two ways to reach that
> > point, one of which is to enter via the bootstrap path, which
> > typically comes from an earlier fixed-loop defined by an earlier
> > version (of the compiler), and the other way to reach that point is
> > to go once around the fixed-loop from itself. Without the bootstrap
> > entry point to that fixed-loop, there's no way to ever have the
> > fixed-loop in the first place.
I never thought of looking for "bootstrapping" per se in WikiPedia
until I saw your agreement on the wording and got the idea that
there ought to be a WikiPedia page on this topic with my wording.
So I checked and found a whole bunch of related pages, two in
particular relevant to our topic:
* Bootstrapping (computing), the process of a simple system
activating a more complicated system that serves the same purpose
* Bootstrapping (compilers), writing a compiler for a computer
language using the language itself
The latter is of course our specific topic, but I'm going to browse
the former, and also another I didn't list here before proceeding ...
Making HTTP connection to en.wikipedia.org (hanging indefinitely)
WikiPedia seems to have gone down!!
Several minutes later it's finally back up ...
OK, done reading the unrelated topic, now reading the first above,
which describes for example how I set up multiple levels of
bootstrap loader on my Altair, the first entered from front panel,
the second entered from terminal using 3n+1 notation, the third
entered from terminal using hexadecimail notation, the fourth
downloaded from a PDP-10 over a modem, whereupon smart-download of
all my software could then occur efficiently and error-protected.
Fascinating example of hypothetical paper-tape bootstrap loader.
OK, done reading that former article, now on to *this* topic ...
Nice explanation of the "chicken and egg" problem involved in
writing a compiler for a language in its own language.
Nice listing of various ways to solve that "chicken and egg"
problem (implementing first compiler in an earlier language, using
subset of language, cross-compilation, etc.), but so far nothing
like my global definition of what's really going on from the "fixed
point loop" point of view.
"Today, a large proportion of programming languages are
bootstrapped, including Basic, C, Pascal, Factor, Haskell,
Modula-2, Oberon, OCaml, Common Lisp, Scheme ..."
Link to "self-hosting" which has this nice text:
If a system is so new that no software has been written for it, then
software is developed on another self-hosting system and placed on a
storage device that the new system can read. Development continues
this way until the new system can reliably host its own development.
Development of the Linux operating system, for example, was initially
hosted on a Minix system.
Note that having Common Lisp "temporarily dependent on C" would
follow that course. Use C to define a subset of Common Lisp that
provides BootLAP and regular LAP and perhaps SYSLISP, and is
capable of self-hosting. At that point, there's no longer any need
for direct use of C. But still there's a philosophical point
involved if C was necessary to build the first version of Common
Lisp along this bootstrapping path. Granting that caveat, that it's
OK to involve C in the "past" if Common Lisp can eventually be
self-hosting, then we might as well take the *current* C-kernel
implementations of Common Lisp, define BootLAP and LAP and SysLisp
from within it, write code to use that system to build a Common
Lisp core to replace our current Common Lisp core, use Lisp source
to build that back up to current CL capability, at which point we
have a self-hosting Common Lisp which needs no further work done by C.
But is that good enough, to have Common Lisp not only dependent on
C all the way up to 2008 and then suddenly install BootLAP and LAP
and SYSLISP and eventually in 2009 make Common Lisp self-hosting,
but still CL is dependent on operating systems such as
Unix/Linux/Windows where every new version of the OS and every
patch to protect against viruses/worms/trojans is written in C and
compiled with a C compiler?? In a sense a symbiosis between C and
the OS is the only thing that's really self-hosting, and everything
else including Common Lisp is dependent on C+OS.
I just thought it'd be nice to have a CL+OS symbiosis that
was truly self-hosting, with no dependency on C whatsoever:
- *ever* (bare-machine boot to DDT to device drivers to SubLisp)
- after some point in time (cross-assemble the CL+OS from current
C+OS hosted CL, and *then* finally divorce CL+OS from C)
The Lisp Machines did that, but I'm thinking of CL+OS self-hosting
on modern Intel processors which are so common nowadays. Imagine
you start up your laptop and it says BIOS then it says LIFO boot
then it says LispOS, and everything after that point is all LispOS.
(Sorry, can't call it CLOS meaning Common Lisp Operating System
because that acronym is already taken by the OOP stuff.)
Continuation of that WikiPedia quote above:
Writing new software development tools "from
the metal" (that is, without using another host system) is rare and in
many cases impossible.
Is it *really* **really** impossible? Don't the engineers who
produce the new CPU chips at Intel Corp. have a physical test rig
where they use an embedded system on each such test board to drive
test signals into the CPU and log its response to determine whether
it's working, for the first prototypes and then again for a
statistical sample of production-line output? Couldn't a slight
variation on that test board be rigged with a "front panel" for
driving signals into the CPU which reset and freeze the CPU then
artificially load a few instructions into address zero and
following then set the CPU address to zero and start executing? And
of course have a UART or ACIA mounted as a device on that test
board, accessible via memory-mapped I/O (two memory bytes which
when written give CONTROL and DATA-OUT and when read give STATUS
and DATA-IN), with some terminal or emulator thereof hooked onto
the RS-232 port? Yeah, it involves some more soldering or
wire-wrapping to the board, beyond what Intel needs for development
and quality control, but isn't it *possible* for some employee at
Intel to take it upon him/herself to use his/her own time to hack
one of the test boards to provide an Altair-like
front-panel-plus-RS232-bootable CPU+RAM? Maybe not likely Intel
would allow their test boards to leak out to "hackers" (in the
MIT/SU sense) to play with, but still *possible*, not *impossible*
as the WikiPedia page claims?
Several programming languages are self-hosting, in the sense that a
compiler for the language, written in the same language, is available.
The first compiler for a new programming language can be written in
another language (in rare cases, machine language) or produced using
bootstrapping. Self-hosting languages include Lisp, Forth, Pascal,
Delphi, C, Modula-2, Oberon, Smalltalk, OCaml, and FreeBASIC.
That section of the WikiPedia page is ambiguous per our current
discussion. Does each of those named languages *really* host itself
by directly producing an executable for the next edition of itself,
or do they not really host themselves because all they generate is
C which must then be compiled by a C compiler to finally produce
the next edition of the named language? (And of course it doesn't
say *which* dialect of Lisp it's making the ambiguous claim about,
so that part is doubly ambiguous.) Is there somebody reading this
thread who can disambiguate for one or more of those named
languages, whether it directly produces executable, or produces
assembly code which requires assembler, or produces C source which
needs to be compiled thereby requiring C compiler?
The first self-hosting compiler (excluding assemblers) was written for
Lisp by Hart and Levin at MIT in 1962. Because Lisp interpreters
existed previously, but no Lisp compilers, they used an original
method to compile their compiler. The compiler, like any other Lisp
program, could be run in a Lisp interpreter. So they simply ran the
compiler in the interpreter, giving it its own source code to
compile.
It would be nice to know whether the interpreted Lisp compiler
produced object code in executable form directly, or produced
relocatable module which then needed to be LOADed then SAVEd to
produce an executable (or on IBM mainframes LINK-EDITed to directly
produce an executable), or assembly code which needed to be
assembled then LoadSaved/LinkEdited. Given the various flavors of
bootstrapping and self-hosting etc., I think I'd like to collect
all these specifics via the newsgroup then edit the WikiPedia
articles to have links to footnotes containing the specifics.
<ftp://publications.ai.mit.edu/ai-publications/pdf/AIM-039.pdf>
I have no way to read PDF here, so I can't look up the answers myself.
<http://en.wikipedia.org/wiki/Self-interpreter>
A self-interpreter is a programming language interpreter written in
the language it interprets. ...
Very good (succinct) statement in next paragraph:
This may sound paradoxical; how can one implement a language in that
language if it doesn't yet exist? However there is little mystery
here. Interpreters are always "mocked up" in some other existing
language and then later converted to the language they interpret. In
these cases the early mockups can be used to develop the source code
of the interpreter. Once the system is bootstrapped new versions of
the interpreter can be developed in the language itself.
My father used to work at Lockheed in Burbank, where his job was
machine work to produce "mock up" stuff, so I understand this jargon!
For newbies to the jargon, it would be nice to have a link to the
definition of that term. Maybe I'll do it some other time when
I'm not already in the middle of something ... yeah, sure, that'll happen.
Lots of good stuff in that WikiPedia article, including:
There are some languages that have a particularly nice and elegant
self-interpreter, such as Lisp or Prolog. ...
Ah, I didn't know this jargon term until I followed the link:
<http://en.wikipedia.org/wiki/Homoiconic>
One of the essential natures of Lisp (another current thread) is
that it's "homoiconic" as defined here. Great stuff in this
WikiPedia article, except for this libel:
Their only great drawback is that programs
written in them look like King Burniburiach's letter to the Sumerians
done in Babylonian cuniform! [...]
I take great exception to that remark! APL may look like cuniform,
but not well-written pretty-printed Common Lisp. In a way it's more
readable than C or most other languages, except for arithmetic
expressions where infix notation really is sometimes easier for
humans to parse. And if anybody really wanted to make that "fix" to
Lisp it'd be simple with a parse-infix macro:
(parse-infix "(-b + sqrt(b**2 - 4*a*c)) / (2*a)")
==> (/ (+ (- b) (sqrt (- (expt b 2) (* 4 a c)))) (* 2 a))
Has anybody actually written that macro and put it to good use?
Languages which are considered to be homoiconic include most members
of the Lisp family, Prolog, Smalltalk, Curl, REBOL, SNOBOL, XSLT,
TRAC, Tcl, Io, Joy, Factor, Clojure, PostScript and V.
In Von Neumann architecture systems (including the vast majority of
systems today), raw machine code also has this property, the data type
being bytes in memory.
Hey, that's exactly what I said in recent weeks/months, except I
didn't know the jargon word "homoiconic", and I am not intimately
familiar with those other languages mentionned so I didn't include
them. I stated that Lisp and machine language are the only
languages I know of where the usual representation of the program
is exactly the same as the usual representation of data processed
by the program, allowing program to manipulate program without
additional hassle. Quoting from what I said before:
"the question as to whether code is data (the essential presumption
of Lisp and machine language programming)"
= <news:rem-2008...@yahoo.com>
"assembly/machine language and Lisp are the only two languages that
take full advantage of this architecture where program equals data
in regard to ability to build it and modify it in useful ways."
= <news:rem-2008...@yahoo.com>
Can somebody confirm, or refute, that the claim I made on Jun.30
about build/modify program in useful ways would apply equally to
those other (unfamiliar) languages listed in the WikiPedia article?
I know that for a subset of Lisp that doesn't have read-time
evaluation or macros or readtables or packages etc. or
keyword/optional arguments, a formal definition of EVAL in terms of
APPLY can be accomplished in less than one page of source code, and
a definition of APPLY for a particular architecture (for example
using association lists to hold all variable bindings, including
function bindings, in a Scheme-like LISP1 way) is likewise less
than one page of source code, assuming the McCarthy primitives for
Lambda calculus are pre-defined (READ PRINT CONS CAR CDR etc.), and
an additional page of code for definitions of ASSOC etc. are also
available. The REP is then one additional line of code. To what
degree can those other listed languages likewise self-express their
essential interpretor and REP loop in just a few pages of sourcecode?
<http://en.wikipedia.org/wiki/SICP>
The program also introduces a practical implementation of the register
machine concept, defining and developing an assembler for such a
construct, which is used as a virtual machine for the implementation
of interpreters and compilers in the book, ...
I looked at the HTML text regarding that:
<http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-4.html#%_toc_start>
and I see it's all abstract within Scheme, not anything like
Knuth's MIX which has realistic machine-language instruction
formats, so I guess I won't use that if and when I implement a
bare-machine-with-front-panel emulator for people to use to
experiment with bootstrapping from a bare machine.
<http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html>
When I was a kid, I learned to program on punched cards. If you made a
mistake, you didn't have any of these modern features like a backspace
key to correct it. You threw away the card and started over.
What a lie! The IBM 026 keypunch had a duplicate-card feature. If
you made a mistake, you could try to duplicate up to the mistake,
manually re-key the correction, then duplicate the rest of the
card. Even insert/delete was possible by manually holding one of
the two cards (original or in-progress copy) so that it wouldn't
move while the other did.
Think you have what it takes? Test Yourself Here!
-> <http://www.joelonsoftware.com/articles/TestYourself.html>
Those were easy. ACCUMULATE is essentially REDUCE but with an
explicit null return value added to avoid the need to special case
on short lists. The rest was trivial in grasp-eye mode.
<http://www.paulgraham.com/avg.html>
(regarding viaweb in 1995:)
It was one of the
first big end-user applications to be written in Lisp, which up till
then had been used mostly in universities and research labs.
I dispute that! At that time I had been writing quite a lot of
*useful* personal software in Lisp, some rather significant in size
(such as my file-maintenance utilities which included a
general-purpose sort-files-by-multiway-merging utility, and the
Lisp implementation of my effective flashcard drill algorithm),
much of unrelated to any university or research lab (except that I
got my copy of MACL as a benefit when I had been employed at
Stanford IMSSS). I claim my file-maintenance system pre-dated
viaweb by years and my flashcard program written in 1995.Jan
probably predated viaweb by a few months. I suspect some people
with Lisp machines at home (Symbolics or other) might have done
some significant Lisp programming for personal use also. I'd be
curious to know if anyone else claims to have written large
personal software systems using Lisp prior to 1995. I count myself,
and my two children, as three end users, OK?
Back in 1995, we knew something that I don't think our competitors
understood, and few understand even now: when you're writing software
that only has to run on your own servers, you can use any language you
want.
Excellent point, exactly one of the two reasons why I liked CGI so
well when I discovered it was available on the new ISP that I had
switched to a few months earlier in mid-2000. (The other reason is
that I can make a demo of my software available from *anywhere*
there is Web access worldwide at no additional cost/effort to me.)
(Unfortunately it's awfully difficult to get anyone to try my
demos, because I'm not a well-established company such as Google
which can put an advertisement for their new services on their
existing services, such as their link to Images on their search
engine main page, and their link to the tagging-images conest on
their Images main page. Short of spamming, does anybody have any
good ideas how to get people to play with my demos of CGI
applications and then get into dialog with me about what
improvements they can suggest?)
P.S. My very first CGI demo (2001.Jan) is here:
<http://shell.rawbw.com/~rem/cgi-bin/topscript.cgi>
... Lisp code, after it's read
by the parser, is made of data structures that you can traverse.
If you understand how compilers work, what's really going on is not so
much that Lisp has a strange syntax as that Lisp has no syntax. You
write programs in the parse trees that get generated within the
compiler when other languages are parsed. But these parse trees are
fully accessible to your programs. You can write programs that
manipulate them. ...
That's a good expression of the point I made about Lisp and machine
language, where in fact Lisp is the only language that treats
program as the same as data and *also* has these be pointy
structures that can be traversed in a nested/recursive way instead
only blocks of consecutive fixed-size numbers in an a giant array
called RAM. I would make only one change to the wording: Lisp
*does* have syntax, namely the minimal syntax necessary to group
levels of nested list and to separate elements within a single
level, namely parens and whitespace. This is the mininum syntax
needed to read and print parse trees in a comprehensible notation.
Forth is the language that truly has NO syntax, merely a tokenizer
that separates words from whitespace, and the REP is simply a loop
that executes each input word in sequence. Now in Forth you could
*hack* s-expression notation by having ( be a word that started
collecting elements of a list and ) be a keyword that stopped such
collecting, but then parens would need whitespace around each, and
you'd have to write:
( define ( accumulate combiner null-value l )
( if ( null? l )
null-value
( combiner ( car l )
( accumulate combiner
null-value
( cdr l ) ) ) ) )
If lisp-haters don't like the parens, imagine needing white space
around each paren too!! You don't know when you got it good!
(But, even with that annoying white-space, it still seems vaguely elegant!)
(And the screen-readers for the blind would probably read that
better than they read regular Lisp source, so maybe I should start
formatting my code with that extra whitespace to make it accessible
to both Forth-greenspun-Lisp and screenreaders-for-the-blind?)
For the past several weeks/months, more and more I've been thinking
that intentional types are the correct way to think about this sort
of stuff, and maybe my HelloPlus CookBook/Matrix needs to be
re-organized to emphasize intentional types instead of built-in
types. But now as I read through this "Beating the Averages" essay,
I'm thinking that intentional types carried to the *extreme* are
actually the best way to teach/learn computer science/programming,
from a bootstrap level that is even more primitive than McCarthy's
original Lisp (implementation of lambda calculus) proposal. Start
with just one primitive data type, the boolean value, i.e. the bit
(false or true), and one container type, the tuple (an ordered list
of bits and/or nested tuples). Then build up intentional types on
top of those. For example:
- a byte (octet) is a tuple (of size 8) of bits, with no additional
intentional information;
- an integer is a tuple (of arbitrary size) of bits, with the
intention that various arithmetic operations treat them as if
twos-complement notation;
- an arithmetic byte, i.e. datatype 'byte' in C, is a byte (octet)
with the addition of the arithmetic semantics, i.e. it's an
integer that happens to be of length 8;
- a 32-bit-word is a tuple (of size 4) of arithmetic bytes, with
the additional intention that the bytes are arranged
little-to-big or big-to-little depending on the architecture.
- a CONS cell is a tuple of two objects.
For run-type dispatching, you need a type-tag accompanying each
boxed object, which is probably best implemented as a tuple of
arithmetic bytes using UTF-8 notation. Then a CONS cell is a tuple
consisting of three objects, the type-tag for "CONS", and the CAR
and CDR cells. (What, you say, having the intrepretor compare four
8-bit charaacters to make sure they match "CONS" every time a CONS
cell is accessed is too much overhead? But what if these type-tags
were canonicalized in a table, so you just do an EQ comparison
between the pointer to the type-tag objects in the dispatch table
and your code, or moreso you do indexing using the pointer address
as an offset, so it takes only one machine instruction to fetch the
type and dispatch on it:
JSR mydisp-tagtable(@ptr)
or maybe seven instructions in a RISC instruction set:
MV x1,ptr ;Simply moves the toplevel object pointer to index register
LDIX a1,x1 ;Loads from memory at whatever address is in index register
LDIM x2,[mydisp-tagtable] ;Loads address of literal constant of difference
LDIX a2,x2 ;Loads the actual constant of difference from memory
ADD a1,a2 ;Computes lookup address in my dispatch table
MV nextpc,a1 ;Sets up the JSR address
JSR ;JSR's to whatever address has been set up
)
So anyway, my new idea is to first introduce intentional types for
each "high level" type (anything above bit and tuple), and have
students write actual code that works at that level to implement
the intentional type (in much the same manner that designers of
arithmetic chips on computers must use shift registers to add one
bit at a time and shift to the next bit with carry). Then we take
that new intentional type "for granted" and build up stuff that
depends on it. Nevermind that the CPU may already have some of
these types built-in so we didn't need to greenspun them ourselves.
Still it was a useful exercise to understand exactly how two's
complement arithmetic works, how tuple-trees are traversed, how
tuple-trees can implement all the intentional data types needed to
greenspun IBM 704 machine language and Lisp 1.0.
So my idea is that the whole range from boolean math and digital
circuit/switching/logic theory and machine language and data
structures and high-level languages can be taught using a uniform
teaching methodology all the way through. And the distinction
between intentional and built-in datatypes, yet the utter
greenspun/emulate-equivalence between them, is paramount in the
students' understanding. So then we won't need to decide whether to
teach machine/assembly langauge or high-level language, we can
sneak it all in under the guise of just one paradigm, bottom-up
intentional-datatype tool-building. Of course we'd have
Web-accessible emulators for all layers of this bottom-up process:
The bit+tuple machine, real and hypothetical machine languages, and
various "high level" languages we pass along the climb.
If I had ever seen a job posting looking for Lisp
hackers, I would have been really worried.
See, I'm not the only person who has noticed a complete absense of
Lisp employment openings openly advertised ever since about 1992.
I'm going back to ones and zeros.
Hey, that's *my* idea, using only two datatypes to start: single
bits, and tuples of arbitrary size.
A WFF is any of:
- a 'p', 'q', 'r' or 's'
- an 'N', followed by another WFF
- a 'C', 'A', 'K', or 'E', followed by two more WFFs
An object is any of:
- a '0' or '1'
- a tuple (ordered list) of any number of objects
As I was bicycling to Target to buy milk, carrying my cellphone
with InterNet access, where my shopping list is on the InterNet in
tiny-screen format for cellphone thereby accessible to my
cellphone, I was thinking about that, and realized there are
exactly 22 different binary functions of two or fewer arguments:
- 16 functions on 2 arguments
- 4 functions on 1 argument
- 2 functions (constants) on 0 arguments
Using function composition, *all* of those many functions can, as
is very well known and has been known since before I was in high
school, implemented with just one (1) function, either NAND or NOR
will do nicely. For my beginning course in computer software, I
plan to ask students to list all those 22 functions per their
truth-tables. Each time a truth table is submitted, my tutorial
will tell them something like the name of that function in Common
Lisp, for example IOR or ANDC1. After the student has supplied more
than half the 22 functions, any time the student can't think of any
more the tutorial will suggest a Common Lisp like name and ask for
the truth table of *that* new function not previously included.
After all 22 functions have been listed one way or another, the
tutorial will give the student the choice of either NAND or NOR (or
any other 2-argument function the student believes would also
work), and asks the student to define each of the other fuctions in
terms of just that one function and any other functions already
defined in this exercise (thus encouraging bottom-up tool building
and re-use of code already at this early stage). Let's see how this
would work if ORC2 were the selected function. (The tutorial would
use non-syntax methodology, as I proposed a few weeks ago. But here
I'll illustrate using Common Lisp DEFUNs:)
(defun arg0-const0 () 0) ;Cheat?
(defun arg0-const1 () 1) ;Cheat?
(defun arg1-const1 (x) (orc2 x x))
(defun arg1-not (x) (orc2 (arg0-const0) x))
(defun arg1-identity (x) (orc2 x (arg0-const1)))
(defun arg1-const0 (x) (orc2 (arg0-const0) (arg0-const1)))
(defun arg2-ior (x y) (orc2 x (arg1-not y)))
(defun arg2-and (x y) (arg1-not (arg2-ior (arg1-not x) (arg1-not y)))) ;Boole's law
(defun arg2-1 (x y) x)
(defun arg2-2 (x y) y)
etc. Did I make any mistakes?
Hey, to prevent student's copying each other's homework assignment,
I'll let the first-comers choose the easy boolean functions and
force late-comers to choose functions not previously chosen, until
all bootable seeds are taken. Of course the class will need to be
small, because only a few functions would work.
Finally out of the Web under SICP link from Meta-circular evaluator (WikiPedia)
Now out of that, back to Self-interpreter (Wikipedia)
Now out of that, back to Bootstrapping (compilers) (Wikipedia)
Oops, I've gotten sidetracked, still looking to see if that second
WikiPedia page contains any succinct/enlightening statement of an
alternate path into the fixed-point closed cycle (self-hosting)
similar to what I wrote (quoted and PJB-approved above). But here I
am at the end of that article, and despite the fine detours I
followed via hyperlinks, I don't recall seeing anything as succinct
as I wrote and you agreed with. So maybe I should propose my text
in the discussion section of this WikiPedia page and let some
expert figure out where to edit it into the main article?
== succinct statement of bootstrapping (compiler) ==
Recently I (Robert Maas) composed and posted this text on a
newsgroup, and Pascal J. Bourguignon approved of it, so I'm
thinking this text is succinct enough to appear in this article:
(text from above)
Somebody expert at WikiPedia want to edit it in somewhere?
DONE: <http://en.wikipedia.org/wiki/Talk:Bootstrapping_%28compilers%29>
(Splitting this long reply here...)
Pascal J. Bourguignon
> I just thought it'd be nice to have a CL+OS symbiosis that
> was truly self-hosting, with no dependency on C whatsoever:
It's all turtles down under.
Movitz.
> Continuation of that WikiPedia quote above:
> Writing new software development tools "from
> the metal" (that is, without using another host system) is rare and in
> many cases impossible.
> Is it *really* **really** impossible?
It has been done once, before the 1940's. There's no reason to do it again.
It is impossible to do it again, because nobody in his sane mind would
finance such an endeavour.
Even if you wanted to design a lisp machine, an electronic processor
able to execute lambda forms directly, you would use your computer to
design it. Otherwise, why would you use paper and pen? Use cave
walls and wood charcoal burnt from a silex started fire.
> It would be nice to know whether the interpreted Lisp compiler
> produced object code in executable form directly, or produced
> relocatable module which then needed to be LOADed then SAVEd to
> produce an executable (or on IBM mainframes LINK-EDITed to directly
> produce an executable), or assembly code which needed to be
> assembled then LoadSaved/LinkEdited.
Most probably, it punched out the object code. Then you could remove
the compiler from memory, and read the punched program in.
> <http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html>
> When I was a kid, I learned to program on punched cards. If you made a
> mistake, you didn't have any of these modern features like a backspace
> key to correct it. You threw away the card and started over.
> What a lie! The IBM 026 keypunch had a duplicate-card feature.
I bet Joel knows better than you whether he worked on a 026 or not.
> (Unfortunately it's awfully difficult to get anyone to try my
> demos, because I'm not a well-established company such as Google
Google doesn't provide a demo of a search engine. It provides a
completely working search engine.
> [...] Short of spamming, does anybody have any
> good ideas how to get people to play with my demos of CGI
> applications and then get into dialog with me about what
> improvements they can suggest?)
Don't make demos. Make useful (to the people) applications.
--
__Pascal Bourguignon__ http://www.informatimago.com/
HANDLE WITH EXTREME CARE: This product contains minute electrically
charged particles moving at velocities in excess of five hundred
million miles per hour.
Robert Maas, http://tinyurl.com/uh3t
> because it was invented by capitalist dogs. So they built a
> computer that worked in ternary. +12V, ~0V, -12V.
> http://www.computer-museum.ru/english/setun.htm
The small (3 pages of 54 words) ferrite RAM that has page exchange
with the main magnetic drum memory works as a cash.
Um, how do you store ternary data in ferrite RAM? Ferrite is very
difficult to get into any but two discrete states. Thus it's
inherently binary. This whole article reads like what Martin
Gardner might have published in Mathematical Games in the April
(Fools) issue, except that he would have spelled "cache" correctly.
symmetrical (balanced) code, i.e. by code with digits 0, +1, -1.
This is actually a good idea, not in logic circuitry itself, but in
higher-level software. In my proposed interval arithmetic software,
for decimal values I use Hollerith keypunch codes for negative
digits and IBM Hexadecimal codes for positive digits past nine.
N M L K J 0 1 2 3 4 5 6 7 8 9 A B C D E
(Explanation: "1" with a minus sign overprinted is negative 1 in
data for machine language on IBM 1620 but prints as "J" on top of
hollerith card, etc. for -2=K -3=L ...)
That allows graceful expression of values that cross over a major
9/0 boundary. For example, nested approximations to PI:
3.[0..2]
3.1[K..8]
3.1[J..6]
3.1[2..5]
3.1[3..5]
3.14[N..E]
3.14[K..3]
3.14[0..2]
3.141[1..9]
3.141[4..7]
3.141[5..7]
3.1416[M..2]
3.1416[K..0]
3.1415[9..A]
(That was a slow-but-steady-convergence algorithm.)
Nested approximations to (- (sqrt (+ 3 (* 2 (sqrt 2)))) (sqrt 2)):
[0..2]
0.[6..D]
0.9[3..C]
0.99[6..E]
0.9999[2..A]
0.99999999[5..B]
0.9999999999999999[8..C]
1.00000000000000000000000000000000[K..3]
1.0000000000000000000000000000000000000000000000000000000000000000[M..4]
(That was by Newton's method, which squares the precision each time.)
> Now, if you think about it, building lambda calculus hardware is
> not so hard. After all there are only four notions to implement
> and some microcode to manage the memory.
Rebuttal: Building a toy computer that will run slowly for a few
minutes before crashing might be easy, but building a fast lambda
calculus machine which can run for days or weeks without crashing
is a serious engineering task, requiring as I said hundreds of
support personal to purify the silicon and draft the etching masks
and mix the impurities/doping and maintain the clean room and
maintain quality control via random-sample testing and maintain
atmosphere during cooling and take care of hazardous waste disposal
and secure the building against terrorists and thieves and
sabotaurs and spies from competing companies etc. etc.
It seems better for Intel to keep making RISC CPUs and let some
software engineer write micro-code to convert them into lambda
calculus systems, rather than convince Intel into building lambda
calculus hardware directly into a new class of LSI chips, or even
worse (1) design+make LSI chips yourself or (2) breadboad MSI
lambda-calculus minicomputers as if this were still 1970, before
the first Intel 4004.
> >> It just happens that once you have a computer it's faster and
> >> cheaper to implement your lambda calculus virtual machine, or your
> >> lisp, programming this computer than soldering electronic
> >> components. Just one student could do it in a few months, while
> >> it would have been a major project if it would have been done with
> >> tubes.
> > Tubes??? Did the IBM 704 use tubes??? I just assumed it used
> > discrete transistors (and resistors, capacitors, etc.) on either
> > wire-wrap or printed-circuit boards, just like the IBM 1620 did (I
> > actually looked at the innerds once when it was being repaired or
> > demonstrated or somesuch), and the IBM 1401 too probably.
> No. The original 704 on which McCarthy and his team started
> implementing LISP was a tube computer. Later they had a 709 and
> then a 7090 which were transistor based.
Wow, thanks for the info. So what was the MTBF of those tubes all
together? Was there any time from one tube failure to the next to
actually use the REP for anything nontrivial?
> Almost all lisp implementations have a LAP. Well, let's discard
> ecls and gcl, since their LAP is C itself.
So CMUCL has LAP? I didn't know that. Looking at 'man cmucl' now ...
-- Calls to library functions (SIN, ...) are optimized to a direct call
to the C library routine (with no number consing.) On hardware with
direct support for such functions, these operations can easily be
open-coded.
Hey!!! CMUCL is a lot more dependent on C than people in this
thread have admitted! Looking at the online CMUCL manual, table of
contents, I see no mention of LAP at all, and no mention of
"assemb..." except for the disassemble function. So it looks like I
would need to roll my own executable-writer using WRITE-BYTE.
The following seems to be the key first-info I would need to know:
<http://en.wikipedia.org/wiki/A.out_%28file_format%29>
An a.out format for the PDP-7, similar to the a.out format used on the
PDP-11, appeared in the first edition of UNIX.^[2] It was superseded
by the COFF format in AT&T Unix System V, which was in turn superseded
by ELF in System V Release 4.
Though Berkeley Unix kept using a.out for some time, modern
BSD-systems have since switched to ELF. NetBSD/i386 switched formally
from a.out to ELF in its 1.5 release. FreeBSD/i386 switched to ELF
during the 2.2 to 3.0 transition.
So presumably the version of FreeBSD Unix on my ISP's shell machine
would use either COFF or ELF, most likely the latter. Let's see which:
(setq ichan (open "/home/users/rem/public_html/cgi-bin/h-c.cgi"
:element-type '(unsigned-byte 8)))
(loop for ix from 0 upto 8 collect (read-byte ichan))
;(127 69 76 70 1 1 1 9 0)
(loop for el in * collect (format nil "~2,'0X" el))
;("7F" "45" "4C" "46" "01" "01" "01" "09" "00")
<http://www-scf.usc.edu/~csci402/ncode/coff_8h-source.html>
00005 struct filehdr {
00006 unsigned short f_magic; /* magic number */
00007 unsigned short f_nscns; /* number of sections */
00015 #define MIPSELMAGIC 0x0162
00017 #define OMAGIC 0407
00018 #define SOMAGIC 0x0701
<http://www.freebsd.org/cgi/man.cgi?query=elf&sektion=5>
The header file <elf.h> defines the format of ELF executable binary
files. Amongst these files are normal executable files, relocatable
object files, core files and shared libraries.
% whereis elf.h
elf.h:
Back to the Web page:
The ELF header is described by the type Elf32_Ehdr or Elf64_Ehdr:
typedef struct {
unsigned char e_ident[EI_NIDENT];
...
} Elf32_Ehdr;
typedef struct {
unsigned char e_ident[EI_NIDENT];
...
} Elf64_Ehdr;
e_ident This array of bytes specifies to interpret the file,
independent of the processor or the file's remaining
contents. Within this array everything is named by
macros, which start with the prefix EI_ and may con-
tain values which start with the prefix ELF. The fol-
lowing macros are defined:
EI_MAG0 The first byte of the magic number. It
must be filled with ELFMAG0.
EI_MAG1 The second byte of the magic number.
It must be filled with ELFMAG1.
EI_MAG2 The third byte of the magic number. It
must be filled with ELFMAG2.
EI_MAG3 The fourth byte of the magic number.
It must be filled with ELFMAG3.
EI_CLASS The fifth byte identifies the architec-
ture for this binary:
Back to Google to find where ELFMAG0 is defined:
<http://tomoyo.sourceforge.jp/cgi-bin/lxr/source/arch/powerpc/boot/elf.h#L126>
126 #define ELFMAG0 0x7f /* EI_MAG */
127 #define ELFMAG1 'E'
128 #define ELFMAG2 'L'
129 #define ELFMAG3 'F'
(Doh!)
<http://tomoyo.sourceforge.jp/cgi-bin/lxr/source/include/linux/elf.h#L325>
325 #define ELFMAG0 0x7f /* EI_MAG */
326 #define ELFMAG1 'E'
327 #define ELFMAG2 'L'
328 #define ELFMAG3 'F'
(as expected now)
(setq bytes123 '("45" "4C" "46")) ;From top of this message
(loop for el in bytes123 collect (code-char (parse-integer el :radix 16)))
;(#\E #\L #\F)
OK, confirmed, it's ELF format!!
P.S. notice how I'm using (loop for el in <givenList> collect <expr>)
more and more, and (mapcar #'(lambda (el) <expr>) <givenList>) less
and less nowadays?
Now as soon as I find out whether ELF format on FreeBSD Unix uses
Little-to-Big or Big-to-End byte order for 16-bit and 32-bit
values, I should in theory be able to directly construct FreeBSD
Unix executables
(well I also need to learn the x386 instruction set, at least a
single instruction sequence that loads a returns the appropriate
completion code to the calling program i.e. the shell; perhaps I
can just study what I see in a compiled less-than-hello-world
program:
int main(int argc, char** argv) {return 42;}
6 -rwx------ 1 rem user 4188 Jul 27 10:34 a.out*
(setq ichan (open "/home/users/rem/CWork/a.out"
:element-type '(unsigned-byte 8)))
(loop for ix from 0 upto 4187 collect (format nil "~2,'0X" (read-byte ichan)))
;("7F" "45" "4C" "46" "01" "01" "01" "09"
"00" "00" "00" "00" "00" "00" "00" "00"
"02" "00" "03" "00" "01" "00" "00" "00"
"4C" "83" "04" "08" "34" "00" "00" "00"
"1C" "07" "00" "00" "00" "00" "00" "00"
"34" "00" "20" "00" "06" "00" "28" "00"
"18" "00" "15" "00" "06" "00" "00" "00" ...)
)
OK, I have some work cut out for me if I decide to write BootLAP myself.
First of course I'll write ReadELF to create a DOM (Document Object
Model, analagous to a parse tree) of the executable, then Prin1ELF
to do the inverse, then BootLAP will consist of generating the DOM,
then doing:
(with-open-file ("b.out" :direction :output :element-type '(unsigned-byte 8))
(Prin1ELF dom))
then twiddling the protection mask to make it executable of course.
> And again, this is something that exist in LISP since day 1, so
> to speak, at least in LISP 1.5:
Yeah, I've known that since I first used Stanford Lisp 1.6, shortly
after I met Larry Masinter at the AI lab and he showed me how he
worked with BBN-lisp, which unfortunately wasn't available locally,
but 1.6 was available, and then Hans Moravec explained how REP
works and I was cooking with fire, and then in 1973 when Les
Earnest temporarily banned me from the lab I got missing Lisp so
badly (like if your favorite food is pizza and suddenly you found
yourself unable to get any pizza, or Homer Simpson if
deprived of donuts)
that I pretended to be a C.S. student and created an account for
myself using the Imlac at the CS library in Polya Hall.
But LAP made only individual SUBRs (compiled function definitions,
i.e. DEFPROPs of compiled function objects) internally, and somehow
could also make FASL files, not executables. My proposed BootLAP
would be needed to directly make an executable from Lisp.
> There's no need for e<E2> to be a S-expression. It can be
> formally a sequence of S-expression.
AHA!! e<E2> is *not* a WFF, it's just a string of text that when
spliced into the stream will cause the surrounding-and-including
segment of that stream to be parsed as a WFF. I never saw that
explained in McCarthy's paper. I guess in those days even McCarthy
was sloppy about the difference between well-formed sub-expressions
in a parse tree or textual representation thereof, and
string-splice macros which act like C preprocessor macros.
> What matters is that (e<E2>) be a S-expression,
Right. Like if we do
(let ((e<E2> "1 2 3"))
(read-from-string (format nil "~A~A~A" "(" e<E2> ")")))
READ will parse a valid list of three numbers.
Thanks for filling in McCarthy's expository gap.
> > Did the typewriter have an underscore character available for such
> > use?
> If you read the AIM-8 paper scanned into PDF or PS,
I have no access to either kind of document reader here on my
Macintosh Performa 600 running System 7.5.5.
> you will see that these symbols were added manually (as well as
> all the square brackets and the arrows).
Yeah, from my high-school typewriter days I seem to recall square
brackets and arrows were not present. But I always thought
underscore was available. But maybe the way I used to get
underscores was by diddling the roller by a half line and typing
hyphens. But if I could diddle the roller, why couldn't McCarthy's
secretary? Or didn't she think of a trick that a high-school math
whiz thought of? Or was she too much of a think-only-inside-box
person who felt she'd lose her job if she diddled the roller on the
typewriter? Oh well. So I guess the OCR from PDF or PS to US-ASCII
(or Latin-1 or UTF-8) wasn't able to recognize the handwritten
symbols? (-: One more proof the A.I. problem hasn't yet been solved. :-)
> > Didn't Russell ever talk with McCarthy face to face, and therefore
> > be privy to ideas that McCarthy expressed in multiple forms at
> > different times, whereupon Russell built up a sense of what
> > McCarthy really meant, such that the formal paper was *not*
> > necessary to tell Russell what he needed to implement, so Russell
> > could DWIM most times there was a typo on McCarthy's paper, and
> > personaly ask McCarthy to clarify (or just re-engineer from the
> > math needs) the few remaining times?
> Of course. But intelligence is usually enough to make sense of papers
> with typoes, and even with gross logical errors.
Well sometimes that works and sometimes it doesn't. In the case of
the semicolon inside parens, where a comma should be present, I
tried to argue that *maybe* it should be a comma, but you insisted
a semicolon was correct, so that left me unsure who was right, but
finally you admitted I was right all along, it should have been a
comma. But in the case of non-WFF string-splice pre-processor
macros, I didn't even begin to think that McCarthy would stoop to
such unstructured notation, so the idea didn't even occur to me as
a possibility, until you explained it.
> > Ah, here's where I pointed out the semicolon problem in my original critique.
> >> Yes of course. Read again:
> >> (We shall use square brackets and semi-colons for writing
> >> functions of S-expressions since parentheses and commas have
> >> been pre-empted. ...)
> > You're missing my point.
> Yes, indeed I did. The semicolon in the parentheses was a typo
> of mine, it doesn't appear in the PDF or PS of the paper. I'll
> have to correct my web page.
(That's where you finally saw the typo I had been asking about all
along. It took several tries but I finally communicated to you well
enough that you would **look** at it and **see** it. All resolved
now, everyone <cliche>on the same [wavelength|page]</cliche>.)
> > ... or somebody (you?) made a mistake "fixing" a correct scan of (e_1,e_3).
> I did.
Ah, taking responsibility, <ot>unlike GWBush who still refuses to admit
he made an immensely bigger mistake diverting forces away from
Afghanistan to invade Iraq</ot>. You're an OK guy. You won't be impeached.
> >> Unfortunately for you, the form (x;y) doesn't appear in the paper.
> >> It's either (x,y) or [x;y].
> > So you're calling me a liar?
> Sorry. I must have been watching the PDF when I wrote that sentence.
(-: Good thing you're not in customer support!! Customer complains that
there's a bug in Windows Vista. You try it in Windows XP and say
there's no bug and refuse to look further. It takes three calls
before you look at WIndows Vista and see there *is* indeed a bug. :-)
> > [...]
> > but I also see some non-USASCII codes which are *not* valid UTF-8,
> > such as:
> > tions has important advantages. Devices such as G<F6>del number-
> No, it must be your how your software renders it. The document
> contains two bytes betwee the 'G' and the 'd'.
If I had the energy, I'd open a TCP stream to your HTTP port and
manually stream the raw HTML to (loop while ... collect (read-byte ...))
and see what's there, but I'll just trust you and try to guess
what's happening. Perhaps lynx is interepreting the two UTF-8 bytes
as the coding for one Latin-1 character, then rendering it in
US-ASCII by discarding the high-order byte, the mathematical
approach of reducing a problem to an already-solved problem.
(You know the joke about the psychologist testing the physicist
engineer and mathematician as to their ability to boil water,
right?)
> Notice that unicode is a superset of ISO-8859-1. The codes from
> 0 to 255 encode the same characters in unicode as in iso-8859-1.
Yes, I'm well familiar with that knowledge. I've written code to
convert between UniCode code points (unsigned integers up to 21
bits in length) and both Latin-1 and UTF-8 byte-sequence
representation. Latin-1 to CodePoint is the IDENTITY function.
CodePoint to Latin-1 is a range check followed by IDENTITY function.
The same goes for US-ASCII <-> CodePoint except the range is narrower.
> > I assume somebody scanned the hardcopy to produce a PDF, then
> > somebody ran some program to convert the PDF to UniCode file,
> > then somebody (probably Pascal Bourguignon) edited the UniCode
> > file in an attempt to produce a UTF-8 WebPage. Is that correct?
> No. I read the PDF and typed in the unicode in emacs.
You mean you manually re-keyed the entire document, or you got 99%
of the document (all the US-ASCII except the handwritten glyphs)
OCRed, then manually keyed/patched the places where OCR failed
(because OCR A.I. couldn't recognize handscribbled glyphs)?
Do you know of any program [PDF --(OCR)-> text] that runs on MacOS 7.5.5
that would do the 99% just the same as what you used there?
Pascal J. Bourguignon
>> > somebody ran some program to convert the PDF to UniCode file,
>> > then somebody (probably Pascal Bourguignon) edited the UniCode
>> > file in an attempt to produce a UTF-8 WebPage. Is that correct?
>> No. I read the PDF and typed in the unicode in emacs.
>
> You mean you manually re-keyed the entire document, or you got 99%
> of the document (all the US-ASCII except the handwritten glyphs)
> OCRed, then manually keyed/patched the places where OCR failed
> (because OCR A.I. couldn't recognize handscribbled glyphs)?
Yes, I did it manyally.
I know no OCR able to do a good job on badly printed or badly scanned
pictures.
> Do you know of any program [PDF --(OCR)-> text] that runs on MacOS 7.5.5
> that would do the 99% just the same as what you used there?
I've not used MacOS for almost twenty years, I don't remember any such
details. You should really try to advance to the 21st century.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Rob Warnock
+---------------
| From: p...@informatimago.com (Pascal J. Bourguignon):
| > Almost all lisp implementations have a LAP. Well, let's discard
| > ecls and gcl, since their LAP is C itself.
|
| So CMUCL has LAP? I didn't know that. Looking at 'man cmucl' now ...
Don't look in the man page, look in the CMUCL sources, for things
called "VOP"s. E.g., from "cmucl-19c/src/compiler/x86/arith.lisp":
(define-vop (signed-byte-32-len)
(:translate integer-length)
(:note "inline (signed-byte 32) integer-length")
(:policy :fast-safe)
(:args (arg :scs (signed-reg) :target res))
(:arg-types signed-num)
(:results (res :scs (any-reg)))
(:result-types positive-fixnum)
(:generator 30
(move res arg)
(inst cmp res 0)
(inst jmp :ge POS)
(inst not res)
POS
(inst bsr res res)
(inst jmp :z zero)
(inst inc res)
(inst shl res 2)
(inst jmp done)
ZERO
(inst xor res res)
DONE))
Is that LAP-y enough for you? ;-}
+---------------
| -- Calls to library functions (SIN, ...) are optimized to a direct call
| to the C library routine (with no number consing.) On hardware with
| direct support for such functions, these operations can easily be
| open-coded.
|
| Hey!!! CMUCL is a lot more dependent on C than people in this
| thread have admitted!
+---------------
As Pascal said, no-one in their right minds is going to completely
re-implement the standard "libc.so" from scratch on any standard
operating system. Why should they?!? Totally wasted effort, and
for what?!? Just use it.
+---------------
| Looking at the online CMUCL manual, table of
| contents, I see no mention of LAP at all, and no mention of
| "assemb..." except for the disassemble function.
+---------------
True, they don't expose it to ordinary users, but any user can
certainly add their own VOPs to the compiler, if they like,
even at run-time -- *without* recompiling the whole system!
[I actually had to do that recently, while investigating an
issue in the VOP for the fast-path of (RANDOM {unsigned-byte-32}).
Worked just fine, thank you very much!]
Remember, the source of CMUCL is completely open, so have at it!
+---------------
| So it looks like I would need to roll my own executable-writer
| using WRITE-BYTE.
+---------------
Sure, go ahead, if you want to re-invent all of computer science
since the wheel... ;-} ;-}
+---------------
| ...[long, rambling, poking at file formats]...
| OK, confirmed, it's ELF format!!
+---------------
Look, if you really, really want to write FreeBSD executables,
just look at the code added to "save.lisp" in CMUCL-19d (or later)
for the new ":EXECUTABLE T" option:
http://common-lisp.net/cgi-bin/viewcvs.cgi/*checkout*/src/code/save.lisp?root=cmucl
...
:executable
If nil (the default), save-lisp will save using the traditional
core-file format. If true, save-lisp will create an executable
file that contains the lisp image built in.
(Not all architectures support this yet.)
It handles all that ELF stuff, including stuffing the heap image
into a new ELF section that the start-up code can "mmap()" back
into memory when you run the result.
Of course, you're probably going to be upset when you discover
that the SAVE-EXECUTABLE function is a call-out to the C-based
run-time library! ;-} ;-} [The stuff in ".../src/lisp/*.[ch]".]
+---------------
| I should in theory be able to directly construct FreeBSD
| Unix executables
+---------------
Save yourself some work. (See above.)
-Rob
-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Andrew Reilly
> +---------------
> | I should in theory be able to directly construct FreeBSD | Unix
> executables
> +---------------
>
> Save yourself some work. (See above.)
I'm sure that Rob knows, but Robert Maas (and other readers) might be
interested to check out man 3 elf, man 5 elf and man 3 gelf on a
FreeBSD-7 (or more recent) system: there's a shiny new libelf (a BSD re-
implementation of the SysV one, I believe) for mucking about with all of
that stuff.
If you don't have such a system handy, then the manual pages can be found
on-line here:
http://www.freebsd.org/cgi/man.cgi?
query=elf&apropos=0&sektion=3&manpath=FreeBSD+7.0-RELEASE&format=html
In particular, man 5 elf has quite a bit of useful-looking information on
the subject.
Cheers,
--
Andrew
Robert Maas, http://tinyurl.com/uh3t
> > demos, because I'm not a well-established company such as Google
> completely working search engine.
That's because it has lots of employees who can bounce ideas off
each other, show early demos to each other, get feedback, get new
ideas, try new ideas, make demos better, bring in a large team to
combine different software modules into a complete application, and
only after all that is done they install a beta version for the
rest of us to try. I'm all alone here, no co-workers to bounce
ideas off, nobody in the newsgroups willing to look at my ideas and
suggest improvements, nobody willing to look at my early demos and
suggest changes, not at all like what must happen inside Google all
the time.
> > [...] Short of spamming, does anybody have any
> > good ideas how to get people to play with my demos of CGI
> > applications and then get into dialog with me about what
> > improvements they can suggest?)
> Don't make demos. Make useful (to the people) applications.
I don't have the resources of a hundred other employees to help me
make a useful application. I don't even have the feedback as to
what anybody would want me to produce, not even ideas from
co-workers who are guessing what customers might like based on
their experience. I don't have even one person around here to
encourage me to work without pay on some big project to help others
who haven't even told me what they might want.
In late 2002 I installed a completely useful program on my Web
site, and I have not yet found even one person to let me give a
full demo of what my program does so that I can get feedback as to
how to make the already-useful program even more useful. One of the
members of this newsgroup lives only about 20 miles from me in San
Mateo. Maybe that person would be interested in educational
software and want to come visit me and see a full demo of my
late-2000 Web application.
Robert Maas, http://tinyurl.com/uh3t
> scanned pictures.
There ought to be such a beast. If not, I could write one if
anybody was willing to pay me for my labor.
> You should really try to advance to the 21st century.
I have no money to pay for that. Perhaps you will hire me to work
for you so that I can pay off credit-card debt and then have some
money for anything new.
Robert Maas, http://tinyurl.com/uh3t
% man 3 elf
No entry for elf in section 3 of the manual
% man 5 elf
NAME
elf -- format of ELF executable binary files
SYNOPSIS
#include <elf.h>
% man 3 gelf
No entry for gelf in section 3 of the manual
% whereis elf.h
elf.h:
Kenny
> jaycx2.3....@spamgourmet.com.remove (Robert Maas, http://tinyurl.com/uh3t) writes:
>
>>(Unfortunately it's awfully difficult to get anyone to try my
>> demos, because I'm not a well-established company such as Google
>
>
> Google doesn't provide a demo of a search engine. It provides a
> completely working search engine.
>
This Error shall be known as The Error of Coming At It From the Wrong
Direction.
If YCAIFTWD, you look at a radical new way to learn Algebra and offer as
your only comment the point size on a web page. Sorry if I made that
sound really retarded, but it does happen. Sometimes the retardee
justifies it by asserting that design matters, which is probably not in
question by any life form higher than moss. The problem is CAIFTWD, aka
half empty/full, as in pint.
Back OT, comparing a fellow Lisper's search engine efforts to those of
Google.... well, that's a bit of a triumph when it comes to missing the
point, aka CAIFTWD.
The second Tilton Prize then must be for making a more retarded
comparison. Just to help, comparing a lawn chair with weather ballons to
an F-16 falls short of earning the prize.
What was that line about the dancing bear and how /well/ it danced not
being the issue?
kt
Pascal J. Bourguignon
> Pascal J. Bourguignon wrote:
>> jaycx2.3....@spamgourmet.com.remove (Robert Maas, http://tinyurl.com/uh3t) writes:
>>
>>>(Unfortunately it's awfully difficult to get anyone to try my
>>> demos, because I'm not a well-established company such as Google
>> Google doesn't provide a demo of a search engine. It provides a
>> completely working search engine.
>
> Back OT, comparing a fellow Lisper's search engine efforts to those of
> Google.... well, that's a bit of a triumph when it comes to missing
> the point, aka CAIFTWD.
We're not discussing a student trying to implement a toy to learn
something. We're talking about a professionnal (if unemployed)
mathematician and software developer with a high IQ, which intend to
develop some professionnal commercial product. If he tries to
develop a concurrent to google search engine, well he chosed his
competition.
--
__Pascal Bourguignon__ http://www.informatimago.com/
"Klingon function calls do not have "parameters" -- they have
"arguments" and they ALWAYS WIN THEM."
Rob Warnock
+---------------
| > I know no OCR able to do a good job on badly printed or badly
| > scanned pictures.
|
| There ought to be such a beast. If not, I could write one if
| anybody was willing to pay me for my labor.
Spammers are always willing to pay for technological advances
in this area, to defeat CAPTCHAs.
Oh, wait, they *did* that already!! See:
http://en.wikipedia.org/wiki/Captcha#Computer_character_recognition
...
Several research projects have broken real world CAPTCHAs,
including one of Yahoo's early CAPTCHAs called "EZ-Gimpy"[1]
and the CAPTCHA used by popular sites such as Paypal [12],
LiveJournal, phpBB, and other open source solutions [13] [14] [15].
In January 2008 Network Security Research released their program
for automated Yahoo! CAPTCHA recognition.[16] Windows Live Hotmail
and Gmail, the other two major free email providers, were cracked
shortly after.[17] [18]
In February 2008 it was reported that spammers had achieved a
success rate of 30% to 35%, using a bot, in responding to CAPTCHAs
for Microsoft's Live Mail service [19] and a success rate of 20%
against Google's Gmail CAPTCHA.[20] A Newcastle University research
team has defeated the segmentation part of Microsoft's CAPTCHA
with a 90% success rate, and claim that this could lead to a
complete crack with a greater than 60% rate.[21]
...