Is lisp this slow?
Richard James Panturis Giuly
The c program takes about 1 second.
The lisp program takes about 20.
Is that the best it can do?
(I'm using ACL on a Linux machine with 64M ram and a 300Mhz
processor.)
----------------------------------------------------
(defun fn ()
(dotimes (h 1000)
(dotimes (i 10000)
(setf j (* 4.33452325 i)) ) ) )
(compile 'fn)
(setf ti (get-universal-time))
(fn)
(print (- (get-universal-time) ti))
-----------------------------------------------------
#include <time.h>
main () {
int h;
int i;
float j = 4.34443;
long l;
l = time(0);
for (h=0; h < 1000; h++)
for (i=0; i < 10000; i++) {
j = 4.33452325 * i;
}
printf("%d", time(0) - l);
}
--
ricky
rgi...@surfsouth.com
Frode Vatvedt Fjeld
> Below I have to similar programs one in lisp, the other in C.
To make them something remotely comparable, do this in lisp:
(defun fn ()
(let (j)
(dotimes (h 1000)
(dotimes (i 10000)
(setf j (* 4.33452325 i))))))
Then compile it and do (time (fn)) in order to see how long it takes
to run.
--
Frode Vatvedt Fjeld
Christophe Rhodes
Firstly, you might want a
(declaim (optimize (speed 3) (safety 0)))
somewhere.
Then, too, you have to realize that this isn't a fair test; lisp will
win on your code (above), since it'll optimize away everything:
[ this is with CMUCL ]
* (compile 'fn) ; fn as above
Compiling LAMBDA NIL:
Compiling Top-Level Form:
FN
NIL
NIL
* (time (fn))
Compiling LAMBDA NIL:
Compiling Top-Level Form:
Evaluation took:
0.1 seconds of real time
0.1 seconds of user run time
0.0 seconds of system run time
0 page faults and
0 bytes consed.
NIL
So, we have a factor of 10 win over C :)
What this is probably saying is that the original poster needs to look
more carefully at what is being compared; and to remember that
simplistic tests don't actually have much to do with the real world.
Christophe
David Hanley
Richard James Panturis Giuly wrote:
> Below I have to similar programs one in lisp, the other in C.
>
> The c program takes about 1 second.
> The lisp program takes about 20.
>
> Is that the best it can do?
Well, firstly, the programs are doing different things.
The lisp program should be run with (time (fn)). This will
remove the difference created by your user-interaction
timing.
Secondly, the lisp program does not have types declared
as the C program does. Declaring the types makes a huge
difference in execution time, as I will show.
Thirdly, your C program is incorrect in several ways:
You are setting J which is never getting used(no big deal)
You use of "time", on my computer, always prints out either
one or two. This is not particularly interesting or helpful.
You are printing a long value, time, but you are only using the
format string for an int. This is wrong, wrong, wrong, and
can have unpredictable or disturbing results, including core
dumps.
To be fair, the lisp program has a problem too; the variable
'j' doesn't come from anywhere. It may be global, but that's
yet another way the programs are different. The C program
has J as a local variable, which can be a register.
Anyways, using "time" from the unix prompt to time the C
program, and using (time (fn)) to test a lisp program with
declared types yields the following:
C version : 2.0 (sun cc)
C version : 1.33 ( gnu gcc )
Lisp version : 0.19 ( CMU-CL )
(all with no optimizations)
The latter result is interesting. CMU-CL is smart enough to
determine that the inner 'i' is a fixnum, and that's all that matters.
In reality, i think this program is a little too simple to use
as a benchmark. A lot of compilers will probably optimize
the entire program away, as 'j' isn't going anywhere.
dave
Christopher J. Vogt
(defun fn2 ()
(declare (optimize (speed 3) (space 0) (debug 0) (safety 0)))
(time
(let (j)
(declare (type single-float j))
(dotimes (h 1000)
(declare (type fixnum h))
(dotimes (i 10000)
(declare (type fixnum i))
(setf j (the single-float (* 4.33452325 i))))))))
Tim Bradshaw
> Below I have to similar programs one in lisp, the other in C.
> The c program takes about 1 second.
> The lisp program takes about 20.
The programs aren't particularly similar -- the C version has a lot of
type information in it that the lisp one completely omits. The lisp
version also assigns to an undeclared variable, which is probably
cripplingly slow.
If you fix these ommissions you might end up with something like this:
(defun fn ()
;;(declare (optimize speed))
(let ((j 0.0))
(declare (type single-float j))
(dotimes (h 1000)
(declare (type fixnum h))
(dotimes (i 10000)
(declare (type fixnum i))
(setf j (* i 4.34443 i))))))
ACL seems not to do too well at this without an (optimize speed)
declaration, if you put that in then it's pretty comparible with gcc
for me. gcc with -O seems to optimize somewhat more aggressively than
ACL, and ends up compiling no floating point ops at all. Whether ACL
can be made to do that I don't know. CMUCL, which is good at dealing
with this kind of tight inner-loop code, seems to compile this
significantly better than ACL, though it's still half as fast as gcc
(both of these are basically measuring loop overhead for this, as the
float stuff is long gone). CMUCL is sometimes significantly worse than
ACL for more interesting code than this however.
Of course, benchmarking some more interesting program might be, well,
more interesting, and tell you a good deal more about the relative
performances. C systems are probably a little faster than typical
serious lisp systems where you can reasonably test similar programs,
so long as you ignore programmer time, reliability and stuff like
that.
--tim
Lieven Marchand
> Below I have to similar programs one in lisp, the other in C.
>
> The c program takes about 1 second.
> The lisp program takes about 20.
>
> Is that the best it can do?
>
> (I'm using ACL on a Linux machine with 64M ram and a 300Mhz
> processor.)
>
> ----------------------------------------------------
>
> (defun fn ()
> (dotimes (h 1000)
> (dotimes (i 10000)
>
> (setf j (* 4.33452325 i)) ) ) )
>
> (compile 'fn)
> (setf ti (get-universal-time))
> (fn)
> (print (- (get-universal-time) ti))
>
> -----------------------------------------------------
>
> #include <time.h>
>
> main () {
> int h;
> int i;
> float j = 4.34443;
> long l;
> l = time(0);
>
> for (h=0; h < 1000; h++)
> for (i=0; i < 10000; i++) {
>
> j = 4.33452325 * i;
> }
>
> printf("%d", time(0) - l);
> }
Sigh. We seem to have this thread about every six months.
First of all, languages are neither slow nor fast. Implementations are.
Second, your programs are not similar.
I compiled your C program on a Linux box with an AMD K6-2 333 MHz and
with an optimization flag of -O3 it prints zero, which indicates sub
second time used. Without optimization, it prints 1.
Now for the differences between the C program and your lisp program.
First of all, tell the Lisp compiler you're interested in speed:
(proclaim '(optimize (debug 0) (speed 3) (safety 0)))
This is unwise for general use, so don't do this for normal lisp
development, but it is the default mode for C.
The first great difference is that you haven't declared your variable
j. This means it will be a special variable. In a multithreaded
implementation as ACL, this comes with a penalty.
So, the first change is
(defun fn ()
(let ((j 0.0))
(dotimes (h 1000)
(dotimes (i 10000)
(setf j (* 4.33452325 i)))))
Now another difference is that you give your C compiler every
information about types. If we do the same for lisp we get:
(defun fn ()
(let ((j 0.0))
(declare (single-float j))
(dotimes (h 1000)
(dotimes (i 10000)
(declare (fixnum h i)
(:explain :boxing :types))
(setf j (* 4.33452325 i))))))
This program gives on my machine also sub second time used.
Your program spent most of its time boxing up floats, discarding them
on the next assignment and garbage collecting the discarded boxed
floats.
Now, was this exercice worth anything?
Not at all. If you want to know how fast an implementation is, use it
on an application you care about. Not on empty loops.
And if the loop is so important measurement shows that it is a
bottleneck, you always have the possibility to rewrite it in
assembler.
--
Lieven Marchand <m...@bewoner.dma.be>
When C++ is your hammer, everything looks like a thumb. Steven M. Haflich
Richard James Panturis Giuly
think that's ridiculous, or can I process data with lisp almost as
quickly as a lower level language?
Joe Marshall
> Below I have to similar programs one in lisp, the other in C.
>
> The c program takes about 1 second.
> The lisp program takes about 20.
>
> Is that the best it can do?
No.
First of all, although these programs seem similar, they are not
identical. Allow me to modify the code for FN to bring it more in
line with what C does.
I'll make H be 10000 because my machine is fast.
First, I'll declare speed 3 and safety 0. I think this is fair
considering that C does this by default.
Second, I'll make j be a local variable (as it is in the C program)
and tell the compiler its type (which C gets to know as well).
(defun fn1 ()
(declare (optimize (speed 3) (safety 0)))
(let ((j 4.34443))
(declare (single-float j))
(dotimes (h 10000)
(dotimes (i 10000)
(setq j (the single-float (* i 4.33452325)))))))
The original code performed like this on my machine:
; cpu time (non-gc) 20,530 msec user, 0 msec system
; cpu time (gc) 460 msec user, 0 msec system
; cpu time (total) 20,990 msec user, 0 msec system
; real time 20,990 msec
; space allocation:
; 240 cons cells, 0 symbols, 1,600,000,928 other bytes, 0 static bytes
The code with the declarations performed like this:
; cpu time (non-gc) 972 msec user, 0 msec system
; cpu time (gc) 0 msec user, 0 msec system
; cpu time (total) 972 msec user, 0 msec system
; real time 972 msec
; space allocation:
; 0 cons cells, 0 symbols, -32 other bytes, 0 static bytes
These declarations produced a 21-fold speedup on my machine, you
should see a similar performance increase.
>
> (I'm using ACL on a Linux machine with 64M ram and a 300Mhz
> processor.)
>
> ----------------------------------------------------
>
> (defun fn ()
> (dotimes (h 1000)
> (dotimes (i 10000)
>
> (setf j (* 4.33452325 i)) ) ) )
>
> (compile 'fn)
> (setf ti (get-universal-time))
> (fn)
> (print (- (get-universal-time) ti))
>
> -----------------------------------------------------
>
> #include <time.h>
>
> main () {
> int h;
> int i;
> float j = 4.34443;
> long l;
> l = time(0);
>
> for (h=0; h < 1000; h++)
> for (i=0; i < 10000; i++) {
>
> j = 4.33452325 * i;
> }
>
> printf("%d", time(0) - l);
> }
> --
> ricky
> rgi...@surfsouth.com
Johan Kullstam
> I want to use lisp to deal with processing a stream of video. Do you
> think that's ridiculous, or can I process data with lisp almost as
> quickly as a lower level language?
if for some reason you can't make lisp go fast enough, you can call a
foreign function (e.g., fortran or C) from lisp. you may want to
check out matlisp. this would give some nice matrix functions and
provide examples of interfacing to fortran.
--
J o h a n K u l l s t a m
[kull...@ne.mediaone.net]
Don't Fear the Penguin!
Erik Naggum
| Below I have to similar programs one in lisp, the other in C.
No, you don't. Others have explained why in great detail.
| Is that the best it can do?
The right question for you to ask is "is that the best I can do in
Common Lisp?". Others have explained better things you could do in
great detail.
I'd like to ask you a different question: What did you expect to
learn from this comparison? In my opinion, using naively coded
examples with bugs in them is _not_ a good way to compare anything.
#:Erik
--
If this is not what you expected, please alter your expectations.
Erik Naggum
| you think that's ridiculous, or can I process data with lisp almost
| as quickly as a lower level language?
Well, as general advice, less than 1% of the code of a program is
truly speed-sensitive, and you can write the remainder of the code
much faster in Common Lisp than in any other language. You won't
notice any differences in response time on that code, anyway.
The speed-ciritical code may require special coding, as with a whole
lot of type declarations, careful examination of bottlenecks with a
profiling tool, etc. Common Lisp environments come with a whole lot
of mature tools in this regard. It is not uncommon for a Common
Lisp project to drop down to C or assembler to take care of special
needs. For instnace, if you're on an Pentium processor, you may
find it useful to exploit the Pentium II or Pentium III facilities,
and this may be hard to do directly from Common Lisp, while some C
or C++ compilers may have reasonable support for them.
More often than not, however, you may be able to experiment with a
variety of algorithms in Common Lisp in the time it takes just to
write _one_ that works correctly in, say, C++. This aspect of the
language is often not appreciated by non-Lispers.
Joe Marshall
> I want to use lisp to deal with processing a stream of video. Do you
> think that's ridiculous, or can I process data with lisp almost as
> quickly as a lower level language?
It's not ridiculous, but it may be challenging. Do you need to
process in real time? If so, you may have to be very careful about GC
pauses.
If you are `offline' processing, I don't see a reason why you couldn't
use Lisp directly, or use a small C module for the intensive stuff and
Lisp for the higher level stuff.
John Clonts
>
> Centuries ago, Nostradamus foresaw a time when Erik Naggum would say:
> >* Richard James Panturis Giuly <n...@spam.com>
> >| Below I have to similar programs one in lisp, the other in C.
> >
> > No, you don't. Others have explained why in great detail.
> >
> >| Is that the best it can do?
> >
> > The right question for you to ask is "is that the best I can do in
> > Common Lisp?". Others have explained better things you could do in
> > great detail.
> >
> > I'd like to ask you a different question: What did you expect to
> > learn from this comparison? In my opinion, using naively coded
> > examples with bugs in them is _not_ a good way to compare anything.
>
> standpoint.
>
> The programs _appear_ to do "the same thing," and the process of
> describing how to tune the Lisp version to perform better is reasonably
> instructive, at least to the relatively naive user.
>
> It does not require either:
> a) A lot of verbiage, or
> b) Anything _tremendously_ complex, conceptually,
> to show how to make the Lisp code run lots faster.
>
> It is also interesting to note that most of the techniques suggested
> _won't_ do much good with CLISP, and it would be instructive to look at
> _why_ that is, as well.
I can attest to what you're saying-- as a "naive lurker" I found it
instructive (especially the one that pointed out that it would be
optimized entirely away!).
So, why does it do no good with CLISP?
Cheers,
John
cs61a-ev
>
> think that's ridiculous, or can I process data with lisp almost as
> quickly as a lower level language?
>
> rgi...@surfsouth.com
Actually, I don't really know how fast your program must run.
In my opinion, it is not ridiculous at all to expect good speeds with
lisp programs, provided that you follow some implementation-dependent
optimization techniques. I recommend CMU/CL which seems to permit the
best speeds.
The techniques for optimization are often easy to apply, even if there
are many cases where I simply get lost (problems in understanding CMU/CL
documentation).
For instance, I developped with 4 comrades a small OpenGL subset in
Common Lisp, using CMU/CL. We finally managed to get 1/4 of the speed of
MesaGL in average, with the possibility to go up to 1/2 (university
project in limited time). We really enjoyed using Common Lisp which
gives so many convenient features and gives you a better view over your
programs. This also permits more advanced optimizations.
But if you really need high speed, maybe you should consider assembly
code only for the time-consuming functions. Notice that you really can
call such speedy functions from CMU Common Lisp, allying convenience of
the language with optimal execution speeds. That is what companies
(Naughty Dog, Nichimen) tend to do when they need terrific speeds for 3d
engines, for instance...
I hope I helped you, I'd really like to know if you will manage to make
it in Common Lisp and how. My knowledge in CMU/CL optimization
techniques needs to be completed...
--
----------------------------------------------
Jonathan BAILLEUL (bail...@emi.u-bordeaux.fr)
Maitrise Informatique, Universite Bordeaux I
Currently attending Berkeley Summer Session Courses CS61a/CS3
cs61a-ev
>
> Richard James Panturis Giuly <n...@spam.com> writes:
>
> > I want to use lisp to deal with processing a stream of video. Do you
> > think that's ridiculous, or can I process data with lisp almost as
> > quickly as a lower level language?
>
> process in real time? If so, you may have to be very careful about GC
> pauses.
Yes.
This can be avoided in avoiding consing as much as possible. Sometimes
it requires handling by hand pre-allocated pools of memory (Paul
Graham's ACL) or forcing certain types (floating-points) to use
non-descriptor representation. I'd like extra help on this topic under
CMU/CL, as far as some special cases cause problems in my programs...
>
> If you are `offline' processing, I don't see a reason why you couldn't
> use Lisp directly, or use a small C module for the intensive stuff and
> Lisp for the higher level stuff.
--
Christopher Browne
>* Richard James Panturis Giuly <n...@spam.com>
>| Below I have to similar programs one in lisp, the other in C.
>
> No, you don't. Others have explained why in great detail.
>
>| Is that the best it can do?
>
> The right question for you to ask is "is that the best I can do in
> Common Lisp?". Others have explained better things you could do in
> great detail.
>
> I'd like to ask you a different question: What did you expect to
> learn from this comparison? In my opinion, using naively coded
> examples with bugs in them is _not_ a good way to compare anything.
Actually, I think the example is _quite_ useful, from a pedagogic
standpoint.
The programs _appear_ to do "the same thing," and the process of
describing how to tune the Lisp version to perform better is reasonably
instructive, at least to the relatively naive user.
It does not require either:
a) A lot of verbiage, or
b) Anything _tremendously_ complex, conceptually,
to show how to make the Lisp code run lots faster.
It is also interesting to note that most of the techniques suggested
_won't_ do much good with CLISP, and it would be instructive to look at
_why_ that is, as well.
--
cbbr...@hex.net - <http://www.ntlug.org/~cbbrowne/lisp.html>
"Some sins carry with them their own automatic punishment. Microsoft
is one such. Live by the Bill, suffer by the Bill, die by the Bill."
-- Tom Christiansen
Christopher Browne
>Christopher Browne wrote:
>>
>> Centuries ago, Nostradamus foresaw a time when Erik Naggum would say:
>> >* Richard James Panturis Giuly <n...@spam.com>
>> >| Below I have to similar programs one in lisp, the other in C.
>> >
>> > No, you don't. Others have explained why in great detail.
>> >
>> >| Is that the best it can do?
>> >
>> > The right question for you to ask is "is that the best I can do in
>> > Common Lisp?". Others have explained better things you could do in
>> > great detail.
>> >
>> > I'd like to ask you a different question: What did you expect to
>> > learn from this comparison? In my opinion, using naively coded
>> > examples with bugs in them is _not_ a good way to compare anything.
>>
>> Actually, I think the example is _quite_ useful, from a pedagogic
>> standpoint.
>>
>> The programs _appear_ to do "the same thing," and the process of
>> describing how to tune the Lisp version to perform better is reasonably
>> instructive, at least to the relatively naive user.
>>
>> It does not require either:
>> a) A lot of verbiage, or
>> b) Anything _tremendously_ complex, conceptually,
>> to show how to make the Lisp code run lots faster.
>>
>> It is also interesting to note that most of the techniques suggested
>> _won't_ do much good with CLISP, and it would be instructive to look at
>> _why_ that is, as well.
>
>instructive (especially the one that pointed out that it would be
>optimized entirely away!).
>
>So, why does it do no good with CLISP?
Because CLISP doesn't provide the (declaim) options that are common
with other Common Lisps, and, more notably, because it doesn't compile
to assembly language, and thus provide _substantial_ optimizations.
Sample:
Running the bit of code:
(defun fn ()
(let (j)
(dotimes (h 1000)
(dotimes (i 10000)
(setf j (* 4.33452325 i))))))
(time (fn))
with CLISP, three times:
a) Compiled, via "clisp -c out.lsp"
b) Compiled, via "clisp -c outfast.lsp", with the "declaim" options
added in
c) Not compiled.
Copyright (c) Bruno Haible, Michael Stoll 1992, 1993
Copyright (c) Bruno Haible, Marcus Daniels 1994-1997
Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998
Copyright (c) Bruno Haible, Sam Steingold 1999
[1]> (load "out.fas")
;; Loading file out.fas ...
Real time: 10.876239 sec.
Run time: 8.69 sec.
Space: 72 Bytes
;; Loading of file out.fas is finished.
T
[2]> (load "outfast.fas")
;; Loading file outfast.fas ...
Real time: 10.951782 sec.
Run time: 8.69 sec.
Space: 72 Bytes
;; Loading of file outfast.fas is finished.
T
[3]> (load "out.lsp")
[runs for several minutes without respite...]
It's definitely _vastly_ faster to "precompile" CLISP code; that still
leaves you with runtimes that are decidedly not impressive.
--
cbbr...@acm.org - <http://www.hex.net/~cbbrowne/lisp.html>
Of course, unless one has a theory, one cannot expect much help from a
computer unless _it_ has a theory)... -- Marvin Minsky
Martin Cracauer
>I want to use lisp to deal with processing a stream of video. Do you
>think that's ridiculous, or can I process data with lisp almost as
>quickly as a lower level language?
Well, if it's real-time, you may get into conflict with the Garbage
Collector.
Looking at your previous posting, the first thing you need is to turn
on maximum warning level on your C compiler. The second thing to do
is to compile your code on CMUCL, even if your final version should
run on a different Lisp. CMUCL compiler emit a lot of message that
help you to make your code fast.
There are no "real" problems with Common Lisp reaching what I'd call
"raw machine speed". Minor problems are that the declarations in Lisp
are ugly and hard to write, so that you loose some of Lisp's
advantage. Using objects from classes in the classical sense in Lisp
(CLOS) is a lot slower than C++ or Objective-C, but for raw speed you
don't want that anyway.
A somewhat more serious problem is that some source file layout forms
may affects current Lisp compilers more than current C compilers. If
your application is spread over several source files, you cannot use
function from a different files as inline functions. That applies to
both Lisp and C. But in Lisp at least CMUCL introduces additional
overhead for calling real functions. It forces argument checks and
the return value is always boxed. But if you're really other speed,
you don't want sperate compilation in either C or Lisp anyway.
I agree to what other said that Lisp makes it a lot easier to
experiment with different algorithms and data structures than C does.
If you overlook a better alorithm, the whole boxed-vs-direct value,
garbage collection etc. is absolutely irrelevant.
I don't agree to the "hot spots" view. For me, when I design an
application from a well-defined specification with performance in
mind, I have some hot spots in the initial try and eliminate them
after the first profiling session, a matter of less than on hour.
After that, my applications are usually no longer subject to speed up
through local changes. If your programming is the same way, that has
far-reaching consequences, especially that using some C functions in
your Lisp program doesn't buy you much.
Martin
--
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <crac...@bik-gmbh.de> http://www.bik-gmbh.de/~cracauer/
FreeBSD - where you want to go. Today. http://www.freebsd.org/
Fernando Rodríguez
"Richard James Panturis Giuly" <n...@spam.com> escribió en el mensaje
news:396B7E25...@spam.com...
> I want to use lisp to deal with processing a stream of video. Do you
> think that's ridiculous, or can I process data with lisp almost as
> quickly as a lower level language?
It would be interesting if someone with greater experience could explain
how to handle the GC delay in this sort of realtime applications... O:-)
john...@my-deja.com
company that specializes in real time Java (boo, hiss). But they
supposedly have a "real time garbage collector". A few years back I
thought they were working on hardware-assisted garbage collection, but
from just visiting their web site I don't see anything about hardware
assist. See www.newmonics.com.
In article <0D_a5.154$5d3....@m2newsread.uni2.es>,
Sent via Deja.com http://www.deja.com/
Before you buy.
Richard James Panturis Giuly
> | Is that the best it can do?
>
> The right question for you to ask is "is that the best I can do in
> Common Lisp?". Others have explained better things you could do in
> great detail.
You are correct. My wording was bad because I was in a hurry and afraid
that lisp was 20 times slower that C. That would have messed up my plans
pretty badly.
Joe Marshall
> "Richard James Panturis Giuly" <n...@spam.com> escribió en el mensaje
> news:396B7E25...@spam.com...
> > I want to use lisp to deal with processing a stream of video. Do you
> > think that's ridiculous, or can I process data with lisp almost as
> > quickly as a lower level language?
>
> It would be interesting if someone with greater experience could explain
> how to handle the GC delay in this sort of realtime applications... O:-)
It depends on how `real time' you need to be.
If there are certain times where you cannot tolerate a GC, but for the
most part you can, then you can simply let the application CONS when
it has to (either let it gobble as much memory as it wants, or run the
GC just before the critical part so there is enough room, or
pre-allocate the amount it needs).
If your application can be made to not CONS during the real-time part
of its operation, there is no need to GC during those parts.
If your application can get away with `fast response' or `almost real
time', a correctly tuned generational GC will probably do the trick.
If you need `hard real time' response, but also need to let the
application cons at will, an `incremental' GC can do it. An
`incremental' GC does a bounded amount of work per cons, so you can
guarantee a real upper bound on response. But since these can be
tricky to code, it might be easier to just buy a lot of ram and go
with the first option.
Rainer Joswig
> I wonder if any lispers have ever heard of Newmonics? They are a
> company that specializes in real time Java (boo, hiss). But they
> supposedly have a "real time garbage collector".
Harlequin has a realtime GC for Lisp which has been
used projects. As I understand the GC of ACL is widely
tuneable and one might get certain capabilities by
setting the right switches. Symbolics also has been
using their system in realtime environments, AFAIK.
--
Rainer Joswig, BU Partner,
ISION Internet AG, Steinhöft 9, 20459 Hamburg, Germany
Tel: +49 40 3070 2950, Fax: +49 40 3070 2999
Email: mailto:rainer...@ision.net WWW: http://www.ision.net/
Erik Naggum
| It would be interesting if someone with greater experience could explain
| how to handle the GC delay in this sort of realtime applications... O:-)
Well, suppose you have a stream that produces about 50 M/s, your
maximal GC delay is 100 ms, and your processing ability is above 50
M/s, you can set up output buffers of about 5 M, which you would
fill before starting actual output and which would drain during GC,
coupled with an input buffer that would fill up during GC and be
drained faster than new input is filling it.
Nobody would ever notice any GC pauses. All you would see is 100 ms
processing latency without any reduction in bandwidth.
If you can control the input bandwidth and read and process faster
than you need to output, your application would block on output and
you wouldn't need to worry about GC pauses at all if you set up big
enough output buffers.
The key is really to have faster-than-real-time processing power.
Robert Monfera
> Well, suppose you have a stream that produces about 50 M/s, your
> maximal GC delay is 100 ms, and your processing ability is above 50
> M/s, you can set up output buffers of about 5 M, which you would
> fill before starting actual output and which would drain during GC,
> coupled with an input buffer that would fill up during GC and be
> drained faster than new input is filling it.
This would not work if real-time output synchronization is needed (e.g.,
displaying a video stream). Maybe an option is to set up two processes,
one would prepare the buffers (e.g., subsequent video frames) and the
other performs the synchronized output of the buffer.
Robert
Erik Naggum
| Well, if it's real-time, you may get into conflict with the Garbage
| Collector.
This is a persistent myth. Real time does not mean zero latency.
With stream data, non-zero latency translates to buffering. You
don't want to saturate the processor, anyway, so you have time to
catch up after a GC.
| There are no "real" problems with Common Lisp reaching what I'd call
| "raw machine speed". Minor problems are that the declarations in
| Lisp are ugly and hard to write, so that you loose some of Lisp's
| advantage.
So use Common Lisp to its advantage and write macros to help with
the type declarations, then.
| Using objects from classes in the classical sense in Lisp (CLOS) is
| a lot slower than C++ or Objective-C, but for raw speed you don't
| want that anyway.
Another persistent myth. Properly declared, CLOS is not at a
disadvantage compared to C++, which does a lot of unnecessary
copying due to its lack of garbage collection. It takes more time
than C++ aficionados are willing to admit.
If CMUCL still uses PCL, it should be noted that a native CLOS is
much, much more efficient than PCL.
Colin Walters
> This would not work if real-time output synchronization is needed
> (e.g., displaying a video stream). Maybe an option is to set up two
> processes, one would prepare the buffers (e.g., subsequent video
> frames) and the other performs the synchronized output of the
> buffer.
Even this won't work in all cases unless you have some guarantees from
the operating system (i.e. a real-time OS), no matter how big your
buffers.
Duane Rettig
> Throwing in type declarations and function-specific optimization hints is a
> lot of work. In my experience just throwing a
>
> (declare (optimize (speed 3) (safety 0)))
>
> at the top of the file will do the trick. Try it with your code and see.
At Franz Inc., we tend to recommend against the wholesale use of the
specific (speed 3) (safety 0) combination, because it may cause correct
code to run incorrectly. Specifically, two extra optimizations it
causes are:
- Declared fixnums remain fixnums when adding or subtracting, which tends
to eliminate the overflow check. Thus, what should be an overflow
into a bignum becomes a wrap around to a large fixnum of the opposite
sign. Care must be given that fixnums will never get large enough
to overflow.
- Interrupt checks are not made. This causes lisp code never to
relinquish a tight loop in a multiprocessing situation, and thus
may cause the lisp to appear to hang or become unresponsive.
Instead, we recommend a
(declaim (optimize (speed 3) (safety 1)))
at the top, with selected declarations of (safety 0) at local contours
which have been thought out and the above cases eliminated or mitigated.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 du...@Franz.COM (internet)
Bruce Tobin
lot of work. In my experience just throwing a
(declare (optimize (speed 3) (safety 0)))
at the top of the file will do the trick. Try it with your code and see.
It may be that because one of your integer counters is being multiplied with
a floating point, you will need to add a specific declaration for that one.
Usually ACL does a decent job of type inferencing when the (speed 3) switch
is on, but the float multiplication may confuse it enough that an explicit
type declaration is required for that variable.
Martin Cracauer
>* Martin Cracauer
>
> If CMUCL still uses PCL, it should be noted that a native CLOS is
> much, much more efficient than PCL.
Last time I spoke to Jim Veitch - at that time at Franz - he told me
they were still using PCL for Allegro as well. The difference is that
they profiled common applications and tuned the methods lookup
mechanism (especially the hashing) to those. The assumptions Gregor
Kiczales made at the time he wrote PCL were different, maybe even not
based on measured data at all. CMUCL uses the original mechanism and
hence is slower than it needs to be.
The whole discussion is a little besides the point. I didn't say that
a CLOS program is slower than a C++ program (although there's a bigger
chance that this is the case than for a non-CLOS CL applications
versus a C app). What I said is that CLOS objects are meant to be
used for larger-scale abstractions than C++ objects and when you use
them for fine-graded purposes, your hash-based lookup taking methods
combinations into account will not be able to meet the speed of C++'s
simple function calls (can even be inlined) or at most using one
pointer indirection through a method (not class) pointer stored in a
C++ object. It goes without saying that C++'s design quickly leads to
slowness through increased space requirements, but it is not sure
whether your applications hits it, while it is sure that the initial
function call speed benefit is there.
That is, BTW, what Dylan's paradigm of classes that may be sealed is
about - give the user one class mechanism where he can express both
paradigms (CLOS flexibility vs. closed-app speed) in a single
mechanism and can change existing classes from one of them to the
other without changing much code, especially without forcing the user
to re-wrap his method's code into a different way to organize methods.
And without the C++ style bloating the RAM amount of *every* object
where for many it doesn't pay off because these objects don't have
methods calls othen enough for the direct function benefit, but are
called frequently enough to trash caches or cause paging.
The whole point is moot anyway when people start requiring using CORBA
over XML-based RPC and at the same time force the use of some OO
patterns where local iteration over collections of remote objects
takes place. Just heard the war story of a case in that line. You
will be able to beat that by speed in every language once you beat
down the stupidness.
Martin Cracauer
>* Martin Cracauer
>| Well, if it's real-time, you may get into conflict with the Garbage
>| Collector.
> This is a persistent myth. Real time does not mean zero latency.
> With stream data, non-zero latency translates to buffering. You
> don't want to saturate the processor, anyway, so you have time to
> catch up after a GC.
That only applies when the playback may be some time behind the
availability of the unfiltered stream of video or audio.
It doesn't work that way when you produce contents that must be
synchronized with real events, i.e. audio for computer games can't
build buffers, the action of the player is supposed to be spiced with
a BOOM or whatever immediately.
>| Using objects from classes in the classical sense in Lisp (CLOS) is
>| a lot slower than C++ or Objective-C, but for raw speed you don't
>| want that anyway.
> Another persistent myth. Properly declared, CLOS is not at a
> disadvantage compared to C++, which does a lot of unnecessary
> copying due to its lack of garbage collection. It takes more time
> than C++ aficionados are willing to admit.
In practice, C++ programs tend to do a lot of unneeded copying, but I
have to rate all of them into pilot errors, where programms use
objects in a way that copies them, while a copy-free mechanism would
be available to the programmer. The problem here is that C++ has too
many complex rules of that kind for even a good programmer to take
into account and that compilers don't warn about it (whereas existing
Lisp compilers warn very well about consing).
Erik Naggum
| This would not work if real-time output synchronization is needed
| (e.g., displaying a video stream).
I said "which would drain during GC" about the output buffers.
That's what solves the constant output bandwidth problem.
I'm insulted that you think I answered his question without
considering constant data flow on the output side. Perhaps you
didn't even read what I suggested? I have to hope so.
Erik Naggum
| Even this won't work in all cases unless you have some guarantees from
| the operating system (i.e. a real-time OS), no matter how big your
| buffers.
Oh, I see, the point of this exercise is to prove it impossible, not
to actually solve any problems. My mistake. I thought were were in
constructive mode, but it is apparently important to find ways to
make things _not_ work. I'll chip in with "Even that won't work if
the power cord isn't connected. How does Lisp solve _that_, huh?"
Martin Cracauer
>* Colin Walters <wal...@cis.ohio-state.edu>
>| Even this won't work in all cases unless you have some guarantees from
>| the operating system (i.e. a real-time OS), no matter how big your
>| buffers.
> Oh, I see, the point of this exercise is to prove it impossible, not
> to actually solve any problems. My mistake. I thought were were in
> constructive mode, but it is apparently important to find ways to
> make things _not_ work. I'll chip in with "Even that won't work if
> the power cord isn't connected. How does Lisp solve _that_, huh?"
For sound, that's easy, you can make a lot of noise by dropping CLtL2
on the floor, much better than that lean C standard you only get as
softcover.
Erik Naggum
| That only applies when the playback may be some time behind the
| availability of the unfiltered stream of video or audio.
So you want hardware solutions. Using software will out of
necessity take non-zero time for non-zero efforts.
| It doesn't work that way when you produce contents that must be
| synchronized with real events, i.e. audio for computer games can't
| build buffers, the action of the player is supposed to be spiced
| with a BOOM or whatever immediately.
I'm so glad we're in this negative mood where the purpose of this
whole crappy exercise is to find ways Lisp will fail. Is it any
wonder Lisp has a bad reputation? Even Lispers you'd think would
have an interest in finding ways to solve the goddamn problems waste
their time finding cases where they think it won't work. I have to
wonder: Are you this bad an engineer?
| In practice, C++ programs tend to do a lot of unneeded copying, but I
| have to rate all of them into pilot errors, where programms use
| objects in a way that copies them, while a copy-free mechanism would
| be available to the programmer.
And now, apologia for C++!? Christ. C++'s lack of GC has one very
important consequence: It has to has to have an ownership protocol
for its allocated objects. Ownership protocols are very interesting
because the are extremely complex. Since the likelihood that they
could be mastered by the kind of people who choose C++ is so close
to zero only number theorists would find anything in that space, the
easy way out is to create a new object with known owners and let the
owner of the original take care of it. Writing C++ is bad enough,
but copy-free C++ is a lot harder than consing-free Lisp.
Tim Bradshaw
> Oh, I see, the point of this exercise is to prove it impossible, not
> to actually solve any problems. My mistake. I thought were were in
> constructive mode, but it is apparently important to find ways to
> make things _not_ work. I'll chip in with "Even that won't work if
> the power cord isn't connected. How does Lisp solve _that_, huh?"
I thought that WITH-POWER-ON was meant to do that? But yeah, to do it
*right* you probably you really need something like Scheme's
CALL-WITH-POWER-ON (CALL/PO) but I think that implementing that really
means you can never stack-allocate protons, which is a real problem,
since it basically means all your programs are radioactive. Of course
in C++ they just bracket things with power->on() and power->off(), I
have no idea what happens if they get an error, presumably they just
end up violating energy conservation all over the place, but they
don't seem to care about that.
--tim
Martin Cracauer
>* Martin Cracauer
>| That only applies when the playback may be some time behind the
>| availability of the unfiltered stream of video or audio.
> So you want hardware solutions. Using software will out of
> necessity take non-zero time for non-zero efforts.
Non-zero delay is not the point, but a constant stream of data with at
most such a delay that the human doesn't recognize it.
For audio, the time the sound travels trough the air is already
non-neglectable. Still, existing Garbage Collectors in commonly
available Common Lisp implementations may introduce pauses that are
much longer than this. This may not even be the GC's direct fault,
i.e. the operating system may have pages swapped out that aren't used
by the working loop, but are swapped back in to scan them for GC-able
or object-pointing material.
>| It doesn't work that way when you produce contents that must be
>| synchronized with real events, i.e. audio for computer games can't
>| build buffers, the action of the player is supposed to be spiced
>| with a BOOM or whatever immediately.
> I'm so glad we're in this negative mood where the purpose of this
> whole crappy exercise is to find ways Lisp will fail. Is it any
> wonder Lisp has a bad reputation? Even Lispers you'd think would
> have an interest in finding ways to solve the goddamn problems waste
> their time finding cases where they think it won't work. I have to
> wonder: Are you this bad an engineer?
Well, I'm not an a negative mood, but feel free to differ.
This discussion is quite clear in what we think Lisp might have
problems with - a very limited set of applications - and anyone not
getting that is too stupid to write working software anyway, so why
bother?
I know of at least one Lisp system without GC and whatever Harlequin
came up with probably isn't stupid as well, so the whole discussion
isn't against Lisp per se.
>| In practice, C++ programs tend to do a lot of unneeded copying, but I
>| have to rate all of them into pilot errors, where programms use
>| objects in a way that copies them, while a copy-free mechanism would
>| be available to the programmer.
> And now, apologia for C++!? Christ. C++'s lack of GC has one very
> important consequence: It has to has to have an ownership protocol
> for its allocated objects. Ownership protocols are very interesting
> because the are extremely complex. Since the likelihood that they
> could be mastered by the kind of people who choose C++ is so close
> to zero only number theorists would find anything in that space, the
> easy way out is to create a new object with known owners and let the
> owner of the original take care of it. Writing C++ is bad enough,
> but copy-free C++ is a lot harder than consing-free Lisp.
Didn't we we just express the same thing?
If you look over Bjarne Stroustrup's C++ page, you will notice that
the 7th external link paragraph is to Boehm's GC, so at least the
clever C++ people seem to be more aware of the problem than their
language spec might imply.
Colin Walters
> * Colin Walters <wal...@cis.ohio-state.edu>
> | Even this won't work in all cases unless you have some guarantees from
> | the operating system (i.e. a real-time OS), no matter how big your
> | buffers.
>
> Oh, I see, the point of this exercise is to prove it impossible, not
> to actually solve any problems.
No. I was trying to point out that the real problem doesn't isn't
with Lisp; it is fact that a non-real-time OS could sometimes (for
example) swap out one of the processes the above poster suggested
using, and then maybe the original poster would blame Lisp for being
slow, when in fact it isn't.
William Deakin
> I thought that WITH-POWER-ON was meant to do that? But yeah, to do it
> *right* you probably you really need something like Scheme's
> CALL-WITH-POWER-ON (CALL/PO) but I think that implementing that really
> means you can never stack-allocate protons, which is a real problem,
> since it basically means all your programs are radioactive.
Did hear the report this morning on the radio that BNFL is currently
gearing up to accept 40 tonnes of SIOD code on which the testing
certification was falsified...
:)will
Robert Monfera
relies on a secondary heavyweight process (maybe a second Lisp image).
Presumably the auxiliary processing can be very simple, e.g., outputting
an array of bytes representing a bitmapped image. GC does not make a
synchronized input/output much more difficult at all.
Robert
ke...@compuserve.com
Have you used Alegro Common Lisp? How does it rate?
In article <396C0329...@cory.eecs.berkeley.edu>,
cs61a-ev <cs61...@cory.eecs.berkeley.edu> wrote:
> Richard James Panturis Giuly wrote:
> >
> > I want to use lisp to deal with processing a stream of video. Do you
> > think that's ridiculous, or can I process data with lisp almost as
> > quickly as a lower level language?
> >
> > --
> > rgi...@surfsouth.com
>
> Actually, I don't really know how fast your program must run.
>
> In my opinion, it is not ridiculous at all to expect good speeds with
> lisp programs, provided that you follow some implementation-dependent
> optimization techniques. I recommend CMU/CL which seems to permit the
> best speeds.
> The techniques for optimization are often easy to apply, even if there
> are many cases where I simply get lost (problems in understanding CMU/CL
> documentation).
>
> For instance, I developped with 4 comrades a small OpenGL subset in
> Common Lisp, using CMU/CL. We finally managed to get 1/4 of the speed of
> MesaGL in average, with the possibility to go up to 1/2 (university
> project in limited time). We really enjoyed using Common Lisp which
> gives so many convenient features and gives you a better view over your
> programs. This also permits more advanced optimizations.
>
> But if you really need high speed, maybe you should consider assembly
> code only for the time-consuming functions. Notice that you really can
> call such speedy functions from CMU Common Lisp, allying convenience of
> the language with optimal execution speeds. That is what companies
> (Naughty Dog, Nichimen) tend to do when they need terrific speeds for 3d
> engines, for instance...
>
> I hope I helped you, I'd really like to know if you will manage to make
> it in Common Lisp and how. My knowledge in CMU/CL optimization
> techniques needs to be completed...
>
> --
> ----------------------------------------------
> Jonathan BAILLEUL (bail...@emi.u-bordeaux.fr)
> Maitrise Informatique, Universite Bordeaux I
> Currently attending Berkeley Summer Session Courses CS61a/CS3
ke...@compuserve.com
for Windows?
In article <8kh5af$2op$1...@counter.bik-gmbh.de>,
crac...@counter.bik-gmbh.de (Martin Cracauer) wrote:
> Richard James Panturis Giuly <n...@spam.com> writes:
>
> >I want to use lisp to deal with processing a stream of video. Do you
> >think that's ridiculous, or can I process data with lisp almost as
> >quickly as a lower level language?
>
> Well, if it's real-time, you may get into conflict with the Garbage
> Collector.
>
> Looking at your previous posting, the first thing you need is to turn
> on maximum warning level on your C compiler. The second thing to do
> is to compile your code on CMUCL, even if your final version should
> run on a different Lisp. CMUCL compiler emit a lot of message that
> help you to make your code fast.
>
ke...@compuserve.com
In article <31724312...@naggum.net>,
Erik Naggum <er...@naggum.net> wrote:
> * Martin Cracauer
> | Well, if it's real-time, you may get into conflict with the Garbage
> | Collector.
>
> This is a persistent myth. Real time does not mean zero latency.
> With stream data, non-zero latency translates to buffering. You
> don't want to saturate the processor, anyway, so you have time to
> catch up after a GC.
>
> | There are no "real" problems with Common Lisp reaching what I'd call
> | "raw machine speed". Minor problems are that the declarations in
> | Lisp are ugly and hard to write, so that you loose some of Lisp's
> | advantage.
>
> So use Common Lisp to its advantage and write macros to help with
> the type declarations, then.
>
> | Using objects from classes in the classical sense in Lisp (CLOS) is
> | a lot slower than C++ or Objective-C, but for raw speed you don't
> | want that anyway.
>
> Another persistent myth. Properly declared, CLOS is not at a
> disadvantage compared to C++, which does a lot of unnecessary
> copying due to its lack of garbage collection. It takes more time
> than C++ aficionados are willing to admit.
>
> If CMUCL still uses PCL, it should be noted that a native CLOS is
> much, much more efficient than PCL.
>
> #:Erik
> --
> If this is not what you expected, please alter your expectations.
>
Fernando Rodríguez
<ke...@compuserve.com> escribió en el mensaje
news:8kmcc5$pla$1...@nnrp1.deja.com...
> Can you please spell out CMUCL and point me at where I can download it
from
> for Windows?
There's no port to win32. For win32, you have: Xanalys, ACL, Corman Lisp
and CLisp
Martin Cracauer
>Can you please spell out CMU/CL and point me at where I can download it from?
Please read the FAQ for this newsgroup or www.lisp.org. CMUCL is on
http://www.cons.org/cmucl/
>Have you used Alegro Common Lisp? How does it rate?
You are sure the speed of the contained CLOS is the most urgent
concern?
Erik Naggum
| So what free or under $200 lisp for Windows would you recommend?
I wouldn't -- I recommend paying people in relative proportion to
the values they provide me, and "free" means I don't get any value,
and under USD 200 means you're moving in the mass market. I don't.
I recommend Franz Inc's Allegro CL Lite for Windows, which is a
trial edition (a full-featured "demo" version), and which you get
free of charge, but can't use for any money-making endeavors.
Eugene Zaikonnikov
> So what free or under $200 lisp for Windows would you recommend?
Depedns on your preferecnes. Corman Lisp is priced exactly $200 for
commercial use, and free for non-profit use. It also has some kind of
IDE, COM interface and ability to write inline assembler code.
Allegro CL Lite is a version of full-blown ACL with nice Delphi-like
GUI builder, but with limited heap size, though it is sufficient for
educational purposes.
Xanalys' LispWorks Personal ed. is a trial of their commercial
product, with limitation on heap size and usage time. It has
Emacs-like IDE, so if you are into The Only True Editor you may like
it.
CLISP is free in GPL sense, but don't expect from it any bells and
whistles of the commercial versions. It also can't compile to native
code, unlike the previous options.
--
Eugene.
Googleplexation is NOT an option!
> Another persistent myth. Properly declared, CLOS is not at a
> disadvantage compared to C++, which does a lot of unnecessary
> copying due to its lack of garbage collection. It takes more time
> than C++ aficionados are willing to admit.
That's Erik, the language bigot, talking out of his arse again. It
seems he knows just enough about C++ to think he's an expert, but not
enough to comment intelligently about it.
Paolo Amoroso
<cs61...@cory.eecs.berkeley.edu> wrote:
> For instance, I developped with 4 comrades a small OpenGL subset in
> Common Lisp, using CMU/CL. We finally managed to get 1/4 of the speed of
What about adding a page with some info about your project to the CLiki
site?
Everybody has write access.
Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/
Reini Urban
>> it from for Windows?
Fernando Rodríguez wrote:
>There's no port to win32. For win32, you have: Xanalys, ACL, Corman Lisp
>and CLisp
he didn't ask for common lisp only, so there're a lot more.
see
www.lisp.org for common lisp and other lisps
www.schemers.org for scheme
--
When C++ is your hammer, everything looks like a thumb. Steven M. Haflich
Fernando Rodríguez
"Googleplexation is NOT an option!" <deja...@my-deja.com> escribió en el
mensaje news:8knkhl$o1t$1...@nnrp1.deja.com...
Flame fests are a lot more fun when you actually sign your messages; trust
me. ;-)