Performance Comparison C++/Eiffel
Dietmar Wolz
For my PHD-thesis I developed an algorithm computing the colimit of
a signature diagram (a construction from category theory which is
useful for parametrization concepts in specification/programming
languages and for graph transformations).
The implementation was done both in Eiffel and C++, so I can present a
performance comparison related to this special algorithm. I was
very suprised about the performance of SmallEiffel 0.9.1 +
Boehm-Demers-Weiser garbage collection.
The Eiffel program consists of 13 Classes where one (dynamic arrays, inherits
from ARRAY[G]) was adapted to the different compilers for performance
optimization. The C++ program uses the same algorithm, a similar
structure and is based on the Standard Template Library.
I tested
a) gnu g++ 2.7.2 with STL (from gnu libg++2.7.1)
b) gnu g++ 2.7.2 with STL (commercial STL from Object Space)
c) ISE ebench 3.3.7 finalize, no precondition test, no garbage collection,
array optimization, inlining, no precompiled libraries
d) ISE ebench 3.3.7 finalize, no precondition test, with garbage collection,
array optimization, inlining, no precompiled libraries
e) SmallEiffel 0.91 (INRIA), boost, no garbage collection
f) SmallEiffel 0.91 (INRIA), boost, with Boehm-Demers-Weiser
g) Sig Eiffel 1.3S no garbage collection
h) Sig Eiffel 1.3S with garbage collection
i) Tower Eiffel 1.5.1 no garbage collection
j) Tower Eiffel 1.5.1 with garbage collection
I tested on a LINUX (kernel 2.0.12, gcc 2.7.2) Pentium P5/200 with
512 PB-Cache and 192 MB EDO-RAM with a diagram containing 2 million symbols.
run time compile time memory in MByte code size in kByte
a) 18 s 25 s 133 67
b) 27 s 28 s 170 86
c) 84 s 145 s 154 184
d) 154 s 145 s 83 184
e) 20.5 s 9 s 96 24
f) 25.5 s 9 s 85 51
g) 48 s 11 s 105 74
h) 90 s 11 s 75 74
i) 21 s 65 s 115 114
j) I got a SEGVIO, perhaps my fault due to an installation problem
The C++ program does memory management by hand, but there are some leaks. So you
cannot compare the memory usage between Eiffel and C++, only between the
different compilers.
The effect of garbage collection can better be seen in a second test, where
I called the algorithm 20 times in a loop for a smaller diagram (55000 Symbols):
run time compile time memory in MByte code size in kByte
a) 8.7 s 25 s 28 67
b) 14.8 s 28 s 12.4 86
c) 50.0 s 145 s 94 184
d) 57.3 s 145 s 4.3 184
e) 12.3 s 9 s 61 24
f) 12.5 s 9 s 4.4 51
Eiffel clearly needs a standardized base library, optimized both for
abstraction and efficiency similar to the STL for C++. Then you could develop
with ISE ebench and finalize with a more efficient compiler producing code
which can really compete with C++.
It follows a small description, how I used the Boehm-Demers-Weiser
(ftp://ftp.parc.xerox.com/pub/gc/gc4.10.tar.gz) together with
SmallEiffel 0.9.1 (ftp://ftp.loria.fr/pub/loria/genielog/SmallEiffel)
Something similar should be added to the SmallEiffel-FAQ.
1) make gc.a, move to /usr/lib as libgc.a
2) move gc.h the to working directory and add the following lines:
#define malloc(n) GC_malloc(n)
#define calloc(m,n) GC_malloc((m)*(n))
#define realloc(p,n) GC_realloc((p),(n))
#define free(p) GC_free(p)
3) instead of calling "compile -o executable -boost -O root make"
call now "compile -o executable -boost -O root make -lgc -include gc.h
--
Dietmar Wolz E-Mail: die...@cs.tu-berlin.de
Technische Universitaet Berlin Tel: (030) 314-73137
FB 13 / Sekr. FR 6-1 / Raum FR 6026 Fax: +49-30-314-23516
Franklinstr. 28/29 10587 Berlin Germany
Dominique Colnet
In article <4v1fk7$k...@news.cs.tu-berlin.de>, die...@cs.tu-berlin.de (Dietmar Wolz) writes:
|> For my PHD-thesis I developed an algorithm computing the colimit of
|> a signature diagram (a construction from category theory which is
|> useful for parametrization concepts in specification/programming
|> languages and for graph transformations).
|>
|> The implementation was done both in Eiffel and C++, so I can present a
|> performance comparison related to this special algorithm. I was
|> very suprised about the performance of SmallEiffel 0.9.1 +
|> Boehm-Demers-Weiser garbage collection.
In your test, are arrays of C++ program extendible as ARRAY of SmallEiffel ?
Have you also use the gcc -O3 ?
|> Eiffel clearly needs a standardized base library, optimized both for
|> abstraction and efficiency similar to the STL for C++. Then you could develop
|> with ISE ebench and finalize with a more efficient compiler producing code
|> which can really compete with C++.
... we are all waiting for a **good** ELKS definition :-)
|>
|> It follows a small description, how I used the Boehm-Demers-Weiser
|> (ftp://ftp.parc.xerox.com/pub/gc/gc4.10.tar.gz) together with
|> SmallEiffel 0.9.1 (ftp://ftp.loria.fr/pub/loria/genielog/SmallEiffel)
|> Something similar should be added to the SmallEiffel-FAQ.
|>
|> 1) make gc.a, move to /usr/lib as libgc.a
|> 2) move gc.h the to working directory and add the following lines:
|> #define malloc(n) GC_malloc(n)
|> #define calloc(m,n) GC_malloc((m)*(n))
|> #define realloc(p,n) GC_realloc((p),(n))
|> #define free(p) GC_free(p)
|> 3) instead of calling "compile -o executable -boost -O root make"
|> call now "compile -o executable -boost -O root make -lgc -include gc.h
I will add this in the -0.90 SmallEiffel release.
Thanks ... for the publicity.
--
------------------------------------------------------------
Dominique COLNET -- Talk Eiffel with SmallEiffel Talk Eiffel
C.R.I.N. (Centre de Recherche en Informatique de Nancy)
POST: CRIN,BP 239,54506 Vandoeuvre les Nancy Cedex,FRANCE
EMAIL: col...@loria.fr VOICE:+33 83593079 FAX:+33 83413079
------
------ READ_ME file of SmallEiffel is append ---------------
-- Part of SmallEiffel -- Read DISCLAIMER file -- Copyright (C)
-- Dominique COLNET and Suzanne COLLIN -- col...@loria.fr
--
-- SmallEiffel is not Smalltalk. D.Colnet 10/2/94
-- Is it still Eiffel ?
--
1 - What is SmallEiffel ? -
SmallEiffel is a free Eiffel compiler distributed under the
terms of the GNU General Public License as published by the
Free Software Foundation.
You can download SmallEiffel at :
ftp://ftp.loria.fr/pub/loria/genielog/SmallEiffel
2 - Brief Description of SmallEiffel -
SmallEiffel is the fruit of a research project done at CRIN
(Centre de Recherche en Informatique de Nancy).
SmallEiffel is intended to be a complete, though small and very
fast, free Eiffel compiler.
SmallEiffel is already used by students of the University Henri
Poincare' at Nancy (FRANCE).
We are using Eiffel as a first langage for teaching OOP
since 1990 (SmallEiffel is used since september 1995).
SmallEiffel is already portable on UNIX, Linux, BSD, HP-UX,
IRIX, XENIX, DOS, OS2, WINDOWS 95, Macintosh NeXT and Amiga.
Current distribution includes a compiler (command 'compile'),
a pretty printer (command 'pretty') and other tools (commands
`finder' and `clean').
An Eiffel interpretor (command 'eval') will be added to the
distribution in a few months.
3 - Contents of the SmallEiffel directory -
3.1 - directory misc : Miscellaneous files including the
INSTALL file.
3.2 - directory man : Miscellaneous help files.
3.3 - directory bin : The binary place including the `compile'
command.
3.4 - directory bin_c : C source code for tools of bin directory.
3.5 - directory lib_std : Standard Eiffel Library (string.e,
integer.e, general.e, array.e, ...).
3.6 - directory lib_show : Demonstration and examples Eiffel
programs.
3.7 - directory lib_test : Set of program to tests SmallEiffel
(see also directory misc/self_test).
4 - Bug Report -
Thank in advance for bug reports. Don't forget to have a look
first in the misc/NOT_YET_IMPLEMENTED file.
Please, try to isolate the bug as much as possible.
The best is to be able to create a single file, as small a
possible (which include the bug).
D.Colnet - col...@loria.fr
Roger Browne
Dietmar Wolz writes:
> For my PHD-thesis I developed an algorithm computing the colimit of
> a signature diagram ... in Eiffel and C++, so I can present a
> performance comparison related to this special algorithm...
Thank you for posting this material, which I'm sure many people
found interesting.
> The C++ program does memory management by hand, but there are
> some leaks. So you cannot compare the memory usage between
> Eiffel and C++, only between the different compilers.
Yet in another sense it provides a very meaningful comparison of
real-life memory usage (leaks and all).
> Eiffel clearly needs a standardized base library ... Then you could
> develop with ISE ebench and finalize with a more efficient
> compiler producing code which can really compete with C++.
NICE announced a few years back that it had no current plans to
standardize the data structures library, but perhaps in time
market forces will do this anyway.
There is another approach that would enable you to develop with
ebench and finalize with another compiler. SIG has published an
"OO Compiler API (OOCAPI)" paper (http://www.sigco.com/oocapi.htm)
that would enable, say, ebench to be used with any Eiffel compiler.
It requires a few changes to the way of thinking about Eiffel
software development, but to my way of thinking they are all
desirable changes (e.g. instead of specifying every cluster in
a LACE file for every project, one "installs" precompiled libraries and
simply names the libraries used by each project).
If the vendors were to embrace this paper, the interoperability of
Eiffel tools and compilers would represent a significant advance
for software development.
Regards,
Roger
--
-- Roger Browne, 6 Bambers Walk, Wesham, PR4 3DG, UK | Ph 01772-687525
-- Everything Eiffel: compilers/libraries/publications | +44-1772-687525
Dietmar Wolz
> In your test, are arrays of C++ program extendible as ARRAY of SmallEiffel ?
I used STL-vectors, which are even more powerful than Eiffel-arrays:
a) there is a push_back routine which is more intelligent as
force(x,upper+1) (except for the ISE base lib),
because it resizes the array by a certain factor.
This behaviour not supported by SmallEiffel, Tower and Sig.
Therefore I introduced class vector, see below.
b) There are standard iterators available for vectors, tables, lists,
sets, etc.., which allow to substitute for instance vectors by
lists using a single typedef (in code restricted to sequential
access). I used this feature heavily by comparing the efficiency of
different data representations in my algorithm.
class VECTOR[ELEM]
inherit ARRAY[ELEM]
redefine out
end
...
push_back(element: ELEM) is
local new_upp, new_cap: INTEGER
do
--force(element,upper+1)
--lets try a more efficient version:
new_upp := upper+1
if capacity = 0 then
with_capacity(16,0)
else
if capacity <= new_upp-lower then
new_cap := 2*capacity
resize(lower,lower+new_cap)
capacity := new_cap
end
end
upper := new_upp
put(element,upper)
end -- push_back
...
I did this kind of optimization with every Eiffel-Compiler. Without it the
speed difference to C++ would be much larger, and ISE ebench would
produce the fastest Eiffel-code.
> Have you also use the gcc -O3 ?
I used the pentium optimized gcc 2.7.2p8 with
options -mpentium -O6 -fswap-for-agi -frisc-const -fschedule-stack-reg-insns
both for the SmallEiffel compiler and for my algorithms. This produces
code which is about 15% faster than with -O3 (only for SmallEiffel
compiled programs).
Brian Strelioff
How about posting the source so we can perform more comparisons?
(Including make files, and which BDW options are in effect)
Also, it would be nice to get Tower 2.0 numbers as the
Tower 1.5.1 run that completed came pretty close to the C++
numbers. Also there are some Eiffel specific capabilities
(exception trace back in particular) that may impose
significant run time overhead, and that may (or may not)
be contributing to the different numbers you are seeing.
SE clearly provides a good Eiffel implementation, and your
hooks to utilize BDW gc make the transition from non-gc
to gc extremely easy. One thing to also consider: recompiling
existing C libraries with similar changes (say X11, Motif, ...)
yields gc support that goes far beyond other Eiffel implementations
which are, by design, restricted to gc only over Eiffel memory.
Another item worth pointing out is that BDW gc is multi-thread safe
on some implementations, so again SE+BDW yields a more advanced
Eiffel implementation than many commercial Eiffel implementations
(Yes, SE needs some work to be thread-safe itself) in this area.
I have done similar exploratory work with BDW & Geodesic/GreatCircle
with similar results with respect to the viability of SE.
Dietmar Wolz
Brian Strelioff <pp00...@interramp.com> writes:
>How about posting the source so we can perform more comparisons?
>(Including make files, and which BDW options are in effect)
The sources will be available when my PHD is finished (in 2 or 3 months).
Unfortunately I have no access to Tower 2.0. I am very much interested
in other comparisons, because my application is for sure not
representative. I expect, that things look better for ISE with larger
applications.
Dominique Colnet
SmallEiffel run time may be better using the new option -no_split added
in release -0.90.
Using -no_split, only one C file is created. Thus, "gcc -O3" can often
do some more inlining.
With SmallEiffel -0.90, the best way to finalize an application is now :
compile root -boost -no_split -O3
Perhaps, SmallEiffel run time is now as fast as C++ run time ?
May be Dietmar Wolz can re-run his benchmark ?
Please let us now :-)
Thanks in advance.
Dietmar Wolz
After my last posting 3 weeks ago I redesigned my algorithm used for
the test, such that there is more or less a one to one correspondence
between the Eiffel and the C++ classes rsp. features. For C++ I use the STL
(Standard Template Library), for Eiffel I had to define my own
iterator classes, because there is no unique realization in
the different Eiffel libraries. Some Eiffel garbage collectors have problems to
handle the frequent iterator creations efficiently, so I tested additionally
an alternative implementation which doesn't use iterators.
Other changes:
1) SmallEiffel version 0.90 appeared, which is faster
and contains several other improvements.
2) The C++ version now contains no memory leaks, so that you
can compare memory usage between Eiffel and C++
3) I tested C++ with the Boehm-Demers-Weiser garbage collection
(I removed partly the hand-coded memory management)
For my PHD-thesis I developed an algorithm computing the colimit of
a signature diagram (a construction from category theory which is
useful for parametrization concepts in specification/programming
languages and for graph transformations).
The implementation was done both in Eiffel and C++, so I can present a
performance comparison related to this special algorithm.
The Eiffel program consists of 13 Classes where one (dynamic arrays, inherits
from ARRAY[G]) was adapted to the different compilers for performance
optimization. The C++ program uses exactly the same algorithm, the same
classes and features. Dynamic arrays are realized by the STL-vector class.
I tested:
a) gnu g++ 2.7.2 with STL (from gnu libg++2.7.1), hand coded memory management
b) gnu g++ 2.7.2 with STL (from gnu libg++2.7.1), Boehm-Demers-Weiser
c) gnu g++ 2.7.2 with STL (commercial STL from Object Space), hand coded gc
d) ISE ebench 3.3.7 melt, precondition test,
array optimization, with precompiled libraries,
garbage collection, no iterators used
e) ISE ebench 3.3.7 finalize, no precondition test,
array optimization, inlining, no precompiled libraries,
garbage collection, iterators used
f) ISE ebench 3.3.7 finalize, no precondition test,
array optimization, inlining, no precompiled libraries,
garbage collection, no iterators used
g) ISE ebench 3.3.7 finalize, no precondition test,
array optimization, inlining, no precompiled libraries,
no garbage collection, iterators used
h) ISE ebench 3.3.7 finalize, no precondition test,
array optimization, inlining, no precompiled libraries,
no garbage collection, no iterators used
i) SmallEiffel 0.90 (INRIA), -boost, -no_split,
with Boehm-Demers-Weiser, iterators used
j) SmallEiffel 0.90 (INRIA), -boost, -no_split,
with Boehm-Demers-Weiser, no iterators used
k) SmallEiffel 0.90 (INRIA), -boost, -no_split,
no garbage collection, iterators used
l) SmallEiffel 0.90 (INRIA), -boost, -no_split,
no garbage collection, no iterators used
m) Sig Eiffel 1.3S, -O, garbage collection, iterators used
n) Sig Eiffel 1.3S, -O, garbage collection, no iterators used
o) Sig Eiffel 1.3S, -O, no garbage collection, iterators used
p) Sig Eiffel 1.3S, -O, no garbage collection, no iterators used
q) Tower Eiffel 1.5.1 no garbage collection, iterators used
r) Tower Eiffel 1.5.1 no garbage collection, no iterators used
s) Tower Eiffel 1.5.1 with garbage collection (resulted in a SEGVIO)
I tested on a LINUX (kernel 2.0.17, gcc 2.7.2p8 pentium version),
Pentium P5/200 with 512 PB-Cache and 192 MB EDO-RAM with compile
options -O6 -fswap-for-agi -frisc-const -fschedule-stack-reg-insns
(doesn't work with the official gcc2.7.2).
The test computes in a loop 12 times the colimit of a signature diagram
containing 50000 symbols.
run time compile time memory in MByte code size in kByte
a) 4.9 s 12 s 5.2 32
b) 9.3 s 14.5 s 9.9 76
c) 8.6 s 17 s 6.6 50
d) 1404 s 15 s 8.5 --
e) 234 s 165 s 6.8 192
f) 219 s 165 s 6.6 192
g) 42.8 s 165 s 108 192
h) 31.0 s 165 s 95 192
i) 8.9 s 13.5 s 12.0 60
j) 7.4 s 13.5 s 10.2 59
k) 8.3 s 13 s 110 33
l) 7.0 s 13 s 99 32
m) 424 s 12 s 11.2 74
n) 39.5 s 12 s 17.0 74
o) 43.2 s 12 s 214 74
p) 36.0 s 12 s 192 74
q) 10.6 s 65 s 78 119
r) 7.5 s 65 s 75 119
s) I got a SEGVIO, perhaps due to an installation problem
Comments
1) There is a close competition between C++, SmallEiffel and Tower.
Unfortunately I couldn't test Towers garbage collection.
2) With automatic garbage collection there is essentially no difference
in efficiency between SmallEiffel and C++ + STL.
3) The Boehm-Demers-Weiser seems to be superior to the other
garbage collection implementations. The results for Sig Eiffel
and ISE ebench are not very impressive (see e) and m)), factor 50-100
is hard to accept. Good programming style using abstract design
patterns like iterators should not be punished by bad efficiency.
4) C++ with hand coded memory management is the most efficient solution
for my application.
5) The code used for this test will be available when my PhD-thesis is
finished (hopefully during the next few months) .
Dietmar Wolz
Changes to the last posting:
-- Joachim Durchholz restructured the table for better readabiblity, thanks.
-- SmallEiffel 0.89 is available, now I use no longer class ARRAY, but
FIXED_ARRAY from their new library. This results in further speed
improvement.
-- The JAVA port of the colimit algorithm is finished. The figures will
be more interesting when kaffe (ftp.sarc.city.ac.uk/pub/kaffe)
finally supports just-in-time compilation. Normally the original interpreter
from SUN (java) is ca. by factor 2 slower as kaffe. But java has an
option to define the maximal heap size, which I have used to improve
execution speed. Because JAVA doesn't support generics (templates), they
are replaced by dynamic type checking in the code. This means, you
cannot expect the same performance as for EIFFEL or C++. But Factor 10
between SmallEiffel and compiled JAVA indicates some room for
optimization.
There is almost an one to one correspondence between the
EIFFEL, C++ and JAVA classes + features in the code.
The program consists of 13 Classes and is compiled and executed on a
LINUX (kernel 2.0.17, gcc 2.7.2p8 pentium version), Pentium P5/200
with 512 PB-Cache and 192 MB EDO-RAM.
It performs a construction from category theory, the computation of
the colimit of a signature diagram containing ca. 20000 symbols. The
computation is repeated 12 times in a loop to test the garbage collection.
The colimit construction can be used, for instance, to implement
structuring operations on parameterized specification languages.
(see D.~E. Rydeheard and R.~M. Burstall, Computational Category Theory,
International Series in Computer Science. Prentice Hall, 1988).
Compilers
g++: gnu g++ 2.7.2
g++ 1: with gnu STL from libg++ 2.7.1
g++ 2: with commercial STL from Object Space
ise: ISE ebench 3.3.7 with array optimization
ise 1: melt, precondition checks on, precompiled libraries, no inlining
ise 2: final, precondition checks off, no precompiled libs, inlining
se: SmallEiffel 0.89 (INRIA), -boost, -no_split
(ftp.loria.fr/pub/loria/genielog/SmallEiffel)
twr: Tower Eiffel 1.5.1
jav: SUN javac 1.0.1 compiled with j2c + Boehm-Demers-Weiser
/ SUN java bytecode interpreter
kaf: SUN javac 1.0.1 compiled with j2c + Boehm-Demers-Weiser
/ kaffe-0.5p3 bytecode interpreter
j2c: j2c JAVA to C compiler from Todo Software
(ftp.webcity.co.jp/pub/andoh/java/j2c-beta4.tar.gz)
Garbage Collectors:
man: Manual garbage collection
bdw: Boehm-Demers-Weiser (ftp.parc.xerox.com/pub/gc/gc4.10.tar.gz)
ise, sig, twr: Built-in GC from vendor
jav, kaf: Built-in GC, but compiler uses Boehm-Demers-Weiser
GC Iterators Run Compile Memory Code
Time Time Usage Size
(sec) (sec) (MB) (kB)
g++ 1 man yes 4.9 12 5.2 32
g++ 1 bdw yes 9.3 14.5 9.9 76
g++ 2 man yes 8.6 17 6.6 50
ise 1 bdw no 1404 15 8.5 --
ise 2 ise yes 234 165 6.8 192
ise 2 ise no 219 165 6.6 192
ise 2 --- yes 42.8 165 108 192
ise 2 --- no 31.0 165 95 192
sig sig yes 424 12 11.2 74
sig sig no 39.5 12 17.0 74
sig --- yes 43.2 12 214 74
sig --- no 36.0 12 192 74
se bdw yes 6.1 12 7.9 57
se bdw no 5.2 12 6.7 56
se --- yes 6.3 12 65.7 30
se --- no 5.3 12 54.6 29
twr twr SEGVIO, perhaps due to an installation problem
twr --- yes 10.6 65 78 119
twr --- no 7.5 65 75 119
jav jav yes 528 0.9 102 16 (bytecode)
kaf kaf yes 4364 0.9 43 16 (bytecode)
j2c bdw yes 53.4 38 18.5 243
Compile denotes overall compile time (from source to executable).
Jeffrey W. Stulin
I would like to publicly thank Dietmar for keeping us informed on his
performance comparison work. I find the results both fascinating and
useful. I hope that, despite their silence, the commercial Eiffel vendors
(ISE, SIG, TOWER) are paying attention to Dietmar's results and will
publicly comment on why, despite some of their claims, their product does
not perform anywhere near C/C++ performance levels.
I am also curious as to why SmallEiffel DOES seem to perform as well as
C/C++. Is SmallEiffel missing some important Eiffel feature that the
others have? Or is SmallEiffel the only vendor who takes performance
seriously? Or is the example somehow unfair to Eiffel? (I don't believe
this one because my benchmarks have also shown about a 5X performance
penalty for using ISE Eiffel in array intensive situations).
Thanks again Dietmar. Please keep up the good work.
Jeffrey W. Stulin
j...@tiac.net
Dominique Colnet
In article <51i7q8$a...@news-central.tiac.net>, j...@tiac.net (Jeffrey W. Stulin) writes:
|>
|>
|> I would like to publicly thank Dietmar for keeping us informed on his
|> performance comparison work. I find the results both fascinating and
|> useful. I hope that, despite their silence, the commercial Eiffel vendors
|> (ISE, SIG, TOWER) are paying attention to Dietmar's results and will
|> publicly comment on why, despite some of their claims, their product does
|> not perform anywhere near C/C++ performance levels.
|>
|> I am also curious as to why SmallEiffel DOES seem to perform as well as
|> C/C++. Is SmallEiffel missing some important Eiffel feature that the
release -0.89 without loosing performances.
I think that SmallEiffel is the only one Eiffel compiler with a serious
type inference mechanism.
|> others have? Or is SmallEiffel the only vendor who takes performance
|> seriously? Or is the example somehow unfair to Eiffel? (I don't believe
|> this one because my benchmarks have also shown about a 5X performance
|> penalty for using ISE Eiffel in array intensive situations).
with a lot of dinamic dispatch calls, SmallEiffel would be better.
|> Thanks again Dietmar. Please keep up the good work.
Yes. Thanks again Dietmar
Dietmar Wolz
>I would like to publicly thank Dietmar for keeping us informed on his
>performance comparison work. I find the results both fascinating and
>useful. I hope that, despite their silence, the commercial Eiffel vendors
>(ISE, SIG, TOWER) are paying attention to Dietmar's results and will
>publicly comment on why, despite some of their claims, their product does
>not perform anywhere near C/C++ performance levels.
>I am also curious as to why SmallEiffel DOES seem to perform as well as
>C/C++. Is SmallEiffel missing some important Eiffel feature that the
>others have? Or is SmallEiffel the only vendor who takes performance
no, TOWER code is almost as fast as SmallEiffel.
>seriously? Or is the example somehow unfair to Eiffel? (I don't believe
The C++-STL is optimized for performance, that may cause some unfairness. For
this reason I made an optimized Eiffel version of my code replacing iterator
creations by simple index counting. In some sense this is unfair to C++ because
here I can use the more abstract concept (iterators) without a performance
penalty.
>this one because my benchmarks have also shown about a 5X performance
has someone else similar results ?
>penalty for using ISE Eiffel in array intensive situations).
>Thanks again Dietmar. Please keep up the good work.
>Jeffrey W. Stulin
>j...@tiac.net
Thanks Jeffrey,
but I want to add some comments to clarify things:
1) related to TOWER
As you can read from another post:
> Ulrich Windl: Walnut Creek Eiffel CD (1/96) 03 Sep
> 1996 12:32
> Is anybody using the Walnut Creek Eiffel CD on a recent Linux system?
> I'd like to exchange some experience (and maybe patches) to get things
> running.
there are a lot of problems to get the TOWER-Demo running. The compiler
crashes occasingly and the problems with gc are not application specific.
TOWER should offer an actualized demo version as soon as possible.
I am shure that the problems are related to Linux library compatibility
issues and that the actual TOWER version will work fine.
2) The performance gain for SmallEiffel from 0.90 to 0.89 is to 80% due to
SmallEiffel improvements, only 20% are caused by the usage of FIXED_ARRAY
(which actually as a special dynamic array, only with fixed lower bound).
3) Keep in mind that compared to JAVA all Eiffel implementations are ok.
(that SmallEiffel code is 10 times faster as compiled JAVA is another story)
4) Normally I use ISE ebench as syntax checker, because sometimes the error messages
from SmallEiffel are not very helpful, specially with generics. Keep this
in mind when comparing the compile times.
5) The original intention of my posting was, to start a discussion on efficiency
issues related to object oriented programming languages. My results are not
very representative, because my application mainly is realized using dynamic
arrays. There are some differences between my expieriences, and what I had
expected from the advertisements. I want to know whether others
share this disappointment with other applications.
6) As you can see from the success of JAVA bytecode interpretation, performance
seems no longer a problem at all on todays fast processors. In GUI-applications
most time critical code is called from some efficiently implemented library.
Jeffrey W. Stulin
In article <51jl1t$a...@news.cs.tu-berlin.de>, die...@cs.tu-berlin.de+
says...
Dietmar:
.....
>5) The original intention of my posting was, to start a discussion on
efficiency
> issues related to object oriented programming languages. My results
are not
> very representative, because my application mainly is realized using
dynamic
> arrays. There are some differences between my experiences, and what
I had
> expected from the advertisements. I want to know whether others
> share this disappointment with other applications.
......
I think that you stated the issue clearly when, in an earlier post, you
lamented that there should not be a large performance penalty for using
abstraction.
In my benchmarks with ISE Eiffel I learned that ISE's compiler can
perform array optimizations in limited situations, but that these
situations are so specialized that almost any use of abstraction would
break them, causing array operations to be performed by function calls
resulting an a 5x performance penalty.
My benchmark was an attempt to find the VERY WORST case for Eiffel as
compared with C (e.g., a program with little need for abstractions but
lots of low level array access).
I state for the record, however, that in the "real" programs I have
developed ISE Eiffel has so far been, in every case, more than fast
enough. Also, I have found that Eiffel's promises of better software to
be correct. Thus, for my uses, the "cost" of Eiffel's abstraction
capability has been well worth the price, and I am a big, big fan of
Eiffel and ISE products.
Despite my happiness with ISE Eiffel this question does come to mind:
Can the "cost" of abstraction, and all the other goodies we get from
Eiffel, be zero.
Your benchmarks for SmallEiffel's and Towers's compilers seem to
indicate that the answer may be "yes".
While you state that your results may not be representative, it is
probably true that your benchmark, like mine, emphasizes Eiffel's weak
points and if Eiffel can do well in these application areas then it will
do well in most application areas.
I hope that your continues posts will help the vendors of object
oriented languages (both Eiffel, Java and any others you intend to
explore) keep their eyes on the performance issue so those who insist on
promoting lower level languages will ultimately have one less argument
to use.
Jeffrey W. Stulin
j...@tiac.net
Steve Tynor
| useful. I hope that, despite their silence, the commercial Eiffel vendors
| (ISE, SIG, TOWER) are paying attention to Dietmar's results and will
| publicly comment on why, despite some of their claims, their product does
| not perform anywhere near C/C++ performance levels.
Please take a look at the numbers again. Don't lump all the commercial
compilers into one basket: TowerEiffel is shown to perform very
closely to C++. Had Dietmar chosen to use TowerEiffel's SIMPLE_ARRAY
as he chose to use SmallEiffel's FIXED_ARRAY (they are essentially the
same thing), I'm confident he would have observed even better
performance from TowerEiffel. Also note that TowerEiffel 2.0.0 (he's
using 1.5.1) introduced some new optimizations that can improve array
performance.
We are corresponding with Dietmar to attempt to understand why he is
having problems with our GC. We suspect that it might be related to
the fact that he's using a bleeding edge version of the Linux kernel
and libraries -- TowerEiffel has not been tested in that environment.
As far as analyzing what aspects of our compilation strategy are
responsible for the differences in performance, our hands are somewhat
tied since Dietmar has not released the sources to his benchmarks.
| others have? Or is SmallEiffel the only vendor who takes performance
| seriously? Or is the example somehow unfair to Eiffel? (I don't believe
I can assure you that Tower has always taken performance seriously!
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Programming needn't be like penance or oat bran.
Steve Tynor Email: Steve...@atlanta.twr.com
Tower Technology WWW: http://www.twr.com/
Bertrand Meyer
This is simply to mention that we cannot discuss any
benchmarks whose source code is not publicly available.
At this point, the comments published mean absolutely nothing.
We strongly object to the use of unsubstantiated figures
to libel carefully optimized products.
A typical example: claiming that ISE's compiler generates code
slower than another compiler's does not seem to be a very
interesting contribution when you read in the fine print
that the other compiler crashes ("I got a SEGVIO, perhaps my fault
due to an installation problem") when garbage collection
is turned on. Small detail indeed.
ISE we'll be glad to deliver a compiler generating code
faster than anything else, including hand-optimized
assembly programs. Sure, it will produce a SEGVIO,
but that will be your fault (an installation problem).
For more details on this special offer just write to us
and we will give you a good price.
In the meantime any discussions of efficiency should
be based on published benchmarks, not on someone claiming
that compiler X is faster. It is also common courtesy
to notify the vendors, so that they can look at the
results and see if, for example, their products have
been inadvertently misused.
Perhaps applying the basic techniques of rational analysis
would help arrive at meaningful results.
--
Bertrand Meyer, ISE Inc., Santa Barbara (California)
805-685-1006, fax 805-685-6869, <bert...@eiffel.com> - ftp://ftp.eiffel.com
Web home page: http://www.eiffel.com
Jared Richardson
Any possibility of releasing the source code so
coders familiar with each system can use optimize
the code for their own vendor's compiler?
I for one am very curious what can done with each
Eiffel when it is the only one you use and you are
more familiar with all of it's optimizations.
Just a thought...
Friedrich Dominicus
I understand that ISE isn't very happy about that not represenative
results. But You have the chance to publish an eiffel-test. And you
can publish the source code.
You don't have done it till now. Why?
So I the tests are not representative but interesting.
If you can do it better, do it.
Best regards
--
Friedrich Dominicus
e-mail: geschaeftl : fr...@imbdec1.bau-verm.uni-karlsruhe.de
privat : fr...@frown.inka.de
phillips
>I understand that ISE isn't very happy about that not represenative
>results. But You have the chance to publish an eiffel-test. And you
>can publish the source code.
published/discussed on this newsgroup, and the problem remains the same:
what does the test and its results have to do with the sort of systems
we are building today? In my (nicht so demuetige) opinion: very little!
A worthwhile test of a language would have to demonstrate:
1) The ease of how a problem can be solved
2) The ability to readily adapt as the requirements change
3) The reliability of the solution
...
99) Efficiency
My experience of using ISE compilers to build large commercial
applications is that performance issues have more to do with the
underlying services (databases, host comms) than the compiled code.
In addition, it is simply too much effort to build systems of the
same scale and reliability in C++ - not impossible, but you want
a life as well! Systems that we built in 1988 are still being used
and re-used today - equivalent C++ systems are already ‘legacy’
systems.
So how can you test such properties? Not easily in such a trite
manner as the tests presented - if software were really that easy,
it would be a hobby!
Pob hwyl,
Roy
Friedrich Dominicus
You're quite right with you opinion about measuring software. But that's not
the point here. It just a informative unrepresenative result of one small
system. But I think it nevertheless interesting.
I don't have seen any comparisons in the past 12-18 month. It's
the first time I see such results.
And I think one result is right. ISEs Compiler need a lot of memory
and hard disk space. I worked a short time with SIGs compiler and with
Towers Compiler. I get the impression that these don't need so much
space.
Maybe I'm wrong. But if we got more results in this news-group we'll
find out which compiler need less resources.
And if you've read my posting, you can seen that I just suggested some
more tests especially from other sides like the compiler manufactures.
But they havn't published anything. They only tell us: "We're quite as
fast as C or C++". Maybe there test aren't quite representative
too. But a lot of none representative tests will give a good
impression of the resource consumption from eiffel-compilers.
Nothing will be said about how good this programms are, here I agree
with you. Eiffel gives us one of the best languages to build reliable
software. But only if you know how to use all mechanismes Eiffel gives
you to write such programs.
Harald Lauer
In article <323ECE4E...@eiffel.com>,
Bertrand Meyer <bert...@eiffel.com> writes:
> This is simply to mention that we cannot discuss any
> benchmarks whose source code is not publicly available.
Dietmar said that he will publish the source code when his
dissertation is finished. While I also want to have a look at the
code, I can also understand his problem.
> At this point, the comments published mean absolutely nothing.
> We strongly object to the use of unsubstantiated figures
> to libel carefully optimized products.
Take it easy. I would probably also be upset if someone posts a
critique with "not-so-good-results" about my product. The point is
that Dietmar never meant to compare the different Eiffel vendors
(just look at the subject line). He is not responsible if the public
discussion moves to this topic.
[big snip]
While all your arguments are valid if Dietmars results are viewed as
actual benchmarks, there is no need for flaming Dietmar if his results
are just taken as a case study. He clearly stated that he has one
specialized application and wanted to do an Eiffel vs. C++ performance
test. I'd also like to see benchmark results, but case studies like
Dietmar's are equally important to me.
Please take the customers viewpoint. We have very limited information
about the performance of the different products (should I as a
customer trust the performance figures published by the company
producing the compiler ?), such that third party information is very
valuable. Customers are also (well, in most cases) not dumb. I know
that Dietmar's results have to be taken with a grain of salt - as it
is the case with all performance evaluations. For example: I never had
any problems with the garbage collector of Tower Eiffel.
The biggest surprise for me is the performance of SmallEiffel. And by
this I mean code efficiency as well as "turnaround times", ie. the
edit-compile-debug cycle. Impressive. These results make me as a
customer happy. SmallEiffel has its deficiencies (eg. incomplete
implementation), but it is improving and commercial Eiffel products
should - IMHO of course - at least be better than the best freeware
product.
This last sentence of course proves your point that these figures
reflect on your product, however the other Eiffel vendors took also a
few hits. I think it would be best if we all wait until Dietmar can
make the source code available.
Harald
--
Harald Lauer Your freedom ends
Wilhelm-Schickard Institut fuer Informatik somewhere between
Universitaet Tuebingen your fist and my nose.
URL: http://www.uni-tuebingen.de/uni/iic/Harald.Lauer.html
Paul F. Dubois
Friedrich Dominicus <fr...@imbalpha.bau-verm.uni-karlsruhe.de> wrote in
article <FRIDO.96S...@imbalpha.bau-verm.uni-karlsruhe.de>...
>
<snip>
>But they havn't published anything. They only tell us: "We're quite as
> fast as C or C++". Maybe there test aren't quite representative
> too. But a lot of none representative tests will give a good
> impression of the resource consumption from eiffel-compilers.
>
This thread has contained an implicit assumption that C++ is a known
reference point. It surely is not. In the Scientific Programming Department
of the journal Computers in Physics we are in the process of publishing a
series of articles on the speed of C++ for scientific programs. Between
compiler improvements and new coding techniques there has been a very
significant increase in performance on some simple benchmarks in just a
year or so. Some of the compiler improvements have been driven in part by
the creation of some small benchmarks (by Scott Haney) and their
publication.
Publication of benchmarks, and cooperation between the vendors and the
benchmark authors, can result in real improvements in compilers. I have
followed this thread with increasing frustration in that I could not see
the Eiffel coding being tested, so that I had no idea what was being
measured. I have no idea if the benchmark author is doing something clever
or something stupid. My view is that a benchmark only contributes if it
doesn't do something a reasonable practictioner of the language would know
is a performance problem, and if its mix of operations is typical of a
significant application area.
The benchmarks that Haney wrote for scientific C++ were very simple. It is
easy to see that a very simple benchmark can be misleading, but on the
other hand it can isolate a certain area of concern in the compiler. One
must combine simple benchmarks with an understanding of their limitations.
I also take it as given that we all understand that execution speed is just
one of many factors one must consider.
If you wish to have a speed reference it is safer to use Fortran, although
even with Fortran a certain amount of care must be taken in writing the
benchmark. Of course, some of the interesting object-oriented operations
are hard to express in Fortran. It should be F77, by the way, not F90,
which is significantly slower I understand.
By the way, the result of the C++ adventures has been that the abstraction
penalty for using a nice real or complex array class can be nearly
eliminated. Statements such as
z = x + y / sqrt (w*w + 1.0);
where x, y, z, and w are arrays, can result in the generation of virtually
the same one loop that a C or Fortran programmer would write. This is
possible because the C++ template facility differs from the Eiffel generic
facility in a interesting way. Whether it is possible to do this trick in
Eiffel is an interesting question.
A side remark: Eiffel is by definition a language which includes garbage
collection. Measuring how it does without garbage collection is measuring
the speed of some other language.
Always in the market for good articles about scientific programming,
Paul Dubois
Editor, Scientific Programming Department
Computers in Physics
phillips
Mae Friedrich <ri...@imbalpha.bau-verm.uni-karlsruhe.de> yn sgrifennu:
>You're quite right with you opinion about measuring software. But that's not
>the point here. It just a informative unrepresenative result of one small
>system. But I think it nevertheless interesting.
>I don't have seen any comparisons in the past 12-18 month. It's
>the first time I see such results.
There was a game implemented in several languages, if I remember correctly,
this was two or three years ago. Anyone got a reference for that?
>And I think one result is right. ISEs Compiler need a lot of memory
>and hard disk space. I worked a short time with SIGs compiler and with
>Towers Compiler. I get the impression that these don't need so much
>space.
It may be true that, for example, SmallEiffel (SE) is more efficient
in that respect. On the other hand, I wouldn't like to loose the
conveniance of Melting Ice, browsing, etc. in my development
environment. If, as the results suggest, SE produces more efficient
code, then an option would be to develop with the environment of your
choice, and `finally' finalise with SE - if SE implemented HASH_TABLE,
for example, I would have tried that today...
>But they havn't published anything. They only tell us: "We're quite as
>fast as C or C++". Maybe there test aren't quite representative
>too. But a lot of none representative tests will give a good
>impression of the resource consumption from eiffel-compilers.
The point I was trying to make is that in larger systems, I *feel*
that an Eiffel system is as efficient as a comparitive C++ system -
or even more efficient. This feeling, is however, worse than only
unrepresentative, it is subjective. But if one is to use performance
criteria as a basis for evaluating tools, then the criteria do need
to address the problem domain.
>Nothing will be said about how good this programms are, here I agree
>with you. Eiffel gives us one of the best languages to build reliable
>software. But only if you know how to use all mechanismes Eiffel gives
>you to write such programs.
Ah, but those mechanisms are few and powerful: I'm constantly amazed at
how quickly newbies pick-up Eiffel and make it there own (especially when
they come from an IT but non-OO background).
- Roy
Joachim Durchholz
phil...@indigo.ie wrote 18.09.96:
> A worthwhile test of a language would have to demonstrate:
> 1) The ease of how a problem can be solved
> 2) The ability to readily adapt as the requirements change
> 3) The reliability of the solution
> ...
> 99) Efficiency
No. Efficiency is as important as the rest. *None* of these issues can be
neglected.
To make a language successful, it must be able to span everything from low-
level operating system programming to GUI dialogs. One of the main reasons
C was so successful was that it didn't leave gaps as the earlier languages
did. MIS departments could reduce costs by throwing out Cobol and
Assembler and doing everything in C (less training cost, no language
barriers when reassigning programmers to new jobs, less cost of ownership
for development systems).
Any language hoping to replace C (or, nowadays, C++) for any market
segment must be able to replace it for any market segment.
As there *are* areas where efficiency is of prime importance, Eiffel
cannot afford to be inefficient. Of course, an efficiency difference of
10% between Eiffel and C++ doesn't interest anybody, as it's always hidden
behind statistic noise. Factors of two do matter a whole lot; wearing my
end-user hat, I don't want to lose half of my processor power to language
quirks!
Or a practical example from my daily work. I'm employed at a company that
produces and sells rolling bearings. When the Iron Curtain went down,
there was a lot of activities to get sales sub-companies up and running in
the now-open Eastern countries. Sales personnel was to get a laptop each
for doing paperwork and reporting results back to the main company. To
write the necessary software, two languages were chosen for review.
Contender 1 was Clarion for Windows (nobody in this newsgroup is expected
to have ever heard of it). Dirt cheap and fast. Has a design with a few
serious flaws, but being a RAD language it was considered good enough to
get some initial software up and running within a few months. A sure loser
according to the above criteria.
Contender 2 was OEL, a home-brewn OO language. Well-designed, but needed a
Pentium and 16 MB RAM (recommended 24 MB) to run. In addition, OEL is my
employer's idea of Ada, i.e. in the long term everything shall be written
in it. A sure winner, right?
Clarion won. MIS decided equipping several dozen salespeople with 6000-
dollar laptops was just too expensive. Clarion didn't require more than a
measly '386 at 44 MHz with 8 MB of RAM - this was still too expensive, of
course, but they didn't come up with anything that required less
resources.
> My experience of using ISE compilers to build large commercial
> applications is that performance issues have more to do with the
> underlying services (databases, host comms) than the compiled code.
That's the point. ISE Eiffel is better for large commercial application.
For anything else, other languages/development environments are considered
better. This niche isn't too bad (after all, MIS departments that need
this are prepared to pay lots of money to get it running), but it's a
niche. You leave out all of the remaining market. There is a huge market
of small companies out there; these buy entry-level IBM compatibles, and
these machines won't be upgraded just because the consultant says Eiffel
needed more RAM; instead, the customer will select a company that does the
programs in Clarion or C++.
There is another reason. Efficiency will impress executives. We all know
the price for the efficiency of C++ is high and comes in terms of
unreliable programs. But an MIS head thinks *his* programmers are good and
tends to underestimate the price. Efficiency is something that the manager
will see immediately, and it is something that he understands. Program
reliability is difficult to measure, and many executives don't really
understand such technical subtleties (I don't speak of Fortune 500
companies here).
Regards,
-Joachim
--
Looking for a new job. Paper resume available on request, WWW resume
available under http://www.franken.de/users/jhd/resume/index.html
Friedrich Dominicus
-- snip we're using eiffel ;-)
>
> >Nothing will be said about how good this programms are, here I agree
> >with you. Eiffel gives us one of the best languages to build reliable
> >software. But only if you know how to use all mechanismes Eiffel gives
> >you to write such programs.
>
> Ah, but those mechanisms are few and powerful: I'm constantly amazed at
> how quickly newbies pick-up Eiffel and make it there own (especially when
> they come from an IT but non-OO background).
Here I disagree with you. I've to maintain Eiffel-programms which have
been written from people who never have done oo-programming
before. I'm sorry to say that they havn't used any mechanism Eiffel
offers.
It's necessary to learn oo-programming. It seems that
oo-programming doesn't come naturally if you use Eiffel. But I have to
say that a lot of books have come out which show good
Eiffel-programming. I'm very keen on the second edition of
oo-software construction. I learned so much from the first
edition. But a quite long time has passed since then.
And I think even Mr Meyer have learned more about good
oo-programming. If he could show us his advances we'll know more.
So I think you got with Eiffel one of the best (the best?) tools for
oo-programming, but it doesn't come for free.
>
> - Roy
Dietmar Wolz
Bertrand Meyer <bert...@eiffel.com> writes:
>This is simply to mention that we cannot discuss any
>benchmarks whose source code is not publicly available.
>At this point, the comments published mean absolutely nothing.
>We strongly object to the use of unsubstantiated figures
>to libel carefully optimized products.
>A typical example: claiming that ISE's compiler generates code
>slower than another compiler's does not seem to be a very
>interesting contribution when you read in the fine print
>that the other compiler crashes ("I got a SEGVIO, perhaps my fault
>due to an installation problem") when garbage collection
>is turned on. Small detail indeed.
That was a LINUX library incompatibility problem, not a problem with
TOWER itself, as I said before. This was a demo version, not the
delivered product, it costs me nothing. Your comment is not fair.
>In the meantime any discussions of efficiency should
>be based on published benchmarks, not on someone claiming
You are right, we need benchmarks. It was my intention to start
a discussion about that issue, because I was disappointed about
the efficiency of some products (after reading their advertisements).
I never claimed nothing beside that my specific application performs
suprisingly different on different compilers, and the question arises,
if others share my experience. Perhaps someone has the time to
provide us with a proposal for an Eiffel benchmark suite ?
Dietmar Wolz
"Jared Richardson" <ja...@nando.net> writes:
>Just a thought...
There will be no release of the code before my PhD is finished (next months).
But the optimisations I used are not a secret:
1) I used all available Eiffel compiler optimisations which are well
documented in the distributions for SmallEiffel, TOWER, ISE and SIG,
nothing special.
2) I use a special pentium optimised gcc with most optimisations enabled
(-mpentium -O6 -fswap-for-agi -frisc-const -fschedule-stack-reg-insns)
3) My application (incremental colimit computation of signature diagrams)
uses dynamic arrays and the UNION-FIND set partition algorithm as
described in
Algorithms, Data Structures, and Problem Solving with C++ from Mark
Allen Weiss, ADDISON Wesley.
A previous version of my algorithm (in C++) can be found in:
Tool Design for Structuring Mechanisms for Algebraic Specification
Languages with Initial Semantics, in Springer LNCS 1130:
Recent Trends in Data Type Specification.
4) The only thing I have done for optimisation which is Eiffel-specific is,
that I have adapted the ARRAY class, such that there is a push_back
routine which increases the memory allocated for a dynamic array by
factor 2, if there is no more free space available. Not every
Eiffel library supports that behaviour which is necessary to get linear
complexity (otherwise it is quadratic). Otherwise ISE, SmallEiffel,
C++ and JAVA would have an unfair advantage.
5) The SmallEiffel people tried hard to adapt their library exactly to
my need: They provided me with a dynamic FIXED_ARRAY, that is a
dynamic array with fixed lower bound. SIMPLE_ARRAY from Tower
is not dynamic, so I cannot use it. There are some arguments against
using SmallEiffels FIXED_ARRAY: 1) it may be unfair to the other vendors
2) I have to maintain 2 versions of the code. I have to think about it.
With normal ARRAY the SmallEiffel version is about 8-10% slower, that
is still the best result compared to the others.
The advises I could give related to efficiency issues are:
1) Try different vendors, there may be large differences related to
efficiency. Push the vendors to provide us with a common
base library (comparable to the STL for C++, or to the JAVA library)
such that we can develop with one environment and finalize with another.
2) Try the the Boehm-Demers-Weiser garbage collector
(ftp://ftp.parc.xerox.com/pub/gc/gc4.10.tar.gz)
Even more important is of course the design of your algorithms itself,
but this is not language specific.
By the way, in the german unix magazine IX from 10/96 (it is out since
yesterday) there is a very interesting article called "Kalter Kaffe" -
"Cold Coffee" related to the drawbacks of JAVA. JAVA really is a step
back from EIFFEL, but it has this powerful common library.
James Mansion
I think the tone of this is most inappropriate given the postings it is
targetted at, and it reflects poorly on Dr Meyer and ISE too.
While it is certainly the case that using publicly available code is
'better', I don't think that the tone of this message is justified at
all.
Personally, on the basis of this, I will be happier to use a Tower
product than an ISE one.
James
Bertrand Meyer wrote:
>
> This is simply to mention that we cannot discuss any
> benchmarks whose source code is not publicly available.
> At this point, the comments published mean absolutely nothing.
> We strongly object to the use of unsubstantiated figures
> to libel carefully optimized products.
>
> A typical example: claiming that ISE's compiler generates code
> slower than another compiler's does not seem to be a very
> interesting contribution when you read in the fine print
> that the other compiler crashes ("I got a SEGVIO, perhaps my fault
> due to an installation problem") when garbage collection
> is turned on. Small detail indeed.
>
> ISE we'll be glad to deliver a compiler generating code
> faster than anything else, including hand-optimized
> assembly programs. Sure, it will produce a SEGVIO,
> but that will be your fault (an installation problem).
> For more details on this special offer just write to us
> and we will give you a good price.
>
> In the meantime any discussions of efficiency should
> be based on published benchmarks, not on someone claiming
Brian Strelioff
Dietmar Wolz wrote:
>
> [ several articles covering the development of a benchmark ]
Great job!! Keep up the good work.
Brian Strelioff
Dietmar Wolz wrote:
>
> [ several articles detailing the ongoing development of
> a benchmark in several "OO" languages, including results
> as things progress]
Great job! Will this information be captured somewhere for
future reference once the source code is released?
As someone who has also wrestled with multi-language benchmarking
(or even basic development portable to all "Eiffel" systems) I
find yout summaries not only very meaningful, but also very
typical of results I have seen with similar efforts for the
various Eiffel systems.
I would recommend that you and/or Tower see about benchmarking
the latest 2.0.* release, and it is very encouraging to see
the SmallEiffel tuning guided by your efforts.
If you are seriously including Java in your PHD project, then
there are some "free" evaluation implementations that you
could also make use of from Sun, Microsoft, and Symantec. Granted
none of them are for Linux, but if you hava sccess to other
platforms they may be worth exploring.
Keep up the good work, and keep the results flowing as best
you can!!!
Bertrand Meyer
I would like once again to protest against the
campaign of slander directed against ISE's compilers.
Publishing benchmarks "results" without the source code
is tantamount to publishing physics results which
no one else can reproduce. In other engineering disciplines
someone who did such a thing would quickly be asked
to apply the basic rules of scientific discourse or
choose another area of excellence.
On comp.lang.eiffel, it is the person who protests against
the carelessness of such irrational and potentially libelous
postings (namely me) who gets insulted and accused of using
"inappropriate tone". Pray, what kind of tone do you adopt when
someone publishes "results" that cast your
products in a bad light, and you have no way to verify
these results, let alone refute them?
While we are careful not to make exaggerated claims, the
published "benchmarks" are so out of touch with our own
measurements (some of which are available, together with
the source code of course, from ISE) that we suspect an
error, a poor use of our products, or worse. There is a
simple way to check: publish the source. If there is an error,
we'll signal it. If there is a simple way to make the code
faster, we'll point it out. If our compiler is indeed bad
on a certain scheme, will grind our teeth and make
it faster in the next release.
You can continue personal invective against me but that
will not make up for technical argument.
I also wish that the other compiler implementers would
refuse to comment on benchmarks that do not include
the source code. First, there is the simple ethical rule.
Second, although a slick salesman might convince a customer
or two on the basis or unverified or misleading information,
one cannot fool any really sophisticated customer for very long.
And finally, a benchmark that turns out to be both accurate and
unfavorable to your products is in the end good news because it brings
a potential deficiency to your attention, and you can quickly
regain the advantage by working on the problem. This has
certainly happened to us in the past. If, on a particular
example, you discover that you are 50% slower than someone
else, you can work on it and become twice as fast a few months
later. These things come and go. This is particularly true for
a compiler written in Eiffel, such as ISE's, which makes it
easier to implement new ideas quickly.
Let me finally note again that clamoring that a certain compiler
beats ISE's hands down and relegating to a footnote that
the compiler in question crashes with garbage collection on is
not the best way to have readers take one's claims seriously.
Not that ISE's compiler never crashes, but if it did on a simple
benchmark I think I would refrain from boasting about the
performance.
In the end, I prefer to continue applying the intellectual standards
that have led to Eiffel. If that means being branded as someone who
uses "inappropriate tone" so be it.
Ian Johnston
Bertrand Meyer wrote:
>
> I would like once again to protest against the
> campaign of slander directed against ISE's compilers.
I think referring to this as slander and libel is a little strong. You
should be sure first that there is malicious intent. My own
subjective impression is that the published figures were an
straightforward, if perhaps less than 100% rigorous, attempt to
highlight an overall picture.
> Publishing benchmarks "results" without the source code
> is tantamount to publishing physics results which
> no one else can reproduce.
Now, hold on a minute. Many times in this newsgroup, I have read claims
that Eiffel code runs faster than C++, or as fast as C++, or as
fast as C, or even faster than C.
I have read in this newgroup, and in other newsgroups, that garbage
collection is faster than manual memory management, or imposes only
negligible overhead.
For none of these claims have I ever seen any numbers, let alone source
code.
Now, at last, someone has apparently taken the trouble to write
the equivalent program in C++ and Eiffel. Yes, the code may not use
Eiffel features as optimally as possible, yes, it may penalise one
particular compiler, and yes, without seeing the source code we
cannot be absolutely sure.
But the numbers are interesting all the same:
- One Eiffel compiler is starting to get into the same ballpark as
C++. Hence there seems to be no inherent reason why Eiffel should be
(significantly) slower than C++. Surely this is a positive message
for Eiffel vendors?
- The free Eiffel compiler is very competitive with C++, albeit that
it probably doesn't implement all Eiffel features (I think someone
mentioned that exceptions were not fully implemented yet?) This too is a
positive message for Eiffel vendors. Note that in the C and C++ world,
the optimsation performed by the GNU compiler is taken as a standard to
equal or exceed; Eiffel vendors should be saying "Hey, we will beat
that, and look how close it is to C++".
- In all cases, *including* the Boehm garbage collector for C++,
the runtime with GC was vastly greater than the runtime without GC.
This tends to refute the (unsubstantiated) argument that GC is more
efficient. In fact, for me it calls into question the very use of
GC for all but the simplest command line tools (e.g. it might be
ok for "wc" or "df").
> On comp.lang.eiffel, it is the person who protests against
> the carelessness of such irrational and potentially libelous
> postings (namely me) who gets insulted and accused of using
> "inappropriate tone". Pray, what kind of tone do you adopt when
> someone publishes "results" that cast your
> products in a bad light, and you have no way to verify
> these results, let alone refute them?
I think you have every right to feel aggrieved that a product you have
worked hard to perfect seems to be under criticism. But if you really
want to succeed, you will take a serious look at these numbers.
Contacting the author to try and arrange release of his source code
would be a better approach than making accusations of libel and slander
in this newsgroup.
> While we are careful not to make exaggerated claims, the
I think some of your claims are at the least
unsubstantiated. If these numbers turn out to be realistic, then
your claims will be seen to be exaggerated.
Besides, haven't you claimed that no language can call itself an
OO language without GC? If you really made that claim, wouldn't
you say that that is an exaggeration?
> published "benchmarks" are so out of touch with our own
> measurements (some of which are available, together with
> the source code of course, from ISE) that we suspect an
> error, a poor use of our products, or worse.
So publish some of these numbers. I think everyone would be
interested to see them.
Ian
Jared Richardson
I agree.
If Deitrich (?) feels it is so important to protect his source code
until after his thesis (or whatever) is completed, he should also have kept
his results under wraps.
Bertrand Meyer <bert...@eiffel.com> wrote in article
<3251DC0A...@eiffel.com>...
> I would like once again to protest against the
> campaign of slander directed against ISE's compilers.
>
> Publishing benchmarks "results" without the source code
> is tantamount to publishing physics results which
> no one else can reproduce. In other engineering disciplines
> someone who did such a thing would quickly be asked
> to apply the basic rules of scientific discourse or
> choose another area of excellence.
<stuff deleted>
Lee Webber
Bertrand Meyer <bert...@eiffel.com> wrote:
>
> I would like once again to protest against the
> campaign of slander directed against ISE's compilers.
>
"Jared Richardson" <ja...@nando.net> wrote:
>
> I agree.
I don't.
[Meyer again]
> Publishing benchmarks "results" without the source code
> is tantamount to publishing physics results which
> no one else can reproduce.
True enough, and if the thesis is not forthcoming soon (as it
is supposed to be) I will change my view. But a preview of the
results with the (unchallenged) intention of eliciting criticism
before publication seems to me to be at the very worst slightly
questionable. Note that constructive criticism was in fact forth-
coming, possibly leading to improvement in the "benchmark".
> ...accused of using "inappropriate tone". Pray, what kind
> of tone do you adopt when someone publishes "results" that
> cast your products in a bad light, and you have no way to verify
> these results, let alone refute them?
You can start by assuming that we are all adults here. In particular:
1. All of us know that benchmarks that haven't been peer-reviewed
need to be viewed with a grain of salt. This does not mean,
however, that they are worthless -- just that they should be
looked upon as indicative rather than conclusive. If I was given
two Pascal compilers to review (say), I could by inappropriate
choice of benchmark bias the absolute performance of the products
by several times, but I would be hard pressed to do the same with
the relative performances. Not impossible -- just difficult.
2. You can assume that anybody reading this newsgroup values proper-
ties of a development environment beyond sheer speed. I've never
actually used the Tower development environment, so take this how
you will, but IMO ISE's environment is a generation ahead of any
other Eiffel product. (Not to mention your networking and concur-
rency clusters, which whatever their limitations are essentially
nonexistent elsewhere.) Fulminating over time trials as if that
were the last word does injustice to your strengths.
> Let me finally note again that clamoring that a certain compiler
> beats ISE's hands down...
Tower, you haven't been using this in your advertising, have you?
That _would_ be something for ISE to complain about. (I forgive
SmallEiffel; it's not a commercial product and for it to perform
on the level of one is something worth a little excessive boasting.)
Finally, about the "campaign of slander". That statement is in itself
defamatory. Are you willing to defend (in the US at least, I don't
know about Germany) that a) you know what is said is untrue without
yourself having seen the source code; b) it was said either to
injure you or without caring whether it was true? (I think that in
the context of this newsgroup you may be considered to be a public
figure.)
There is nothing wrong with your fostering healthy scepticism
about this "benchmark". Knock the rest of it off.
Frieder Monninger
Bertrand Meyer wrote:
. Publishing benchmarks "results" without the source code
. is tantamount to publishing physics results which
. no one else can reproduce.
.. or in other words: NOISE
I agree 100% with him (thats a rare event ;-)
But this problem can be resolved very easy .. I assume that there
is some ftp address where anybody can access the progrems. Am I right ?
--
Frieder Monninger SIG Computer GmbH
<f...@sigco.com> D 35619 Braunfels
http://www.sigco.com + 49 6472-2096 (fax -911 031)
Joachim Durchholz
bert...@eiffel.com wrote 01.10.96:
> On comp.lang.eiffel, it is the person who protests against
> the carelessness of such irrational and potentially libelous
> postings (namely me) who gets insulted and accused of using
> "inappropriate tone".
Bertrand, your message was totally right. It would be inappropriate to
quote the figures from that benchmark before we can see the source.
> Pray, what kind of tone do you adopt when
> someone publishes "results" that cast your
> products in a bad light, and you have no way to verify
> these results, let alone refute them?
Yes, I know from my own experience that it is infuriating to see
implausible claims against oneself published, regardless of the intent of
the publishing person. It is even more infuriating to have to sit and see
everybody gloat over the figures and no way to check their background. It
is even worse if you are an entrepreneur who will suffer significant
damage from bad reputation. (I know that. I had a tiny business for a few
years, and I had to restrain myself more than once. And that was a
business that didn't have so clear-cut quality criteria like execution
speed, so I had to swallow most unfavorable claims just because they were
not falsifyable. Such an experience teaches lots of humility, I can tell.)
However, infuriating as the situation may be, the tone or your post was
exaggerated. Just read it again and imagine you were the person against
whom the post was directed, and you will see what I mean. (At least I hope
you won't put my prediction to shame.)
> While we are careful not to make exaggerated claims, the
> published "benchmarks" are so out of touch with our own
> measurements
Such a statement would have been appropriate. I value facts more than
outrage, however justified the outrage may be.
> Not that ISE's compiler never crashes, but if it did on a simple
> benchmark I think I would refrain from boasting about the
> performance.
I can't let that stand uncommented. I used the Personal Eiffel for Windows
product from ISE. True enough, the compiler did not crash. It had no real
opportunity to do so - the workbench crashed after an average of five
minutes of work, rendering it next to useless. I tried to get help from
the distributor and from ISE, but none of the hints and updates that I got
solved the problem. I was informed by SIG (the German distributor) that
other customers had the same problems, that the problem seemed to depend
on Windows version and configuration, and that no real fix was available,
though an update might reduce the problem to a tolerable level (it did
not).
I don't want this to be read as an outright disqualification of
EiffelBench or other ISE products. I assume they run satisfactorily under
Unix or VMS. I'm not sure of the current state of Windows versions of ISE
tools.
(Unluckily, the consequence of this unhappy episode was that I'm now
forced to use Microsoft Visual C++ as development platform. Condolences
are accepted - though I still can't prove that the change was for the
worse, and that's what I resent most about the situation.)
OK, enough of ranting.
I think this is the right place for two remarks, to put a few things into
the right place:
1) Bertrand, the design of Eiffel is excellent work. When I first read
about Smalltalk, I thought "wow, that object-oriented stuff is great - but
without static typing, it is not what we really need." C++ was a horrible
Frankenstein of disparate parts - a valiant effort, but much too
complicated to actually use. When I read about Eiffel, I found it was not
only statically typed, but extremely well-designed in other ways, too!
2) Bertrand, the design of Eiffel makes the work of your life an admirable
success, regardless of wether you succeed or fail with any other endeavour
in the future. Even if the "benchmark" figures were justified, and if
(even worse) ISE were unable to do anything about the supposed compiler
inefficiencies, this would not change much about the high regard that I
hold the designer of Eiffel in. (And there is no need to blush because of
the praise - I think you honestly earned every letter of this remark.)
And I don't think either of these remarks is a minority position in this
newsgroup.
Regards,
-Joachim
--
Looking for a new job. Resume available on request. WWW version of resume
available under http://www.franken.de/users/jhd/resume/index.html
Paul Johnson
In article <325226...@ubs.com>, ian.jo...@ubs.com says...
>- In all cases, *including* the Boehm garbage collector for C++,
>the runtime with GC was vastly greater than the runtime without GC.
>This tends to refute the (unsubstantiated) argument that GC is more
>efficient. In fact, for me it calls into question the very use of
>GC for all but the simplest command line tools (e.g. it might be
>ok for "wc" or "df").
The problem is that the costs of not having GC are hidden in the structure of
the program. Furthermore they often only turn up in large, complicated
systems. As I understand the benchmark, its not difficult for the C++ code to
manage its own garbage, and the programmer does not have to take special
precuations (such as copying structures so that each module has exactly one
copy to delete).
Unfortunately this is the kind of argument that is almost impossible to back up
with figures. One would have to develop the same large program in two
different languages.
Paul.
--
Paul Johnson | GEC-Marconi Ltd is not responsible for my opinions. |
+44 1245 242244 +-----------+-----------------------------------------+
Work: <paul.j...@gecm.com> | You are lost in a twisty maze of little
Home: <Pa...@treetop.demon.co.uk> | standards, all different.
Ian Johnston
Paul Johnson wrote:
>
> In article <325226...@ubs.com>, ian.jo...@ubs.com says...
>
> >- In all cases, *including* the Boehm garbage collector for C++,
> >the runtime with GC was vastly greater than the runtime without GC.
[...]
> The problem is that the costs of not having GC are hidden in the structure of
> the program. Furthermore they often only turn up in large, complicated
> systems. As I understand the benchmark, its not difficult for the C++ code to
> manage its own garbage, and the programmer does not have to take special
> precuations (such as copying structures so that each module has exactly one
> copy to delete).
So what are these hidden costs? Do you think it is likely that these
hidden
costs will compensate for the (minimum of) 100% overhead imposed by GC
according to the figures?
For the systems I write, 100% GC overhead is just not acceptable,
period.
And no, I don't have any memory allocation problems, thank you.
> Unfortunately this is the kind of argument that is almost impossible to back up
> with figures. One would have to develop the same large program in two
> different languages.
Yes, and now someone apparently has developed the same program in two
different languages. We don't know how large the program is, but
apparently it manages
a large number of objects (20000 x 12 computations are performed). The
(as yet unsubstantiated) figures actually refute your argument, not back
it up.
Ian
Graham Perkins
I think we are all suffering from the lack of thorough and
open work in this area. The original measurements given
on this thread were of interest (if only we could see the
source) but apply to one particular project.
Measurements need to be made over many projects, perhaps
subgrouped into major application areas. We also need
some way of ensuring or factoring out comparability issues,
such as relative programming skills in the two languages
being compared.
But what about true software costs? Isn't memory and time
consumption less important (within broad bounds) than
development time or maintenance frequency?
Maybe as an academic I am isolated from reality, but I'd
rather have an easy time developing robust software with
little maintenance and rapid evolution capability even if
it is nearly as big and slow as the average Microsoft
product. The attractions of gaining a little runtime
efficiency by sweating it out over a low level language
from the C family are minimal, particularly as I know that
nobody (including myself) will be able to maintain the stuff
once I've written it.
On the reliability front, Joachim (earlier this thread) was
quite right in his descriptions of PEfW. I was
another one of those users who had massive problems with the
workbench. But it was the particular combination of Windows 3.1
and PEfW that was very sensitive to environment. The professional
version, the windows 95/NT personal version, and the unix
versions are much more robust. I'm now using PEfW under
windows 95 and the full works on RS6000 - they're
both brilliant.
--
person: Graham Perkins paper: School of Computing
bits: g...@dmu.ac.uk De Montfort University
voice: +44 190 883 4936 Milton Keynes MK7 6HP
dots: +44 190 883 4948 United Kingdom
Bertrand Meyer
Ian Johnston writes in reply to Paul Johnson:
> The (as yet unsubstantiated) figures actually refute your argument.
A real gem.
Franck Arnaud
Ian Johnston:
> the (minimum of) 100% overhead imposed by GC according to the figures?
Could you actually _read_ the figures? Some results are x times slower
with GC, some are slightly slower, some are actually quicker with GC. So
these figures do not tell that GC implies any overhead.
---
By the way, the whole controversy about Dietmar's "benchmarks" really
seems to be a storm in a tea cup. He repeated at least 5 times (see
dejanews) that his test was (a) not a benchmark (b) a very specific
and not representative application. If people cannot read and see
benchmarks and general results where there are none, it is not
his fault.
Darius Blasband
Paul Johnson wrote:
>
> In article <325226...@ubs.com>, ian.jo...@ubs.com says...
>
> >- In all cases, *including* the Boehm garbage collector for C++,
> >the runtime with GC was vastly greater than the runtime without GC.
> >efficient.
>
> the program. Furthermore they often only turn up in large, complicated
> systems. As I understand the benchmark, its not difficult for the C++ code to
> manage its own garbage, and the programmer does not have to take special
> precuations (such as copying structures so that each module has exactly one
> copy to delete).
>
> with figures. One would have to develop the same large program in two
> different languages.
Besides, Boehm's collector is atypical in the sense that it works in a hostile
world, that was not designed with GC in mind. We benchmarked asymptotical GC
(this is how we qualify a system that manages whatever is required to perform
GC, but never allocates. Hence, no GC) in YAFL (which is a language that
can be compared to Eiffel in many ways) and it appeared to take about
15% of the total execution time. These 15% represent keeping track of all visible
variables at any given time in order to be able to perform precise GC, as opposed
to a conservative GC as Boehm's.
The garbage collector in a precise environment is significantly more efficient
than a conservative one, and claiming that because Boehm's (by the way, excellent)
collector did induce a significant performance penalty in a specific case, all
GC's should be banned seems a bit naive to me.
In my opinion...
D.
Richard Smol
In a previous article, phil...@indigo.ie (phillips) says:
>
>It may be true that, for example, SmallEiffel (SE) is more efficient
>in that respect. On the other hand, I wouldn't like to loose the
>conveniance of Melting Ice, browsing, etc. in my development
>environment. If, as the results suggest, SE produces more efficient
>code, then an option would be to develop with the environment of your
>choice, and `finally' finalise with SE - if SE implemented HASH_TABLE,
>for example, I would have tried that today...
>
>
*NEWBIE ALERT* ...
*grin* just so you know ;)
Well, I just have gotten into Eiffel like a week ago... although I made
some little proggies in SmallEiffel for decrypting a CGI-form.
The prob is that I don't seem to be able to find a proper Eiffel
book anywwhere.
Anyway, I figure there is enough difference between SmallEiffel and
the ISE version. Even I as a newbie see that features are declared
in a different version in SmallEiffel (with {ANY} and such). So,
can ISE stuff easily be used with SmallEiffel?
Actually, I am not so much interested in designing large aplications
right now. Would Sather not be more suitable for my purposes?
*ducks* ;0
Greetz,
RS
Brian Strelioff
Richard Smol wrote:
>
>
> Well, I just have gotten into Eiffel like a week ago... although I made
> some little proggies in SmallEiffel for decrypting a CGI-form.
> The prob is that I don't seem to be able to find a proper Eiffel
> book anywwhere.
Look for Jacob Gore's Eiffel book.
>
> Anyway, I figure there is enough difference between SmallEiffel and
> the ISE version. Even I as a newbie see that features are declared
> in a different version in SmallEiffel (with {ANY} and such). So,
> can ISE stuff easily be used with SmallEiffel?
Probably not, but it depends on the program. In general there is
little code that runs unchanged on all vendor's products. Stick
with SmallEiffel if you are just "dabbling".
>
> Actually, I am not so much interested in designing large aplications
> right now. Would Sather not be more suitable for my purposes?
>
Hard to say. If you are making progress with SmallEiffel, stick to it.
If you run into any roadblocks, check with the SmallEiffel folks about
when any missing feature will be available.
Sather is good, but essentially a completely different language.
With Microsoft's Java product (Visual J++) hitting the shelves at
$80.00 (here in the US) that may be a better bet.
phillips
Mae Richard Smol <bl...@cleveland.Freenet.Edu> yn sgrifennau:
>Well, I just have gotten into Eiffel like a week ago... although I made
>some little proggies in SmallEiffel for decrypting a CGI-form.
>The prob is that I don't seem to be able to find a proper Eiffel
>book anywwhere.
Here is a sample of some good Eiffel texts:
- Object-Oriented Programming in Eiffel (Rist & Terwilliger) Addison
Wesley
- Eiffel: An Introduction (Robert Switzer) Prentice Hall ISBN
0-13-105909-2
*- Object-Oriented Programming in Eiffel (Thomas & Weedon) Addison Wesley
- Software Development Using Eiffel - There Can Be Life after C++
(R. Wiener) Prentice Hall ISBN 0-13-100686-X
* A colleague, new to Eiffel, particularly recommended this one
for beginners - he certainly got up to speed very quicky in Eiffel.
The second edition of Betrand Meyer's Object-Oriented Software
Construction,
recently announced on this newsgroup, promises to be an essential work
- all 1300 pages of it - but you should be able to find what you need
in the list above (I think ISE can supply all of these titles, try:
MAILTO:in...@eiffel.com).
>Anyway, I figure there is enough difference between SmallEiffel and
>the ISE version. Even I as a newbie see that features are declared
>in a different version in SmallEiffel (with {ANY} and such). So,
>can ISE stuff easily be used with SmallEiffel?
There is very little - if any - differences in the syntax implemented
by SmallEiffel and commerical offerings, the problem is one of SE not
implementing the same basic library classes. Does anyone know if there
are any plans to do this? So far, I have modified some of the classes
shipped with SE to match my ISE code: primarily LINKED_LIST and STRING,
and the STD_FILE classes - this at least allows me to compile a small
GUI library with SE.
>Actually, I am not so much interested in designing large aplications
>right now. Would Sather not be more suitable for my purposes?
It's your call: do you want to be able to re-use experience and classes
that you do now later in large applications? I would recommend trying
Sather: it's the best argument for covariance over contravariance!
SmallEiffel is set to fulfill an important role for the Eiffel community:
that of a free, fast and simple environment to get people started with the
language - the Eiffel community would do well to support this effort in
anyways possible (i.e., provide standard library inmplementations for SE,
make data access, user interface and CORBA interface classes available).
- Roy
Brian Strelioff
Graham Perkins wrote:
>
> I think we are all suffering from the lack of thorough and
> open work in this area. The original measurements given
> on this thread were of interest (if only we could see the
> source) but apply to one particular project.
Nevertheless, they are the only recent attempt to measure
performance of various Eiffel vendor products as well as
C++ and Java. As such the work should be encouraged by all.
>
> Measurements need to be made over many projects, perhaps
> subgrouped into major application areas. We also need
> some way of ensuring or factoring out comparability issues,
> such as relative programming skills in the two languages
> being compared.
True, and granted that "Tower Eiffel programming skills"
are sometimes different them "SmallEiffel programming skills"
are sometimes different then "SIG Eiffel programming skills ..."
in that all products implement a different dialect of the
Eiffel language. Nevertheless, the numbers presented seem
to suggest sufficient competency in Eiffel variations that
I doubt the relatively poor performance of some vendors
is indicative of lack of "Vendor-X Eiffel programming skills".
>
> But what about true software costs? Isn't memory and time
> consumption less important (within broad bounds) than
> development time or maintenance frequency?
>
Perhaps the author of this somparison can provide some
insight into his experiences with the various products.
OTOH, "correctness" is also very significant, and the
Eiffel system providing the shortest development time
or maintainance frequency may not produce "correct" code
to start with. This is especially evident to anyone who
has tried to port Eiffel code, then quite often code
that compiles & runs cleanly with vendor X won't even
compile with vendor Y, and once compile errors are taken
care of won't even run on Y. If the program took less time
to develop with vendor X, but vendor Y has the "correct"
implementation of Eiffel, how do you assign "software cost"?
> Maybe as an academic I am isolated from reality, but I'd
> rather have an easy time developing robust software with
> little maintenance and rapid evolution capability even if
> it is nearly as big and slow as the average Microsoft
> product. The attractions of gaining a little runtime
> efficiency by sweating it out over a low level language
> from the C family are minimal, particularly as I know that
> nobody (including myself) will be able to maintain the stuff
> once I've written it.
>
Definitely isolated, but your comments are valuable anyway :-).
Actually, the only difference I have found between "academic"
and "commercial" worlds is who the "customer" is. Anyone
writing software for their own use (primarily) will have a
different experience then writing software for others to use
(primarily).
And I quite agree that a factor of 2 slowdow is irrelevant
especially given the rapid advance in hardware speeds. But
the results presented seem to indicate a much larger
factor then 2 for some Eiffel implementations?
Even that factor of 2 is acceptable only if other areas
of programming are "improved", which is not necessarily the
case with some Eiffel implementations. Given the often
ambiguous descriptions of the Eiffel language, the lack of
portability of Eiffel code, the lack of any standard library
for Eiffel, etc. I am not convinced that Eiffel is an
appropriate choice for large, long-term, vendor/platform
independant projects.
Lee Webber
If anyone cares to pursue this further, Paul R. Wilson of the
University of Texas devotes a chapter of his GC survey to "Overall
Cost of Garbage Collection". The chapter cites five various
studies of GC cost, ranging from 5% to 40% overhead (Wilson
considers the two highest to be special cases), and contains
his own conclusion:
"Our own estimate of the typical cost of garbage
collection [significant qualifications omitted]
is that it _should_ cost roughly ten percent of
running time ... with a space cost of roughly a
factor of two in data memory size."
(_Italics_ in original.) The survey (about 67 pages, PostScript)
may be found at
ftp://ftp.cs.utexas.edu/pub/garbage/bigsurv.ps
Ian Johnston
Franck Arnaud wrote:
>
> Ian Johnston:
>
> > the (minimum of) 100% overhead imposed by GC according to the figures?
>
> Could you actually _read_ the figures? Some results are x times slower
> with GC, some are slightly slower, some are actually quicker with GC. So
> these figures do not tell that GC implies any overhead.
You're right, I looked only at the first few figures when I posted.
Yes, for SmallEiffel, the Boehm collected version outperforms the non
collected version. The SIG version without iterators runs only 10% more
slowly with GC than without. In these three cases GC performance looks
good.
For G++ with iterators, ISE with and without iterators and SIG with
iterators, GC performance looks bad.
4 cases with massive overhead; 3 without. Yet you say that these figures
do not imply any overhead?
Ian
Ian Johnston
Bertrand Meyer wrote:
>
> Ian Johnston writes in reply to Paul Johnson:
>
> > The (as yet unsubstantiated) figures actually refute your argument.
>
> A real gem.
Person A makes a statement and says that it is almost impossible
to produce figures to back up this statement.
Person B quotes some as yet unsubstantiated figures that, if
substantiated, would refute person A's argument.
Person C considers this a "real gem".
Marvellous intellectual discussion here.
Ian
James Mansion
Jared Richardson wrote:
>
> I agree.
>
> If Deitrich (?) feels it is so important to protect his source code
> until after his thesis (or whatever) is completed, he should also have kept
> his results under wraps.
This is crazy. The results were interesting, the claims very limited,
and the approach open and intelligent.
I must strongly object to your implied suggestion that we cannot even
discuss performance issues without actually achieving wide publication
of a frozen benchmark. That would stifle discussion and innovation.
It is a fact of life that 'freezing' code can be expensive and time
consuming. It is too much to ask that everyone who has an insite from a
real-world system be required to make a public code freeze and release.
If they want to claim an open scientific benchmark, then yes.
Otherwise, forget it.
I too am most impressed by the performance of Free Eiffel and I hope
that we can see more comparisons. And, I would like to see the
performance claims for Eiffel vs C++ (and I guess we should consider vs
Java with a decent JIT too) substantiated. But I do not see any benefit
in stifling discussion in the absence of a 'perfect' benchmark. Except
perhaps to cover the embarasment of certain vendors with vested
interests and a jaundiced approach (at least we're not dealing with Mr
Cox here ...)
James
James Mansion
Joachim Durchholz wrote:
>
> bert...@eiffel.com wrote 01.10.96:
>
> > On comp.lang.eiffel, it is the person who protests against
> > the carelessness of such irrational and potentially libelous
> > postings (namely me) who gets insulted and accused of using
> > "inappropriate tone".
>
> Bertrand, your message was totally right. It would be inappropriate to
> quote the figures from that benchmark before we can see the source.
>
Bertrand's tone is not right for the simple reason that the original
post did not claim to be a scientific benchmark.
I would support his viewpoint to some extent if it did. (Though I would
not support the way that he put his case forward, implying 'irrational'
and 'potentially libelous' behavour, which is I think clearly
bottom-speak from Dr Meyer. If I say 'I have a program and if I compile
it with BC5 and VC++ then the VC++ version runs faster' is that libelous
to BC++? I don't think so. If I say 'if you compile *any* program with
BC5 and VC++ then the VC++ one will be much faster 'cos BC5 is crap'
then maybe it is libellous. But more likely it would just make me look
silly.)
If we take every anecdotal 'war story' as a libelous benchmark against
one or more vendors where will be be? (Answer: we will be in the same
dumb situation as we are with SQL databases, whose license agreements
often specifically prohibit publication of test results. This makes
purchase decisions rather problematic.)
James Mansion
Graham Perkins wrote:
>
> I think we are all suffering from the lack of thorough and
> open work in this area. The original measurements given
> on this thread were of interest (if only we could see the
> source) but apply to one particular project.
I think we all agree that we suffer from the lack of real 'open'
work, but I dispute that measurements are ONLY of interest if
you can see the source.
For sure, any tests that someone does will be very heavily
loaded by:
- the particular problem domain
- the coding style
- the interaction with particular products implementations
In this particular case, the results were not particularly relevant
to my own line of work. But I did find them interesting nevertheless.
It doesn't bother me if this user finds that, say, ISE Eiffel 3.0 was
slow at that application. It would bother me a the majority of posts
from people who had tried different Eiffel technologies found that ISE
was (a lot) slower, or that all Eiffel systems had disappointing
performance. But that isn't where we are and Meyer's reaction
was, I thought, indicative of a certain paranoia about performance.
Its tempting to read something into that, though it would be mistaken
to do so I think.
>
> Measurements need to be made over many projects, perhaps
> subgrouped into major application areas. We also need
> some way of ensuring or factoring out comparability issues,
> such as relative programming skills in the two languages
> being compared.
I think the best you can do in practice is to hope for a large
sample size. No-one is being paid for this after all (well,
if you are paid by a company to do measurements to select a tool,
then most likely the results will be proprietary anyway - why let
your competitor benefit from your work, after all).
>
> But what about true software costs? Isn't memory and time
> consumption less important (within broad bounds) than
> development time or maintenance frequency?
In applications development, sure. But probably not for system
libraries and tools, and frequently executed code. You wouldn't
convince a derivatives trader that performance of multi-variable
options valuation was unimportant - it affects when they can go
home at night. ;-)
>
> Maybe as an academic I am isolated from reality, but I'd
> rather have an easy time developing robust software with
> little maintenance and rapid evolution capability even if
> it is nearly as big and slow as the average Microsoft
> product. The attractions of gaining a little runtime
> efficiency by sweating it out over a low level language
> from the C family are minimal, particularly as I know that
> nobody (including myself) will be able to maintain the stuff
> once I've written it.
As an academic, maybe YOU set the priorities - that's a big
difference.
Its also an issue of whether you can realistically propose certain
technologies as solutions these days. Eiffel is a hard sell at
present because it has a low profile with senior management and
the claimed benefits for engineering and maintenance etc have not
been justified well enough. I happen to believe them, but I'd
still be loath to propose its use, while I 'm fairly happy to
consider it in an ISV situation where I'm not selling source.
It doesn't matter if Eiffel code is easier to maintain that C++ code
if you have a choice of 10 good C++ programmers with relevant industry
knowledge and one claimed Eiffel hot-shot whose experiences are
primarily academic, for example.
>
> On the reliability front, Joachim (earlier this thread) was
> quite right in his descriptions of PEfW. I was
> another one of those users who had massive problems with the
> workbench. But it was the particular combination of Windows 3.1
> and PEfW that was very sensitive to environment. The professional
> version, the windows 95/NT personal version, and the unix
> versions are much more robust. I'm now using PEfW under
> windows 95 and the full works on RS6000 - they're
> both brilliant.
Wasn't a very good edvertisement for the reliability benefits of
design by contract and Eiffel was it?
>
> --
> person: Graham Perkins paper: School of Computing
> bits: g...@dmu.ac.uk De Montfort University
> voice: +44 190 883 4936 Milton Keynes MK7 6HP
> dots: +44 190 883 4948 United Kingdom
James
Paul Johnson
In article <3254B0...@ubs.com>, ian.jo...@ubs.com says...
>
>Paul Johnson wrote:
>> The problem is that the costs of not having GC are hidden in the structure
of
>> the program. Furthermore they often only turn up in large, complicated
>> systems. As I understand the benchmark, its not difficult for the C++ code
to
>> manage its own garbage, and the programmer does not have to take special
>> precuations (such as copying structures so that each module has exactly one
>> copy to delete).
>
>So what are these hidden costs? Do you think it is likely that these
>hidden
>costs will compensate for the (minimum of) 100% overhead imposed by GC
>according to the figures?
Not 100%. That point has been dealt with by others. More like 10-15%.
Yes, I think it can. There are several ways in which non-GC languages
impose overhead.
Suppose we have two modules, A and B, which both need to use a particular
object. The programmers of A and B (on a large project this means two
people) either have to share detailed knowledge of the workings of their
modules (so that A and B can agree together when the object can be
deleted) or they have to maintain two separate copies of the object so
that each module has exactly one to delete.
The first solution creates a hidden dependency between A and B. You can
get it to work, but future maintenance becomes much more difficult.
Most programmer effort today is spent on maintenance. The whole point
of OO programming is to increase the modularity of the systems we
produce.
The second solution is of course less efficient. Certainly more than
10 or 15 percent, but it won't show up in a benchmark because the extra
costs are hidden in the design rather than the language run-time support.
>For the systems I write, 100% GC overhead is just not acceptable,
>period.
>And no, I don't have any memory allocation problems, thank you.
How many people in your project team? The argument above starts to
bite at around half a dozen.
>> Unfortunately this is the kind of argument that is almost impossible to back
u
>p
>> with figures. One would have to develop the same large program in two
>> different languages.
>
>Yes, and now someone apparently has developed the same program in two
>different languages.
No, I said *large* program, as in a minimum of 10 programmer-years.
We don't know how large the program is, but
>apparently it manages
>a large number of objects (20000 x 12 computations are performed).
You missed the point. I can create a 10 line program that manages a million
objects, but it is still not a large program.
James Mansion
Paul Johnson wrote:
>
> In article <3254B0...@ubs.com>, ian.jo...@ubs.com says...
> >
> >Paul Johnson wrote:
> Not 100%. That point has been dealt with by others. More like 10-15%.
>
> Yes, I think it can. There are several ways in which non-GC languages
> impose overhead.
>
> Suppose we have two modules, A and B, which both need to use a particular
> object. The programmers of A and B (on a large project this means two
> people) either have to share detailed knowledge of the workings of their
> modules (so that A and B can agree together when the object can be
> deleted) or they have to maintain two separate copies of the object so
> that each module has exactly one to delete.
>
In practice this is actually quite straightforward so long as you
remember
that an object is only owned in one place and ownership is passed with
explicit documentation to that effect.
What's more, Purify has revolutionised the detection of such faults.
I think you also need this sort ownership model in systems with GC.
While
the responsibility for actual deletion passes to the system, you may
still
need to explicitly release system resources before the GC system will
collect
the objects (particularly mutex and critical section locks, database
locks,
graphics contexts, and so on). Also, you need to have some idea whether
the guy that passed you an object is expecting to use it again - if so
you'd
better not use it as a scratch area and had better clone it if you need
to
call non-const functions.
A trivialised example might be a key object that is used in a map.
Making a
clone of the key object for every insertion is rather expensive, while
placing
an object that you retain a reference to into the key structure is
dangerous,
since you might chanmge one of its properties inadvertantly when
generating a
further key value.
I think you still need to track ownership carefully. GC does not
eliminate that,
except in trivial cases. Granted, its nice not to worry about free
store management,
but once you have done the design and documentation work for ownership
and
responsibilities, making the store management explicit is not a huge
overhead. It
*was* however very difficult to verify, but as I say, Purify has made
that task
very much easier. (And I guess other products, like Sentinel etc)
James
Ian Johnston
Paul Johnson wrote:
>
> In article <3254B0...@ubs.com>, ian.jo...@ubs.com says...
> >Paul Johnson wrote:
>
> of
> >> the program. Furthermore they often only turn up in large, complicated
> >> systems.
[...]
> >So what are these hidden costs? Do you think it is likely that these
> >hidden
> >costs will compensate for the (minimum of) 100% overhead imposed by GC
> >according to the figures?
>
> Not 100%. That point has been dealt with by others. More like 10-15%.
>
> Yes, I think it can. There are several ways in which non-GC languages
> impose overhead.
>
> Suppose we have two modules, A and B, which both need to use a particular
> object. The programmers of A and B (on a large project this means two
> people) either have to share detailed knowledge of the workings of their
> modules (so that A and B can agree together when the object can be
> deleted) or they have to maintain two separate copies of the object so
> that each module has exactly one to delete.
If instead of purely memory resources (like, say, a text string) you
think of other types of resource--file handles, mutexes, shared
memory segments, sockets--I think you can see that these resources
still need to be managed carefully. I don't think the presence of GC
removes the need to plan management of object lifetimes.
[...]
> Most programmer effort today is spent on maintenance. The whole point
> of OO programming is to increase the modularity of the systems we
> produce.
Agreed.
However, it seems to me that there is an assumption that "if you've got
GC, resource management is solved". But I think this assumption is
wrong (for any resource except memory); and that concerns me.
> The second solution is of course less efficient.
The second solution may not even be possible, depending on the type
of resource. But that doesn't mean the problem can be ignored.
> >For the systems I write, 100% GC overhead is just not acceptable,
> >period.
> >And no, I don't have any memory allocation problems, thank you.
>
> How many people in your project team? The argument above starts to
> bite at around half a dozen.
The systems I write use many different types of resources; memory is
just one resource. All these resources need to be managed; that
management needs to be designed. If I could use GC for free, then I
would, as it would remove one resource from the design cycle.
However, removing memory management from the design cycle would
not save me very much planning and design time: perhaps 5%. Let's
say it would save 10% of the 80% of maintenance time to be spent
on this system over its lifetime. I would in total save around 13%
of the development time.
On the other hand, the systems I write have fairly rigorous timing
constraints. Even 10% time overhead could be hard to justify; and
I'm not yet convinced that GC will run with such little overhead
on all the platforms I need to write for. In addition, there is
currently no collector that I know of that:
- runs on all our platforms
- works with threads
- works with the languages we use.
There is currently a discussion in comp.programming.threads about
how to stop all threads except one, so that a collector (at least
a Boehm-style collector for C) can do its thing. It seems that not
all threading systems allow arbitrary sets of threads to be suspended.
It seems that the state of the art in GC in a hostile environment
in the presence of threads is still very much in the experimental
stage.
[...]
> No, I said *large* program, as in a minimum of 10 programmer-years.
>
> We don't know how large the program is, but
> >apparently it manages
> >a large number of objects (20000 x 12 computations are performed).
>
> You missed the point. I can create a 10 line program that manages a million
> objects, but it is still not a large program.
The size of the program would affect design and implementation time.
But this thread is about GC *runtime* performance. And for that, the
number of managed objects is significant; the size of the program
is not.
--
Ian Johnston, Contracting at UBS, Zurich
Hacked electronic address to defeat junk mail; please edit when replying
Matt Kennel
Ian Johnston ("ian.johnston"@ubs.com) wrote:
: If instead of purely memory resources (like, say, a text string) you
: think of other types of resource--file handles, mutexes, shared
: memory segments, sockets--I think you can see that these resources
: still need to be managed carefully. I don't think the presence of GC
: removes the need to plan management of object lifetimes.
The presence of GC allows entirely new categories of solutions to this problem.
--
Matthew B. Kennel/m...@caffeine.engr.utk.edu/I do not speak for ORNL, DOE or UT
Oak Ridge National Laboratory/University of Tennessee, Knoxville, TN USA/
I would not, could not SAVE ON PHONE, |==================================
I would not, could not BUY YOUR LOAN, |The US Government does not like
I would not, could not MAKE MONEY FAST, |spam either. It is ILLEGAL!
I would not, could not SEND NO CA$H, |USC Title 47, section 227
I would not, could not SEE YOUR SITE, |p (b)(1)(C) www.law.cornell.edu/
I would not, could not EAT VEG-I-MITE, | /uscode/47/227.html
I do *not* *like* GREEN CARDS AND SPAM! |==================================
M A D - I - A M!
James Mansion
> : think of other types of resource--file handles, mutexes, shared
> : memory segments, sockets--I think you can see that these resources
> : still need to be managed carefully. I don't think the presence of GC
> : removes the need to plan management of object lifetimes.
>
Good grief! Now *that's* a helpful and informative answer ... not.
I agree with Ian, GC doesn't help out entirely. The problem isn't
so much that finalisation (or whatever your language of choice calls
it) will release the resources as they owning objects are destroyed,
just that you don't have enough control over when that happens.
Sure, in most systems you can try to force a collect, but you can't
be very specific. Normally there is just a small subset of objects
that you want to have collected 'as soon as possible'.
I would be prepared to guess that for most such objects a use-count
scheme would work well - its not often that they will remain in an
orphaned circular structure. They might, but the usage of these
resource owners in my experience doesn't work quite like that.
James
Ian Johnston
> : think of other types of resource--file handles, mutexes, shared
> : memory segments, sockets--I think you can see that these resources
> : still need to be managed carefully. I don't think the presence of GC
> : removes the need to plan management of object lifetimes.
>
Matt, would you care to elaborate? Any new techniques are something
I'd be interested in learning more about.
In particular, locks of various kinds and network connections are
types of resources I use a a great deal.
Jon S Anthony
In article <53l32r$f...@ubszh.fh.zh.ubs.com> Ian Johnston <"ian.johnston "@ ubs . com> writes:
> Matt Kennel wrote:
> >
> > Ian Johnston ("ian.johnston"@ubs.com) wrote:
> > : If instead of purely memory resources (like, say, a text string) you
> > : think of other types of resource--file handles, mutexes, shared
> > : memory segments, sockets--I think you can see that these resources
> > : still need to be managed carefully. I don't think the presence of GC
> > : removes the need to plan management of object lifetimes.
> >
> > The presence of GC allows entirely new categories of solutions to this problem.
>
> Matt, would you care to elaborate? Any new techniques are something
> I'd be interested in learning more about.
>
> In particular, locks of various kinds and network connections are
> types of resources I use a a great deal.
It is relatively well known in the GC community that the sort of
resource management that you are talking about (other than memory) is
a topic of ongoing research level work. It is not a solved problem,
at least not at a level most are willing to accept.
/Jon
--
Jon Anthony
Organon Motives, Inc.
Belmont, MA 02178
617.484.3383
j...@organon.com
Dietmar Wolz
Changes to the last posting
-- removed ISE results, because I don't like to argue with Bertrand
Meyer. It seems that the intention of my posting was misunderstood
by many people. And again I want to say, that I clearly prefer ISE
ebench as development environment.
-- TOWER fixed their linux GC bug, so I have GC results for TOWER 2.0.
-- for SmallEiffel I included a version with normal ARRAYs, and one
with special arrays with fixed lower bound 0 provided by their
library.
Some commment about GC, Paul wrotes sometime ago:
> As I understand the benchmark, its not difficult for the C++ code to
> manage its own garbage,
Yes, thats true. For me it is absolutely not clear if you can achieve
any substantial performance gain with manual garbage collection in
larger
projects, so I would prefer the Boehm-Demers-Weiser.
Darius Blasband wrote:
>Besides, Boehm's collector is atypical in the sense that it works in a hostile
>world, that was not designed with GC in mind. We benchmarked asymptotical GC
>(this is how we qualify a system that manages whatever is required to perform
>GC, but never allocates. Hence, no GC) in YAFL (which is a language that
>can be compared to Eiffel in many ways) and it appeared to take about
>15% of the total execution time. These 15% represent keeping track of all visible
>variables at any given time in order to be able to perform precise GC, as opposed
>to a conservative GC as Boehm's.
>The garbage collector in a precise environment is significantly more efficient
>than a conservative one, and claiming that because Boehm's (by the way, excellent)
>collector did induce a significant performance penalty in a specific case, all
GC's should be banned seems a bit naive to me.
I share your opinion, but my results indicate suprisingly that the
Boehm-Demers-Weiser seems to perform better than most of the
specialized collectors. I really would like to know why.
It seems at least for my application that Eiffel is the winner
compared to C++. Eiffel clearly is the superior language and
if you use C++ without dirty tricks, with design
patterns and good programming style you can have the same performance
using Eiffel. There seems to be no language inherent performance
penalty. Time will show, whether this will be true also for
JAVA. For me this result was surprising. For a long
time I had strong objections against Eiffel and garbage collection
because I thought it would be not applicable to time critical
applications.
Compilers
g++: gnu g++ 2.7.2
g++ 1: with gnu STL from libg++ 2.7.1
g++ 2: with commercial STL from Object Space
se: SmallEiffel 0.89 (INRIA), -boost, -no_split
(ftp.loria.fr/pub/loria/genielog/SmallEiffel)
se1: normal array class used
se2: special array class with fixed lower bound 0 used
twr: Tower Eiffel 2.0.0.13
twr 1: option inline(no)
twr 2: option inline(yes)
jav: SUN javac 1.0.1 compiled with j2c + Boehm-Demers-Weiser
/ SUN java bytecode interpreter
kaf: SUN javac 1.0.1 compiled with j2c + Boehm-Demers-Weiser
/ kaffe-0.5p3 bytecode interpreter
j2c: j2c JAVA to C compiler from Todo Software
(ftp.webcity.co.jp/pub/andoh/java/j2c-beta4.tar.gz)
Garbage Collectors:
man: Manual garbage collection
bdw: Boehm-Demers-Weiser (ftp.parc.xerox.com/pub/gc/gc4.10.tar.gz)
ise, sig, twr: Built-in GC from vendor
jav, kaf: Built-in GC, but compiler uses Boehm-Demers-Weiser
GC Iterators Run Compile Memory Code
Time Time Usage Size
(sec) (sec) (MB) (kB)
g++ 1 man yes 4.9 12 5.2 32
g++ 1 bdw yes 9.3 14.5 9.9 76
g++ 2 man yes 8.6 17 6.6 50
se 1 bdw yes 6.9 11 8.0 58
se 1 bdw no 5.6 11 6.6 57
se 1 --- yes 7.3 10.8 71.2 31
se 1 --- no 5.8 10.8 60.0 30
se 2 bdw yes 6.1 11 7.9 57
se 2 bdw no 5.2 11 6.7 56
se 2 --- yes 6.3 10.8 65.7 30
se 2 --- no 5.3 10.8 54.6 29
twr 1 twr yes 13.7 57 9.6 132
twr 1 twr no 8.6 57 9.5 131
twr 1 --- yes 9.6 57 77.7 132
twr 1 --- no 7.1 57 66.5 131
twr 2 twr yes 12.7 147 9.6 230
twr 2 twr no 8.2 147 9.5 228
twr 2 --- yes 8.3 147 77.7 230
twr 2 --- no 6.9 147 66.5 228
jav jav yes 528 0.9 102 16 (bytecode)
kaf kaf yes 4364 0.9 43 16 (bytecode)
j2c bdw yes 53.4 38 18.5 243
Compile denotes overall compile time (from source to executable).
There is almost an one to one correspondence between the
EIFFEL, C++ and JAVA classes + features in the code.
The program consists of 13 Classes and is compiled and executed on a
LINUX (kernel 2.0.17, gcc 2.7.2p8 pentium version), Pentium P5/200
with 512 PB-Cache and 192 MB EDO-RAM.
It performs a construction from category theory, the computation of
the colimit of a signature diagram containing ca. 20000 symbols. The
computation is repeated 12 times in a loop to test the garbage
collection.
The colimit construction can be used, for instance, to implement
structuring operations on parameterized specification languages.
(see D.~E. Rydeheard and R.~M. Burstall, Computational Category Theory,
International Series in Computer Science. Prentice Hall, 1988).
--
Dietmar Wolz E-Mail:
die...@cs.tu-berlin.de
Technische Universitaet Berlin Tel: (030) 314-73137
FB 13 / Sekr. FR 6-1 / Raum FR 6026 Fax: +49-30-314-23516
Franklinstr. 28/29 10587 Berlin Germany
Dietmar Wolz
Sorry, Dominique Colnet reminded me to change the SmallEiffel line to
se: SmallEiffel -0.89 (CRIN-INRIA), -boost, -no_split
The minus before 0.89 is important because of negative
version numbering (-0.88 (hopefully soon) is newer than -0.90).
Steve Tynor suggested another compile flag for TOWER
twr 3: option optimize_kind("validity")
which for my application combines the advantages of the other settings:
twr 1 twr no 8.6 57 9.5 131
twr 2 twr no 8.2 147 9.5 228 (inline)
twr 3 twr no 8.3 59 66.5 130 (validity)
Dietmar Wolz
> ..It seems that the intention of my posting was misunderstood
> by many people. ...
>
> Perhaps like me they are don't understand why you (continue to) post
> results without making the source available (at least under NDA) to
> interested parties? By not releasting the source you put vendors in
> an impossible position -- they cannot verify your results and/or
> rectify their implementations if need be.
This was no benchmark, it is a special application. For me it makes a
difference if I publish an unfinished work to the world or just to
a vendor. I never got any personal mail from ISE, I have no problem
to send them the sources. How would you
react after being flamed in such a way? The question is, whether we
customers are allowed to discuss personal expieriences with
different tools without being flamed. May be we need a separate
news group. Note that I recommended ISE ebench several times.
Roger Browne
Dietmar Wolz <die...@cs.tu-berlin.de> writes:
> The question is, whether we ... are allowed to discuss personal expieriences
> with different tools
Yes, absolutely. This is an unmoderated discussion group, not a scientific
journal. I am always interested to hear people's experiences with Eiffel.
--
-- Roger Browne, 6 Bambers Walk, Wesham, PR4 3DG, UK | Ph 01772-687525
-- Everything Eiffel: compilers/libraries/publications | +44-1772-687525
Dietmar Wolz
Another thing I don't understand:
Why everyone is concentrating so much on the execution speed of a
special application ? Another fact seems to be more interesting.
Due to a statement from Bertrand Meyer ISEs code performs well
with their own tests (they don't publish). We all have a very big
Eiffel application to check its performance: The compiler itself.
Both the ISE and the SmallEiffel compiler are written in Eiffel.
Everyone can write a program which compiles both with SmallEiffel
and ISE. Compare the compilation times, even better the time needed
to produce the C-code. Yes, in SmallEiffel some features are missing,
the libraries are smaller, etc.. But still you will be suprised about
the performance of ISEs excellent code.
BTW. I will try to make my code availailable this month, not as
performance test, but mainly for all of you interested in category
theory.
Michael Reiche
Roger Browne wrote:
>
> Dietmar Wolz <die...@cs.tu-berlin.de> writes:
>
> > The question is, whether we ... are allowed to discuss personal expieriences
> > with different tools
>
> Yes, absolutely. This is an unmoderated discussion group, not a scientific
> journal. I am always interested to hear people's experiences with Eiffel.
I find it strange that nobody else has posted performance
figures for their applications. Here's mine.
The following figures are not intended to be scientific,
nor representative of the overall performance of a
particular compiler. But they are representative of
the difficulties we are facing with Eiffel. The program
was intended to be as simple as possible while still
demonstrating the performance problem. In the end, I
wimped out and didn't show which vendor is which, the
point is that neither compiler comes close to "C".
We still use Eiffel for almost all our development,
but if I hear someone say how efficient it is, I'm
gonna puke.
Both programs generate a CRC for a fixed 'key' (10000 times).
The program was compiled and run on a Sparc 10, with a Ross 125 MHz
HyperSparc CPU. The OS is Solaris 2.5.
Compiler User Seconds (from timex) Factor Slower
------------------------ ------------------------ ------------
CenterLine "C" 2.0.2a 0.36 1.0
Eiffel - Vendor "A" 2.96 8.2
Eiffel - Vendor "B" 15.50 43.0
Michael Reiche
CSE/National Defense Canada
-------------------------------------------------------------
-------------------------------------------------------------
-- The correct CRC is 0xb4, or 11010100, or 00101011
-- depending how you look at it
------------------------------------------------------------
class BITS_TEST
creation
make
feature -- Initialization
make is
local
i : INTEGER;
z : INTEGER;
do
!!key.make(1,128);
-- set bits 1-120 to 0 0 0 0 0 1 0 0 0 0 1 ...
from i:=key.lower until i>key.upper-8 loop
key.put( ((i-1) \\ 5) = 0, i);
i:=i+1;
end;
-- set bits 121-128 to 0 0 0 0 0 0 0 0
from i:=key.upper-8 until i>key.upper loop
key.put( False, i);
i:=i+1;
end;
-- calculate CRC 10000 times
from z:=1 until z> 10000 loop
parity;
z:=z+1;
end;
print(ccrc); print("%N");
end
key : ARRAY[BOOLEAN]
ccrc : BIT 8;
parity is
local
bit7 : BOOLEAN;
i : INTEGER;
do
ccrc :=00000000B;
from i := key.lower until i > key.upper
loop
bit7 := ccrc.item(8)
ccrc.put((ccrc.item(7)) xor bit7, 7)
ccrc.put((ccrc.item(2)) xor bit7, 2)
ccrc:=ccrc ^ 1 -- For Tower, use "ccrc:=ccrc ^ -1"
ccrc.put((key @ i) xor bit7, 1)
i := i + 1
end
end
end
-------------------------------------------------------------
-- ISE Ace file
system
BITS_TEST
root
BITS_TEST (root_cluster): "make"
default
assertion (no);
inlining (yes);
array_optimization (yes);
cluster
root_cluster: ".";
kernel: "$EIFFEL3/library/base/kernel";
support: "$EIFFEL3/library/base/support";
access: "$EIFFEL3/library/base/structures/access";
cursors: "$EIFFEL3/library/base/structures/cursors";
cursor_tree: "$EIFFEL3/library/base/structures/cursor_tree";
dispenser: "$EIFFEL3/library/base/structures/dispenser";
iteration: "$EIFFEL3/library/base/structures/iteration";
list: "$EIFFEL3/library/base/structures/list";
obsolete: "$EIFFEL3/library/base/structures/obsolete";
set: "$EIFFEL3/library/base/structures/set";
sort: "$EIFFEL3/library/base/structures/sort";
storage: "$EIFFEL3/library/base/structures/storage";
table: "$EIFFEL3/library/base/structures/table";
traversing: "$EIFFEL3/library/base/structures/traversing";
tree: "$EIFFEL3/library/base/structures/tree";
end
-------------------------------------------------------------
-- Tower Ace file
system
BITS_TEST
root
BITS_TEST: make
default
assertion (no);
cluster
root_cluster: ".";
kernel: "$EIFFEL/clusters/kernel";
support: "$EIFFEL/clusters/support";
end
-------------------------------------------------------------
/* This is "C", not C++, I know, I know. */
#include <stdio.h>
#define m_shift(a,b) ((a)<<(b))
#define m_item(a,i) ( ((a) & (1<<(i))) !=0 )
#define m_put(a,i,b) ( (b) ? (a) | (1<<(i)) : (a) & ~(1<<(i)) )
int ccrc;
int key[128];
int parity()
{
int bit7;
int i;
ccrc=0;
for(i=0;i<128; i++){
bit7 = m_item(ccrc,7);
ccrc=m_put(ccrc,6,m_item(ccrc,6) ^ bit7 );
ccrc=m_put(ccrc,1,m_item(ccrc,1) ^ bit7 );
ccrc=m_shift(ccrc,1);
ccrc=m_put(ccrc,0, key[i] ^ bit7 );
}
return 0;
}
main()
{
int i;
int z;
for(i=0;i<128; i++) /* 0 0 0 0 0 1 0 0 0 0 1 ... */
key[i]= (i % 5) == 0;
for(i=119;i<128; i++) /* 0 0 0 0 0 0 0 0 */
key[i]= 0;
for(z=0; z<10000; z++)
i=parity();
printf("%0x\n",ccrc & 0xff );
}
The Rt Rev'd Colin James III, KOTM 1/96
Michael Reiche <mre...@cse.dnd.ca> posted with deletions:
| -------------------------------------------------------------
| -- The correct CRC is 0xb4, or 11010100, or 00101011
| -- depending how you look at it
| ------------------------------------------------------------
From the above, only, the first and second values are the same, correct
CRC. The third value is meaningless because of the format of either CRC
XMODEM or CRC-16-CCITT. See Numerical Recipes in FORTRAN.
Roger Browne
Michael Reiche <mre...@cse.dnd.ca> writes:
> The program was compiled and run on a Sparc 10, with a Ross 125 MHz
> HyperSparc CPU. The OS is Solaris 2.5.
>
> Compiler User Seconds (from timex) Factor Slower
> ------------------------ ------------------------ ------------
> CenterLine "C" 2.0.2a 0.36 1.0
> Eiffel - Vendor "A" 2.96 8.2
> Eiffel - Vendor "B" 15.50 43.0
For what it's worth, I got faster times for Eiffel under Windows NT 3.51
on a Pentium Pro 200MHz Gateway 2000 machine. As you did not mention the
vendor, neither will I.
Compiler User Seconds (from timex)
------------------------ -------------------------
Eiffel 1.62
I didn't attempt to optimize your benchmark (other than by compiling in
"finalize" mode), so the Eiffel time could probably be improved further.
By the way, your code says:
-- set bits 1-120 to 0 0 0 0 0 1 0 0 0 0 1 ...
but it actually does this:
-- set bits 1-120 to 1 0 0 0 0 1 0 0 0 0 1 ...
Regards,
Roger
Michael Reiche
The Rt Rev'd Colin James III, KOTM 1/96 wrote:
>
> Michael Reiche <mre...@cse.dnd.ca> posted with deletions:
>
> | -- The correct CRC is 0xb4, or 11010100, or 00101011
> | -- depending how you look at it
> | ------------------------------------------------------------
>
> CRC. The third value is meaningless because of the format of either CRC
> XMODEM or CRC-16-CCITT. See Numerical Recipes in FORTRAN.
Yeah, I goofed.
The "C" program prints b4,
Eiffel Vendor A prints 10110100B
Eiffel Vendor B prints 00101101b
Compiler 'A' prints the bits starting with .item(<n>),
Compiler 'B' prints the bits starting at .item(1).
Mike.
Dietmar Wolz
Michael Reiche <mre...@cse.dnd.ca> writes:
>Roger Browne wrote:
>>
>> Dietmar Wolz <die...@cs.tu-berlin.de> writes:
>>
>> > The question is, whether we ... are allowed to discuss personal expieriences
>> > with different tools
>>
>> Yes, absolutely. This is an unmoderated discussion group, not a scientific
>> journal. I am always interested to hear people's experiences with Eiffel.
>I find it strange that nobody else has posted performance
>figures for their applications. Here's mine.
>The following figures are not intended to be scientific,
>nor representative of the overall performance of a
>particular compiler. But they are representative of
>the difficulties we are facing with Eiffel. The program
>was intended to be as simple as possible while still
>demonstrating the performance problem. In the end, I
>wimped out and didn't show which vendor is which, the
>point is that neither compiler comes close to "C".
>We still use Eiffel for almost all our development,
>but if I hear someone say how efficient it is, I'm
>gonna puke.
On my LINUX (kernel 2.0.23, gcc 2.7.2p8) Pentium P5/200 machine
with 512 PB-Cache and 192 MB EDO-RAM I got the following results:
Compiler User Seconds (from time) Factor Slower
------------------------ ------------------------ ------------
gcc 2.7.2p8 0.22 1.0
Eiffel - Vendor "A" 0.84 3.8
(version 2.0.0.13)
Eiffel - Vendor "B" 11.92 54.1
(version 3.3.7)
Unfortunately Vendor "C" (version -0.89) doesn't support the xor
operation.
It seems a typical case where an external C-routine should be used,
but the differences between the compilers are interesting.
I tried hard to improve the results for Eiffel (see the code below),
but had not much success with Vendor "B". May be Vendor "B" - experts
have more luck.
Compiler User Seconds (from time) Factor Slower
------------------------ ------------------------ ------------
Eiffel - Vendor "A" 0.59 2.7
(version 2.0.0.13)
Eiffel - Vendor "B" 11.33 51.5
(version 3.3.7)
even worse are the compile times:
Compiler compile time (from time) Factor Slower
------------------------ ------------------------ ------------
gcc 2.7.2p8 0.27 1.0
Eiffel - Vendor "A" 26 96.3
(version 2.0.0.13)
Eiffel - Vendor "B" 150 555.6
(version 3.3.7)
Compiler code size (stripped) Factor Bigger
------------------------ ------------------------ ------------
gcc 2.7.2p8 2924 1.0
Eiffel - Vendor "A" 112288 38.4
(version 2.0.0.13)
Eiffel - Vendor "B" 171864 58.8
(version 3.3.7)
------------------------------------------------------------
-- OPTIMIZED for Vendor "A" (version 2.0.0.13)
------------------------------------------------------------
class BITS_TEST
creation
make
feature -- Initialization
make is
local
i : INTEGER;
z : INTEGER;
ccrc : BIT 8;
key : SIMPLE_ARRAY[BOOLEAN]
do
!!key.make(128);
-- set bits 1-120 to 0 0 0 0 0 1 0 0 0 0 1 ...
from i:=key.lower until i>key.upper-8 loop
key.put( ((i-1) \\ 5) = 0, i);
i:=i+1;
end;
-- set bits 121-128 to 0 0 0 0 0 0 0 0
from i:=key.upper-8 until i>key.upper loop
key.put( False, i);
i:=i+1;
end;
-- calculate CRC 10000 times
from z:=1 until z> 10000 loop
ccrc := parity(key);
z:=z+1;
end;
print(ccrc); print("%N");
end
parity (key : SIMPLE_ARRAY[BOOLEAN]) : BIT 8 is
local
bit7 : BOOLEAN;
i : INTEGER;
do
Result :=00000000B;
from i := key.lower until i > key.upper
loop
bit7 := Result.item(8)
Result.put((Result.item(7)) xor bit7, 7)
Result.put((Result.item(2)) xor bit7, 2)
Result:=Result ^ -1 -- For Vendor "B", use "Result:=Result ^ 1"
Result.put((key @ i) xor bit7, 1)
i := i + 1
end
end
end
------------------------------------------------------------
-- "OPTIMIZED" for Vendor "B" (version 3.3.7)
------------------------------------------------------------
class BITS_TEST
creation
make
feature -- Initialization
make is
local
i : INTEGER;
z : INTEGER;
ccrc : BIT 8;
key : TO_SPECIAL[BOOLEAN]
do
!!key.make_area(128);
-- set bits 1-120 to 0 0 0 0 0 1 0 0 0 0 1 ...
from i:=0 until i>119 loop
key.area.put( ((i-1) \\ 5) = 0, i);
i:=i+1;
end;
-- set bits 121-128 to 0 0 0 0 0 0 0 0
from i:=120 until i>127 loop
key.area.put( False, i);
i:=i+1;
end;
-- calculate CRC 10000 times
from z:=1 until z> 10000 loop
ccrc := parity(key.area);
z:=z+1;
end;
print(ccrc); print("%N");
end
parity (key : SPECIAL[BOOLEAN]) : BIT 8 is
local
bit7 : BOOLEAN;
i : INTEGER;
do
Result :=00000000B;
from i := 0 until i >= key.count
loop
bit7 := Result.item(8)
Result.put((Result.item(7)) xor bit7, 7)
Result.put((Result.item(2)) xor bit7, 2)
Result:=Result ^ 1 -- For Vendor "A", use "Result:=Result ^ -1"
Result.put((key.item(i)) xor bit7, 1)
i := i + 1
end
end
end
--
Dietmar Wolz
James Mansion <ja...@wgold.demon.co.uk> writes:
>While it is certainly the case that using publicly available code is
>'better', I don't think that the tone of this message is justified at
>all.
So, here it is:
An alpha release of the colimit library can be found at
ftp://ftp.cs.tu-berlin.de:/pub/local/tfs/colimitlib/colimitlib.tar.gz
Linux binaries for the performance comparison and some related
papers are included. For other platforms you can compile the sources
yourself. Use the same C/C++-compile options for all compilers
(as I did). Feel free to make optimizations / propose alternative
algorithms for the colimit computation.
Dietmar Wolz
I wrote:
> Compiler User Seconds (from time) Factor Slower
> ------------------------ ------------------------ ------------
> Eiffel - Vendor "A" 0.59 2.7
> (version 2.0.0.13)
> Eiffel - Vendor "B" 11.33 51.5
> (version 3.3.7)
>
Unfortunately for some reason Steves posting is still not on my
news reader. So was not suspicious because of the original results
which are even worse for vendor "A". I got mail from Steve and
first thought the difference has to do with the different operating
systems. But then I saw that my C-compile options for the vendor
"A" compiler was wrongly set. I can confirm Steve's results more
or less. Of course I checked also the options for vendor "B".
They were correctly set to "-O6 -mpentium -fswap-for-agi -frisc-const
-fschedule-stack-reg-insns" which is the best setting for the
Pentium optimized gcc2.7.2p8.
Compiler User Seconds (from time) Factor Slower
------------------------ ------------------------ ------------
gcc 2.7.2p8 0.22 1.0
Eiffel - Vendor "A" 0.29 1.3
(version 2.0.0.13)
Eiffel Code for Vendor "A":
class BITS_TEST
creation
make
feature -- Initialization
make is
local
i : INTEGER;
z : INTEGER;
ccrc : BIT 8;
key : SIMPLE_ARRAY[BOOLEAN]
do
!!key.make(128);
-- set bits 1-120 to 0 0 0 0 0 1 0 0 0 0 1 ...
from i:=key.lower until i>key.upper-8 loop
key.put( (i \\ 5) = 0, i);
i:=i+1;
end;
-- set bits 121-128 to 0 0 0 0 0 0 0 0
from i:=key.upper-8 until i>key.upper loop
key.put( False, i);
i:=i+1;
end;
-- calculate CRC 10000 times
from z:=1 until z> 10000 loop
ccrc := parity(key);
z:=z+1;
end;
print(ccrc); print("%N");
end
parity (key : SIMPLE_ARRAY[BOOLEAN]) : BIT 8 is
local
bit7 : BOOLEAN;
i : INTEGER;
do
Result :=00000000B;
from i := 0 until i > 127
loop
bit7 := Result.item(8)
Result.put((Result.item(7)) xor bit7, 7)
Result.put((Result.item(2)) xor bit7, 2)
Result:=Result ^ -1 -- For Tower, use "Result:=Result ^ -1"
Result.put((key @ i) xor bit7, 1)
i := i + 1
end
end
end
--
Dietmar Wolz
Sorry, I forgot to replace i-1 by i in both optimized versions. But it
neither influences the performance figures nor the result of the computation
(which was the reason I haven't noticed it earlier).
Dietmar Wolz
Dietmar Wolz wrote:
> Compiler User Seconds (from time) Factor Slower
> ------------------------ ------------------------ ------------
> gcc 2.7.2p8 0.22 1.0
> (version 2.0.0.13)
>
Before all Eiffel fans got over-enthusiastic, lets see what we can
do with the C-code:
#include <stdio.h>
#define m_shift(a,b) ((a)<<(b))
#define m_item(a,i) ( ((a) & (1<<(i))) !=0 )
int ccrc;
int key[128];
void parity()
{
int bit7;
int i;
ccrc=0;
for(i=0;i<128; i++){
bit7 = m_item(ccrc,7);
ccrc ^= m_shift(bit7,6) | m_shift(bit7,1);
ccrc = m_shift(ccrc,1);
ccrc |= key[i] ^ bit7;
}
}
main()
{
int i;
int z;
for(i=0;i<128; i++) /* 0 0 0 0 0 1 0 0 0 0 1 ... */
key[i]= (i % 5) == 0;
for(i=119;i<128; i++) /* 0 0 0 0 0 0 0 0 */
key[i]= 0;
for(z=0; z<10000; z++)
parity();
printf("%0x\n",ccrc & 0xff );
}
And now we have:
Compiler User Seconds (from time) Factor Slower
------------------------ ------------------------ ------------
gcc 2.7.2p8 0.073 1.0
Eiffel - Vendor "A" 0.29 4.0
(version 2.0.0.13)
Eiffel - Vendor "B" 11.33 155.2
(version 3.3.7)
Seems Vendor "B" has some room left for improvements.
Dietmar Wolz
Just to show you how clever GNU gcc is:
Everyone would expect that the optimized
void parity()
{
int i;
int mask;
ccrc=0;
mask = m_shift(1,7) | m_shift(1,2) | 1;
for(i=0;i<128; i++) {
ccrc = m_shift(ccrc,1) | key[i];
if (ccrc & m_shift(1,8)) ccrc ^= mask;
}
}
is faster than the straightforward
void parity()
{
int bit7;
int i;
ccrc=0;
for(i=0;i<128; i++){
bit7 = m_item(ccrc,7);
ccrc = m_shift(ccrc,1);
ccrc ^= m_shift(bit7,7) | m_shift(bit7,2) | (key[i] ^ bit7);
}
}
but this is only the case without optimization enabled.
With -O2 they are equally fast and with -mpentium -O6 the second
solution is faster. (This holds for the Pentium optimized gcc 2.7.2p8,
for the ordinary gcc 2.7.2 -O2 and -O6 make no difference.
Don Harrison
Dietmar Wolz wrote:
:Before all Eiffel fans got over-enthusiastic, lets see what we can
:do with the C-code:
For a performance comparison between languages to be meaningful, the
languages must be of a similar level of abstraction. So, it may make sense
to compare Eiffel and C++ but it doesn't make sense to compare Eiffel and
C except to measure the performance difference you would expect between
a high and a low level language.
:Eiffel - Vendor "A" 0.29 4.0
:(version 2.0.0.13)
:Eiffel - Vendor "B" 11.33 155.2
:(version 3.3.7)
The result for Vendor "A" is what you might expect.
:Seems Vendor "B" has some room left for improvements.
Agree.
BTW, we might also expect Eiffel to be slightly slower than C++ because it
is slightly higher level. So, if an Eiffel compiler can outperform a C++
compiler, it is a significant acheivement (unless, of course, the C++ compiler
is a lemon).
Don.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Don Harrison do...@syd.csa.com.au
Alexander Kjeldaas
Alexander Kjeldaas <as...@lucifer.guardian.no> writes:
> #3 is a particularly serious performance bottleneck. Reorganizing the
> algorithm slightly makes it a better approximation to the
> eiffel-version and at the same time makes it run about 65% faster (on
> my machine), which is faster than any of your other 'optimized'
> versions.
Hmm..looking at the numbers that would be 45% I guess..
astor
--
Alexander Kjeldaas
Guardian Networks AS as...@guardian.no
Michael Reiche
Don Harrison wrote: with deletions
> For a performance comparison between languages to be meaningful, the
> languages must be of a similar level of abstraction. So, it may make sense
> to compare Eiffel and C++ but it doesn't make sense to compare Eiffel and
> C except to measure the performance difference you would expect between
> a high and a low level language.
I agree completely. But the performance difference I expect
between Eiffel and C is definitely less than a factor of 155.
Here's an excerpt from "Why your next project should use
Eiffel" which was provided by an Eiffel vendor at OOPSLA '96.
"Thanks to a performance-obsessed language design and ten years of
research and competition on compiling algorithms, the speed of
Eiffel-generated code (in such modes as what is known as
"finalization" in ISE Eiffel) is as good as that of hand-produced
C code, or better."
(the following numbers are from the calculation of a CRC)
seconds times slower than C
> :Eiffel - Vendor "A" 0.29 4.0
> :Eiffel - Vendor "B" 11.33 155.2
>
Dietmar Wolz
do...@syd.csa.com.au (Don Harrison) writes:
>For a performance comparison between languages to be meaningful, the
>languages must be of a similar level of abstraction. So, it may make sense
>to compare Eiffel and C++ but it doesn't make sense to compare Eiffel and
>C except to measure the performance difference you would expect between
>a high and a low level language.
Wait a moment, the level of abstraction has not much to do with the
choosen language, as you can see by the code given below.
Of course we can optimize the Eiffel code similar to the C-code, leading
to code which is more than 3 times faster than the original C-code !!
For some reason I don't understand it is even faster than the
optimized C-code, but test yourself. We have to be very careful
when claiming superior performance of C.
Unfortunately I had problems getting similar code running for vendor "B"
(replacing SIMPLE_ARRAY by SPECIAL and adapting the BIT 8 stuff).
Perhaps someone else can post an optimized vendor "B" - version.
Compiler User Seconds (from time) Factor Slower
------------------------ ------------------------ ------------
gcc 2.7.2p8 (optimized) 0.078 1.0
Eiffel - Vendor "A" 0.065 0.83
(version 2.0.0.13)
(optimized code)
gcc 2.7.2p8 (old code) 0.220 2.8
Eiffel - Vendor "A" 0.840 10.8
(version 2.0.0.13)
(old code)
------------------------- C - VERSION ------------------------------
#include <stdio.h>
#define m_shift(a,b) ((a)<<(b))
int ccrc;
int key[128];
void parity()
{
int i;
ccrc = 0;
for(i=0;i<128; i++) {
ccrc = m_shift(ccrc,1) | key[i];
if (ccrc & m_shift(1,8))
ccrc ^= m_shift(1,7) | m_shift(1,2) | 1;
}
}
main()
{
int i;
int z;
for(i=0;i<128; i++) /* 0 0 0 0 0 1 0 0 0 0 1 ... */
key[i]= (i % 5) == 0;
for(i=119;i<128; i++) /* 0 0 0 0 0 0 0 0 */
key[i]= 0;
for(z=0; z<10000; z++)
parity();
printf("%0x\n",ccrc & 0xff );
}
------------------------- Vendor A VERSION ------------------------------
class BITS_TEST
creation
make
feature -- Initialization
make is
local
i : INTEGER;
z : INTEGER;
ccrc : BIT 8;
key : SIMPLE_ARRAY[BIT 8]
do
!!key.make(128);
-- set bits 1-120 to 0 0 0 0 0 1 0 0 0 0 1 ...
from i:=key.lower until i>key.upper-8 loop
if (i \\ 5) = 0 then
key.put(00000001B, i)
else
key.put(00000000B, i)
end
i:=i+1;
end;
-- set bits 121-128 to 0 0 0 0 0 0 0 0
from i:=key.upper-8 until i>key.upper loop
key.put(00000000B, i);
i:=i+1;
end;
-- calculate CRC 10000 times
from z:=1 until z> 10000 loop
ccrc := parity(key);
z:=z+1;
end;
print(ccrc); print("%N");
end
parity (key : SIMPLE_ARRAY[BIT 8]) : BIT 8 is
local
i : INTEGER;
do
Result :=00000000B;
from i := 0 until i > 127
loop
if Result.item(8) then
Result:= ((Result ^ -1) or (key @ i)) xor
((00000001B ^ -7) or (00000001B ^ -2) or 00000001B)
else
Result:= (Result ^ -1) or (key @ i)
end
i := i + 1
end
end
end
---------------------------------------------------------------------------
Darius Blasband
>
> Darius Blasband wrote:
> >Besides, Boehm's collector is atypical in the sense that it works in a hostile
> >world, that was not designed with GC in mind. We benchmarked asymptotical GC
> >(this is how we qualify a system that manages whatever is required to perform
> >GC, but never allocates. Hence, no GC) in YAFL (which is a language that
> >can be compared to Eiffel in many ways) and it appeared to take about
> >15% of the total execution time. These 15% represent keeping track of all visible
> >variables at any given time in order to be able to perform precise GC, as opposed
> >to a conservative GC as Boehm's.
> >The garbage collector in a precise environment is significantly more efficient
> >than a conservative one, and claiming that because Boehm's (by the way, excellent)
> >collector did induce a significant performance penalty in a specific case, all
> GC's should be banned seems a bit naive to me.
>
> I share your opinion, but my results indicate suprisingly that the
> Boehm-Demers-Weiser seems to perform better than most of the
> specialized collectors. I really would like to know why.
>
I have no explanation for your experiment, that do contradict ours. We generated
a GC-less version of the YAFL compiler, linked together with Boehm's GC, and
the memory consumption was significantly higher (+60%) and the performance was
poor. It is true however that our experiment was straightforward, and did not
take advantage of such features as the ability to indicate that a given allocation
is pointer-free.
Cheers,
Darius
Joachim Durchholz
do...@syd.csa.com.au wrote 23.10.96:
> For a performance comparison between languages to be meaningful, the
> languages must be of a similar level of abstraction. So, it may make sense
> to compare Eiffel and C++ but it doesn't make sense to compare Eiffel and
> C except to measure the performance difference you would expect between
> a high and a low level language.
This strongly depends on what you want to use the language for.
For business client applications, performance isn't really an issue.
For business server applications, it is important in a moderate degree.
For number crunching, performance is all-important. Eiffel should
outperform FORTRAN if possible!
I think success of a language is largely a function of how many areas are
adequately covered. I wouldn't defend Eiffel performance penalties on the
grounds that performance isn't important, or that Eiffel is a higher-level
language. After all, the exec deciding on the language will be looking for
arguments to back his decision against Eiffel, and bad performance has
always been an excellent auxiliary argument.
Regards,
-Joachim
--
Looking for a new job. Resume available on request. WWW version of resume
available under http://www.franken.de/users/herold/jhd/resume/index.html
Don Harrison
Dietmar said:
:>For a performance comparison between languages to be meaningful, the
:>languages must be of a similar level of abstraction. So, it may make sense
:>to compare Eiffel and C++ but it doesn't make sense to compare Eiffel and
:>C except to measure the performance difference you would expect between
:>a high and a low level language.
:
:Wait a moment, the level of abstraction has not much to do with the
:choosen language, ...
By 'level of abstraction', I mean 'high level vs low level'. I don't know of
a specific term describing 'high level vs low level'.
What I'm saying is simply that high level code which is well written and
compiled efficiently won't run as fast as low level code which is well written
and compiled efficiently. This should be obvious.
More specifically, well written Eiffel (high level) which is translated into C
(low level) and subsequently compiled will run slower than well written C
compiled by the same C compiler.
Such comparisons are highly subjective and influenced by many factors (as
shown by the difference between vendors in Dietmar's results) and can only
be made meaningful by identifying those factors and reducing their effect.
Using the same C compiler is an obvious way of removing subjectivity.
Only then can the results give any indication of how slow Eiffel is compared
to C.
Did someone say once "There are lies, lies and results of performance
tests". (Actually, it was 'statistics' - poetic license). ;-)
Robb Nebbe
Don Harrison wrote:
> What I'm saying is simply that high level code which is well written and
> compiled efficiently won't run as fast as low level code which is well
> written and compiled efficiently. This should be obvious.
Actually this isn't obvious at all. There are even counter examples. When
you write in a lower-level language you almost invariably wind up "over
specifying" the code since implementation details are practically
unseparable from the intended semantics. The consequence is that you
inhibit certain useful optimizations.
High-level languages sometimes define semantics that inhibit optimization
as well. In the absence of such "design choices" it is often much easier to
determine which optimizations will preserve the semantics. The downside is
that there is more optimization required to reach the same performance
level of the low-level language.
It does not automatically follow that just because a language is higher
level than some other language it is necessarily less efficient. At the
same time it is understandable when they are not because it isn't an easy
task to design or optimize high-level languages.
Robb Nebbe
James Mansion
Don Harrison wrote:
>
> What I'm saying is simply that high level code which is well written and
> compiled efficiently won't run as fast as low level code which is well written
> and compiled efficiently. This should be obvious.
>
> (low level) and subsequently compiled will run slower than well written C
> compiled by the same C compiler.
I don't think that's true. First of all (in your previous post) you
suggested
that it might not matter too much. The trouble is that for some
applications
you do have to twiddle bits and in those cases you really don't want to
have
to break out to a lower level lenguage - you want to be able to code the
same
stuff in the high level language.
Secondly, the input to the C compiler that an Eiffel compiler can
generate do
not have the same constraints on readability and maintainability
(cough!) that
a C programmer must use. And the Eiffel compiler has much more
knowledge of
the intent of the programmer, so for example it can explicitly code
loads and
saves of values accessed through aliases (ie pointers) and help the C
optimiser
out.
An Eiffel compiler knows a lot about what's going on. One might even
consider
cases where the pre and post conditions could be used to help the
optimiser.
The delayed code generation also helps - hence the rather surprising
speed evidenced
by the SmallEiffel system in replacing a virtual function call with an
if/else
test and inlining the bodies.
James
PS if you want to see how well a C++ compiler can do when it compiles to
C, look at the
Kuck and Associates page. It gives the lie to any claims that a direct
compiler
will always do best.
Juergen Schlegelmilch
In article <DzrKG...@syd.csa.com.au>, Don Harrison wrote:
>
>What I'm saying is simply that high level code which is well written and
>compiled efficiently won't run as fast as low level code which is well written
>and compiled efficiently. This should be obvious.
I think this is not true in general. I expect high level code to
carry more semantics, so it can be optimized much better than low-level
code where you have less information. For example, the high-level
language might allow you to specify properties of operations such as being
a closure operator, commute with itself or other operators, or specifying
the neutral element of a user-defined data type. There is a lot of
information about user-defined operations that a compiler cannot find
by looking at the code alone; the declarations are necessary. In low-level
languages there are no language constructs to specify such properties,
so the compiler is left with the code which doesn't tell anything about
the algebraic properties of the operators it implements (except for
trivial cases). Another aspect is usage of statements that inhibit
optimisation: if a statement that makes optimisation hard (e.g. use of
pointers, aliasing) can be used by the programmer in any context, the
compiler has to be pessimistic, thus leaving it untouched. If the same
statement was generated from a high-level language where its use is
restricted to a few well-known cases, the compiler can optimize it since
it knows that the dangerous usage does not occur.
Juergen
--
+-----------------------------------------------------------------------------+
Dipl.-Inf. Juergen Schlegelmilch University of Rostock
email: schl...@Informatik.Uni-Rostock.de Computer Science Department
http://www.informatik.uni-rostock.de/~schlegel Database Research Group
Tel: ++49 381 498 3402 18051 Rostock
Fax: ++49 381 498 3426 Germany
+-----------------------------------------------------------------------------+
Matt Kennel
Juergen Schlegelmilch (schl...@pella.informatik.uni-rostock.de) wrote:
: In article <DzrKG...@syd.csa.com.au>, Don Harrison wrote:
: >
: >What I'm saying is simply that high level code which is well written and
: >compiled efficiently won't run as fast as low level code which is well written
: >and compiled efficiently. This should be obvious.
: I think this is not true in general. I expect high level code to
: carry more semantics, so it can be optimized much better than low-level
: code where you have less information.
I definitely understand your point, but
Can be != is.
The difference is that a human producing a "hand optimized low level"
algorithm for one specific circumstance is still much easier than creating
a robust and general purpose compiler optimization framework to do the same
thing.
And secondly, oftentimes the hand-optimization relies on domain knowledge
(say about distribution of run-time inputs) generally not available to
conventional compilers.
: Juergen
For instance, it appeared that in the recent bit-twiddling benchmark, one
was comparing an implementation in C which intentionally used bit-masking on
long integers, versus an Eiffel implementation of ARRAY[BOOLEAN].
The semantics of a general ARRAY[*] is quite different than a few bits
packed into an unsigned integer (consider arbitrary sizing) and that
the implementation of the ARRAY[*] generic can't easily be special cased for
one particular instantiation (BOOLEAN).
The point is that the C language doesn't even have the concept of
ARRAY[BOOLEAN].
So we are comparing a generic library data structure ARRAY[*] with a special
case, bit masking, value-style bit-twiddling. Maybe maybe maybe if the
compiler could prove 1) never more than 32 (or 64) bits will be used,
2) this thing is never aliased 3) BOOLEAN has no descendants and is a built in,
then maybe special case implementations devolving back into C-level bit
twiddling could be built into a compiler.
If one actually implements a bit-twiddling algorithm in Eiffel then it is
merely testing the compiler's ability to inline some classes of elementary
library operations, something which may be considered important for some
particular circumstances.
--
Matthew B. Kennel/m...@caffeine.engr.utk.edu/I do not speak for ORNL, DOE or UT
Oak Ridge National Laboratory/University of Tennessee, Knoxville, TN USA/
I would not, could not SAVE ON PHONE, |==================================
I would not, could not BUY YOUR LOAN, |The US Government does not like
I would not, could not MAKE MONEY FAST, |spam either. It is ILLEGAL!
I would not, could not SEND NO CA$H, |USC Title 47, section 227
I would not, could not SEE YOUR SITE, |p (b)(1)(C) www.law.cornell.edu/
I would not, could not EAT VEG-I-MITE, | /uscode/47/227.html
I do *not* *like* GREEN CARDS AND SPAM! |==================================
M A D - I - A M!
Michael Reiche
Matt Kennel wrote: (with deletions)
>
> The difference is that a human producing a "hand optimized low level"
> algorithm for one specific circumstance is still much easier than creating
> a robust and general purpose compiler optimization framework to do the same
> thing.
Yeah, but we're not talking 50% slower here. I've got code that runs
50+ times slower in Eiffel than the equivalent C code.
(finalized, assertions(no), inlining(yes), array_optimization(yes))
>
> For instance, it appeared that in the recent bit-twiddling benchmark, one
> was comparing an implementation in C which intentionally used bit-masking on
> long integers, versus an Eiffel implementation of ARRAY[BOOLEAN].
Whoaa!!! Where the Eiffel program used an ARRAY[BOOLEAN], the C
program used int key[128]. Where the Eiffel program used BIT 8,
the C program used masking. But that's irrelevant.
>
> The semantics of a general ARRAY[*] is quite different than a few bits
> packed into an unsigned integer (consider arbitrary sizing) and that
> the implementation of the ARRAY[*] generic can't easily be special cased for
> one particular instantiation (BOOLEAN).
>
> The point is that the C language doesn't even have the concept of
> ARRAY[BOOLEAN].
(At least it runs fast enough to use.)
I'm not saying that C is better than Eiffel just because it's faster.
I would expect it to be maybe 50% slower, not 50 times slower.
Remember that it was the Eiffel compiler vendors who told us just how
efficient it was.
>
> So we are comparing a generic library data structure ARRAY[*] with a special
> case, bit masking, value-style bit-twiddling.
-- stuff deleted
I was comparing the same job done in two different languages, I used
the BIT 8 class 'cause it seemed to fit the application.
The point of the program was to demonstrate that, contrary to marketing
hype, Eiffel is far from being as efficient as C.
Mike Reiche
CSE/DND Canada
Don Harrison
Robb Nebbe writes:
:Don Harrison wrote:
:
:> What I'm saying is simply that high level code which is well written and
:> compiled efficiently won't run as fast as low level code which is well
:> written and compiled efficiently. This should be obvious.
:
:Actually this isn't obvious at all.
Oh well. Sounded good for a short while. :)
:There are even counter examples. When
:you write in a lower-level language you almost invariably wind up "over
:specifying" the code since implementation details are practically
:unseparable from the intended semantics. The consequence is that you
:inhibit certain useful optimizations.
Juergen's second point.
:High-level languages sometimes define semantics that inhibit optimization
:as well. In the absence of such "design choices" it is often much easier to
:determine which optimizations will preserve the semantics. The downside is
:that there is more optimization required to reach the same performance
:level of the low-level language.
:
:It does not automatically follow that just because a language is higher
:level than some other language it is necessarily less efficient. At the
:same time it is understandable when they are not because it isn't an easy
:task to design or optimize high-level languages.
Agree. And probably simpler languages (such as Eiffel) would be easier
to optimise than more complex ones (such as Ada).
Don Harrison
Juergen Schlegelmilch writes:
:In article <DzrKG...@syd.csa.com.au>, Don Harrison wrote:
:>
:>What I'm saying is simply that high level code which is well written and
:>compiled efficiently won't run as fast as low level code which is well written
:>and compiled efficiently. This should be obvious.
:
:I think this is not true in general.
True.
:I expect high level code to
:carry more semantics, so it can be optimized much better than low-level
:code where you have less information.
True.
[...]
: Another aspect is usage of statements that inhibit
:optimisation: if a statement that makes optimisation hard (e.g. use of
:pointers, aliasing) can be used by the programmer in any context, the
:compiler has to be pessimistic, thus leaving it untouched. If the same
:statement was generated from a high-level language where its use is
:restricted to a few well-known cases, the compiler can optimize it since
:it knows that the dangerous usage does not occur.
Agree.
However, my statement does hold for the special case where the 'well
written' C program is the code produced by compiling the Eiffel program.
(It reflects a good high level design and is consistently implemented, at
least). This program can be further optimised by hand and made more
efficient. I'm not saying it would be easy to do (or it should be done),
just that it is possible.
Of course, the same would not necessarily be true of a well written C
program coded from scratch. I originally failed to make this distinction.
Don Harrison
James Mansion writes:
:Don Harrison wrote:
:>
:> What I'm saying is simply that high level code which is well written and
:> compiled efficiently won't run as fast as low level code which is well written
:> and compiled efficiently. This should be obvious.
:>
:> More specifically, well written Eiffel (high level) which is translated into C
:> (low level) and subsequently compiled will run slower than well written C
:> compiled by the same C compiler.
:
:I don't think that's true. First of all (in your previous post) you
:suggested
:that it might not matter too much.
No, I've said other things that aren't true. But not that, at least. :)
:The trouble is that for some
:applications
:you do have to twiddle bits and in those cases you really don't want to
:have
:to break out to a lower level lenguage - you want to be able to code the
:same
:stuff in the high level language.
Agree. You should be able to do everything in the high level language.
Though, in order to do some things, you may need to sacrifice safety eg.
for type conversions. Also, you should probably have to put a sign up to
publicise that you are breaking the rules as Modula 3 does by declaring
a module to be 'UNSAFE'.
Could such an approach be used in Eiffel? ...
unsafe class DODGY
-- dodgy stuff
end
:Secondly, the input to the C compiler that an Eiffel compiler can
:generate do
:not have the same constraints on readability and maintainability
:(cough!) that
:a C programmer must use. And the Eiffel compiler has much more
:knowledge of
:the intent of the programmer, so for example it can explicitly code
:loads and
:saves of values accessed through aliases (ie pointers) and help the C
:optimiser
:out.
Juergen's first point (about extra semantic information).
:An Eiffel compiler knows a lot about what's going on. One might even
:consider
:cases where the pre and post conditions could be used to help the
:optimiser.
Yes.
Alexander Kjeldaas
[ I changed the title - we're not talking about C++ vs. eiffel here..]
Michael Reiche <mre...@cse.dnd.ca> writes:
> Yeah, but we're not talking 50% slower here. I've got code that runs
> 50+ times slower in Eiffel than the equivalent C code.
> (finalized, assertions(no), inlining(yes), array_optimization(yes))
>
I've used some time implementing the necessary functionality in
SmallEiffel and looking at the problem of optimizing the crc algorithm
from the sources at hand. I won't publish the results yet as I haven't
figured out where some of the deficiencies of the eiffel-generated
assembly-output originates from. But there are a few things I'd like
you to be aware of.
The eiffel and C versions are not yet comparable IMHO. Earlier I
looked at the C version and changed it somewhat to look more like the
eiffel version and at the same time the performance increased. Now I
have a C-code generated by SmallEiffel and it is clear that the
semantics of the inner loop in the eiffel version is quite different
from the c version. IMO, the bug is in the BIT class which doesn't
include the efficient operations that are necessary to express the
same semantics as the C version does. Look at the following lines of
eiffel vs. C code:
Eiffel1:
Result.xor_item(bit7,2);
Result.xor_item(bit7,7);
Eiffel2:
Result.put((Result.item(6)) xor bit7, 6)
Result.put((Result.item(1)) xor bit7, 1)
C1:
#define m_shift(a,b) ((a)<<(b))
ccrc ^= m_shift(bit7,6) | m_shift(bit7,1);
C2:
#define m_put(a,r,i) { int _r=(r); \
(a)&=~(1<<(i)); \
(a)|=_r<<(i); }
#define m_item(a,i) (!! ((a) & (1<<(i))) )
m_put(ccrc, bit7 ^ m_item(ccrc,6),6);
m_put(ccrc, bit7 ^ m_item(ccrc,1),1);
I think it's pretty clear that the Eiffel1 semantics resembles C1,
while the Eiffel2 semantics resembles C2 [You may choose to implement
m_put differently as some architectures, notably the PowerPC, can do
this operation way more efficient, but I'm testing this on a pentium
pro]. So far there seems to be no way to get C2 to execute nearly as
fast as C1.
The problem with the BIT class, however is that the feature xor_item
does not exist! In SmallEiffel I added this feature (along with all
the other bit operations) to get some performance. I would expect the
other vendors to be able to show significantly better results using
this feature instead of put.
The problem I'm currently facing with SmallEiffel is that it emits C
generated functions in an order which prohibits gcc from inlining
them. Secondly, it doesn't support using the gcc keyword 'inline' to
inline small functions. Cheating somewhat and moving the emitted
functions to the top of the file and adding the 'inline' keyword shows
that eiffel-code _could_ perform within what you'd expect. I would
expect that the other vendors got this right [but as usual, don't take
my word for it!].
> > The semantics of a general ARRAY[*] is quite different than a few bits
> > packed into an unsigned integer (consider arbitrary sizing) and that
> > the implementation of the ARRAY[*] generic can't easily be special cased for
> > one particular instantiation (BOOLEAN).
> >
> > The point is that the C language doesn't even have the concept of
> > ARRAY[BOOLEAN].
>
> (At least it runs fast enough to use.)
> I'm not saying that C is better than Eiffel just because it's faster.
> I would expect it to be maybe 50% slower, not 50 times slower.
> Remember that it was the Eiffel compiler vendors who told us just how
> efficient it was.
I would expect that the semantics of the array classes used in the
vendor-optimized versions is similar to SIMPLE_ARRAY which I use in
the SmallEiffel-optimized version. SIMPLE_ARRAY corresponds directly
to an array in C and as such does not influence the performance of the
algorithm.
astor
--
Alexander Kjeldaas | Contrary to popular belief, Unix
<as...@guardian.no> | is user friendly. It just happens to be
Guardian Networks AS | selective about who it makes friends with.
Roger Browne
James Mansion wrote:
> The trouble is that for some applications you do have to twiddle bits and
> in those cases you really don't want to have to break out to a lower
> level lenguage - you want to be able to code the same stuff in the
> high level language.
I agree completely. Lots of Eiffel sales have been lost because vendors
sometimes take the attitude that the user can do all the low-level stuff in C.
There are many Delphi users who would like to move up to Eiffel, but they
regard a move from Delphi to an Eiffel/C mixture as a backwards step.
Don Harrison wrote:
> Agree. You should be able to do everything in the high level language.
> Though, in order to do some things, you may need to sacrifice safety eg.
> for type conversions.
I'm not convinced that this is true. In other words, I believe that you can
add low-level facilities to a language without compromising the safety of the
existing high-level facilities. Of course the low-level facilities have the
potential for misuse, but they do not make the existing high-level facilities
any less safe.
Naturally you can't do low-level work without the ability to access the
bitwise representation of structures, or to set up interrupt handlers,
or to copy bytes from one memory location to another. But these operations
can be expressed within the existing strongly-typed Eiffel notation.
> Also, you should probably have to put a sign up to publicise that you
> are breaking the rules as Modula 3 does by declaring a module to be
> 'UNSAFE'.
I don't think an "unsafe" flag is needed, because these specialised features
(e.g. memory_copy) are their own "unsafe" flag. But in any case,
no new language construct is required - the vendor can simply supply these
features in class UNSAFE.
Dietmar Wolz
Perhaps someone can help me with the ISE-code below. It works with
"collection_off" and not with "collection_on". There must be a bug
either in the code or in the GC.
Add additional output inside the loop to check whats
going on.
--------------optimized Vendor "B" version-------------------
-------------------------------------------------------------
-- The correct CRC is 0xb4, or 11010100, or 00101011
-- depending how you look at it
------------------------------------------------------------
class BITS_TEST
inherit MEMORY
creation
make
feature -- Initialization
make is
local
i : INTEGER;
z : INTEGER;
ccrc : BIT 8;
key : ARRAY[BIT 8]
do
-- collection_off
!!key.make(0,127);
-- set bits 1-120 to 0 0 0 0 0 1 0 0 0 0 1 ...
from i:=key.lower until i>key.upper-8 loop
if (i \\ 5) = 0 then
key.put(00000001B, i)
else
key.put(00000000B, i)
end
i:=i+1;
end;
-- set bits 121-128 to 0 0 0 0 0 0 0 0
from i:=key.upper-8 until i>key.upper loop
key.put(00000000B, i);
i:=i+1;
end;
-- calculate CRC 10000 times
from z:=1 until z> 10000 loop
ccrc := parity(key.area);
-- print(ccrc); print("%N");
z:=z+1;
end;
print(ccrc); print("%N");
end
parity (key : SPECIAL[BIT 8]) : BIT 8 is
local
i : INTEGER;
do
Result :=00000000B;
from i := 0 until i > 127
loop
if Result.item(1) then
Result:= ((Result ^ -1) or (key.item(i))) xor
((00000001B ^ -7) or (00000001B ^ -2) or 00000001B)
else
Result:= (Result ^ -1) or (key.item(i))
end
i := i + 1
end
end
end
Joachim Durchholz
do...@syd.csa.com.au wrote 25.10.96:
> Agree. You should be able to do everything in the high level language.
So do I.
> Though, in order to do some things, you may need to sacrifice safety eg.
> for type conversions.
Huh? I'd be hard pressed to find a non-contrived example for that in
Eiffel. Especially since the advent of the ?= operator (and I'd reserve
even that for quick hacks that I can't to cleanly in time, and for
importing objects from the untyped external world).
Though I actually can think of two such things: memory access and
specifying processor instructions directly. Both is useful for hand-
optimizing the code, for device drivers, and operating system kernels.
Memory access could be done via a class MEMORY. The mere presence of
MEMORY on the list of ancestors would be ample warning that unclean things
are about to happen, so no need for an "unsafe" keyword here.
For processor instructions, I don't actually think they are inherently
unclean. The only place to write them would be in a routine, which is the
perfect place to put a few preconditions and postconditions to fix the
semantics and make them clean (similar to the way OOSC proposes to
"dignify" external C routines).
Dominique Colnet
|> The problem I'm currently facing with SmallEiffel is that it emits C
|> generated functions in an order which prohibits gcc from inlining
|> them. Secondly, it doesn't support using the gcc keyword 'inline' to
|> inline small functions.
I have some idea to produce BIT C code exactely as fast as C.
It is not a problem at all ... but is it not in my priority list :-)
I think that the most important is dynamic binding and I think that
SmallEiffel is one of the best from this point of vue.
I am now testing release -0.88 before delivery. As the delivery test
has begun, I don't want to change BIT compilation in -0.88.
I will certainly work on BIT compilation for release -0.87 if people
continue to compare Eiffel from the BIT point of vue (I will try to
be the best if I can :-).
Thanks
--
------------------------------------------------------------
Dominique COLNET -- Talk Eiffel with SmallEiffel Talk Eiffel
C.R.I.N. (Centre de Recherche en Informatique de Nancy)
POST: CRIN,BP 239,54506 Vandoeuvre les Nancy Cedex,FRANCE
EMAIL: col...@loria.fr VOICE:+33 0383593079 FAX:+33 0383413079
Robb Nebbe
Don Harrison wrote:
>
> Robb Nebbe writes:
>
> :It does not automatically follow that just because a language is higher
> :level than some other language it is necessarily less efficient. At the
> :same time it is understandable when they are not because it isn't an easy
> :task to design or optimize high-level languages.
>
> Agree. And probably simpler languages (such as Eiffel) would be easier
> to optimise than more complex ones (such as Ada).
What is important is the ability to determine what is semantically important in
order to asses the applicability of certain types of optimizations. This is
somewhat orthogonal to the issue of complexity.
For example Ada's numeric types a relatively complex because you can declare
your own by specifying the range and other characteristics such as the delta
for fixed point types or the signifigant digits for floating point types. These
define the semantics. The compiler may then choose, in the absence of further
representation clauses, any implementation that guarantees the semantics
required by the declaration. Most languages define the semantics by specifying
the implementation.
Most of the complexity in Ada, when compare to Eiffel, comes from separating
semantics from implementation. Efficiency was a major concern and they
recognized that separating semantics from implementation was important for
optimization.
The real question is if the extra effort required to implement the language
would have been better spent elsewhere in terms of improving the efficiency of
the generated code. I don't have enough competence in the domain to answer this
particular question.
Robb Nebbe
Alexander Kjeldaas
Alexander Kjeldaas <as...@lucifer.guardian.no> writes:
> I would expect that the semantics of the array classes used in the
> vendor-optimized versions is similar to SIMPLE_ARRAY which I use in
> the SmallEiffel-optimized version. SIMPLE_ARRAY corresponds directly
> to an array in C and as such does not influence the performance of the
> algorithm.
>
[It's called FIXED_ARRAY in smalleiffel. My mistake]
Actually it is not true that FIXED_ARRAY corresponds to an array in C
as I recently discovered. FIXED_ARRAY in smalleiffel is the following
structure:
struct sT34{int id;void * _storage;int _capacity;int _upper;};
which requires _two_ indirections to get to one array element. This
requires an extra intermediate register on the x86 platform increasing
register pressure. The result is that the gcc allocates registers
badly for the inner loop in the parity function. And this in turn
makes the smalleiffel+cheat version perform badly. This should,
however, be possible to fix by trading the load for an add. On the
other hand, this should not be a major problem on RISC architectures
where you can have more 'base' pointer registers available.
Having found what seems to be the sorce of the performance
degradation, here are the results of the smalleiffel+cheat version
running on my system:
100_000 iterations:
smalleiffel+cheat:
gcc bits_test.c -O3 -fomit-frame-pointer
0.11user 1.28system 0:01.39elapsed 99%CPU
c-translation:
gcc ccrc.c -O3 -fomit-frame-pointer
0.07user 0.65system 0:00.73elapsed 98%CPU
astor
--
Alexander Kjeldaas | Contrary to popular belief, Unix is user
<as...@guardian.no> | friendly. It just happens to be selective
Guardian Networks AS | about who it makes friends with.
Don Harrison
Robb Nebbe writes:
:Don Harrison wrote:
:
:> Agree. And probably simpler languages (such as Eiffel) would be easier
:> to optimise than more complex ones (such as Ada).
:
:What is important is the ability to determine what is semantically important in
:order to asses the applicability of certain types of optimizations.
Yes.
:This is
:somewhat orthogonal to the issue of complexity.
However, isn't it true that less semantics (as with Eiffel), imposes fewer
constraints on what optimisations may be used thus making the task easier?
:For example Ada's numeric types a relatively complex because you can declare
:your own by specifying the range and other characteristics such as the delta
:for fixed point types or the signifigant digits for floating point types. These
:define the semantics. The compiler may then choose, in the absence of further
:representation clauses, any implementation that guarantees the semantics
:required by the declaration. Most languages define the semantics by specifying
:the implementation.
:
:Most of the complexity in Ada, when compare to Eiffel, comes from separating
:semantics from implementation. Efficiency was a major concern and they
:recognized that separating semantics from implementation was important for
:optimization.
:
:The real question is if the extra effort required to implement the language
:would have been better spent elsewhere in terms of improving the efficiency of
:the generated code. I don't have enough competence in the domain to answer this
:particular question.
Likewise.
Don Harrison
Roger Browne wrote:
:Lots of Eiffel sales have been lost because vendors
:sometimes take the attitude that the user can do all the low-level stuff in C.
:
:There are many Delphi users who would like to move up to Eiffel, but they
:regard a move from Delphi to an Eiffel/C mixture as a backwards step.
Understandable. Having to interface to C gives the impression that the
language is "not quite there yet".
:> ... You should be able to do everything in the high level language.
:> Though, in order to do some things, you may need to sacrifice safety eg.
:> for type conversions.
:
:I'm not convinced that this is true. In other words, I believe that you can
:add low-level facilities to a language without compromising the safety of the
:existing high-level facilities.
This is what I'm suggesting ought to be the case: add the requisite low-level
facilities in ways that do not compromise the safety of existing facilities.
:Of course the low-level facilities have the
:potential for misuse, but they do not make the existing high-level facilities
:any less safe.
:
:Naturally you can't do low-level work without the ability to access the
:bitwise representation of structures, or to set up interrupt handlers,
:or to copy bytes from one memory location to another.
Agree.
:But these operations
:can be expressed within the existing strongly-typed Eiffel notation.
:
:> Also, you should probably have to put a sign up to publicise that you
:> are breaking the rules as Modula 3 does by declaring a module to be
:> 'UNSAFE'.
:
:I don't think an "unsafe" flag is needed, because these specialised features
:(e.g. memory_copy) are their own "unsafe" flag. But in any case,
:no new language construct is required - the vendor can simply supply these
:features in class UNSAFE.
Yes, that approach could be taken. If all the required low-level facilities
can be expressed in this way, (or in a configuration file), this solution
would have the advantage of not requiring any new construct. This is
true of type conversion. In the case of interrupt handlers, we would need
some way of assigning operations to specific addresses. I'm not aware of any
existing mechanism for doing this. For bit fiddling, features such as
functions size_in_bits, address_of may be useful (Are these already
available?).
The approach would follow the precedent of other facilities that bypass
Eiffel's safety features (eg. assigning integer values to descendants of
INTEGER_REF: set_item?)
I would prefer that unsafe facilities were more obvious than showing up in
an inheritance hierarchy because the classes that use them can be given
innocuous names, hiding the fact that potentially dodgy things are happening.
But the downside is added complexity in the language. :(
--
Don Harrison
Joachim Durchholz writes:
:> Though, in order to do some things, you may need to sacrifice safety eg.
:> for type conversions.
:
:Huh? I'd be hard pressed to find a non-contrived example for that in
:Eiffel. Especially since the advent of the ?= operator (and I'd reserve
:even that for quick hacks that I can't to cleanly in time, and for
:importing objects from the untyped external world).
You need something more than the ?= operator if you need to process an object
as a bunch of bytes, for example.
:Though I actually can think of two such things: memory access and
:specifying processor instructions directly. Both is useful for hand-
:optimizing the code,
Don't know about anyone else, but I'd rather have the compiler do this
for me.
:for device drivers, and operating system kernels.
:
:Memory access could be done via a class MEMORY. The mere presence of
:MEMORY on the list of ancestors would be ample warning that unclean things
:are about to happen, so no need for an "unsafe" keyword here.
If I understand Roger's suggestion, it would be best to have a single class
UNSAFE where unsafe *routines* appeared. Functions from anywhere could be
used, of course (including MEMORY).
Matt Kennel
Don Harrison (do...@syd.csa.com.au) wrote:
: :This is
: :somewhat orthogonal to the issue of complexity.
: However, isn't it true that less semantics (as with Eiffel), imposes fewer
: constraints on what optimisations may be used thus making the task easier?
No, not necessarily: consider Smalltalk without value style variables.
This language has more 'uniform' and 'simpler' semantics than Ada or Eiffel or
Sather or whatever, but is not easier to optimize.
'More semantics' might also mean that the compiler can *assume more*, which
is generally a prerequisite to successful optimization.
For example, "pointers to anything" are conceptually simple but present
optimization problems (aliasing) because their properties are insufficiently
compile-time provable.
Bertrand Meyer
Dietmar Wolz wrote:
>
> Perhaps someone can help me with the ISE-code below. It works with
> "collection_off" and not with "collection_on". There must be a bug
> either in the code or in the GC.
> Add additional output inside the loop to check whats
> going on.
[Code extract removed.]
We have reproduced the problem at ISE. This is a bug with BITS
that has been fixed (along with others in this area and in expanded
types) for ISE Eiffel version 4.0, including the current beta versions.
We tested Mr. Wolz's example with 4.0 and everything works fine.
It was a code generation problem, not a GC problem. This means
the fix cannot be retrofitted to ISE Eiffel 3
We'll be glad to give Mr. Wolz access to 4.0i (the current
beta) of 4.0j (the next one, scheduled for next
week). Just get in touch with <sup...@eiffel.com> to tell us
which one you want.
For information about ISE Eiffel 4 see our Web page
at http://www.eiffel.com.
--
Bertrand Meyer, ISE Inc., Santa Barbara (California)
805-685-1006, fax 805-685-6869, <bert...@eiffel.com> - ftp://ftp.eiffel.com
Web home page: http://www.eiffel.com