This is NOT flamebait... it just has me concerned that by deciding on
Python (after becoming semi-proficient in PHP and Perl) I might be making a
mistake.
Python is not slow at all. There are however tasks that you would not
normally do in a scripting language (heavy number crunching etc.). One
of the good things about Python is that you can still write those types
of applications in Python and only write the performance critical parts
in C/C++ versus writing the entire application in C/C++.
Many of us also have to deal with large libraries of pre-existing C/C++
libraries that we can't afford to rewrite in Python. Since it is almost
trivial to wrap these libraries as Python extension modules it lets us
have the best of two worlds.
I'm not a Perl expert, but from what I hear it is considerably harder to
extend/embed Perl (someone please correct me if I'm wrong <wink>).
-- bjorn
> I am a bit
> disheartened at how common it seems to code only some of a program in
> Python and use C for other parts, and how much discussion
> here involves using C for X and Y.
It is actually quite rare for most people using python to explicitly use
C<->Python directly. There are many modules which are written in C, however
for most users this is completely transparent - from Python they behave
exactly the same as a module written in Python.
> language. I don't
> see nearly as much discussion about doing this in the Perl newgroups.
That would probably be because it's a lot harder to do in Perl. It is
actually quite easy to write extensions in C for Python which will work on
every platform that Python itself runs on.
> Should I just be learning C instead? Is this just because of
> the different
> applications python is being put to use for or because Python
> programmers
> are more "multi-lingual" or is Python really so slow that one
> needs to go
> to C on a more regular basis?
Definitely do not learn C *instead* of Python. Python programmers do tend to
"use the best tool for the job". In most cases, the best tool is pure
python, using modules (e.g. Numeric Python) which are written in C/C++ to
improve performance when needed.
Most of the discussions you see are from people who are writing modules for
others to use, or who are investigating the Python codebase or C extension
codebase to scratch an itch.
Python is normally "fast enough", and in general is about the same
performance as perl (of course, the actual performance depends on the
specific task and the algorithm chosen). Most of the time poor performance
is due to a poor algorithm - it is much easier to get the algorithm right in
Python than in most languages.
Tim Delaney
What draws me to Python is:
1. The pragmatic approach it encourages.
2. The multi-paradigmatic approach it encourages (it's not just OO, it's
not just functional, it's not just procedural).
3. The clarity it makes possible.
4. Batteries are included (the standard library has lots to offer).
I've been using Python for almost a year and haven't felt the need to write
a C extension. Well, that's partially true. A colleague of mine wrote one
(to interface with Verisign's PayFlowPro SDK) and I fixed a tiny bug in it.
I wouldn't expect Verisign to provide a native Python API, so in this case
the fact that Python allows for C extensions is an expression, again, of its
pragmatic nature. In this case, the need for the C extension had nothing to
do with speed, but rather with utilizing a third party library.
I'd say if you find another language that suits your needs better than
Python, by all means that should be where you spend your energy.
Cheers,
// mark
Yes, you should learn C, but you'll find it's much more hassle to get
things done in C than in Python. So Pythoneers like to write the
framework of their program in Python, and use C for the parts that are
speed critical.
As others have noted, Python is the same speed as Perl, and the
use of C comes more from the desire to integrate with other
software than from a need for improving on Python's speed.
My comment is that in 2.5 years of writing Python code, with a
group averaging eight developers during that time all writing Python,
the only time we have had to write C code to work with Python was
to interface with a dual-port RAM device in an embedded system in
which we use Python -- not because of speed concerns.
We use Python for web work (via Zope), factory automation, utilities,
GUI applications, embedded systems and other areas. We pretty
much cover the range, and have not yet found a reason to write
any C code. Those who do for performance reasons are generally
working in specialized areas (e.g. image processing) or they
may be optimizing prematurely.
In any case, compared to the other "P" languages you mention,
Python is definitely not slower. Note however that none of
those other languages, including the "C" one :-), would be
as suitable for such a wide range of application areas as Python.
-Peter
Newsgroup discussion gives a very biased picture of language use. I
am sure that there are thousands of programmers quietly and
successfully using Python without necessarily knowing C(sharp) from
D(flat). Nothing much to post about except for the occasional
unsolicited testimonial.
Extending Python with C or embedding Python in C is doable but
nontrivial. Hence, people do it but sometimes run into problems and
questions which they bring to the newsgroup -- and which usually get
answered. One result of the effort and help is the availability of
several extensions written im C which the pure Python programmer can
make use of.
Terry J. Reedy
> I'm not a Perl expert, but from what I hear it is considerably harder to
> extend/embed Perl (someone please correct me if I'm wrong <wink>).
I've never looked into extending Python with C in terms of its
difficulty, but I can say that doing it in Perl is a real pain in the
proverbial. No doubt, once one is up to speed on it, it would probably
be less of a hassle, but for a new user, especially one with little C
experience, it can be daunting.
David Trudgett
Well, of course Python is going to be about an order of magnitude or
more slower than C just because of its interpreted nature. But, then,
it takes an order of magnitude more time to develop an application in
C, generally speaking.
For most applications, this speed difference doesn't matter too much,
and it's more important to get the application done quickly.
The same reasoning goes for Perl, though I find Perl to be generally
faster than Python for what I do with it [completely unscientific, take it
with a grain of salt], which is mostly text processing tasks, but plenty
of other things, as well, like TCP/IP network daemons.
There will often be places in a program that require speed, or that
must be written in a lower-level language for some other reason. I
don't see this as a problem, though. Why should one write the whole
application in C just to optimise 2% of it?
The bottom line is, not everyone can expect to do everything in Python or
Perl, and so C is the natural choice to supplement the capabilities of
the higher-level languages.
By the way, C is a "small" language and not hard to learn. It might be
hard to learn to program *well* in it, but that's another story. :-)
David Trudgett
I've heard that interfacing Perl & C is challenging. Still, there is
Inline::C.
--
Skip Montanaro (sk...@pobox.com - http://www.mojam.com/)
"Excellant Written and Communications Skills required" - seen on chi.jobs
Well, not really - this is almost the central issue: C code goes wrong in
subtle ways (eg, double-free() of a pointer, over-indexing an array),
Python doesn't (usually). When it goes wrong, Python encourages interactive
exploration of your code, whereas C, you have go into the time consuming
"edit-compile-debug-iterate" phase.
If you're new to programming, a language like Python would seem to be the
right place to start.
--
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
http://www.bagley.org/~doug/shootout/
// mark
It's worth mentioning the Numeric/Numpy module and associated extensions
provides a lot of C-coded number crunching goodness for the Python-only
coder.
I think you're seeing a lot of people discussing development of python
extensions more then that actual Python app development per se. I doubt that
most people create extensions on an on-going basis, but on this mailing list
and newsgroup extension development gets highlighted somewhat.
I just turned an associate on to Python and he's so happy with the speed
compared to Java, he's converting old programs from Java to Python for his
customers. He ranted for some time about the increase in speed and huge
reductions in coding effort and size too.
I think most scripters are more multi-lingual then their compiled language
peers. You yourself are adding Python to your Perl background :-)
David LeBlanc
Seattle, WA USA
> >By the way, C is a "small" language and not hard to learn. It might be
> >hard to learn to program *well* in it, but that's another story. :-)
>
> Well, not really - this is almost the central issue: C code goes wrong in
> subtle ways (eg, double-free() of a pointer, over-indexing an array),
> Python doesn't (usually). When it goes wrong, Python encourages interactive
> exploration of your code, whereas C, you have go into the time consuming
> "edit-compile-debug-iterate" phase.
>
> If you're new to programming, a language like Python would seem to be the
> right place to start.
That's quite right. I didn't mean to give the impression that C is a
good language to cut one's teeth on. Far from it. However, it is a
small language, and therefore easy to learn. That doesn't mean that
it's easy to write good programs in it. C goes out of its way to give
the programmer all the power required to shoot himself* in the foot.
It assumes the programmer knows exactly what he is doing -- a bad
assumption most of the time ;-)
David Trudgett
*Which is not to imply that all programmers are male. Apart from being
normal usage in several languages to use the masculine in the generic
sense, the fact is that most programmers probably are male!
> Should I just be learning C instead? Is this just because of the
> different
> applications python is being put to use for or because Python
> programmers
> are more "multi-lingual" or is Python really so slow that one needs to
> go
> to C on a more regular basis?
Some projects have are a mixture of Python and C. This is especially
true of heavy-duty library modules, either due for efficiency's sake or
because C acts as the glue between Python and some third-party library
or tool.
For your own personal projects, the question of whether Python alone
will be sufficient really depends on what you plan on doing. The answer
is likely that it will be more than satisfactory. While there certainly
are cases where applications are so CPU intensive that from the start,
Python alone would be inappropriate, these tasks are relatively rare,
particularly for user end stuff. Even with intensive applications, it's
not a question of whether it's as fast as possible, it's whether it's
fast _enough_. On a modern machine, Python is almost certain to be more
than sufficient for your needs, whatever they are.
--
Erik Max Francis / m...@alcyone.com / http://www.alcyone.com/max/
__ San Jose, CA, US / 37 20 N 121 53 W / ICQ16063900 / &tSftDotIotE
/ \ Who'd ever think it / Such a squalid little ending
\__/ The American and Florence, _Chess_
Church / http://www.alcyone.com/pyos/church/
A lambda calculus explorer in Python.
I didn't explain myself very well - the point I meant to make is that
you don't really know C until you've spent a decade investigating core
dumps (and not even then in some prominent cases)... 8-) 8-)
> I am a bit disheartened at how common it seems to code only some of a
> program in Python and use C for other parts, and how much discussion
> here involves using C for X and Y.
Hi Chris,
Bear in mind that the reason it gets discussed more than the simpler stuff
is because the simpler stuff is, well, simpler (so no need for
discussion). If you could somehow survey all Python users to see what
percentage have to end up writing C extensions you'd probably find that
percentage to be much lower than what you might infer from the discussions
around here.
Have fun,
Dave
I've spent a fair amount of time on Perl newsgroups, too, and writing
stuff in Perl. Here are my hypotheses on the causes of this difference:
1. Python is really slow, because it's highly dynamic and because it's
interpreted. This is true of Perl, too. Primitive operations in both
languages are roughly 100 to 1000 times slower than in languages
compiled to machine code. Perl is usually marginally faster than
Python, although there are some cases I've seen where it's drastically
faster. Both languages also use bucketloads of memory.
2. Perl programmers have a lot of basic problems with things like
failing to open files, leaving variables uninitialized, and
accidentally clobbering global variables that are inadvertently shared
between different parts of a program. Python catches these errors and
tells you about them, so you don't need to post to a newsgroup to find
out what went wrong. Fewer basic questions means the advanced
questions constitute a greater portion of the traffic.
3. Perl is extremely popular among novice programmers and
nonprogrammers (compared, anyway, to other programming languages.)
These people do not typically try to extend Perl (or Python) with C,
because it's a lot harder than writing stuff in Perl (or Python).
4. Perl is a pain in the butt to extend in C. Python is easy to
extend in C. So more people extend Python in C, but that still
requires writing C, which is comparatively more difficult than Python
to debug.
People usually write C extensions to either Python or Perl for one of
three reasons:
- their Perl or Python code is too slow or too space-inefficient
- they have other code (written in some other language) that they want to
incorporate into their program, so they have to write some C glue to
make it all work
- they have to do something that can only be done in a low-level language,
such as access to some system call; in a sense, this is the same as the
previous point
One project I'm working on at the moment has about 3000 lines of
Python and about 400 lines of C. The 400 lines of C allowed us to cut
our memory footprint by a factor of five.
>Actually I've created array and did something like this :
>for k in xrange(1,10000):
> for m in range(1,10000):
> x[m] = .... #some calculation of the value of an array's element x[m]
>and this work's realy slow compared to C. I don't mind it working 3 times
>slower, but ..
>Perhaps it is possible, to do something like implementing loops in C inside
>Python ??
>(though it seems to me it will not help)
Trey Pyrex .. it's a cool tool :)
http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/
Armin
> On Tuesday 2002-05-21 at 19:12:19 -0600, Bjorn Pettersen wrote:
>
> > I'm not a Perl expert, but from what I hear it is considerably harder to
> > extend/embed Perl (someone please correct me if I'm wrong <wink>).
>
> I've never looked into extending Python with C in terms of its
> difficulty,
I find I can do it whilst asleep, drunk and with my hands behind my
back. I have done it quite a lot by now, though...
Cheers,
M.
--
I also feel it essential to note, [...], that Description Logics,
non-Monotonic Logics, Default Logics and Circumscription Logics
can all collectively go suck a cow. Thank you.
-- http://advogato.org/person/Johnath/diary.html?start=4
Bjorn> I'm not a Perl expert, but from what I hear it is considerably
Bjorn> harder to extend/embed Perl (someone please correct me if I'm
Bjorn> wrong <wink>).
Skip> I've heard that interfacing Perl & C is challenging. Still,
Skip> there is Inline::C.
I wrote two extensions using perlxstut and was still none the wiser
about what was really going on. From the first bit of code I wrote
for my first Python extension, I had a pretty good idea about what was
happening.
So, from the statistically significant sample of one, I would say that
it is much harder to extend or embed Perl than it is for Python.
- Dave
From no doubt a very different background, my experience has shown the
number to be more in the range of 10 to 100 times slower, averaging
about 30 times. In fact, the 100 times was usually with non-Pythonic
code, written with fairly heavy for-loops and lots of strings being
concatenated inefficiently.
(In any case, clearly it is lots slower than C.)
-Peter
Hmm. I love Python, but I have not found it as fast as Perl. It seems to
run about the speed of Tcl 8.3, but Perl seems to leave it in the dust, as
least for the comparisons that I've done.
One of those was generating prime numbers, so it was highly CPU intensive
operations. The Python and Tcl versions were about the same speed. The Perl
was roughly 50-100% faster.
Mike
--
Michael P. Soulier, QX41, SKY Tel: 613-765-4699 (ESN: 39-54699)
Optical Networks, Nortel Networks, SDE Pegasus
"...the word HACK is used as a verb to indicate a massive amount
of nerd-like effort." -Harley Hahn, A Student's Guide to Unix
You're not wrong. Both Tcl, Python and Ruby use the concept of including a
simple header file, learning some new types and semantics, and you're off and
running. Easy.
Perl, on the other hand, uses the XS framework, which requires a bunch of
extra macros and such in your C to make it merge with Perl. It's a major pain,
and it's something they're planning to fix in Perl 6 (currently vapourware)
because they've been so heavily criticized for it.
What draws me is all of the above, plus coding in Perl for 8 hours a day.
Python is a relief after that. ;-)
?? How did Java score higher than Python? Any use for Java I've tried it's
been incredibly slow.
> I spend a fair amount of time on perl newsgroups, having the need to use
> perl for various parts of my job. I am now learning Python. I am a bit
> disheartened at how common it seems to code only some of a program in
> Python and use C for other parts, and how much discussion here involves
> using C for X and Y. I don't have time to learn another language. I don't
> see nearly as much discussion about doing this in the Perl newgroups.
> Should I just be learning C instead? Is this just because of the different
> applications python is being put to use for or because Python programmers
> are more "multi-lingual" or is Python really so slow that one needs to go
> to C on a more regular basis?
>
> This is NOT flamebait... it just has me concerned that by deciding on
> Python (after becoming semi-proficient in PHP and Perl) I might be making a
> mistake.
"Python" is *much* "faster" than "Perl", "php", or "C" at answering
this question:
What is the integer value of 2 raised to the power of 10000?
$ time python -c 'print 2**10000'
[answer snipped]
real 0m0.416s
user 0m0.410s
sys 0m0.000s
So what do you mean? Perl coders --- bring it! C coders -- bring it!
Faster here means getting the job done with the "canonical
implementation" of the language (that's why all of the languages were
quoted). Lisp hackers would snicker having endured years of this line
of badgering, no doubt, but they are off coding logistical systems for
airlines, or some such. I've read most of the responses to your
question, and for the most part the reiterate well thought out and
accurate advice give semi-weekly on this topic. Even so, why is it
that everyone loves chining in on this question?
To the specifics of your question: If you don't have time to learn
another language, then you will likely never realize any of the
benefits that Python, and learning Python, might have to offer. These
benefits are usually longer term, and if all you value is the
instantaneous, then learning something new may not be in your best
interest. Don't let the Python zealots tell you that you can't write
maintainable code in Perl, you most certainly can. However, to do so,
you require skill and discipline that require time and dedication to
cultivate. The same is true of Python. However, Python the language is
much more consistent and free of idiosyncrasies than is Perl the
language. You can be writing idiomatic Python code relatively quickly,
which means that you can leverage the experience of legions of Master
Python Hackers just by emulating them and figure out why it's a good
idea later. Please think about why this might be true.
I love learning Perl weirdness, but that for my spare time not eaten
by netnews. If I am trying to solve problems, I often "explore" the
problem interactively in Python. If it's straight forward, it's not
really a problem, it's more of a task, and you can simply apply "the
canonical pattern" to eliminate that task. As far as speed is
concerned, I find that the time consumed by design, development,
implementation, and maintenance far exceeds run time for most of the
stuff I do. I have *never* found an instance where the difference in
runtime speed between the standard Python and Perl implementations has
been a deal breaker, YMMV.
--
George Demmy
Layton Graphics, Inc
$ time python -c 'print 2**10000'
[answer snipped]
real 0m0.226s
user 0m0.070s
sys 0m0.010s
$ time clisp -q -x '(expt 2 10000)'
[answer snipped]
real 0m0.038s
user 0m0.030s
sys 0m0.000s
| Lisp hackers would snicker having endured years of this line of
| badgering, no doubt,
No doubt.
Jeremy.
$ time python -c 'print 2**10000'
real 0m0.347s
user 0m0.044s
sys 0m0.023s
$ lisp -noinit
CMU Common Lisp 18d, running on vista.netmemetic.com
* (time (expt 2 10000))
Compiling LAMBDA NIL:
Compiling Top-Level Form:
Evaluation took:
0.0 seconds of real time
5.0e-6 seconds of user run time
5.0e-6 seconds of system run time
0 page faults and
0 bytes consed.
(No idea if the two timings are comparable.)
--
Ng Pheng Siong <ng...@netmemetic.com> * http://www.netmemetic.com
I don't apply any special magic to my Python programs, and none of them
have ever left me waiting around for results. Until a few weeks ago, the
fastest computer I had at my disposal (other than a few work-related
exceptions) was a P-II 266, and every single Python program I ran on it
spat out its results in a flash. (This includes a host of programs people
have submitted to Useless Python that I have run just to see what they do.)
Python's speed is just fine.
Rob Andrews
http://uselesspython.com
Peter Hansen <pe...@engcorp.com> wrote previously:
|From no doubt a very different background, my experience has shown the
|number to be more in the range of 10 to 100 times slower, averaging
|about 30 times. In fact, the 100 times was usually with non-Pythonic
|code, written with fairly heavy for-loops and lots of strings being
|concatenated inefficiently.
I think Kragen and Peter are probably both right in their estimates,
because they are talking about slightly different things. If you just
compare a raw loop, for example, C is easily 1000 times faster, e.g.
for ln in lines: pass vs.
for (i=0; lines[i][0]; i++) {ln = lines[i];}
But once you *do* something inside a loop, Python typically makes use of
some higher-level facility that is implemented in a more-or-less optimal
way, e.g.
import re
new = []
for ln in lines:
new.append(re.sub('[Ff][Oo][Oo]','Bar',ln))
Doing that in C takes a lot more lines... but moreover, the lines you
actually write won't be any better than what re.sub() does internally.
Yours, Lulu...
--
mertz@ _/_/_/_/_/_/_/ THIS MESSAGE WAS BROUGHT TO YOU BY:_/_/_/_/ v i
gnosis _/_/ Postmodern Enterprises _/_/ s r
.cx _/_/ MAKERS OF CHAOS.... _/_/ i u
_/_/_/_/_/ LOOK FOR IT IN A NEIGHBORHOOD NEAR YOU_/_/_/_/_/ g s
Python in and of itself is a great language that stands on it's own merits!
It's faster then some, slower then others. Some will like it, some won't.
Don't worry, be happy.
Dave LeBlanc
Seattle, WA USA
i.e. return 1L<<exponent
I'm guessing the reason is "no one really uses this much, so there's not
much point putting the effort into the change", but I'm still curious if
I'm missing something about 2-exponentiation.
Given that the test was stated as an English word problem, the 1L<<10000
solution would probably even be acceptable in the speed tests. We just
need to restate the problem as:
"What is the integer/long value of 2 raised to the power of 10000000?"
And Python will be universally acknowledged as the speediest language on
the planet ;) .
Mike
In another thread Jeremy Yallop wrote:
> * George Demmy
> | "Python" is *much* "faster" than "Perl", "php", or "C" at answering
> | this question:
> |
> | What is the integer value of 2 raised to the power of 10000?
...
_______________________________________
Mike C. Fletcher
http://members.rogers.com/mcfletch/
> I've noticed the use of 2**HUGENUMBER as a speed test a couple of
> times now. I'm curious why the language(s) doesn't (don't) optimise
> 2**X operations for really huge exponents. Python obviously optimises
> 1**x (since 1**1000000 returns instantly), so why not 2**x where x >
> (30 or 62)?
I guess I didn't do a good job of making the point that the "speed" of
a language and its implementation depends hugely on context. I was
able to go from problem to solution in less than two seconds (I'm not
the worlds fastest typest...) for the toy problem. Execution time was
less than half a second, the other second and a half being spent
typing
python -c 'print 2**10000'
OK, I lied. Probably closer to five seconds typing, but it better
makes the point. Using clisp (in my limited experience, a *much*
faster bignum implementation, btw, than Guido Python), I could have
shaved four tenths of a second... either way, execution time was a
small fraction of total execution time.
>
> i.e. return 1L<<exponent
>
> I'm guessing the reason is "no one really uses this much, so there's
> not much point putting the effort into the change", but I'm still
> curious if I'm missing something about 2-exponentiation.
Probably not missing anything about 2-exponentiation. 7 ** 10000 would
have been find. I confess, I "rigged" it to eliminate the obvious Perl
one-liner. Perl would the spit out a floating point answer just
fine. But I'm unrepentant in the rigging -- "Is python fast 'enough'?"
is a loaded question!
> Given that the test was stated as an English word problem, the
> 1L<<10000 solution would probably even be acceptable in the speed
> tests. We just need to restate the problem as:
>
> "What is the integer/long value of 2 raised to the power of
> 10000000?"
Problems are usually specifed in words... or less! I always get a
chuckle (in the gallows humor sense) when I see the folks trying to
elicit just what it is that the customer wants. Much of my time as a
programmer is spent exploring problems and trying to nail down specs
not hacking :(
How long does it take to fire up emacs (OK, zero, it's *always*
running), edit the c code, compile, and exectute. Again, illustrating
that "speed" can (and should!) be measured more ways than one.
> And Python will be universally acknowledged as the speediest language
> on the planet ;) .
>
> Mike
Wink aside, if you're interested in flat out execution speed for
bignums, you might check out clisp... Optimization of 2**HUGEL may be
premature at this point. I like Python and the Guido Python
implementation because it lets me go from problem statement to
solution quickly for a wide variety of problems... small and
large.
G
emap = {
2:1,
4:2,
8:3,
16:4,
32:5,
64:6,
128:7,
256:8,
512:9,
1024:10,
2048:11,
4096:12,
8192:13,
16384:14,
32768:15,
65536:16,
131072:17,
262144:18,
524288:19,
1048576:20,
2097152:21,
4194304:22,
8388608:23,
16777216:24,
33554432:25,
67108864:26,
134217728:27,
268435456:28,
536870912:29,
1073741824:30,
}
def dumbpow( number, exponent ):
# is number 1?
if abs(number) == 1 or exponent <= 0:
return number**exponent
# is number a power of 2?
size = emap.get(abs(number))
if size:
size = 1L << (size*exponent)
if exponent %2 and number < 0:
return -size
else:
return size
else:
return number ** exponent
Sure, it's an extra map call, but oh well, pointless "optimisation" is
fun :) .
Enjoy,
Mike
Mike C. Fletcher wrote:
...
> "What is the integer/long value of 2 raised to the power of 10000000?"
...
I am in a situation where I want to learn a language that is capable
of being used to make coherent, larger applications. I know it can be
done in Perl, but my experience with it leads me to think that *I*
need something different. After investigating various languages, it
came down to either Python or C (for now at least). But as I delved
into Python more I became nervous that the implication of the regular
discussions hereabouts that involve dropping out of python into C at
X, Y and Z might be that python by itself is not up to this task.
I was beginning to feel like the unspoken sentiment was "if you want
to do something really cool in python, you will need to learn C as
well"
I am glad to find out otherwise for all the good reasons given in the
earlier thread. I simply don't have time right now to learn two more
languages and want to focus my efforts on that which will be of the
most use for me... performance is an issue, but not the ultimate
issue!
> On Tue, 21 May 2002 20:34:01 -0500, Mark McEahern <mark...@mceahern.com>
> wrote:
>> For what it's worth, the OP might want to check this out:
>>
>> http://www.bagley.org/~doug/shootout/
>
> ?? How did Java score higher than Python? Any use for Java I've tried it's
> been incredibly slow.
Any Java performance evaluation that doesn't take into account the effect of
JITC/HotSpot is worthless. Unless the test program runs for quite a while,
runtime will include byte-code interpretation (before the frequently-visited
code is compiled) and compilation.
-- F
Why bother? i**j is performed by a general algorithm that does a number of
long-int multiplications, but no more than 2*ceiling(log_base_2(j)) of them.
i==1 is not a special case; it zooms simply because all of the O(log(j))
intermediate products are exactly 1, so go very quickly.
> I'm guessing the reason is "no one really uses this much, so there's not
> much point putting the effort into the change", but I'm still curious if
> I'm missing something about 2-exponentiation.
No, there's simply no point to burning code space to special-case power-of-2
bases; if you know your base is a power of 2, feel free to shift yourself --
I do <wink>.
Exponentiation can be overloaded. Look at __pow__ ...
Tim Delaney
> Any Java performance evaluation that doesn't take into account the effect of
> JITC/HotSpot is worthless. Unless the test program runs for quite a while,
> runtime will include byte-code interpretation (before the frequently-visited
> code is compiled) and compilation.
As long as what you're trying to evaluate will be a long running
program. Otherwise, regardless of how well your test program may
eventually do after runtime permits the compilation, it's still going
to be the interpretation overhead in typical use (e.g., for short
tasks).
As usual, benchmarking and performance evaluation is an area fraught
with peril :-)
--
-- David
--
/-----------------------------------------------------------------------\
\ David Bolen \ E-mail: db...@fitlinxx.com /
| FitLinxx, Inc. \ Phone: (203) 708-5192 |
/ 860 Canal Street, Stamford, CT 06902 \ Fax: (203) 316-5150 \
\-----------------------------------------------------------------------/
> really huge exponents. Python obviously optimises 1**x (since 1**1000000
> returns instantly), so why not 2**x where x > (30 or 62)?
Obviously? It actually doesn't optimize it at all, it's just that you
can calculate 1*1 a lot faster than (2**99999999)*2
--
Christopher A. Craig <com-n...@ccraig.org>
"No one has ever called us untrustworthy" -- Steve Balmer
Want to post the implementations used for both? My guess is that
the Python version was not very Pythonic...
-Peter
You're right. I didn't catch the "primitive operations" part
of Kragen's post... no doubt Python is up to 1000 times slower
doing little baby steps like that.
-Peter
Maybe I'll just give up on trying to interest people in Python
optimisation challenges. The good-old days of idle speculation and
optimisation are dead. Time to stop living in the past. Idea dropped,
back to boring GUI work.
*poof*
Tim Peters wrote:
> [Mike C. Fletcher]
...
> Why bother? i**j is performed by a general algorithm that does a number of
> long-int multiplications, but no more than 2*ceiling(log_base_2(j)) of them.
> i==1 is not a special case; it zooms simply because all of the O(log(j))
> intermediate products are exactly 1, so go very quickly.
...
> No, there's simply no point to burning code space to special-case power-of-2
> bases; if you know your base is a power of 2, feel free to shift yourself --
> I do <wink>.
...
Cheer up!
> Maybe I'll just give up on trying to interest people in Python
> optimisation challenges. The good-old days of idle speculation and
> optimisation are dead. Time to stop living in the past. Idea dropped,
> back to boring GUI work.
Don't say such sad words.
Nothing is dead but maybe some of our expectations.
Python was never eager to adopt new features (which
is good), but putting ideas down this short wasn't
usual, too.
I hereby encourage you to write that in the best
way you can, since I know of a Python branch
that will happily adopt any short and sensible optimization.
I can contribute a bunch to this, too.
> Tim Peters wrote:
>
>> [Mike C. Fletcher]
>
> ...
>
>> Why bother? i**j is performed by a general algorithm that does a
>> number of
>> long-int multiplications, but no more than 2*ceiling(log_base_2(j)) of
>> them.
>> i==1 is not a special case; it zooms simply because all of the O(log(j))
>> intermediate products are exactly 1, so go very quickly.
>
> ...
>
>> No, there's simply no point to burning code space to special-case
>> power-of-2
>> bases; if you know your base is a power of 2, feel free to shift
>> yourself --
>> I do <wink>.
Viruses pretending to be real emails are becoming
a problem. This one was just simple to detect:
The real Timbot would either implement it in less
bytes than the text of this response, or propose
to provide an appropriate optional extension mechanism
for special optimization modules.
dissing-neat-stuff-is-another-wise-mans-job<snort>-ly y'rs - chris
--
Christian Tismer :^) <mailto:tis...@tismer.com>
Mission Impossible 5oftware : Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/
14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
> Maybe I'll just give up on trying to interest people in Python
> optimisation challenges. The good-old days of idle speculation and
> optimisation are dead. Time to stop living in the past. Idea
> dropped, back to boring GUI work.
Do you feel like implementing Karatsuba multiplication in
longobject.c? That might actually be some use...
Cheers,
M.
--
GET *BONK*
BACK *BONK*
IN *BONK*
THERE *BONK* -- Naich using the troll hammer in cam.misc
Although this was on the table two or three years
ago, and dissed as well, although cheap to implement.
import time
def karatsuba( x,y, threshold = 1000 ):
"""Karatsuba multiplication for Python longs
1) how should threshold be determined
2) should there be multiple splits for large numbers?
"""
thresholdValue = (2L**threshold)-1
# b is the power of 2 at which the split occurs...
b = 1L<<threshold
# short circuit if both values are small...
if x&thresholdValue == x and y&thresholdValue == y:
return x*y
elif x<0 or y<0:
# don't think this works for signed longs
return x*y
x0 = x&thresholdValue
x1 = x>>threshold
assert x == (x1*b+x0)
y0 = y&thresholdValue
y1 = y>>threshold
assert y == (y1*b+y0)
return ((1L<<(threshold*2))+b)*x1*y1 - b*(x1-x0)*(y1-y0) + (b+1)*x0*y0
def test( ):
data = [
(2L<<1000000,3L<<100000),
(5L<<1000000,23L<<100000),
(23L<<1000,45L<<2000),
(23L<<10000,45L<<20000),
]
for x,y in data:
t = time.clock()
a = x*y
t = time.clock()-t
print 'regular', t
t = time.clock()
b = karatsuba(x,y)
t = time.clock()-t
print 'Karatsuba', t
assert a == b
if __name__ == "__main__":
test()
Is the idea that you'd want to take a threshold like 30 and run blocks
of 30 all the way along the number? Oh, the gmp system seems to have
Karatsuba multiplication for long ints, incidentally.
Okay, I feel better now, afternoon's work whacked by a chance to learn
something new :) . Thanks Michael!
Mike
Michael Hudson wrote:
> "Mike C. Fletcher" <mcfl...@rogers.com> writes:
>
>
>>Maybe I'll just give up on trying to interest people in Python
>>optimisation challenges. The good-old days of idle speculation and
>>optimisation are dead. Time to stop living in the past. Idea
>>dropped, back to boring GUI work.
>
>
> Do you feel like implementing Karatsuba multiplication in
> longobject.c? That might actually be some use...
>
> Cheers,
> M.
>
--
I'm not particularly attached to the idea of optimising 2**x for large
values of x (never actually used it myself), or even generic large-int
multiplication. I was just hoping for some resulting discussions of
algos for speeding up multiplication, optimising big integers (e.g.
shared storage or the like), and generally bringing the "fun place to
learn about programming" atmosphere back to the list.
So far only Karatsuba is on the table (that was fun, maybe someone will
want to play with the Python version I just posted). Apparently the
area of long-int-optimisation is just too well-trod to elicit much
interest among the general populace.
Tim's rather busy these days. I suppose the days of timbot postings
that take 2 hours to read (what with tracking down all the references
given and learning exotic CS concepts) are a thing of the past. Still a
fun person, but now more of an adult figure, distant, authoritative :o)
;) . Life continues on. Someday I too will need to leave
Never-Never-Land, but darn it, I'll fight it all the way.
I still think it's all a plot by that dratted Wendy Darling!
Mike
Christian Tismer wrote:
> Mike C. Fletcher wrote:
...
> I hereby encourage you to write that in the best
> way you can, since I know of a Python branch
> that will happily adopt any short and sensible optimization.
> I can contribute a bunch to this, too.
>
...
> Viruses pretending to be real emails are becoming
> a problem. This one was just simple to detect:
> The real Timbot would either implement it in less
> bytes than the text of this response, or propose
> to provide an appropriate optional extension mechanism
> for special optimization modules.
...
Ya, but "optimizing" 2**N to a shift is like, instead of trying to speed
flattening of arbitrary nested sequences, trying to speed flattening of a
list nested no more than 3 deep, whose first element is a non-nested tuple,
there are no more than 4 elements in all, and the last list element is an
int. Face it: it's boring <wink>.
[Christian Tismer]
> Don't say such sad words.
> Nothing is dead but maybe some of our expectations.
> Python was never eager to adopt new features (which
> is good), but putting ideas down this short wasn't
> usual, too.
It's truly trivial for a user to a write a shift themself, and long-int pow
is almost always used in real life in its 3-argument form, where this
optimization doesn't apply (except to waste cycles determining that it's of
no use). Now you know perfectly well that it's possible to write a simple
pow() in Python that runs faster than the current C version in *many*
cases -- that's a lot more interesting.
[Michael Hudson]
> Do you feel like implementing Karatsuba multiplication in
> longobject.c? That might actually be some use...
[back to Christian]
> Although this was on the table two or three years
> ago, and dissed as well, although cheap to implement.
I don't recall it being dissed. I do recall that nobody cared enough at the
time to write a patch to do it. Hell, we still don't have a reasonable,
cheap way to determine the number of significant bits in a long, although
for 2.2 math.log and math.log10 were generalized to work on arbitrary longs,
and math.log(n)/math.log(2) is a darned good guess now.
>>> x = 1L << 1000000
>>> import math
>>> math.log(x)
693147.18055994529
>>> _/math.log(2)
1000000.0
>>>
you-can't-change-history-when-a-bot-is-watching<wink>-ly y'rs - tim
> Michael Hudson wrote:
> > Do you feel like implementing Karatsuba multiplication in
> > longobject.c? That might actually be some use...
>
> Although this was on the table two or three years ago,
I know. I remember. That thread was probably the first place I heard
of Karatsuba multiplication...
> and dissed as well, although cheap to implement.
I don't think it got dissed; as I recall Tim and yourself slung Python
versions around for a bit and then said "now who's going to translate
this into C for longobject.c?" whereupon the thread died very quickly.
Cheers,
M.
PS: longobject.c is surely the most horrible bit of source in the
Python distribution. I certainly wouldn't want to hack on it.
--
Hmmm... its Sunday afternoon: I could do my work, or I could do a
Fourier analysis of my computer's fan noise.
-- Amit Muthu, ucam.chat (from Owen Dunn's summary of the year)
The message you're looking for may be in a thread from July 1999:
http://groups.google.com/groups?hl=en&lr=&threadm=37922175.FEEBE203%40appliedbiometrics.com&rnum=4&prev=/groups%3Fq%3Dgroup:comp.lang.python%2Bkaratsuba%26hl%3Den%26lr%3D%26selm%3D37922175.FEEBE203%2540appliedbiometrics.com%26rnum%3D4
It'd be a bit nicer now that the builtin "long" class can be subclassed.
Too bad you can't write
long.__mul__ = karatsuba_mul
or
long = karatsuba_long
and have it "work" (longs constructed in the easy way, 'x=1L', would have
karatsuba multiplication behavior by default)
Jeff
> On Fri, May 24, 2002 at 10:06:12AM +0000, Michael Hudson wrote:
> > Christian Tismer <tis...@tismer.com> writes:
> >
> > > Michael Hudson wrote:
> > > > Do you feel like implementing Karatsuba multiplication in
> > > > longobject.c? That might actually be some use...
> > >
> > > Although this was on the table two or three years ago,
> >
> > I know. I remember. That thread was probably the first place I heard
> > of Karatsuba multiplication...
>
> The message you're looking for may be in a thread from July 1999:
[snip megaurl]
Yep that's the one. I'd already found it (not very hard -- google for
Karatsuba on comp.lang.python...). Good reading, though, so anyone
else out there should read it too!
> It'd be a bit nicer now that the builtin "long" class can be subclassed.
That's a point.
> Too bad you can't write
> long.__mul__ = karatsuba_mul
> or
> long = karatsuba_long
> and have it "work" (longs constructed in the easy way, 'x=1L', would have
> karatsuba multiplication behavior by default)
I think overall I'd glad you can't do things like that...
Cheers,
M.
--
If you give someone Fortran, he has Fortran.
If you give someone Lisp, he has any language he pleases.
-- Guy L. Steele Jr, quoted by David Rush in comp.lang.scheme.scsh
Give it a try! You'd be surprised. A saving grace is that operations on
longs are very well defined, and independent of competing Standard Committee
whims. For this reason, longobject.c is one of the most #ifdef-free of all
the Python modules, and the ifdefs it does suffer (on HAVE_LONG_LONG and
Py_USING_UNICODE) don't really have anything to do with longs. So this is a
refreshingly Dead Right versus Dead Wrong module.
The only *artificial* pain (== not inherent to the subject matter) is due to
implementing portable numerics in a language (C) that leaves lots of crucial
issues undefined (like how many bits a thing has, and how sign bits work in
all cases). It's also one of the more algorithmically interesting modules.
Just pray that x_divrem never sprouts a bug <0.89 wink>.
> Do you feel like implementing Karatsuba multiplication in
> longobject.c? That might actually be some use...
Done. See patch 560379.
Please be warned, though, this is my first attempt at a patch.
--
Christopher A. Craig <com-n...@ccraig.org>
"Only two things are infinite, the universe and human stupidity, and I'm not
sure about the former." Albert Einstein
[longobject]
>>Although this was on the table two or three years ago,
>
> I know. I remember. That thread was probably the first place I heard
> of Karatsuba multiplication...
>
>>and dissed as well, although cheap to implement.
>
> I don't think it got dissed; as I recall Tim and yourself slung Python
> versions around for a bit and then said "now who's going to translate
> this into C for longobject.c?" whereupon the thread died very quickly.
You are right, I remember. I think at that time
I wasn't yet able to write C extensions.
(Maybe, or evenly properly was too lazy :)
> Cheers,
> M.
> PS: longobject.c is surely the most horrible bit of source in the
> Python distribution. I certainly wouldn't want to hack on it.
I don't think so. It took me much shorter to understand
than ceval.c ...
ciao - chris
Long-int speedups didn't interest the general public ten years ago, either.
It's a specialty market, and one well served by GMP (for which at least two
viable Python wrappers currently exist).
> Tim's rather busy these days. I suppose the days of timbot postings
> that take 2 hours to read (what with tracking down all the references
> given and learning exotic CS concepts) are a thing of the past.
That's for sure! I had a lot more time to give to Python before I got a job
putatively working on it <0.3 wink>. Now it's mostly my job to sabotage
people if I suspect they may scowl at Guido when he's not looking. Also, I
read, to dream up gratuitous ways to break working code for no reason at
all. It's a living.
> Still a fun person, but now more of an adult figure, distant,
> authoritative :o) ;) .
But still vicious! I haven't lost any any truly important virtue.
> Life continues on. Someday I too will need to leave
> Never-Never-Land, but darn it, I'll fight it all the way.
I've seen Peter Pan too, and there wasn't a long-int exponentiation in the
whole story. Nested tuples, yes, but no truly big numbers. Neither decimal
points nor implied binary points. So I conclude you got into this mess by
poking your head *out* of Never-Never Land! Believe me, you don't even want
to *see* what goes on out here.
grownups-everywhere-you-turn-and-some-even-smell-bad-ly y'rs - tim
Currently being designed. Yes, it's a bit behind schedule, but you can
hardly call something vapourware if it's still in the design process
(although Parrot [the interpreter] does exist).
cheers,
--
Iain.
Well, you get that when people don't use warnings or strict. And they don't
check their error conditions. Some of this is being remedied in Perl 6. See
www.perl.com for the Apocalypses and Exegeses of the design process (should
be a new pair out next week, covering regular expressions).
PHP programmers have it worse with variables that they don't initialise. At
least if they don't use the newer features of the language. Using the
automatic variable assignation from the HTTP parameters and not defining any
variables you are using can lead to variables having values which they
aren't meant to. Can lead to security problems.
Perl at least lets you use a taint mode so that unchecked input is not
usable until it's been checked.
I'm only a beginner in Python, so I'm afraid I can't compare there. Probably
covered in a FAQ (I'd check, but my ISP's external access appears to have
died - so all I can play with is the news server and the mail server).
The thing is, you can program badly in any language - some just protect you
a bit more. Not 'use'ing the warnings or strict pragmas in Perl is like not
checking the values of pointers in C, or your array indices.
> Python catches these errors and
> tells you about them, so you don't need to post to a newsgroup to find
> out what went wrong. Fewer basic questions means the advanced
> questions constitute a greater portion of the traffic.
Well, Perl catches the file errors, and it lets you handle them however you
like. If you don't handle them, then it won't. It can also catch the others
- you just have to ask it to. Sometimes you may not want it to.
[My apologies if this is weirdly formatted - I'm having to use Outlook
Express rather than my preferred slrn+vim]
cheers,
--
Iain.
Iain Truskett:
> Well, Perl catches the file errors, and it lets you handle them however
> you like. If you don't handle them, then it won't. It can also catch the
> others - you just have to ask it to. Sometimes you may not want it to.
Admittedly, there is an advantage in the default being "abort with stack
trace" rather than "ignore silently".
Python can do "ignore silently", but you have to specifically ask it to.
Jiri
--
Jiri Baum <ji...@baum.com.au> http://www.csse.monash.edu.au/~jirib
MAT LinuxPLC project --- http://mat.sf.net --- Machine Automation Tools