Types, Cython, program readability
bearoph...@lycos.com
differently from the dead-looking Pyrex. Hopefully Cython will become
more user-friendly too (Pyrex is far from being user-friendly for
Windows users, it doesn't even contain a compiler, I think. The
ShedSkin Windows installer contains an old but working MinGW, and this
is positive). It seems Cython is going to become an efficient and
general purpose language after all, with optional static typing (its
purpose: mostly for speed), and it may even gain some kind of macros
soon. So it may even end replacing Python itself in some situations
where running efficiency is important, and where Psyco can't be used
or isn't enough.
Apparently unrelated: few days ago I was translating a 200-lines long
program from Python to D (D is mostly statically typed, like C++,
Java) for personal use. That Python program was without unit tests,
comments, docstrings, documentation, etc. So finding the types of all
the variables required some time. So I have fed the whole program to
ShedSkin, and in 20 seconds on my old PC it has produced a Python
program with full type annotations (as comments on each line of code),
that later I have translated quickly to D.
In such "blind" situations (that are often enough in some scripts I
find around) I think a statically typed program (without type
inference) can be more readable than a Python program, because even if
the programmer is super-lazy types must be present (otherwise the
program doesn't compile), and with the variable names they give me
some information that's absent (well, it's less easy to extract.
ShedSkin is often able to extract it) from the Python code.
The (little) point of this post: sometimes (when the programmer is
quite lazy) statically typed code is more "readable" than Python code.
Bye,
bearophile
Tim Golden
> It seems the development of Cython is going very well, quite
> differently from the dead-looking Pyrex.
I'll leave others to comment on how dead Pyrex is or isn't ...
> Hopefully Cython will become
> more user-friendly too (Pyrex is far from being user-friendly for
> Windows users, it doesn't even contain a compiler, I think. The
> ShedSkin Windows installer contains an old but working MinGW, and this
> is positive).
I'm not entirely sure why you think Pyrex should "contain a compiler".
It certainly works well enough with the free [beer] MS VS 2008 Express
and I'm fairly sure it's fine with MingW. Both of those are readily
available and I don't imagine anyone who's going to use Pyrex / Cython /
ShedSkin is going to baulk at downloading a compiler set :)
TJG
Stefan Behnel
bearoph...@lycos.com wrote:
> It seems Cython is going to become an efficient and
> general purpose language after all, with optional static typing (its
> purpose: mostly for speed), and it may even gain some kind of macros
> soon. So it may even end replacing Python itself in some situations
> where running efficiency is important, and where Psyco can't be used
> or isn't enough.
Even without any static type annotations, Cython currently runs the supported
portion of pybench some 30% faster than CPython 2.5.1, with for-loops and
if-then-else being multiple times faster. Even on plain Python numbers,
arithmetic gives you a factor of 2 in Cython.
http://comments.gmane.org/gmane.comp.python.cython.devel/680?set_lines=100000
If you need more, C arithmetic and integer loops can easily be enabled with
type annotations, and the Cython project is working on further optimisations
for normal Python code. So if speed is important and the existing code is
supported by Cython (or can be made working), compilation is definitely an option.
Stefan
sturlamolden
> It seems the development of Cython is going very well, quite
> differently from the dead-looking Pyrex. Hopefully Cython will become
> more user-friendly too (Pyrex is far from being user-friendly for
> Windows users, it doesn't even contain a compiler, I think. The
Pyrex is not dead. And although it may come as a surprise to you,
Pyrex IS a compiler.
Pyrex does not contain a C compiler. But why should it? There are
Microsoft Visual C++, MinGW, Cygwin, lcc, Borland C++, Intel C++
compiler, OpenWatcom, among others. Get your favourite.
Being written in pure Python, Pyrex is equally easy to use on Windows
and Linux. You have to use the command prompt or disttools on either
platform.
> The (little) point of this post: sometimes (when the programmer is
> quite lazy) statically typed code is more "readable" than Python code.
If your task is to translate a Python program into a statically typed
language, type annotations can be helpful. But you don't need types to
read a/b as 'a divided by b'.
Stefan Behnel
> On 16 Mar, 15:32, bearophileH...@lycos.com wrote:
>> It seems the development of Cython is going very well, quite
>> differently from the dead-looking Pyrex. Hopefully Cython will become
>> more user-friendly too (Pyrex is far from being user-friendly for
>> Windows users, it doesn't even contain a compiler, I think. The
>
> Pyrex is not dead.
Technically, that's true. There has been a release not so long ago, although
after a rather lengthy idle period.
However, the current development cycle of Cython is so fast that Pyrex will
have a hard time catching up. So there isn't much of a reason why you should
prefer Pyrex over Cython.
Stefan
bearoph...@lycos.com
> I'm not entirely sure why you think Pyrex should "contain a compiler".
I think lot of Win users (computational biologists?), even people that
know how to write good Python code, don't even know how to install a C
compiler.
>I'm fairly sure it's fine with MingW<
(In the past?) I think you have to tweak it a bit.
---------------
sturlamolden:
>And although it may come as a surprise to you, Pyrex IS a compiler.<
I meant a compiler that spits out the final executable a person can
click on.
>Being written in pure Python, Pyrex is equally easy to use on Windows and Linux.<
Or maybe equally difficult :-) The computational biology students I
was with think PythonWin is "easy" enough (or take a look at the
"Processing" Java-like language user interface). Lowering entry
difficulties is positive. Can things be made simpler for newbies?
Bye,
bearophile
sturlamolden
> I think lot of Win users (computational biologists?), even people that
> know how to write good Python code, don't even know how to install a C
> compiler.
If you don't know how to install a C compiler like Microsoft Visual
Studio, you should not be programming computers anyway.
> I meant a compiler that spits out the final executable a person can
> click on.
You don't click on compiled Python C extensions. You call it from your
Python code.
> >Being written in pure Python, Pyrex is equally easy to use on Windows and Linux.<
>
> Or maybe equally difficult :-) The computational biology students I
> was with think PythonWin is "easy" enough (or take a look at the
> "Processing" Java-like language user interface). Lowering entry
> difficulties is positive. Can things be made simpler for newbies?
Being a computational biologist myself (PhD), I do not agree. Any
student smart enough to program Python can learn to use disttools or
even a command prompt (whether it's bash or DOS does not matter).
There is no way of avoiding the keyboard when your are programming
computers, except for some strange drag-and-drop languages like
LabView. The way I see it, using a C compiler should be a part of the
general education for any student in computational sciences.
Java is not less complicated. It requires compilation step as well
(javac and possibly gnumake). Students are either forced to learn an
IDE (Eclispe og NetBeans), or deal with the command prompt.
sturlamolden
> You don't click on compiled Python C extensions. You call it from your
> Python code.
By the way, disttools will invoke Pyrex and the C compiler, and
produce the binary .pyd-file you can access from Python. It's not
rocket science (not even computational biology.)
Ben Finney
> If you don't know how to install a C compiler like Microsoft Visual
> Studio, you should not be programming computers anyway.
Utter elitist nonsense.
Programming should be made easier, and I see Python as a very good
language for making programming easier. Lowering the barrier for
prospective hackers to get into programming is a good thing. The
installation of a C compiler is incidental to that, and has no
necessary connection with programming.
Placing meaningless roadblocks, as you suggest should be done, is
anathema to education.
--
\ "The best mind-altering drug is truth." -- Jane Wagner, via |
`\ Lily Tomlin |
_o__) |
Ben Finney
Jeff Schwab
> sturlamolden <sturla...@yahoo.no> writes:
>
>> If you don't know how to install a C compiler like Microsoft Visual
>> Studio, you should not be programming computers anyway.
>
> Utter elitist nonsense.
>
> Programming should be made easier, and I see Python as a very good
> language for making programming easier. Lowering the barrier for
> prospective hackers to get into programming is a good thing. The
> installation of a C compiler is incidental to that, and has no
> necessary connection with programming.
>
> Placing meaningless roadblocks, as you suggest should be done, is
> anathema to education.
Hear, hear!
I continue to be shocked when an OS doesn't come with a compiler for the
language in which the OS is written. It seems like one of the most
basic things that should just be there, along with some kind of command
shell (be it CLI or graphical), and ideally a well-defined scripting
interface that is easily extensible from the "core" language.
Nowadays, scripts are often intended to work on multiple different
platforms; various scripting languages therefore list portability among
their most valuable features. One issue from which most of them suffer
is that they really are not suitable for systems programming. As
scripts evolve into applications, native-language compilers end up being
necessary, after all, such that portability of non-trivial programs is
severely limited. (It's not really the languages' fault. After all,
they were only really meant to enable portable scripting.)
Python, I think, helps improve the situation with specific support for
applications programming. Unlike Perl or Tcl, Python is not just a
scripting language with a set of ad-hoc extensions. There are still
issues, and Python probably will never be a general-purpose replacement
for system-native language compilers, but it does enable a smooth ramp
from "just a user," through "a user who does some scripting," to
"application developer."
I am of the opinion that the more regular users write and share code,
the faster we will see improvement in overall software quality, and the
more pleasant the world will be for all of us to live in.
See also:
http://cm.bell-labs.com/cm/cs/upe/
http://www.linfo.org/unix_philosophy.html
Tom Stambaugh
> and I'm fairly sure it's fine with MingW. Both of those are readily
> available and I don't imagine anyone who's going to use Pyrex / Cython /
> ShedSkin is going to baulk at downloading a compiler set :)
Anything like this is a lot more attractive to me if I can download and
install a binary without *needing* a compiler. Most of the time, I just want
something to run -- I don't want to risk diving in compiler-hell if I can
avoid it. It isn't downloading the compiler set that worries me, it's that I
almost never even want to think about it.
For example, the new (!) simplejson (v1.7.4) doesn't compile correctly (on
my WinXP system, at least) with either any current MS or MinGW compiler. Oh,
I know I can make it work if I spend enough time on it -- but the binary egg
I eventually found seems to work just fine.
Tim Golden
>> I'm not entirely sure why you think Pyrex should "contain a compiler".
>> It certainly works well enough with the free [beer] MS VS 2008 Express
>> and I'm fairly sure it's fine with MingW. Both of those are readily
>> available and I don't imagine anyone who's going to use Pyrex / Cython /
>> ShedSkin is going to baulk at downloading a compiler set :)
>
> Anything like this is a lot more attractive to me if I can download and
> install a binary without *needing* a compiler. Most of the time, I just want
> something to run -- I don't want to risk diving in compiler-hell if I can
> avoid it. It isn't downloading the compiler set that worries me, it's that I
> almost never even want to think about it.
I certainly sympathise with the compiler-hell syndrome here: the
subject of compiling Python itself and its extensions under
Windows has long been a source of questions and qualified answers
on the Python lists. This cuts both ways: Python itself is effectively
only buildable with the MS tools [1] (notwithstanding some valiant
efforts to provide patchsets so that it builds under MingW [2]). On
the other hand, some useful libraries, say ffmpeg, only *won't* build
with the MS tools[3]. Both sides of that debate have a valid position,
but it doesn't ultimately help the developers. Which is probably why
avbin/pyglet[4] took the decoupled route and used ctypes to join the
two together!
In the case of a general extension (like simpleJSON or PIL etc.)
I'm usually just after a binary. In the case of Pyrex, though, the
Pyrex code *itself* is pure Python; it's the code it generates which
is C -- one step further away. Granted, it might be possible to produce
some sort of bundle which shipped a MingW-based extension along with
the Pyrex release bundle, but my own philosophy is that, if you've
got to the point of using Pyrex then downloading a compiler isn't
going to be too much work.
It's clear, though, from the posts in this thread, that other
people don't necessarily agree. Admittedly I have the advantage
of being a professional programmer who is therefore bound to know
what a compiler *is*!
TJG
[1]
http://coding.derkeiler.com/Archive/Python/comp.lang.python/2006-04/msg03627.html
[2] http://jove.prohosting.com/iwave/ipython/pyMinGW.html
[3] http://ffmpeg.mplayerhq.hu/faq.html#SEC36
[4] http://code.google.com/p/avbin/
Duncan Booth
> For example, the new (!) simplejson (v1.7.4) doesn't compile correctly
> (on my WinXP system, at least) with either any current MS or MinGW
> compiler. Oh, I know I can make it work if I spend enough time on it
> -- but the binary egg I eventually found seems to work just fine.
I've also just spent a while getting simplejson 1.7.4 to install on a (non-
windows) system without a C compiler.
The trick is to unzip the tar file and then before you try to install it
delete everything in simplejson.egg-info. Then 'setup.py install' will run
to completion and while it warns about 'speedups are not enabled.' it
doesn't take it as fatal error.
Without this step it preserves the reference to native_libs.txt in
SOURCES.txt even though the native_libs.txt file itself gets deleted.
Robert Hicks
<snip>
> Unlike Perl or Tcl, Python is not just a
> scripting language with a set of ad-hoc extensions. There are still
> issues, and Python probably will never be a general-purpose replacement
> for system-native language compilers, but it does enable a smooth ramp
> from "just a user," through "a user who does some scripting," to
> "application developer."
Danger! My crap-o-meter went to 100%! You really need to explain what
you mean better here. What enables Python to give you a "smooth ramp"?
Inquiring minds want to know.I am sure there are a whole lotta
programmers in the Perl and Tcl camps that would like to know what you
mean as well.
I await your enlightenment.
Robert