Programming in Python with a view to extending in C at a later date.
dug.ar...@googlemail.com
Say you set out to program in Python knowing that you will be
converting parts of it into C ( or maybe C++) at a later date, but you
do not know which parts.
Can you give any general Python structure / syntax advice that if
implemented from the start, will make this future task less painful.
In addition, advice that will make this easier for automatic tools
will be good too.
This advice might include stuff like "avoid using Python yield
as ...."
Thank you in advance,
Douglas
Scott David Daniels
> Say you set out to program in Python knowing that you will be
> converting parts of it into C ( or maybe C++) at a later date, but you
> do not know which parts.
>
> Can you give any general Python structure / syntax advice that if
> implemented from the start, will make this future task less painful.
> In addition, advice that will make this easier for automatic tools
> will be good too.
The best advice I have is, "Ignore the conversion task, and get your
algorithms and data structures right. Once you have that, _measure_
where your inner loop is and rethink.
When you've identified your inner loop(s), and only then, isolate that
code and shift its style to a more static style (essentially testing
your C/C++ algorithms in Python).
Now you can either go for the translation or use something like Cython
or Pyrex to see if automatic translation will be good enough.
This technique takes advantage of the fact that trying several different
algorithms will be much easier in Python than in C or C++.
--Scott David Daniels
Scott....@Acm.Org
Lawson English
> On Mon, Apr 20, 2009 at 1:57 PM, dug.ar...@googlemail.com
> <mailto:dug.ar...@googlemail.com> <dug.ar...@googlemail.com
>
> Say you set out to program in Python knowing that you will be
> converting parts of it into C ( or maybe C++) at a later date, but you
> do not know which parts.
>
>
> in C for speed. It's always been a pleasant experience, and I've yet
> to come across a Python construct that made my life especially
> difficult later. The value of rapid development and debugging
> algorithmic problems in Python has always outweighed the cost of later
> translating some of the code.
>
> Hope that helps,
Linden Lab, makers of Second LIfe virtual world, found it was good to
create a C++ class similar
to a Python dictionary in order to leverage some of the faster
prototyping advantages of that Python type
when converting to C++.
Google LLSD Linden Lab for more info.
L
Lawson English
> Daniel Stutzbach wrote:
>> On Mon, Apr 20, 2009 at 1:57 PM, dug.ar...@googlemail.com
>> <mailto:dug.ar...@googlemail.com> <dug.ar...@googlemail.com
>> <mailto:dug.ar...@googlemail.com>> wrote:
>>
>> converting parts of it into C ( or maybe C++) at a later date,
>> but you
>> do not know which parts.
>>
>>
>> it in C for speed. It's always been a pleasant experience, and I've
>> yet to come across a Python construct that made my life especially
>> difficult later. The value of rapid development and debugging
>> algorithmic problems in Python has always outweighed the cost of
>> later translating some of the code.
>>
>> Hope that helps,
>
> Linden Lab, makers of Second LIfe virtual world, found it was good to
> create a C++ class similar
> to a Python dictionary in order to leverage some of the faster
> prototyping advantages of that Python type
> when converting to C++.
>
> Google LLSD Linden Lab for more info.
>
Incidentally LLSD is a proposed IETF standard so there's no licensing
issues involved in using the technique, and
I believe there are implementations in C++ (GPL) and C#(BSD) and
possible other languages as well.
L.
Terry Reedy
> Hi,
>
> Say you set out to program in Python knowing that you will be
> converting parts of it into C ( or maybe C++) at a later date, but you
> do not know which parts.
I presume because you intend to wait for profile results. Good.
Of course, you might even find your program *fast enough*.
> Can you give any general Python structure / syntax advice that if
> implemented from the start, will make this future task less painful.
> In addition, advice that will make this easier for automatic tools
> will be good too.
I would hate for you to make your Python experience more painful by
overly restricting yourself. A good test suite (in Python!) will make
any translation less painful. This goes also for trying out different
algorithms in Python, or otherwise refactoring. You also might be able
to get the speedup you need by compiling with PyRex, Cython, Shedskin,
Weave, or Pscho, or by using existing C code that you find when you need it.
> This advice might include stuff like "avoid using Python yield
> as ...."
I am not sure which, if any, of the above handle generators directly,
but generators often make Python programming easier, and there are
several other solutions.
0) Replace the body that calculates what is yielded with a C-coded
function, and leave the generator alone.
1) If you are explicitly calling next(somegen), rewrite the generator in
C as a next function.
2) Rewrite the generator as an iterator and translate that.
tjr
Carl Banks
<dug.armad...@googlemail.com> wrote:
> Say you set out to program in Python knowing that you will be
> converting parts of it into C ( or maybe C++) at a later date, but you
> do not know which parts.
>
> Can you give any general Python structure / syntax advice that if
> implemented from the start, will make this future task less painful.
> In addition, advice that will make this easier for automatic tools
> will be good too.
I'd just avoid things that don't have a straightforward translation to
C (such as clever metaprogramming, overdependence on keyword
arguments, nested scopes and closures, and yes, generators). Most
things in Python do correspond pretty well to things in C.
However, don't avoid objects that might be difficult to implement in C
(such as dicts and sets); fact is, code that uses these objects a lot
probably won't benefit much from translation to C. OTOH, if you need
dict-like behavior in an otherwise static computation, it's probably
because there's no other easy way to do the computation, so you'll
have to tackle the problem in C anyway.
Also, don't avoid regular exception handling. When you rewrite
something in C you'll find that it is farily obvious how to write the
C code so that it returns errors in a similar way.
If you do a lot of OO stuff I strongly recommend that you study up on
the right way to implement new-style types from C, such that they can
serve as a base for Python classes. That way, if you have a class and
you want to implement a few methods in C while leaving the rest in
Python, you can factor out all the C methods into a base class written
in C.
Carl Banks
bobicanprogram
Actually you can have your cake and eat it too. We've used the SIMPL
toolkit (http://www.icanprogram.com/simpl) and its ultra lightweight
toolkit to join Python programs to those written in other languages
(C,C++,Tcl/Tk or JAVA). That way you can keep the parts that Python
is good at in Python and keep the parts that C or C++ are good at
written in those languages and join the lot together seamlessly.
SIMPL will even allow those parts to be deployed on separate network
nodes often without any code change or recompile.
There is an onlime tutorial/course with lots of examples at:
http://www.icanprogram.com/06py/main.html
bob
Stefan Behnel
dug.ar...@googlemail.com wrote:
> Say you set out to program in Python knowing that you will be
> converting parts of it into C ( or maybe C++) at a later date, but you
> do not know which parts.
>
> Can you give any general Python structure / syntax advice that if
> implemented from the start, will make this future task less painful.
> In addition, advice that will make this easier for automatic tools
> will be good too.
My advice is to
1) write the program in Python, in a way that makes it well comprehensible
and maintainable
2) take care to use suitable algorithms and builtin types
3) benchmark important parts of the code early, fix your algorithms
4) benchmark/profile the code when you think it's ready, optimise it where
required
If you find places where you require plain performance that you think
cannot be achieved in Python,
5) extract those functions or classes into one or more separate modules,
compile them with Cython and import them into your normal code. Try to also
extract surrounding code sections (especially loops) to avoid Python's call
overhead.
6) add type declarations until it runs fast enough (usually a couple of
hundred times faster for I/O free code)
You may be able to compile whole files in Cython (so that you can skip the
refactoring required for step 5), but that is neither required nor
necessarily the best way to do it, since binary modules are always harder
to handle/fix/understand than plain Python modules. It will also keep you
from adding type declarations in places where it might hurt readabilty more
than it brings performance.
> This advice might include stuff like "avoid using Python yield
> as ...."
My advice regarding this paragraph is to not take advices like this. Ever.
Stefan