Cythonizing a Python Library

[Trokon Johnson]

So, I've a summer project that focuses on accelerating a library. I thought at first to move some of the computationally intense functions to Cython. During profiling, I found that the function with the highest percentage runtime from the custom library, that is, not from a default python library, took up about 76% of run time. I shifted this to a cython file, compiled, and ran, but I found that the run-time for the cython version was about the same as the pure python version. I didn't do any optimizations through static typing or the like, and I didn't expect a huge speedup, but I expected a noticeable change in run time just from the compilation alone; the lack of this makes me wonder if I've done something wrong in my implementation.

Does anyone have any experience or sources with this workflow, that is using cython to accelerate a pre-existing python library?

[Hanno Klemm]

On 26. Jun 2017, at 17:33, Trokon Johnson <tkjoh...@gmail.com> wrote:

So, I've a summer project that focuses on accelerating a library. I thought at first to move some of the computationally intense functions to Cython. During profiling, I found that the function with the highest percentage runtime from the custom library, that is, not from a default python library, took up about 76% of run time. I shifted this to a cython file, compiled, and ran, but I found that the run-time for the cython version was about the same as the pure python version. I didn't do any optimizations through static typing or the like, and I didn't expect a huge speedup, but I expected a noticeable change in run time just from the compilation alone; the lack of this makes me wonder if I've done something wrong in my implementation.

Does anyone have any experience or sources with this workflow, that is using cython to accelerate a pre-existing python library?

Hi, 

of course it depends a lot on what your code is actually doing, but I had very similar experiences when trying to "just compile something". No speed-up or even worse performance. This changed dramatically when I used static typing. 

If you have just one function that takes up the majority of your computation time, it should be easy to add some types to speed this up. However, if your function is already relying on numpy heavily, there are situations where getting incremental gains can be tricky. Again, it depends highly on your code. 

Good luck,

Hanno

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Trokon Johnson]

Understood. Thank you.

[Daπid]

On 26 June 2017 at 19:54, Hanno Klemm <h.k...@gmx.de> wrote:

Hi, 

of course it depends a lot on what your code is actually doing, but I had very similar experiences when trying to "just compile something". No speed-up or even worse performance. This changed dramatically when I used static typing. 

On this topic, I have experienced that, in some particular cases (mostly numerical, I believe), just cythonising code gets up to a 20% speed up for free. What is Cython doing there? Is there some simple type inference? Or the compiler is seeing through all the allocation and deallocation of floats and removing intermediate values?

/David.

[Robert Bradshaw]

There is some simple type inference, some optimization of
builtin/common calls (stuff like isinstance and list.append), and the
removal of interpreter overhead. Depending on the code, I've seen this
help a lot or not at all (and in some rare cases it can be even worse,
e.g. if it ends up converting a type back and forth that would have
remained a Python object). Of course judicious type annotation is
where th real gains are.

- Robert

[Denis Akhiyarov]

There is also jedityper to help with cythonization.