When I statically declare a Python type, is there a performance benefit?

[Zak]

Hello All,

When I statically declare a variable as a Python type, I am wondering if there is any performance benefit in any case. I have run some tests and seen no performance impact at all. I am talking about doing things like this:

# Normal dynamic Python:
my_dict = {'foo': 'bar'}

# Possibly faster static Cython:
cdef dict c_dict = {'baz': 'quux'}

The only benefit of the second line, as far as I know, is type checking. Is it possible to get a performance benefit with a statically declared Python type, the same way you get a benefit from a C type?

Thanks,

Zak

[yi huang]

When you declare it as `dict`, when you access if like `c_dict[key]`,
cython would generate `PyDict_GetItem` rather than `PyObject_GetItem`.
There is some minor performance gain, but not significant.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
http://yi-programmer.com/

[Robert Bradshaw]

Yep. There are a couple of optimizations we do when we know a variable
is of a special type (e.g. for key, value in d.items()), but there is
the overhead of type checking on assignment.

The performance benefits from C types can be much better because
there's much less overhead for simple C value types than full Python
objects.

[Stefan Behnel]

Zak, 26.07.2013 01:24:

Absolutely. However, if you do this inside of a function, then Cython will
already know that my_dict contains a dict (because it sees the assignment)
and optimise for it. Adding a type declaration won't have an effect then.
Only if it's a global variable (which Cython doesn't have control over),
it's usually worth statically typing it.

Cython also does several optimistic optimisations internally that even
dynamic code will benefit from. So the visible performance benefit of
statically declaring Python types is certainly limited. Basically, you
should always validate the need static type declarations (C or Python
types) with benchmarks. The less you clutter your code with them, the
better for readability.

If you're really interested in the details, try compiling with "cython -a"
(or cythonize(..., annotate=True)) to get an annotated HTML version of your
compiled code. Click on the code lines to see the C code that Cython
generates for them.

Hmm, maybe we should add an additional feature to this that also inserts
the inferred type for each usage of a variable. That sounds like a good idea...

Stefan

[Yury V. Zaytsev]

On Fri, 2013-07-26 at 05:49 +0200, Stefan Behnel wrote:
>
> Hmm, maybe we should add an additional feature to this that also
> inserts the inferred type for each usage of a variable. That sounds
> like a good idea...

Sounds like a brilliant idea, even.

--
Sincerely yours,
Yury V. Zaytsev

[Sturla Molden]

On 26.07.2013 01:24, Zak wrote:

> When I statically declare a variable as a Python type, I am wondering if
> there is any performance benefit in any case. I have run some tests and
> seen no performance impact at all.

Sometimes!

* Declaring a Python type (or just "cdef object") avoids the dictionary
lookups when you access the variable. It can also avoid unneeded
reference counting when accessing a variable. This can be a huge
performance gain if the variable is accessed often.

* cdef variables are not accessible from Python. Thus you can protect
them from modification from Python.

* Declaring a Python type can make Cython optimize certain calls, such
as indexing and appending.


Sturla

[Robert Bradshaw]

On Jul 26, 2013 9:58 AM, "Sturla Molden" <stu...@molden.no> wrote:
>
> On 26.07.2013 01:24, Zak wrote:
>
>> When I statically declare a variable as a Python type, I am wondering if
>> there is any performance benefit in any case. I have run some tests and
>> seen no performance impact at all.
>
>
> Sometimes!
>
> * Declaring a Python type (or just "cdef object") avoids the dictionary lookups when you access the variable. It can also avoid unneeded reference counting when accessing a variable. This can be a huge performance gain if the variable is accessed often.
>
> * cdef variables are not accessible from Python. Thus you can protect them from modification from Python.

These two points are for global variables, and in that case there can be quite a boost.

> * Declaring a Python type can make Cython optimize certain calls, such as indexing and appending.
>
>
> Sturla
>
>
>
>
>