Cython 0.18 released

[Stefan Behnel]

Hi all,

Cython 0.18 is out of the door. This is a feature release that adds some
long requested language features (const!), better Python compatibility and
optimisations.

You can get it from here:

http://cython.org/release/Cython-0.18.tar.gz

http://cython.org/release/Cython-0.18.zip

Release notes: https://github.com/cython/cython/blob/0.18/CHANGES.rst

Documentation: http://docs.cython.org/


The major new features are:

* A new directive ``overflowcheck`` which raises an OverflowException
when arithmetic with C ints overflows. This has a modest performance
penalty, but is much faster than using Python ints.

* C ``const`` declarations are supported in the language.

* Calls to nested Python functions are resolved at compile time.

* Type inference works across nested functions.

* ``py_bytes_string.decode(...)`` is optimised.

* Named Unicode escapes ("\N{...}") are supported.

* Python functions/classes provide the special attribute "__qualname__"
as defined by PEP 3155.


Have fun,

Stefan

[Czarek Tomczak]

Cython FAQ is not yet updated, see "How do I use 'const'"?

http://wiki.cython.org/FAQ#HowdoIuse.27const.27.3F

Also in the documentation:

Don’t use const. Cython doesn’t know anything about const..

http://docs.cython.org/src/userguide/external_C_code.html#referencing-c-header-files

Czarek.

[Stefan Behnel]

Czarek Tomczak, 30.01.2013 00:18:

> Cython FAQ is not yet updated, see "How do I use 'const'"?
> http://wiki.cython.org/FAQ#HowdoIuse.27const.27.3F
>
> Also in the documentation:
>
>> Don’t use const. Cython doesn’t know anything about const..
>
> http://docs.cython.org/src/userguide/external_C_code.html#referencing-c-header-files

Thanks! I fixed both. The updated page is here:

https://sage.math.washington.edu:8091/hudson/job/cython-docs/doclinks/1/src/userguide/external_C_code.html

Stefan

[Dave Hirschfeld]

Stefan Behnel <stefan_ml <at> behnel.de> writes:

>
> Hi all,
>
> Cython 0.18 is out of the door. This is a feature release that adds some
> long requested language features (const!), better Python compatibility and
> optimisations.
>

> Have fun,
>
> Stefan
>

Hi,
I've been eagerly awaiting the "array expressions" functionality but I see
that the development has stalled (https://github.com/cython/cython/pull/144).
Is it still planned to merge this functionality?

Whilst keeping in mind Brian Granger's post about feature creep in open source
projects, I think array expressions would be a killer feature for cython. Even
if it doesn't give any performance improvements over nested loops as can be
seen in numbaPRO, it allows you to write your code in a much more numpythonic
way.

http://docs.continuum.io/numbapro/arrayexprs.html

Thanks,
Dave

[mark florisson]

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

Hey Dave,

Yes, it's a nice feature, I've unfortunately been busy with other
things. It however doesn't neatly address the math issue, you can use
element-wise functions such as the libc math functions which are
mapped over your data, but it doesn't do the math, cmath or numpy math
stuff. It also doesn't do reductions, scans, etc at this point.
However, it can be much faster than indexing since it is much more
cache aware, especially if you mix different array data layouts. For
small arrays there is however some overhead that might negate any
potential performance gain. But it's certainly more expressive :)

Anyway, your enthusiasm might just have been enough to make me take
another look at merging it. There is probably some divergence by now,
last I looked some tests were still broken, I'll assess the
feasibility of a merge.

Mark

[Dave Hirschfeld]

mark florisson <markflorisson88 <at> gmail.com> writes:

>
> On 30 January 2013 03:37, Dave Hirschfeld <dave.hirschfeld <at> gmail.com>
wrote:

> > Hi,
> > I've been eagerly awaiting the "array expressions" functionality but I see
> > that the development has stalled
(https://github.com/cython/cython/pull/144).
> > Is it still planned to merge this functionality?
> >

> > Thanks,
> > Dave

>
> Hey Dave,
>
> Yes, it's a nice feature, I've unfortunately been busy with other
> things. It however doesn't neatly address the math issue, you can use
> element-wise functions such as the libc math functions which are
> mapped over your data, but it doesn't do the math, cmath or numpy math
> stuff. It also doesn't do reductions, scans, etc at this point.
> However, it can be much faster than indexing since it is much more
> cache aware, especially if you mix different array data layouts. For
> small arrays there is however some overhead that might negate any
> potential performance gain. But it's certainly more expressive :)
>
> Anyway, your enthusiasm might just have been enough to make me take
> another look at merging it. There is probably some divergence by now,
> last I looked some tests were still broken, I'll assess the
> feasibility of a merge.
>
> Mark
>

Glad I posted then! It does appear there are merge conflicts so hopefully
they're easy to resolve. I also tried to get the branch working but I think I
did something wrong with the minivect submodule and couldn't get it working.

Thanks,
Dave

[mark florisson]

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

Sorry, reflecting some more on this, I don't think I should merge it,
since I'll have the time to get it in, but I'm not sure I'll have the
time to support it and fix bugs in a timely manner. Instead I'd like
to keep fixing bugs for the memoryviews and other features as we have
them now (and perhaps add other, smaller, features later).

[Stefan Behnel]

mark florisson, 07.02.2013 03:32:
> On 30 January 2013 11:10, Dave Hirschfeld wrote:
>> mark florisson writes:

>>> On 30 January 2013 03:37, Dave Hirschfeld wrote:
>>>> I've been eagerly awaiting the "array expressions" functionality but I see
>>>> that the development has stalled
>>>> (https://github.com/cython/cython/pull/144).
>>>> Is it still planned to merge this functionality?
>>>

>>> Yes, it's a nice feature, I've unfortunately been busy with other
>>> things. It however doesn't neatly address the math issue, you can use
>>> element-wise functions such as the libc math functions which are
>>> mapped over your data, but it doesn't do the math, cmath or numpy math
>>> stuff. It also doesn't do reductions, scans, etc at this point.
>>> However, it can be much faster than indexing since it is much more
>>> cache aware, especially if you mix different array data layouts. For
>>> small arrays there is however some overhead that might negate any
>>> potential performance gain. But it's certainly more expressive :)
>>>
>>> Anyway, your enthusiasm might just have been enough to make me take
>>> another look at merging it. There is probably some divergence by now,
>>> last I looked some tests were still broken, I'll assess the
>>> feasibility of a merge.
>>

>> Glad I posted then! It does appear there are merge conflicts so hopefully
>> they're easy to resolve. I also tried to get the branch working but I think I
>> did something wrong with the minivect submodule and couldn't get it working.
>

> Sorry, reflecting some more on this, I don't think I should merge it,
> since I'll have the time to get it in, but I'm not sure I'll have the
> time to support it and fix bugs in a timely manner. Instead I'd like
> to keep fixing bugs for the memoryviews and other features as we have
> them now (and perhaps add other, smaller, features later).

I just thought I'd follow up with a quick reminder that Cython is an
Open-Source project. If anyone's interested in taking this over, I'm sure
there will be some interest out there.

Stefan

[Dave Hirschfeld]

mark florisson <markflorisson88 <at> gmail.com> writes:

>

> On 30 January 2013 11:10, Dave Hirschfeld <dave.hirschfeld <at> gmail.com>
wrote:

That's a shame, but I completely understand and it's better to have fewer, very
solid features than lots of poorly supported ones IMO. I doubt I'd have the time
to pick it up myself even if I had the requsite skills!

Thanks,
Dave

[mark florisson]

Indeed. We've discussed building an array based computation engine
directly in LLVM and building a community around that. It would be
nice to have the previously discussed double[...] syntax to allow an
arbitrary number of dimensions and JIT the actual array expressions
according to the runtime data layout and dimensionality. This means
throwing minivect out the door, and it means a dependency on LLVM...

[Chris Barker - NOAA Federal]

On Fri, Mar 8, 2013 at 3:40 AM, mark florisson
<markflo...@gmail.com> wrote:

> Indeed. We've discussed building an array based computation engine
> directly in LLVM and building a community around that.

How is this different than numba?

-Chris


--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[mark florisson]

On 11 March 2013 15:58, Chris Barker - NOAA Federal

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

It's at a lower level than numba, which would allow numba as well as
other front-end compilers to reuse the functionality (perhaps even
without ever having a Python dependency).

[Dag Sverre Seljebotn]

On 03/08/2013 11:46 AM, Dave Hirschfeld wrote:
> mark florisson <markflorisson88 <at> gmail.com> writes:
>
>>
>> On 30 January 2013 11:10, Dave Hirschfeld <dave.hirschfeld <at> gmail.com>
> wrote:
>>> mark florisson <markflorisson88 <at> gmail.com> writes:
>>
>> Sorry, reflecting some more on this, I don't think I should merge it,
>> since I'll have the time to get it in, but I'm not sure I'll have the
>> time to support it and fix bugs in a timely manner. Instead I'd like
>> to keep fixing bugs for the memoryviews and other features as we have
>> them now (and perhaps add other, smaller, features later).
>>
>
> That's a shame, but I completely understand and it's better to have fewer, very
> solid features than lots of poorly supported ones IMO. I doubt I'd have the time
> to pick it up myself even if I had the requsite skills!

Yes, I think Mark's decision makes sense at the moment. It's not good
enough to simply have the array operations merged in; there need to be
some active plan for making Cython proceed for numerical computation and
compete head-to-head against Fortran.

I think there's a fundamental reason to this shift towards Numba; for
numerical computation, it really makes a lot of sense to have the
compiler around at run-time to specialize for different cases as you hit
them. That is how the future will look like, and Numba and LLVM (and
Julia for that matter) allows preparing for this in ways which the
Cython compile-to-C paradigm doesn't.

Dag Sverre

[Dag Sverre Seljebotn]

I should be clearer:

Cython has a prominent role as the de facto standard way of making
wrappers the scientific Python community (and a lot of non-scientific
libraries too, though it isn't de facto standard there).

I think that will continue. But as for numerical computation, my guess
is that Numba will take over more of that in the future. The numerical
features currently in Cython have served many very well so far though.

Dag Sverre

[Stefan Behnel]

Dag Sverre Seljebotn, 11.03.2013 20:23:

I agree. I think it makes sense that Numba's wrapper is (apparently)
written in Cython, rather than the code that it generates and runs.

Stefan