Cython 0.28 beta 1 is available!

[Stefan Behnel]

Hi all,

after almost half a year of development and a late couple of weeks of bug
hunting and feature cleanup, I'm happy to announce the availability of the
first beta of Cython 0.28.

https://github.com/cython/cython/releases/tag/0.28b1

This is a rather big feature release, please give it some testing. I'll not
even start listing the major improvements here, but it comes with some
really long-requested language features and lots of new optimisations, so
please take a look at the changelog:

https://github.com/cython/cython/blob/0.28b1/CHANGES.rst

Have fun,

Stefan

[Jeroen Demeyer]

There are still a lot of superfluous "redeclared" warnings, see
https://github.com/cython/cython/issues/2013

This doesn't technically break anything, but it's an annoying regression
since 0.27.3. So it would be good to fix that before the final release.

[Jeroen Demeyer]

Breakage in SageMath due to

https://github.com/cython/cython/issues/2132

[Jeroen Demeyer]

Just a comment: this change

> * The signature of the special ``__richcmp__()`` method was corrected to recognise
> the type of the first argument as ``self``. It was previously treated as plain
> object, but CPython actually guarantees that it always has the correct type.

might cause compilation errors for code that did things like

def __richcmp__(self, other, int op):
self = .....

We had 2 instances of that in SageMath.

Of course, the change to Cython's handling of __richcmp__ makes sense.
However, it might be good to make that change more prominent in the
changelog by putting it in the "Features added" section instead of "Bugs
fixed".

[Stefan Behnel]

Hi again,

thanks to your feedback, I was able to solve a couple of issues and
regressions in the first beta. Here's the second one.

Release:
https://github.com/cython/cython/releases/tag/0.28b2

Differences:
https://github.com/cython/cython/compare/0.28b1...0.28b2

Overall changelog for 0.28:
https://github.com/cython/cython/blob/0.28b2/CHANGES.rst

Looks like the final release is close. If nothing major comes up, I'll
prepare the final release in a couple of days.

Stefan

[Stefan Behnel]

... and one more iteration, since in order to fix an error reporting bug, I
had to change a setting for internally generated code, which should not
impact user code, but might, especially in C++ mode.

So here's the release candidate:

https://github.com/cython/cython/releases/tag/0.28rc1

Stefan

[Jeroen Demeyer]

Two comments about the changelog:

1. Issue #2013 should not appear in the changelog, since it fixes a bug
which was only introduced after the release of 0.27.3. Users who never
installed Cython master or the 0.28betas wouldn't know about #2013.

2. Nothing is mentioned about the verbatim include feature

http://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html?highlight=verbatim#including-verbatim-c-code

[Stefan Behnel]

Hi Jeroen, thanks for your comments.

Jeroen Demeyer schrieb am 11.03.2018 um 11:37:
> Two comments about the changelog:
>
> 1. Issue #2013 should not appear in the changelog, since it fixes a bug
> which was only introduced after the release of 0.27.3. Users who never
> installed Cython master or the 0.28betas wouldn't know about #2013.

I added it there because it was already a warning before, just not shown by
default. Thus, the bug might have had an impact on users who increased the
warning level. Also, the change is not strictly reducing the scope of the
warning and I can't guarantee that some users might see warnings now that
were not there before.

But you're right, #1874 is actually the one to mention, not #2013.

> 2. Nothing is mentioned about the verbatim include feature
>
> http://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html?highlight=verbatim#including-verbatim-c-code

Thanks, yes, that was missing.

Stefan

[Jeroen Demeyer]

On 2018-03-11 10:55, Stefan Behnel wrote:
> https://github.com/cython/cython/releases/tag/0.28rc1

Works great!

[Hai Nguyen]

On Sun, Mar 11, 2018 at 5:55 AM, Stefan Behnel <stef...@behnel.de> wrote:

... and one more iteration, since in order to fix an error reporting bug, I
had to change a setting for internally generated code, which should not
impact user code, but might, especially in C++ mode.

So here's the release candidate:

https://github.com/cython/cython/releases/tag/0.28rc1

Hi

I've tried the rc1 with my program (pytraj) and got two errors that I don't know how to fix.

1. "Constructor cannot be called without GIL unless all base constructors can also be called without GIL"

Following the message, I removed all the "nogil" I added before (works fine with older cython version (0.24, 0.25) but that does not help.

2.  " pytraj/datasets/c_datasets.pxd:172:19: Base constructor defined here."

To reproduce, just try "cythonize pytraj/datasets/c_datasets.pyx" from "nogil-remove" branch in

https://github.com/hainm/pytraj/blob/nogil-remove/pytraj/datasets/c_datasets.pyx

The offending line is here: https://github.com/hainm/pytraj/blob/nogil-remove/pytraj/datasets/c_datasets.pxd#L172

cheers

Hai

[Loïc Estève]

Hi,

In scikit-learn we really appreciate that the cython const memoryview feature
(https://github.com/cython/cython/pull/1869) has been merged so thanks a lot
for that! The main reason: we encourage the use of read-only memmaps for
parallel computing through joblib and until #1869 we actually had to go back to
the older ndarray interface (see for example
https://github.com/scikit-learn/scikit-learn/pull/4775/files#diff-5d8b82d506a543bd684a1d403d5941e0).
Similar work-arounds were used in many other instances in pandas and
scikit-learn, some of them can be found in the original cython issue:
https://github.com/cython/cython/issues/1605.

After #1869 was merged, I was quite excited to try it out in
scikit-learn. I bumped into a few limitations that for now prevents
scikit-learn to fully move to const memoryviews:
- https://github.com/cython/cython/issues/1772: const can not be used in
  combination with fused types. A lot of our cython functions can take both 32
  and 64 bit arrays so we would like to use const floating[:] for these
  function parameters.
- https://github.com/cython/cython/issues/2134: a non-const memoryview can not
  be created from the copy of a const memoryview. This one is more a nice to
  have than a blocker.

It would be really nice if both these issues were fixed in 0.28, but of course
as a non cython developer, it is rather hard to know how much time and effort
this represents. I saw that both issues were labelled "first good issues" so
maybe that does not represent to much effort from someone familiar with cython
code base? If you think that with a few pointers a cython newbie could get the
work done, I would be willing to work on PRs for these issues. If not, of
course I would be more than keen to test potential candidate fixes in
scikit-learn.

Cheers,
Loïc

[Stefan Behnel]

Salut Loic,

0.28 is far overdue. Let me get it out first, then we can look into these
issues. I'm happy to consider at least the first one a bug that can be
fixed anytime in a point release.

> I saw that both issues were labelled "first good issues" so
> maybe that does not represent to much effort from someone familiar with
> cython
> code base? If you think that with a few pointers a cython newbie could get
> the
> work done, I would be willing to work on PRs for these issues. If not, of
> course I would be more than keen to test potential candidate fixes in
> scikit-learn.

Usually it works like this:

- write a failing test case
- figure out where it goes wrong
- fix it.

No surprise here. ;) In this case, my guess is that there is some "if
type.is_const: type = type.const_base_type" missing somewhere, either in
FusedNode.py or in the fused types related parts of PyrexTypes.py. Some
special cased integration between const types and fused types.where a const
type needs to be unpacked into its (non-const) base type when comparing
fused types. Something like that.

The fact that this touches the combination of two orthogonal type modifiers
(const+fused) means that we'll probably need some more tests for different
ways of combining the two. It happens fairly often that writing tests is
more work than the actual change.

Stefan

[Jeroen Demeyer]

On 2018-03-12 21:35, Stefan Behnel wrote:
> It happens fairly often that writing tests is
> more work than the actual change.

+1

Writing good testcases is an art by itself.

[Loïc Estève]

0.28 is far overdue. Let me get it out first, then we can look into these
issues. I'm happy to consider at least the first one a bug that can be
fixed anytime in a point release.

OK fair enough, this is completely understandable. Thanks a lot for the pointers, I may try to give it a go at one point and dive a bit deep in the cython code!

Cheers,
Loïc

[Stefan Behnel]

Hi everyone,

I'm pleased to announce the immediate availability of Cython 0.28, after
almost half a year of development.

https://pypi.python.org/pypi/Cython/0.28


The major new features include:

* Cdef classes can now multiply inherit from ordinary Python classes,
as long as (exactly) one base class is an extension type.

* The "const" modifier can be applied to memoryview declarations to allow
read-only buffers as input.

* C code in the docstring of a "cdef extern" block is copied verbatimly
into the generated file.


The changelog lists the various other big and small features, optimisations
and bug fixes that went into this release:

https://github.com/cython/cython/blob/0.28/CHANGES.rst


Have fun,

Stefan

[Stefan Behnel]

Hi,

Hai Nguyen schrieb am 11.03.2018 um 21:47:
> I've tried the rc1 with my program (pytraj) and got two errors that I don't
> know how to fix.
>
> 1. "Constructor cannot be called without GIL unless all base constructors
> can also be called without GIL"
>
> Following the message, I removed all the "nogil" I added before (works fine
> with older cython version (0.24, 0.25) but that does not help.
>
> 2. " pytraj/datasets/c_datasets.pxd:172:19: Base constructor defined
> here."

That second line is only a hint for the first error, not a separate error.

> To reproduce, just try "cythonize pytraj/datasets/c_datasets.pyx" from
> "nogil-remove" branch in
> https://github.com/hainm/pytraj/blob/nogil-remove/pytraj/datasets/c_datasets.pyx
>
> The offending line is here:
> https://github.com/hainm/pytraj/blob/nogil-remove/pytraj/datasets/c_datasets.pxd#L172

The error message is misleading, but the name of the constructor is
misspelled here:

https://github.com/hainm/pytraj/blob/nogil-remove/pytraj/datasets/c_datasets.pxd#L236

Meaning, there is no declared constructor, and thus Cython auto-declares
one for you, and uses the "nogil" modifier for it. That fails because it
would require the base class constructors to also be callable without GIL,
but they are not in your declarations. Thus the error. It's a bug in
Cython, it should only declare it "nogil" if that's actually valid.

I'll fix it for the next bug fix release.

Stefan

[Stefan Behnel]

Fixed:
https://github.com/cython/cython/issues/2157

Stefan

[Hai Nguyen]

Great, thanks.

Hai

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.