Cython 0.26 and Python 3

2,346 views
Skip to first unread message

Nathan Dunfield

unread,
Jul 25, 2017, 10:54:08 PM7/25/17
to cython-users
Dear Cython Folks,

After upgrading to Cython 0.26 from 0.25.2, I am now getting the following error at runtime with the Python package SnapPy [1]:

  File "/Users/dunfield/work/SnapPy/build/lib.macosx-10.6-intel-3.6/snappy/__init__.py", line 7, in <module>
    from .SnapPy import (AbelianGroup, HolonomyGroup, FundamentalGroup,
  File "/Users/dunfield/work/SnapPy/build/lib.macosx-10.6-intel-3.6/snappy/browser.py", line 18, in <module>
    from .SnapPy import SnapPeaFatalError
  File "cython/core/basic.pyx", line 676, in init SnapPy
    cdef class AbelianGroup(object):
AttributeError: type object 'SnapPy.AbelianGroup' has no attribute '__reduce_cython__'

This problem manifests itself on both macOS (clang) and Windows (msvc) but only with Python 3.* as good old Python 2.7 continues to work fine.  I believe it not anything special about the AbelianGroup class (which is cdef'ed but quite basic) as moving code around I can get similar error for other cdef'd classes. 

Any suggestions for what might be going wrong?  Build logs available at [2].  

Thanks,

Nathan



Nathan Dunfield

unread,
Jul 26, 2017, 10:43:49 AM7/26/17
to cython-users
I found a workaround, namely adding 

#cython: auto_pickle=False 

to my "pyx" files.

Nathan

Stefan Behnel

unread,
Jul 26, 2017, 11:59:16 AM7/26/17
to cython...@googlegroups.com
Hi!

Thanks for the report.

Nathan Dunfield schrieb am 26.07.2017 um 04:54:
> After upgrading to Cython 0.26 from 0.25.2, I am now getting the following
> error at runtime with the Python package SnapPy [1]:
>
> File
> "/Users/dunfield/work/SnapPy/build/lib.macosx-10.6-intel-3.6/snappy/__init__.py",
> line 7, in <module>
> from .SnapPy import (AbelianGroup, HolonomyGroup, FundamentalGroup,
> File
> "/Users/dunfield/work/SnapPy/build/lib.macosx-10.6-intel-3.6/snappy/browser.py",
> line 18, in <module>
> from .SnapPy import SnapPeaFatalError
> File "cython/core/basic.pyx", line 676, in init SnapPy
> cdef class AbelianGroup(object):
> AttributeError: type object 'SnapPy.AbelianGroup' has no attribute
> '__reduce_cython__'

The package failed to complete its build for me from a bitbucket zip
download (some python/twister directory missing), but at least from a
glance at the C code it generated up to that point, I couldn't see anything
wrong with it. Tried it with CPython 3.5. The file definitely contains the
method above (after removing the "auto_pickle=False" directive again).

Could you search for occurrences of "__reduce_cython__" in SnapPy.c on your
side? There should be a line like this in a PyMethodDef struct:

{"__reduce_cython__",
(PyCFunction)__pyx_pw_6SnapPy_12AbelianGroup_21__reduce_cython__,
METH_NOARGS, __pyx_doc_6SnapPy_12AbelianGroup_20__reduce_cython__},


> This problem manifests itself on both macOS (clang) and Windows (msvc) but
> only with Python 3.* as good old Python 2.7 continues to work fine.

That is very surprising, because there shouldn't be any Py2/3 difference
here. Could you check if both Py2 and Py3 generate the exact same .c file
for you?


> I believe it not anything special about the AbelianGroup class (which is
> cdef'ed but quite basic) as moving code around I can get similar error for
> other cdef'd classes.

It seems to be the first that is being created during module
initialisation, so probably it just fails for all of your cdef classes.

Stefan

Nathan Dunfield

unread,
Jul 27, 2017, 3:01:27 PM7/27/17
to cython-users, stef...@behnel.de
Stefan, 

Thanks for looking into this!


Could you search for occurrences of "__reduce_cython__" in SnapPy.c on your
side? There should be a line like this in a PyMethodDef struct:

{"__reduce_cython__",
(PyCFunction)__pyx_pw_6SnapPy_12AbelianGroup_21__reduce_cython__,
METH_NOARGS, __pyx_doc_6SnapPy_12AbelianGroup_20__reduce_cython__},

There definitely some occurrence like this.  I have posted the whole ".c" file here at 


if you want to have a look.

> This problem manifests itself on both macOS (clang) and Windows (msvc) but
> only with Python 3.* as good old Python 2.7 continues to work fine.

That is very surprising, because there shouldn't be any Py2/3 difference
here. Could you check if both Py2 and Py3 generate the exact same .c file
for you?

The two cythonized files are identical except for a couple whitespace differences in the first 20 lines (the metadata comment at the top). 

Thanks,

Nathan 

Nathan Dunfield

unread,
Jul 27, 2017, 3:36:10 PM7/27/17
to cython-users, stef...@behnel.de
Stefan,

Another interesting point:  I tried it out on Linux, specifically the "manylinux1" Docker image, and SnapPy builds and tests fine with Python 3.6 and Cython 0.26 there (after of course removing the "auto_pickle=False" directive).  I checked and the "SnapPy.c" file generated there is identical to the other two, so that's not the issue. 

So seemingly the problem has something to do with the fact that the compiler is not "gcc" but rather "clang" on macOS and "msvc" on Windows.  Also, it should be noted that the "setup.py" file for SnapPy compiles everything as C++ rather than C code, even though this particular file is called "SnapPy.c".

By the way, if you want to try building SnapPy again --- unfortunately I don't recognize the problem you refer to --- I recommend doing:

cd snappy
hg update -r 092c5c15a4e8

as the current tip is rather in flux and unlikely to work for you in the best of circumstances. 

Nathan




Hai Nguyen

unread,
Jul 27, 2017, 6:07:00 PM7/27/17
to cython-users
: Tested with different python versions on Linux:

 I got similar issue with pytraj: https://github.com/Amber-MD/pytraj

with slightly different message (Tested with different python versions on Linux:)

      File "stringsource", line 2, in pytraj.trajectory.c_traj.c_trajectory.TrajectoryCpptraj.__reduce_cython__
    TypeError: no default __reduce__ due to non-trivial __cinit__

Hai

Robert Bradshaw

unread,
Jul 27, 2017, 6:28:25 PM7/27/17
to cython...@googlegroups.com
This is unrelated, it means you tried to pickle a
pytraj.trajectory.c_traj.c_trajectory.TrajectoryCpptraj. It shouldn't
have worked before (did it?).

Hai Nguyen

unread,
Jul 27, 2017, 9:54:21 PM7/27/17
to cython-users
It has been working like a charm last two years.
Only happens to fail with v0.26

Hai 


--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Robert Bradshaw

unread,
Jul 27, 2017, 10:18:49 PM7/27/17
to cython...@googlegroups.com
Could you clarify what you mean by"it" that's been working fine?

Hai Nguyen

unread,
Jul 27, 2017, 10:37:49 PM7/27/17
to cython-users
On Thu, Jul 27, 2017 at 10:18 PM, Robert Bradshaw <robe...@gmail.com> wrote:
Could you clarify what you mean by"it" that's been working fine?


Hi, I just refer "it" as "it" in your previous post.
Basically pytraj has been using cython for two years without any trouble of __reduce__ issue.
The error as you saw happens with v0.26

Hai

Stefan Behnel

unread,
Jul 28, 2017, 2:16:57 AM7/28/17
to cython...@googlegroups.com
Hai Nguyen schrieb am 28.07.2017 um 04:37:
> On Thu, Jul 27, 2017 at 10:18 PM, Robert Bradshaw wrote:
>> On Jul 27, 2017 6:54 PM, "Hai Nguyen" wrote:
>>> On Thu, Jul 27, 2017 at 6:27 PM, Robert Bradshaw wrote:
>>>> On Thu, Jul 27, 2017 at 3:06 PM, Hai Nguyen wrote:
>>>>> I got similar issue with pytraj: https://github.com/Amber-MD/pytraj
>>>>>
>>>>> with slightly different message (Tested with different python versions
>>>>> on Linux:)
>>>>>
>>>>> File "stringsource", line 2, in
>>>>> pytraj.trajectory.c_traj.c_trajectory.TrajectoryCpptraj.__re
>>>>> duce_cython__
>>>>> TypeError: no default __reduce__ due to non-trivial __cinit__
>>>>
>>>> This is unrelated, it means you tried to pickle a
>>>> pytraj.trajectory.c_traj.c_trajectory.TrajectoryCpptraj. It shouldn't
>>>> have worked before (did it?).
>>>>
>>>>
>>> It has been working like a charm last two years.
>>> Only happens to fail with v0.26
>>>
>> Could you clarify what you mean by"it" that's been working fine?
>>
>>
> Hi, I just refer "it" as "it" in your previous post.
> Basically pytraj has been using cython for two years without any trouble of
> __reduce__ issue.
> The error as you saw happens with v0.26

What (I think) Robert meant, was: are you sure that the pickling of a
TrajectoryCpptraj instance worked correctly before? Looking at your
implementation, I cannot see any dedicated pickling support, and extension
types (cdef classes) do not support pickling without specifically
implementing the pickle protocol for them. Automatic support for that is
the new feature in Cython 0.26 that we're talking about here.

What previously (pre-0.26) happened instead, was: Python tried to pickle
them, didn't find any internal state for them, and pickled an empty class.
On unpickling, it instantiated a fresh instance without remembering the
actual state.

That means that it is quite possible that it *seemed* to work, in the sense
that there was no obvious immediate failure, but you'd end up with a
probably unusable default instance of the pickled class instead of what you
actually intended to pickle.

Could you please confirm that you validated the result of pickling a
TrajectoryCpptraj instance and that it previously unpickled a correct
representation of the pickled instance and not just something bogus?

Regarding the error you get, that's a safety measure, probably still a bit
too broad for now. You can explicit enable auto-pickling support for this
class in Cython 0.26 by adding the decorator "@cython.auto_pickle(True)".

Stefan

Stefan Behnel

unread,
Jul 28, 2017, 3:26:47 AM7/28/17
to cython...@googlegroups.com
Nathan Dunfield schrieb am 27.07.2017 um 21:36:
> Another interesting point: I tried it out on Linux, specifically the
> "manylinux1" Docker image, and SnapPy builds and tests fine with Python 3.6
> and Cython 0.26 there (after of course removing the "auto_pickle=False"
> directive). I checked and the "SnapPy.c" file generated there is identical
> to the other two, so that's not the issue.
>
> So seemingly the problem has something to do with the fact that the
> compiler is not "gcc" but rather "clang" on macOS and "msvc" on Windows.
> Also, it should be noted that the "setup.py" file for SnapPy compiles
> everything as C++ rather than C code, even though this particular file is
> called "SnapPy.c".

Lacking access to these two platforms, I tried at least using clang (3.8.0)
on my Linux system, but it gets past the code that fails for you (I tried
it from the local checkout and it fails at a later import). It initialises
the extension types without errors for me.

Having this as a platform related problem seems even more surprising.
There's really nothing platform specific in the code, and I also cannot see
anything suspicious regarding a C compiler dependency. The code in the
internal helper function "__Pyx_setup_reduce()", that results in the
exception for you, is really just calling straight forward C-API functions.
I tried both 0.26 and current Cython master (which has some supposedly
unrelated changes here), but both work for me on your code.


> unfortunately I don't recognize the problem you refer to

I'm getting this error at the end of the build:

"""
copying
build/lib.linux-x86_64-3.6-pydebug/snappy/twister/twister_core.cpython-36dm-x86_64-linux-gnu.so
-> python/twister

error: could not create
'python/twister/twister_core.cpython-36dm-x86_64-linux-gnu.so': No such
file or directory
"""

There's a top-level "twister" directory on my side, but none in the
"python" directory right next to it.


> hg clone https://bitbucket.org/t3m/snappy
> cd snappy
> hg update -r 092c5c15a4e8

Could maybe someone with a MacOS system try to reproduce the original
failure? I.e. this:

"""
File "cython/core/basic.pyx", line 676, in init SnapPy
cdef class AbelianGroup(object):
AttributeError: type object 'SnapPy.AbelianGroup' has no attribute
'__reduce_cython__'
"""

It's supposed to show under Py3 on MacOS and Windows.

Stefan

Jason Madden

unread,
Jul 28, 2017, 8:01:11 AM7/28/17
to cython-users, stef...@behnel.de


On Friday, 28 July 2017 02:26:47 UTC-5, Stefan Behnel wrote:

> hg clone https://bitbucket.org/t3m/snappy
> cd snappy
> hg update -r 092c5c15a4e8

Could maybe someone with a MacOS system try to reproduce the original
failure? I.e. this:

"""
  File "cython/core/basic.pyx", line 676, in init SnapPy
    cdef class AbelianGroup(object):
AttributeError: type object 'SnapPy.AbelianGroup' has no attribute
'__reduce_cython__'
"""

It's supposed to show under Py3 on MacOS and Windows.

I'm on MacOS 10.12.6. I did the above steps, and then the steps from the Appveyor log referenced earlier ("setup.py install", "python -m snappy.test") in a fresh virtualenv of python 3.6 (macports) with Cython 0.26 installed. I could not reproduce the problem.

Now, I didn't have OpenGL or tkinter installed, so it looked like the `browser` module wasn't even being imported. So I tried to manually import the source of the trouble based on the stack trace in the log:

    $  python -c 'from snappy.SnapPy import AbelianGroup; print(AbelianGroup)'
    <class 'SnapPy.AbelianGroup'>

I tried with both clang 8.1 and clang 9.0 successfully.

I did run into a separate problem compiling spherogram-1.6.1[1], but fortunately they had wheels published to PyPI that I was able to install.

Jason

[1] Stacktrace:

Running spherogram-1.6.1/setup.py -q bdist_egg --dist-dir /var/folders/y5/x7pvzk651c3dqkllbxd1jd280000gn/T/easy_install-ake2lv_n/spherogram-1.6.1/egg-dist-tmp-rml3ypna
Compiling planarmap_src/planarmap.pyx because it depends on //VirtualEnvs/tmp-60c4989469ba9cc6/lib/python3.6/site-packages/Cython/Includes/libc/string.pxd.
[1/1] Cythonizing planarmap_src/planarmap.pyx
Traceback (most recent call last):
  File "Cython/Compiler/Visitor.py", line 180, in Cython.Compiler.Visitor.TreeVisitor._visit
  File "//VirtualEnvs/tmp-60c4989469ba9cc6/lib/python3.6/site-packages/Cython/Compiler/ParseTreeTransforms.py", line 2416, in visit_StatListNode
    if not self.current_directives['remove_unreachable']:
TypeError: 'NoneType' object is not subscriptable
 

Hai Nguyen

unread,
Jul 28, 2017, 9:10:18 AM7/28/17
to cython...@googlegroups.com
Hi, I see what's going on after you said. No, that class is not pickled correctly in pytraj package before.


Regarding the error you get, that's a safety measure, probably still a bit
too broad for now. You can explicit enable auto-pickling support for this
class in Cython 0.26 by adding the decorator "@cython.auto_pickle(True)".

I see. Thanks.

Hai

Stefan


--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.

Nathan Dunfield

unread,
Jul 28, 2017, 10:15:30 AM7/28/17
to cython-users, stef...@behnel.de
I'm on MacOS 10.12.6. I did the above steps, and then the steps from the Appveyor log referenced earlier ("setup.py install", "python -m snappy.test") in a fresh virtualenv of python 3.6 (macports) with Cython 0.26 installed. I could not reproduce the problem.

Now, I didn't have OpenGL or tkinter installed, so it looked like the `browser` module wasn't even being imported. So I tried to manually import the source of the trouble based on the stack trace in the log:

    $  python -c 'from snappy.SnapPy import AbelianGroup; print(AbelianGroup)'
    <class 'SnapPy.AbelianGroup'>

I tried with both clang 8.1 and clang 9.0 successfully.

Jason,

Thanks for looking into this!  I get basically the same error with snippet above, details below.  I'm using macOS 10.12.5, with clang "Apple LLVM version 8.1.0 (clang-802.0.42)", but one difference I see between our setups is that your Python comes via macports and mine is from the Python.org installer.  Here are the exact steps I used:

hg clone https://bitbucket.org/t3m/snappy SnapPyCythonIssue
cd SnapPyCythonIssue
hg update -r 092c5c15a4e8
py3 -m virtualenv venv
venv/bin/pip install cython sphinx
venv/bin/python setup.py pip_install
venv/bin/python -c 'from snappy.SnapPy import AbelianGroup; print(AbelianGroup)'

which resulted in:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/dunfield/work/SnapPyCythonIssue/venv/lib/python3.6/site-packages/snappy/__init__.py", line 6, in <module>
    from .SnapPy import (AbelianGroup, HolonomyGroup, FundamentalGroup,
  File "/Users/dunfield/work/SnapPyCythonIssue/venv/lib/python3.6/site-packages/snappy/browser.py", line 18, in <module>
    from .SnapPy import SnapPeaFatalError
  File "cython/core/basic.pyx", line 676, in init SnapPy
    cdef class AbelianGroup(object):
AttributeError: type object 'SnapPy.AbelianGroup' has no attribute '__reduce_cython__'

After the usual substitution ('venv/bin' -> '/venv/Scripts') the same series of commands on Windows 10 with Python 3.6 (Python.org installer) and MSVC 14 gives essentially the same error message.
 
I did run into a separate problem compiling spherogram-1.6.1[1], but fortunately they had wheels published to PyPI that I was able to install.

Weird, I am also one of the developers of spherogram and I have had no problems with Cython 0.26 there.   

Best,

Nathan

Jason Madden

unread,
Jul 29, 2017, 7:40:22 AM7/29/17
to cython-users, stef...@behnel.de


On Friday, 28 July 2017 09:15:30 UTC-5, Nathan Dunfield wrote:
but one difference I see between our setups is that your Python comes via macports and mine is from the Python.org installer.  

That seems to be it. If I use the python.org installer, I *do* reproduce the error as described. I checked both Python3.6.0 and 3.6.2 with clang 8.1, and Python 3.6.2 with clang 9.0; all combos produced the error.

Jason.

Jason Madden

unread,
Jul 29, 2017, 8:21:13 AM7/29/17
to cython-users, stef...@behnel.de


On Friday, 28 July 2017 09:15:30 UTC-5, Nathan Dunfield wrote:

I did run into a separate problem compiling spherogram-1.6.1[1], but fortunately they had wheels published to PyPI that I was able to install.

Weird, I am also one of the developers of spherogram and I have had no problems with Cython 0.26 there.   

If I install spherogram standalone (i.e., grab the .tar.gz, unpack it, and either run 'pip install .' or 'setup.py install'), I have no problems. It's only when it's installed as a dependency of  snappy via 'setup.py install' in the snappy directory that I get that error. Presumably it's some sort of isolation issue, then.

Jason
 

Nathan Dunfield

unread,
Jul 29, 2017, 3:17:12 PM7/29/17
to cython-users, stef...@behnel.de

On Friday, 28 July 2017 09:15:30 UTC-5, Nathan Dunfield wrote:
but one difference I see between our setups is that your Python comes via macports and mine is from the Python.org installer.  

That seems to be it. If I use the python.org installer, I *do* reproduce the error as described. I checked both Python3.6.0 and 3.6.2 with clang 8.1, and Python 3.6.2 with clang 9.0; all combos produced the error.

Glad to hear it's not just me!  Another data point: I installed Python 3.6.2 via homebrew on the same machine as before and it *also* exhibits the error.  I wonder what is special about macports' Python... 

Nathan


Jason Madden

unread,
Jul 29, 2017, 4:10:28 PM7/29/17
to cython-users, stef...@behnel.de

On Saturday, 29 July 2017 14:17:12 UTC-5, Nathan Dunfield wrote:
I installed Python 3.6.2 via homebrew on the same machine as before and it *also* exhibits the error.  I wonder what is special about macports' Python... 

Interesting.  MacPorts does apply a small variety of patches[1] to the binary it builds. A few of them look to do with linker args, but the seem pretty minor, but I didn't look all that close.

One other way in which things differ: The python.org distribution is a universal binary (i386 and x86_64 in one file). The macports python *isn't* universal by default (just x86_64 on my machine). I don't know what homebrew does by default...but I rebuilt my macports python as universal, and didn't reproduce the error (either when using the dependency wheels from PyPI or installing them from source).

Another way: python.org installs the Tk bindings by default, macports does not. So I installed the Tk bindings into Macports, created a fresh virtualenv and a fresh clone of snappy and...BINGO! 

(Or at least, we seem to have tracked down the relevant difference, if not the root cause.)

$ python --version
Python 3.6.2
$ python -c 'import tkinter'
$ python -c 'from snappy.SnapPy import AbelianGroup; print(AbelianGroup)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "//lib/python3.6/site-packages/snappy/__init__.py", line 6, in <module>
    from .SnapPy import (AbelianGroup, HolonomyGroup, FundamentalGroup,
  File "//lib/python3.6/site-packages/snappy/browser.py", line 18, in <module>
    from .SnapPy import SnapPeaFatalError
  File "cython/core/basic.pyx", line 676, in init SnapPy
    cdef class AbelianGroup(object):
AttributeError: type object 'SnapPy.AbelianGroup' has no attribute '__reduce_cython__'

Let's see what happens if we uninstall tkinter:

$ rm -f $VIRTUAL_ENV/lib/python3.6/site-packages/_tkinter.cpython-36m-darwin.so
$ python -c 'import tkinter'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/tkinter/__init__.py", line 36, in <module>
    import _tkinter # If this fails your Python may not be configured for Tk
ModuleNotFoundError: No module named '_tkinter'
$ python -c 'from snappy.SnapPy import AbelianGroup; print(AbelianGroup)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "//lib/python3.6/site-packages/snappy/__init__.py", line 6, in <module>
    from .SnapPy import (AbelianGroup, HolonomyGroup, FundamentalGroup,
  File "cython/core/basic.pyx", line 50, in init SnapPy
    from .horoviewer import HoroballViewer
  File "//lib/python3.6/site-packages/snappy/horoviewer.py", line 3, in <module>
    from .CyOpenGL import *
  File "opengl/CyOpenGL.pyx", line 36, in init CyOpenGL
    cdef class vector3:
AttributeError: type object 'CyOpenGL.vector3' has no attribute '__reduce_cython__'

Let's uninstall and rebuild in this tkinter-less environment:

$ pip uninstall -y snappy
Uninstalling snappy-2.5.3a1:
  Successfully uninstalled snappy-2.5.3a1
$ pip install .
Processing /private/tmp/snappy
...
Installing collected packages: snappy
  Running setup.py install for snappy ... \
Successfully installed snappy-2.5.3a1
$ python -c 'from snappy.SnapPy import AbelianGroup; print(AbelianGroup)'
<class 'SnapPy.AbelianGroup'>

As expected, with to tkinter to be found at build time, it works.

Nathan Dunfield

unread,
Jul 29, 2017, 9:52:00 PM7/29/17
to cython-users, stef...@behnel.de
Jason,

I investigated and believe I know why "tkinter" is relevant and even better what the (essentially unrelated) root cause is.  Some quick background: snappy provides a Tk-based GUI, complete with an OpenGL widget for 3D graphics, but can also run "headless" on systems where tkinter is not available.  This is detected at build time when it decides whether or not to build the CyOpenGL module; at run time, it checks whether the CyOpenGL module is available to know whether or not to load the module "browser.py" which requires CyOpenGL.  The issue seems to be line 18 of "browser.py" which reads:

from .SnapPy import SnapPeaFatalError

If I keep tkinter available but change this line to (the source file is "snappy/python/browser.py") 

SnapPeaFatalError = None

then the error goes away and all the doctests pass ("python -m snappy.test"), including the 3D graphics.  

----------

I now believe this is some kind of circular import issue: "snappy/__init__.py" imports the binary module "snappy.SnapPy" which imports "snappy/browser.py" which in turn tries to import "SnapPeaFatalError" from "snappy.SnapPy".   The relevant part of the Cython code is "cython/core/basic.pyx" whose first 60 lines include both the definition of "SnapPeaFatalError" and then the import of "browser.py".   Since "SnapPeaFatalError" is not "cdef'd", presumably we could (and indeed should) break the circular import by moving it to another file.  This doesn't explain why it stopped working, though.  

Best,

Nathan


Robert Bradshaw

unread,
Jul 31, 2017, 2:05:18 PM7/31/17
to cython...@googlegroups.com
To summarize, there's two issues in this thread.

The first is an attribute error with __reduce_cython__ in snappy,
triggered by some import issue (which is suppressed if tkinter is not
present). I suspect that this is due to the exception from a hasattr
call not getting properly cleared, and then surfacing later (and
perhaps this was formerly an import error which was being caught and
ignored), but I think there's still some fix/cleanup to be done here.

The second is with "pytraj" (and possibly others) where we now throw
an explicit error rather than generate broken pickles. This fix in
this case is to fix the project to either add pickling logic or avoid
pickling. (One can revert back to the old logic of producing broken
pickles by setting auto_pickle=False too, though obviously this is not
recommended--the fact that broken pickles were being generated
(leading to silent or difficult to hunt down bugs) was one of the
motivations for this change in the first place.)

Does that sound right?

Hai Nguyen

unread,
Jul 31, 2017, 2:31:14 PM7/31/17
to cython-users
On Mon, Jul 31, 2017 at 2:04 PM, Robert Bradshaw <robe...@gmail.com> wrote:
To summarize, there's two issues in this thread.

The first is an attribute error with __reduce_cython__ in snappy,
triggered by some import issue (which is suppressed if tkinter is not
present). I suspect that this is due to the exception from a hasattr
call not getting properly cleared, and then surfacing later (and
perhaps this was formerly an import error which was being caught and
ignored), but I think there's still some fix/cleanup to be done here.

The second is with "pytraj" (and possibly others) where we now throw
an explicit error rather than generate broken pickles.

 
This fix in
this case is to fix the project to either add pickling logic or avoid
pickling.

sounds good.
 
(One can revert back to the old logic of producing broken
pickles by setting auto_pickle=False too, though obviously this is not
recommended--the fact that broken pickles were being generated
(leading to silent or difficult to hunt down bugs) was one of the
motivations for this change in the first place.)


 
Does that sound right?


Yes, sounds good.  thanks.

Hai
 

> For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe@googlegroups.com.

Nathan Dunfield

unread,
Jul 31, 2017, 3:10:29 PM7/31/17
to cython-users
On Monday, July 31, 2017 at 1:05:18 PM UTC-5, Robert Bradshaw wrote:
The first is an attribute error with __reduce_cython__ in snappy,
triggered by some import issue (which is suppressed if tkinter is not
present). I suspect that this is due to the exception from a hasattr
call not getting properly cleared, and then surfacing later (and
perhaps this was formerly an import error which was being caught and
ignored), but I think there's still some fix/cleanup to be done here.

Yes, that is correct.   

Nathan
Reply all
Reply to author
Forward
0 new messages