Cython in jupyter notebook

[Simon King]

Hi!

The following cython code compiles fine in SageMath command line version
and it *used* to compile fine in the jupyter notebook one year ago:

cython("""
def mantisse():
cdef double a = 1
cdef double b = 2
cdef int i = 0
while True:
a += b**(-i)
print("%2d: %.53f"%(i,a))
if a==1:
return i-1
i += 1
a = 1
""")

However, in the jupyter notebook I no get the error

RuntimeError Traceback (most recent call last)
<ipython-input-3-672ba140476c> in <module>()
12 i += 1
13 a = 1
---> 14 """)

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/lazy_import.pyx in sage.misc.lazy_import.LazyImport.__call__ (build/cythonized/sage/misc/lazy_import.c:3756)()
352 True
353 """
--> 354 return self.get_object()(*args, **kwds)
355
356 def __repr__(self):

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/cython.pyc in cython_compile(code, **kwds)
1005 with open(tmpfile,'w') as f:
1006 f.write(code)
-> 1007 return cython_import_all(tmpfile, get_globals(), **kwds)

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/cython.pyc in cython_import_all(filename, globals, **kwds)
895 code
896 """
--> 897 m = cython_import(filename, **kwds)
898 for k, x in iteritems(m.__dict__):
899 if k[0] != '_':

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/cython.pyc in cython_import(filename, **kwds)
870 - the module that contains the compiled Cython code.
871 """
--> 872 name, build_dir = cython(filename, **kwds)
873
874 oldpath = sys.path

/home/king/Sage/git/sage/local/lib/python2.7/site-packages/sage/misc/cython.pyc in cython(filename, verbose, compile_message, use_cache, create_local_c_file, annotate, sage_namespace, create_local_so_file)
669 except Exception as msg:
670 msg = str(msg) + "\n" + distutils_messages
--> 671 raise RuntimeError(msg.strip())
672
673 if verbose >= 0:

RuntimeError: fileno

What goes wrong?

I'd like to have it solved till the day after tomorrow, as the notebook
is to be used in some exercise group.

Best regards,
Simon

[Nils Bruin]

On Sunday, April 15, 2018 at 7:11:06 AM UTC-7, Simon King wrote:

Hi!

The following cython code compiles fine in SageMath command line version
and it *used* to compile fine in the jupyter notebook one year ago:

 cython("""
 def mantisse():
     cdef double a = 1
     cdef double b = 2
     cdef int i = 0
     while True:
         a += b**(-i)
         print("%2d: %.53f"%(i,a))
         if a==1:
             return i-1
         i += 1
         a = 1
 """)

However, in the jupyter notebook I no get the error

You can trigger the error by a straight call

sage.misc.cython.cython(<cython file>)

The traceback (there is only one frame):

...

    669     except Exception as msg:
    670         msg = str(msg) + "\n" + distutils_messages
--> 671         raise RuntimeError(msg.strip())
    672 
    673     if verbose >= 0:

RuntimeError: fileno

I suspect it's this commit:

https://git.sagemath.org/sage.git/commit/?h=c59f37ad5ee719b23293a0f677422f5f818b7981

where this code was committed:

    try:
        # Capture errors from distutils and its child processes
        with open(os.path.join(target_dir, name + ".err"), 'w+') as errfile:
            try:
                with redirection(sys.stderr, errfile, close=False):
                    dist.run_command("build")
            finally:
                errfile.seek(0)
                distutils_messages = errfile.read()
    except Exception as msg:

        msg = str(msg) + "\n" +

distutils_messages
        raise RuntimeError(msg.strip())

This works on the command line:

sage: F=open("er",'w+')
sage: sage.misc.sage_ostools.redirection(sys.stderr,F)

but in jupyter we get an error:

UnsupportedOperation: fileno

because sys.stderr is now:

<ipykernel.iostream.OutStream object at 0x7ff909f6ed10>

It looks like sage_ostools.redirection might need some love.

[Simon King]

Hi Nils,

On 2018-04-15, Nils Bruin <nbr...@sfu.ca> wrote:
> ...

> where this code was committed:
>
> try:
> # Capture errors from distutils and its child processes
> with open(os.path.join(target_dir, name + ".err"), 'w+') as errfile:
> try:
> with redirection(sys.stderr, errfile, close=False):
> dist.run_command("build")
> finally:
> errfile.seek(0)
> distutils_messages = errfile.read()
> except Exception as msg:
> msg = str(msg) + "\n" + distutils_messages
> raise RuntimeError(msg.strip())

Thank you! I'll try to undo that commit till the exercise group is
done...

Best regards,
Simon

[Nils Bruin]

Problems like this have been encountered before. This seems to work around problems in the other direction (i.e., let Jupyter deal with an actual file rather than one of its streams):

https://stackoverflow.com/questions/34145950/is-there-a-way-to-redirect-stderr-to-file-in-jupyter

The quickest workaround for now is to just eliminate the whole error redirection. We didn't have it before either. If you have control over the distribution you use to teach, that might get you through your lecture.

The next step would be to disable the redirection if stderr isn't an actual file.

The full solution could probably make do with saving sys.stderr, assigning the temporary file to it and afterwards putting it back. You would need the right amount of flushes. Furthermore, this is not at all thread-safe (the current solution isn't either), so general hygiene would probably require forking before redirecting and then reaping the child once the build has completed.

[Simon King]

Hi Nils,

On 2018-04-15, Nils Bruin <nbr...@sfu.ca> wrote:

> The quickest workaround for now is to just eliminate the whole error
> redirection. We didn't have it before either. If you have control over the
> distribution you use to teach, that might get you through your lecture.

Indeed I'll just use the Sage installation on my laptop.

Best regards,
Simon

[Jeroen Demeyer]

On 2018-04-15 16:08, Simon King wrote:
> Hi!
>
> The following cython code compiles fine in SageMath command line version
> and it *used* to compile fine in the jupyter notebook one year ago:

Fix at https://trac.sagemath.org/ticket/25177