snappy in Sage

[kcrisman]

Regarding SnapPy: 

I had the usual SSL problems on Mac, so I did as suggested at https://www.math.uic.edu/t3m/SnapPy/installing.html#sagemath and 

sage -i openssl
sage -f python2

did indeed get me to actually download SnapPy.  But then I got (even when using ./sage -pip install --no-binary :all: snappy) the following:

    Complete output from command python setup.py egg_info:

    Please put "# distutils: language=c++" in your .pyx or .pxd file(s)

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/private/var/folders/k8/nj0z1bkd11dcs1v5hbh_tknc92p43w/T/pip-build-41Ncor/snappy/setup.py", line 435, in <module>

        new = max(os.path.getmtime(file) for file in glob('cython/core/*.pyx'))

    ValueError: max() arg is an empty sequence

Since I don't actually use SnapPy but was testing out the installation for a friend, that was kind of disappointing.  Any clues?  This is with 8.3.rc0 (I didn't see anything indicating I needed an 8.4 beta).

- kcrisman

[Jeroen Demeyer]

On 2018-10-01 20:19, kcrisman wrote:
> new = max(os.path.getmtime(file) for file in
> glob('cython/core/*.pyx'))
>
> ValueError: max() arg is an empty sequence

It's bad packaging by upstream: it's running Cython but the Cython
source files are not in the snappy source tarball.

[Thierry]

Hi,

I had a similar issue. The fix was

- to install mercurial on your distro or with:

sage -pip install mercurial

- to install snappy from the repo

sage -pip install hg+https://bitbucket.org/t3m/snappy

Ciao,
Thierry



> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[kcrisman]

I had a similar issue. The fix was

 -  to install mercurial on your distro or with:

        sage -pip install mercurial

No way!  Hg rises from the dead!  Love it.

 

 - to install snappy from the repo

        sage -pip install hg+https://bitbucket.org/t3m/snappy

Thanks, Thierry.  Sounds like we need a ticket for this.  In particular, I wasn't even able to get sage -pip install mercurial to work on OS X.

    building 'mercurial.cext.osutil' extension

    gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -Imercurial -I/Users/karl.crisman/Downloads/sage/local/include/python2.7 -c mercurial/cext/osutil.c -o build/temp.macosx-10.9-x86_64-2.7/mercurial/cext/osutil.o -DHAVE_BSD_STATFS

    In file included from /System/Library/Frameworks/CoreGraphics.framework/Headers/CGContext.h:18:0,

                     from /System/Library/Frameworks/CoreGraphics.framework/Headers/CGBitmapContext.h:9,

                     from /System/Library/Frameworks/CoreGraphics.framework/Headers/CoreGraphics.h:11,

                     from /System/Library/Frameworks/ApplicationServices.framework/Headers/ApplicationServices.h:35,

                     from mercurial/cext/osutil.c:1320:

    /System/Library/Frameworks/CoreGraphics.framework/Headers/CGFont.h:53:40: error: initializer element is not constant

     static const CGFontIndex kCGGlyphMax = kCGFontIndexMax;

                                            ^~~~~~~~~~~~~~~

    error: command 'gcc' failed with exit status 1

    

Quite the difficult matter, this snappy.

Note also (in general) that 

You are using pip version 9.0.3, however version 18.0 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

though perhaps this is fixed in 8.4beta? 

[Jeroen Demeyer]

On 2018-10-02 16:50, kcrisman wrote:
> Thanks, Thierry. Sounds like we need a ticket for this.

I assume you mean a Snappy ticket since the problem with Snappy is
completely unrelated to Sage.

> Note also (in general) that
>
> You are using pip version 9.0.3, however version 18.0 is available.
>
> You should consider upgrading via the 'pip install --upgrade pip' command.
> though perhaps this is fixed in 8.4beta?

Yes.

[Samuel Lelievre]

One can also get the "Computational topology Sage Docker image":

  https://hub.docker.com/r/computop/sage/

It ships among other things:

- SageMath

- SnapPy

- Regina

- PHCpack

There is a link to it from the "Distribution" page on the Sage wiki:

  https://wiki.sagemath.org/Distribution

[kcrisman]

On Tuesday, October 2, 2018 at 10:52:43 AM UTC-4, Jeroen Demeyer wrote:

On 2018-10-02 16:50, kcrisman wrote:
> Thanks, Thierry.  Sounds like we need a ticket for this.

I assume you mean a Snappy ticket since the problem with Snappy is
completely unrelated to Sage.

Oh!  I guess that is probably the case, since sage -pip is just pip really except with a few variables? Actually there is one:

https://bitbucket.org/t3m/snappy/issues/13/trouble-with-installing-snappy-in-sagemath

(But nonetheless I still mean a Sage ticket to keep track of upstream; if users can't use Snappy, they can't use Sage-related behavior.)

[kcrisman]

On Tuesday, October 2, 2018 at 10:55:14 AM UTC-4, Samuel Lelievre wrote:

One can also get the "Computational topology Sage Docker image":

  https://hub.docker.com/r/computop/sage/

It ships among other things:

- SageMath

- SnapPy

- Regina

- PHCpack

Absolutely!  I actually passed this link on to the person I was trying this out for yesterday.  But it's good to remind anyone reading this thread about it :) 

[Nathan Dunfield]

This is a known bug and we are working on it (there's even a ticket).  It should be fixed soon.  It's some issue with our configuration script.  

[Nathan Dunfield]

It's bad packaging by upstream: it's running Cython but the Cython
source files are not in the snappy source tarball.

Yes, we ship the Cython generated C/C++ files rather than Cython code itself.  My understanding from the Cython docs is that this is the recommended approach for non-Sage packages, and we have always done this.  But apparently we accidentally introduced a bug in the setup.py script in SnapPy 2.6 that is causing it to look for the Cython files anyway...

Nathan

[Dima Pasechnik]

On Tue, Oct 2, 2018 at 4:44 PM Nathan Dunfield <nat...@dunfield.info> wrote:
>>
>> It's bad packaging by upstream: it's running Cython but the Cython
>> source files are not in the snappy source tarball.
>
>
> Yes, we ship the Cython generated C/C++ files rather than Cython code itself. My understanding from the Cython docs is that this is the recommended approach for non-Sage packages, and we have always done this.

Really? Could you point out the place in the docs that recommends this?
Letting Cython to do necessary adjustments for the target C/C++
compiler seems like the most natural thing to do...

> But apparently we accidentally introduced a bug in the setup.py script in SnapPy 2.6 that is causing it to look for the Cython files anyway...
>
> Nathan
>

[Nils Bruin]

On Tuesday, October 2, 2018 at 8:54:12 AM UTC-7, Dima Pasechnik wrote:

On Tue, Oct 2, 2018 at 4:44 PM Nathan Dunfield <nat...@dunfield.info> wrote:
>>
>> It's bad packaging by upstream: it's running Cython but the Cython
>> source files are not in the snappy source tarball.
>
>
> Yes, we ship the Cython generated C/C++ files rather than Cython code itself.  My understanding from the Cython docs is that this is the recommended approach for non-Sage packages, and we have always done this.

Really? Could you point out the place in the docs that recommends this?
Letting Cython to do necessary adjustments for the target C/C++
compiler seems like the most natural thing to do...

https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#distributing-cython-modules

The idea is that cython produces fully compliant C code, so that no tuning of the generated C code is required. Distributing the cythonized c-files has the advantage that the installing user does not need cython installed. In sage we have run into trouble with that due to API incompatibilities between Py2 and Py3, and it doesn't seem possible to specify the right conditionals in the cython file to produce an appropriate c file with the right conditional pieces of code in.

Note that the cython documentation recommends to distribute the c files AS WELL AS the cython files, and to just not have cythonization as a default build option (I'm not sure it always said that). So it seems cython recommended practice now is to also include the cython code. With that in place, we could just to a cythonizing build in sage (because we'll have cython available for sure).

[Vincent Delecroix]

Le 02/10/2018 à 18:31, Nils Bruin a écrit :
>
>
> On Tuesday, October 2, 2018 at 8:54:12 AM UTC-7, Dima Pasechnik wrote:
>>
>> On Tue, Oct 2, 2018 at 4:44 PM Nathan Dunfield <nat...@dunfield.info
>> <javascript:>> wrote:
>>>>
>>>> It's bad packaging by upstream: it's running Cython but the Cython
>>>> source files are not in the snappy source tarball.
>>>
>>>
>>> Yes, we ship the Cython generated C/C++ files rather than Cython code
>> itself. My understanding from the Cython docs is that this is the
>> recommended approach for non-Sage packages, and we have always done this.
>>
>> Really? Could you point out the place in the docs that recommends this?
>> Letting Cython to do necessary adjustments for the target C/C++
>> compiler seems like the most natural thing to do...
>>
>
> https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#distributing-cython-modules
>
> The idea is that cython produces fully compliant C code, so that no tuning
> of the generated C code is required. Distributing the cythonized c-files
> has the advantage that the installing user does not need cython installed.
> In sage we have run into trouble with that due to API incompatibilities
> between Py2 and Py3, and it doesn't seem possible to specify the right
> conditionals in the cython file to produce an appropriate c file with the
> right conditional pieces of code in.

As far as I remember, snappy deals with that by providing file.py2.c and
file.py3.c. Nathan could infirm/confirm.

> Note that the cython documentation recommends to distribute the c files AS
> WELL AS the cython files, and to just not have cythonization as a default
> build option (I'm not sure it always said that). So it seems cython
> recommended practice now is to also include the cython code. With that in
> place, we could just to a cythonizing build in sage (because we'll have
> cython available for sure).

But what should be the default behavior of pip for a given Python/Sage
package:

(1) do cythonize if Cython is available
(2) do cythonize if asked by the user

>>
>>> But apparently we accidentally introduced a bug in the setup.py script
>> in SnapPy 2.6 that is causing it to look for the Cython files anyway...
>>>
>>> Nathan
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>> Groups "sage-devel" group.
>>> To unsubscribe from this group and stop receiving emails from it, send

>> an email to sage-devel+...@googlegroups.com <javascript:>.

>>> To post to this group, send email to sage-...@googlegroups.com

>> <javascript:>.

[Thierry]

I was thinking to include those in a future version of Sage Debian Live
and the scripts in the computop docker could help me.

However, it could be much nicer to have those packaged directly in Sage so
that the packaging work is done only once for all situations (computop,
SDL, Karl-Dieter, ...).

[Jeroen Demeyer]

On 2018-10-02 17:53, Dima Pasechnik wrote:
> Letting Cython to do necessary adjustments for the target C/C++
> compiler seems like the most natural thing to do...

Cython doesn't need to do any such "necessary adjustments". It's
completely independent of C/C++ compilers (and it should be).

[Dima Pasechnik]

So you need to ship generated C files for C99, C11, and C++ files for
c++11, c++14, c++17, blah++foo, etc etc etc?

[Jeroen Demeyer]

On 2018-10-02 23:01, Dima Pasechnik wrote:
> So you need to ship generated C files for C99, C11, and C++ files for
> c++11, c++14, c++17, blah++foo, etc etc etc?

No. You *do* need to tell Cython the language (C or C++). But beyond
that, it generates code compatible with any version of the C/C++ standards.

[Nathan Dunfield]

Thanks, Thierry.  Sounds like we need a ticket for this.  In particular, I wasn't even able to get sage -pip install mercurial to work on OS X.

I was just helping a colleague install SnapPy into SageMath on OS X and hit the above issue.  I just downloaded the source tarball from BitBucket directly and all of SnapPy built fine, even though Mercurial didn't.  So probably a problem with Mercurial...

Nathan

[Nathan Dunfield]

> The idea is that cython produces fully compliant C code, so that no tuning
> of the generated C code is required. Distributing the cythonized c-files
> has the advantage that the installing user does not need cython installed.
> In sage we have run into trouble with that due to API incompatibilities
> between Py2 and Py3, and it doesn't seem possible to specify the right
> conditionals in the cython file to produce an appropriate c file with the
> right conditional pieces of code in.

As far as I remember, snappy deals with that by providing file.py2.c and
file.py3.c. Nathan could infirm/confirm.

For SnapPy itself, we don't hit this API issue and so there is just a single version of each Cython-generated ".c" file.  For CyPari, I believe Marc Culler did have to use the scheme Vincent mentions above.

Nathan

[Nathan Dunfield]

On Tuesday, October 2, 2018 at 10:40:28 AM UTC-5, Nathan Dunfield wrote:

This is a known bug and we are working on it (there's even a ticket).  It should be fixed soon.  It's some issue with our configuration script.  

FYI, this issue has now been fixed and the corrected version is on PyPI as version 2.6.1.post1.  

 

Nathan