The Annual Sage on Python 3 Thread

[R. Andrew Ohana]

But this time with a report!

Broken spkgs:

PIL

setuptools (really distribute)

sympy

pycrypto

pynac

rpy2

pexpect

gdmodule

sqlalchemy

networkx

mpmath

zn_poly

sagenb

sagetex (maybe?)

scons

(almost certainly polybori -- it depends on scons)

and a number of others that use python 2 spkg-install scripts, or need other minor updates.

Of these, setuptools, sympy, rpy2, sqlalchemy, networkx, mpmath, and pycrypto should be fixed with simple updates (see #15510, #15511, and #15512 for the first 3). For the others:

gdmodule, PIL, sqlalchemy -- do we use these anywhere? (I don't see any imports, or even tests of these libraries in our test suite, although I could be missing something) If not, I think would vote that we should change these to optional spkgs; reduce the bloat of the base distribution.

PIL -- If we keep this a standard package, we should switch to Pillow, as upstream PIL is dead.

pynac -- I have a small patch that makes this build against python3, and hopefully this will be sufficient to make it work.

pexpect, zn_poly -- 2to3 seems to be sufficient for these

scons -- there is on going work to port to python3 (seems likely to come with the next release)

polybori -- no clue

This leaves a very short list of what packages we are waiting on:

sagenb

scons

polybori

sagetex?

sage library + scripts

In other words, almost everything upstream is ready for Python 3.

--

Andrew

[Jeroen Demeyer]

I did a quick search for these:

> gdmodule
Only used in sage/matrix/matrix2.pyx and
sage/matrix/matrix_modn_sparse.pyx in the visualize_structure() method.
Should be replaced by a different imaging library.

> PIL
Used in various places: PolyBoRi, Pygments, Sphinx, SciPy

> sqlalchemy
sagenb's SPKG.txt *claims* this is needed, but it is almost certainly
not (did earlier versions use this?). Sphinx and IPython have files
using sqlalchemy, but it doesn't look like a requirement.

[Volker Braun]

On Wednesday, December 11, 2013 10:53:25 AM UTC, R. Andrew Ohana wrote:

and a number of others that use python 2 spkg-install scripts, or need other minor updates.

Since we have a bunch of independent scripts written in Python, how about we start with building python3 next to python2. Then we can transition the scripts to #!/usr/bin/env python3, and make it a policy than any new script has to be py3.

 

[François Bissey]

On Wed, 11 Dec 2013 02:53:25 R. Andrew Ohana wrote:
> But this time with a report!
>
> Broken spkgs:
>
> PIL

We actually now make it possible to use pillow in sage-on-gentoo
one small patch in the sage library is needed.

> setuptools (really distribute)
> sympy

Recent sympy can do py3

> pycrypto
> pynac

Ask Burcin for that.

> rpy2
> pexpect
> gdmodule
> sqlalchemy

newer sqlalchemy support python 3.3. But has mentioned elsewhere
it is not an essential component of sage. I think wstein added it
because he wanted to use the functionality.

> networkx

I believe recent networkx can use py3.

> mpmath

After reading some stuff on the sympy mailing list I think
we should dump mpmath and use the copy from sympy - that is shipped
in sage and that has fix that are not in a mpmath release. Paulo
had a patch to do that when he was working sage-on-mandriva.

> zn_poly
> sagenb
> sagetex (maybe?)
> scons
> (almost certainly polybori -- it depends on scons)

Polybori: ask Alexander Drier

[Volker Braun]

On Wednesday, December 11, 2013 8:37:15 PM UTC, François wrote:

> (almost certainly polybori -- it depends on scons)
Polybori: ask Alexander Drier

 

Not exactly fair if its via the scons dependency ;-)   

http://scons.org says that the current release will be the last one to support python2, so there is hope...

[François Bissey]

I am fairly sure it has python bits (apart from the ipython bindings)
but I could be wrong.

[Dan Drake]

On Wed, 11 Dec 2013 at 02:53AM -0800, R. Andrew Ohana wrote:
> But this time with a report!
>
> Broken spkgs:
>

> sagetex (maybe?)

Why the "maybe"? What happens if you try to use Py3?

In January I might be able to update SageTeX to Python 3. It should be
easy. (Famous last words?)


Dan

--
--- Dan Drake
----- www.math.wisc.edu/~ddrake/
-------

[R. Andrew Ohana]

I've made #15530 to track the progress on supporting python3 (and there is already a good amount there that is up for review).

Also, other than the trivial usage in sage.monoids.string_monoid(_element), I'm not finding any real usage of pycrypto. Am I missing something? If not, I would elect to make it an optional package.

--

Andrew

[Jean-Pierre Flori]

On Wednesday, December 18, 2013 6:24:40 AM UTC+1, R. Andrew Ohana wrote:

I've made #15530 to track the progress on supporting python3 (and there is already a good amount there that is up for review).

Also, other than the trivial usage in sage.monoids.string_monoid(_element), I'm not finding any real usage of pycrypto. Am I missing something? If not, I would elect to make it an optional package.

I think having pycrypto in Sage is cool for ... cryptographers :)
Hint: fix for FreeBSD at #12399 and update to latest upstream at #14854!

[Martin Albrecht]

On Wednesday 25 Dec 2013 09:09:58 Jean-Pierre Flori wrote:
> I think having pycrypto in Sage is cool for ... cryptographers :)

I am not sure this is true. PyCrypto is quite high-level, it gives you RSA,
AES, SHA256 and sutff like that. At such a high level I guess it would be
useful for testing/implementing protocols, but for the kind of crypto where
Sage is probably used most - i.e. actually diving into the algorithms - I
don't think it's that useful.

Put another way: did anybody on this list ever use it? I think I may have once
briefly.

Cheers,
Martin


--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x6532AFB4
_otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
_www: http://martinralbrecht.wordpress.com/
_jab: martinr...@jabber.ccc.de

[Jean-Pierre Flori]

On Wednesday, December 25, 2013 7:58:27 PM UTC+1, Martin Albrecht wrote:

On Wednesday 25 Dec 2013 09:09:58 Jean-Pierre Flori wrote:
> I think having pycrypto in Sage is cool for ... cryptographers :)

I am not sure this is true. PyCrypto is quite high-level, it gives you RSA,
AES, SHA256 and sutff like that. At such a high level I guess it would be
useful for testing/implementing protocols, but for the kind of crypto where
Sage is probably used most - i.e. actually diving into the algorithms - I
don't think it's that useful.

Put another way: did anybody on this list ever use it? I think I may have once
briefly. 

I think it can also be quite useful for teaching as well when you want to jump from more crypto oriented to more math oriented stuff.

[Jean-Pierre Flori]

On a slightly unrelated topic, currently the only function of pycrypto used in the Sage library is indeed in the monoid stuff and that's the byte_to_long function.
Although I don't feel we should remove pycrypto from standard packages, I feel we could implement this ourselves if this prevents a boring warning from breaking tests (see #14854).

[R. Andrew Ohana]

I disagree with the sentiment that we should make python package X a standard package because it is useful for Y. At that rate, every (compatibly licensed) mathematics and science python package will be a standard part of sage. Because pycrypto is not truly a dependency of the sage library (i.e. used in a non-trivial manner), nor is it nicely integrated into to sage (you have to `from pycrypto import foo`), to me it clearly falls into this boat.

--

Andrew

[William Stein]

On Wed, Dec 25, 2013 at 10:58 AM, Martin Albrecht
<martinr...@googlemail.com> wrote:
> On Wednesday 25 Dec 2013 09:09:58 Jean-Pierre Flori wrote:
>> I think having pycrypto in Sage is cool for ... cryptographers :)
>
> I am not sure this is true. PyCrypto is quite high-level, it gives you RSA,
> AES, SHA256 and sutff like that. At such a high level I guess it would be
> useful for testing/implementing protocols, but for the kind of crypto where
> Sage is probably used most - i.e. actually diving into the algorithms - I
> don't think it's that useful.

Even if you implement one of these, you might want to be able to very
easily compare your implementation against one that actually works.

> Put another way: did anybody on this list ever use it? I think I may have once
> briefly.

I have used it dozens of times in a teaching context (for undergrads).

-- William

>
> Cheers,
> Martin
>
>
> --
> name: Martin Albrecht
> _pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x6532AFB4
> _otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF
> _www: http://martinralbrecht.wordpress.com/
> _jab: martinr...@jabber.ccc.de

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[William Stein]

Sage has a fairly clear mission statement: "Mission: Creating a viable
free open source alternative to Magma, Maple, Mathematica and Matlab."

Magma doesn't have crypto included, so far as I can tell:

http://magma.maths.usyd.edu.au/magma/handbook/linear_codes_over_finite_fields

Do any other MA's?

William


>
> --
> Andrew
>
> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/groups/opt_out.

[Jean-Pierre Flori]

On Wednesday, December 25, 2013 8:40:27 PM UTC+1, Jean-Pierre Flori wrote:

On a slightly unrelated topic, currently the only function of pycrypto used in the Sage library is indeed in the monoid stuff and that's the byte_to_long function.
Although I don't feel we should remove pycrypto from standard packages, I feel we could implement this ourselves if this prevents a boring warning from breaking tests (see #14854).

Anyway, I've completely disabled any call to pycrypto in the monoid stuff.
Feel free to review http://trac.sagemath.org/ticket/14854
The topic of removing, making optional or whatever with pycrypto can be dealt independently of that.

[William Stein]

This discussion on Hacker News is relevant to "The Annual Sage on
Python 3 Thread"

https://news.ycombinator.com/item?id=6985207

http://alexgaynor.net/2013/dec/30/about-python-3/

1. Argument against switching now:

One quote from the HN discussion: "My last few Python projects have
started out as Python 3, but ended up as 2 due to missing library
support." The takeaway for me is that even if we could easily just
switch Sage to Python 3, it's not 100% clear that we actually should
do so today. Doing so could alienate users, and restrict the
third-party code and packages that can be installed into Sage. For
example, we might see "I started out using Sage (which uses Python 3),
but had to switch to Numpy/Scipy/etc. and Python 2 due to missing
library support. Anyway, I wanted to point this out loud and clear
in case anybody on the list has been silently worrying about this.

2. Argument for switching now:

On the other hand, the main suggestion in the discussion there for how
to fix this situation is to get all the popular Linux distros to
switch to Python3. Sage is like a linux distro, in that if we switch
it would help move things along. Also, Sage isn't meant to be
mainly a "python platform", but an "alternative to Mathematica/etc."
platform.

To me personally, a killer feature I really like in Python3 is the
massively improved unicode support. I don't personally use it, but I
think it makes Sage much, much more palatable to the international
community, who like to write code like the following, which was sent
to me by Ширшов Андрей in Russia, who just one first prize in a
contest there for applying Sage to ship-building:

#Sage variable names can be only in Latin characters
numbers=[1,2,3.5]
strings=["alpha", "beta", "gamma"]
names=["Зоя", "Яна", "Кит"]

второе_имя="Яна"

print(numbers)
print(strings)
print(names)
if второе_имя in names:
print("Второе имя в списке %s" % второе_имя)


According to http://stackoverflow.com/questions/2649544/unicode-identifiers-in-python
this will never work in Python2 no matter what you do, but it does
work fine with python3. So you can put the above in a file cyr.py,
and type "python3 cyr.py", and it works.

I look at analytics a lot, and the distribution of Sage users is very
world-wide, with a lot from non-native-English countries, and some of
them would really appreciate being able to name their variables using
their own language. In Mathematica (I just checked), there's no
problem at all with unicode identifiers, so far as I can tell. So
from the Sage mission statement -- "create a viable alternative to the
Ma's" -- it seems critical to our goals to switch to Python 3.

William

[R. Andrew Ohana]

I don't think that we are quite there with switching. For starters we need both scons and polybori to be updated to support python 3, but beyond that, the Sage library needs a massive amount of work to make it Python 3 compatible. Moreover, I don't think we have to force the switch, it should be perfectly possible to support both python 2 and python 3 for a period of time.

--

Andrew

[Nils Bruin]

On Monday, December 30, 2013 12:35:37 PM UTC-8, R. Andrew Ohana wrote:

Moreover, I don't think we have to force the switch, it should be perfectly possible to support both python 2 and python 3 for a period of time.

What's the benefit of that? The sage process itself will be running on one python process. We can't be hybrid with that. So either you make the whole sage library Py3-capable or not. We could ship both Py2 and Py3 so that we can run some scripts under Py3 and the sage process under Py2 (or vice versa), but having two pythons seems a lot of admin for very little benefit. I could see a development version that for a while has both to aid debugging adapting the sage library, but outside of that I don't see a benefit of having both.

[William Stein]

The benefit is that we could switch to python3 by default; however, if
somebody runs into a show-stopper issue, we can tell them: "heh, just
do 'sage --py2' to start your copy of Sage up using python2 instead".
It directly addresses the main argument that I made against switching
above. It sounds technically difficult, but it is certainly possible.

Alternatively, we could have a build flag that determines whether
everything gets built with py3 or py2.

It all sounds like a lot of extra pain and work, but it is possible
and addresses the main argument against switching.

Your main argument against this is "for very little benefit" --
however, given that only 2% of download from PyPi are for py3, and in
light of the discussions I linked to, evidence suggests there's
potentially substantial real benefit to continuing to support the
python2 ecosystem, at least for now.

-- William

[R. Andrew Ohana]

On Mon, Dec 30, 2013 at 1:04 PM, William Stein <wst...@gmail.com> wrote:

On Mon, Dec 30, 2013 at 12:47 PM, Nils Bruin <nbr...@sfu.ca> wrote:
> On Monday, December 30, 2013 12:35:37 PM UTC-8, R. Andrew Ohana wrote:
>>
>> Moreover, I don't think we have to force the switch, it should be
>> perfectly possible to support both python 2 and python 3 for a period of
>> time.
>
>
> What's the benefit of that? The sage process itself will be running on one
> python process. We can't be hybrid with that. So either you make the whole
> sage library Py3-capable or not. We could ship both Py2 and Py3 so that we
> can run some scripts under Py3 and the sage process under Py2 (or vice
> versa), but having two pythons seems a lot of admin for very little benefit.
> I could see a development version that for a while has both to aid debugging
> adapting the sage library, but outside of that I don't see a benefit of
> having both.

The benefit is that we could switch to python3 by default; however, if
somebody runs into a show-stopper issue, we can tell them: "heh, just
do 'sage --py2' to start your copy of Sage up using python2 instead".
 It directly addresses the main argument that I made against switching
above.  It sounds technically difficult, but it is certainly possible.

This would certainly be very nice, but

Alternatively, we could have a build flag that determines whether
everything gets built with py3 or py2.

this would probably be much easier.

It all sounds like a lot of extra pain and work, but it is possible
and addresses the main argument against switching.

I don't think it is that much work to maintain, as for most python code there isn't that large of a difference between python 2 and 3. There will of course be some trickier bits, but how we deal with most of these issues should occur during the initial porting process.
 

--

Andrew