Dropping support for 2.7 in 2020

[Stéfan van der Walt]

Hi everyone

Python 2.7 support has been extended to 2020.  Would you agree that we can drop 2.7 support after that date as well?

Stéfan

[Egor Panfilov]

Hi Stefan!

Although I'm a 3.5 user for quite a long time, many people from industry who I'm familiar with are still using 2.7, both for R&D and deplyoment.

Let's see how thing will change. Intuitively, it's too early to get rid of 2.7 in 2020.

--
You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
To post to this group, send email to scikit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/CABDkGQmJyoCWrBu%3DpV41V2-dJD0wp6mrvrqHuaYCDeYmXYeW2A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Stéfan van der Walt]

Note that end of life for Python 2.x is actually now-ish, and that support is being offered exactly so that industry and others have time to switch over. It does not mean that folks should aim at using 2.7 until 2020!

Maintaining support for an old version of Python carries a maintenance burden that I'd like to avoid if possible.

Also, once our dependencies such as numpy, scipy or matplotlib make the move, we can theoretically keep supporting 2.x for a while longer, but not *much* longer.

Folks in industry often run on platforms such as Anaconda, where this will probably be a non issue by then. I'm more concerned about big clusters, University labs, etc. But even there we are now able to provide both virtual environments or Conda envs much more easily than before.

Stéfan

To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/CAP0v5tn%2BojdyKOrzs8LS8_YhRHTCv1ar7zudZAVLBaLkAJaCGw%40mail.gmail.com.

[Johannes Schönberger]

Hi Stefan,

Even though official support for 2.7 will be dropped on the Python side, my feeling is that it will stick around in official linux repositories for quite some time longer. Especially on clusters etc., as you mentioned. I would suggest to drop 2.7 support as soon as Numpy/Scipy drops it - this is an upper bound as we depend on it and, at the same time, numpy/scipy are important enough libraries that should also convince the last developer to make the switch to 3.x.

Anyway, until 2020 is a long time :-)

Cheers, Johannes

> To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/CABDkGQm3bvXF45AEo1fZ8aWUQNuESaYgBRcndrzWr9OQYsE%2BjA%40mail.gmail.com.

[Stéfan van der Walt]

Hi Johannes

On Tue, 17 May 2016 at 23:26 Johannes Schönberger <js...@demuc.de> wrote:

Even though official support for 2.7 will be dropped on the Python side, my feeling is that it will stick around in official linux repositories for quite some time longer. Especially on clusters etc., as you mentioned. I would suggest to drop 2.7 support as soon as Numpy/Scipy drops it - this is an upper bound as we depend on it and, at the same time, numpy/scipy are important enough libraries that should also convince the last developer to make the switch to 3.x.

It is true that Python 2.7 will stick around beyond 2020, but it will not receive bug-fixes any longer.  scikit-image 0.18 (or whatever :) will also stick around along with 2.7, and we can keep supporting that release.  But for any new releases beyond that point it makes little sense to support 2.7.

Stéfan

[Emmanuelle Gouillart]

At the moment, can you estimate the cost of supporting Python 2.7, in
terms of additional code? (I would say that testing 2.7 in CI is not a
true cost, since it represents server time and not human time)

Best,
Emma

[Stéfan van der Walt]

On Tue, 17 May 2016 at 23:47 Emmanuelle Gouillart <emmanuelle...@nsup.org> wrote:

At the moment, can you estimate the cost of supporting Python 2.7, in
terms of additional code? (I would say that testing 2.7 in CI is not a
true cost, since it represents server time and not human time)

 

It's probably not terribly much, but it does prohibit us from making use of newer language features as they develop.

I should emphasize that I think it's perfectly reasonable to keep supporting an older release of scikit-image for 2.7, just not the latest release.

Stéfan

[Emmanuelle Gouillart]

> It's probably not terribly much, but it does prohibit us from making use of
> newer language features as they develop.

Any pointer about such new features so that ignorant people like myself
can educate themselves :-D?

> I should emphasize that I think it's perfectly reasonable to keep
> supporting an older release of scikit-image for 2.7, just not the
> latest release.

Of course. It's possible that in 2020 we'll have reached a "stationary
state" as NumPy, and that new releases will mostly be about bug fixes and
improved documentation. But we're still on a fast growing curve, so it's
hard to know.

[Stéfan van der Walt]

On Wed, 18 May 2016 at 00:40 Emmanuelle Gouillart <emmanuelle...@nsup.org> wrote:

> It's probably not terribly much, but it does prohibit us from making use of
> newer language features as they develop.

Any pointer about such new features so that ignorant people like myself
can educate themselves :-D?

Here are a few, compliments of Aaron Muerer:

http://asmeurer.github.io/python3-presentation/python3-presentation.pdf 

> I should emphasize that I think it's perfectly reasonable to keep
> supporting an older release of scikit-image for 2.7, just not the
> latest release.

Of course. It's possible that in 2020 we'll have reached a "stationary
state" as NumPy, and that new releases will mostly be about bug fixes and
improved documentation. But we're still on a fast growing curve, so it's
hard to know.

:D  Feature complete!  That will be a day for celebration.

Stéfan 

[Himanshu Mishra]

Just a pointer : Ubuntu 16.04 has officially removed Python 2 from the distribution. So yes, *people will be* moving to Python 3 now.

--

You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
To post to this group, send email to scikit...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/CABDkGQm_SE8fjU1ZiUg1kWne1eLTicjvtV14Bhp1x8fOupOQNA%40mail.gmail.com.

[Thomas Caswell]

The proposed release schedule for mpl is being discussed at https://mail.python.org/pipermail/matplotlib-devel/2016-May/000374.html

The key point is that 'dropping python2' only means 'for new releases', all of the existing packaged code will continue to work.  How often do users want to install cutting edge skimage on top of system python2?

Tom

To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/CANpJW-URfDAKKzmbeLQz5%3DksxjJ8QbFCU2Hdm4bedZ2zx1AAAw%40mail.gmail.com.

[Emmanuelle Gouillart]

On Wed, May 18, 2016 at 05:42:23PM +0000, Thomas Caswell wrote:
> The proposed release schedule for mpl is being discussed at https://
> mail.python.org/pipermail/matplotlib-devel/2016-May/000374.html

> The key point is that 'dropping python2' only means 'for new releases', all of
> the existing packaged code will continue to work.  How often do users want to
> install cutting edge skimage on top of system python2?

It happened to me yesterday :-). When I wanted to install 0.12 on a
server at the synchrotron, where they have powerful stations really
useful for my processing pipeline, but they are running Debian Wheezy and
don't have ipython or matplotlib for python3. Also on such machines, I
only have ssh access, no http/s, so no pip install... Such situations are
not so unusual in the industry / at large facilities.

Cheers,
Emma

[Nelle Varoquaux]

On 18 May 2016 at 19:54, Emmanuelle Gouillart

I think that still happens to a lot of people. In the two institutes I
currently work at, one has recently moved from python 2.3 to python
2.6, and the other one from python 2.6 to python 2.7. In both cases,
it took around 8 years to have an OS upgrade, so I wouldn't hold my
breath for a new one any time soon.

Cheers,
N

>
> Cheers,
> Emma

>
> --
> You received this message because you are subscribed to the Google Groups "scikit-image" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.

> To post to this group, send an email to scikit...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/20160518175455.GA4080630%40phare.normalesup.org.

[Emmanuelle Gouillart]

Just to be clear: I very much hope that we'll have lots of great releases
until 2020 so that users bound to legacy 2.7 will be happy with a
not-so-up-to-date release. It's just that I trust us more to produce
these releases than big institutions to make their systems evolve!

[François Boulogne]

> It's just that I trust us more to produce
> these releases than big institutions to make their systems evolve!

:)


To add a little piece to the thoughts shared on this question, we may
wonder if there was or there is or there will be soon a feature to
implement that would cause a lot of pain to make it work with python 2.7.

We may also wonder if for those users who need to use old systems, will
it be possible to install scikitimage if we constantly increase the
minimal version of our dependencies (cython etc). If they are not also
available on their system, it means that in practice, they will need to
stuck on an older version of scikit-image anyway.

I never learned python2, but I don't know the nice features of python3
only because I can't contribute with them. That's sad.


--
François Boulogne.
http://www.sciunto.org
GPG: 32D5F22F

[Nathaniel Smith]

FYI, if you can copy files to the server at all, then this is easy to solve -- just download and copy over this one file,

  http://repo.continuum.io/archive/Anaconda3-4.0.0-Linux-x86_64.sh

run it, and bam, complete up-to-date python scientific stack, no need to bother the sysadmins.

I actually recommend this in general for everyone trying to do work on weird old institutional OSes, regardless of whether they need python 3 in particular -- life is too short to waste time fiddling with some ancient version of matplotlib or buggy scipy or whatever when you have real work to get done. Stable distributions are great for providing a stable baseline for the background tools that don't really matter, but if you're a scientific programmer then your python environment is too important; you need more control than that.

-n

[Emmanuelle Gouillart]

> FYI, if you can copy files to the server at all, then this is easy to solve --
> just download and copy over this one file,

>   http://repo.continuum.io/archive/Anaconda3-4.0.0-Linux-x86_64.sh

> run it, and bam, complete up-to-date python scientific stack, no need to bother
> the sysadmins.

> I actually recommend this in general for everyone trying to do work on weird
> old institutional OSes, regardless of whether they need python 3 in particular
> -- life is too short to waste time fiddling with some ancient version of
> matplotlib or buggy scipy or whatever when you have real work to get done.
> Stable distributions are great for providing a stable baseline for the
> background tools that don't really matter, but if you're a scientific
> programmer then your python environment is too important; you need more control
> than that.

Thanks for the tip; indeed I almost resorted to this solution, but
tweaking a little bit scikit-image's setup.py and requirements.txt was
faster :-) (if not cleaner).

It's very good to have Anaconda in such cases, as a last resort solution.
The downside is that we lose the modularity of the scientific Python
ecosystem, having to use a whole distribution in a very monolithic way. I
used to think that Anaconda was mostly for these Windows people :-)...

Cheers,
Emma

[Stuart Mumford]

Hi,

> It's very good to have Anaconda in such cases, as a last resort solution.
> The downside is that we lose the modularity of the scientific Python
> ecosystem, having to use a whole distribution in a very monolithic way.

There is always miniconda which only installs Python and conda, which
means you can build up your own environments. Somewhat off topic, but we
have a nice centralised installation of miniconda on our cluster at
Sheffield.
(http://docs.iceberg.shef.ac.uk/en/latest/software/apps/python.html)

On topic, I am all for not releasing new (non-bug fix) versions of
libraries with Python 2.7 support in or even before 2020.

Stuart

[Emmanuelle Gouillart]

On Fri, May 20, 2016 at 01:15:33PM +0100, Stuart Mumford wrote:
> Hi,
> > It's very good to have Anaconda in such cases, as a last resort solution.
> > The downside is that we lose the modularity of the scientific Python
> > ecosystem, having to use a whole distribution in a very monolithic way.

> There is always miniconda which only installs Python and conda, which
> means you can build up your own environments. Somewhat off topic, but we
> have a nice centralised installation of miniconda on our cluster at
> Sheffield.
> (http://docs.iceberg.shef.ac.uk/en/latest/software/apps/python.html)

Thanks, I should give it a try.

[Michael Sarahan]

There is also Constructor, which helps you create your own miniconda-like installers with only the packages you actually want: https://github.com/conda/constructor

It may save you some steps with installing miniconda, then installing other packages.

I don't think this is a concern for cython or scikit-image, but many people at bumping into the language support limit in the C++11 sense with Python 2.7 on Windows.  Since VS 2008 is the de-facto standard compiler for Python 2.7, people are unable to use C++11 code in modules for Python 2.7.  Some people use newer compilers anyway, which sometimes works, but is mixing runtimes, and can lead to bugs or crashes.  Many people would like to support Python 2.7 using a different compiler for the whole ecosystem.  One example is Ilastik, by folks at HHMI, using VS 2012 to have a custom stack: https://github.com/ilastik/ilastik-build-conda

Perhaps not central to this discussion, but something to be aware of.

--
You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.

To post to this group, send an email to scikit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/20160520141800.GD3583487%40phare.normalesup.org.

[Nathaniel Smith]

On May 20, 2016 07:30, "Michael Sarahan" <msar...@gmail.com> wrote:
> I don't think this is a concern for cython or scikit-image, but many people at bumping into the language support limit in the C++11 sense with Python 2.7 on Windows.  Since VS 2008 is the de-facto standard compiler for Python 2.7, people are unable to use C++11 code in modules for Python 2.7.  Some people use newer compilers anyway, which sometimes works, but is mixing runtimes, and can lead to bugs or crashes.  Many people would like to support Python 2.7 using a different compiler for the whole ecosystem.  One example is Ilastik, by folks at HHMI, using VS 2012 to have a custom stack: https://github.com/ilastik/ilastik-build-conda

I think getting mingwpy finished and polished is probably an easier solution for this problem than forking the entire py27-on-windows ecosystem :-)

-n

[Juan Nunez-Iglesias]

I would drop 2.7 sooner. NumPy/SciPy dropping it is the absolute cutoff, but there’s no reason why we can’t jump the gun (and lead rather than follow). Py2.7 users can make do with older releases. As Stéfan mentioned, this is not about erasing Py2.7 support, but not releasing new features on Py2.7. Very different. I envision that we stop producing Py2.7 releases after 0.13, but we call that an “LTS” release which will get bugfix backports until 2020.

imho, with Py3.4 and especially Py3.5, Py3-only suddenly became very attractive. My three favourite features are type annotations, keyword-only arguments (these two together make it much easier to produce correct code and debug), and the @ matrix multiplication operator. The latter makes linear algebra code *so* *much* nicer to read and write, and we have our fair share in scikit-image. For my own projects, I am now Py3.5-only, always.

Finally, as others have mentioned, but is worth restating:

- Conda allows user-space installs of Python versions and packages, so you don’t ever depend on your sysadmin-controlled environment.

- It is *absolutely false* that conda packages are only useful on Windows — I have failed to compile scipy both on OSX and Ubuntu boxes.

- It is also *not* a requirement to download the monolithic Anaconda distro — with miniconda (which should be the default), you just install precisely what you need.

- With conda-forge we now have a community-run repository of binary conda packages.

So, in short, the switch to Py3 is easier than ever for *all* users, some might just not realise it yet ;), and switching to a Py3-only programming environment is a boon to all the scikit-image devs… though again some might not realise it yet ;).

Juan.

--

You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.

To post to this group, send email to scikit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/CAPJVwB%3DjRQtHn9Xj0xwv3hxtDsjBvKYtJTB38ewBuZ84cNR4xw%40mail.gmail.com.

[Thomas Kluyver]

I'd like to revive this discussion with a bit of added context. We (principally people from IPython/Jupyter, Matplotlib, Sympy and Scikit-bio) are putting together a statement for Scientific Python community projects to signal that we're not planning to maintain Python 2 support forever. We're saying that we will end Python 2 support in or before 2020, to correspond with the end of support for Python 2.7 itself.

http://python3statement.github.io/

The more projects that get on board with this, the better we can make the case that researchers and users need Python 3 to be available. Then we can all benefit from a reduced maintenance burden. So scikit-image wouldn't be making a lone stand on this - there are a number of projects already agreeing to drop Python 2 support by 2020, and we hope there will be more soon.

Thanks,
Thomas

[François Boulogne]

Le 04/07/2016 à 11:38, Thomas Kluyver a écrit :
> I'd like to revive this discussion with a bit of added context. We
> (principally people from IPython/Jupyter, Matplotlib, Sympy and
> Scikit-bio) are putting together a statement for Scientific Python
> community projects to signal that we're not planning to maintain
> Python 2 support forever. We're saying that we will end Python 2
> support in or before 2020, to correspond with the end of support for
> Python 2.7 itself.
>
> http://python3statement.github.io/
>
> The more projects that get on board with this, the better we can make
> the case that researchers and users need Python 3 to be available.
> Then we can all benefit from a reduced maintenance burden. So
> scikit-image wouldn't be making a lone stand on this - there are a
> number of projects already agreeing to drop Python 2 support by 2020,
> and we hope there will be more soon.
>

Thanks for your note.

Does it mean that all new version of those libraries will support
python2.7 until 2020 or just that you can release a bugfix to already
released version supporting 2.7?

If it's the first option, it means we can't use python3 features before
2020 and we will be "in late" with these features.

[Thomas Kluyver]

On 4 July 2016 at 10:47, François Boulogne <fbou...@sciunto.org> wrote:

Does it mean that all new version of those libraries will support
python2.7 until 2020 or just that you can release a bugfix to already
released version supporting 2.7?

If it's the first option, it means we can't use python3 features before
2020 and we will be "in late" with these features.

The statement doesn't commit projects to support 2.7 until any specific date - scikit-bio already went Python 3 only with its release last month, for instance. We're agreeing to end Python 2 support by 2020, but projects are welcome to do it earlier than that.

For IPython, we're planning for the 6.0 feature release (some time in 2017) to be Python 3 only, while we'll make bugfix releases of the 5.x series for longer than normal to support users on Python 2.

[Josh Warner]

I think we should support 2.7 through end-of-life. 2020 sounds right.

There are a tremendous amount of systems out there still running 2.7. I know more than one lab which standardized on 2.7 years ago, and while we're evaluating the move to 3.x it won't be without major pains. Small, niche packages which work perfectly under 2.7 break in 3.x, and they may or may not be maintained. It only takes a couple of these to render lab equipment with price tags in the 6 figures buggy, problematic, or useless in 3.x. This is problematic when you want a consistent Python experience across a lab or team, but certain pieces just won't work in 3.x. 

Sounding the trumpets is appropriate, as 2.7 won't be around forever. Quarterly or bi-annually it would be reasonable to send out a reminder. Three years sounds like a long time, but that's about right for this kind of transition.

[Juan Nunez-Iglesias]

For what it's worth, I strongly support the statement, and would prefer to drop 2.7 support much earlier. This does *not* mean ignoring people submitting 2.7 bug reports. It means that, like Python itself, *new* versions would not support Py2.7. Bug fixes would be ported to e.g. the 0.13.x branch. This in no way interferes people with systems that "just work" and who understandably don't want to risk breaking them.

Juan.

--

You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
To post to this group, send email to scikit...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/dc739979-e63f-4ce0-849b-66f8a659c8fc%40googlegroups.com.