Why Python3
Terry Reedy
specifically, of the changes that break Python2 code. I attempt here to
give a relatively full explanation.
SUMMARY: Python3 completes (or makes progress in) several transitions
begun in Python2.
In particular, Python3 bunches together several feature removals (which
always break someone's code) and a few feature changes (which also break
code). The alternative would have been to make the same changes, a few
at a time, over several releases, starting with about 2.5.
Another alternative would have been to declare 2.0 or 2.1 complete at
far as it went and forbid adding new features that duplicate and
supersede existing features.
Another would have been to add but never remove anthing, with the
consequence that Python would become increasingly difficult to learn and
the interpreter increasingly difficult to maintain with volunteers. I
think 2.7 is far enough in that direction.
SPECIFIC REPLACEMENTS:
1. Integer division
In Python1, arithmetic expressions are mostly generic (polymophic) in
that they have the same meaning for all builtin number types. The
glaring exception is division, which has a special meaning for pairs of
integers. Newbies were constantly surprised that 1/2 equals 0.
Guido proposed to fix the situation with a second division operator
'//', with a standard warning/deprecation/removal process for the old
behavior over three releases, perhaps ending with 2.5. In response to
complaints about having to find and examine every use of '/', Guido
promised that there would be a helper program. I proposed that the
version that had only the new behavior, without 'from __future__ import
division', be named 3.0 to signal the change.
Guido obviously decide to make a lot more changes in a 3.0 release. And
the helper program was expanded to the current 2to3. In any case,
Python3 finished the integer division transition.
2. User classes
In Python1, user classes, defined with a class statement, are instances
of a class type and are otherwise separate from and cannot inherit from
builtin types. Instances of user classes are actually instances of an
instance type, not of the user class.
Python2.2 introduced a unified class-type system. Instead of 'from
__future__ import newclass', the new system was invoked by defining
__metaclass__ or inheriting from a future class. Python3 finished this
transition by dropping ClassType and InstanceType and making the
built-in class 'object' the default baseclass for all user classes.
3. User-defined exceptions
In Python1, user usually defined exceptions by using strings as
exceptions, with the caveat that it was the identity and not the value
of the string that defined the exception. The ability to subclass
built-in exceptions made this obsolete. String exceptions were
discouraged in 2.5 and removed in 3.0, completing another transition.
Now all exceptions are instances of subclasses of BaseException.
4. Function application
Suppose one has a function f, object sequence args, and name-object
mapping kwds, and one wants to call f with the *contents* of args and
kwds as arguments. Calling f(args,kwds) does not do that as it would
pass the collections themselves as a pair of arguments. In Python1, one
called apply(f, args, kwds). Python2 introduced the synonym syntac
f(*args,**kwds) and deprecated 'apply' by 2.5. Python3 removed apply,
completing the transition.
5. Date interchange
In Python1, lists are the primary data interchange type used by built-in
functions such as filter, map, and range. Python2.2 introduced a new
iterator/iterable protocal, the iterator class 'generator', and
generator functions. One advantage is that iterables and iterators can
compactly represent virtual sequences of indefinite length. Another is
that non-sequence collections, like sets and dicts, used be made
iterable. Python2.3 introduced iterator versions of filter map, and zip
in the itertools module. It also defined the new built-in function
enumerate as returning an iterator rather than a list, as would have
been the case if introduced much earlier. 2.4 introduce 'reversed' as
returning an iterator. (The new in 2.4 'sorted' returns a list because
it has to contruct one anyway.)
Pyhon3 moved much closer to replacing lists with iterators as the
primary data interchange format. Ifilter, imap, and izip replaced
filter, map, and zip. Xrange, which preceded 2.2, replaced range.
Getting lists, as previously, is easy with 'list()'. The transition is
not complete (and may never be), as slicing still returns a subsequence
rather than an iterator, so itertools still has islice.
6. Integers
Python1 had two separate integer types, int and long. I believe that
integer operations overflowed if the answer was too large. At some
point, the two types were partially unified in that ints were converted
to long when necessary to avoid overflow. At this point, having two
types, with 'L' appended or not on output, became something of a
nuisance. Python3 finishes int-long unification.
7. Order comparisonS
In early Python1, I believe all objects could be (arbitrarily) compared
and sorted. When Guido added the complex type, he decided not to add an
arbitrary order, as he thought that could mask bugs. I believe other
classes were added later that only allowed comparisons between their own
instances. Python3 completed the transition from comparable by default
to incomparable by default.
More controversially, it also completed a transition from __cmp__ to the
full comparison set. Similarly, list.sort dropped the 'cmp' parameter
after gaining the 'key' parameter.
8. Text
Python1 had a byte string type that doubled as a text string type. (Some
would put this the other way.) Python2 introduced a second text type,
unicode, but kept using bytes as the default, as in its namespace dicts.
Python3 replaced bytes with unicode, including in its namespaces,
thereby making Python3 a more univeresally useful language. It kept
bytes both for generic binary data use and for specialized encoded text
use. This part of the transition is not complete, especially for some of
the internet interfacing libraries.
I think that covers the main transitions in core Python.
--
Terry Jan Reedy
MRAB
> Some people appear to not understand the purpose of Python3 or more
> specifically, of the changes that break Python2 code. I attempt here to
> give a relatively full explanation.
>
> SUMMARY: Python3 completes (or makes progress in) several transitions
> begun in Python2.
>
> In particular, Python3 bunches together several feature removals (which
> always break someone's code) and a few feature changes (which also break
> code). The alternative would have been to make the same changes, a few
> at a time, over several releases, starting with about 2.5.
>
> Another alternative would have been to declare 2.0 or 2.1 complete at
> far as it went and forbid adding new features that duplicate and
> supersede existing features.
>
> Another would have been to add but never remove anthing, with the
> consequence that Python would become increasingly difficult to learn and
> the interpreter increasingly difficult to maintain with volunteers. I
> think 2.7 is far enough in that direction.
>
It's clear that Guido's time machine is limited in how far it can travel
in time, because if it wasn't then Python 1 would've been more like
Python 3 and the changes would not have been necessary! :-)
Stephen Hansen
> Terry Reedy wrote:
>> Another would have been to add but never remove anthing, with the
>> consequence that Python would become increasingly difficult to learn
>> and the interpreter increasingly difficult to maintain with
>> volunteers. I think 2.7 is far enough in that direction.
>>
> [snip]
> It's clear that Guido's time machine is limited in how far it can travel
> in time, because if it wasn't then Python 1 would've been more like
> Python 3 and the changes would not have been necessary! :-)
I'm pretty sure he wrote the Time Machine in Python 1.4, or maybe 1.3?
Either way, its well established that a time machine can't go back in
time any farther then the moment its created.
I don't at all remember why, don't even vaguely understand the physics
behind it, but Morgan Freeman said it on TV, so its true.
So he couldn't go back and fix 1.0, physics won't allow him. So we're
stuck with the Py3k break. :)
--
... Stephen Hansen
... Also: Ixokai
... Mail: me+list/python (AT) ixokai (DOT) io
... Blog: http://meh.ixokai.io/
MRAB
> On 6/27/10 6:09 PM, MRAB wrote:
>> Terry Reedy wrote:
>>> Another would have been to add but never remove anthing, with the
>>> consequence that Python would become increasingly difficult to learn
>>> and the interpreter increasingly difficult to maintain with
>>> volunteers. I think 2.7 is far enough in that direction.
>>>
>> [snip]
>> It's clear that Guido's time machine is limited in how far it can travel
>> in time, because if it wasn't then Python 1 would've been more like
>> Python 3 and the changes would not have been necessary! :-)
>
> I'm pretty sure he wrote the Time Machine in Python 1.4, or maybe 1.3?
> Either way, its well established that a time machine can't go back in
> time any farther then the moment its created.
>
> I don't at all remember why, don't even vaguely understand the physics
> behind it, but Morgan Freeman said it on TV, so its true.
>
>>> import wormhole
Unfortunately it's not part of the standard library. :-(
eric dexter
planned obselence.. It would be nice if a pause was taken at 3.5 and
a huge number of libraries were made available for 3.5..
Benjamin Kaplan
You mean as opposed to a 2-year pause at 3.1 so that a huge number of
libraries and alternate Python implementations could catch up?
http://www.python.org/dev/peps/pep-3003/
geremy condra
> Stephen Hansen wrote:
>>
>> On 6/27/10 6:09 PM, MRAB wrote:
>>>
>>> Terry Reedy wrote:
>>>>
>>>> Another would have been to add but never remove anthing, with the
>>>> consequence that Python would become increasingly difficult to learn
>>>> and the interpreter increasingly difficult to maintain with
>>>> volunteers. I think 2.7 is far enough in that direction.
>>>>
>>> [snip]
>>> It's clear that Guido's time machine is limited in how far it can travel
>>> in time, because if it wasn't then Python 1 would've been more like
>>> Python 3 and the changes would not have been necessary! :-)
>>
>> I'm pretty sure he wrote the Time Machine in Python 1.4, or maybe 1.3?
>> Either way, its well established that a time machine can't go back in time
>> any farther then the moment its created.
>>
>> I don't at all remember why, don't even vaguely understand the physics
>> behind it, but Morgan Freeman said it on TV, so its true.
>>
> That's if the time machines uses a wormhole:
>
>>>> import wormhole
>
> Unfortunately it's not part of the standard library. :-(
Batteries- but not flux capacitors- included.
Geremy Condra
Steven D'Aprano
> 7. Order comparisonS
>
> In early Python1, I believe all objects could be (arbitrarily) compared
> and sorted. When Guido added the complex type, he decided not to add an
> arbitrary order, as he thought that could mask bugs.
I should point out that this wasn't a mere whimsy on Guido's part.
Mathematically, supporting larger-than and less-than comparisons on
complex numbers *is* a bug -- they're simply meaningless mathematically.
(Which is greater, 2-1i or -1+2i?)
What Python needs[1] is a "sorting" operator, which is allowed to return
a consistent if arbitrary sort order (perhaps lexicographic sort order?),
separate from the ordinary > and < operators. This would allow the caller
to sort lists of arbitrary items for display purposes, without implying
anything about the relative size of items.
[1] For some definition of "need".
--
Steven
Carl Banks
> I think that covers the main transitions in core Python.
Nice post, but it's missing one thing.
The main benefit of Python 3 for Joe Q. Scripter is this:
The Python team doesn't have to spend any effort on maintaining a lot
of old obsolete cruft, and can devote that time to fixing bugs and
maintaining useful stuff instead.
So they will benefit even if they don't actually use new features.
Carl Banks
John Nagle
Unfortunately, that's not what's happening in the development
pipeline. PyPy targets Python 2.5. Unladen Swallow targets Python
2.6.1. IronPython targets Python 2.6. C module support for
CPython 3.x is still very spotty. We have a long way to go
before Python 3.x is ready for prime time.
Yes, it's a better language, but it's just not there yet.
John Nagle
Steven D'Aprano
> Unfortunately, that's not what's happening in the development
> pipeline. PyPy targets Python 2.5. Unladen Swallow targets Python
> 2.6.1. IronPython targets Python 2.6. C module support for CPython 3.x
> is still very spotty. We have a long way to go before Python 3.x is
> ready for prime time.
None of PyPy, Unladen Swallow or IronPython are dependencies for Python
3.x to be "ready for prime time". Neither is C module support.
Python 3.1 itself is solid, reliable release of the Python language. It
and the standard library are more than ready to be put into production.
Of course, if you personally require some C module "ham" which only
supports 2.6, you'll have good reason to stick with 2.6. But then if your
project absolutely depends on module "spam" which only supports Python
1.5, you'll be still using Python 1.5. So what?
For the rest of us, you can do a lot with just Python 3.1, with or
without C modules. Whether it does *enough* to be considered for
deployment depends on what you're deploying it to do. I for one would not
hesitate to use Python 3.1 as a scripting language, or for any
application where the standard library is all you need. You can do a lot
with just the standard library.
For the rest, the question isn't "is Python 3 ready for production?",
because the answer for that is "absolutely". The question is, "are the
libraries I need ready for Python 3?", and the answer to that is often
No, but sometimes a provisional or experimental Yes.
Personally, I'm getting tired of all the negative nellies who seem to
think that take up of Python 3 is a race, and that if anyone is still
using 2.x by next Tuesday that means Python 3 is a failure and we should
all just dump it as a bad idea. Python 3 uptake is not a race. Both
Python 2.7 and 3.x will be supported for many years to come. If you can't
use 3 *now*, that's fine, nobody says you should -- but by the same
token, try to tone down the negativity.
--
Steven
OKB (not okblacke)
> None of PyPy, Unladen Swallow or IronPython are dependencies for
> Python 3.x to be "ready for prime time". Neither is C module
> support.
I think this is being overoptimistic. For me, "ready for prime
time" means "I can rely on being able to find a way to do what I want to
do with it." This includes being able to find third-party libraries
that do what I want to do. Right now, I can't really rely on Python 3
in this way.
<snip>
> For the rest of us, you can do a lot with just Python 3.1, with or
> without C modules. Whether it does *enough* to be considered for
> deployment depends on what you're deploying it to do. I for one
> would not hesitate to use Python 3.1 as a scripting language, or
> for any application where the standard library is all you need. You
> can do a lot with just the standard library.
The thing is that, for me at least, this isn't sufficient, because
I often don't know what all I'm going to need when I start off. I may
decide to add some new feature that requires an extra library, and
only then find out that I can't, because that library doesn't exist
for Python 3. Some things are part of the standard lib, some aren't. I
want to be able to start a project and be able to find what I need,
whether that's part of the standard lib or not.
--
--OKB (not okblacke)
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is
no path, and leave a trail."
--author unknown
Terry Reedy
> Unfortunately, that's not what's happening in the development
> pipeline.
Please do some research before posting year-old news as current news.
> Unladen Swallow targets Python 2.6.1.
It used 2.6 for development because that was the current stable release
when they started. They are targeting 3.3 for possible integration with
CPython.
http://www.python.org/dev/peps/pep-3146/
Note that this is accepted, at least in principle.
> IronPython targets Python 2.6.
They plan to release a 2.7 version sometime this year after CPython2.7
is released. They plan to release a 3.2 version early next year, soon
after CPython. They should be able to do that because they already have
a 3.1 version mostly done (but will not release it as such) and 3.2 has
no new syntax, so the 3.1 development version will easily morph into a
3.2 release version. I forget just where I read this, but here is a
public article.
http://www.itworld.com/development/104506/python-3-and-ironpython
Cameron Laird, Python/IronPython developer '''
As Jimmy Schementi, a Program Manager with Microsoft, e-mailed me last
week, "IronPython's roadmap over the next year includes compatibility
with Python 3. Also, we're planning on a release ... before our first
3.2-compatible release which will target 2.7 compatibility."
'''
> PyPy targets Python 2.5.
I believe that wherever I read about IronPython, I also read about PyPy
people planning to do a 3.2 compatible PyPy about a year after it comes
out. But this seems more tentative and I cannot verify.
> CPython 3.x is still very spotty.
I have no idea what you mean here. 3.2 will have all the bugfixes in 2.7
that are relevant.
--
Terry Jan Reedy
Martin v. Loewis
> Mathematically, supporting larger-than and less-than comparisons on
> complex numbers *is* a bug -- they're simply meaningless mathematically.
> (Which is greater, 2-1i or -1+2i?)
However, that's true for many other values that *where* ordered in 2.x.
Which is greater, (1,2) or [1,2]? It's meaningless mathematically.
Likewise (if you claim that comparing lists and tuples is like comparing
apples and oranges) - how should these be ordered:
file("/etc/passwd"), file("/etc/group"), and sys.stdin?
> What Python needs[1] is a "sorting" operator, which is allowed to return
> a consistent if arbitrary sort order (perhaps lexicographic sort order?),
> separate from the ordinary > and < operators. This would allow the caller
> to sort lists of arbitrary items for display purposes, without implying
> anything about the relative size of items.
And indeed, that's available, by means of the key= argument to list.sort.
Regards,
Martin
Paul Rubin
> And indeed, that's available, by means of the key= argument to list.sort.
Unfortunately what's needed for more generality is the ability to supply
a comparison function, which Python2 also offers, but Python3 removes.
I gave an example a while back of wanting to compare two tree
structures, and Raymond H. explained how to do it with just a key
function, which seemed ok at the time. But thinking about it further
afterwards, I believe both of us missed then that the method suggested
doesn't always work, so you really do need cmp. I'll see if I can find
the old post and reconstruct the problem.
Martin v. Loewis
> "Martin v. Loewis" <mar...@v.loewis.de> writes:
>> And indeed, that's available, by means of the key= argument to list.sort.
>
> Unfortunately what's needed for more generality is the ability to supply
> a comparison function, which Python2 also offers, but Python3 removes.
That isn't really a problem. You can readily have one:
http://code.activestate.com/recipes/576653-convert-a-cmp-function-to-a-key-function/
> I gave an example a while back of wanting to compare two tree
> structures, and Raymond H. explained how to do it with just a key
> function, which seemed ok at the time. But thinking about it further
> afterwards, I believe both of us missed then that the method suggested
> doesn't always work, so you really do need cmp. I'll see if I can find
> the old post and reconstruct the problem.
If you remember, don't forget to post it to the recipe, so that you can
find it more easily the next time.
Regards,
Martin
Dino Viehland
> > IronPython targets Python 2.6.
>
> They plan to release a 2.7 version sometime this year after CPython2.7
> is released. They plan to release a 3.2 version early next year, soon
> after CPython. They should be able to do that because they already have
> a 3.1 version mostly done (but will not release it as such) and 3.2 has
> no new syntax, so the 3.1 development version will easily morph into a
> 3.2 release version. I forget just where I read this, but here is a
> public article.
> http://www.itworld.com/development/104506/python-3-and-ironpython
> Cameron Laird, Python/IronPython developer '''
> As Jimmy Schementi, a Program Manager with Microsoft, e-mailed me last
> week, "IronPython's roadmap over the next year includes compatibility
> with Python 3. Also, we're planning on a release ... before our first
> 3.2-compatible release which will target 2.7 compatibility."
Close but not 100% correct - we do plan to release 2.7 sometime this year
but 3.2 is going to be sometime next year, not early, I would guess EOY.
I guess Jimmy misspoke a little there but the "2.7 this year 3.2 next year"
plan is what I said during my PyCon State of IronPython talk and it hasn't
changed yet.
Also we have only a few 3.x features implemented (enabled w/ a -X:Python30
option since 2.6) instead of having a different build for 3.x. Running
with that option isn't likely to run any real 3.x code though but it gives
people a chance to test out a few new features. Of course implementing 2.7
also gets us much closer to 3.x then we are today w/ all its backports so
we are certainly making progress.
John Yeung
<brenNOSPAMb...@NObrenSPAMbarn.net> wrote:
> Steven D'Aprano wrote:
> > For the rest of us, you can do a lot with just Python 3.1,
> > with or without C modules. Whether it does *enough* to be
> > considered for deployment depends on what you're deploying
> > it to do. I for one would not hesitate to use Python 3.1
> > as a scripting language, or for any application where the
> > standard library is all you need. You can do a lot with
> > just the standard library.
>
> The thing is that, for me at least, this isn't
> sufficient, because I often don't know what all I'm going
> to need when I start off. I may decide to add some new
> feature that requires an extra library, and only then find
> out that I can't, because that library doesn't exist for
> Python 3. Some things are part of the standard lib, some
> aren't. I want to be able to start a project and be able
> to find what I need, whether that's part of the standard
> lib or not.
Ah, but what version of Python has a package for everything that you
will need, including the things you haven't thought of? What happens
when you want to provide a feature that requires a library that
doesn't exist for 2.6? (Or 2.5 or whatever it is that you feel has
the most complete coverage.)
My point is simply that you have not said anything that goes against
any of Steven's points.
John