handling PY2/3 issues (like range) with `future`

[Chris Smith]

I was talking with @asmeurer and @jayesh92 this afternoon about removing xrange from the SymPy codebase. There's an interesting dilemma that arises: there is nothing in PY2 that will emulate the new range in PY3. The most crippling example is that while range in PY3 can accept a slice like range(10)[::2], xrange cannot. So in a file where one needs to use an xrange-like range, it can be imported from compatibility as range (and compatibility supplies the xrange function which will no longer be necessary once PY2 is dropped). *But if, in the same file, one slices a range an error will be raised because the imported xrange (under the alias range) cannot be sliced that way:

from sympy.core.compatibility import range

range(10**8)  # ok. We imported range which is really xrange so this works

...

range(10)[::2] --> error since xrange cannot be sliced this way

We can write list(range(10))[::2] to work around the issue but then we are are forcing people to use PY2 idioms to work around PY2 functions (xrange) disguised as PY3 functions (range). That's about as bad as allowing users to use PY2 keywords (like xrange) in a codebase where we are trying to keep things compatible with PY3.

There's some really good news to help in this (and other issues): the "future" project at 

I was talking with @asmeurer and @jayesh92 this afternoon about removing xrange from the SymPy codebase. There's an interesting dilemma that arises: there is nothing in PY2 that will emulate the new range in PY3. The most crippling example is that while range in PY3 can accept a slice like range(10)[::2], xrange cannot. So in a file where one needs to use an xrange-like range, it can be imported from compatibility as range (and compatibility supplies the xrange function which will no longer be necessary once PY2 is dropped). *But if, in the same file, one slices a range an error will be raised because the imported xrange (under the alias range) cannot be sliced that way:

EXAMPLE

========

from sympy.core.compatibility import range

range(10**8)  # ok. We imported range which is really xrange so this works

...

range(10)[::2] --> error since xrange cannot be sliced this way

We can write list(range(10))[::2] to work around the issue but then we are are forcing people to use PY2 idioms to work around PY2 functions (xrange) disguised as PY3 functions (range). That's about as bad as allowing users to use PY2 keywords (like xrange) in a codebase where we are trying to keep things compatible with PY3.

There's some really good news to help in this (and other issues): the "future" project at http://python-future.org/ . They have already workd through the issues related to this (and other PY2/3 incompatibilities) so that in our compatibility file all we would have to do is

if PY3:

    range = range

else:

    from future.builtins import range

And then in a file where we want to use xrange we use the compatibility range instead and now the code in our EXAMPLE above will work -- no workaround is necessary. That's the way it should be. I would recommend that as long as we support PY2 we should include `future` like we included `mpmath` until we drop PY2 support.

[Aaron Meurer]

I don't think it's necessary to depend on future, although we can certainly borrow code from it. But how many places in the code slice a range (and of them, how many of them have to be xrange in Python 2)?

Aaron Meurer

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To post to this group, send email to sy...@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.
To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/0e4afab2-6e47-4544-b985-3e670469ff11%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Chris Smith]

4 places (https://github.com/sympy/sympy/pull/8914) which have been marked with a note telling how they should be modified once PY2 is dropped.

[Aaron Meurer]

I used astsearch to search all combinations and also only found four (which were not sliced, just indexed)

sympymaster%=$astsearch 'range(?)[?]' sympy/

sympy/matrices/dense.py

  77|                    i = range(self.rows)[i]

  83|                    j = range(self.cols)[j]

sympy/matrices/sparse.py

  99|                    i = range(self.rows)[i]

 107|                    j = range(self.cols)[j]

sympymaster%=$astsearch 'range(?)[?:?]' sympy/

sympymaster%=$astsearch 'range(?)[?:?:?]' sympy/

sympymaster%=$astsearch 'range(?)[?::?]' sympy/

sympymaster%=$astsearch 'range(?)[:?:?]' sympy/

sympymaster%=$astsearch 'range(?)[?:?:]' sympy/

sympymaster%=$astsearch 'range(?)[?:]' sympy/

sympymaster%=$astsearch 'range(?)[:?]' sympy/

sympymaster%=$astsearch 'range(?)[:]' sympy/

sympymaster%=$astsearch 'range(?)[::]' sympy/

sympymaster%=$astsearch 'range(?)[?::]' sympy/

sympymaster%=$astsearch 'range(?)[:?:]' sympy/

sympymaster%=$astsearch 'range(?)[::?]' sympy/

What is the point of "i = range(self.rows)[i]"? That's the same as

if not 0 <= i < self.rows:

    raise IndexError

And there's probably a better way to manage the logic than by using IndexError anyway.

Aaron Meurer

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/0a1c98aa-2e7d-4bce-9684-ca8edde6fb91%40googlegroups.com.

[Chris Smith]

astsearch does not tell you that i or j are slices ;-)