A little Python 3 status update

[Erik Bray]

Just achieved a result I've been trying to get to pretty much since
diving into helping with the Python 3 port of Sage:

$ ./sage -t src/sage/doctest/
too many failed tests, not using stored timings
Running doctests with ID 2017-11-29-16-09-22-b19471aa.
Git branch: u/embray/python3/doctest-fixes
Using --optional=mpir,python2,sage
Doctesting 11 files.
sage -t src/sage/doctest/util.py
[141 tests, 1.30 s]
sage -t src/sage/doctest/external.py
[38 tests, 1.62 s]
sage -t src/sage/doctest/__init__.py
[0 tests, 0.00 s]
sage -t src/sage/doctest/fixtures.py
[59 tests, 1.18 s]
sage -t src/sage/doctest/sources.py
[366 tests, 2.16 s]
sage -t src/sage/doctest/all.py
[0 tests, 0.00 s]
sage -t src/sage/doctest/reporting.py
[111 tests, 1.41 s]
sage -t src/sage/doctest/parsing.py
[271 tests, 2.39 s]
sage -t src/sage/doctest/control.py
[198 tests, 18.71 s]
sage -t src/sage/doctest/forker.py
[450 tests, 106.73 s]
sage -t src/sage/doctest/test.py
[23 tests, 51.69 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 246.0 seconds
cpu time: 24.2 seconds
cumulative wall time: 187.2 seconds


This required a number of (mostly little) fixes to the doctest
framework itself, miscellaneous support frameworks, and a large chunk
of the rest of Sage (since the test suite for sage.doctest itself runs
the tests for a few modules as part of its self-test). I haven't made
tickets for all these fixes yet either, so you won't be able to
reproduce this outside my private branch just yet.

But having a test runner whose results can be somewhat relied on
should allow for rapid progress on the rest of the work needed for
Python 3 support, since running the test suites for individual modules
and/or entire packages will make it easier to pinpoint where work is
needed. I'll keep working to organize my existing fixes into a
reasonable set of patches so that the doctest runner is working for
everyone. Then perhaps we can organize a little Python 3 bug bash.

Best,
Erik

[Jeroen Demeyer]

Do tracebacks work? That is obviously important to debug stuff.

I notice from the bug report on https://trac.sagemath.org/ticket/24221
that there is something wrong with traceback formatting.

[Erik Bray]

I'm not exactly sure how you mean. I've had tracebacks working,
seemingly normally, for a while now. I don't know if I personally did
anything toward that or not--I don't specifically recall encountering
a problem with it (though I have fixed a few issues in the sageinspect
module which could have something to do with it...?).

[Frédéric Chapoton]

Great ! I am impatient that the 24 positive-reviewed tickets for python3 get closed, so that we can move forward again safely.

[Dima Pasechnik]

by the way, is there a python3 meta-ticket? I can't find it.

[Frédéric Chapoton]

Vanilla sage is now building and starting with python3. But mostly not yet working smoothly.

Next logical steps would be : make so that (1) documentation builds (2) doctest framework works

Concerning meta-tickets, you can look at

https://trac.sagemath.org/query?status=!closed&component=python3

and

https://trac.sagemath.org/ticket/15530

Frederic

[François Bissey]

Well I should add that the current vbraun branch where Volker merges tickets
before releasing betas is starting with python3. There is still a lot of work
around unicode to do before either building documentation or running doctests.

François

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/d/optout.

[Frédéric Chapoton]

No need for Volker branch. Sage 8.2.b8 itself starts when built with python3.

[Erik Bray]

On Tue, Mar 13, 2018 at 2:45 AM, Dima Pasechnik <dim...@gmail.com> wrote:
> by the way, is there a python3 meta-ticket? I can't find it.

There are a couple very old ones that aren't entirely relevant anyore.
A starting point is

https://trac.sagemath.org/ticket/15530

I started doing some work last week to reorganize things into
something coherent and cohesive, but I got side-tracked before I could
really complete that work. In any case IMO the best "meta-ticket" is
just the list of tickets under the python3 component:

https://trac.sagemath.org/query?status=!closed&component=python3

Most of these are small fixes that are mostly independent of each other.

Some of the meta-tickets do list known issues that don't necessarily
have tickets yet, such as the division ticket:

https://trac.sagemath.org/ticket/15995

[Erik Bray]

On Tue, Mar 13, 2018 at 8:48 AM, François Bissey <frp.b...@gmail.com> wrote:
> Well I should add that the current vbraun branch where Volker merges tickets
> before releasing betas is starting with python3. There is still a lot of work
> around unicode to do before either building documentation or running doctests.

To be clear, I've had the doctest framework working almost perfectly
in my personal Python 3 branch for *months* now. Most of the work
needed for that has tickets for it but has been slow to get accepted.

[Frédéric Chapoton]

Could you give us the precise list of tickets that are needed to make the doctest framework work ?

[Erik Bray]

On Tue, Mar 13, 2018 at 2:42 PM, Frédéric Chapoton <fchap...@gmail.com> wrote:
> Could you give us the precise list of tickets that are needed to make the
> doctest framework work ?

Let me go through and make sure the exact fixes that are needed all
have tickets now. I believe they should but I'm not 100% certain.
I'll go through the existing tickets and apply them to 8.1beta8 and
see.

I should also mention that most of the tickets I do know of are nearly
complete by now, at least.

[Erik Bray]

On Tue, Mar 13, 2018 at 2:46 PM, Erik Bray <erik....@gmail.com> wrote:
> On Tue, Mar 13, 2018 at 2:42 PM, Frédéric Chapoton <fchap...@gmail.com> wrote:
>> Could you give us the precise list of tickets that are needed to make the
>> doctest framework work ?
>
> Let me go through and make sure the exact fixes that are needed all
> have tickets now. I believe they should but I'm not 100% certain.
> I'll go through the existing tickets and apply them to 8.1beta8 and
> see.
>
> I should also mention that most of the tickets I do know of are nearly
> complete by now, at least.

From my experiments so far it seems that these are the only two
tickets (both already positive review) that need to be merged for the
doctest framework to basically work:

https://trac.sagemath.org/ticket/24343
https://trac.sagemath.org/ticket/24922

Most tests will still *fail* (although a good many of them are fixed
by other open tickets in the python3 component).

[Frédéric Chapoton]

Thanks Erik. So we are getting close indeed to be able ot use the doctest framework ! And what about doc-building (not so close to be working, I am afraid ) ?

[Erik Bray]

On Tue, Mar 13, 2018 at 3:40 PM, Frédéric Chapoton <fchap...@gmail.com> wrote:
> Thanks Erik. So we are getting close indeed to be able ot use the doctest
> framework ! And what about doc-building (not so close to be working, I am
> afraid ) ?

I know I had doc builds mostly working at one point too, but there
were some modules that were broken badly-enough at import time that it
would cause an error during the doc build. But there's not much about
the docbuild framework itself that needs fixing for python 3 AFAICT.

[Frédéric Chapoton]

Thanks. I think for the doc we need at least

https://trac.sagemath.org/ticket/24312

and

https://trac.sagemath.org/ticket/24936

But this does not seem to be enough..