doctests: non-sorted output

[Daniel Krenn]

A patchbot reported two failing doctests because the result is a set and
the elements were in a different order, see
https://trac.sagemath.org/ticket/23637
I somehow remember that such an issue was discussed here and I thought
it was about the output of dict and set being sorted by a display hook.
Am I wrong and this is not done or even not possible?

Best

Daniel

[Maarten Derickx]

The output of both dict and set are ordered by a display hook from IPython, the problem is that this sorting is done on the basis of __cmp__ and the default way of sorting using __cmp__ is using the id of an object which is not deterministic, so even though the output is sorted it is done on a nondeterministic basis for objects which have this nondeterministic __cmp__.

I see basically 2 options going forward:

1. Just fix the doctests of pynormaliz themselves by making them deterministic. 

2. Make the doctesting of sets more robust by ordering the output basted on the string representation instead of __cmp__ . This has two ways of doing it.

2.1 Only change the order of the output in the doctesting framework

2.2 Change it both in the doctesting framework and the user interface

At https://trac.sagemath.org/ticket/23586 there is a partial solution using 2, I stopped working on this because I want feedback on wether we should do 2.1 or 2.2. But the progress on this ticket is slow.

So I propose doing the quick fix 1 independently of 2, because this will remove a lot of false negatives of the patchbot on other tickets helping greatly in the sage development.

Do people have feedback on 2.1 vs 2.2? 

Main advantage of 2.1, the user will get more meaningful output in the case where __cmp__ is deterministic. For example with 2.2 the user will get the following awkward output:

    sage: set(GF(11)) 

    {0, 1, 10, 2, 3, 4, 5, 6, 7, 8, 9}

The main advantage of 2.2 is that the doctest and the actual output of sage are guaranteed to agree and that the code paths for doctesting and actual output are more similar.

My personal preference is for 2.1 since sets are inherently unordered, but I am open for feedback.

[Maarten Derickx]

Option 2.3 might be to make solution 2.1 optional and dependent on the presence of something like  # unorderedset in the doctest. I actually like this the best since it allows us to use it only in the cases where it is really needed. So we can have the advantages of 2.1 and 2.2 depending on the situation!

[David Roe]

Given the transition to Python 3, I don't think we should directly compare the elements in a set (though perhaps an exception should be made if they all have the same parent).  Comparing their string representations seems better.

If we do sort the string representation of sets, we can reduce surprise for users by using something like natural sort (see https://github.com/SethMMorton/natsort for a library, or various code snippets available online).

As for whether to do this only for doctests or for the user interface as well, I think it's nicer for the two to be the same.  So I would prefer 2.2 over 2.1.

David

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscribe@googlegroups.com.
To post to this group, send email to sage-...@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.

[Maarten Derickx]

Thanks for your input David. We discussed it on trac, and decided to leave the situation as is on python 2.7 because it is stable enough as is given the pynormaliz fix.

For python 3 we propese to fix the sorting in upstream ipython. Using str based sort only if the standard sorting raises an error because things are not comparable. This strategy would be like 2.2 both for doctests and for end user printing.

[David Roe]

On Sat, Sep 9, 2017 at 11:15 AM, Maarten Derickx <m.derick...@gmail.com> wrote:

Thanks for your input David. We discussed it on trac, and decided to leave the situation as is on python 2.7 because it is stable enough as is given the pynormaliz fix.

For python 3 we propese to fix the sorting in upstream ipython. Using str based sort only if the standard sorting raises an error because things are not comparable. This strategy would be like 2.2 both for doctests and for end user printing.

That seems reasonable to me.

David