Adding a trivial docstring confuses testing
Rob Beezer
that if I add a trivial docstring (empty, one-line, other variants of
trivial) to the first function in the file (which is "have_dvipng")
then a test run on that file really goes haywire. Take out the
trivial docstring and all tests pass.
It would appear that the value of the boolean constant EMBEDDED_MODE
might be affected, as much of the erroneous doctest output gets
wrapped in HTML tags, when the expected output is not in this format.
I haven't put much work into trying to isolate the problem (other than
removing/adding the additional docstring repeatedly - that sound you
hear is me banging my head on the wall). I'm wondering if this sounds
familiar to anybody, or if there is output I could post that would be
helpful in diagnosing the problem.
Thanks,
Rob
William Stein
I've never heard of anything like the above.
You could post your file.
William
Rob Beezer
This is the fubared version of sage/misc/latex.py (which I am in the
middle of working on). It contains one doctest that I've edited
heavily, so some changes seem to be allowed OK. I'm running sage -t
on it within my development tree, which has been upgraded progessively
to 3.4.1.rc2.
When I take out the docstring on the first function, it passes all
tests.
Thanks,
Rob
John H Palmieri
innocuous changes near the top screws up the doctests for
latex_variable_name at the bottom, acting like EMBEDDED_MODE is True.
Are those the doctests which fail for you? I couldn't figure out why,
though.
John
John H Palmieri
adding a docstring to have_png -- just something trivial, as Rob says,
like
"""
docstring
docstring
"""
then doctests suddenly fail for that file.
John
John H Palmieri
I figured out how to fix the problem, although I still don't know why
adding a docstring should cause it.
To fix it: the docstring for "print_or_typeset" contains the lines
sage: sys.displayhook = sage.misc.latex.pretty_print
sage: sage.misc.latex.print_or_typeset(3)
<html><span class="math">3</span></html>
sage: sage.misc.latex.EMBEDDED_MODE=False
You need to add a line to the beginning and a line to the end, making
it:
sage: TEMP = sys.displayhook
sage: sys.displayhook = sage.misc.latex.pretty_print
sage: sage.misc.latex.print_or_typeset(3)
<html><span class="math">\newcommand{\Bold}[1]{\mathbf{#1}}3</
span></html>
sage: sage.misc.latex.EMBEDDED_MODE=False
sage: sys.displayhook = TEMP
This is included in the patch for <http://trac.sagemath.org/sage_trac/
ticket/5610>, although that patch includes lots of other things, too.
As part of that patch, I added a few doctests to latex.py, but I don't
have time to get anywhere close to 100%.
John
Rob Beezer
Thanks for the fix! That behavior was driving me crazy. ;-) I've
been away for a few days, so I'm sorry I didn't reply to your other
messages as you diagnosed this.
I'll get back to this soon and I'll keep an eye out for any
explanations, in addition to looking at #5610. I've only written a
few new doctests for this file, but have spent enough time with the
code that I might be able to add some more, more quickly.
While on the topic, is there a need for the global "_have_dvipng"
variable in this file? it only seems to be used within the have_dvipng
() function, and nowhere else. I was thinking of removing it.
Rob
John H Palmieri
> Hi John,
>
> Thanks for the fix! That behavior was driving me crazy. ;-) I've
> been away for a few days, so I'm sorry I didn't reply to your other
> messages as you diagnosed this.
>
> I'll get back to this soon and I'll keep an eye out for any
> explanations, in addition to looking at #5610. I've only written a
> few new doctests for this file, but have spent enough time with the
> code that I might be able to add some more, more quickly.
>
> While on the topic, is there a need for the global "_have_dvipng"
> variable in this file? it only seems to be used within the have_dvipng
> () function, and nowhere else. I was thinking of removing it.
think this description is accurate:
"""
Return True if this computer has the program dvipng.
The first time it is run, this function caches its result in the
variable ``_have_dvipng``, and any subsequence time, it just
checks the value of the variable.
"""
So I don't think the variable can be discarded.
John
Carl Witty
<jhpalm...@gmail.com> wrote:
> I figured out how to fix the problem, although I still don't know why
> adding a docstring should cause it.
>
> To fix it: the docstring for "print_or_typeset" contains the lines
Well, I can tell you why adding a docstring will cause the problem.
The Sage doctest system works by translating files with Sage-style
doctests into files with Python-style doctests and then running the
Python doctest system. (You can see the translated file at
$SAGE_ROOT/tmp/.doctest_latex.py .)
During this translation, the doctests get named example_1 through
example_NN (example_12 in our case). print_or_typeset ends up being
example_9.
Then the Python doctest system sorts the names ALPHABETICALLY, and
runs them in this order:
example_1
example_10
example_11
example_12
example_2
...
example_9
Adding a new doctest to a function near the beginning of the file
bumps print_or_typeset from example_9 (where it is the last doctest to
be run, so it doesn't affect any other doctests) to example_10 (where
it is the second doctest to be run, and affects almost all the other
doctests).
Probably there should be a way to randomize the doctest order, so we
can verify that it doesn't matter what order they are run in. I'll
add that to my list of small Sage projects...
Carl
Rob Beezer
Thanks for the explanation - good to know just why this was
happening. I'd noticed the tests being run in a different order as I
tried to debug this, but hadn't dug deep enough to discover the cause.
In misc/latex.py how the latex is created often depends on the value
of the boolean EMBEDDED_MODE, typically wrapping the latex in HTML
tags, or not. So then it must be critical to assume a default value
of this switch and restore that state at the end of example that
exercises the alternate behavior. Maybe there are other places where
this situation could arise?
Thinking as I write - a randomized order for tests might cause errors
like this to not occur during the review process, but then surface
spuriously during testing after releases? In the long-run it would
lead to more bullet-proof tests (long-term gain, better tests), but
maybe inject a confusing element of randomness in initial testing
(short-term pain, more heartburn for the release manager)?
An alternative would be to name the examples with names like
"example_0006" so the alphabetical sorting is the same as the order of
the tests in the file. Just thoughts - I don't have enough experience
to say which makes more sense.
Thanks again for explaining the mystery.
Rob
William Stein
I posted a patch so that
(1) doctests are ran in the same order as the file
(2) doctests can be run in random order
(3) doctests can be run in random order specified by a seed
Carl, maybe you can referee it:
http://trac.sagemath.org/sage_trac/ticket/5816
William
John H Palmieri
John
William Stein
Oops. I've posted it now.
William
William Stein
>> Where's the patch?
>>
>> John
>>
>
> Oops. I've posted it now.
>
> William
>
It's very interesting to run on the full rc3 tree with a fixed random
seed. I think this
reveals *numerous* errors and subtle problems:
./sage -tp 20 -long -rand=1 devel/sage/sage/ > testlong-rand1.log&
This already turns up the following *interesting* problems:
sage -t -long -rand=1 devel/sage/sage/modular/modform/space.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/modular/modform/space.py",
line 763:
sage: M.q_expansion_basis()
Expected:
[
q - 2*q^2 - q^3 + 2*q^4 + q^5 + O(q^6),
1 + 12/5*q + 36/5*q^2 + 48/5*q^3 + 84/5*q^4 + 72/5*q^5 + O(q^6)
]
Got:
[
q - 2*q^2 - q^3 + 2*q^4 + q^5 + 2*q^6 - 2*q^7 - 2*q^9 - 2*q^10 +
q^11 - 2*q^12 + 4*q^13 + O(q^14),
1 + 12/5*q + 36/5*q^2 + 48/5*q^3 + 84/5*q^4 + 72/5*q^5 + 144/5*q^6
+ 96/5*q^7 + 36*q^8 + 156/5*q^9 + 216/5*q^10 + 12/5*q^11 + 336/5*q^12
+ 168/5*q^13 + O(q^14)
]
**********************************************************************
1 items had failures:
1 of 5 in __main__.example_3297184748
sage -t -long -rand=1 devel/sage/sage/misc/latex.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/misc/latex.py",
line 62:
sage: latex([1,2,3])
Expected:
\left[1,
[[everything breaks as you know]]
sage -t -long -rand=1 devel/sage/sage/libs/pari/gen.pyx
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/libs/pari/gen.pyx",
line 8280:
sage: a = pari('1.2'); a, a.type(), a.precision()
Expected:
(1.20000000000000, 't_REAL', 3)
Got:
(1.20000000000000, 't_REAL', 4)
**********************************************************************
1 items had failures:
1 of 18 in __main__.example_0036674092
sage -t -long -rand=1 devel/sage/sage/misc/randstate.pyx
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/misc/randstate.pyx",
line 59:
: rtest()
Expected:
(207, 0.505364206568040, 4*x^2 + 1/2, (1,2)(4,5), [ 0, 0, 1, 0, 1
], 2137873234, 27695, 0.19982565117278328)
Got:
(207, 0.505364206568040, 4*x^2 + 1/2, (2,3)(4,5), [ 0, 0, 1, 0, 1
], 2137873234, 27695, 0.19982565117278328)
**********************************************************************
1 items had failures:
1 of 62 in __main__.example_0299976341
***Test Failed*** 1 failures.
The following might be related to http://trac.sagemath.org/sage_trac/ticket/5789
sage -t -long -rand=1 devel/sage/sage/rings/real_rqdf.pyx
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/rings/real_rqdf.pyx",
line 1116:
sage: RQDF(-1)._complex_double_(CDF)
Expected:
-1.0
Got:
doctest:1172: DeprecationWarning: RQDF is deprecated; use
RealField(212) instead.
-1.0
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/rings/real_rqdf.pyx",
line 4:
: RQDF(1)
Expected:
doctest:...: DeprecationWarning: RQDF is deprecated; use
RealField(212) instead.
1.000000000000000000000000000000000000000000000000000000000000000
Got:
1.000000000000000000000000000000000000000000000000000000000000000
**********************************************************************
2 items had failures:
1 of 4 in __main__.example_0007003101
1 of 12 in __main__.example_0299976341
sage -t -long -rand=1 devel/sage/sage/libs/fplll/fplll.pyx
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/libs/fplll/fplll.pyx",
line 715:
sage: A = gen_ntrulike(5,10,12); A
Expected:
[ 1 0 0 0 0 320 351 920 714 66]
[ 0 1 0 0 0 351 920 714 66 320]
[ 0 0 1 0 0 920 714 66 320 351]
[ 0 0 0 1 0 714 66 320 351 920]
[ 0 0 0 0 1 66 320 351 920 714]
[ 0 0 0 0 0 12 0 0 0 0]
[ 0 0 0 0 0 0 12 0 0 0]
[ 0 0 0 0 0 0 0 12 0 0]
[ 0 0 0 0 0 0 0 0 12 0]
[ 0 0 0 0 0 0 0 0 0 12]
Got:
[ 1 0 0 0 0 116 331 303 963 456]
[ 0 1 0 0 0 331 303 963 456 116]
[ 0 0 1 0 0 303 963 456 116 331]
[ 0 0 0 1 0 963 456 116 331 303]
[ 0 0 0 0 1 456 116 331 303 963]
[ 0 0 0 0 0 12 0 0 0 0]
[ 0 0 0 0 0 0 12 0 0 0]
[ 0 0 0 0 0 0 0 12 0 0]
[ 0 0 0 0 0 0 0 0 12 0]
[ 0 0 0 0 0 0 0 0 0 12]
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/libs/fplll/fplll.pyx",
line 656:
sage: A = gen_uniform(10,10,12); A
[[tons more problems]]
age -t -long -rand=1
devel/sage/sage/rings/polynomial/polynomial_modn_dense_ntl.pyx
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/rings/polynomial/polynomial_modn_dense_ntl.pyx",
line 379:
sage: f.small_roots()
Expected:
[4]
Got:
verbose 2 (<module>) epsilon = 0
verbose 2 (<module>) m = 3
verbose 2 (<module>) t = 0
verbose 2 (<module>) X = 4
verbose 1 (<module>) LLL of 9x12 matrix (algorithm fpLLL:wrapper)
verbose 1 (<module>) LLL finished (time = 0.0)
[4]
**********************************************************************
1 items had failures:
1 of 7 in __main__.example_0208607704
sage -t -long -rand=1 devel/sage/sage/rings/polynomial/polynomial_element.pyx
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/rings/polynomial/polynomial_element.pyx",
line 3991:
sage: p.roots(ring=RR, algorithm='numpy')
Expected:
Traceback (most recent call last):
...
ValueError: array must not contain infs or NaNs
Got:
Traceback (most recent call last):
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/ncadoctest.py",
line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/sagedoctest.py",
line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example,
filename, compileflags)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/ncadoctest.py",
line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_0061063177[118]>", line 1, in <module>
p.roots(ring=RR, algorithm='numpy')###line 3991:
sage: p.roots(ring=RR, algorithm='numpy')
File "polynomial_element.pyx", line 4235, in
sage.rings.polynomial.polynomial_element.Polynomial.roots
(sage/rings/polynomial/polynomial_element.c:34787)
return self.change_ring(L).roots(multiplicities=multiplicities,
algorithm=algorithm)
File "polynomial_element.pyx", line 4156, in
sage.rings.polynomial.polynomial_element.Polynomial.roots
(sage/rings/polynomial/polynomial_element.c:33064)
raise
File "polynomial_element.pyx", line 4146, in
sage.rings.polynomial.polynomial_element.roots
(sage/rings/polynomial/polynomial_element.c:32862)
ext_rts1 = numpy.roots(numpy_array)
File "/home/wstein/build/sage-3.4.1.rc3/local/lib/python2.5/site-packages/numpy/lib/polynomial.py",
line 180, in roots
roots = _eigvals(A)
File "/home/wstein/build/sage-3.4.1.rc3/local/lib/python2.5/site-packages/numpy/lib/polynomial.py",
line 35, in _eigvals
return eigvals(arg)
File "/home/wstein/build/sage-3.4.1.rc3/local/lib/python2.5/site-packages/numpy/linalg/linalg.py",
line 597, in eigvals
_assertFinite(a)
File "/home/wstein/build/sage-3.4.1.rc3/local/lib/python2.5/site-packages/numpy/linalg/linalg.py",
line 125, in _assertFinite
raise LinAlgError, "Array must not contain infs or NaNs"
LinAlgError: Array must not contain infs or NaNs
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/rings/polynomial/polynomial_element.pyx",
line 4002:
sage: p.roots(ring=RR)
Expected:
[(0.000000000000000, 1)]
Got:
[(-0.000000000000000, 1)]
sage -t -long -rand=1 devel/sage/sage/combinat/sloane_functions.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/combinat/sloane_functions.py",
line 7203:
sage: initial = len(sloane.A001358._b)
Exception raised:
Traceback (most recent call last):
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/ncadoctest.py",
line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/sagedoctest.py",
line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example,
filename, compileflags)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/ncadoctest.py",
line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_0246131326[2]>", line 1, in <module>
initial = len(sloane.A001358._b)###line 7203:
sage: initial = len(sloane.A001358._b)
AttributeError: 'A001358' object has no attribute '_b'
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/combinat/sloane_functions.py",
line 7205:
sage: len(sloane.A001358._b) - initial > 0
Exception raised:
Traceback (most recent call last):
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/ncadoctest.py",
line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/sagedoctest.py",
line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example,
filename, compileflags)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/ncadoctest.py",
line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_0246131326[4]>", line 1, in <module>
len(sloane.A001358._b) - initial > Integer(0)###line 7205:
sage: len(sloane.A001358._b) - initial > 0
NameError: name 'initial' is not defined
**********************************************************************
1 items had failures:
sage -t -long -rand=1 devel/sage/sage/combinat/words/word.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/combinat/words/word.py",
line 119:
sage: w
Expected:
word: 2,1,3,12
Got:
word: 2-1-3-12
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/combinat/words/word.py",
line 837:
sage: Words(range(1000))([0,1,10,101]).__str__()
Expected:
'word: 0,1,10,101'
Got:
'word: 0-1-10-101'
**********************************************************************
2 items had failures:
1 of 10 in __main__.example_1192818567
1 of 5 in __main__.example_1949818771
***Test Failed*** 2 failures.
sage -t -long -rand=1 devel/sage/sage/calculus/equations.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/calculus/equations.py",
line 895:
sage: bool( x^2 > x )
Expected:
False
Got:
True
**********************************************************************
1 items had failures:
1 of 13 in __main__.example_3038663466
***Test Failed*** 1 failures.
sage -t -long -rand=1 devel/sage/sage/interfaces/r.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/interfaces/r.py",
line 1071:
sage: t = a.trait_names()
Expected nothing
Got:
<BLANKLINE>
Building R command completion list (this takes
a few seconds only the first time you do it).
To force rebuild later, delete /scratch/wstein/sage//r_commandlist.sobj.
**********************************************************************
1 items had failures:
1 of 5 in __main__.example_1346836858
sage -t -long -rand=1
devel/sage/sage/interfaces/singular.py**********************************************************************File
"/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/interfaces/singular.py",
line 988: sage: singular.trait_names()Expected: ['headStand',
... 'stdfglm']Got: ['pause', 'writelist', 'tab', 'split',
'showrecursive', 'show', 'rMacaulay', 'pmat', 'lprint', 'allprint',
'newtonDiag', 'subrInterred', 'id2mod', 'mod2id', 'denominator',
'numerator', 'content', 'lcm', 'rad_con', 'normalize', 'mindeg1',
'mindeg', 'maxdeg1', 'maxdeg', 'maxcoef', 'is_zero', 'freerank',
'kat_var', 'katsura', 'cyclic', 'substitute', 'hilbPoly', 'select1',
'select', 'sat', 'nselect', 'elim1', 'elim', 'blowup0', 'triangMH',
'triangM', 'triangLfak', 'triangL', 'absFactorize', 'newZero_decom
[[tons of other issues due to bad doctests]]
sage -t -long -rand=1 devel/sage/sage/interfaces/maxima.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/interfaces/maxima.py",
line 1872:
sage: f.diff('x')
Expected:
34*y*'diff(y,x,1)+2*x
Got:
2*x
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/interfaces/maxima.py",
line 1602:
sage: f = maxima('1/(x-1)^3'); f
Expected:
1/(x-1)^3
Got:
1
And *NUMEROUS* other issues!!
sage -t -long -rand=1 devel/sage/sage/combinat/partition.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/combinat/partition.py",
line 2372:
sage: RestrictedPartitions(5,[3,2,1]).__repr__()
Expected:
doctest:...: DeprecationWarning: RestrictedPartitions is
deprecated; use Partitions with the parts_in keyword instead.
'Partitions of 5 restricted to the values [1, 2, 3] '
Got:
doctest:1: DeprecationWarning: RestrictedPartitions is deprecated;
use Partitions with the parts_in keyword instead.
doctest:2324: DeprecationWarning: RestrictedPartitions_nsk is
deprecated; use Partitions with the parts_in keyword instead.
'Partitions of 5 restricted to the values [1, 2, 3] '
[[and more]]
sage -t -long -rand=1 devel/sage/sage/calculus/calculus.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/calculus/calculus.py",
line 8211:
sage: latex(factorial(x))
Exception raised:
Traceback (most recent call last):
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/ncadoctest.py",
line 1231, in run_one_test
self.run_one_example(test, example, filename, compileflags)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/sagedoctest.py",
line 38, in run_one_example
OrigDocTestRunner.run_one_example(self, test, example,
filename, compileflags)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/bin/ncadoctest.py",
line 1172, in run_one_example
compileflags, 1) in test.globs
File "<doctest __main__.example_2333006911[2]>", line 1, in <module>
latex(factorial(x))###line 8211:
sage: latex(factorial(x))
File "/scratch/wstein/build/sage-3.4.1.rc3/local/lib/python2.5/site-packages/sage/calculus/calculus.py",
line 8249, in __call__
return factorial(n, **kwds)
File "/scratch/wstein/build/sage-3.4.1.rc3/local/lib/python2.5/site-packages/sage/rings/arith.py",
line 328, in factorial
raise ValueError, "factorial -- must be nonnegative"
ValueError: factorial -- must be nonnegative
(not much more)
sage -t -long -rand=1 devel/sage/sage/server/notebook/worksheet.py
**********************************************************************
File "/scratch/wstein/build/sage-3.4.1.rc3/devel/sage-main/sage/server/notebook/worksheet.py",
line 147:
sage: sage.server.notebook.worksheet._a_sage
Expected nothing
Got:
Sage
**********************************************************************
1 items had failures:
1 of 5 in __main__.example_3577862625
The following tests failed:
sage -t -long -rand=1 devel/sage/sage/modular/modform/space.py
# 1 doctests failed
sage -t -long -rand=1 devel/sage/sage/misc/session.pyx # 1
doctests failed
sage -t -long -rand=1 devel/sage/sage/misc/latex.py # 17 doctests failed
sage -t -long -rand=1 devel/sage/sage/libs/pari/gen.pyx # 1
doctests failed
sage -t -long -rand=1 devel/sage/sage/misc/randstate.pyx # 1
doctests failed
sage -t -long -rand=1 devel/sage/sage/rings/real_rqdf.pyx # 2
doctests failed
sage -t -long -rand=1 devel/sage/sage/libs/fplll/fplll.pyx # 5
doctests failed
sage -t -long -rand=1
devel/sage/sage/rings/polynomial/polynomial_modn_dense_ntl.pyx # 1
doctests failed
sage -t -long -rand=1
devel/sage/sage/rings/polynomial/polynomial_element.pyx # 2 doctests
failed
sage -t -long -rand=1
devel/sage/sage/combinat/sloane_functions.py # 2 doctests failed
sage -t -long -rand=1 devel/sage/sage/combinat/words/word.py #
2 doctests failed
sage -t -long -rand=1 devel/sage/sage/calculus/equations.py #
1 doctests failed
sage -t -long -rand=1 devel/sage/sage/interfaces/r.py # 1
doctests failed
sage -t -long -rand=1 devel/sage/sage/interfaces/singular.py #
6 doctests failed
sage -t -long -rand=1 devel/sage/sage/interfaces/maxima.py #
14 doctests failed
sage -t -long -rand=1 devel/sage/sage/combinat/partition.py #
2 doctests failed
sage -t -long -rand=1 devel/sage/sage/calculus/calculus.py # 5
doctests failed
sage -t -long -rand=1
devel/sage/sage/server/notebook/worksheet.py # 1 doctests failed
----------------------------------------------------------------------
Carl Witty
> I posted a patch so that
>
> (1) doctests are ran in the same order as the file
> (2) doctests can be run in random order
> (3) doctests can be run in random order specified by a seed
>
> Carl, maybe you can referee it:
>
> http://trac.sagemath.org/sage_trac/ticket/5816
OK, what should happen now? I like the patch (except for the name of
the command-line argument); but it can't be applied because it makes
doctests fail.
As far as I can tell, there are two options:
1) go through and fix all the broken doctests, and add new patches to
#5816 for all of them
2) leave out the zero-padding from this patch, so that the default
doctest order doesn't change. Then once all the broken doctests are
eventually fixed, the zero-padding can be reinstated.
Carl
mabshoff
On Apr 18, 5:05 pm, Carl Witty <carl.wi...@gmail.com> wrote:
> On Sat, Apr 18, 2009 at 2:46 PM, William Stein <wst...@gmail.com> wrote:
> > I posted a patch so that
>
> > (1) doctests are ran in the same order as the file
> > (2) doctests can be run in random order
> > (3) doctests can be run in random order specified by a seed
>
> > Carl, maybe you can referee it:
>
> > http://trac.sagemath.org/sage_trac/ticket/5816
>
> OK, what should happen now? I like the patch (except for the name of
> the command-line argument); but it can't be applied because it makes
> doctests fail.
> As far as I can tell, there are two options:
>
> 1) go through and fix all the broken doctests, and add new patches to
> #5816 for all of them
>
> 2) leave out the zero-padding from this patch, so that the default
> doctest order doesn't change. Then once all the broken doctests are
> eventually fixed, the zero-padding can be reinstated.
with the fallout from this patch and it is trivial to apply for anyone
who wants to play with it.
> Carl
Cheers,
Michael
Rob Beezer
> the command-line argument); but it can't be applied because it makes
> doctests fail.
possible. It requires a positive action to invoke it, and its
*purpose* is to break doctests. ;-) In other words, it is working
properly when doctests break.
On the other hand, padding the example numbers to change the current
order (even if it is somewhat unpredictable) would seem to be too
chaotic.
So maybe two passes are in order.
1. Allow the randomization switch, advertise problem areas, get
doctests cleaned up.
2. Test strict ordering, perhaps optionally for a while and clean up
further problems with advertisements of problem areas.
3. Make strict ordering the default.
Rob
William Stein
>
> On Sat, Apr 18, 2009 at 2:46 PM, William Stein <wst...@gmail.com> wrote:
>> I posted a patch so that
>>
>> (1) doctests are ran in the same order as the file
>> (2) doctests can be run in random order
>> (3) doctests can be run in random order specified by a seed
>>
>> Carl, maybe you can referee it:
>>
>> http://trac.sagemath.org/sage_trac/ticket/5816
>
> OK, what should happen now? I like the patch (except for the name of
> the command-line argument); but it can't be applied because it makes
> doctests fail.
>
> As far as I can tell, there are two options:
>
> 1) go through and fix all the broken doctests, and add new patches to
> #5816 for all of them
I found a bug in the patch -- it was always running things in random
order. We will of course have to do 1) above eventually, since we
really want all tests to pass in any order.
> 2) leave out the zero-padding from this patch, so that the default
> doctest order doesn't change. Then once all the broken doctests are
> eventually fixed, the zero-padding can be reinstated.
I just posted a part 2 to the patch that does exactly that. I think
this should go in sage-3.4.1, since once it is there, it will be
trivial for people to run the randomized tests in any order (with any
seed) once sage-3.4.1 is released, and this will make it much easier
for people to fix all the bugs mentioned in 1 above.
William
William Stein
+1 to this plan. I've posted an amendment to my patch that implements
just 1 for now, which I think should go into 3.4.1, assuming it gets
a positive review.
William
mabshoff
On Apr 18, 5:20 pm, William Stein <wst...@gmail.com> wrote:
One thing that would be nice to see is that one should be able to run
the doctest N times with N something like 100 or even 1,000 for
example and each time a random seed would be picked. Then if any
failure occurred the doctesting framework should also print the random
see that was used. That way one could flush out issues on a file by
file basis.
> William
Cheers,
Michael
Rob Beezer
> One thing that would be nice to see is that one should be able to
run
> the doctest N times with N something like 100 or even 1,000 for
> example and each time a random seed would be picked. Then if any
> failure occurred the doctesting framework should also print the random
> see that was used. That way one could flush out issues on a file by
> file basis.
on one file at a time and they can hopefully spot all the possible
problems in one, or just a few, iterations. If latex.py is an
indicator, the root issue may be the same throughout a file anyway.
William Stein
I rewrote the patch and restored "traditional order".
>
> One thing that would be nice to see is that one should be able to run
> the doctest N times with N something like 100 or even 1,000 for
> example and each time a random seed would be picked. Then if any
> failure occurred the doctesting framework should also print the random
> see that was used. That way one could flush out issues on a file by
> file basis.
That's a good idea.
William