5 doctest errors when building 4.7.2

[Emil Widmann]

I compiled sage 4.7.2 (and eventually use the new make micro_install
option) on my Live CD base distro. But I got some doctest failures.
With two of them I get different correct solutions compared to the
solutions in the doctests. Other failures seem specific for my setup.
Could you give me advice or hints how those could be be fixed (I'd
like to have zero doctest failures)? Or at least give an expert
assessment if those failures are critical?

some info:
md5sum of source download was correct
I use gcc 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
Kernel 2.6.33.2
2xIntel Duo E4600 CPU@2.4 Ghz / 1 GB RAM

- failure 1 -------------

sage -t -force_lib "devel/sage/doc/en/thematic_tutorials/
linear_programming.rst"
**********************************************************************
File "/mnt/sda6/COMPILE/sage-4.7.2/devel/sage/doc/en/
thematic_tutorials/linear_programming.rst", line 366:
sage: [e for e,b in matching.iteritems() if b == 1]
Expected:
[(0, 1), (6, 9), (2, 7), (3, 4), (5, 8)]
Got:
[(1, 6), (0, 4), (2, 3), (5, 8), (7, 9)]
**********************************************************************

1 items had failures:
1 of 11 in __main__.example_2
***Test Failed*** 1 failures.
For whitespace errors, see the file /root/.sage//tmp/
linear_programming_3381.py
[3.2 s]

This is about finding a matching set in the Peterson graph. I read
about it and this is obviously an alternativ solution compared to the
one in the doctest.
Out of symmetrie reasons there are still other possibilities to solve
this.
Would it makes sence to choose doctests with unique solutions? Or does
this result points out that something is seriously odd in my system?


- failure 2 ---------------

sage -t -force_lib "devel/sage/sage/structure/sage_object.pyx"
**********************************************************************
File "/mnt/sda6/COMPILE/sage-4.7.2/devel/sage/sage/structure/
sage_object.pyx", line 1042:
sage: sage.structure.sage_object.picklejar(1, dir + '/noaccess')
Expected:
Traceback (most recent call last):
...
OSError: ...
Got nothing
**********************************************************************
1 items had failures:
1 of 11 in __main__.example_24
***Test Failed*** 1 failures.
For whitespace errors, see the file /root/.sage//tmp/
sage_object_3831.py
[7.7 s]

This is about testing access to a directory which had permissions set
to "0000" before.
The doctest expects some sort of error message, but on my machine
there is no error.
I checked that the directory was created and the permissions were set
to 0000 correctly.
Is this critical or can it be fixed with some sort of setting?

EDIT: I found the culprit - I ran doctest as root.
If I run the doctest as restricted user I get the correct OSError
message.

Maybe there could be a check/warning if the tests run as root?


- failure 3 ---------------

sage -t -force_lib "devel/sage/sage/combinat/integer_vector.py"
**********************************************************************
File "/mnt/sda6/COMPILE/sage-4.7.2/devel/sage/sage/combinat/
integer_vector.py", line 262:
sage: print gale_ryser_theorem(p1, p2)
Expected:
[1 1 0]
[1 0 1]
[0 1 0]
Got:
[0 1 1]
[1 1 0]
[1 0 0]
**********************************************************************
File "/mnt/sda6/COMPILE/sage-4.7.2/devel/sage/sage/combinat/
integer_vector.py", line 298:
sage: print gale_ryser_theorem(p1, p2, algorithm="gale")
Expected:
[1 1 1 0 0]
[1 1 0 0 1]
[1 0 1 0 0]
[0 0 0 1 0]
Got:
[1 0 1 1 0]
[1 0 1 0 1]
[1 1 0 0 0]
[0 1 0 0 0]
**********************************************************************
1 items had failures:
2 of 31 in __main__.example_3
***Test Failed*** 2 failures.
For whitespace errors, see the file /root/.sage//tmp/
integer_vector_4275.py
[3.2 s]

Here we have another example of a different possible solution. I read
about the gale ryser theorem and
p1 = [2,2,1], p2 = [2,2,1] and the row sum of the matrix should be p1
and the column sum of the matrix should be p2
which is obviously true for both matrices.

Same is true for the second doctest error, both matrices have
[3,3,2,1] [3,2,2,1,1] as row resp column sum.
Why choose doctests with multiple possible solutions?


- failure 4 --------------

sage -t -force_lib "devel/sage/sage/sandpiles/sandpile.py"
*** *** Error: TIMED OUT! PROCESS KILLED! *** ***

[360.2 s]

This module gave me headaches before, just because doctests needs so
much time.
I went through the doctests manualy, most went through smooth, only
which took several seconds was
line 196: S.resolution() - took maybe 15 sec
line 1897: S._set_resolution() - took maybe 35 sec
line 1939: S.resolution() - same as line 196
line 2026: S.betti() - took long, I timed it and got: CPU 2.19 s,
Wall: 48.64 s ?
line 4689: K.betti() - took ages, like 60 sec
line 4979: S.resolution() - took > 30 sec
line 5049: D=SandpileDivisor(S, [1,1,1,12,0]) - takes like 60 sec
line 5134: sage: for p in P: sum([partition_sandpile(S,
i).betti(verbose=False)[-1] for i in p]) ... betti in a loop :(, takes
about 20 sec
line 5138: S.betti() betti again
line 5172: sage: for p in P: sum([partition_sandpile(S,
i).betti(verbose=False)[-1] for i in p]) ... again, betti in loop
line 5181: S.betti ...

As one can see, most of the time is used for "betti" and this is
tested over and over again.
I think here there is a conflict with the function of the docstrings
as "help and documentation", and
as a way to test the correctness of the build. Is there a specific
problem with my machine so that this function is so slow?
Otherways it would make no sense for me to test this slow function
multiple times and have the test ultimately time out because of it. (6
minutes are 5% of overall test time)

Another point I don't understand is e.g in line 1367, there is the
example
sage: [i.effective_div() for i in ns] # optional - 4ti2

which gives a warning because I don't have "4ti2" installed - is this
another dependency?
in the documentation it says:
r"""
To calculate linear systems associated with divisors, 4ti2 must be
installed.
One way to do this is to run sage -i to install glpk, then 4ti2. See
http://sagemath.org/download-packages.html to get the exact names of
these
packages. An alternative is to install 4ti2 separately, then point
the
following variable to the correct path.
"""

I didn't know that there are hidden dependencies in regular sage
modules (other examples who need this dependency in this module are in
Lines 1644,1684ff, 3505ff, 4045, 4147f,4211, 4296, 4346, 4415, 5050)
Given this long list I assume the module sandbox is only fully usable
if this "4ti2" dependency is installed. IMHO this is questionable for
a standard module. Are there such hidden dependencies in other
standard sage modules? Would it be advisable to include all such
dependencies in a Live CD release?


- failure 5 ------------

sage -t -force_lib "devel/sage/sage/graphs/digraph.py"
The doctested process was killed by signal 6
[5.6 s]

I get an error for the test in line 857:
sage: all( random_acyclic(100, .2).is_directed_acyclic() # long
time
... for i in range(50)) # long
time

Line 1340:
sage: dcycle.feedback_edge_set(value_only=True)
give no result while it should give 5.0

also the next tests of the function "feedback_edge_set" do not work
properly
sage: g = graphs.RandomGNP(5,.3)
sage: dg = DiGraph(g)
sage: feedback = dg.feedback_edge_set()
sage: (u,v,l) = g.edge_iterator().next()
sage: (u,v) in feedback or (v,u) in feedback

gives no result while it should give True

other Doctests in this specific function fail with Traceback error.

so I think there is an error somewhere here wich results in the
"Killed/crashed" error. I tried the rest of the doctests in the
module, but those where OK.


ok thanks if you read this far :-) any help appreciated.
sincerly
emil

[Nathann Cohen]

Helloooooooooooooo Emil !

Well, it looks like I am to blame for most of these errors :-/

The first one is about a matching in Petersen's Graph. Then the error in the gale_ryser theorem function. Both codes are calling GLPK (I assume that you have not installed the CBC package ? Or CPLEX ?), and it seems its results depend on the architecture (though I believe they are not random, so they should always return the same on your architecture).

Why choose doctests with multiple possible solutions?

Well. It's not that I was exactly looking for examples which had several solutions, but then the doctests are also made to lead the user to understand how the commands are to be used and what they do, and there's no point in building a very complicated graph while losing the reader only because of our technical problems. These days, I am really keen on adding many EXAMPLES whith the flag "not tested", and a TEST section afterwards ensuring that the methods work with symmetry-free doctests.

I mean... Doctests are nice, and the way they are checked create technical problems, but the solution is not to change the documentation (meant for users) so that it solves our technical problems. Doctests are a tool, not a master :-p

So the best is probably to comment these tests out (this is a thematic tutorial anyway, and those methods are tested extensively elsewhere) and add correction tests somewhere else.

You said that you got an error on :

sage: all( random_acyclic(100, .2).is_directed_acyclic()    # long time 

...        for i in range(50))                              # long time 

Could you say what happened ? It is also wrong that feedback_edge_set would return nothing, though this function intensively calls the "is_directed_acyclic" method, so this is probably the same problem.

... and I am totally ignorant about the other bugs you found :-)

Nathann

 

[Emil Widmann]

On Nov 13, 4:53 pm, Nathann Cohen <nathann.co...@gmail.com> wrote:
> Helloooooooooooooo Emil !
>
> Well, it looks like I am to blame for most of these errors :-/

Hi Nathan, thanks for quick response!
I am not sure if those are errors, but when I, as a non export, try to
build sage then I take failing doctests
seriously. But to go into the details with each failing test takes
time and is tedious, so I think it is better to avoid possible
problems (like those with the multible solutions) whenever possible.

> The first one is about a matching in Petersen's Graph. Then the error in
> the gale_ryser theorem function. Both codes are calling GLPK (I assume that
> you have not installed the CBC package ? Or CPLEX ?), and it seems its
> results depend on the architecture (though I believe they are not random,
> so they should always return the same on your architecture).

I know GLPK (Gnu Linear Programming Kit) but I am, yet, totally
ignorant of the other two packages.
And yes, results are not random. The Functions give always the same
solution on my machine

>
> Why choose doctests with multiple possible solutions?
>
>
>
> Well. It's not that I was exactly looking for examples which had several
> solutions, but then the doctests are also made to lead the user to
> understand how the commands are to be used and what they do, and there's no
> point in building a very complicated graph while losing the reader only
> because of our technical problems. These days, I am really keen on adding
> many EXAMPLES whith the flag "not tested", and a TEST section afterwards
> ensuring that the methods work with symmetry-free doctests.
> I mean... Doctests are nice, and the way they are checked create technical
> problems, but the solution is not to change the documentation (meant for
> users) so that it solves our technical problems. Doctests are a tool, not a
> master :-p
> So the best is probably to comment these tests out (this is a thematic
> tutorial anyway, and those methods are tested extensively elsewhere) and
> add correction tests somewhere else.

I fully agree - is it okay if I create a "track ticket" for this? I
haven't done this before but I think I'll manage.

> You said that you got an error on :
>
> sage: all( random_acyclic(100, .2).is_directed_acyclic()    # long time
> ...        for i in range(50))                              # long time
>
> Could you say what happened ? It is also wrong that feedback_edge_set would
> return nothing, though this function intensively calls the
> "is_directed_acyclic" method, so this is probably the same problem.

I will look into this error again try to give more details

> ... and I am totally ignorant about the other bugs you found :-)

Well, I think the error with
sage -t -force_lib "devel/sage/sage/structure/sage_object.pyx"

is based on testing as root. So there could be a check if this test is
run as root, however I have no idea how this would be implemented.
Or one could give a warning in the beginning of all tests. Or on could
ignore this case entirely and say people who run as root have to live
with such minor inconveniences (Don't miss the hilarious
http://www.garyshood.com/root/!)

My issue with sandpile.py is closely related to the issues with gyle &
ryler. I think there is lot's of double testing of the same
functionality.
If I got it right a possible solution would be to include some TEST
sections which doctest the functionality once (S.betti ...), then
the bunch of the other examples would be ignored?

I am aware that those are minor issues, but if building sage on a
stock PC they might cause some confusion and extra detective work.
Does this qualify to file a bug or open an issue on trac?

sincerly
emil

[Nathann Cohen]

Hellooooo !!

> seriously. But to go into the details with each failing test takes
> time and is tedious, so I think it is better to  avoid possible
> problems (like those with the multible solutions) whenever possible.

Yepyep. All that needs to be fixed :-)

> I fully agree - is it okay if I create a "track ticket" for this? I
> haven't done this before but I think I'll manage.

Please do ! And if you see the "cc" field in the Trac form, please add
my username (ncohen) inside :-)

> I will look into this error again try to give more details

Thaaaaaanks !

> My issue with sandpile.py is closely related to the issues with gyle &
> ryler. I think there is lot's of double testing of the same
> functionality.

Gyle & ryler ? Gale Ryser ?

Gale Ryser problem's also comes from GLPK !

> If I got it right a possible solution would be to include some TEST
> sections which doctest the functionality once (S.betti ...), then
> the bunch of the other examples would be ignored?

Yep, especially in this case where the tests are not present to test
Sage but rather to explain things to the users. Complicating them
would make the bugs vanish, but if the users do not get what it means
anymore then there's no point. So it would be better in this case to
write doctests only for the user, and to put the real tests elsewhere,
or at the end of the document.

> I am aware that those are minor issues, but if building sage on a
> stock PC they might cause some confusion and extra detective work.
> Does this qualify to file a bug or open an issue on trac?

Anything that makes the doctests scream qualifies as a bug. Anything
that is not even a bug but that is not the behaviour you expected also
qualifies as a bug, even though it may just mean rewriting some
documentation :-)

Thanks for your help !

Nathann

[Emil Widmann]

> Anything that makes the doctests scream qualifies as a bug. Anything
> that is not even a bug but that is not the behaviour you expected also
> qualifies as a bug, even though it may just mean rewriting some
> documentation :-)

Ok I created some tickets:

doctest failure in linear_programming.rst:
http://trac.sagemath.org/sage_trac/ticket/12019

doctest failure insage_object.pyx when test run as root
http://trac.sagemath.org/sage_trac/ticket/12021

failing doctest in integer_vector.py (gale ryser theorem)
http://trac.sagemath.org/sage_trac/ticket/12022

doctest of sandpile.py TIMED OUT!
http://trac.sagemath.org/sage_trac/ticket/12023

about the last error in digraph.py I will have a look, but I will need
some time, maybe tomorrow.

[Volker Braun]

On Sunday, November 13, 2011 3:05:32 PM UTC+1, Emil Widmann wrote:

I checked that the directory was created and the permissions were set
to 0000 correctly.
Is this critical or can it be fixed with some sort of setting?

EDIT: I found the culprit - I ran doctest as root.
If I run the doctest as restricted user I get the correct OSError
message.

We should change the doctest to execute "rm -rf /" instead. This will also provide a valuable lesson for people that use privileged accounts for everything :-)

[Nathann Cohen]

> We should change the doctest to execute "rm -rf /" instead. This will also provide a valuable lesson for people that use privileged accounts for everything :-)

Not to mention that all free softwares are distributed with the
warning "You use this code at your own risk" and that we never had any
fun with it. What's the point then ? :-p

Nathann_also_known_as_root_on_#sagemath

[Emil Widmann]

About the failing test
sage -t -force_lib "devel/sage/sage/graphs/digraph.py" # Killed/
crashed
I ran the test again with -verbose

I think its connected to the routine dcycle.feedback_edge_set (in line
1252) and maybe its an integer / floating error

Trying:
cycle.size()###line 1338:_sage_ >>> cycle.size()
Expecting:
5
# >>> dcycle.feedback_edge_set(value_only=True)
5.0
**********************************************************************
File "/mnt/sda6/COMPILE/sage-4.7.2/devel/sage/sage/graphs/digraph.py",
line 998, in __main__.example_22
Failed example:
cycle.size()###line 1338:_sage_ >>> cycle.size()
Expected:
5
# >>> dcycle.feedback_edge_set(value_only=True)
5.0
Got:
5

and then:

Trying:
feedback = dg.feedback_edge_set()###line 1348:_sage_ >>>
feedback = dg.feedback_edge_set()
Expecting nothing
Assertion failed: orig->m > 0 && orig->n > 0
Error detected in file glpapi09.c at line 289
/mnt/sda6/COMPILE/sage-4.7.2/local/lib/libcsage.so(print_backtrace
+0x3b)[0xb71f231e]
/mnt/sda6/COMPILE/sage-4.7.2/local/lib/libcsage.so(sigdie+0x17)
[0xb71f235e]
/mnt/sda6/COMPILE/sage-4.7.2/local/lib/libcsage.so(sage_signal_handler
+0x1c4)[0xb71f1e97]
[0xffffe400]
/lib/libc.so.6(abort+0x182)[0xb7472ab2]
/usr/local/lib/libglpk.so.0(+0x43b0e)[0xaea1eb0e]
/usr/local/lib/libglpk.so.0(+0x439ff)[0xaea1e9ff]
/usr/local/lib/libglpk.so.0(glp_intopt+0xae0)[0xae9e89b0]
/mnt/sda6/COMPILE/sage-4.7.2/local/lib/python/site-packages/sage/
numerical/backends/glpk_backend.so(+0x5c47)[0xaea99c47]
/mnt/sda6/COMPILE/sage-4.7.2/local/lib/python/site-packages/sage/
numerical/mip.so(+0x29036)[0xb0bcf036]
/mnt/sda6/COMPILE/sage-4.7.2/local/lib/libpython2.6.so.
1.0(PyCFunction_Call+0x128)[0xb7642c58]

.lotsof that sort..

/mnt/sda6/COMPILE/sage-4.7.2/local/lib/libpython2.6.so.1.0(Py_Main
+0xc7d)[0xb76d0c5d]
python(main+0x27)[0x8048667]
/lib/libc.so.6(__libc_start_main+0xe6)[0xb745bbc6]
python[0x80485a1]

------------------------------------------------------------------------
Unhandled SIGABRT: An abort() occurred in Sage.
This probably occurred because a *compiled* component of Sage has a
bug
in it and is not properly wrapped with sig_on(), sig_off(). You might
want to run Sage under gdb with 'sage -gdb' to debug this.
Sage will now terminate.
------------------------------------------------------------------------

The doctested process was killed by signal 6

[5.7 s]

Personally I am fine with doctesting rm -rf /. People who run as root
will handle it with dignity (ouch, maybe not?).

[Nathann Cohen]

Yoooooooooo !!!

> doctest failure in linear_programming.rst:
> http://trac.sagemath.org/sage_trac/ticket/12019

This ticket is now waiting for a review.

> failing doctest in integer_vector.py (gale ryser theorem)
> http://trac.sagemath.org/sage_trac/ticket/12022

Same here. And these tickets could not be shorter :-)

Nathann

[Emil Widmann]

I applied the patch and it passes doctest - great !.
I changed status to positiv review.

check also system information appended at ticket 12019, and the
processor information in 12022.
The system is maybe not so uncommon - it is the Live CD base
distribution which had approx 2000 Downloads from the sage mirrors
this year.

hurray for those two patches - what do you think about the others?

About the sandpile.py doctests - I could try to write a patch myself
which is similar to the one you wrote (ad the # not tested flag to
some of the multiple calls I mentioned in the first post. I am sure
sandpile.py running out of time is not uncommon at all.

The one with the root issue is trickier - is it save to flag it as not
tested or is there some option to make a doctest conditional (e.g. if
UID = 0 then # not tested) ?

I just opened a ticket for the last one (crash in digraph.py).
http://trac.sagemath.org/sage_trac/ticket/12054

thanks
emil

[Volker Braun]

I propose that we just add a doctest that makes it clear that you shouldn't run doctests as root. No more mysterious failures.

    sage: import os

    sage: try:    

    ...    uid = os.getuid()

    ... except AttributeError:     

    ...    uid = -1

    sage: if uid==0:

    ...       raise OSError('You must not run the doctests as root, geez!')

[Emil Widmann]

this would probably be the right thing to do :-)! So you think it is
better to add it somewhere on the top level, or should I just take
this code and apply at the place where it fails?
it is in:
devel/sage/sage/structure/sage_object.pyx, line 1042:

[Volker Braun]

Since there is apparently only one doctests that fails with elevated privileges I'd put it right there, at the place where it fails.

[Keshav Kini]

Why not put it in the doctester module itself? That way you'll avoid actually running any doctests as root. If you make it just a doctest in the suite, it will fail, but all the doctests will be run as root anyway.

-Keshav

----
Join us in #sagemath on irc.freenode.net !

[Jeroen Demeyer]

On 2011-11-19 12:23, Volker Braun wrote:
> I propose that we just add a doctest that makes it clear that you
> shouldn't run doctests as root. No more mysterious failures.

Well, I certainly agree that running doctests as root is not very
clever, I don't think we should *prevent* anyone from doing that.
Note also that in a system-wide install, the person installing Sage
(e.g. root) must run doctests before Sage will function completely. So,
if you do this, make it depend on #5155.

[Volker Braun]

On Saturday, November 19, 2011 1:22:09 PM UTC, Jeroen Demeyer wrote:

Well, I certainly agree that running doctests as root is not very
clever, I don't think we should *prevent* anyone from doing that.

My suggestion wouldn't prevent anyone from running the doctest as root, they'd just get a doctest failure that is more illuminating than the current OSError.

[William Stein]

A tangentially related remark, I do not recommend that in a
system-wide install, then person
installing has to be root. For example, I have Sage installs
systemwide on sage.math (and the other *.math) computers, but I made
them by creating a /usr/local/sage directory (as root), and making
that
directory owned by the user wstein. Then everything else related to
the system-wide Sage install is done as the user "wstein". I
consider this safer.

- William

>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>

--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org

[Emil Widmann]

I have written a patch with this code - it throws no error message any
more.
http://trac.sagemath.org/sage_trac/ticket/12021
I have also flagged several doctests in sandpile.py as # long time
patch is in http://trac.sagemath.org/sage_trac/ticket/12023
Those were my first tries with hg and creating patches so I would be
grateful if somebody could comment if those submissions and entries
are OK or if I made some serious blunders.