Hi. Could we maybe turn off reproducible "randomness" in doctests by default. There are probably thousands of meaningless doctests because of this.
or the doctests for the namespace function `bernoulli` or the `number_of_partitions` function also in the namespace.Alternatively we should add `set_random_seed` to all of those places.Either way, this might be a huge project, because of possible bugs.Jonathan
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/6a5005ec-717e-400c-b253-514f9101bc72o%40googlegroups.com.
sage: ls = [randint(-100,100) for _ in range(4)] sage: intervals = [[x, x+randint(1,50)] for x in ls] sage: P = polytopes.hypercube(4, intervals, backend='field')
On Mon, 22 Jun 2020, 10:01 'Jonathan Kliem' via sage-devel, <sage-...@googlegroups.com> wrote:Hi. Could we maybe turn off reproducible "randomness" in doctests by default. There are probably thousands of meaningless doctests because of this.I don't understand, you say there that it went unnoticed due to a too easy doctest. I don't see how that one had anything to do with randomness.Also, could you explain what you mean by "randomness" there. Doctests are either tagged "random" (and do matching of answers skipped),or use random inputs. If a doctest with random input produces inconsistent results due to not properly set seeds, it would have been caught all along.
--or the doctests for the namespace function `bernoulli` or the `number_of_partitions` function also in the namespace.Alternatively we should add `set_random_seed` to all of those places.Either way, this might be a huge project, because of possible bugs.Jonathan
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-...@googlegroups.com.
I had a doctest
sage: ls = [randint(-100,100) for _ in range(4)]sage: intervals = [[x, x+randint(1,50)] for x in ls]sage: P = polytopes.hypercube(4, intervals, backend='field')
which then checked `P` for consistency. However it was somewhat meaningless (missing `set_random_seed()`) as all entries in `ls` where positive. It never caught a sign error that we introduced in #28866.Adding `set_random_seed` to it, does catch the error.
--Now this check for consistency certainly wasn't very good in the first place, as two polyhedra might compare equal, even if one is set up incorrectly (I just noticed this because of your post, thank you). But it still would have caught the error even randomness was true randomness.You might say, I should have known to add `set_random_seed()` in the first place, but I'm definitely not the only one who made that mistake and there are plenty of doctests that claim to test a random event, but they really test the same event all over and over.Am Montag, 22. Juni 2020 11:24:59 UTC+2 schrieb Dima Pasechnik:On Mon, 22 Jun 2020, 10:01 'Jonathan Kliem' via sage-devel, <sage-...@googlegroups.com> wrote:Hi. Could we maybe turn off reproducible "randomness" in doctests by default. There are probably thousands of meaningless doctests because of this.I don't understand, you say there that it went unnoticed due to a too easy doctest. I don't see how that one had anything to do with randomness.Also, could you explain what you mean by "randomness" there. Doctests are either tagged "random" (and do matching of answers skipped),or use random inputs. If a doctest with random input produces inconsistent results due to not properly set seeds, it would have been caught all along.--or the doctests for the namespace function `bernoulli` or the `number_of_partitions` function also in the namespace.Alternatively we should add `set_random_seed` to all of those places.Either way, this might be a huge project, because of possible bugs.Jonathan
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/6a5005ec-717e-400c-b253-514f9101bc72o%40googlegroups.com.
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/fcc3910a-e851-4f58-9a26-4eaf3dced699o%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/fcc3910a-e851-4f58-9a26-4eaf3dced699o%40googlegroups.com.
Currently, the doctest setup produces the very same random numbers on every instance.If you call `set_random_seed()` first you will get somewhat different values on every single doctest run. It can still be annoying as not every bot will catch every failure, but it is by far more likely to catch regression eventually.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/f5004f9e-be47-47a2-9646-6e3eaf6ac1b6o%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/f5004f9e-be47-47a2-9646-6e3eaf6ac1b6o%40googlegroups.com.
On 2020-06-22 05:01, 'Jonathan Kliem' via sage-devel wrote:
> Hi. Could we maybe turn off reproducible "randomness" in doctests by
> default. There are probably thousands of meaningless doctests because of
> this.
>
> E.g. https://trac.sagemath.org/ticket/29904
>
> or the doctests for the namespace function `bernoulli` or the
> `number_of_partitions` function also in the namespace.
>
> Alternatively we should add `set_random_seed` to all of those places.
>
> Either way, this might be a huge project, because of possible bugs.
>
For anyone who doesn't know what you're talking about, e.g.
sage: ZZ.random_element()
in a doctest always produces the same number.
I'm violently opposed to the existing behavior because it's caused me to
introduce several embarrassing bugs.
It's counterintuitive and unique to
sage, so new users and reviewers never learn about it until it's too
late. And finally, the reasoning behind it is just silly. There are
plenty of ways to test random functions without being able to predict
their literal output. For example, instead of,
"""
Ensure that we can produce random integers::
sage: ZZ.random_element()
42
"""
One can easily use
"""
Ensure that we can produce random integers::
sage: ZZ.random_element() in ZZ
True
"""
Changing the behavior will break a lot of tests (as well as shed light
upon many bugs), but the working tests at least can be addressed with
set_random_seed(), in essence by removing their randomness again.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/ed175873-fc41-812f-a92e-edbb2180ee9e%40orlitzky.com.
Yes there is need. Please go into any file of your choice and print out a "random" value in a doctest and see what happens.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/ac071eda-2f54-45a2-9434-5104341dcb36o%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/ac071eda-2f54-45a2-9434-5104341dcb36o%40googlegroups.com.
And this design has problems e.g. https://trac.sagemath.org/ticket/28599Basic story is: The random splitting that is being done to find real roots is far from stable. Apparently the doctests don't catch on, because they always use the same "random state". In real life, you cannot obtain a regular polygon with 20 vertices with this method.So some doctests claim that sage can do something, which can be totally wrong (unless you have your lucky day). Maybe if the doctests would have shown some variation, people designing this would have caught on that their random choice could be real bad and they would have to accounted for that.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/bdd1663f-9898-417a-890f-84d4ed25425bo%40googlegroups.com.
Doing so won't consume any extra time
on any individual machine, and the multitudes of reviewers and patchbots
running the test on different examples will ferret out any corner cases.