can tests involing warnings be put in doctests rather than the test suite?

[smichr]

I prefer not to see warnings while running tests. Would it be a better
idea to put these in docstrings? Or perhaps some warnings management
could be done to change the warnings into exceptions:

[ http://docs.python.org/library/warnings.html ]

[Alexey U. Gudchenko]

26.04.2011 08:20, smichr пишет:

> can tests involing warnings be put in doctests rather than the test suite?

Hi!

May be it is related with the issue 101 so I asked some questions.

First of all what do you classify as warnings? I run ./bin/doctestand
don't see warnings in master now. The only report with failed and passed
tests.

But if I add in the docstring, e.g. for depricated "block_diag"
procedure (sympy/matrices/matrices.py),

-----------------
def block_diag(matrices):
"""
Warning: this function is deprecated. See .diag()

Constructs a block diagonal matrix from a list of square matrices.

>>> from sympy.matrices.matrices import block_diag
>>> block_diag([1, 1])

See also: diag(), eye()
"""
import warnings
warnings.warn("block_diag() is deprecated, use diag() instead",
DeprecationWarning)
return diag(*matrices)
-----------------

then while I have warning message indeed:

"""
DeprecationWarning: block_diag() is deprecated, use diag() instead
warnings.warn("block_diag() is deprecated, use diag() instead",
DeprecationWarning)
"""

Also, could you explain how do you imagine use-case with the warnings in
details: how should it be formed in the docstring (by the some example)?
E.g it may be sufficient to do not shaped at whole in the docstring as
in the above example.

And what is more significant, what should the reaction of ./bin/doctest
be? Or should it be controlled by "-W" option?

--
Alexey U.

[Haz]

What about having our own (light) warning functionality -- goes to standard out under normal circumstances, and any testing can (and probably should) disable the output of the warning.

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To post to this group, send email to sy...@googlegroups.com.
To unsubscribe from this group, send email to sympy+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sympy?hl=en.

[Vinzent Steinberg]

On Apr 26, 5:24 pm, Haz <christian.mu...@gmail.com> wrote:
> What about having our own (light) warning functionality -- goes to standard
> out under normal circumstances, and any testing can (and probably should)
> disable the output of the warning.

I think it should be possible to silence specific unwanted warnings. I
wouldn't silence all, because they might be relevant for debugging.

Vinzent

[Alexey U. Gudchenko]

26.04.2011 19:24, Haz пишет:

> What about having our own (light) warning functionality -- goes to standard
> out under normal circumstances, and any testing can (and probably should)
> disable the output of the warning.
>

I do not understand exactly the purposes for our own warning functionality.
I note that warnings can be used for:
a) for our development and debugging
b) for end-user whom use sympy as CAS with IPython (e.g. about sympy
version, or deprectations)
c) for end-user about environment (e.g. about absence about some modules)
d) for users whom use SymPy as module.
and so on.

I can agree that for a)-c) our warning functionality can be used
somehow, but for exterior usage d) they will be operate with standard
warning module.

And BTW, I have just now read the documentation [1], and can advise to
Chris how temporary to suppress the warning messages while doctest'ing:

$ python -Wignore ./bin/doctest

[1] http://www.python.org/dev/peps/pep-0230/

--
Alexey U.

[Alexey U. Gudchenko]

26.04.2011 20:20, Vinzent Steinberg пишет:

Also in [1], e.g. to suppress DeprecationWarnings, one can do it with:

$ python -Wi::DeprecationWarning ./bin/doctest

[1] http://www.python.org/dev/peps/pep-0230/

In other words I thin that usage standard warning functionality is
helpful for main tasks.

The only question: should this option to be encapsulated to the
"doctest" script itself?

--
Alexey U.

[Chris Smith]

On Tue, Apr 26, 2011 at 3:09 PM, Alexey U. Gudchenko <pr...@goodok.ru> wrote:

26.04.2011 08:20, smichr пишет:

> can tests involing warnings be put in doctests rather than the test suite?

> I prefer not to see warnings while running tests. Would it be a better
> idea to put these in docstrings? Or perhaps some warnings management
> could be done to change the warnings into exceptions:
>
>     [ http://docs.python.org/library/warnings.html ]
>

Hi!

May be it is related with the issue 101 so I asked some questions.

First of all what do you classify as warnings? I run ./bin/doctestand
don't see warnings in master now. The only report with failed and passed
tests.

OK, then maybe this is platform dependent...I am getting a warning when running the tests of geometry:

    ============================= test process starts ==============================

    executable:   C:\Python27\python.exe  (2.7.1-final-0)

    ground types: python

    sympy\geometry\tests\test_geometry.py[9] ..ff..Warning: Polygons may intersect p

    roducing erroneous output

    Warning: Polygons may intersect producing erroneous output

    ...                          [OK]

    ======== tests finished: 7 passed, 2 expected to fail, in 3.40 seconds =========

Also, could you explain how do you imagine use-case with the warnings in
details: how should it be formed in the docstring (by the some example)?
E.g it may be sufficient to do not shaped at whole in the docstring as
in the above example.

If everyone were getting the above warning then I would prefer that it be raised in a doctest as you have done in your example. So the test suite wouldn't raise the condition but the doctest (which captures the output) would.

 

And what is more significant, what should the reaction of ./bin/doctest
be? Or should it be controlled by "-W" option?

I'm not sure. I would just prefer not to see anything except failures during testing. Maybe a -W could be used if you want to see the warnings. But I'm not sure that would be necessary if the warnings module were used (in the tests) to just convert the warning to an exception.

[Chris Smith]

On Tue, Apr 26, 2011 at 9:09 PM, Haz <christi...@gmail.com> wrote:

What about having our own (light) warning functionality -- goes to standard out under normal circumstances, and any testing can (and probably should) disable the output of the warning.

Yes, that is what I am looking for.

[Chris Smith]

I still get the warning:

    > \Python27\python.exe -Wignore bin\test geometry

    ============================= test process starts ==============================

    executable:   C:\Python27\python.exe  (2.7.1-final-0)

    ground types: python

    sympy\geometry\tests\test_geometry.py[9] ..ff..Warning: Polygons may intersect p

    roducing erroneous output

    Warning: Polygons may intersect producing erroneous output

    ...                          [OK]

    ======== tests finished: 7 passed, 2 expected to fail, in 2.53 seconds =========

 

[Alexey U. Gudchenko]

26.04.2011 23:47, Chris Smith пишет:

Yes, I see that there is this line (#373) in the code of
"sympy/geometry/polygon.py" :

print("Warning: Polygons may intersect producing erroneous output")

That's why:

> I still get the warning:

It is better to use "warning.warn" standard procedure. (I think issue
must created for it in any case)

Only then we can manage with those messages (e.g. as I described in
other messages) or at least discussed how to manage.

>
>> And what is more significant, what should the reaction of ./bin/doctest
>> be? Or should it be controlled by "-W" option?
>>
>> I'm not sure. I would just prefer not to see anything except failures
> during testing. Maybe a -W could be used if you want to see the warnings.
> But I'm not sure that would be necessary if the warnings module were used
> (in the tests) to just convert the warning to an exception.
>

I think, the warnings are needed for the warn notifications.
So they are meaningless if they are suppressed by default.

But in any case, they must be handled correctly in the report's display
output . Not as the dust text as now.

E.g. marked as "W" symbol (like ".", "F", "X", "s") and with the
statistic printed in the report's summary.
But the message itself in my opinion must be appeared too.

--
Alexey U.

[Mateusz Paprocki]

Hi,

2011/4/26 Alexey U. Gudchenko <pr...@goodok.ru>

That's a very good idea. Lets do it this way.

 

--
Alexey U.

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To post to this group, send email to sy...@googlegroups.com.
To unsubscribe from this group, send email to sympy+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sympy?hl=en.

Mateusz

[Aaron S. Meurer]

I already thought of much of this a while back.  See http://code.google.com/p/sympy/issues/detail?id=2142.  We should consider creating our own SymPyDeprecationWarning, since in Python 2.5+ DeprecationWarnings are not shown by default, but I think we ought to enable them at least in isympy.  

Also, we should run the tests with warnings turned into exceptions.  

Testing warnings in doctests is a bad idea, I think.  Just turn them into exceptions and test them with raises().

Aaron Meurer

[Alexey U. Gudchenko]

27.04.2011 22:22, Aaron S. Meurer пишет:

> I already thought of much of this a while back. See
> http://code.google.com/p/sympy/issues/detail?id=2142. We should
> consider creating our own SymPyDeprecationWarning, since in Python
> 2.5+ DeprecationWarnings are not shown by default, but I think we
> ought to enable them at least in isympy.
>
> Also, we should run the tests with warnings turned into exceptions.
>
> Testing warnings in doctests is a bad idea, I think. Just turn them
> into exceptions and test them with raises().
>
> Aaron Meurer
>

I something not understand why to turn warnings to exceptions is needed.
Suppose that I use SymPy as library. I don't what handle and catch the
insignificant exceptions to maintain my code to be always running.

[Aaron S. Meurer]

I only mean that they should be turned into exceptions for the tests. Of course, otherwise they should stay as warnings.

Aaron Meurer

[Alexey U. Gudchenko]

27.04.2011 23:09, Aaron S. Meurer пишет:

If so, how you imagine it in details?

As I understand the main task is: to handle warnings and inform about it
while testing.

If so then the SymPyDeprecationWarning, is one variant of realization of
this task.
As we have to handle warnings manually in any case (while testing), so
we can manage them without this auxiliary exception, because we can
direct supply info straitly to the report. And it is better, because we
can tune reporter specially how to deal with them.

Another question (which can be related with previous): whether to print
"DO NOT COMMIT" or not when warnings are raised?

--
Alexey U.

[Alexey U. Gudchenko]

Or, do you tell about the ./bin/test utility (not doctest)?

If you do, I am sorry, I supposed that we talk about ./bin/doctest utulity.

But for `./bin/test` the clearance of the usage details is needed too.

27.04.2011 23:09, Aaron S. Meurer пишет:

--
Alexey U.

[Aaron S. Meurer]

On Apr 27, 2011, at 1:26 PM, Alexey U. Gudchenko wrote:

> 27.04.2011 23:09, Aaron S. Meurer пишет:
>
> If so, how you imagine it in details?
>
> As I understand the main task is: to handle warnings and inform about it while testing.
>
> If so then the SymPyDeprecationWarning, is one variant of realization of this task.
> As we have to handle warnings manually in any case (while testing), so we can manage them without this auxiliary exception, because we can direct supply info straitly to the report. And it is better, because we can tune reporter specially how to deal with them.
>
> Another question (which can be related with previous): whether to print "DO NOT COMMIT" or not when warnings are raised?

Yeah, I think so. Code in SymPy shouldn't be raising warnings.

Aaron Meurer

>
>
>> I only mean that they should be turned into exceptions for the tests. Of course, otherwise they should stay as warnings.
>>
>> Aaron Meurer
>>
>> On Apr 27, 2011, at 1:08 PM, Alexey U. Gudchenko wrote:
>>
>>> 27.04.2011 22:22, Aaron S. Meurer пишет:
>>>> I already thought of much of this a while back. See
>>>> http://code.google.com/p/sympy/issues/detail?id=2142. We should
>>>> consider creating our own SymPyDeprecationWarning, since in Python
>>>> 2.5+ DeprecationWarnings are not shown by default, but I think we
>>>> ought to enable them at least in isympy.
>>>>
>>>> Also, we should run the tests with warnings turned into exceptions.
>>>>
>>>> Testing warnings in doctests is a bad idea, I think. Just turn them
>>>> into exceptions and test them with raises().
>>>>
>>>> Aaron Meurer
>>>>
>>>
>>> I something not understand why to turn warnings to exceptions is needed.
>>> Suppose that I use SymPy as library. I don't what handle and catch the
>>> insignificant exceptions to maintain my code to be always running.
>>
>
>
> --
> Alexey U.
>