Writing Test cases in SymPy
83 views
Skip to first unread message
Gaurav Dhingra
Nov 5, 2015, 1:50:25 AM11/5/15
to sympy
Hi everyone
There have been cases with me while solving bugs in SmyPy in certain cases that i feel the need to follow certain rules for writing cases. I think that there are no hard and fast rules to write test cases for everything in genral for `new implementation`(for any software). For example when i made the PR https://github.com/sympy/sympy/pull/10079 for fixing a bug, i think i wrote too many test-cases but i thought it is better to have more test-cases since nothing similar to that was done before in SymPy. But if i write too many test cases i fear that overall time of tests suite would increase(i think that is bad), so i am very concerned about the increasing test-suite testing time . But for bugs solving i feel that every new line has to be tested. My question is: "We as in SymPy, do we follow rules for writing test-case(hard and fast rules)?, for both the new implementation and for bug fixing?"
Even if there is no wiki about that. It would be great if i could have the link to some blog (which is very similar to rules we follow in SymPy). I remeber when Joachim Durcholz(@toolforger) mentioned about that something, writing clever test cases but may be skipping some lines of code from testing. Like on this comment https://github.com/sympy/sympy/issues/9660#issuecomment-120720283
Thanks
Aaron Meurer
Nov 5, 2015, 11:29:18 AM11/5/15
to sy...@googlegroups.com
I don't think there are any rules that would apply to every situation.
Some general things, though:
- Tests should provide coverage of the code. This is more than just
lines covered (which is what the coverage tool will tell you). It
really means you should cover all the possible code paths.
- As one instance of this, you should try to test any corner cases.
Corner cases usually represent trivial input, so they generally are
very fast tests, but they are important because it's easy to code them
wrong.
- As for the speed of the tests, you should avoid tests that are
slower than necessary (like don't use a very large expression if a
small expression would test the same thing). However, I would worry
more about coverage than test speed. Don't be afraid to add a lot of
tests. That PR you referenced doesn't seem to have too many tests to
me.
- It's also a good idea to stress test every once in awhile. We don't
do this enough in SymPy IMHO. This means test that are larger than
usual. It could also mean tests that are only fast if the code is
properly tuned, to test against performance regressions.
- Tests often test more than one thing implicitly (e.g., a test may
test the output of a function, but also implicitly test that the
function does not raise an exception). However, it should be clear
from a test what it is explicitly testing.
Aaron Meurer
> --
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/a5ef6de8-1e83-4ea6-bdb6-ec351d9011b9%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Some general things, though:
- Tests should provide coverage of the code. This is more than just
lines covered (which is what the coverage tool will tell you). It
really means you should cover all the possible code paths.
- As one instance of this, you should try to test any corner cases.
Corner cases usually represent trivial input, so they generally are
very fast tests, but they are important because it's easy to code them
wrong.
- As for the speed of the tests, you should avoid tests that are
slower than necessary (like don't use a very large expression if a
small expression would test the same thing). However, I would worry
more about coverage than test speed. Don't be afraid to add a lot of
tests. That PR you referenced doesn't seem to have too many tests to
me.
- It's also a good idea to stress test every once in awhile. We don't
do this enough in SymPy IMHO. This means test that are larger than
usual. It could also mean tests that are only fast if the code is
properly tuned, to test against performance regressions.
- Tests often test more than one thing implicitly (e.g., a test may
test the output of a function, but also implicitly test that the
function does not raise an exception). However, it should be clear
from a test what it is explicitly testing.
Aaron Meurer
> You received this message because you are subscribed to the Google Groups
> "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/a5ef6de8-1e83-4ea6-bdb6-ec351d9011b9%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
Jason Moore
Nov 5, 2015, 11:40:57 AM11/5/15
to sy...@googlegroups.com
Thanks Aaron. Someone should copy this to a wiki page!
To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6Jn8Hk1t88x_oorcT9NsTFYi5sDcWYDyGx9uHwEoWKfwA%40mail.gmail.com.
Gaurav Dhingra
Nov 8, 2015, 4:08:25 AM11/8/15
to sympy
Thanks Jason, Aaron. I have made a new wiki page and added the text written by Aaron here https://github.com/sympy/sympy/wiki/Test-Case-Writing-in-SymPy
Gaurav(gxyd)
Sumith 1896
Nov 8, 2015, 4:15:39 AM11/8/15
to sympy
Thanks Gaurav,
I have done some minor changes and the page now resides here: https://github.com/sympy/sympy/wiki/Writing-test-cases-in-SymPyTo view this discussion on the web visit https://groups.google.com/d/msgid/sympy/0ea09215-341a-43e2-9dbc-946f6b8b9b23%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages