"Terry Reedy" <tjre...@udel.edu> writes: > > But assert statements vanish when you turn on the optimizer. If > > you're going to run your application with the optimizer turned on, I > > certainly hope you run your regression tests with the optimizer on.
> I don't see why you think so. Assertion statements in the test code make > it harder, not easier for the test to pass. Ditto, I believe, for any in > the run code, if indeed there are any.
If the unit tests are expressed as assert statements, and the assert statements get optimized away, then running the unit tests on the optimized code can obviously never find any test failures.
Paul Rubin wrote: > "Terry Reedy" <tjre...@udel.edu> writes:
>>>But assert statements vanish when you turn on the optimizer. If >>>you're going to run your application with the optimizer turned on, I >>>certainly hope you run your regression tests with the optimizer on.
>>I don't see why you think so. Assertion statements in the test code make >>it harder, not easier for the test to pass. Ditto, I believe, for any in >>the run code, if indeed there are any.
> If the unit tests are expressed as assert statements, and the assert > statements get optimized away, then running the unit tests on the > optimized code can obviously never find any test failures.
Any code depending upon __debug__ being 0 won't be tested. Sometimes test structures update values as a side-effect of tracking the debugging state. Not massively likely, but it makes for a scary environment when your tests cannot be run on a non-debug version.
Scott David Daniels <Scott.Dani...@Acm.Org> wrote:
> Any code depending upon __debug__ being 0 won't be tested. Sometimes > test structures update values as a side-effect of tracking the debugging > state. Not massively likely, but it makes for a scary environment when > your tests cannot be run on a non-debug version.
def verify (value): if not value: throw AssertionError
and then everyplace in your py.test suite where you would normally have done "assert foo", you now do "verify (foo)"? A quick test shows that it appears to do the right thing. I made a little test file:
when I run that with "python py.test", I get two failures. When I run it with "python -O py.test", I get one pass and one fail, which is what I expected to get if the assert gets optimized away.
The output is a little more verbose, since it shows the exception raised in verify(), but it gives you a stack dump, so it's not that hard to look one frame up and see where verify() was called from.
It's interesting that given the penchant for light-weight-ness in py.test, that the default output is so verbose (and, to my mind, confusing) compared to unittest. I guess one could write their own output formatter and cut down on the verbosity?
Raymond Hettinger wrote: > [Peter Hansen] >>(I'm not dissing py.test, and intend to check it >>out.
> Not to be disrepectful, but objections raised by someone > who hasn't worked with both tools equate to hot air.
Not to be disrespectful either, but criticism by someone who has completely missed my point (and apparently not read my posts) doesn't seem entirely fair.
At no time in this thread have I objected to py.test. The sole point of my posts has been to object to those claiming unittest as "heavy" when in the same breath they seem to think you have to know all kinds of details about TestSuite, TestRunner, and TestResult objects just to use it. I tried to demonstrate that my way of using it appears to be on the same order of "lightness" as some of the samples that were being used to show how much lighter py.test was.
> Until you've exercised both packages, you haven't helped the OP > whose original request was: "Is there anybody out there who has > used both packages and can give a comparative review?"
It seems possible to me that I might have helped him solely by pointing out that unittest might not be so "heavy" as some people claimed. I got the impression that he might be swayed by some unfounded claims not even to look further at unittest, which I felt would be a bad thing. (Not to say your comments are unfounded, as clearly they are valid... I happen to believe mine have been, in this thread, as well. I guess you're free to believe otherwise. Cheers.)
>It seems possible to me that I might have helped him >solely by pointing out that unittest might not be so >"heavy" as some people claimed. I got the impression >that he might be swayed by some unfounded claims not >even to look further at unittest, which I felt would >be a bad thing.
I'm the "him" referred to above. I've been using unittest ever since it was first added to the standard library (actually, now that I think about it, I believe I may have been using it even before then).
And yes, I think unittest brings along a certain amount of baggage. There is something attractive about having the same basic framework work in many languages (PyUnit, JUnit, C++Unit, etc), but on the other hand, it does add ballast. I use it, I certainly don't hate it, but on the other hand, there are enough things annoying about it that it's worth investing the effort to explore alternatives.
From the few days I've been playing with py.test, I think I like what I see, but it's got other issues. The "optimization elides assert" issue we've been talking about is one.
It's also neat that I can write unittest-style test classes or go the simplier route of just writing static test functions, but there's a certain amount of TIMTOWTDI (did I spell that right?) smell to that.
I'm also finding the very terse default output from unittest (basicly a bunch of dots followed by "all N tests passed") highly preferable to py.test's verbosity.
In short, I haven't made up my mind yet, but I do appreciate the input I've gotten.
"Raymond Hettinger" <vze4r...@verizon.net> writes: > [Peter Hansen] > > (I'm not dissing py.test, and intend to check it > > out.
> Not to be disrepectful, but objections raised by someone > who hasn't worked with both tools equate to hot air.
[...]
Why? Peter had a reasonable question which, AFAICT, doesn't depend on any more detailed knowledge that what he had to work with.
What I don't understand about py.test (and trying it out seems unlikely to answer this) is why it uses the assert statement. unittest used to do that, too, but then it was pointed out that that breaks when python -O is used, so unittest switched to self.assert_ &c. Does py.test have some way around that?
On Mon, 04 Apr 2005 22:50:35 +0000, John J. Lee wrote: > What I don't understand about py.test (and trying it out seems > unlikely to answer this) is why it uses the assert statement. > unittest used to do that, too, but then it was pointed out that that > breaks when python -O is used, so unittest switched to self.assert_ > &c. Does py.test have some way around that?
"Don't use -O because it doesn't do anything significant?"
Is this an issue in practice? (Honest question.) If -O did something interesting I might use it, but I don't think it does.
In article <pan.2005.04.04.23.26.28.468...@jerf.org>, Jeremy Bowers <j...@jerf.org> wrote:
> On Mon, 04 Apr 2005 22:50:35 +0000, John J. Lee wrote: > > What I don't understand about py.test (and trying it out seems > > unlikely to answer this) is why it uses the assert statement. > > unittest used to do that, too, but then it was pointed out that that > > breaks when python -O is used, so unittest switched to self.assert_ > > &c. Does py.test have some way around that?
> "Don't use -O because it doesn't do anything significant?"
> Is this an issue in practice? (Honest question.) If -O did something > interesting I might use it, but I don't think it does.
The following program produces different output depending on whether you run it with -O or not:
try: assert 0 print "I am running with -O" except AssertionError: print "I'm not"
On Mon, 04 Apr 2005 19:59:20 -0400, Roy Smith wrote: > The following program produces different output depending on whether you > run it with -O or not:
> try: > assert 0 > print "I am running with -O" > except AssertionError: > print "I'm not"
But my question is whether it does anything *else*; if all -O does is remove asserts (and set __debug__), and you're (not you personally) complaining about -O removing asserts in testing code, the solution is to not use -O. There's only a real *problem* if -O does something *more*, such that you want one but not the other.
Up to this point, after years of Python use, I just ignore -O, because it's not like it accelerates code or anything. (And I don't work on embedded systems where my docstrings are a memory problem.) If everybody else is like me, then if we even do have "real" optimizations, then we'll actually need to use another switch.
Roy Smith <r...@panix.com> writes: > In article <424e73a3.100782...@news.oz.net>, b...@oz.net (Bengt Richter) > > Is there a package that is accessible without svn?
Peter Hansen <pe...@engcorp.com> writes: > Raymond Hettinger wrote: > > [Peter Hansen]
> >>This is pretty, but I *want* my tests to be contained > >>in separate functions or methods. > > In py.test, those would read: > > def test1(): > > assert a == b > > def test2(): > > raises(Error, func, args) > > Enclosing classes are optional.
> So basically py.test skips the import statement, > near as I can tell, at the cost of requiring a > utility to be installed in the PATH.
> Where was all that weight that unittest supposedly > has?
For PyPy we wanted to do some things that the designers of unittest obviously hadn't expected[1], such as formatting tracebacks differently. This was pretty tedious to do[2], involving things like accessing __variables, defining subclasses of certain classes, defining subclasses of other classes so the previous subclasses would actually get used, etc. I've not programmed in Java, but I imagine this is what it feels like all the time...
(Not to knock unittest too much, we did manage to get the customizations we needed done, but it wasn't fun).
Cheers, mwh
[1] this in itself is hardly a criticism: there are many special things about PyPy. [2] *this* is the criticism.
-- Well, you pretty much need Microsoft stuff to get misbehaviours bad enough to actually tear the time-space continuum. Luckily for you, MS Internet Explorer is available for Solaris. -- Calle Dybedahl, alt.sysadmin.recovery
Michael Hudson wrote: > Peter Hansen <pe...@engcorp.com> writes: >>Where was all that weight that unittest supposedly >>has?
> For PyPy we wanted to do some things that the designers of unittest > obviously hadn't expected[1], such as formatting tracebacks > differently. This was pretty tedious to do[2],
I'd agree with that! The guts of unittest I find to be somewhat opaque, and not straightforward to extend. Trying to do so here has also proven tedious, though ultimately feasible, as you say the PyPy folks have found.
Which, in the end, is precisely why I just use the simplest vanilla approach that I can, no different than anything done, for example, in the excellent chapter on testing in http://www.diveintopython.org or other examples of using unittest.
unitttest is surely not the be all and end all of Python unit testing frameworks... but it's one of the batteries included in the standard distribution, and it's pretty trivial to get started using it, unless maybe you try to go by the documentation instead of by the examples...
py.test is awesome, but there is one slight flaw in it. It produces to much output. All I want to see when all tests pass is "All X passes succeded!" (or something similar). py.test's output can be distracting.
<bjou...@gmail.com> wrote: >py.test is awesome, but there is one slight flaw in it. It produces to >much output. All I want to see when all tests pass is "All X passes >succeded!" (or something similar). py.test's output can be >distracting.
|
