To give an example use-case where I would like to see it, this is a
function I have written with it in mind:
func AssertEquals(t *testing.T, a interface{}, b interface{}) {
if a != b {
t.IndirectlyLogf("expected %v to equal %v", a, b)
t.Fail()
}
}
// SetCaller fixes reporting location to be// level frames above the current frame.// The function returned reverts the reporting// location to the previous value.//// Example://// defer t.SetCaller(1)()func (t *T)SetCaller(level int) (func ())
AssertEquals provides no information, just a line number and whether
the assertion was true or false. This call
AssertEquals(t, foo, bar)
can, at most, say something like "At line x: fooval != barval". Add a
new arg to give context:
AssertEquals(t, "Quux(args)", foo, bar)
so now you have "at line x: Quux(args) == fooval, want barval" which
is a somewhat informative if rigidly formatted error message. And you
need to remember the order of arguments for your AssertEquals
function. This kind of design quickly spirals into dozens of AssertFoo
variations which aren't much fun to use.
Give me this any day:
if foo != bar {
t.Errorf("Quux(%q) == %q, want %q", args, foo, bar)
}
Unless you can propose a more palatable use case for IndirectlyLogf, I
don't think you'll get much traction with this idea in the core Go
testing package.
Andrew
On Thu, Feb 16, 2012 at 4:40 PM, Steven Degutis <sdeg...@8thlight.com> wrote:
> The current solution of the 'testing' package seems temporary, so when
> I proposed a solution, I didn't propose a completely ideal one, but
> rather one that allows us to keep using 'testing'. This change gives
> us relatively a lot of extra flexibility, with relatively little
> addition to the stdlib code, and it's completely isolated from the
> rest of the code.
>
> In the future though, I think we could introduce a more flexible
> testing package that uses interfaces instead for extensibility. But
> that's not pertinent to this proposed code addition, which will keep
> us going with good old 'pkg/testing' for quite a while I would think.
You might consider gocheck, which is more full-featured by design and
built atop the testing package:
It has matchers which let you extend it in interesting and novel ways
and comes with a set of basic things, so you can write code like:
c.Assert(foo, Equals, bar)
c.Assert(err, IsNil)
... and so on. Right now the only issue I have with it is that 'go
test' doesn't accept gocheck specific command-line arguments (at
least, not in golang-weekly 2012-02-07-1~11764~oneiric1 on Ubuntu),
but I guess that's a temporary one.
Thanks,
J.
> ... and so on. Right now the only issue I have with it is that 'go
> test' doesn't accept gocheck specific command-line arguments (at
> least, not in golang-weekly 2012-02-07-1~11764~oneiric1 on Ubuntu),
> but I guess that's a temporary one.
I don't think the command line arguments ever stopped working. What's
the issue you're facing there?
--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/plus
http://niemeyer.net/twitter
http://niemeyer.net/blog
-- I'm not absolutely sure of anything.
> AssertEquals provides no information, just a line number and whether
> the assertion was true or false. This call
>
> AssertEquals(t, foo, bar)
>
> can, at most, say something like "At line x: fooval != barval". Add a
This is somewhat of a pessimistic view on what one can do.
Given this:
func Quux(s string) string { return "baz" }
func (S) TestDetails(c *C) {
c.Assert(Quux("foo"), Equals, "bar")
}
This is the real output from gocheck
----------------------------------------------------------------------
FAIL: foo_test.go:21: S.TestDetails
foo_test.go:22:
c.Assert(Quux("foo"), Equals, "bar")
... obtained string = "baz"
... expected string = "bar"
As stated, the testing package follows different principles, but you
can use gocheck today for that:
Easier to understand than an if statement? Really?
Andrew
http://golang.org/doc/go_faq.html#assertions is relevant, if you
haven't already read it.
Another problem with things like assert.Equal is that it's hard to
agree on what 'equals' means: shallow versus deep, slice/map
references versus contents, tolerance margins for floating point,
dealing with NaN, type differences (is myInt(3) 'equal' to
yourInt(3))? I have already seen a Go assertion package that converted
interface{} to string before comparing, so that int32(3) was 'equal'
to float64(3.0) was 'equal' to the string "3". That's a perfectly
consistent concept of 'equality', but it is a surprising one, whose
semantics are not obvious just from the call site: assert.Equal(foo,
bar). Future maintainers might inadvertently miss false positives.
In comparison, the meaning of '==' is defined by the language
specification, and is something that all Go programmers should be
familiar with. If I need to check a more exotic predicate, I already
have a perfectly good language to describe it: the Go programming
language.
Please try using "if foo != bar" instead of assert.Equal(foo, bar).
Equal(s) should mean the same as ==. Anything else is obviously incoherent.
This is something I got wrong early on with gocheck (it does
reflect.DeepEqual), but that will be fixed in the next release.
On 17 February 2012 14:04, liigo <com....@gmail.com> wrote:
> +1 for AssertEqual(). It's much easier to use (less code to type) andhttp://golang.org/doc/go_faq.html#assertions is relevant, if you
> to understand,
haven't already read it.
Another problem with things like assert.Equal is that it's hard to
agree on what 'equals' means: shallow versus deep, slice/map
references versus contents, tolerance margins for floating point,
dealing with NaN, type differences (is myInt(3) 'equal' to
yourInt(3))?
I have already seen a Go assertion package that converted
interface{} to string before comparing, so that int32(3) was 'equal'
to float64(3.0) was 'equal' to the string "3". That's a perfectly
consistent concept of 'equality', but it is a surprising one, whose
semantics are not obvious just from the call site: assert.Equal(foo,
bar). Future maintainers might inadvertently miss false positives.
In comparison, the meaning of '==' is defined by the language
specification, and is something that all Go programmers should be
familiar with.
If I need to check a more exotic predicate, I already
have a perfectly good language to describe it: the Go programming
language.
Please try using "if foo != bar" instead of assert.Equal(foo, bar).
The second paragraph of that FAQ entry starts with: "The same
arguments apply to the use of assert() in test programs." That
paragraph applies equally well to any AssertEquals proposals.
You misunderstand my point. I'm not saying that a built-in assert
function, if it existed, should be used for testing. I am saying that
replacing a language-provided "assert()" with a package-provided
"AssertEquals" in that paragraph gives a relevant argument:
The same arguments apply to the use of AssertEquals in test programs.
Proper error handling means letting other tests run after one has
failed, so that the person debugging the failure gets a complete
picture of what is wrong. It is more useful for a test to report that
isPrime gives the wrong answer for 2, 3, 5, and 7 (or for 2, 4, 8, and
16) than to report that isPrime gives the wrong answer for 2 and
therefore no more tests were run. The programmer who triggers the test
failure may not be familiar with the code that fails. Time invested
writing a good error message now pays off later when the test breaks.
+1 for AssertEqual(). It's much easier to use (less code to type) and
to understand, it has been widely used in other languages / test-
frameworks, such as JUnit and TestNG.
Hey Andrew,> AssertEquals provides no information, just a line number and whether
> the assertion was true or false. This call
>
> AssertEquals(t, foo, bar)
>
> can, at most, say something like "At line x: fooval != barval". Add aThis is somewhat of a pessimistic view on what one can do.
Given this:
func Quux(s string) string { return "baz" }
func (S) TestDetails(c *C) {
c.Assert(Quux("foo"), Equals, "bar")
}This is the real output from gocheck
----------------------------------------------------------------------
FAIL: foo_test.go:21: S.TestDetailsfoo_test.go:22:
c.Assert(Quux("foo"), Equals, "bar")
... obtained string = "baz"
... expected string = "bar"
On Thu, Feb 16, 2012 at 10:43 PM, Gustavo Niemeyer <gus...@niemeyer.net> wrote:
>> ... and so on. Right now the only issue I have with it is that 'go
>> test' doesn't accept gocheck specific command-line arguments (at
>> least, not in golang-weekly 2012-02-07-1~11764~oneiric1 on Ubuntu),
>> but I guess that's a temporary one.
>
> I don't think the command line arguments ever stopped working. What's
> the issue you're facing there?
Oh, awesome, this turned out to be a case of EPEBKAC. I was passing
-gocheck.v instead of --gocheck.v. I guess I was confused by the 'go
test helpflag' documentation, because the builtin flags all use a
single dash. Anyway, after you said things should work I tried it and
typed two dashes instinctively and it Just Worked(tm)!
You fix bugs without writing code... it's like magic. ;)
Take care,
J.
With gocheck you can do this instead to do non-terminating verifications:
c.Check(Quux("foo"), Equals, "bar")
> and not forbidding the old way: if a!=b {t.Errorf()} else {}
Right, it allows you to do that as well, when it's more appropriate.
Stop arguing with them and go test some software. :)
+1 for having Equals() mean "==".
A possible extension: have Equals() check equality of structures /
references for Go's referenced types, i.e. slice & map. (This respects
Go's ref semantics for those types. It may be useful that Go defines
"==" as meaning precisely this for slice & map. For slice, this
translates to equality of pointer & len (cap left out). Cheap and useful.)
Denis
On Fri, Feb 17, 2012 at 01:04, liigo wrote:
> +1 for AssertEqual(). It's much easier to use (less code to type) and
> to understand, it has been widely used in other languages / test-
> frameworks, such as JUnit and TestNG. I would like to see AsserEqual()
> adding to testing pkg, please. Thanks.As stated, the testing package follows different principles, but you
can use gocheck today for that:
I completely agree about usefulness of full information reports for
testing --more precisely for diagnosis at development (or modification)
time. Thus, testing checks should not behave like typical asserts which
just stop there. Moreover, they should extensively report on each
failure found, and even possibly on successful checks (eg explicitely
knowing for which combinations of values a bugged func works as expected
is often as useful as knowing for which it fails.)
Things are quite different for regression/maintenance tests, where one
just wants to know whether everything (still) works fine after
compilation on a given system, or after changes in (seemingly) unrelated
parts of an app. In this case, all tests should still run (no break
after a failing check) but information should be minimal. Then a
developper could quickly choose which precise test or testsuite launch,
this time in diagnosis mode.
My dream testing check func(s) would thus have 2 modes, one verbose for
dev/diagnosis, one nearly silent for regression/maintenance. But in both
cases check failures would cause output and go on the tests.
Denis
I would love a compiler-assisted testing check func like:
check(square(-3), 9)
printing on failure:
*** Test Error: check failure in package Foo, function testSquare, l 123
expression : square(3)
expected : 9
result : -9
It needs I guess compiler assistance to give back the original
expression; possibly (haven't yet explored reflection capabilities of
Go) for exact location of call to check().
In diagnosis mode, it may also print out on success succintly:
* check: square(-9) --> 9
Denis
That's great!
Would you explain (or point to relevant code/doc) how you obtain the
original expressoin? (possibly off list)
Thank you,
Denis
Denis
That's what DeepEquals does, and it will continue to exist with that
name. Equals should choke on that since == will too.
Using the go/* packages:
http://weekly.golang.org/pkg/go/
I agree. Let's all go back to CVS.
Hum, I'll explore your package soon. Bit if DeepEquals performs deep
equality as commonly understood, that's not what I meant, rather the
opposite: for slices and maps, just check equality of structures
wrapping the contents, and stops there (no equality xhexk of contents,
even less recusively). It's analog to equality of object refs.
Maybe that's what DeepEquals does as you sat, then great! but if so the
name of the func is /for me/ misleading.
Denis
It's not just analogous. That's exactly what it is: &a == &b works if
both a and b are slices.
This is a good use case for table-styled tests:
var tests := []struct{s string; e Expr}{
{"(foo bar)", NewList(NewSymbol("foo"), NewSymbol("bar")},
}
Put as many entries as you want, write one short for loop.
> I should mention that I wrote the gobdd lib, which I prefer over
> gocheck or any other third party lib for unit testing in Go. But the
> goal I'm reaching for in proposing this change (or at least some
> extensibility addition!) is to make use of the built-in testing
> package, instead of moving away from it.
>
> The aim here is to reduce duplication as much as possible in my unit
> tests, while making it easy to see what failed and where in my console
> output. Anything which does this, I'm fine with.
package testing sounds like just the ticket.
-rob
Yeah, table-driven testing FTW!
Seriously I think launchpad.net/gocheck works very well, and although it does make it easier to write long-winded tests, it also relieves some of the tedium of writing any tests, table-driven or not.
Assert and Check print the values that were compared and the code context, and that's often the only thing you need to diagnose the problem.
I'm not sure about the idiom of importing gocheck to '.' though. I'm not sure I want it *that* special.
c.Assert(err, C.IsNil)
would read Ok to me if gocheck imported as C.
If you tease out the *testing.T arg, and reduce it to
func match(error, string) bool
then you don't need to off-by-one the call stack and double-print line
numbers, and you can also control whether or not a mismatch is fatal:
if err := doSomething(); !match(err, testcase.want) {
t.Errorf("doSomething: got %v want %v", err, testcase.want)
}
AssertEqual()-like functions with a external configuration (maybe a cmd's arg/flag, an XML or other), can do this well. Just as junit/testng did.
But the currently pkg/testing's if-statement-like tests only report information when test fails. So user don't know explicitly weather or not the tests did really done, if not fails.
Lingo Zhuang
In assertion-based tests, you can use table-drive too; in if-statement-based tests, you can copy-paste codes too. NO one tends to write bad codes.
The fact that the file:line information in the testing errors
discourages wrappers like this is one of the best things
about them. It is second only to the fact that they produce
useful information that makes debugging broken tests easier.
Russ
AssertEqual()-like functions with a external configuration (maybe a cmd's arg/flag, an XML or other), can do this well. Just as junit/testng did.
But the currently pkg/testing's if-statement-like tests only report information when test fails. So user don't know explicitly weather or not the tests did really done, if not fails.
Not the same things. Most tests in std pkgs, are coded as: if a != b { reports errors } No else-statement, so if a == b, nothing information will be shown.
在 2012-2-19 下午3:15,"minux" <minu...@gmail.com>写道:
>
>
> On Sat, Feb 18, 2012 at 11:39 AM, Liigo Zhuang <com....@gmail.com> wrote:
>>
>> AssertEqual()-like functions with a external configuration (maybe a cmd's arg/flag, an XML or other), can do this well. Just as junit/testng did.
>
> I think the discussion is not about capability (I'm pretty sure these two approaches are equivalent), but about (testing) style.
AssertEqual() do the same things (or a little more) with the same style, it do let users write less codes.
> Not the same things. Most tests in std pkgs, are coded as: if a != b {
> reports errors } No else-statement, so if a == b, nothing information will
> be shown.
Passing tests are not particularly interesting, and (hopefully) there
will be a great many of them, so showing no information is the right
choice.
Failing tests are interesting, so showing something is the right choice
for them.
Chris
--
Chris "allusive" Dollin
This is true for regression testing only, where you only want to check
that/whether all (still) works fine. The field of testing is, imo, far
wider than that. During development, i want tests to provide me with all
relevant info telling me how the presently considered/developped piece
of code works, or helping me and diagnose issues I have with it. Passing
checks are as informative as failing ones.
Denis
Just as Danis said, sometimes we need that information, and sometimes not. I don't want to change the test source code every times. AssertEqual() + external configure can do this well.
Hence -test.v
> 在 2012-2-20 上午3:23,"Jan Mercl" <jan....@nic.cz>写道:
>>
>> Use t.Log("passed X") and -test.v please.
Chris
--
Chris "allusive" Dollin
在 2012-2-20 下午9:58,"chris dollin" <ehog....@googlemail.com>写道:
>
> On 20 February 2012 13:54, Liigo Zhuang <com....@gmail.com> wrote:
> > Just as Danis said, sometimes we need that information, and sometimes not. I
> > don't want to change the test source code every times.
>
> Hence -test.v
At least 3 lines code to be add: else-statement.
V.S.
AssertEqual() still that one line with any change.
if a != b {
t.Error(...)
} else {
t.Log(...)
}
VS.
AssertEqual(a, b, ...)
So the change I'm proposing is one new and isolated function in
'testing':
func (c *T) IndirectlyLogf(format string, args ...interface{})
This function would be identical to Logf() except it would print out
the file:line one higher in the stack trace than its existing
counterpart.