XML output for gotest
xenolinguist
output test results in a way that is particularly friendly or useful
to most continuous integration tools, e.g. Hudson[1]. For my own
purposes, I'd very much like to have this feature, and I'm prepared to
put some serious effort into it myself. However, this seems like a
feature that might be useful to more folks than just me, so I thought
I'd gauge the interest from this group.
Basically, if this isn't something that's likely to be eventually
integrated into the main Go codebase, then a quick and nasty hack will
certainly suffice for my own needs. However, if there's enough
general interest that a feature patch might eventually get accepted,
then I'll "do it right" and try to code using the official Go coding
standards, and document the feature properly.
So: anybody want to vote for this feature?
Ian Lance Taylor
Can you outline what this feature would look like? I spent 30 seconds
looking at the Hudson web site, but I couldn't figure out what you are
proposing to do.
Ian
xenolinguist
> Can you outline what this feature would look like? I spent 30 seconds
> looking at the Hudson web site, but I couldn't figure out what you are
> proposing to do.
>
> Ian
At its core, what I'm looking to make is mostly a convenience feature
for integration into existing continuous build infrastructure(s). The
problem isn't specific to Hudson; the CruiseControl family of tools,
TeamCity, and other tools also lean heavily towards XML for the
reporting of failed tests in automated builds. (Likewise for compiler
output, but that's a very different problem I'm not even thinking
about.)
Having test results formatted in this manner enables them to be parsed
by the tool for a quick summary on a web interface or via email (or
other messaging) - this is a commonplace feature in the Continuous
Integration tooling in the Java, .NET, and Ruby ecosystems (et al.).
The quickest real-world examples I can find are from one an open-
source .NET project using TeamCity [1] and a Java/Eclipse project
using Hudson. [2] I want to be able to see gotest results in an
interface like this and with the off-the-shelf setups, this usually
requires outputting to XML files.
Just to reiterate: this isn't a "need" - grepping or equivalent
through console logs is still very possible. (See [3,4] for the
console logs from the previously-linked builds) And the go/gotest
equivalent of that output is indeed *much* friendlier to that grepping
than the average ant/junit (or equivalent) output. However, for my
other projects, when a build is broken with test failures, I can tell
at a glance what tests I need to examine. I'd just like to enable
this same convenience for my (and potentially other folks') Go
projects too.
n.b. - AFAIK, none of the relevant test runners in other ecosystems
defaults to outputting XML, so neither should/would gotest. This
feature would be controlled entirely by command line flags, with
sensible defaults (i.e. current behaviour).
[1] http://teamcity.codebetter.com/viewLog.html?buildId=8892&tab=buildResultsDiv&buildTypeId=bt65&guest=1
[2] https://build.eclipse.org/hudson/job/cbi-tm-3.2-nightly-ant/11/testReport
[3] http://teamcity.codebetter.com/viewLog.html?buildId=8892&buildTypeId=bt65&tab=buildLog&guest=1
[4] https://build.eclipse.org/hudson/job/cbi-tm-3.2-nightly-ant/11/console
Russ Cox
then it makes much more sense to use an encoding
designed for data structures, like JSON. That is already
used for machine-readable structures in other packages,
like the dumps exported from package expvar.
XML is for formatting documents, not for describing
data structures. The fact that it gets used often
for the latter doesn't lessen its fundamental inability
to do the job well.
Russ
Esko Luontola
servers, or does each testing framework use its own custom format? I
might consider adding that feature to GoSpec [1], although there are
still many other features with higher priority on my TODO list (if
it's important to you, you can implement it yourself). GoSpec supports
already now plugging in a custom printer - create an implementation of
the ResultVisitor interface [2], similar to the default printer [3],
and pass it to the test runner [4]. You might even be able to do it
already now, without changing GoSpec, by duplicating less than 10
lines of code from [4]. Gotest on the other hand would require
changing the testing.Main() method [5].
[1] http://github.com/orfjackal/gospec
[2] http://github.com/orfjackal/gospec/blob/release/src/gospec/results.go#L87-90
[3] http://github.com/orfjackal/gospec/blob/release/src/gospec/printer.go
[4] http://github.com/orfjackal/gospec/blob/release/src/gospec/main.go#L57
[5] http://golang.org/src/pkg/testing/testing.go#L160