I noticed that TestNG does not capture the output to System.out and
System.err unlike JUnit. Those output are often very convenient when
you are looking at test failures.
I can work around the problem to some extent by doing
System.setOut/setErr myself, but:
1. it doesn't work well if I run tests in parallel
2. it's tedious
Is there any reason why TestNG can't do that for us?
In JUnit, http://parallel-junit.dev.java.net/ can capture output done
by multiple threads.
The thing is, it's not that my test code is trying to write to
System.out.
For example, some of my tests call into Javac, which produces messages
on System.out. Or my tests call the schema compiler of JAXB, which does
something similar.
Or I might be launching a web container like Tomcat through Cargo, and
then Tomcat writes a whole bunch of stuff into System.out.
Authors of those respective libraries is not wrong in writing output to
System.out. They do mean to "print this on the standard output". But
when I integrate those libraries as dependencies into a test, I want
the output to be captured into my test execution log.
So given this reality, what is my option?
I'm afraid I'm not quite following when you say I'd "still have the
problem of rogue output". If I run it outside a test framework, all the
output goes System.out and that's it. I don't see anything wrong with
that.
> Now, to your question: why not implement your own suggestion and set
> System.out and System.err before starting your system?
Yeah, if I can do it without changing TestNG, I think I'm OK with it.
But there are a few tricky aspects.
First, when tests are to be run in parallel, we can't have each test
individually set and restore System.out and System.err. They need to be
installed once before parallel execution starts, then have each thread
co-operatively use it, and finally restored back to the original value
when the parallel execution ends. I don't know what the place to do
this in TestNG.
Hmm... maybe I can do that before and after I call into
org.testng.TestNG.run()? from my custom PrintWriter I might be able to
send them to Reporter.log().
(BTW I'm looking at the Reporter class and I don't think this is
thread-safe. For example, setCurrentTestResult has just one field,
instead of ThreadLocal.)
Second, err... I think that might be the only issue.
Thank you very much for all your help. (I'd be very grateful if you
could take a look at another e-mail I posted yesterday, too)
Hmm... maybe I can do that before and after I call into
org.testng.TestNG.run()? from my custom PrintWriter I might be able to
send them to Reporter.log().
(BTW I'm looking at the Reporter class and I don't think this is
thread-safe. For example, setCurrentTestResult has just one field,
instead of ThreadLocal.)
Second, err... I think that might be the only issue.
Thank you very much for all your help. (I'd be very grateful if you
could take a look at another e-mail I posted yesterday, too)
Or is it fixed already :-) :-) ?
Would you like me to send it to this list, or create a branch and
commit it there for your review? You guidance will be appreciated.