In container/server tests

[Daniel R.]

Hi,
I am trying to implement in-container/server side tests, as suggested
by several posts on this forum. So I have a servlet which executes the
tests. Now I am wondering how to integrate the tests and results into
a normal TestNG execution flow from the out of container/client side.
So basically folks would be able to execute the same tests as the
usually do but somehow they would trigger that they would use the in-
container execution and then the results would be seamlessly reported
back to the user. So I guess my question is two fold:

- What is the best way to invoke the in-container tests from a regular
TestNG test? Maybe factories? Or just have a single dummy regular test
case which then invokes the servlet and the in-container tests?

- What is the best way to inject the results from the in-container/
server side into the actual TestNG execution on the client side? Right
now I am returning the contents of testng-results.xml from the
servlet, but not sure what is the best way to inject this into TestNG.

Hope the questions are clear. Any help is much appreciated. I hope
that other people who have tackled in container tests have cleared
these hurdles already.

Thanks

[Cédric Beust ♔]

Hi Daniel,

Quick and lazy answer:  we cover this topic extensively in our book (sorry for plugging it but hey, I don't do this very often :-)).

--

Cédric

[Daniel Rudman]

Ah yes, I have seen that. I suspected you might tell me to buy the book :)

Thanks

2009/6/29 Cédric Beust ♔ <cbe...@google.com>

[Daniel Rudman]

Hi Cedric,

So I have ordered your book and read through the relevant sections, but I am not sure if it exactly covers the scenario I am shooting for. You explain that you can hook in listeners/reports within the container and report those results from the container as an email, web page, etc. That is fine if you only have in-container tests. We will have a mix of both and I want both to be kicked off regularly like any out of container test.  So ideally I would like to receive the result back from the container (this part I am able to do) and inject those results into the regular test execution. Currently I just have a factory which runs the tests in the container and gets back the results, but does not return any regular out of container tests to run, so the results show up as 0 test run. Is there any way I can fake out TestNG to think that some test cases were run even though the factory does not return any tests and TestNG does not run them out of container?

Hope that makes sense and my question is clear.

Thanks

[Cédric Beust ♔]

Hi Daniel,

Sorry but I'm not quite following.

You have an instance of TestNG running inside your container which runs tests, and that part is working fine.  But now you want to communicate these results to a front-end TestNG?

--

Cédric

[Daniel Rudman]

Yeah, exactly. I want the front-end TestNG to show the container results just as if they were run on the front-end.

If you think this is feasible I would greatly appreciate any suggestions. If you think this is craziness, I guess I will just stick to reporting them from the container.

Thanks

2009/7/14 Cédric Beust ♔ <cbe...@google.com>

[Cédric Beust ♔]

Hi Daniel,

On Tue, Jul 14, 2009 at 4:26 PM, Daniel Rudman <dru...@gmail.com> wrote:

Yeah, exactly. I want the front-end TestNG to show the container results just as if they were run on the front-end.

If you think this is feasible I would greatly appreciate any suggestions. If you think this is craziness, I guess I will just stick to reporting them from the container.

No, it's not craziness, just a bit unusual :-)

However, I think it won't be too hard.

Part of the work to make TestNG distributed involved making the input and output data fully serializable, which includes ITestResult, so one way to achieve what you want would be to send these objects passed to the listeners in the container to the client.

Are you sure you really need this?  Wouldn't it be more convenient to just make the HTML results in the container accessible from the browser?

--

Cédric

[Daniel Rudman]

I am not sure I need this, that is there is no concrete issue right now which requires this approach, but I think that this is the preferable way to go, although it is harder. My thinking is that first there might be issues with our specific reporters (or any future reporters) if they run from the restricted container environment. Also it is easier having one report on the front-end if you have a mix of in-container and out-of-container tests. Lastly, since our tests run either from Eclipse or a fully automated zero-touch test system, so looking at a browser is not a good option.

I guess I would rather have two options available just in case.

Can you please point me in the right direction for using the distributed system? Would this be mimicing what the TetsNG Eclipse plugin does executing a remote TestNG?

Thanks

2009/7/14 Cédric Beust ♔ <cbe...@google.com>

Hi Daniel,

[Cédric Beust ♔]

On Tue, Jul 14, 2009 at 9:12 PM, Daniel Rudman <dru...@gmail.com> wrote:

Can you please point me in the right direction for using the distributed system? Would this be mimicing what the TetsNG Eclipse plugin does executing a remote TestNG?

I was not able to finish it, so it's not really usable, though, but this doesn't change the fact that all the Xml* and result classes are serializable, which is all you should need to communicate between two endpoints.

Eclipse uses a different, simpler protocol, which was written way before my distributed work.

--

Cédric