Hi,
I am trying to implement in-container/server side tests, as suggested
by several posts on this forum. So I have a servlet which executes the
tests. Now I am wondering how to integrate the tests and results into
a normal TestNG execution flow from the out of container/client side.
So basically folks would be able to execute the same tests as the
usually do but somehow they would trigger that they would use the in-
container execution and then the results would be seamlessly reported
back to the user. So I guess my question is two fold:
- What is the best way to invoke the in-container tests from a regular
TestNG test? Maybe factories? Or just have a single dummy regular test
case which then invokes the servlet and the in-container tests?
- What is the best way to inject the results from the in-container/
server side into the actual TestNG execution on the client side? Right
now I am returning the contents of testng-results.xml from the
servlet, but not sure what is the best way to inject this into TestNG.
Hope the questions are clear. Any help is much appreciated. I hope
that other people who have tackled in container tests have cleared
these hurdles already.
On Mon, Jun 29, 2009 at 3:03 PM, Daniel R. <drud...@gmail.com> wrote:
> Hi,
> I am trying to implement in-container/server side tests, as suggested
> by several posts on this forum. So I have a servlet which executes the
> tests. Now I am wondering how to integrate the tests and results into
> a normal TestNG execution flow from the out of container/client side.
> So basically folks would be able to execute the same tests as the
> usually do but somehow they would trigger that they would use the in-
> container execution and then the results would be seamlessly reported
> back to the user. So I guess my question is two fold:
> - What is the best way to invoke the in-container tests from a regular
> TestNG test? Maybe factories? Or just have a single dummy regular test
> case which then invokes the servlet and the in-container tests?
> - What is the best way to inject the results from the in-container/
> server side into the actual TestNG execution on the client side? Right
> now I am returning the contents of testng-results.xml from the
> servlet, but not sure what is the best way to inject this into TestNG.
> Hope the questions are clear. Any help is much appreciated. I hope
> that other people who have tackled in container tests have cleared
> these hurdles already.
> On Mon, Jun 29, 2009 at 3:03 PM, Daniel R. <drud...@gmail.com> wrote:
>> Hi,
>> I am trying to implement in-container/server side tests, as suggested
>> by several posts on this forum. So I have a servlet which executes the
>> tests. Now I am wondering how to integrate the tests and results into
>> a normal TestNG execution flow from the out of container/client side.
>> So basically folks would be able to execute the same tests as the
>> usually do but somehow they would trigger that they would use the in-
>> container execution and then the results would be seamlessly reported
>> back to the user. So I guess my question is two fold:
>> - What is the best way to invoke the in-container tests from a regular
>> TestNG test? Maybe factories? Or just have a single dummy regular test
>> case which then invokes the servlet and the in-container tests?
>> - What is the best way to inject the results from the in-container/
>> server side into the actual TestNG execution on the client side? Right
>> now I am returning the contents of testng-results.xml from the
>> servlet, but not sure what is the best way to inject this into TestNG.
>> Hope the questions are clear. Any help is much appreciated. I hope
>> that other people who have tackled in container tests have cleared
>> these hurdles already.
Hi Cedric,So I have ordered your book and read through the relevant
sections, but I am not sure if it exactly covers the scenario I am shooting
for. You explain that you can hook in listeners/reports within the container
and report those results from the container as an email, web page, etc. That
is fine if you only have in-container tests. We will have a mix of both and
I want both to be kicked off regularly like any out of container test. So
ideally I would like to receive the result back from the container (this
part I am able to do) and inject those results into the regular test
execution. Currently I just have a factory which runs the tests in the
container and gets back the results, but does not return any regular out of
container tests to run, so the results show up as 0 test run. Is there any
way I can fake out TestNG to think that some test cases were run even though
the factory does not return any tests and TestNG does not run them out of
container?
On Mon, Jun 29, 2009 at 3:54 PM, Daniel Rudman <drud...@gmail.com> wrote:
> Ah yes, I have seen that. I suspected you might tell me to buy the book :)
> Thanks
>> On Mon, Jun 29, 2009 at 3:03 PM, Daniel R. <drud...@gmail.com> wrote:
>>> Hi,
>>> I am trying to implement in-container/server side tests, as suggested
>>> by several posts on this forum. So I have a servlet which executes the
>>> tests. Now I am wondering how to integrate the tests and results into
>>> a normal TestNG execution flow from the out of container/client side.
>>> So basically folks would be able to execute the same tests as the
>>> usually do but somehow they would trigger that they would use the in-
>>> container execution and then the results would be seamlessly reported
>>> back to the user. So I guess my question is two fold:
>>> - What is the best way to invoke the in-container tests from a regular
>>> TestNG test? Maybe factories? Or just have a single dummy regular test
>>> case which then invokes the servlet and the in-container tests?
>>> - What is the best way to inject the results from the in-container/
>>> server side into the actual TestNG execution on the client side? Right
>>> now I am returning the contents of testng-results.xml from the
>>> servlet, but not sure what is the best way to inject this into TestNG.
>>> Hope the questions are clear. Any help is much appreciated. I hope
>>> that other people who have tackled in container tests have cleared
>>> these hurdles already.
You have an instance of TestNG running inside your container which runs
tests, and that part is working fine. But now you want to communicate these
results to a front-end TestNG?
On Tue, Jul 14, 2009 at 4:15 PM, Daniel Rudman <drud...@gmail.com> wrote:
> Hi Cedric,So I have ordered your book and read through the relevant
> sections, but I am not sure if it exactly covers the scenario I am shooting
> for. You explain that you can hook in listeners/reports within the container
> and report those results from the container as an email, web page, etc. That
> is fine if you only have in-container tests. We will have a mix of both and
> I want both to be kicked off regularly like any out of container test. So
> ideally I would like to receive the result back from the container (this
> part I am able to do) and inject those results into the regular test
> execution. Currently I just have a factory which runs the tests in the
> container and gets back the results, but does not return any regular out of
> container tests to run, so the results show up as 0 test run. Is there any
> way I can fake out TestNG to think that some test cases were run even though
> the factory does not return any tests and TestNG does not run them out of
> container?
> Hope that makes sense and my question is clear.
> Thanks
> On Mon, Jun 29, 2009 at 3:54 PM, Daniel Rudman <drud...@gmail.com> wrote:
>> Ah yes, I have seen that. I suspected you might tell me to buy the book :)
>> Thanks
>>> On Mon, Jun 29, 2009 at 3:03 PM, Daniel R. <drud...@gmail.com> wrote:
>>>> Hi,
>>>> I am trying to implement in-container/server side tests, as suggested
>>>> by several posts on this forum. So I have a servlet which executes the
>>>> tests. Now I am wondering how to integrate the tests and results into
>>>> a normal TestNG execution flow from the out of container/client side.
>>>> So basically folks would be able to execute the same tests as the
>>>> usually do but somehow they would trigger that they would use the in-
>>>> container execution and then the results would be seamlessly reported
>>>> back to the user. So I guess my question is two fold:
>>>> - What is the best way to invoke the in-container tests from a regular
>>>> TestNG test? Maybe factories? Or just have a single dummy regular test
>>>> case which then invokes the servlet and the in-container tests?
>>>> - What is the best way to inject the results from the in-container/
>>>> server side into the actual TestNG execution on the client side? Right
>>>> now I am returning the contents of testng-results.xml from the
>>>> servlet, but not sure what is the best way to inject this into TestNG.
>>>> Hope the questions are clear. Any help is much appreciated. I hope
>>>> that other people who have tackled in container tests have cleared
>>>> these hurdles already.
Yeah, exactly. I want the front-end TestNG to show the container results
just as if they were run on the front-end.
If you think this is feasible I would greatly appreciate any suggestions. If
you think this is craziness, I guess I will just stick to reporting them
from the container.
> You have an instance of TestNG running inside your container which runs
> tests, and that part is working fine. But now you want to communicate these
> results to a front-end TestNG?
> --
> ***Cédric
> *
> On Tue, Jul 14, 2009 at 4:15 PM, Daniel Rudman <drud...@gmail.com> wrote:
>> Hi Cedric,So I have ordered your book and read through the relevant
>> sections, but I am not sure if it exactly covers the scenario I am shooting
>> for. You explain that you can hook in listeners/reports within the container
>> and report those results from the container as an email, web page, etc. That
>> is fine if you only have in-container tests. We will have a mix of both and
>> I want both to be kicked off regularly like any out of container test. So
>> ideally I would like to receive the result back from the container (this
>> part I am able to do) and inject those results into the regular test
>> execution. Currently I just have a factory which runs the tests in the
>> container and gets back the results, but does not return any regular out of
>> container tests to run, so the results show up as 0 test run. Is there any
>> way I can fake out TestNG to think that some test cases were run even though
>> the factory does not return any tests and TestNG does not run them out of
>> container?
>> Hope that makes sense and my question is clear.
>> Thanks
>> On Mon, Jun 29, 2009 at 3:54 PM, Daniel Rudman <drud...@gmail.com> wrote:
>>> Ah yes, I have seen that. I suspected you might tell me to buy the book
>>> :)
>>> Thanks
>>>> On Mon, Jun 29, 2009 at 3:03 PM, Daniel R. <drud...@gmail.com> wrote:
>>>>> Hi,
>>>>> I am trying to implement in-container/server side tests, as suggested
>>>>> by several posts on this forum. So I have a servlet which executes the
>>>>> tests. Now I am wondering how to integrate the tests and results into
>>>>> a normal TestNG execution flow from the out of container/client side.
>>>>> So basically folks would be able to execute the same tests as the
>>>>> usually do but somehow they would trigger that they would use the in-
>>>>> container execution and then the results would be seamlessly reported
>>>>> back to the user. So I guess my question is two fold:
>>>>> - What is the best way to invoke the in-container tests from a regular
>>>>> TestNG test? Maybe factories? Or just have a single dummy regular test
>>>>> case which then invokes the servlet and the in-container tests?
>>>>> - What is the best way to inject the results from the in-container/
>>>>> server side into the actual TestNG execution on the client side? Right
>>>>> now I am returning the contents of testng-results.xml from the
>>>>> servlet, but not sure what is the best way to inject this into TestNG.
>>>>> Hope the questions are clear. Any help is much appreciated. I hope
>>>>> that other people who have tackled in container tests have cleared
>>>>> these hurdles already.
On Tue, Jul 14, 2009 at 4:26 PM, Daniel Rudman <drud...@gmail.com> wrote: > Yeah, exactly. I want the front-end TestNG to show the container results > just as if they were run on the front-end. > If you think this is feasible I would greatly appreciate any suggestions. > If you think this is craziness, I guess I will just stick to reporting them > from the container.
No, it's not craziness, just a bit unusual :-)
However, I think it won't be too hard.
Part of the work to make TestNG distributed involved making the input and output data fully serializable, which includes ITestResult, so one way to achieve what you want would be to send these objects passed to the listeners in the container to the client.
Are you sure you really need this? Wouldn't it be more convenient to just make the HTML results in the container accessible from the browser?
I am not sure I need this, that is there is no concrete issue right now
which requires this approach, but I think that this is the preferable way to
go, although it is harder. My thinking is that first there might be issues
with our specific reporters (or any future reporters) if they run from the
restricted container environment. Also it is easier having one report on the
front-end if you have a mix of in-container and out-of-container tests.
Lastly, since our tests run either from Eclipse or a fully automated
zero-touch test system, so looking at a browser is not a good option.
I guess I would rather have two options available just in case.
Can you please point me in the right direction for using the distributed
system? Would this be mimicing what the TetsNG Eclipse plugin does executing
a remote TestNG?
> On Tue, Jul 14, 2009 at 4:26 PM, Daniel Rudman <drud...@gmail.com> wrote:
>> Yeah, exactly. I want the front-end TestNG to show the container results
>> just as if they were run on the front-end.
>> If you think this is feasible I would greatly appreciate any suggestions.
>> If you think this is craziness, I guess I will just stick to reporting them
>> from the container.
> No, it's not craziness, just a bit unusual :-)
> However, I think it won't be too hard.
> Part of the work to make TestNG distributed involved making the input and
> output data fully serializable, which includes ITestResult, so one way to
> achieve what you want would be to send these objects passed to the listeners
> in the container to the client.
> Are you sure you really need this? Wouldn't it be more convenient to just
> make the HTML results in the container accessible from the browser?
On Tue, Jul 14, 2009 at 9:12 PM, Daniel Rudman <drud...@gmail.com> wrote: > Can you please point me in the right direction for using the distributed > system? Would this be mimicing what the TetsNG Eclipse plugin does executing > a remote TestNG?
I was not able to finish it, so it's not really usable, though, but this doesn't change the fact that all the Xml* and result classes are serializable, which is all you should need to communicate between two endpoints.
Eclipse uses a different, simpler protocol, which was written way before my distributed work.
|
