Attaching TestNG listeners via Annotation?

[S M Humayun]

Is there a way to attach one or more TestNG listeners via annotations?
I've been forced to create a testng.xml file just because I had to
attach a listener otherwise annotations were doing just fine for me.

--------------------------------------------
S M Humayun
Software Architect, IBL Group
--------------------------------------------
Email : smhu...@iblgrp.com
MSN : smhuma...@hotmail.com
Skype : smhumayun
Cell : 92.321.8707783
Blog : http://smhumayun.blogspot.com
Link : http://www.linkedin.com/in/smhumayun
Facebook : http://www.facebook.com/smhumayun
Twitter : http://twitter.com/smhumayun
--------------------------------------------

[Cédric Beust ♔]

On Thu, Nov 5, 2009 at 2:05 AM, S M Humayun <smhu...@gmail.com> wrote:

Is there a way to attach one or more TestNG listeners via annotations?
I've been forced to create a testng.xml file just because I had to
attach a listener otherwise annotations were doing just fine for me.

No, you have to use either XML, the command line or ant.

Annotations are not a very good match for this because listeners are global to TestNG while annotations have to be tied to a specific test class, which doesn't make a lot of sense.

Another thing you can do is write your own TestNG wrapper (in Java) and specify your annotation on the TestNG object there before invoking run().

--

Cédric

[S M Humayun]

Why all annotations have to be tied to a specific "test" class? can't
we have @Listener annotation for those "non test" classes which
implements one or more of the listener interfaces? making sure that
the annotations defined in those classes have a "global" impact on
testing environment or context?

--------------------------------------------
S M Humayun
Software Architect, IBL Group
--------------------------------------------
Email : smhu...@iblgrp.com
MSN : smhuma...@hotmail.com
Skype : smhumayun
Cell : 92.321.8707783
Blog : http://smhumayun.blogspot.com
Link : http://www.linkedin.com/in/smhumayun
Facebook : http://www.facebook.com/smhumayun
Twitter : http://twitter.com/smhumayun
--------------------------------------------

2009/11/5 Cédric Beust ♔ <cbe...@google.com>:

[Cédric Beust ♔]

On Thu, Nov 5, 2009 at 8:06 AM, S M Humayun <smhu...@gmail.com> wrote:

Why all annotations have to be tied to a specific "test" class? can't
we have @Listener annotation for those "non test" classes which
implements one or more of the listener interfaces? making sure that
the annotations defined in those classes have a "global" impact on
testing environment or context?

I was talking about the way you tell TestNG about a listener, and it seems to me that an annotation is not the best match for that.  Where would you put that annotation?  On a class?  Which one, a test clas or a normal class?  Why pick one test class over another one to specify a setting that's global in nature?  And finally, how do you tell TestNG about that class that contains the listener annotation or that is a listener?  You will probably have to use an XML file, so we're back to square one.

Having said that, what you're asking is possible:  we could imagine just launching TestNG with a bunch of classes and TestNG can determine which ones are tests and which ones are listeners.

It's just that XML files become pretty much necessary once your test base grows and you need to specify additional parameters or group your tests logically, so I have never really felt the need to extend the command line syntax past a certain level of complexity...

--

Cédric

[S M Humayun]

Quote - we could imagine just launching TestNG with a bunch of classes

and TestNG can determine which ones are tests and which ones are

listeners - Unquote

Thats exactly what I was thinking and in this case we don't even need
any @listener annotation, presence of interface is more than enough to
sense that class is indeed a testng listener.

Thanx for the clarifications.

--------------------------------------------
S M Humayun
Software Architect, IBL Group
--------------------------------------------
Email : smhu...@iblgrp.com
MSN : smhuma...@hotmail.com
Skype : smhumayun
Cell : 92.321.8707783
Blog : http://smhumayun.blogspot.com
Link : http://www.linkedin.com/in/smhumayun
Facebook : http://www.facebook.com/smhumayun
Twitter : http://twitter.com/smhumayun
--------------------------------------------





2009/11/5 Cédric Beust ♔ <cbe...@google.com>:
>
>

[borojevic]

Hi,
I have around 1500-2000 tests that needs to be maintained.
I specify customized listeners in suite files or when I run the suite
with ant.
Having control on listeners is much better for me then allowing anyone
in my team to specify listener just by annotating some class.
Bad implementation of listener interface, especially ITestListener
could make problem during the execution of large base of tests.

Listeners are not something you do every day and I don't see very good
scenario where you need to create them with annotation.

On Nov 6, 5:42 am, S M Humayun <smhuma...@gmail.com> wrote:
> Quote - we could imagine just launching TestNG with a bunch of classes
> and TestNG can determine which ones are tests and which ones are
> listeners - Unquote
>
> Thats exactly what I was thinking and in this case we don't even need
> any @listener annotation, presence of interface is more than enough to
> sense that class is indeed a testng listener.
>
> Thanx for the clarifications.
>
> --------------------------------------------
> S M Humayun
> Software Architect, IBL Group
> --------------------------------------------

> Email : smhuma...@iblgrp.com
> MSN   : smhumayun2...@hotmail.com

[S M Humayun]

Hi borojevic,

You've got a point and may be I'm wrong but shouldn't code maintenance
and governance be handled via other means? for example, what would you
do if one of your team member add one or more listeners to your
testng.xml file? how would you detect this? and more importantly how
would you protect this?

--------------------------------------------
S M Humayun
Software Architect, IBL Group
--------------------------------------------

Email : smhu...@iblgrp.com
MSN : smhuma...@hotmail.com

[Cédric Beust ♔]

On Fri, Nov 6, 2009 at 10:46 PM, S M Humayun <smhu...@gmail.com> wrote:

Hi borojevic,

You've got a point and may be I'm wrong but shouldn't code maintenance
and governance be handled via other means? for example, what would you
do if one of your team member add one or more listeners to your
testng.xml file? how would you detect this? and more importantly how
would you protect this?

This is  an orthogonal issue, but  if that developer can modify the testng.xml file, they can probably just as well modify a source file.

There are several ways to address this problem, code reviews being probably the one that makes most sense (obviously, code reviews have a lot of other benefits).

Back to the annotation discussion, my rule of thumb to use an annotation is:  "Does this information apply to a specific Java element, such as a package, class, method, field or variable?".

If the answer is yes, an annotation is probably a good choice since using another means (e.g. XML file) will force you to describe which Java element to apply this information to, and you end up with monstrosities such as:

<package name="com.example">
  <class name="A">
    <method name="foo">
     <... the metadata>

Since TestNG listeners are global in nature, I think they should not be described in an annotation.

An argument could be made to have listeners that only apply to certain test classes, and in this case, an annotation would be more appropriate, but I don't think anyone has ever requested such a thing.

--

Cédric

[borojevic]

Hi Humayun,
I can not protect testng.xml from being modified, but I could ask the
svn admin to give me only exclusive rights on these files.

testng.xml is actually modified all the time, because developers are
adding tests all the time, but nobody is adding the listener and there
is
no need to add listener frequently.
If anybody modifies this file I will see the modification as we drive
tests from cruisecontrol and list of changes are always presented.
Anyway I have a quick look over the tests and suite files every
morning and that takes some 10-15 min.
I prefer to implement interface ITestListener or IReporter and
explicitly specify that I am going to use it, either in suite file,
ant, command line,
then doing this via annotation.
Via annotation my listener would be hidden in large base of test
classes and it would be quite difficult to troubleshoot if anything
goes wrong and it usually goes wrong when you start write listener for
the first time...
And one more thing to add, nobody is interested in my team to write
listeners and most of the people does not know about them, they just
want to know how to add their tests to nightly build execution.

On Nov 7, 7:46 am, S M Humayun <smhuma...@gmail.com> wrote:
> Hi borojevic,
>
> You've got a point and may be I'm wrong but shouldn't code maintenance
> and governance be handled via other means? for example, what would you
> do if one of your team member add one or more listeners to your
> testng.xml file? how would you detect this? and more importantly how
> would you protect this?
>
> --------------------------------------------
> S M Humayun
> Software Architect, IBL Group
> --------------------------------------------

> Email : smhuma...@iblgrp.com
> MSN   : smhumayun2...@hotmail.com
> Skype : smhumayun
> Cell  : 92.321.8707783
> Blog  :http://smhumayun.blogspot.com
> Link  :http://www.linkedin.com/in/smhumayun
> Facebook :http://www.facebook.com/smhumayun
> Twitter :http://twitter.com/smhumayun
> --------------------------------------------
>