testng slow to start running tests

[Michael Ondrejko]

We are using testng with a large source base. With testng 6.3.2 and 6.4 there is a long pause before tests start running. It seems that this is the time that testng is scanning classes for testng annotations. For our source base, this requires 15+ minutes to complete. Unfortunately, our test classes are in the same package structure as the product classes themselves and it is not possible to change this organization. I have looked for a way to limit that classes (and jars) that testng is scanning, but it seems to pick up the test classes, everything in the classpath that matches the package definition specified in testng.xml are scanned.

 

Is it possible to further limit this scanning or otherwise speed up the startup time for testng.

[Cédric Beust ♔]

Hi Michael,

It's a bit hard to comment without any data, do you have the output of a profiler to share?

Having said that, you are right that specifying packages is expensive, consider specifying classes explicitly or only running your tests with a minimal classpath.

-- 

Cédric

On Tue, Jun 12, 2012 at 3:09 PM, Michael Ondrejko <mic...@ondrejko.name> wrote:

We are using testng with a large source base. With testng 6.3.2 and 6.4 there is a long pause before tests start running. It seems that this is the time that testng is scanning classes for testng annotations. For our source base, this requires 15+ minutes to complete. Unfortunately, our test classes are in the same package structure as the product classes themselves and it is not possible to change this organization. I have looked for a way to limit that classes (and jars) that testng is scanning, but it seems to pick up the test classes, everything in the classpath that matches the package definition specified in testng.xml are scanned.

 

Is it possible to further limit this scanning or otherwise speed up the startup time for testng.

--
You received this message because you are subscribed to the Google Groups "testng-dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/testng-dev/-/iVXlAbeoN3MJ.
To post to this group, send email to testn...@googlegroups.com.
To unsubscribe from this group, send email to testng-dev+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/testng-dev?hl=en.

[Michael Ondrejko]

There are thousands of test classes, so specifying each individual test class is not feasible. Previously, with testng 6.0.1m the project team used an include sub-element in the package specification too filter specific tests. For example,

 

  <package name="com.company.*">

    <include name=".*Test"/>

  </package>

 

To process only classes that end in "Test" in the "com.company" package hierarchy. However, this now results in all tests being excluded as the include here only filters based upon package name. I tried to use wildcards in the class element for testng.xml; however, class specification does not seem to support wildcards.

 

Is there a way in testng.xml to specify a pattern for the classes to scan for annotations within a package structure?

[Cédric Beust ♔]

On Wed, Jun 13, 2012 at 6:54 AM, Michael Ondrejko <mic...@ondrejko.name> wrote:

Is there a way in testng.xml to specify a pattern for the classes to scan for annotations within a package structure?

This won't make a difference, the problem is that TestNG has to look up all the classes and packages found on the class path, if only to apply that filter.

But again, we're just guessing at this point until you run a profiler to find out exactly where the performance problems are coming from.

-- 

Cédric

[Alexandru Baxanean]

When using testng inside maven, is there any way to tell testng to scan for classes only in test-classes directory ?

That's to avoid the useless scanning of hundreds of jar files and the classes directory where source (not test) classes are located.

[g_...@msn.com]

We have 15 suite-file and use gradle to lunch the test, and we observed that test class objects are created and then paused for ~15 minutes to start running the actual test methods....very poor performance.

[Julien Herr]

Did you try recent releases like 6.11 or 6.12? Is it better?

--

You received this message because you are subscribed to the Google Groups "testng-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to testng-dev+...@googlegroups.com.

To post to this group, send email to testn...@googlegroups.com.

Visit this group at https://groups.google.com/group/testng-dev.
For more options, visit https://groups.google.com/d/optout.