TestNG 7.5 released !!

[Krishnan Mahadevan]

Hello everyone,

On behalf of the TestNG Dev team, I wish all TestNG users a very happy, healthy and prosperous New Year.

We are pleased to announce the availability of TestNG 7.5 (org.testng:testng:7.5)

TestNG 7.5 is now available in Maven Central.

Contributors who helped make TestNG become better in this release:

• ZhangJian He
• Roman Morskyi
• Dzmitry Sankouski
• cdalexndr
• Nan Liang
• Elis Edlund
• dianny
• Stuart Marks 
• Ian Springer
• Pavel Sakharchuk
• baflQA
• Vitalii Diravka
• Marcono1234
• Anindya Roy

Special thanks go to

• Julien Herr for helping out with all the code reviews
• Vladimir Sitnikov for helping with refactoring the codebase into multiple modules and also streamlining a lot of areas around the TestNG gradle build files and of course,
• to all the other contributors who made this release possible.

In this release, we spent a lot of time squashing a lot of bugs (Please see below for a full list of issues that were fixed)

Note:

TestNG 7.5 would be the last of the TestNG releases that would run on JDK8. The next TestNG release would have a minimum JDK dependency of JDK11

Complete list of bugs that were fixed are as below:

• GITHUB-2701: Bump gradle version to 7.3.3 to support java17 build
• GITHUB-2646: Streamline Logging Across TestNG
• GITHUB-2658: Inheritance + dependsOnMethods
• GITHUB-2664: Order for DependsOnGroups has changed after TestNg 7.4.0
• GITHUB-2501: TestNG 7.4.0 throws an exception "sun.net.www.protocol.file.FileURLConnection cannot be cast to java.net.HttpURLConnection" when xml file contain "ENTITY SYSTEM" grammer 
• GITHUB-2693: TestNG ignores 'dataproviderthreadcount' CLA
• GITHUB-2685: TestInvoker should clear Thread.interrupted flag before calling ITestListeners
• GITHUB-2684: AfterGroups config annotation does not consider retries for tests
• GITHUB-2689: Yaml parser: implement loadClasses flag
• GITHUB-2676: NPE is triggered when working with ITestObjectFactory
• GITHUB-2674: Run onTestSkipped for each value from data provider
• GITHUB-2672: Log real stacktrace when test times out.
• GITHUB-2669: A failed retry with ITestContext will lose the ITestContext.
• GITHUB-2643: assertEquals(Set,Set) now ignores ordering as it did before.
• GITHUB-2653: Assert methods requires casting since TestNg 7.0 for mixed boxed and unboxed primitives in assertEquals.
• GITHUB-2229: Restore @BeforeGroups and @AfterGroups Annotations functionality
• GITHUB-2563: Skip test if its data provider provides no data
• GITHUB-2535: TestResult.getEndMillis() returns 0 for skipped configuration - after upgrading testng to 7.0 +
• GITHUB-2638: "[WARN] Ignoring duplicate listener" appears when running .xml suite with and 
• GITHUB-1297: Passed configuration methods appear in testng-failed.xml, when failure was after passed test
• New: Decouple configuration unit tests from main suite 
• GITHUB-2536: Problems with Nested Test Classes
• GITHUB-2558:Make IExecutionListener, ITestListener, IInvokedMethodListener, IConfigurationListener, ISuiteListener finish method with reverse order 
• GITHUB-2532: Apply commandline switches for suites in jar files 
• GITHUB-2558: Make IExecutionListener, ITestListener, IInvokedMethodListener, IConfigurationListener, ISuiteListener execute in the order of insertion
• GITHUB-2611: Config Failures not included in testng-failed.xml when its part of a different test class
• GITHUB-2613: Ignored Tests are not retrieved for a mixed test class (test with enabled, disabled and ignored test method)
• GITHUB-849: Performance improvement by fixing hashCode
• GITHUB-2570: Use Guice injector for instantiate IRetryAnalyzer
• Fix: use proper instances for beforeClass callback when different instances collide on hash codes
• Fix parallel and configfailurepolicy parsing in tr_TR locale
• Wrong results from GuiceBasedObjectDispenser when there's Object#hashCode collision
• GITHUB-2564: Source code is split into several modules for better modularity in the future (for now only a combined jar is released as it was before)
• GITHUB-2564: Added license files as META-INF/LICENSE.txt within the released jar
• GITHUB-2564: Added pax-exam-based OSGi test to verify the manifest
• GITHUB-2564: Migrated the build to Gradle 7.0.2
• GITHUB-2576: Guice 5.0 drops no-aop variant, so TestNG should probably upgrade and avoid no-aop dependency
• GITHUB-2566: Reporter#getOutput(ITestResult tr) uses Map.get(tr.hashCode()) which might result in surprising results
• GITHUB-2565: Dataprovider only supporting a raw type for Iterator return type
• GITHUB-2557: Flaky test: ThreadAffinityTest#testThreadAffinity
• GITHUB-2567: MethodHelper#CANONICALNAMECACHE is never reset, so it could result in a memory leak
• GITHUB-2540: assertEquals(Collection) need check order
• GITHUB-2360: Groovy 3 internal generated methods are detected as test methods
• GITHUB-2522: TestNG 7.4.0 Can not skip test through listener
• GITHUB-2529: Link to testng.xml in CONTRIBUTING.md file was dead
• GITHUB-2521: The method has a separate string with 'invocation-number' parameters for each failure (does not group) in 'testng-failed.xml' file for DataProvider + Factory
• GITHUB-2426: New feature TestNG - getFactoryMethodParamsInfo on ConfigurationMethod
• GITHUB-2517: Factory data-provider parameters not displayed in 'testng-failed.xml' file
• GITHUB-279: Guice dependency injection into listeners and reporters
• GITHUB-2504: Data provider data cannot be populated to onTestStart hook when BeforeMethod is failed
• GITHUB-2489: Hierarchical base- and test-class @AfterClass methods out of order using groups
• GITHUB-2493: Avoid NPE from TextReporter execution when a dataprovider method provides null
• GITHUB-2483: Asymmetric not equals
• GITHUB-2486: assertSame/assertNotSame broken after GITHUB-2296
• GITHUB-2490: assertNotEquals returns fast when argument is null, not calling equals(Object)
• GITHUB-2500: Mention in assertEqualsNoOrder doc that arrays are not compared deeply
• GITHUB-2544: TestNG Retry Fails on complex data-provider arguments

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

 

[dr...@broad.mit.edu]

I get an error running testng 7.5 in parallel. 7.3.0 worked fine

[ERROR] ExecutionException There was an error in the forked process
[ERROR] org.testng.xml.XmlSuite.setParallel(Ljava/lang/String;)V
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException There was an error in the forked process
[ERROR] org.testng.xml.XmlSuite.setParallel(Ljava/lang/String;)V
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:510)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:457)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:298)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:246)
[ERROR]         at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183)
[ERROR]         at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
[ERROR]         at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
[ERROR]         at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
[ERROR]         at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
[ERROR]         at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
[ERROR]         at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
[ERROR]         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
[ERROR]         at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
[ERROR]         at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
[ERROR]         at org.apache.maven.cli.MavenCli.execute(MavenCli.java:957)
[ERROR]         at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:289)
[ERROR]         at org.apache.maven.cli.MavenCli.main(MavenCli.java:193)
[ERROR]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]         at java.lang.reflect.Method.invoke(Method.java:498)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:282)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:225)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:406)
[ERROR]         at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:347)
[ERROR] Caused by: org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR] org.testng.xml.XmlSuite.setParallel(Ljava/lang/String;)V
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:656)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$600(ForkStarter.java:115)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:444)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:420)
[ERROR]         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[ERROR]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[ERROR]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[ERROR]         at java.lang.Thread.run(Thread.java:748)

[Krishnan Mahadevan]

What version of surefire plugin are you working with ? Can you upgrade to the latest surefire plugin and see if that helps?

This is because surefire plugin is attempting to use a method in TestNG that was deprecated and removed later. 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

---

From: testng...@googlegroups.com <testng...@googlegroups.com> on behalf of dr...@broad.mit.edu <dr...@broad.mit.edu>
Sent: Friday, January 14, 2022 2:52:10 AM
To: testng-users <testng...@googlegroups.com>
Subject: [testng-users] Re: TestNG 7.5 released !!

 

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/cc1c7e9a-e3e2-488a-8526-b4c0a55185cbn%40googlegroups.com.

[dr...@broad.mit.edu]

I am running 2.22.2 but I have also tested 3.0.0-M5. What version of surefire is known to work?

[Krishnan Mahadevan]

I don’t remember exactly. I do vaguely remember that there was an incompatibility between sure fire plugin and TestNG around this method. 

Can you create a sample project that can be used to reproduce the problem ? 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

From: testng...@googlegroups.com <testng...@googlegroups.com> on behalf of dr...@broad.mit.edu <dr...@broad.mit.edu>

Sent: Friday, January 14, 2022 8:26:49 PM
To: testng-users <testng...@googlegroups.com>
Subject: Re: [testng-users] Re: TestNG 7.5 released !!

 

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/ac9420b8-f08d-433e-83ca-ff5ba2cde564n%40googlegroups.com.

[dr...@broad.mit.edu]

the problem isn't related to the test itself (the test is never run), it's a runtime issue. To reproduce this I simply run any test in parallel. 

<property>
    <name>parallel</name>
    <value>classes</value>
</property>

On Friday, January 14, 2022 at 10:01:09 AM UTC-5 Krishnan Mahadevan wrote:

I don’t remember exactly. I do vaguely remember that there was an incompatibility between sure fire plugin and TestNG around this method. 

Can you create a sample project that can be used to reproduce the problem ? 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

[Krishnan Mahadevan]

Without a sample project which shows what your dependencies are and the plugins that you use and their versions, I personally dont think I can be much of a help. 

So let me step aside and let someone else chime in. 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

From: testng...@googlegroups.com <testng...@googlegroups.com> on behalf of dr...@broad.mit.edu <dr...@broad.mit.edu>

Sent: Friday, January 14, 2022 9:31:06 PM

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/21c2b5d6-e064-454e-8f5e-9b0a8ff1ba93n%40googlegroups.com.

[dr...@broad.mit.edu]

here you go. the only dependency is testng 7.5 and surefire 2.22.2

https://github.com/therealdjryan/ParallelTest

On Friday, January 14, 2022 at 12:47:29 PM UTC-5 Krishnan Mahadevan wrote:

Without a sample project which shows what your dependencies are and the plugins that you use and their versions, I personally dont think I can be much of a help. 

So let me step aside and let someone else chime in. 

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"

[Krishnan Mahadevan]

This is a problem with surefire plugin, but I am not sure if the fix for this has been released (the latest released version of surefire plugin doesn’t have the fix)

 

For more details, you can take a look at this Stackoverflow post: https://stackoverflow.com/questions/68143219/maven-surefire-failsafe-testng-7-4-0-parallel-error-void-org-testng-xml-xmlsuit

 

PS: The github link you shared is not accessible to me.

 

​Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

 

 

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/fe50c673-6951-4de5-af26-0bd8096d33cbn%40googlegroups.com.

[dr...@broad.mit.edu]

So since 7.4, parallelism in testNG has a dependency on unreleased software? Good to know!

Thanks again Krishnan,

Dennis

[⇜Krishnan Mahadevan⇝]

Dennis,

No I wouldn't say so.

Rephrasing it, I would say that the Surefire plugin needs to upgrade itself to be able to be compatible with the latest released version of TestNG.

You can still control parallelism by creating a suite xml file wherein you mention the parallelism strategy and surefire plugin can be fed with the suite file path and it would work. The issue is strictly related to when you try to control parallelism via surefire exposed properties (which eventually get propagated to TestNG)

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/c4067f46-16a5-4ba4-8cb1-b01ec33e3fb5n%40googlegroups.com.

[Baubak]

Wonderful news!

Seems to have some conflict when a listener thows a SkipException, and skips due to config issues. Will need to analyze this more.

[Konstantin Shkvarun]

Hi everyone. I have an issue.
I'm using Maven+TestNg+Selenide. And now, when I'm updated from 7.4.0 to 7.5 I cannot use @Listeners({SoftAsserts.class}) to turn on soft asserts for class. 

My test is always green even if I broke them down on purpose.

@Listeners({SoftAsserts.class - this is class integrated into com.codeborne.selenide.testng; })
public class BannersCreate extends BannersAbstractTest {

}

Please, help me!)

среда, 26 января 2022 г. в 01:18:19 UTC+2, Baubak:

[⇜Krishnan Mahadevan⇝]

There's not much information to figure out what is going on.

Can you please do the following?

1. Upgrade to TestNG 7.6.0 and retry your scenario (You will need JDK11)

2. If the problem persists, please create a sample project that can be used to recreate the problem (Please don't share the entire project that you are working with since its not going to be possible to delve into its specifics to find out what is going wrong and at the same time, please help keep the sample project's dependencies to the bare minimum i.e., just testng and print statements to reproduce the problem)

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/4453d76e-de5d-49db-adf3-70903e8def61n%40googlegroups.com.