Retries(skips) in GitLab CI/CD fails with ERROR: Job failed...

[Matt Wetzel]

So with my test suite, I've implemented a retry listener.  

Everything works fine when running locally.   When tests are retried, they are marked as skipped and the suite passes as long as there are no failures.   

When executing the same on GitLab CI/CD and there are retries(skips), the job fails with...

Total tests run: 224, Failures: 0, Skips: 1

ERROR: Job Failed: command terminated with exit code 1

Can't figure out why GitLab is doing this so I'm looking for a little help with resolving this.

Thanks ahead of time

Matt

[⇜Krishnan Mahadevan⇝]

Matt,

Have you tried setting the verbose=2 or higher on the suite file and then checking if that helps bring out something ?

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

--
You received this message because you are subscribed to the Google Groups "testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to testng-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/f4defdd7-dcf1-4aa2-ad02-3861153db09bn%40googlegroups.com.

[Matt Wetzel]

Krishnan, I tried adding verbose to the testing.xml file in the suite line and while it did give me some output, frankly it wasn't terribly useful.  

I set it to 10 and it only gave me the splash with Cedric's name then a list of the Invoked Methods.   Does it need to be use in conjunction with some other logging tool?   I did comment out all the log4j settings so it would be just testing...   

thank you BTW for the answer.

-matt

[⇜Krishnan Mahadevan⇝]

Matt,

Let's take a step back:

1. What sort of a project are you running on GitLab CI? Is that a plain vanilla java project that is being built or are you talking about some UI tests that involve Selenium?

2. What version of TestNG are you using? Please make sure that you use TestNG 7.3.0 (the latest released version). This I dont think has anything to do with your problem because TestNG would have failed locally as well, if it was a pure TestNG related problem (or atleast that's the expectation here).

3. Did you look at the reports or anything that you have access to, after running the tests to see what is happening?

4. Does the console not tell you which is the test that is failing/skipping ? The verbose level of 2 should be able to start printing that out for you.

Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening or in love with someone else!"
My Scribblings @ http://wakened-cognition.blogspot.com/

My Technical Scribblings @ https://rationaleemotions.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/testng-users/2f6b137d-c589-46c5-afdb-07e05257bb14n%40googlegroups.com.