setting test results

[Andy Dingfelder]

Hi

I'm working on a reporting summary extension and I could use a hint

I've created a listener and it does seems to be registered as listener methods are firing such as:

@Override
public void afterExample(ExampleEvent event) {

but if I dump the event's ResultSummary it contains

successCount=0
failureCount=0
ignoredCount=0
exceptionCount=0

And at the end of my test, it is printing:

Successes: 0, Failures: 0

3 tests completed, 1 failed

Its interesting to me that failed is 1 but failures is 0

Can someone tell me where the numbers that are printed at the end are getting set?

And where should the event's ResultSummary get populated with test results?

Cheers,

Andy

[Andy Dingfelder]

I think I see a bit more about of whats going on:

I am launching my test via an intellij configuration using a gradle task with the class name passed in as an argument eg --tests "*RunExampleTests*"

If rather than launching a test directly, I instead launch a markdown test that just launches an additional test via a run command e.g. 

- **[MarkdownExampleSpec](testComparison/MarkdownExampleSpec.md "c:run")**

then in this case the run summary in the event is populated 

 

Is there something I can do to get the ResultSummary to be used for a single test or a set of tests without having to have a markdown that hardcodes the tests to run?

The problem is that I dont always know what tests I want to run ahead of time - Im testing a dynamic build system that ahas a list of things that has changed and I need to launch just the tests that are relevant 

Im also still curious  about where the success/failure numbers are getting set?

e.g. where is the event's ResultSummary populated with test results as there seems to be 2 sets of results:  1 after each test, and one that summarizes all tests

Successes: 0, Failures: 0

3 tests completed, 1 failed

I need to intercept this logic and store the results along the way so I can generate a dynamic report

Thoughts?

[Nigel Charman]

We have had a report that Gradle's "--tests" does not work correctly with Concordion. Could you try “-DtaskName.single = testNamePattern”? Our focus will be on getting JUnit 5 working (which has a cleaner model), rather than fixing issues with JUnit 4.

Sorry I don't have time to delve into the result summary logic right now. The result reporting is tied in with the way that JUnit works. When using the run command we get the test results back from JUnit of the test that was run.

Since this is delving into internals, could you move the discussion to the concordion-dev list please?

--
You received this message because you are subscribed to the Google Groups "concordion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concordion+...@googlegroups.com.
To post to this group, send email to conco...@googlegroups.com.
Visit this group at https://groups.google.com/group/concordion.
To view this discussion on the web, visit https://groups.google.com/d/msgid/concordion/2374baa0-f9f3-4bd2-88b7-0ad0bbed4d97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.