running via annotations

[Andy Dingfelder]

Hi

quick question - one of the challenges I am facing with Concordion adoption is that some of our currently implemented tests have been done in Cucumber, and I'm struggling to replicate the logic they use, it would be great if someone has ideas on how I can do something similar in Concordion

Our existing feature files contain multiple tests - e.g. one feature file per logical feature, and all related tests for that feature are in one feature/spec file

It seems like Concordion isn't set up this way, instead we have one file per test, and I guess we just group related tests into folders (by feature) or whatever makes sense. 

The functionality I'm hoping to replicate is related to annotations / tags.  Some of our tests (in cucumber) logically belong more than one type of test (e.g. a test could be both a smoke test and a regression test) or test suite.  

If I understand correctly with Concordion, you would need to do something like create SmokeTest and RegressionTest folders to store tests, and then filter on that name to run all the tests for the phase you want? L

imitation being of course that a file cannot be in 2 folders at the same time, so if you want the test to be included in 2 phases at once, you would have to make a clone of the test (which would make maintenance a nightmare)

Is that right? or is there something I missed?  

Cheers,

Andy

[Jimmy Kemp]

Hi Andy

In Concordion, you have a specification file (e.g. html or md), which matches with a Fixture file (e.g. java/c#).  The Specification/Fixture can contain multiple tests.  There is a helpful Concordion Presentation, which does a nice job of introducing Concordion in this regard.

On the first Question > It seems like Concordion isn't set up this way, instead we have one file per test, and I guess we just group related tests into folders (by feature) or whatever makes sense. 

Using the Concordion Presentation as a guide.  Each fixture can have multiple tests.  

The Concordion/Cubano project has a demo project which provides working examples.  There are plenty of specification/fixtures with multiple tests. Take a look at the Living Docs for the demo project.  Which you can pull down from git btw.

On the second Question > Executing differing test types (e.g. Regression vs Smoke)

In terms of executing differing test types (e.g. smoke vs regression), one way to manage this is via Index Files.

e.g. The demo project contains an index/specification named 'Example.md'.  

1. Take a copy of this index > and rename to  'Smoke.md'

2. Include only the fixtures you wish to run.  You can include indexes or navigate directly to fixtures.

3. Take a copy of Example.java and rename to  Smoke.java)

4. Execute Smoke

5. There is no duplication/cloning of folders or tests

6. If I maintain/refactor a test which is in both indexes (e.g. Example and Smoke), then both indexes result in the same behaviour for that test

hope this helps.

Jimmy

[Andy Dingfelder]

yes the index file approach does make sense Jimmy, thanks

Having said that, I do find the cucumber way of doing it a bit more straightforward as you can quickly add an annotation to the test and it immediately is part of the relevant suite

and you can tell what suites it is part of just by looking at the test whereas with the index file approach, you would have to look at each index file to see if it includes that test (or search for usages)

Just a different way of doing things I guess

also fyi the "demo project" link sends you to a 404 on https://concordion.github.io/cubano-demo/ 

Cheers,

Andy

[Jimmy Kemp]

Apologies > Cubano Demo

https://github.com/concordion/cubano-demo

On Monday, October 1, 2018 at 9:41:24 AM UTC+13, Andy Dingfelder wrote:

[Nigel Charman]

Andy, I think you should also be able to create a Concordion extension that uses the new

withImplementationStatusModifier()

method to define individual examples to be ignored (but still create the output specifications). The filtering could be based off the fixtures (eg using annotations) or the specifications (eg using attributes).