camunda-bpm-testing

[Jan Galinski]

Moved from users forum. Last question by Bernd: 

Rafa started a discussion about naming of modules this morning which I think should be best move here into the forum:

My only comment would be to explicitly think about the naming of the Maven modules to reflect purpose. I believe it is important that developers coming to the camunda-bpm-testing project, are able to answer pretty fast the question "Why do I need this for? What problem is this going to solve?". The current proposed names for the new modules

camunda-bpm-junit 
camunda-bpm-jbehave 
camunda-bpm-fluent-assertions 
camunda-bpm-fluent-api 
camunda-bpm-needle 
camunda-bpm-arquillian 

indicate technologies and not purpose. I just ask myself whether other names would be better to guide the users of the libraries to the right place. If I'm a software developer building an process-driven app with Spring/CDI and I want to do unit test, integration testing and BDD: how do I do that  with camunda BPM? Which libraries can I use and how do they help me?

My answer to this:

I understand what you mean. But some goals could be realized by different frameworks (e.g. BDD by Cucumber or JBehave; Unit Testing by JUnit or TestNG; …) and I think it should be clear in the Maven Artifact what framework is used as this has quite some impact (I need to write JUnit tests, JBehave fixtures, Arquillian deployments, …). So it cannot be easily hidden and I would prefer to expose it clearly in that case and do a proper documentation to guide people through the selection of the proper testing approach.

 

Opinions?

[Simon Zambrovski]

Hi,

I think we should start naming and need the framework indication only after alternatives appear. Currently, we are building tests based on Junit, Jbevahe and Acuirillian. For me it is ok to name it unit, behavior and integration. If TestNG joins camunda-BPM we can still refactor or repackage...

Simon Zambrovski

Senior Consultant

Holisticon AG - Member of Sigma Group

Griegstraße 75, Haus 25

22763 Hamburg

Mobile: +49 176 1616 9010

Office: +49 40 6094 430-0

Fax: +49 40 6094 430-30

E-Mail: simon.za...@holisticon.de

www.holisticon.de

Amtsgericht Hamburg, HRB 107396

Vorstand: Oliver Ihns, Dierk Harbeck

Aufsichtsrat: Sune Nilsson (Vorsitz)

--
You received this message because you are subscribed to the Google Groups "camunda BPM platform internal" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camunda-bpm-d...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Jan Galinski]

We should not aim for multi framework support. When we talk about unit testing, we think "junit", when we talk about mocking, its mockito, integration = arquillian and (after some thought and evaluation) Simon an I decided that (for Java projects in general and BPM projects in particular)  

bdd=jbehave. 

Especially the "GivenStories" annotation that allows you to take a successful story ("process ran and now waits in user task") as precodition for the follow up scenario ("process now continues with ...") is essential for incremental process testing and is not supported by cucumber.

The fluent assertion extension itself is a good example: it hides the fact that it relies on junit and mockito (and fest) to work correctly by the term "fluent".

My opinion? Besides naming the components, we should establish a set of test tools with concrete versions in a pom module and use these allover camunda. That being done, we would be free to name modules after their purpose, not their technology.

[Bernd Rücker (camunda)]

Any other oppinions? Especially some from the core team would be interessting to take into account.

 

At the oment Jan, Simon and Rafa tend to name it "unit" / "bdd"; I tend to name it "junit" / "jbehave". Hence democracy goes for the first ;-)

 

Does anybody knows other frameworks solving this naming problem? Maybe we can use Best Practices from there? I have bad Internet at the moment so I cannot google myself…

--

[Daniel Meyer]

Hi Guys,

 

2 thoughts on this:

-          If we provide integration with a particular framework (junit, arquillian, spring, camel, cdi, ejb, test-ng, cucumber, ….) we should name the corresponding module in a way that people can find it if they look for that integration

-          Testing frameworks can be used for writing different “kinds” of tests. I for my part have written countless “integration tests” with junit and loads of unit tests based on arquillian. Saying that “junit = unit test” and “arquillian = integration” test is simply not true.

 

But that is just my opinion I don’t want to impose or anything.

 

Cheers,

Daniel Meyer

[Daniel Meyer]

Hi Jan,

 

Besides naming the components, we should establish a set of test tools with concrete versions in a pom module and use these allover camunda.

 

Feel free to open a “where is camunda-bpm-bom???” discussion!

 

Cheers,

Daniel Meyer

 

Von: camunda...@googlegroups.com [mailto:camunda...@googlegroups.com] Im Auftrag von Jan Galinski

Gesendet: Sonntag, 8. September 2013 12:23
An: camunda...@googlegroups.com
Betreff: Re: camunda-bpm-testing

 

We should not aim for multi framework support. When we talk about unit testing, we think "junit", when we talk about mocking, its mockito, integration = arquillian and (after some thought and evaluation) Simon an I decided that (for Java projects in general and BPM projects in particular)  

--

[Bernd Rücker (camunda)]

Exactly my oppionon as well (and believe me - I have done nothing to influence Daniel :-))

[Martin Schimak]

Hi all,

My apologies pls that you didn't get my feedback on that development (camunda-bpm-testing) earlier - you caught me at the wrong foot, as I just started into a new project of mine which needed quite a bit of attention.

The birth of camunda-bpm-testing: I think it's just great and fine! I am very happy to see the interest in that baby growing after all - which will for sure help to grow that baby into a little child - and this makes me very confident that we can maybe even form a little special-interest-contributors-community around that topic of testing camunda bpm process applications!

Therefore: Many thx to Jan and Simon. I've seen quite a lot cleanup-work with respect to pom structure / build and besides the whole new needle contribution even an addition to the "fluent" engine-api (Deployment). Thank you! 

I also saw that we now have some holisticon dependencies, but did not investigate any further why we need them. As a tendency I would vote to avoid those, but don't know whether there are strong reasons to keep them (or to keep them for the moment). Sorry for adding new aspects to that thread, but it also serves to get started with the discussion about camunda-bpm-testing :-)! We should then move on by separating the issues into separate threads.

With respect to the main discussion here about modules / naming:

Even though Rafa and me called the stuff "fluent-assertions" etc I see now more upsides to Bernds and Daniels POV. I also agree with Jan that we should not "aim for" multi-framework support but focus on a "best-of", at the same time I also would like to remain open for further contributions like e.g. for cucumber - especially in case they are contributed by somebody else!-) 

It was already mentioned that people will find it easier to find their way when we name modules alongside specific framework integration. I want to add another aspect: It will make it more straightforward for us to separate the concerns. I think we already have an example for what I mean in the current codebase: there exists now code for deeper mockito integration inside the "needle" module (automatically registering named fields as mocks), but something very similar already pre-existed in the "fluent-assertions" module. Actually it does not belong in both of them.

My suggestions for what we already have:

- camunda-bpm-needle => just fine

- camunda-bpm-fluent-assertions => camunda-bpm-fest (or camunda-bpm-fest-assertions?)

- camunda-bpm-fluent-api => would keep it, because it is a self-contained topic in itself - so (as a goal) just dependent on camunda-bpm-engine itself...

Furthermore, we should probably aim to separate out the mockito-support into

- camunda-bpm-mockito

We can then of course add

- camunda-bpm-jbehave 

- camunda-bpm-arquillian 

…

My two or five cents, how many you prefer…, for the moment. I will keep posting other topics that are on my mind…

Looking forward to your feedback,

Many greetings,

Martin.

[Simon Zambrovski]

Hi Guys,

@Martin: holisticon dependency is a maven-central published pom-only project that combines correct versions of mockito/hamcrest/junit. 

@Martin: could you explain the camunda-bpm-mockito artifact? What should it contain? We use needle for mocking/injecting dependencies with mockito as a mock provider behind it. I would expect that there is a general camunda-bpm-testing-support project contaning small helpers for mockito, unit etc... Is there so much mockito specific code that it is worth to create an own project for it?

We already started the JBehave part (look on the branch include_bdd) and named it camunda-bpm-bdd. It is not a problem to rename it to camunda-bpm-jbehave, since it has a direct dependency to the JBehave framework.

So I'm fine with naming propsed by Martin.

What we would get is a parent camunda-bpm-testing with modules:

camunda-bpm-needle 

camunda-bpm-fest-assertions

camunda-bpm-fluent-api (at some point moved out of testing, if depends only on the engine)

camunda-bpm-jbehave

camunda-bpm-arcquillian

camunda-bpm-testing-examples (explained below)

camunda-bpm-testing-support

I think it is a good idea to have some common examples, all this projects are testing. So in order to show some specific test strategy or feature we should not create an isolated process example, but rely on the common example processes. It would make the knowledge tranfser and eductaion easier, since people have to learn the processes under test only once.

Kind regards,

Simon 

[Martin Schimak]

Hi Simon, all,

@Martin: holisticon dependency is a maven-central published pom-only project that combines correct versions of mockito/hamcrest/junit. 

Yep, saw that, but would vote to see / bundle those dependencies directly in the camunda-bpm(-testing) pom structure. 

@Martin: could you explain the camunda-bpm-mockito artifact? What should it contain? We use needle for mocking/injecting dependencies with mockito as a mock provider behind it. I would expect that there is a general camunda-bpm-testing-support project contaning small helpers for mockito, unit etc... Is there so much mockito specific code that it is worth to create an own project for it?

I agree that there is probably not much code at the moment, but made that proposal for more systematic reasons. If we decide to move those things into a camunda-bpm-testing-support project (which is a possible alternative) then we will make our life easier but will probably also produce more inter-framework dependencies right there. 

At the same time I don't see it as contradicting that approach that e.g. needle support is based on mockito as a provider but tend to think in a direction which makes that as transparent as possible...

We already started the JBehave part (look on the branch include_bdd) and named it camunda-bpm-bdd. It is not a problem to rename it to camunda-bpm-jbehave, since it has a direct dependency to the JBehave framework.

So I'm fine with naming propsed by Martin.

Fine. So I figure we can probably almost call that a decision?!-)

What we would get is a parent camunda-bpm-testing with modules:

camunda-bpm-needle 

camunda-bpm-fest-assertions

camunda-bpm-fluent-api (at some point moved out of testing, if depends only on the engine)

camunda-bpm-jbehave

camunda-bpm-arcquillian

camunda-bpm-testing-examples (explained below)

Agreed from my side...

camunda-bpm-testing-support 

camunda-bpm-testing-support discussion is open... let's also remember and include into that discussion here that Bernd actually suggested two more upcoming modules:

- camunda-bpm-testing-core: core testing code independant of testing framework (no JUnit dependencies here!)

- camunda-bpm-junit: junit stuff

More opinions on that?

I think it is a good idea to have some common examples, all this projects are testing. So in order to show some specific test strategy or feature we should not create an isolated process example, but rely on the common example processes. It would make the knowledge tranfser and eductaion easier, since people have to learn the processes under test only once.

Fully agreed. This gives me the opportunity to mention the currently existing integration-test project which is disabled at the moment. Because this project is - at least in part - already showing executable examples which go far beyond "unit testing" the testing code. I also saw that you also have such examples in your needle module. With respect to all that I propose that we

- write more unit tests "testing the testing code" in the respective module (yes, comment primarily targeted at myself!-).

- create that camunda-bpm-testing-examples project which shows executable examples for all the modules. Those examples then also serve as further integration tests for the testing code. 

I would therefore move some of the examples we currently have in the integration test module into that new examples module. I would suggest that you do the same for examples in the needle project. Based on that we can step by step improve those examples by adding more cross-module testing code over time. (When all that is done, I would also at some point remove that disabled integration test module...)

Many greetings,

Martin.

[Simon Zambrovski]

Great,

than let me create the modules tonight so we can continue refactoring. 

I like the camunda-bpm-testing-core and more than my proposed name of test-support.

I'll also create the camunda-bpm-testing-junit, which remains empty at the moment but will be filled at some point with stuff like Junit-Rules etc...

Kind regards,

Simon

 

2013/9/9 Martin Schimak <martin....@plexiti.com>

--
You received this message because you are subscribed to the Google Groups "camunda BPM platform internal" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camunda-bpm-d...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
--
Simon Zambrovski

[Martin Schimak]

Ok, great that you move forward!-) 

Just one thing: I don't think that Bernd meant with the "testing-core" module a module that combines several helpers for integration frameworks into one module... I more understood it as a module which would rely on no framework at all but just on the bpm-engine, with the option to move at some point in time existing testing code into that module.

Maybe Bernd or Daniel can comment on that? What did you actually mean?

Martin. 

[Bernd Rücker (camunda)]

Great discussion guys. I am really happy with the results so far - thanks for pushing this forward!

 

Let's see hwta testing-core will contain - the idea was to keep it pretty independant of JUnityes (as we might not need it for Arquillian and the like and it avoids unwanted dependencies) - but from my side it is not a must. Best we get something working the best way we envision at the moment - I fear over engineering more that having something mingled together at this stage.

 

For the testing examples I would like to move them to https://github.com/camunda/camunda-bpm-examples later on or show testing variaties in the various example we already have. But I think it is a good idea to collect it there first and maybe later on move it to a proper location. Or maybe even keep it there if it proves to be the best location.

 

So it will be this list - correct?

 

camunda-bpm-testing-core

camunda-bpm-junit

camunda-bpm-needle

camunda-bpm-fest-assertions

camunda-bpm-fluent-api

camunda-bpm-jbehave

camunda-bpm-arquillian

camunda-bpm-testing-examples

 

+1!

 

Cheers

Bernd

[Martin Schimak]

Correct and +1, for the comment on over engineering, too! (The tricky thing is that one cannot find a correct position to this topic once and for all, it's more that we have to continuously find the balance and help each other achieving that balance…)

A small matter of taste: in case Simon comes to the conclusion while refactoring that the junit module remains totally empty for the time being, I personally would prefer not to open that module at all at the moment, but wait whether we want to open it later. 

Many greetings,

Martin.

[Jan Galinski]

+1 from me too (finally had 5 minutes on my machine to read and answer). 

One last question:
I think we will have some Mockito utils (doAnswer/doReturn for Delegates for example). Will I put these in camunda-bpm-mockito, -junit or -core?

And:
We already talked about moving all test tools (Rule, Helper) from the engine package to the bpm-testing project, what would the correct locations be?

Jan

[Bernd Rücker (camunda)]

Mockito utils could go to core for the moment I think (as it maybe adds to much overhead to add an own module for that) - or does this impose a mockito runtime dependency? Or the needle module if it is "only" used there?

 

I think all the helpers you mention could go into core (or JUnit if it exists).

 

Von: camunda...@googlegroups.com [mailto:camunda...@googlegroups.com] Im Auftrag von Jan Galinski
Gesendet: Montag, 9. September 2013 17:20
An: camunda...@googlegroups.com
Betreff: Re: camunda-bpm-testing

 

+1 from me too (finally had 5 minutes on my machine to read and answer). 

--

[Martin Schimak]

My interpretation of the discussion result:

(1) camunda-bpm-mockito does not exist for the moment, 

(2) rather we move those small helpers and stuff that supports all the modules to the -core

(3) junit is a module which adds junit framework support - in case and when it's possible to factor that out

We then wait and see what kind of code collects in -core, which runtime dependencies we produce and decide then whether and when it makes sense to move stuff into separate modules.

Cheers,

Martin.

--

[Rafael Cordones]

Hi all,

First of all, great to see the camunda BPM testing project picking up speed!

I am currently focusing the very little free time I have to move forward the Apache Camel integration. So basically I do not want to add my opinion here since I will not be able to "support it" later with feedback and actions. I will just say that I like a lot what I have read so far in this thread! :-)

Jan wrote: "I think we will have some Mockito utile (doAnswer/doReturn for Delegates for example). Will I put these in camunda-bpm-mockito, -junit or -core?"

@Jan: do you mean Mockito utils to test ServiceTasks implemented with Java delegates?! 

Funnily enough, while exploring today how to implement the camunda BPM --> Apache Camel integration with a Java Delegate I found myself with the need of… testing it! :-) Do you have already that Mockito code? :-D 

In any case, once the Apache Camel integration is in better shape, I will come back to the testing support for end-to-end integration tests for camunda BPM + Apache Camel.

Looking forward to see you all at the BPMCon!

/rafa

[Jan Galinski]

Jan wrote: "I think we will have some Mockito utile (doAnswer/doReturn for Delegates for example). Will I put these in camunda-bpm-mockito, -junit or -core?"

@Jan: do you mean Mockito utils to test ServiceTasks implemented with Java delegates?! 

Yes, I do. Its not much though, just a "setVariable" tool. https://github.com/camunda/camunda-bpm-testing/blob/master/camunda-bpm-needle/src/main/java/org/camunda/bpm/engine/test/SetVariablesOnDelegateExecutionAnswer.java

Funnily enough, while exploring today how to implement the camunda BPM --> Apache Camel integration with a Java Delegate I found myself with the need of… testing it! :-) Do you have already that Mockito code? :-D 

In any case, once the Apache Camel integration is in better shape, I will come back to the testing support for end-to-end integration tests for camunda BPM + Apache Camel.

Looking forward to see you all at the BPMCon!

Me too. I think you gonna love the needle approach. 

Jan

[Simon Zambrovski]

Hi,

I just pushed the results to the master. I commented out the camunda-bpm-jbehave module, since we need the snapshot of jbehave to be put into camunda repository.
I hope Christian can help us with it.

Kind regards,

Simoni

Am 10.09.2013 00:08, schrieb Jan Galinski:

Jan wrote: "I think we will have some Mockito utile (doAnswer/doReturn for Delegates for example). Will I put these in camunda-bpm-mockito, -junit or -core?"

@Jan: do you mean Mockito utils to test ServiceTasks implemented with Java delegates?!�

Yes, I do. Its not much though, just a "setVariable" tool. https://github.com/camunda/camunda-bpm-testing/blob/master/camunda-bpm-needle/src/main/java/org/camunda/bpm/engine/test/SetVariablesOnDelegateExecutionAnswer.java

Funnily enough, while exploring today how to implement the camunda BPM --> Apache Camel integration with a Java Delegate I found myself with the need of� testing it! :-) Do you have already that Mockito code? :-D�

In any case, once the Apache Camel integration is in better shape, I will come back to the testing support for end-to-end integration tests for camunda BPM + Apache Camel.

Looking forward to see you all at the BPMCon!

Me too. I think you gonna love the needle approach.�

Jan

--
You received this message because you are subscribed to the Google Groups "camunda BPM platform internal" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camunda-bpm-d...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Simon Zambrovski

Dipl.-Ing.- Inf.
E-Mail: si...@zambrovski.org
Tel.: +49 176 16169010
Internet: http://simon.zambrovski.org/

[Bernd Rücker (camunda)]

Cool - thanks for all the extra time you put in it. Looks already really good! I quickly adjusted the dependency in needle from camunda-bpm-fluent-assertions to camunda-bpm-fest-assertions. Now it builds perfectly. Hope I find time to have a deeper look later today.

 

@Christian: Would be lovely if you can help out with the jbehave dependency.

 

Von: camunda...@googlegroups.com [mailto:camunda...@googlegroups.com] Im Auftrag von Simon Zambrovski
Gesendet: Dienstag, 10. September 2013 00:50
An: camunda...@googlegroups.com; c...@camunda.com
Betreff: Re: camunda-bpm-testing

 

Hi,

I just pushed the results to the master. I commented out the camunda-bpm-jbehave module, since we need the snapshot of jbehave to be put into camunda repository.
I hope Christian can help us with it.

Kind regards,

Simoni

Am 10.09.2013 00:08, schrieb Jan Galinski:

Jan wrote: "I think we will have some Mockito utile (doAnswer/doReturn for Delegates for example). Will I put these in camunda-bpm-mockito, -junit or -core?"

 

@Jan: do you mean Mockito utils to test ServiceTasks implemented with Java delegates?! 

 

 

Yes, I do. Its not much though, just a "setVariable" tool. https://github.com/camunda/camunda-bpm-testing/blob/master/camunda-bpm-needle/src/main/java/org/camunda/bpm/engine/test/SetVariablesOnDelegateExecutionAnswer.java

 

 

 

 

Funnily enough, while exploring today how to implement the camunda BPM --> Apache Camel integration with a Java Delegate I found myself with the need of… testing it! :-) Do you have already that Mockito code? :-D 

 

In any case, once the Apache Camel integration is in better shape, I will come back to the testing support for end-to-end integration tests for camunda BPM + Apache Camel.

 

Looking forward to see you all at the BPMCon!

 

Me too. I think you gonna love the needle approach. 

 

Jan

--
You received this message because you are subscribed to the Google Groups "camunda BPM platform internal" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camunda-bpm-d...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Christian Lipphardt]

Hi guys,

I modified the camunda-bpm-fluent-integration-tests module so it builds again and added some arquillian profiles for jboss (managed/remote). By default, there will be no IT executed on any server, just the included unit tests.

Cheers

Christian

[Martin Schimak]

Hi Christian,

Thats great - thank you!

Martin.

--