Cucumber Jvm: Method examples() of gherkin.formatter.Formatter interface not called from JUnit tests

[Liviu Carausu]

Hi,
  I try to implement a custom Formatter for Cucumber Jvm.

  When starting the test with Runtime.run() the examples() method of the Formatter interface is called but when running with the JUnitRunner,
the examples() method is not called.

  When starting the test using the Runtime.run() , this is my stack trace :
    at my.class.implementing.Formatter.examples()
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at cucumber.runtime.Utils$1.call(Utils.java:34)
    at cucumber.runtime.Timeout.timeout(Timeout.java:13)
    at cucumber.runtime.Utils.invoke(Utils.java:30)
    at cucumber.runtime.RuntimeOptions$1.invoke(RuntimeOptions.java:172)
    at com.sun.proxy.$Proxy18.examples(Unknown Source)
    at gherkin.formatter.model.Examples.replay(Examples.java:56)
    at cucumber.runtime.model.CucumberExamples.format(CucumberExamples.java:45)
    at cucumber.runtime.model.CucumberScenarioOutline.run(CucumberScenarioOutline.java:43)
    at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:154)
    at cucumber.runtime.Runtime.run(Runtime.java:120)
    at cucumber.runtime.Runtime.run(Runtime.java:108)
    at cucumber.runtime.Runtime$run.call(Unknown Source)

  So, everything starts with Runtime.run() , method which has as comment the following lines :

  /**
     * This is the main entry point. Used from CLI, but not from JUnit.
     */

   When starting with jUnitRunner, I have found no clue when my Example starts ...

  Does somebody please have an ideea how to deal with the Examples in the Formatters with JUnit ?


   Thanks,
     Liviu

[aslak hellesoy]

On Thu, Aug 28, 2014 at 10:33 PM, Liviu Carausu <liviu....@gmail.com> wrote:

Hi,
  I try to implement a custom Formatter for Cucumber Jvm.

  When starting the test with Runtime.run() the examples() method of the Formatter interface is called but when running with the JUnitRunner,
the examples() method is not called.

I think you cut off the most important part of the stack trace, at the top. Please send it again.

Also - what Cucumber-JVM version are you using?

Aslak

--
Posting rules: http://cukes.info/posting-rules.html
---
You received this message because you are subscribed to the Google Groups "Cukes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cukes+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Björn Rasmusson]

Hi,

When each example from an scenario outline is instantiated as a scenario and executed, the scenario() method of the formatter is called (with the keyword argument "Scenario Outline"). This behavior is exactly the same when using the command line runner or the junit runner. The calls to the scenario_outline() and the examples() methods, does not represent the actual execution of the scenario outline and examples, they summarizes what will be executed next. They will only occur when using the command line runner (if using the pretty formatter you will see that the scenario outline and examples will be color coded as "skipped", since they are not yet executed). See also this post.

You know that your examples have started when the scenario() method is called with the keyword argument "Scenario Outline".

Best Regards
Björn
 

[Liviu Carausu]

Hi Aslak,
   This is my complete stacktrace. I trigger Runtime.run() from a Spock test :

        at java.lang.Thread.dumpStack(Thread.java:1365)
    at net.thucydides.cucumber.ThucydidesReporter.examples(ThucydidesReporter.java:144)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at cucumber.runtime.Utils$1.call(Utils.java:34)
    at cucumber.runtime.Timeout.timeout(Timeout.java:13)
    at cucumber.runtime.Utils.invoke(Utils.java:30)
    at cucumber.runtime.RuntimeOptions$1.invoke(RuntimeOptions.java:172)
    at com.sun.proxy.$Proxy18.examples(Unknown Source)
    at gherkin.formatter.model.Examples.replay(Examples.java:56)
    at cucumber.runtime.model.CucumberExamples.format(CucumberExamples.java:45)
    at cucumber.runtime.model.CucumberScenarioOutline.run(CucumberScenarioOutline.java:43)
    at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:154)
    at cucumber.runtime.Runtime.run(Runtime.java:120)
    at cucumber.runtime.Runtime.run(Runtime.java:108)
    at cucumber.runtime.Runtime$run.call(Unknown Source)

    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at net.thucydides.cucumber.outcomes.WhenCreatingThucydidesTestOutcomesForTableDrivenScenarios.$spock_feature_0_0(WhenCreatingThucydidesTestOutcomesForTableDrivenScenarios.groovy:56)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
    at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:330)
    at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:311)
    at org.spockframework.runtime.BaseSpecRunner.invokeFeatureMethod(BaseSpecRunner.java:285)
    at org.spockframework.runtime.BaseSpecRunner.doRunIteration(BaseSpecRunner.java:256)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
    at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:330)
    at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:311)
    at org.spockframework.runtime.BaseSpecRunner.runIteration(BaseSpecRunner.java:223)
    at org.spockframework.runtime.BaseSpecRunner.initializeAndRunIteration(BaseSpecRunner.java:214)
    at org.spockframework.runtime.BaseSpecRunner.runSimpleFeature(BaseSpecRunner.java:205)
    at org.spockframework.runtime.BaseSpecRunner.doRunFeature(BaseSpecRunner.java:199)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
    at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:330)
    at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:311)
    at org.spockframework.runtime.BaseSpecRunner.runFeature(BaseSpecRunner.java:175)
    at org.spockframework.runtime.BaseSpecRunner.runFeatures(BaseSpecRunner.java:152)
    at org.spockframework.runtime.BaseSpecRunner.doRunSpec(BaseSpecRunner.java:112)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
    at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:330)
    at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:311)
    at org.spockframework.runtime.BaseSpecRunner.runSpec(BaseSpecRunner.java:91)
    at org.spockframework.runtime.BaseSpecRunner.run(BaseSpecRunner.java:82)
    at org.spockframework.runtime.Sputnik.run(Sputnik.java:63)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)


Thanks,
    Liviu

[Liviu Carausu]

Hi Aslak,

  I trigger Runtime.run() directly , I Use Cucumber-jvm version 1.1.8.

  Here is my stack trace :

       at java.lang.Thread.dumpStack(Thread.java:1365)
    at net.thucydides.cucumber.ThucydidesReporter.examples(ThucydidesReporter.java:144)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at cucumber.runtime.Utils$1.call(Utils.java:34)
    at cucumber.runtime.Timeout.timeout(Timeout.java:13)
    at cucumber.runtime.Utils.invoke(Utils.java:30)
    at cucumber.runtime.RuntimeOptions$1.invoke(RuntimeOptions.java:172)
    at com.sun.proxy.$Proxy18.examples(Unknown Source)
    at gherkin.formatter.model.Examples.replay(Examples.java:56)
    at cucumber.runtime.model.CucumberExamples.format(CucumberExamples.java:45)
    at cucumber.runtime.model.CucumberScenarioOutline.run(CucumberScenarioOutline.java:43)
    at cucumber.runtime.model.CucumberFeature.run(CucumberFeature.java:154)
    at cucumber.runtime.Runtime.run(Runtime.java:120)
    at cucumber.runtime.Runtime.run(Runtime.java:108)
    at cucumber.runtime.Runtime$run.call(Unknown Source)

    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at net.thucydides.cucumber.outcomes.WhenCreatingThucydidesTestOutcomesForTableDrivenScenarios.$spock_feature_0_0(WhenCreatingThucydidesTestOutcomesForTableDrivenScenarios.groovy:56)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
    at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:330)
    at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:311)
    at org.spockframework.runtime.BaseSpecRunner.invokeFeatureMethod(BaseSpecRunner.java:285)
    at org.spockframework.runtime.BaseSpecRunner.doRunIteration(BaseSpecRunner.java:256)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
    at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:330)
    at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:311)
    at org.spockframework.runtime.BaseSpecRunner.runIteration(BaseSpecRunner.java:223)
    at org.spockframework.runtime.BaseSpecRunner.initializeAndRunIteration(BaseSpecRunner.java:214)
    at org.spockframework.runtime.BaseSpecRunner.runSimpleFeature(BaseSpecRunner.java:205)
    at org.spockframework.runtime.BaseSpecRunner.doRunFeature(BaseSpecRunner.java:199)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
    at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:330)
    at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:311)
    at org.spockframework.runtime.BaseSpecRunner.runFeature(BaseSpecRunner.java:175)
    at org.spockframework.runtime.BaseSpecRunner.runFeatures(BaseSpecRunner.java:152)
    at org.spockframework.runtime.BaseSpecRunner.doRunSpec(BaseSpecRunner.java:112)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at org.spockframework.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:138)
    at org.spockframework.runtime.BaseSpecRunner.invokeRaw(BaseSpecRunner.java:330)
    at org.spockframework.runtime.BaseSpecRunner.invoke(BaseSpecRunner.java:311)
    at org.spockframework.runtime.BaseSpecRunner.runSpec(BaseSpecRunner.java:91)
    at org.spockframework.runtime.BaseSpecRunner.run(BaseSpecRunner.java:82)
    at org.spockframework.runtime.Sputnik.run(Sputnik.java:63)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)

    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)

    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

Thanks,
    Liviu

[Liviu Carausu]

Hi Björn,

   Thanks for your answer. First of all, the Version of cucumber-jvm that I use is 1.1.8.

   The Examples object that I receive from the examples() method is extremely valuable for my reporting because it contains a list of  ExamplesTableRow.

   In the step() callback mehod, the step.getRows() returns always null...

      Thanks and best regards,
         Liviu

[aslak hellesoy]

On Fri, Aug 29, 2014 at 8:09 AM, Liviu Carausu <liviu....@gmail.com> wrote:

Hi Aslak,
   This is my complete stacktrace.

A stacktrace consists of an error message (e.getMessage()) and stack frames (list of paths with line numbers).

Your stacktrace has the frames, but not the message.

This makes it hard to know what the error is.

Aslak

[Liviu Carausu]

Hi Aslak,

  There is no error message. There is only a stackTrace ( like in "Thread.dumpStack() " ) that i have triggered in my implementation of the "examples()" method to find out
who is calling the method and why is the method not called when I trigger the test with the JUnitRunner.

    Thanks,
       Liviu

[Björn Rasmusson]

Liviu Carausu wrote:

Hi Björn,

   Thanks for your answer. First of all, the Version of cucumber-jvm that I use is 1.1.8.

   The Examples object that I receive from the examples() method is extremely valuable for my reporting because it contains a list of  ExamplesTableRow.

Hi,

Well, then maybe you should decide to use the command line runner (cucumber.api.cli.Main) ...

   In the step() callback mehod, the step.getRows() returns always null...

the step.getRows() returns the data table of the step

  Given the following data
     | x | y |
     | m | n |

so in your case it is likely to be null.

I believe that the argument from the example table that was used when instantiating the step, is passed to the formatter in the match() call.

Best Regards
Björn
 

[Liviu Carausu]

Hi Björn,

   I have found no util information in the argument of the match call(). I created a pull request to have the methods examples() and scenarioOutline() called also when JUnit runner is used.
Do you also think that this will be useful ?

  Thanks,
      Liviu