[Cucumber-JVM]: Unable to access testng pickle.getSteps() in Cucumber version 5.6.0

[Anukul Singhal]

Hi,

I am a user of Cucumber-JVM with TestNGCucumberRunner, and my existing tests use cucumber version 4.2.0.  In my TestRunner, I have to access the PickleStep object for my own custom framework.  I am able to access it in version 4.2.0 with the following code:

@Test(description = "Runs Cucumber Scenarios", dataProvider = "scenarios")
public void scenario(PickleEventWrapper pickleEvent, CucumberFeatureWrapper cucumberFeature) throws Throwable {
    List<PickleStep> steps = pickleEvent.getPickleEvent().pickle.getSteps();

I plan to upgrade the cucumber version to 5.6.0 (cucumber-testng, cucumber-java, cucumber-picocontainer).  When I did that, I was able to perform all necessary migrations, package updations, etc, however, 

since gherkin based Pickle class is package-private inside the testng based Pickle class, the above does not work for me.  Tried using the following in version 5.6.0:

@Test(description = "Runs Cucumber Scenarios", dataProvider = "scenarios")
public void scenario(PickleWrapper pickleEvent, FeatureWrapper cucumberFeature) throws Throwable {
    List<Step> steps = pickleEvent.getPickle().getPickle().getSteps();

It shows an error on the highlighted getPickle() above as this:

'getPickle()' is not public in 'io.cucumber.testng.Pickle'. Cannot be accessed from outside package

When I inspected the Pickle class in io.cucumber.testng, found this:

package io.cucumber.testng;

import java.net.URI;
import java.util.List;
import org.apiguardian.api.API;
import org.apiguardian.api.API.Status;

@API(
    status = Status.STABLE
)
public final class Pickle {
    private final io.cucumber.core.gherkin.Pickle pickle;

    Pickle(io.cucumber.core.gherkin.Pickle pickle) {
        this.pickle = pickle;
    }

    io.cucumber.core.gherkin.Pickle getPickle() {
        return this.pickle;
    }

    public String getName() {
        return this.pickle.getName();
    }

    public int getScenarioLine() {
        return this.pickle.getScenarioLocation().getLine();
    }

    public int getLine() {
        return this.pickle.getLocation().getLine();
    }

    public List<String> getTags() {
        return this.pickle.getTags();
    }

    public URI getUri() {
        return this.pickle.getUri();
    }
}

Can you please help as to how I can access the pickle.getSteps() method in my TestRunner implementation since it is working just fine with version 4.2.0, and this is blocking me from upgrading to cucumber-jvm 5.6.0?

Thanks,
Anukul

[MP Korstanje]

Heya,

this change was entirely intentional. `Pickles` are an implementation detail of Cucumber and an abstraction you shouldn't need when using the `TestNGCucumberRunner`.

What were you using the steps for?

If it is reporting I would recommend that you look into the `Plugin` system.

[praveen halhalli]

Hi Anukuk, MP,

Just to confirm, can we get Scenario step names with  TestNGCucumberRunner, with cucumber version 4.2.0? I am in need of this implementation from months.

Could you guys help me here?

[Anukul Singhal]

Hi MP,

I already have a plugin written to capture the step and its text for our custom reporter we have written.  However, this requirement is different, as I have another DataProvider created within my TestRunner to generate possible combinations (as required by our framework) based on the Step.  Here is our DataProvider implementation as residing in our TestRunner class:

@DataProvider(name = "TestCaseContainer", parallel = true)
public Object[][] scenarios() {
    Object[][] scenarios = testNGCucumberRunner.provideScenarios();
    ArrayList<Object[]> dataProvider = new ArrayList<>();
    int totalScenario = scenarios.length;
    CombinationGenerator combinationGenerator = new CombinationGenerator();
    List<TestCaseContainer> testCaseContainerList = new ArrayList<>();
    for (int i = 0; i < totalScenario; i++) {
        PickleWrapper pickleEventWrapper = (PickleWrapper) scenarios[i][0];
        Pickle pickle = pickleEventWrapper.getPickle();
        testCaseContainerList.addAll(combinationGenerator.generateCombinations(pickle.getPickle().getSteps()));
        testCaseContainerList.forEach(testCaseContainer -> {
            Object[] dataPr = new Object[]{pickleEventWrapper, testCaseContainer};
            dataProvider.add(dataPr);
        });
    }
    return dataProvider.toArray(new Object[0][]);
}

As you can see, the first highlighted line, I am getting all the scenarios from the testNGCucumberRunner object. I need to internally use it and get

access to the Pickle object to get access to the Steps so I can create my own custom data provider based on our framework requirement.

Since the gherkin pickle implementation is hidden now, I am unable to use pickle.getPickle().getSteps().  I cannot do this in the plugin system since my 

@Test needs my custom data provider to work with in the beginning, and the plugin system would generally be invoked in the following order - @BeforeClass -> @Test -> TestStepEventStarted -> Hooks.

My implementation requires me to form the data provider initially, and let it rip that in parallel.

Please advise how I can access steps at the TestRunner level? (as I was able to do it in version 4.2.0)

Thanks,

Anukul

[Anukul Singhal]

Hi Praveen,

I had written a formatter plugin which has the following generic code that you can use to capture the step text (this is as per cucumber 4.2.0, note that this is not the complete code as I have removed my custom reporting implementation), but this should get you going:

public class CustomFormatter implements ConcurrentEventListener {

    private static final org.slf4j.Logger LOGGER = org.slf4j.LoggerFactory.getLogger(CustomFormatter.class);

    private EventHandler<TestStepStarted> stepStartedHandler = new EventHandler<TestStepStarted>() {
        @Override
        public void receive(TestStepStarted event) {
            handleTestStepStarted(event);
        }
    };

    private EventHandler<TestStepFinished> stepFinishedHandler = new EventHandler<TestStepFinished>() {
        @Override
        public void receive(TestStepFinished event) {
            try {
                handleTestStepFinished(event);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    };

    @Override
    public void setEventPublisher(EventPublisher publisher) {
        publisher.registerHandlerFor(TestStepStarted.class, stepStartedHandler);
        publisher.registerHandlerFor(TestStepFinished.class, stepFinishedHandler);
    }

    private void handleTestStepStarted(TestStepStarted event) {
        if (event.getTestStep() instanceof PickleStepTestStep) {
            PickleStepTestStep testStep = (PickleStepTestStep) event.getTestStep();
            LOGGER.info("TEST STEP STARTED : " + testStep.getStepText());
        }
    }

    private void handleTestStepFinished(TestStepFinished event) throws IOException {
        if (event.getTestStep() instanceof PickleStepTestStep && event.getResult().getStatus().is(Status.PASSED)) {
            PickleStepTestStep testStep = (PickleStepTestStep) event.getTestStep();
            LOGGER.info("TEST STEP FINISHED : " + testStep.getStepText());
        } else if (event.getTestStep() instanceof PickleStepTestStep && event.getResult().getStatus().is(Status.SKIPPED)) {
            //Do your stuff
        }
    }

}

You need to consume this CustomFormatter plugin in the CucumberOptions in your Runner class as follows (see highlighted use of plugin):

@CucumberOptions(features = {"<path_to_your_features>"},
        glue = {"<your_glue>"},
        plugin = {"path.to.your.CustomFormatter"},
        tags = {"@<your_tag>"})
public class TestRunner {
  ...

See if that helps.

Thanks,
Anukul

[M.P. Korstanje]

So why are you generating all combinations of steps? 

At a glance this doesn't seem like a core feature of Cucumber. So I don't see a good reason to support it. You may want to parse scenarios yourself using Gherkin directly.

-MP

--
Posting rules: https://cucumber.io/support/posting-rules
---
You received this message because you are subscribed to a topic in the Google Groups "Cukes" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cukes/_FyuiSUHqqE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cukes+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cukes/a5f00eb8-3e63-4ffd-a2e3-39bd279b6b06%40googlegroups.com.

[Anukul Singhal]

Hey MP,

Combination generation using steps is a little internal to our company, but in a nutshell, we are trying to create an exhaustive testing framework which kind of creates this based on customized gherkin we follow.  This feature of our framework might not be in accordance with the cucumber core, but our objective is to extend cucumber for our needs as well.

My point is, if it was working in version 4.2.0, and not working in version 5.6.0, I feel there is one less capability available.  I feel that cucumber as a tool should be extensible and the user should be able to create their own implementations by accessing public cucumber api's.  However, restricting the public api restricts us too.

Is there any specific reason to make getSteps() as private?

Also, as you mentioned - "You may want to parse scenarios yourself using Gherkin directly." - how can I do this at the TestRunner level?

Thanks,

Anukul

On Wednesday, May 6, 2020 at 12:48:25 PM UTC-5, M.P. Korstanje wrote:

So why are you generating all combinations of steps? 

At a glance this doesn't seem like a core feature of Cucumber. So I don't see a good reason to support it. You may want to parse scenarios yourself using Gherkin directly.

-MP

To unsubscribe from this group and all its topics, send an email to cu...@googlegroups.com.

[M.P. Korstanje]

> Is there any specific reason to make getSteps() as private?

The primary goal of Cucumber is to support BDD and the primary goal of the TestNGCucumberRunner is to facilitate integration with TestNG for the purpose to executing Cucumber scenarios. Now you may feel that Cucumber should be extensible. However extension is something that should be designed for and comes at a significant cost.

While you may feel this worth it just for your usecase; yours is but one usecase. As one of the maintainer of Cucumber I have to balance new features, bugs, general improvements and backwards compatibility against a rather limited amount of time. The smaller Cucumbers public API the easier this becomes and more effectively this can be done.

As I mentioned before Pickles are an implementation detail of Cucumber. Between v5 and v6 they've gone through significant changes to facilitate the new Rule keyword. A change that most users of Cucumber will not notice because the implementation is hidden. That they were exposed this way in v4 is unfortunate, that you came to depend on them even more so. Hiding these implementation details behind a well defined API will exactly prevent these misunderstandings in the future.

> how can I do this at the TestRunner level?

You can find examples of how to use the Gherkin parser here:

https://github.com/cucumber/cucumber/blob/master/gherkin/java/src/test/java/io/cucumber/gherkin/GherkinTest.java#L54

I reckon you'll be able to work out all the details from there.

To unsubscribe from this group and all its topics, send an email to cukes+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cukes/3ac1f444-5205-475d-a0a4-f6cd3a650c4a%40googlegroups.com.

[Anukul Singhal]

MP, this is an excellent answer and now I am totally aligned with your thoughts.  I understand the need for making the public API smaller, and hopefully, this will be for the betterment of Cucumber.

Will try out the solution you provided.

Thanks!

On Wednesday, May 6, 2020 at 3:13:50 PM UTC-5, M.P. Korstanje wrote:

> Is there any specific reason to make getSteps() as private?

The primary goal of Cucumber is to support BDD and the primary goal of the TestNGCucumberRunner is to facilitate integration with TestNG for the purpose to executing Cucumber scenarios. Now you may feel that Cucumber should be extensible. However extension is something that should be designed for and comes at a significant cost.

While you may feel this worth it just for your usecase; yours is but one usecase. As one of the maintainer of Cucumber I have to balance new features, bugs, general improvements and backwards compatibility against a rather limited amount of time. The smaller Cucumbers public API the easier this becomes and more effectively this can be done.

As I mentioned before Pickles are an implementation detail of Cucumber. Between v5 and v6 they've gone through significant changes to facilitate the new Rule keyword. A change that most users of Cucumber will not notice because the implementation is hidden. That they were exposed this way in v4 is unfortunate, that you came to depend on them even more so. Hiding these implementation details behind a well defined API will exactly prevent these misunderstandings in the future.

> how can I do this at the TestRunner level?

You can find examples of how to use the Gherkin parser here:

https://github.com/cucumber/cucumber/blob/master/gherkin/java/src/test/java/io/cucumber/gherkin/GherkinTest.java#L54

I reckon you'll be able to work out all the details from there.

To view this discussion on the web visit https://groups.google.com/d/msgid/cukes/3ac1f444-5205-475d-a0a4-f6cd3a650c4a%40googlegroups.com.