Various questions from a newcomer

[anonymous tester]

Hello! I work as a software tester and have been trying to incorporate QAF in an automated testing project I have been working on. The main project uses Selenium, but before I start including QAF in it, I have been working on a small pilot project of sorts just to try out QAF and see if I can make it work. I have been struggling to make it work and have some questions I could use help with:

(My project's pom file uses TestNG 6.14.3 and QAF 2.1.14)

1. When I try to initiate a simple run of a scenario from Gherkin feature that uses a JSON dataFile as Examples (and also when I try to enter the indata examples in the feature file), I get the following error text:

[RemoteTestNG] detected TestNG version 6.14.3

log4j:WARN No appenders could be found for logger (com.qmetry.qaf.automation.testng.pro.QAFAnnotationTransformer2).

log4j:WARN Please initialize the log4j system properly.

org.testng.TestNGException: 

The factory method class com.qmetry.qaf.automation.step.client.ScenarioFactory.getTestsFromFile() threw an exception

at org.testng.internal.FactoryMethod.invoke(FactoryMethod.java:197)

at org.testng.internal.TestNGClassFinder.processFactory(TestNGClassFinder.java:223)

at org.testng.internal.TestNGClassFinder.processMethod(TestNGClassFinder.java:179)

at org.testng.internal.TestNGClassFinder.processClass(TestNGClassFinder.java:171)

at org.testng.internal.TestNGClassFinder.<init>(TestNGClassFinder.java:121)

at org.testng.TestRunner.initMethods(TestRunner.java:370)

at org.testng.TestRunner.init(TestRunner.java:271)

at org.testng.TestRunner.init(TestRunner.java:241)

at org.testng.TestRunner.<init>(TestRunner.java:192)

at org.testng.remote.support.RemoteTestNG6_12$1.newTestRunner(RemoteTestNG6_12.java:33)

at org.testng.remote.support.RemoteTestNG6_12$DelegatingTestRunnerFactory.newTestRunner(RemoteTestNG6_12.java:66)

at org.testng.SuiteRunner$ProxyTestRunnerFactory.newTestRunner(SuiteRunner.java:713)

at org.testng.SuiteRunner.init(SuiteRunner.java:260)

at org.testng.SuiteRunner.<init>(SuiteRunner.java:198)

at org.testng.TestNG.createSuiteRunner(TestNG.java:1295)

at org.testng.TestNG.createSuiteRunners(TestNG.java:1273)

at org.testng.TestNG.runSuitesLocally(TestNG.java:1128)

at org.testng.TestNG.runSuites(TestNG.java:1049)

at org.testng.TestNG.run(TestNG.java:1017)

at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:115)

at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:251)

at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:77)

Caused by: java.lang.IllegalArgumentException: wrong number of arguments

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

at java.lang.reflect.Method.invoke(Unknown Source)

at org.testng.internal.FactoryMethod.invoke(FactoryMethod.java:167)

... 21 more

Why does it keep happening?

2. When I use QAF to run test scenarios from a Gherkin file, it's supposed to make use of the class GherkinScenarioFactory to parse the Gherkin features. The QAF website says Cucumber hooks have to be converted "to appropriate TestNG listener". Does this include hooks such as Given, Then, etc.? Do the Cucumber steps need to be tagged as @QAFTestStep or is it enough to tag the step definition class as @QAFTestStepProvider and leave the Cucumber hooks as they are? I found a StackOverflow entry (https://stackoverflow.com/questions/39899242/is-it-possible-to-migrate-from-cucumber-jvm-to-qaf), a commenter wrote "QAF only supports @Given@then` @when & not the cucumber hooks"; is this accurate?

3. When feature examples are given in feature files as "Examples: {'dataFile':'path/To/File.json'}", does a DataProvider need to be given in the code?

4. Where is it appropriate to use the @QAFDataProvider tag? I have done some testing using TestNG, where the DataProvider tag was given in the metadata for test functions. In those, you could link the providers by name as well as data provider class, but from what I've been able to tell, the @QAFDataProvider tag doesn't allow those parameters.

5. The Stack Overflow post linked to in 2. also included a comment mentioning that QAF doesn't need a runnertest class to run the test scenarios. Does that mean that it should be possible to launch the tests using only the testng.xml file?

I appreciate any help and advice I can get.

Best regards

[cjayswal]

Welcome to QAF users community....

I would recommend to go through qaf-step-by-step-tutorial for quick understanding. QAF uses TestNG version 6.10 so you need to try with TestNG 6.10 instead of 6.14.3. Please find comments inline hope that will address to your questions.

सोमवार, 25 फ़रवरी 2019 को 1:26:20 अपर UTC-8 को, anonymous tester ने लिखा:

Try after removing TestNG dependency or use TestNG version 6.10

2. When I use QAF to run test scenarios from a Gherkin file, it's supposed to make use of the class GherkinScenarioFactory to parse the Gherkin features. The QAF website says Cucumber hooks have to be converted "to appropriate TestNG listener". Does this include hooks such as Given, Then, etc.?

Given, When, Then are not hooks so you can continue using Given, When, Then step annotation from cucumber.

Do the Cucumber steps need to be tagged as @QAFTestStep or is it enough to tag the step definition class as @QAFTestStepProvider and leave the Cucumber hooks as they are?

It's matter of choice. If you don't want cucumber library in dependency you can replace @Given/When/Then with  @QAFTestStep otherwise add  @QAFTestStepProvider at class level where you have step with @Given/When/Then annotation. Keep in mind hooks are different than step (method with @Given/When/Then). I would suggest to use @QAFTestStep instead of @Given/When/Then because, if you are using @QAFTestStep :

1. You don't need to write regular expression
   
2. You don't need additional cucumber dependency

I found a StackOverflow entry (https://stackoverflow.com/questions/39899242/is-it-possible-to-migrate-from-cucumber-jvm-to-qaf), a commenter wrote "QAF only supports @Given@then` @when & not the cucumber hooks"; is this accurate?

 Yes, QAF doesn't supports cucumber hooks instead it supports TestNG listeners and qaf listeners. If you are migrating from existing cucumber project to QAF convert Cucumber hooks to appropriate TestNG listener, for example convert @before implementation to testng method listener’s beforeMethod

3. When feature examples are given in feature files as "Examples: {'dataFile':'path/To/File.json'}", does a DataProvider need to be given in the code?

If you want to use custom data provider then only you need to provide DataProviderName and DataProviderClass. Refer documentation.

4. Where is it appropriate to use the @QAFDataProvider tag? I have done some testing using TestNG, where the DataProvider tag was given in the metadata for test functions. In those, you could link the providers by name as well as data provider class, but from what I've been able to tell, the @QAFDataProvider tag doesn't allow those parameters.

 @QAFDataProvider is used with @Test to specify qaf inbuilt data-provider behavior when you are authoring your test in Java. If you want to use your custom data provider you don't need to use @QAFDataProvider but provide data-provider name and class in @Test annotation . For custom-data provider with BDD you need to set using dataProvider and dataProviderClass meta-data.

5. The Stack Overflow post linked to in 2. also included a comment mentioning that QAF doesn't need a runnertest class to run the test scenarios. Does that mean that it should be possible to launch the tests using only the testng.xml file?

 You need to create Testng config file with appropriate factory class for BDD syntax you are using.

[anonymous tester]

Hi, cjaswal! Thanks for replying quickly. I'm afraid the Step by Step guide is of limited use for me since I'm trying to apply QAF to a project that originally used Cucumber with Gherkin features and the BDD example in the guide isn't practical to apply to the larger project I plan to apply it to later.

Regarding 1, I tried to change the TestNG dependency to 6.10. The noted exception doesn't occur then, but unfortunately, the test also doesn't run the available test scenario, regardless of whether or not the scenarios tag is included in the <include> tag in the testng.xml file. It also doesn't work if I remove the tag from the feature file and/or remove the entire <groups> tag from the XML file. When I tried to remove the TestNG dependency entirely, the test runs with TestNg 6.14.3 and the original exception occurs. Have Cucumber dependencies or any other dependencies been known interfere with QAF? (I also tried to run again with TestNG 6.10 after changing the Given, Then, etc. hooks to QAFTestSteps; no tests were run still, but the feature file can link to the steps)

Re: 3 and 4: Let's say I don't want to use a custom data provider. If I add the @QAFDataProvider tag with a JSON file as dataFile to any steps that require indata of some sort, should it be able to parse a Map<String, Object> from the file as illustrated with a data provider using a CSV file here?

Re: 5: I know that and I am doing so, but my question was whether or not a runnertest class, such as one normally used to launch Cucumber tests, is necessary anymore if the tests can be launched using the XML file? Also, the <classes> tag in my current file contains this: <class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />

Best regards

[anonymous tester]

Hello again!

Another question occurred to me. Presently, in my bigger test project that uses Selenium and Cucumber, my WebDriver is defined as a static variable in the runnertest class that extends to the step definition class and any class that uses the WebDriver. There is also a fully functioning infrastructure for performing WebDriver actions such as clicking elements and for obtaining WebElement locators that I would like to keep using. If I were to implement QAF to this project, would I still be able to use those original functions and variables?

Best regards

[cjayswal]

मंगलवार, 26 फ़रवरी 2019 को 1:59:23 पूर्व UTC-8 को, anonymous tester ने लिखा:

Hi, cjaswal! Thanks for replying quickly. I'm afraid the Step by Step guide is of limited use for me since I'm trying to apply QAF to a project that originally used Cucumber with Gherkin features and the BDD example in the guide isn't practical to apply to the larger project I plan to apply it to later.

Step by Step guide is kind of quick start helpful to play with, understand different concept and get hands on. You can deep dive by trying different features following documentation using that sample project once you successfully completed first test.

Regarding 1, I tried to change the TestNG dependency to 6.10. The noted exception doesn't occur then, but unfortunately, the test also doesn't run the available test scenario, regardless of whether or not the scenarios tag is included in the <include> tag in the testng.xml file. It also doesn't work if I remove the tag from the feature file and/or remove the entire <groups> tag from the XML file. When I tried to remove the TestNG dependency entirely, the test runs with TestNg 6.14.3 and the original exception occurs. Have Cucumber dependencies or any other dependencies been known interfere with QAF? (I also tried to run again with TestNG 6.10 after changing the Given, Then, etc. hooks to QAFTestSteps; no tests were run still, but the feature file can link to the steps)

Make sure that you have provided scenario.file.loc property with correct value. If not provided QAF will consider scenarios directory for bdd/feature files. Refer document for bdd execution configuration.

Re: 3 and 4: Let's say I don't want to use a custom data provider. If I add the @QAFDataProvider tag with a JSON file as dataFile to any steps that require indata of some sort, should it be able to parse a Map<String, Object> from the file as illustrated with a data provider using a CSV file here?

The example provided in documentation is with CSV file and you can use any of the supported data-provider. For Json, refer json data provider to get idea of example json file.

Re: 5: I know that and I am doing so, but my question was whether or not a runnertest class, such as one normally used to launch Cucumber tests, is necessary anymore if the tests can be launched using the XML file? Also, the <classes> tag in my current file contains this: <class name="com.qmetry.qaf.automation.step.client.gherkin.GherkinScenarioFactory" />

You don't need any additional runner class while using QAF.

Best regards

[cjayswal]

Having webdriver object static will limit you to run tests in parallel. If you will keep WebDriver as a static variable you will not be able to take benefit of parallel execution or

see issues while running tests in parallel. While using QAF, driver management is taken care by QAF and you don't need unwanted complexity in code. All you need to set driver name and set appropriate capabilities. The migration approach depends on the current code, still as quick suggestion, i can say that remove  WebDriver static variable and add static getter method in the class as below. For example

    //public static Webdriver driver;
    public static WebDriver getDriver() {
        return new WebDriverTestBase().getDriver();
    }

Globally replace driver static variable call with getDriver() method call.

बुधवार, 27 फ़रवरी 2019 को 12:41:59 पूर्व UTC-8 को, anonymous tester ने लिखा:

[anonymous tester]

I have already set the scenario.file.loc as well as the step.provider.pkg parameter in the xml file. I have tried various ways of entering the feature file location; using the absolute path and using a path relative to the project location; entering the path to just the folder and including the exact feature file name. I even tried to place the feature in the "scenarios" directory and not include the parameter in the xml file at all so it would read the default location. I have included the tag of the scenario to run in <groups> and I have left <groups> out. I have tried removing the tags from the feature and xml file altogether. I even tried including the txt.scenario.file.ext parameter set to ".feature", even though the page says that isn't applicable for the Gherkin parser. I have tried entering the parameters on suite and test level. The test scenarios have never been able to run. What am I doing wrong?

Best regards

On Wednesday, February 27, 2019 at 9:07:49 PM UTC+1, cjayswal wrote:

...

Regarding 1, I tried to change the TestNG dependency to 6.10. The noted exception doesn't occur then, but unfortunately, the test also doesn't run the available test scenario, regardless of whether or not the scenarios tag is included in the <include> tag in the testng.xml file. It also doesn't work if I remove the tag from the feature file and/or remove the entire <groups> tag from the XML file. When I tried to remove the TestNG dependency entirely, the test runs with TestNg 6.14.3 and the original exception occurs. Have Cucumber dependencies or any other dependencies been known interfere with QAF? (I also tried to run again with TestNG 6.10 after changing the Given, Then, etc. hooks to QAFTestSteps; no tests were run still, but the feature file can link to the steps)

Make sure that you have provided scenario.file.loc property with correct value. If not provided QAF will consider scenarios directory for bdd/feature files. Refer document for bdd execution configuration.

... 

[cjayswal]

I would suggest to first try and play with sample project and then apply to you existing project so you will get idea what is missing. Alternate is try moving required code in sample project and see if it is working or not for few tests to get idea what is missing.

गुरुवार, 28 फ़रवरी 2019 को 1:34:29 पूर्व UTC-8 को, anonymous tester ने लिखा:

[anonymous tester]

I tried running the step by step example project. I had to make some adjustments to the step definition file to use a Robot to press Enter instead of pressing the Search button in the google method, but it ran fine when I ran it as a java test (as described in Step 3). However, when I tried to run it with the BDDTestFactory, using the SampleTest scenario in Step 2, an error occurred during the "When sendkeys 'Git reporsitory QAF' into 'txt.searchbox'" step:

org.openqa.selenium.TimeoutException: Timed out after 10 seconds waiting for presence of element located by: By.xpath: //*[@name=' into ' or @id=' into ' or @value=' into ']

When I Ctrl+click the step in the BDD file, it connects to the CommonStep class method sendKeys. Based on the test step description and the BDD step, I suppose it should get the txt.searchbox value from the home.properties and form an xpath from that, but the exception message suggests that the xpath is built from the text between the strings in the step text.