BDD Cucumber Testing

[Iain Macdonald]

I have moved this to the relevant forum as suggested..

Hi,

Firstly great work on the Client which is brilliant.

I'm really interested in integrating the ZAP API alongside our current test automation so I'm keen to understand the current status of the project using cucumber.

 https://github.com/continuumsecurity/bdd-security/tree/cukesecure

I appreciate its still a work in progress and there are obviously a few things not working.

Essentially we are just wanting to run the (app_scan.feature) tests initially then expand from there. 

When I run them as a JUnit Runner I keep getting an error when running via the JUnit runner:

package junit;
import cucumber.api.CucumberOptions;

@CucumberOptions(
        features = {
                "src/test/resources/cucumber/features"
        },
        glue = {"stepdefs"},
        tags = { "@scan_sql_injection" }
)
public class AppScanTestSuite extends BaseCucumberTestCase {

}

net.continuumsecurity.proxy.ProxyException: org.zaproxy.clientapi.core.ClientApiException: URL Not Found in the Scan Tree

at net.continuumsecurity.proxy.ZAProxyScanner.scan(ZAProxyScanner.java:201)

at stepdefs.security.AppScanningSteps.runScanner(AppScanningSteps.java:272)

at ✽.When the scanner is run(app_scan.feature:18)

Interestingly the tests for the Feature: Navigate App work absolutely fine out of the box but that uses the spider rather than scan method.

For now I have a local Tomcat deployed war of the ropeytasks website which is pointing at ropeytask in my config.xml

<baseUrl>http://localhost:8080/ropeytasks-0.1/</baseUrl> 

We are using the 2.4.3.jar included in the git project and running on a mac if that is relevant. 

Any advise or extra logs I need to include?

Thanks

Iain 

[Stephen de Vries]

Hi Iain,

Looks like you're using an old or customised version of AppScanTestSuite.  The current version explicitly sets the features to run and this works on my system at least!

https://github.com/continuumsecurity/bdd-security/blob/cukesecure/src/test/java/net/continuumsecurity/junit/AppScanTestSuite.java

[Iain Macdonald]

Thanks Stephen - All I have done is change the location of the feature files and the glue which is set in the BaseCucumberTestCase

Just to be clear the test and ZAP all start up perfectly fine its this step thats causing the issue.

@When("the scanner is run")
public void runScanner() throws Exception {
    
    getScanner().scan(Config.getInstance().getBaseUrl());
    int complete = 0;
    int scanId = getScanner().getLastScannerScanId();
    while (complete < 100) {
        complete = getScanner().getScanProgress(scanId);
        System.out.println(("Scan is " + complete + "% complete."));
        Thread.sleep(2000);
    }
}

Let me do some more investigation and I will come back to you.

Thanks

Iain

[Stephen de Vries]

I think you're getting that error because the navigate app doesn't run before the app scan test, so ZAP tries to scan first but has no knowledge of the URL.  

Forcing the order of scenarios in Cucumber is not officially supported, but they're supposed to run in alphabetic order, which is why app_nXXX is supposed to run before app_sXXX

Try setting the feature files explicitly and see if that fixes it.  I've posted a question on the Cucumber list to see whether there's a better way of doing this.

[Iain Macdonald]

Stephen,

OK I have pulled a clean copy from git of cukessecure and done a clean gradle build.

When I run the following Class net.continuumsecurity.junit.AppScanTestSuite I get exactly the same error. :-)

Can you rerun your test again with just the following tags set as I'm not too concerned by the detailed app_navigation features at this time.

tags = { "@scan_sql_injection" }

Can you ping me over your console output so I can see whats different.

Once I can get things working I would like to get involved in helping on the documentation side.

Thanks

Iain

[Stephen de Vries]

Can you rerun your test again with just the following tags set as I'm not too concerned by the detailed app_navigation features at this time.

tags = { "@scan_sql_injection" }

This is the problem!  You _must_ run the app_navigate feature before running the app_scan feature, else ZAP has no HTTP traffic to work with.  Try adding the tags as they were.

[Iain Macdonald]

Sorry Stephen your original request was crossed with an earlier post of mine the Application thing makes sense.

When I add it back in I get the same message unfortunately.

What are the expected outputs of the spider by the way as I'm not convinced its pulling details in properly. I guess its some kind of list of URI's?

int scanId = getSpider().getLastSpiderScanId();

status = getSpider().getSpiderProgress(scanId);

This shows the scan progress to 100% but I cant see what methods interact with the output.

When I debug the spider steps and run the following command the list is empty. Even though the scan progress is 100%.

getSpider().getSpiderResults(0)

If I'm using the ZAP IDE and hit http://localhost:8080/ropeytasks-0.1/ and start a spider I get the correct 15 URI's - Programatically this doesn't seem to be the case.

Any ideas ?

Thanks

Iain

[Stephen de Vries]

Any ideas ?

Try adding a slash to the end of the baseUrl in the config file:

<baseUrl>http://localhost:8080/</baseUrl>

I removed the slash and got the same error as you.  Have raised this as an issue.

[Iain Macdonald]

For the completeness of this thread I have successfully deployed the application using the ropeytasks.jar standalone jar i.e. java -jar ropeytasks.jar  I haven't fully diagnosed whats wrong but running under tomcat I have issues.

Thanks Stephen I have everything working now as expected on live webapps and localhost deployments.

I will document up your md file in git to give people a better guide on getting started.