Is URL a required field for Spider Scan or just Context is sufficient?

[Eswar Jayaraman]

Greetings, 

I have been using Zap for a couple of weeks now and I am trying to understand a behaviour in Zap Api spider scan. I was hoping I could get some help here. 

We use Zap Api to automate our security tests. As part of the setup we, 

1. Create a new Context by calling /json/context/action/newContext?contextName=<contextname>

2. Include Urls in the context  /json/context/action/includeInContext?contextName=<contextname>&regex=<url>

3. And then trigger a spider scan using /json/spider/action/scan?contextName=<contextname>

Since the context already has the Url, we didn't provide a URL again in the spider can endpoint. And this is what the documentation says as well:

Runs the spider against the given URL (or context). Optionally, the 'maxChildren' parameter can be set to limit the number of children scanned, the 'recurse' parameter can be used to prevent the spider from seeding recursively, the parameter 'contextName' can be used to constrain the scan to a Context and the parameter 'subtreeOnly' allows to restrict the spider under a site's subtree (using the specified 'url').

But we are getting an error if we don't provide a URL:

1340984 [ZAP-ProxyThread-66] WARN org.zaproxy.zap.extension.api.API  - Bad request to API endpoint [/JSON/spider/action/scan/] from [127.0.0.1]:

Missing Parameter (missing_parameter) : url

        at org.zaproxy.zap.extension.spider.SpiderAPI.scanURL(SpiderAPI.java:451)

        at org.zaproxy.zap.extension.spider.SpiderAPI.handleApiAction(SpiderAPI.java:225)

So is URL a required field to trigger spider scan even when the Context already has the URL?

This is how the context looks via /context/view/context/:

{

• context:

  {

  • excludeRegexs: [ ],
  • loggedInPattern: "",
  • urlParameterParserClass: "org.zaproxy.zap.model.StandardParameterParser",
  • description: "",
  • includeRegexs: "[http://localhost:9000/my-path]",
  • inScope: "true",
  • authenticationDetectionMethodId: "0",
  • loggedOutPattern: "",
  • postParameterParserConfig:

    {

    • kvps: "&",
    • kvs: "=",
    • struct: [ ]

    },
  • name: "7b91a78c-bed1-4040-b679-5961fa10430d",
  • urlParameterParserConfig:

    {

    • kvps: "&",
    • kvs: "=",
    • struct: [ ]

    },
  • id: "1",
  • authType: "Manual Authentication",
  • postParameterParserClass: "org.zaproxy.zap.model.StandardParameterParser"

  }

}

Any help or pointers is appreciated. 

Thank You

Eswar

[thc...@gmail.com]

Hi.

> Since the context already has the Url

Note that it's not a URL but a regular expression, ZAP does not use that
as seed for the spider (you might also need to include "child" URLs,
e.g. http://localhost:9000/my-path.* ).

Even if you specify a context ZAP still needs at least one URL (in its
session, e.g. accessed previously, proxy, sent manually, imported...) to
start spidering. Since ZAP does not have one it's requiring it.

Best regards.

> - context:
> {
> - excludeRegexs: [ ],
> - loggedInPattern: "",
> - urlParameterParserClass:
> "org.zaproxy.zap.model.StandardParameterParser",
> - description: "",
> - includeRegexs: "[http://localhost:9000/my-path]",
> - inScope: "true",
> - authenticationDetectionMethodId: "0",
> - loggedOutPattern: "",
> - postParameterParserConfig:
> {
> - kvps: "&",
> - kvs: "=",
> - struct: [ ]
> },
> - name: "7b91a78c-bed1-4040-b679-5961fa10430d",
> - urlParameterParserConfig:
> {
> - kvps: "&",
> - kvs: "=",
> - struct: [ ]
> },
> - id: "1",
> - authType: "Manual Authentication",
> - postParameterParserClass:

[Eswar Jayaraman]

Thank You thc202 for immediate response. 

Your clarification helps. So the regex provided in the context is only to limit the spider scan to the provided regex?

Regards
Eswar

[thc...@gmail.com]

Right, the context allows the spider to know which parts of the
application it can access.

Best regards.

[Eswar Jayaraman]

Thank You...