What does Spider attack actually do?

[Beccy Stafford]

Hi folks,

Apologies for the probably basic question, I've spent a lot of time googling and not been able to find an answer to my question.

So, I'm completely new to ZAP and I'm learning how to best use it by myself. I've seen/heard some guidance around the internet that I should run a spider first, and then an active scan - to achieve the things I want to achieve. I was looking into what the Spider actually does, and found this page:

https://github.com/zaproxy/zap-core-help/wiki/HelpStartConceptsSpider

The thing that's confusing me is, those words indicate that a Spider will find all the urls in the application, but it doesn't give me any information on if the spider will also run some scans on each URL - but when I use ZAP from the UI I can see that some alerts come back, so the spider must be running some scans, but I don't understand what scans are being run.

I'm ultimately going to be running this programmatically via the ZAP API, and I'm just trying to understand if it's worth running a spider first, and then an active scan - or if I can just run an active scan which will cover the same things. If anyone can answer my question - what sort of scans does the spider do, if any? I'd be really grateful. 

Thanks,

Beccy

[Simon Bennetts]

Hi Beccy,

No need to appologize - if you couldnt find the answer googling then we havnt done a good enough job of documentation :)

All requests that are proxied through ZAP or initialised by tools like the spider are passively scanned: https://github.com/zaproxy/zap-core-help/wiki/HelpStartConceptsPscan
Passive scanning is always performed because it is completely safe - ZAP just looks at the requests and responses rather than making any additional requests.
This is good for finding problems like missing security headers or missing anti CSRF tokens but is no good for finding vulnerabilities like XSS which require malicious requests to be sent - thats the job of the active scanner.

Does that make more sense now?

Cheers,

Simon

[Beccy Stafford]

Hi Simon,

Ah - that makes a lot of sense now! Thank you so much :)

[Beccy Stafford]

Hi Simon,

Just to double check then, would an active scan find the same issues as a passive scan, with extra on top?

I'm trying to understand if it's worth running spider and then active, or if I can just run active and find everything :).

Thanks,
Beccy

[Simon Bennetts]

Hi Beccy,

No, they all do different things:)

The spider(s) explore the site. They dont actually do any scanning.
The passive scan rules examine all of the requests and responses flowing through ZAP and report the issues they can spot.
The active scan rules dont bother with the things the passive scan rules look for. They also wont explore a site.
If you just try running the active scan rules without exploring a site (either manually or using the spiders) then they wont find anything because there will be nothing to work on.
We try to keep things separate for flexibility, while providing packaged options like the Quick Scan and Baseline scan for those people who want them.

Does that make any more sense now?

Cheers,

Simon

[Sivabalan Ravichandran]

Hello Simon, Beccy,

Thanks for the fruitful discussion.

@Simon, the number of urls crawled by Spider differs every time. In most cases, the value of "URLs Found" is increasing when we run Spider for second time. Is there any logic behind this behaviour?

Thanks,

Siva.

[kingthorin+owaspzap]

Has the state or content of the app changed between runs of the spider? Have the access role or permissions of the user changed?

[thc...@gmail.com]

Some of the Spider options [1] might also affect how much is discovered
in the "first run", for example:
- Maximum depth to crawl;
- Maximum duration;
- Maximum children to crawl.

Also, following runs will have more seeds, which might allow to reach
other parts of the application (per constraints of previous options).


[1] https://github.com/zaproxy/zap-core-help/wiki/HelpUiDialogsOptionsSpider

Best regards.

[Sivabalan Ravichandran]

Hi,

No. There is no change [in spider setting/ target application/ user role] at all. But I do not understand, how following runs can crawl more urls.

Thanks,

Siva.

[Simon Bennetts]

Can you give us some sanitized examples of site nodes (URLs plus params) that were not found on the first run but were found on subsequent runs?

Cheers,

Simon

[kingthorin+owaspzap]

Did you proxy anything in the time between launching the first scan and the second scan?

[Sivabalan Ravichandran]

No. I have not and also couldn't find the exact urls that are newly found in second scan. But while scanning http://demo.testfire.net application, 97 urls were found in first scan and 105 were found in second time.

[thc...@gmail.com]

> But while scanning http://demo.testfire.net application, 97
> urls were found in first scan and 105 were found in second time.

How are the scans being started? Is the starting URL being manually
specified in both cases?

Could you provide the list of URLs found in both scans? If you are using
the UI you can select them all and copy them (with usual keyboard
shortcuts). If in daemon mode they can be obtained with
spider/view/fullResults.

Thanks.
Best regards.

[Sivabalan Ravichandran]

The scans were started via site tree [after intercepting via browser]. Also, I have set the context to demo.testfire and unable to export the urls that are out of context. Kindly find attached urls list obtained from two scans[with context].