Spider scan in Zap 2.10.0

[Vicky]

Hello,

I am using Zap desktop version 2.10.0. I have below questions and I will be very grateful to you if you can answer them.

Q1. I am using a simple Bodgeit application. I have configured the authentication and to test it, enabled the force user mode and verified that zap automatically logs in the application. Then I am doing the standard spider scan.

However, As you can see in the image above, only 2 urls found and out of which 1 is out of scope. Am I missing something here?

Q2. Some applications use auth tokens for authentication purpose. Those auth tokens expire in an hour. If the scanning and attacking takes more than 1 hour, will it cause any impact on the scan if the token gets expired?

Q3. While doing the baseline or full scan, we pass the target url. However, is it a right approach that first run your happy path functional test cases proxying through zap so that zap can learn your application and create the proper site tree. Then save this session and then start the full or baseline scan where you can pass the session created by the automated functional tests?

Q4. In the Context (Session Properties) menu, there is a section for Technology, where we can select the technology used by the application.But it seems limited. What if the technology which my application uses is not listed in it? Will it impact the scan and the results?

Q5. Context - Authentication - Verification Strategy:

Where do I see and how do I confirm if the verification strategy is successful or failed?

Q6. What is the difference between Persist Session and Snapshot Session as. My intention is to run the automated functional test cases proxying through zap and then save the session. So, which method should I use, persist or snapshot?

Q7. Suppose If I save a session. In few days, couple of pages get added to my application. Then I open the session saved earlier, click on the links on the newly added pages. This will add those pages in Zap's Site Tree (Please correct me if I am wrong here). Do I need to save this session again as new links/pages are added to site tree?

Q8. What is the use of the 2nd parameter "overwrite"?

What I have understood is that if the session with the given name (1st parameter) is already exists, then it responds saying "Already Exists". If that is the case then what is supposed to be entered in overwrite input field? Is there any way we can overwrite the existing session?

Q9. Is there any way to stop the Ajax Spider scan or an Active scan from zap desktop? As of now, I simply start the new session which kills the active scans and starts the new session

Thanks in advance,

Vicky

[Simon Bennetts]

Replies inline

Cheers,

Simon

On Monday, 12 July 2021 at 13:34:27 UTC+2 Vicky wrote:

Hello,

I am using Zap desktop version 2.10.0. I have below questions and I will be very grateful to you if you can answer them.

Q1. I am using a simple Bodgeit application. I have configured the authentication and to test it, enabled the force user mode and verified that zap automatically logs in the application. Then I am doing the standard spider scan.

However, As you can see in the image above, only 2 urls found and out of which 1 is out of scope. Am I missing something here?

Difficult to say without knowing exactly what your configuration is, but my first guess would be that you've made a mistake somewhere.

It looks like you are scanning as a user and enabling forced user mode - you shouldnt do that, you should use one or the other.

 

Q2. Some applications use auth tokens for authentication purpose. Those auth tokens expire in an hour. If the scanning and attacking takes more than 1 hour, will it cause any impact on the scan if the token gets expired?

If the app expires the token and rejects future requests with that token then yes, this will impact scanning - essentially you will then be performing an unauthenticated scan.

However if you can create a new token then you can configure ZAP to use that. Exactly how will depend on how you are generating and using your token.

 

Q3. While doing the baseline or full scan, we pass the target url. However, is it a right approach that first run your happy path functional test cases proxying through zap so that zap can learn your application and create the proper site tree. Then save this session and then start the full or baseline scan where you can pass the session created by the automated functional tests?

That will depend on the application - try out different strategies and see whats works best for your app.

In general proxying test cases is good as they will hopefully use more realistic data and should expose more of the application.

However then may not be complete (eeither now or in the future), in which case the spiders could find functionality not covered by your tests.

And if you dont have tests then the spiders become essential.

If you do have tests then I recommend proxying them through ZAP first and then using one or both of the spiders.

FYI if you want to "save" your session then do so at the start rather than halfway though.

ZAP sessions are stored in a db, if you persist it then the db files are copied - its much more efficiaent to do that early on rather than later.

One you have persisted a session then any updates to it will also be persisted.

 

Q4. In the Context (Session Properties) menu, there is a section for Technology, where we can select the technology used by the application.But it seems limited. What if the technology which my application uses is not listed in it? Will it impact the scan and the results?

The technology section can be used to tell ZAP if you do not use certain tech, in which case rules that target that tech can be ignore.

Many ZAP rules a tech agnostic, they should find the vulnerabilities they are looking for regardless of the tech you are using.

 

Q5. Context - Authentication - Verification Strategy:

Where do I see and how do I confirm if the verification strategy is successful or failed?

Via the ZAP authentication stats: https://www.zaproxy.org/docs/desktop/start/features/stats/

 

Q6. What is the difference between Persist Session and Snapshot Session as. My intention is to run the automated functional test cases proxying through zap and then save the session. So, which method should I use, persist or snapshot?

As above, persist the session at the very start.

A snapshot takes a full copy of the db. I think thats more useful when manual pentesting but if you do want to copy the entire ZAP session db in automation then you can do it as well.

 

Q7. Suppose If I save a session. In few days, couple of pages get added to my application. Then I open the session saved earlier, click on the links on the newly added pages. This will add those pages in Zap's Site Tree (Please correct me if I am wrong here). Do I need to save this session again as new links/pages are added to site tree?

No, the ZAP session is a db, any changes will also be persisted.

However if you keep using the same session then it will just grow bigger and bigger.

I'd recommend starting with a new session everytime.

 

Q8. What is the use of the 2nd parameter "overwrite"?

What I have understood is that if the session with the given name (1st parameter) is already exists, then it responds saying "Already Exists". If that is the case then what is supposed to be entered in overwrite input field? Is there any way we can overwrite the existing session?

"true"

 

Q9. Is there any way to stop the Ajax Spider scan or an Active scan from zap desktop? As of now, I simply start the new session which kills the active scans and starts the new session

Yes, all of the ZAP "scanners" have a "Stop" button, eg https://www.zaproxy.org/docs/desktop/ui/tabs/spider/

You can also pause and resume them.

 

Thanks in advance,

Vicky

[Vicky]

Thanks Simon for the quick reply

Regarding Q1, in the spider scan setting, Maximum Depth to Crawl, Maximum Children to Crawl was set to 1. This was the reason for the spider not to find many urls.

I am not sure if these settings have changed in 2.10.0. In 2.9.0, I guess the values were set to 5

Regarding Q3. as you mentioned " FYI if you want to "save" your session then do so at the start rather than halfway though."

I am using zap docker image. In order to save the session at the start:

zap.sh -v ${pwd}: /zap/wrk/:rw -daemon -host 0.0.0.0 -port 8090 -newsession "/zap/wrk "

This way the newly created session in /zap/wrk folder will be saved in ${pwd}. Is it a correct way to do it?

Also, whatever command line options are available (https://www.zaproxy.org/docs/desktop/cmdline/) , can they also be used for zap docker and automation framework?

Thanks again,

Vicky

[Vicky]

HI Simon,

I tried to create the session using zap docker image and pass the session in baseline scan but could not succeed.

Approach 1: using Zap docker image:

Below is my docker-compose.yml file

When I do docker-compose up, then in the logs, I could see that 12698 [ZAP-daemon] INFO  org.parosproxy.paros.control.Control - New session file created: /zap/wrk.session

After running the automated tests proxying through zap, running spider scan, I could see that no session file is created in the session folder in project directory

What is wrong here?

Approach 2: using baseline scan:

I am running baseline scan using the command. I am using -z option from command line scan to pass the session file.

Session file is kept at: C:/Users/abc/Downloads/zap-test/SessionFile/ My_Session.session

docker run -v C:/Users/abc/Downloads/zap-test:/zap/wrk/:rw -v C:/Users/abc/Downloads/zap-test/SessionFile:/home/zap/.ZAP/session/:rw -t owasp/zap2docker-stable zap-baseline.py -t http://www.example.com -c gen.conf -r testreport.html -p Progress.json --hook My_Hook.py -z "-session 'My_Session.session'"

I am getting the below error: Failed to open session: /home/zap/.ZAP/session/My_Session.session. However, I have done the volume mapping for this folder

What is wrong here?

Quick question: How does the Package API scan works? I guess it does not use the spider, then how it scans and attacks the APIs just based on the open api spec?

Also, do you have any demo example/video on how to configure the authentication for API scan?

Thanks and Regards,

Vicky

[Vicky]

Hello Simon,

I could fix the issue with Approach 1. I am able to create the session while running the automated tests proxying through Zap

Now, I want to pass this session to my baseline scan as I mentioned in my previous post in Approach 2.

Can someone help me to resolve the error in  Approach 2: using baseline scan: as mentioned in my previous post? What is the correct command to pass the session? Do I need to mount the directory where the session file is stored?

Documentation says (https://www.zaproxy.org/docs/desktop/cmdline/) , relative paths to session file are resolved against the “session” directory located in ZAP's home directory (default or specified with -dir option). But, in the owasp/zap2docker-stable image, there is no such "session" folder.

 

Thanks and Regards,

Vicky

[Vicky]

Hi Simon,

I could progress a little bit on this issue. It seems I could successfully pass the session to baseline scan (Please correct me if I am wrong)

Thanks and Regards,

Vicky

[Zhili X]

Hi Vicky,

I created the session via the first option on popup window when ZAP UI first starts.

Scanning with a saved session works fine on my local Docker container.  Not sure why it errors out when I push it to Azure DevOps.

Regards,

Zhili

 

On Mon, Jul 19, 2021 at 2:30 AM Vikrant Chaudhari <vikrantcha...@gmail.com> wrote:

Hi Zhil,

Sue, I will let you know if I get the solution on it.

How are you creating your session?

From the error "java.sql.SQLException: Database lock acquisition failure: lockFile: org.hsqldb.persist.LockFile@3bf9a665[file =/zap/wrk/session/google.session.lck, exists=false, locked=false, valid=false, ] method: openRAF reason: java.io.FileNotFoundException: /zap/wrk/session/google.session.lck (Permission denied) ", it seems that the session file is being used

by other processes and hence it is locked to avoid opening and modifying it by multiple users.

Once this session file is created, try closing that session and opening it in a new session. Because if the session which is creating this session file is still running then it means it is still accessing this file and hence it will not allow any other user/ session to access this file.

Thanks and Regards,

Vicky

On Sat, Jul 17, 2021 at 8:56 PM Zhili X <zhili...@gmail.com> wrote:

Hi Vicky,

I am experiencing the same issue but it seems no one has the answer so far.

https://groups.google.com/g/zaproxy-users/c/8zvH3Mlxcu4

If you find a solution, please let me know!

Regards,

Zhili

[Vicky]

Hi Simon,

Can you guide me on passing the session to package scan. How do I resolve the error?

Thanks and Regards,

Vicky

[Simon Bennetts]

Hiya Vicky,

Sorry, but why are you trying to pass the ZAP session to the packaged scans?

Thats not an approach I would normally recommend.

If its for authentication then the recommended approach is to get the authentication working in the desktop, then exporting the context and passing that to the packahed scan.

Cheers,

Simon

[Vikrant Chaudhari]

Hi Simon,

My application has forms and requires specific data to enter to move to the next page. Hence I am exploring the application from my automated test cases proxying through zap, running spiders on it, creating the site tree, and saving the session. Then pass this session to the packaged scans. Just passing the target url to package scans may not explore the application completely.

Is there any better approach to this?

Thanks and Regards,

Vicky

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/5c2943fa-c53f-456b-aef5-599f3c89f2ccn%40googlegroups.com.

[Simon Bennetts]

Hi Vicky,

You can configure the traditional spider to use specific values for form fields: https://www.zaproxy.org/docs/desktop/addons/form-handler/

Unfortunately thats not supported by the Ajax Spider yet (#3343).

You can always implement an Http Sender script which can change the requests and responses in any way you like: https://github.com/zaproxy/community-scripts/tree/main/httpsender

Can you explain in a bit more detail how these form values work?

Cheers,

Simon