Zap automation

[Hector Luna]

I am not sure if this has been asked already, but I have an automation plan in ZAP that works well when run from the GUI, but when run from the command line I get some weird issues:

1. The context does not seem to get included, or rather, it doesn't seem to be added to the context list in much the same way it does when run from the automation tab from the GUI.

2. When the scan completes, there is nothing in the report. It shows as empty, despite history showing everything that went on. Attempting to generate the report from the GUI returns the results as if I just run the automation from the gui.

3. Running the same automation plan with no daemon or gui always fails. Something about not being able to connect to localhost.

Thank you in advance!

[Simon Bennetts]

Does you context include any scripts?

If so you will need to make sure they are available in the environment where you are running ZAP from and have the right paths in the yaml file.

Also, if in doubt check the ZAP logs: https://www.zaproxy.org/faq/somethings-not-working-what-should-i-do/#check-the-log-file

Cheers,

Simon

[Hector Luna]

The context itself does not include any scripts (ie authentication, session, etc), but basically, the automation configuration includes a context name, the urls, and the included paths.
All other context parameters, etc are left with their default values.

In the jobs section, I specify all the rules (all of them) I want to use, the passive scan settings, a standalone script I want to run (which is a zest script to simulate login and out), and generate a repot after.

When I run the automation configuration in the GUI, it works as expected. It even adds the context I defined in the context section of the configuration file, with all the paths and all.
When I run the automation configuration from the command line, it runs the standalone script as expected, it even triggers the same alerts but it does not create a new context entry like the gui does, and the report does not show any results. Like nothing happened.

When I run the automation configuration from the command line and ZAP is not running in GUI or daemon mode, I get the connection issues when it tries to execute the standalone script.
I am not sure if that is because I need to actually run ZAP in some way or form in the background or not.

Thank you Simon!

[Hector Luna]

Also, when I look at the log file, there are no exceptions. Everything seems to work fine.
Weird thing is... when I run it, the passive scanner does not seem to wait to complete, tho it claims it does complete.

ie, there is no real wait at all, while the GUI does wait.
2022-03-24 09:32:41,689 [main ] INFO  CommandLine - Job passiveScan-config finished
2022-03-24 09:32:41,689 [main ] INFO  CommandLine - Job script started
2022-03-24 09:32:41,690 [main ] INFO  CommandLine - Job: script Start action: run
2022-03-24 09:33:37,361 [main ] INFO  CommandLine - Job script finished
2022-03-24 09:33:37,362 [main ] INFO  CommandLine - Job passiveScan-wait started
2022-03-24 09:33:38,867 [main ] INFO  CommandLine - Job passiveScan-wait finished

Thanks again!

[Hector Luna]

For added context, here are a few files that also seem to have the same issues:

Running it using this command: ./zap.sh -cmd -autorun `pwd`/automation/config-demo.yaml
"automation" is just the name of the folder where I have my automation files.
I do not specify the path to the zst file because it is already loaded into ZAP.
The path to the reports can be anywhere...

All the zest script is doing is going to https://zaproxy.org and that is it. I did a manual navigation to it and copied all the items in there into the "ZAPContext", and then added them to the "Zaproxy Test.zst" file.
The automation plan works well from the GUI. The reports generates as expected, and no issues. The command line on the other hand shows an empty report.

I am not sure what I am doing wrong, but this should work without issue shouldn't it?
Thanks!

[Simon Bennetts]

This is strange.

I've just tried running ZAP with those 2 files and it works fine for me using both the GUI and the command line.

In both cases the report is generated correctly.

Can you add the full output when running the automation plan from the commandline?

Also dopuble check to see if there are any errors in the zap.log file: https://www.zaproxy.org/faq/somethings-not-working-what-should-i-do/#check-the-log-file

Cheers,

Simon

[Hector Luna]

sure thing!
By the way, I am running zap from a VM running ubuntu. Not sure if that is the source of the issue?

Thanks Simon.

[Hector Luna]

Oh, and the gui has never been an issue. It always works. The command line is the issue.

The report generates "empty" when the running from the command line and:
1. The GUI is running.
2. Zap runs in daemon mode.

[Hector Luna]

There is the output when running the command no GUI, no daemon.

$ zap.sh -cmd -autorun `pwd`/zap/automation/config-zap.yaml
Found Java version 11.0.14
Available memory: 3889 MB
Using JVM args: -Xmx972m
Job authentication set parameters = {}
Job verification set method = response
Job verification set pollFrequency = 60
Job verification set pollUnits = requests
Job verification set pollUrl =
Job verification set pollPostData =
Job sessionManagement set method = cookie
Job sessionManagement set parameters = {}
Job passiveScan-config set maxAlertsPerRule = 10
Job passiveScan-config set scanOnlyInScope = true
Job passiveScan-config set maxBodySizeInBytesToScan = 0
Job passiveScan-config set enableTags = true
Job script set action = run
Job script set type = standalone
Job script set engine =
Job script set name = Zaproxy Test.zst
Job passiveScan-wait set maxDuration = 0
Job report set template = risk-confidence-html
Job report set theme = original
Job report set reportDir = /home/hmluna00/zap/reports
Job report set reportFile =
Job report set reportTitle = UAT ZAP Scanning Report
Job report set reportDescription =
Job report set displayReport = true
Job passiveScan-config started
Job passiveScan-config set rule 10,202 threshold to MEDIUM
Job passiveScan-config set rule 10,020 threshold to MEDIUM
Job passiveScan-config set rule 90,022 threshold to MEDIUM
Job passiveScan-config set rule 10,055 threshold to MEDIUM
Job passiveScan-config set rule 90,011 threshold to MEDIUM
Job passiveScan-config set rule 10,019 threshold to MEDIUM
Job passiveScan-config set rule 10,010 threshold to MEDIUM
Job passiveScan-config set rule 10,011 threshold to MEDIUM
Job passiveScan-config set rule 10,054 threshold to MEDIUM
Job passiveScan-config set rule 10,017 threshold to MEDIUM
Job passiveScan-config set rule 10,098 threshold to MEDIUM
Job passiveScan-config set rule 10,023 threshold to MEDIUM
Job passiveScan-config set rule 10,025 threshold to MEDIUM
Job passiveScan-config set rule 10,024 threshold to MEDIUM
Job passiveScan-config set rule 10,027 threshold to MEDIUM
Job passiveScan-config set rule 90,001 threshold to MEDIUM
Job passiveScan-config set rule 90,033 threshold to MEDIUM
Job passiveScan-config set rule 2 threshold to MEDIUM
Job passiveScan-config set rule 10,015 threshold to MEDIUM
Job passiveScan-config set rule 50,001 threshold to MEDIUM
Job passiveScan-config set rule 10,040 threshold to MEDIUM
Job passiveScan-config set rule 10,037 threshold to MEDIUM
Job passiveScan-config set rule 3 threshold to MEDIUM
Job passiveScan-config set rule 50,003 threshold to MEDIUM
Job passiveScan-config set rule 10,096 threshold to MEDIUM
Job passiveScan-config set rule 10,057 threshold to MEDIUM
Job passiveScan-config set rule 10,032 threshold to MEDIUM
Job passiveScan-config set rule 10,003 threshold to MEDIUM
Job passiveScan-config set rule 90,030 threshold to MEDIUM
Job passiveScan-config set rule 10,105 threshold to MEDIUM
Job passiveScan-config set rule 10,061 threshold to MEDIUM
Job passiveScan-config set rule 10,021 threshold to MEDIUM
Job passiveScan-config set rule 10,056 threshold to MEDIUM
Job passiveScan-config finished
Job script started

Job: script Start action: run

org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1] failed: Connection refused (Connection refused)
Job script finished
Job passiveScan-wait started
Job passiveScan-wait finished
Job report started
Job report generated report /home/hmluna00/zap/reports/2022-04-05-ZAP-Report-www.google-analytics.com.html
Job report finished
Automation plan succeeded!
ATTENTION: default value of option mesa_glthread overridden by environment.


This is what it looks like when I run it from the command line and the GUI is running.

$ zap.sh -cmd -autorun `pwd`/zap/automation/config-zap.yaml
Found Java version 11.0.14
Available memory: 3889 MB
Using JVM args: -Xmx972m
Job authentication set parameters = {}
Job verification set method = response
Job verification set pollFrequency = 60
Job verification set pollUnits = requests
Job verification set pollUrl =
Job verification set pollPostData =
Job sessionManagement set method = cookie
Job sessionManagement set parameters = {}
Job passiveScan-config set maxAlertsPerRule = 10
Job passiveScan-config set scanOnlyInScope = false
Job passiveScan-config set maxBodySizeInBytesToScan = 0
Job passiveScan-config set enableTags = true
Job script set action = run
Job script set type = standalone
Job script set engine =
Job script set name = Zaproxy Test.zst
Job passiveScan-wait set maxDuration = 0
Job report set template = risk-confidence-html
Job report set theme = original
Job report set reportDir = /home/hmluna00/zap/reports
Job report set reportFile =
Job report set reportTitle = UAT ZAP Scanning Report
Job report set reportDescription =
Job report set displayReport = true
Job passiveScan-config started
Job passiveScan-config set rule 10,202 threshold to MEDIUM
Job passiveScan-config set rule 10,020 threshold to MEDIUM
Job passiveScan-config set rule 90,022 threshold to MEDIUM
Job passiveScan-config set rule 10,055 threshold to MEDIUM
Job passiveScan-config set rule 90,011 threshold to MEDIUM
Job passiveScan-config set rule 10,019 threshold to MEDIUM
Job passiveScan-config set rule 10,010 threshold to MEDIUM
Job passiveScan-config set rule 10,011 threshold to MEDIUM
Job passiveScan-config set rule 10,054 threshold to MEDIUM
Job passiveScan-config set rule 10,017 threshold to MEDIUM
Job passiveScan-config set rule 10,098 threshold to MEDIUM
Job passiveScan-config set rule 10,023 threshold to MEDIUM
Job passiveScan-config set rule 10,025 threshold to MEDIUM
Job passiveScan-config set rule 10,024 threshold to MEDIUM
Job passiveScan-config set rule 10,027 threshold to MEDIUM
Job passiveScan-config set rule 90,001 threshold to MEDIUM
Job passiveScan-config set rule 90,033 threshold to MEDIUM
Job passiveScan-config set rule 2 threshold to MEDIUM
Job passiveScan-config set rule 10,015 threshold to MEDIUM
Job passiveScan-config set rule 50,001 threshold to MEDIUM
Job passiveScan-config set rule 10,040 threshold to MEDIUM
Job passiveScan-config set rule 10,037 threshold to MEDIUM
Job passiveScan-config set rule 3 threshold to MEDIUM
Job passiveScan-config set rule 50,003 threshold to MEDIUM
Job passiveScan-config set rule 10,096 threshold to MEDIUM
Job passiveScan-config set rule 10,057 threshold to MEDIUM
Job passiveScan-config set rule 10,032 threshold to MEDIUM
Job passiveScan-config set rule 10,003 threshold to MEDIUM
Job passiveScan-config set rule 90,030 threshold to MEDIUM
Job passiveScan-config set rule 10,105 threshold to MEDIUM
Job passiveScan-config set rule 10,061 threshold to MEDIUM
Job passiveScan-config set rule 10,021 threshold to MEDIUM
Job passiveScan-config set rule 10,056 threshold to MEDIUM
Job passiveScan-config finished
Job script started

Job: script Start action: run

Response: http://zaproxy.org/ passed = true code=302
Response: http://www.zaproxy.org/ passed = true code=301
Response: https://www.zaproxy.org/ passed = true code=200
Response: https://www.zaproxy.org/main.7dbba1.css passed = true code=200
Response: https://www.zaproxy.org/main.08cf90.js passed = true code=200
Response: https://fonts.googleapis.com/css?family=Quicksand:500,700 passed = false code=200
572451 [ZAP-ProxyThread-93] WARN  org.parosproxy.paros.core.proxy.ProxyThread - Failed to read https://www.googletagmanager.com/gtag/js?id=UA-71455536-1 within 20 seconds, check to see if the site is available and if so consider adjusting ZAP's read time out in the Connection options panel.
Response: https://www.googletagmanager.com/gtag/js?id=UA-71455536-1 passed = false code=504
Response: https://fonts.googleapis.com/css?family=Istok+Web%257COpen+Sans:400,700%257CRubik&display=swap passed = false code=200
Response: https://cdnjs.cloudflare.com/ajax/libs/lunr.js/2.3.6/lunr.min.js passed = true code=200
Response: https://fonts.gstatic.com/s/quicksand/v28/6xK-dSZaM9iE8KbpRA_LJ3z8mH9BOJvgkBgv58a-wg.woff2 passed = true code=200
Response: https://fonts.gstatic.com/s/opensans/v28/memSYaGs126MiZpBA-UvWbX2vVnXBbObj2OVZyOOSr4dVJWUgsg-1x4gaVI.woff2 passed = true code=200
Response: https://fonts.gstatic.com/s/opensans/v28/memSYaGs126MiZpBA-UvWbX2vVnXBbObj2OVZyOOSr4dVJWUgsjZ0B4gaVI.woff2 passed = true code=200
Response: https://fonts.gstatic.com/s/quicksand/v28/6xK-dSZaM9iE8KbpRA_LJ3z8mH9BOJvgkM0o58a-wg.woff2 passed = true code=200
593499 [ZAP-ProxyThread-97] WARN  org.parosproxy.paros.core.proxy.ProxyThread - Failed to read https://www.google-analytics.com/analytics.js within 20 seconds, check to see if the site is available and if so consider adjusting ZAP's read time out in the Connection options panel.
Response: https://www.google-analytics.com/analytics.js passed = false code=504
613538 [ZAP-ProxyThread-97] WARN  org.parosproxy.paros.core.proxy.ProxyThread - Failed to read https://www.google-analytics.com/j/collect?v=1&_v=j96&a=1405494435&t=pageview&_s=1&dl=https%3A%2F%2Fwww.zaproxy.org%2F&ul=en-us&de=UTF-8&dt=OWASP%20ZAP&sd=24-bit&sr=1870x1044&vp=1268x830&je=0&_u=YEBAAUABAAAAAC~&jid=244393815&gjid=216306495&cid=1907860492.1648141728&tid=UA-71455536-1&_gid=280050515.1648141728&_r=1&gtm=2ou3e0&z=980902847 within 20 seconds, check to see if the site is available and if so consider adjusting ZAP's read time out in the Connection options panel.
Response: https://www.google-analytics.com/j/collect?v=1&_v=j96&a=1405494435&t=pageview&_s=1&dl=https%253A%252F%252Fwww.zaproxy.org%252F&ul=en-us&de=UTF-8&dt=OWASP%2520ZAP&sd=24-bit&sr=1870x1044&vp=1268x830&je=0&_u=YEBAAUABAAAAAC~&jid=244393815&gjid=216306495&cid=1907860492.1648141728&tid=UA-71455536-1&_gid=280050515.1648141728&_r=1&gtm=2ou3e0&z=980902847 passed = false code=504
Job script finished
Job passiveScan-wait started
Job passiveScan-wait finished
Job report started
Job report generated report /home/hmluna00/zap/reports/2022-04-05-ZAP-Report-www.google-analytics.com.html
Job report finished
Automation plan succeeded!
ATTENTION: default value of option mesa_glthread overridden by environment.

Here is what the GUI looks like when I run the plan from the command line.

Here is what the GUI looks like when I run the plan from the GUI

Also included are the two reports it generates.

Maybe there is a setting that I am missing or something that I am doing wrong. Is there something I have to do in ZAP so that this works that I am not doing?

Thanks again!

[thc...@gmail.com]

The Zest script proxies through ZAP but in command line mode the proxy
is not started, in versions <= 2.11.

I'd suggest trying with a recent weekly release. Maybe Simon tested with
a weekly (or source?) and that's why it worked fine also in command line
mode.

Best regards.

>>>>> Running it using this command: *./zap.sh -cmd -autorun
>>>>> `pwd`/automation/config-demo.yaml*

[Hector Luna]

Oh!? I am going to give that a try.
Thanks a bunch!

[Hector Luna]

Well sir, I must thank you for that very useful suggestion. That fixed my issue.
I am wondering if I made the mistake of assuming the latest release (2.11.1) would do this, or if I missed the part where the documentation says that 2.11.1 doesn't support this feature.

At any rate. I thank you very much for the suggestion because latest does do what I need.
Does this mean that if I want to do the same thing but using a docker image that I should also look at weekly as well?

Thank you again.

[thc...@gmail.com]

I don't think that was documented but it behaved that way from the
beginning, it was changed recently to allow to e.g. proxy integration
tests while using the automation framework.

Right, you would have to use the weekly (or live) image.

You are welcome.

Best regards.