Docker Authenticated Scan Error - ERROR org.zaproxy.zap.extension.script.ScriptParam

[Meir Zeevi]

Hello,

I have M1 Pro chip, and I am trying to run an authenticated scan using a zst script (script works on UI) with the following command using Docker - 

Docker command

docker run  --platform linux/amd64 -p 8088:8088 -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-full-scan.py -t https://address.com -P 8088 -c zap-config.conf -x results-full.xml -n file.context -U user...@mail.com -z "-config script.scripts.name=auth.zst -config script.script.engine=MozillaZest -config script.script.type=authentication -config script.script.enabled=true -config script.script.file=$(pwd):/zap/wrk/auth.zst"

I can see the context, conf, and script files were copied successfully into the container. However, I get the following 2 errors:

First error

2023-01-18 13:04:27 24939 [ZAP-daemon] INFO  org.zaproxy.zap.extension.authentication.ExtensionAuthentication - Loaded authentication method types: [Form-based Authentication, HTTP/NTLM Authentication, Manual Authentication, Script-based Authentication, JSON-based Authentication]
2023-01-18 13:04:27 24960 [ZAP-daemon] INFO  org.parosproxy.paros.extension.ExtensionLoader - Initializing Log4j Extension - Logs errors to the Output tab in development mode only
2023-01-18 13:04:27 24961 [ZAP-daemon] INFO  org.parosproxy.paros.extension.ExtensionLoader - Initializing Users Extension - Users Extension
2023-01-18 13:04:27 24978 [ZAP-daemon] INFO  org.parosproxy.paros.extension.ExtensionLoader - Initializing Parameters Extension - Summarise and analyse FORM and URL parameters as well as cookies
2023-01-18 13:04:27 24983 [ZAP-daemon] INFO  org.parosproxy.paros.extension.ExtensionLoader - Initializing Script Extension - Script integration
2023-01-18 13:04:27 25011 [ZAP-daemon] ERROR org.zaproxy.zap.extension.script.ScriptParam - Error while loading the script: auth.zst
2023-01-18 13:04:27 java.lang.NullPointerException: null
2023-01-18 13:04:27     at java.io.File.<init>(File.java:278) ~[?:?]
2023-01-18 13:04:27     at org.zaproxy.zap.extension.script.ScriptParam.parse(ScriptParam.java:76) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.common.AbstractParam.load(AbstractParam.java:63) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.model.OptionsParam.addParamSet(OptionsParam.java:184) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.extension.ExtensionLoader.hookOptions(ExtensionLoader.java:1196) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.extension.ExtensionLoader.hookAllExtension(ExtensionLoader.java:925) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.extension.ExtensionLoader.startLifeCycle(ExtensionLoader.java:782) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.control.AbstractControl.loadExtension(AbstractControl.java:58) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.control.Control.init(Control.java:172) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.control.Control.initSingletonWithoutView(Control.java:405) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.zaproxy.zap.HeadlessBootstrap.initControl(HeadlessBootstrap.java:59) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.zaproxy.zap.DaemonBootstrap$1.run(DaemonBootstrap.java:75) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at java.lang.Thread.run(Thread.java:829) ~[?:?]
2023-01-18 13:04:27 25046 [ZAP-daemon] INFO  org.parosproxy.paros.extension.ExtensionLoader - Initializing ExtensionScripts - Scripting console, supports all JSR 223 scripting languages
2023-01-18 13:04:27 26046 [ZAP-daemon] INFO  org.parosproxy.paros.extension.ExtensionLoader - Initializing Forced User Extension - Forced User Extension
2023-01-18 13:04:27 26052 [ZAP-daemon] INFO  org.parosproxy.paros.extension.ExtensionLoader - Initializing HTTP Sessions Extension - Extension handling HTTP sessions

second error:

2023-01-18 13:04:27 48875 [ZAP-daemon] INFO  org.zaproxy.addon.network.ExtensionNetwork - ZAP is now listening on 0.0.0.0:8088
2023-01-18 13:04:27 50906 [ZAP-IO-Server-1-1] ERROR org.zaproxy.zap.authentication.ScriptBasedAuthenticationMethodType - Unable to find script while loading Script Based Authentication Method for name: auth
2023-01-18 13:04:27 51245 [ZAP-IO-Server-1-1] ERROR org.zaproxy.zap.extension.api.ContextAPI - null
2023-01-18 13:04:27 java.lang.NullPointerException: null
2023-01-18 13:04:27     at org.zaproxy.zap.authentication.ScriptBasedAuthenticationMethodType.persistMethodToSession(ScriptBasedAuthenticationMethodType.java:729) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.zaproxy.zap.extension.authentication.ExtensionAuthentication.persistContextData(ExtensionAuthentication.java:405) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.model.Model.saveContext(Model.java:548) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.model.Session.saveContext(Session.java:1266) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.parosproxy.paros.model.Session.importContext(Session.java:1565) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.zaproxy.zap.extension.api.ContextAPI.handleApiAction(ContextAPI.java:284) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.zaproxy.zap.extension.api.API.handleApiRequest(API.java:521) ~[zap-2.12.0.jar:2.12.0]
2023-01-18 13:04:27     at org.zaproxy.addon.network.internal.server.http.handlers.ZapApiHandler.handleApiRequest(ZapApiHandler.java:100) ~[?:?]
2023-01-18 13:04:27     at org.zaproxy.addon.network.internal.server.http.handlers.ZapApiHandler.handleRequest(ZapApiHandler.java:74) ~[?:?]
2023-01-18 13:04:27     at org.zaproxy.addon.network.internal.server.http.handlers.ZapApiHandler.handleMessage(ZapApiHandler.java:59) ~[?:?]
2023-01-18 13:04:27     at org.zaproxy.addon.network.internal.server.http.MainServerHandler.notifyMessageHandlers(MainServerHandler.java:147) ~[?:?]
2023-01-18 13:04:27     at org.zaproxy.addon.network.internal.server.http.MainServerHandler.processMessage(MainServerHandler.java:129) ~[?:?]
2023-01-18 13:04:27     at org.zaproxy.addon.network.internal.server.http.LocalServerHandler.processMessage(LocalServerHandler.java:66) ~[?:?]
2023-01-18 13:04:27     at org.zaproxy.addon.network.internal.server.http.MainServerHandler.process(MainServerHandler.java:94) ~[?:?]
2023-01-18 13:04:27     at org.zaproxy.addon.network.internal.server.http.MainServerHandler.lambda$channelRead0$0(MainServerHandler.java:82) ~[?:?]
2023-01-18 13:04:27     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
2023-01-18 13:04:27     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
2023-01-18 13:04:27     at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[?:?]
2023-01-18 13:04:27     at java.lang.Thread.run(Thread.java:829) ~[?:?]

Appreciate any help

[thc...@gmail.com]

Hi.

Some of the keys are not correct (missing s char).
https://www.zaproxy.org/faq/how-do-you-add-a-script-to-zap-from-the-command-line/

The value of "script.scripts.file" should be just "/zap/wrk/auth.zst".

(The initial output of ZAP shows the config keys/values being set.)


The second error is caused by the fist error.

Best regards.

On 18/01/2023 18:29, Meir Zeevi wrote:
> Hello,
>
> I have M1 Pro chip, and I am trying to run an authenticated scan using a
> zst script (script works on UI) with the following command using Docker -
>

> *Docker command*

>
> docker run --platform linux/amd64 -p 8088:8088 -v $(pwd):/zap/wrk/:rw -t
> owasp/zap2docker-stable zap-full-scan.py -t https://address.com -P 8088 -c
> zap-config.conf -x results-full.xml -n file.context -U user...@mail.com -z
> "-config script.scripts.name=auth.zst -config
> script.script.engine=MozillaZest -config script.script.type=authentication
> -config script.script.enabled=true -config
> script.script.file=$(pwd):/zap/wrk/auth.zst"
>
> I can see the context, conf, and script files were copied successfully into
> the container. However, I get the following 2 errors:
>

> *First error*

> *second error:*

[Meir Zeevi]

Hi thc202,

I updated the command line based on the FAQ page you shared above. Here is the new syntax:

docker run --platform linux/amd64 -p 8088:8088 -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-full-scan.py -t https://address.com -P 8088 -c zap-config.conf -x results-full.xml -n file.context -U use...@mail.com -z -config script.scripts.name="auth" -config script.script.engine="Mozilla Zest" -config script.script.type=authentication -config script.script.enabled=true -config script.script.file="/Users/mzeevi/zap/wrk/auth.zst"

Here is the new error:

$ cat zap.out
Found Java version 11.0.16
Available memory: 7851 MB
Using JVM args: -Xmx1962m
Failed due to invalid parameters: [-daemon, -port, 8088, -host, 0.0.0.0, -config, database.recoverylog=false, -config, api.disablekey=true, -config, api.addrs.addr.name=.*, -config, api.addrs.addr.regex=true, -config, spider.maxDuration=0, -addonupdate, -addoninstall, pscanrulesBeta, -addoninstall, ascanrulesBeta, -config]
Index 21 out of bounds for length 21
Use '-h' for more details.

I wonder if the quotations, and the script.script.type=authentication are correct...Any idea?

[thc...@gmail.com]

The -z is a parameter of the zap-full-scan.py script, you need to quote
its value (if you use single quotes I don't think you need to escape the
double quotes).

Note that the path to the script should be /zap/wrk/auth.zst
It's the path inside the Docker container.

Best regards.

[Meir Zeevi]

Hi thc202,

I tried to modify the command many times without a success. Do you know what might be a typo in the following command line?

docker run  --platform linux/amd64 -p 8088:8088 -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-full-scan.py -t https://address.com -P 8088 -c zap-config.conf -x results-full.xml -n file.context -U use...@mail.com -z '-config script.scripts.name="auth" -config script.script.engine="Mozilla Zest" -config script.script.type=authentication -config script.script.enabled=true -config script.script.file="/zap/wrk/auth.zst"'

Also, can I run the same scan with the same output with ZAP GUI? I might prefer that instead of continue troubleshooting the docker run.

[thc...@gmail.com]

There are still some config keys that are not correct, the key prefix
should be "script.scripts." (note the latter part is plural).


Yes, that should work the same if you do the same steps as the scan script.

Best regards.

On 18/01/2023 21:42, Meir Zeevi wrote:
> Hi thc202,
>

> I tried to modify the command many times without a success. Do you know
> what might be a typo in the following command line?
>
> docker run --platform linux/amd64 -p 8088:8088 -v $(pwd):/zap/wrk/:rw -t
> owasp/zap2docker-stable zap-full-scan.py -t https://address.com -P 8088 -c
> zap-config.conf -x results-full.xml -n file.context -U use...@mail.com

> <https://groups.google.com/> -z '-config script.scripts.name="auth" -config

[thc...@gmail.com]

I'd suggest looking at the Automation Framework, you can create (and
test) the plan in the GUI and once working as expected execute it in the
Docker container.
https://www.zaproxy.org/docs/automate/automation-framework/

(It allows to add scripts through the plan.)

Best regards.

[Meir Zeevi]

Hi thc202,

Thank you for your support. It looks like the script is loaded successfully after changing the script/s typo. However, I come across the following error now

9706 [ZAP-daemon] INFO  org.zaproxy.zap.control.ExtensionFactory - Installed add-ons: [[id=alertFilters, version=15.0.0], [id=ascanrules, version=51.0.0], [id=automation, version=0.21.0], [id=bruteforce, version=12.0.0], [id=callhome, version=0.6.0], [id=commonlib, version=1.12.0], [id=database, version=0.1.0], [id=diff, version=12.0.0], [id=directorylistv1, version=5.0.0], [id=domxss, version=14.0.0], [id=encoder, version=1.0.0], [id=exim, version=0.3.0], [id=formhandler, version=6.1.0], [id=fuzz, version=13.9.0], [id=gettingStarted, version=14.0.0], [id=graaljs, version=0.3.0], [id=graphql, version=0.12.0], [id=help, version=15.0.0], [id=hud, version=0.15.0], [id=invoke, version=12.0.0], [id=network, version=0.6.0], [id=oast, version=0.14.0], [id=onlineMenu, version=10.0.0], [id=openapi, version=31.0.0], [id=pscanrules, version=45.0.0], [id=quickstart, version=36.0.0], [id=replacer, version=12.0.0], [id=reports, version=0.18.0], [id=requester, version=7.1.0], [id=retest, version=0.5.0], [id=retire, version=0.18.0], [id=reveal, version=5.0.0], [id=scripts, version=34.0.0], [id=selenium, version=15.11.0], [id=soap, version=16.0.0], [id=spider, version=0.2.0], [id=spiderAjax, version=23.10.0], [id=tips, version=10.0.0], [id=webdriverlinux, version=47.0.0], [id=websocket, version=28.0.0], [id=zest, version=38.0.0]]
9713 [ZAP-daemon] INFO  org.zaproxy.zap.control.ExtensionFactory - Loading extensions
13186 [ZAP-daemon] INFO  org.zaproxy.addon.network.internal.TlsUtils - Using supported SSL/TLS protocols: [TLSv1.2, TLSv1.3]
14305 [ZAP-daemon] ERROR org.zaproxy.zap.ZAP.UncaughtExceptionLogger - Exception in thread "ZAP-daemon"
java.awt.AWTError: Can't connect to X11 window server using ':1.0' as the value of the DISPLAY variable.
        at sun.awt.X11GraphicsEnvironment.initDisplay(Native Method) ~[?:?]
        at sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:102) ~[?:?]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
        at sun.awt.X11GraphicsEnvironment.<clinit>(X11GraphicsEnvironment.java:61) ~[?:?]
        at java.lang.Class.forName0(Native Method) ~[?:?]
        at java.lang.Class.forName(Class.java:315) ~[?:?]
        at java.awt.GraphicsEnvironment$LocalGE.createGE(GraphicsEnvironment.java:101) ~[?:?]
        at java.awt.GraphicsEnvironment$LocalGE.<clinit>(GraphicsEnvironment.java:83) ~[?:?]
        at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:129) ~[?:?]
        at sun.awt.X11.XToolkit.<clinit>(XToolkit.java:232) ~[?:?]
        at java.lang.Class.forName0(Native Method) ~[?:?]
        at java.lang.Class.forName(Class.java:315) ~[?:?]
        at java.awt.Toolkit$2.run(Toolkit.java:588) ~[?:?]
        at java.awt.Toolkit$2.run(Toolkit.java:583) ~[?:?]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
        at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:582) ~[?:?]
        at javax.swing.ImageIcon.<init>(ImageIcon.java:198) ~[?:?]
        at javax.swing.ImageIcon.<init>(ImageIcon.java:217) ~[?:?]
        at org.zaproxy.zap.extension.hud.ExtensionHUD.<clinit>(ExtensionHUD.java:87) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
        at org.zaproxy.zap.control.AddOnLoaderUtils.loadAndInstantiateClassImpl(AddOnLoaderUtils.java:113) ~[zap-2.12.0.jar:2.12.0]
        at org.zaproxy.zap.control.AddOnLoaderUtils.loadAndInstantiateClass(AddOnLoaderUtils.java:64) ~[zap-2.12.0.jar:2.12.0]
        at org.zaproxy.zap.control.AddOnLoader.loadAddOnExtension(AddOnLoader.java:897) ~[zap-2.12.0.jar:2.12.0]
        at org.zaproxy.zap.control.AddOnLoader.loadAddOnExtensions(AddOnLoader.java:886) ~[zap-2.12.0.jar:2.12.0]
        at org.zaproxy.zap.control.AddOnLoader.getExtensions(AddOnLoader.java:844) ~[zap-2.12.0.jar:2.12.0]
        at org.zaproxy.zap.control.AddOnLoader.getExtensions(AddOnLoader.java:815) ~[zap-2.12.0.jar:2.12.0]
        at org.zaproxy.zap.control.ExtensionFactory.loadAllExtension(ExtensionFactory.java:108) ~[zap-2.12.0.jar:2.12.0]
        at org.parosproxy.paros.control.Control.addExtension(Control.java:201) ~[zap-2.12.0.jar:2.12.0]
        at org.parosproxy.paros.control.AbstractControl.loadExtension(AbstractControl.java:54) ~[zap-2.12.0.jar:2.12.0]

        at org.parosproxy.paros.control.Control.init(Control.java:172) ~[zap-2.12.0.jar:2.12.0]

        at org.parosproxy.paros.control.Control.initSingletonWithoutView(Control.java:405) ~[zap-2.12.0.jar:2.12.0]

        at org.zaproxy.zap.HeadlessBootstrap.initControl(HeadlessBootstrap.java:59) ~[zap-2.12.0.jar:2.12.0]

        at org.zaproxy.zap.DaemonBootstrap$1.run(DaemonBootstrap.java:75) ~[zap-2.12.0.jar:2.12.0]

        at java.lang.Thread.run(Thread.java:829) ~[?:?]

What do you suggest to do as a workaround? Is there a way to remove this extension as part of the command line?

[Meir Zeevi]

I tried to add the following config line to the command I used above -

-config hud.enabledForDaemon=false

so the command is:

docker run  --platform linux/amd64 -p 8088:8088 -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-full-scan.py -t https://address.com -P 8088 -c zap-config.conf -x results-full.xml -n file.context -U use...@mail.com -z '-config script.scripts.name="auth" -config script.scripts.engine="Mozilla Zest" -config script.scripts.type=authentication -config script.scripts.enabled=true -config script.scripts.file="/zap/wrk/auth.zst"

-config hud.enabledForDaemon=false'

But I get the same error..anything else possible as a workaround?

[thc...@gmail.com]

You would have to remove the add-on before starting ZAP, I'd suggest
using the live image which has that issue fixed.

The stable image will fixed at the latest in February.

Best regards.

[Meir Zeevi]

I believe that the Live docker image runs successfully, but doesn't produce the report file, even though it is part of the syntax I use

-x results-full.xml

Do you know why can be the reason? I don't see any errors on zap.out file

[Meir Zeevi]

Thank you for all your help thc202

Following on this thread...is there another way to get the results.xml file? The docker container doesn't create one on my local machine (I am using the Live docker image)

[Simon Bennetts]

We have a FAQ all about diagnosing docker issues.

In this case see https://www.zaproxy.org/docs/docker/diagnosing-problems/#cannot-access-created-files

Cheers,

Simon

[Meir Zeevi]

Hi Simon and thc202,

It seems like the tool doesn't generate the scan results because it fails to complete the scan, potentially, when it gets to the DomXssScanRule. I assume this is the issue because I don't see any logs after the following line

176444 [ZAP-Scanner-0] INFO  org.parosproxy.paros.core.scanner.HostProcess - start host https://address.com | DomXssScanRule strength MEDIUM threshold MEDIUM

I am trying to troubleshoot and see why the scan breaks. Is the zap.out file the only file to view logs inside the container? Can it happen because I use the Live docker image because of the bug in the stable version?

Thank you

[Meir Zeevi]

Hi Simon,

I followed the package scanning troubleshoot guide, and added the following to my syntax - "--hook=LogMessagesHook.py". Somehow, when I do it, the scan is completed and generates and results.xml file..

[Simon Bennetts]

So it works when you essentially add "debug" :/

What if thats an empty file?? :)

You could try removing sections of code to see what the minimum content is that makes it work?

Very strange...

Cheers,

Simon