ERROR org.parosproxy.paros.network.HttpBody - Failed to set charset: %s

[Vincent Yang]

Hi Team,

I use zap2docker to do the full scan, here are steps:

1. start zap:

docker run -v $(pwd):/zap/wrk/:rw --name zap_test -d owasp/zap2docker-stable zap.sh -daemon \
-port 8090 \
-host 0.0.0.0 \
-config api.disablekey=true \
-config scanner.attackOnStart=true \
-config scanner.delayInMs=0 \
-config scanner.maxScanDurationInMins=0 \
-config scanner.threadPerHost=1 \
-config api.addrs.addr.name=.* \
-config api.addrs.addr.regex=true

2. enter the docker: docker exec -it zap_test bash

3. run the full scan script: 

zap-full-scan.py -t https://192.168.1.1 -g gen.conf -r testreport.html -n /zap/wrk/testcontext

(I export the testcontext from ZAP GUI on Windows)

The scan can generate the report, but the Number of Alerts are less than the Manual scan report on Windows.

Check the zap.out, I found many errors: Failed to set charset: %s

23429 [ZAP-SpiderInitThread-0] INFO  org.zaproxy.zap.spider.Spider - Spider initializing...
23440 [ZAP-SpiderInitThread-0] INFO  org.zaproxy.zap.spider.Spider - Starting spider...
23552 [ZAP-SpiderThreadPool-0-thread-2] ERROR org.parosproxy.paros.network.HttpBody - Failed to set charset: %s
java.nio.charset.IllegalCharsetNameException: %s
        at java.nio.charset.Charset.checkName(Charset.java:308) ~[?:?]
        at java.nio.charset.Charset.lookup2(Charset.java:482) ~[?:?]
        at java.nio.charset.Charset.lookup(Charset.java:462) ~[?:?]
        at java.nio.charset.Charset.forName(Charset.java:526) ~[?:?]
        at org.parosproxy.paros.network.HttpBody.setCharset(HttpBody.java:563) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.network.HttpMessage.setResponseBody(HttpMessage.java:506) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.network.HttpSender.send(HttpSender.java:639) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.network.HttpSender.sendAuthenticated(HttpSender.java:602) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.network.HttpSender.sendAuthenticated(HttpSender.java:585) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.network.HttpSender.sendAndReceive(HttpSender.java:490) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.network.HttpSender.sendAndReceive(HttpSender.java:460) [zap-2.11.1.jar:2.11.1]
        at org.zaproxy.zap.spider.SpiderTask.fetchResource(SpiderTask.java:415) [zap-2.11.1.jar:2.11.1]
        at org.zaproxy.zap.spider.SpiderTask.runImpl(SpiderTask.java:178) [zap-2.11.1.jar:2.11.1]
        at org.zaproxy.zap.spider.SpiderTask.run(SpiderTask.java:150) [zap-2.11.1.jar:2.11.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]
23556 [ZAP-SpiderThreadPool-0-thread-2] ERROR org.parosproxy.paros.network.HttpBody - Failed to set charset: %s
java.nio.charset.IllegalCharsetNameException: %s
        at java.nio.charset.Charset.checkName(Charset.java:308) ~[?:?]
        at java.nio.charset.Charset.lookup2(Charset.java:482) ~[?:?]
        at java.nio.charset.Charset.lookup(Charset.java:462) ~[?:?]
        at java.nio.charset.Charset.forName(Charset.java:526) ~[?:?]
        at org.parosproxy.paros.network.HttpBody.setCharset(HttpBody.java:563) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.network.HttpMessage.setResponseBody(HttpMessage.java:506) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.network.HttpMessage.<init>(HttpMessage.java:258) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.db.RecordHistory.<init>(RecordHistory.java:64) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.db.paros.ParosTableHistory.build(ParosTableHistory.java:564) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.db.paros.ParosTableHistory.read(ParosTableHistory.java:390) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.db.paros.ParosTableHistory.write(ParosTableHistory.java:532) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.db.paros.ParosTableHistory.write(ParosTableHistory.java:429) [zap-2.11.1.jar:2.11.1]
        at org.parosproxy.paros.model.HistoryReference.<init>(HistoryReference.java:378) [zap-2.11.1.jar:2.11.1]
        at org.zaproxy.zap.extension.spider.SpiderThread.notifySpiderTaskResult(SpiderThread.java:533) [zap-2.11.1.jar:2.11.1]
        at org.zaproxy.zap.spider.Spider.notifyListenersSpiderTaskResult(Spider.java:847) [zap-2.11.1.jar:2.11.1]
        at org.zaproxy.zap.spider.SpiderTask.runImpl(SpiderTask.java:234) [zap-2.11.1.jar:2.11.1]
        at org.zaproxy.zap.spider.SpiderTask.run(SpiderTask.java:150) [zap-2.11.1.jar:2.11.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]
23628 [ZAP-SpiderThreadPool-0-thread-2] ERROR org.parosproxy.paros.network.HttpBody - Failed to set charset: %s

What's the reason zap report this kind errors?

Regards,

Vincent

[kingthorin+owaspzap]

Looks like the content-type header in the response doesn't contain a valid character set but instead specifies "%s" literally.

[Vincent Yang]

My test target https://192.168.1.1 did not use any special content-type, I captured the packets log when access the target, I can see the content-type:

Content-Type: text/html; charset=UTF-8\r\n

or

Content-Type: application/x-javascript\r\n

Looks no special.

[kingthorin+owaspzap]

Have you only been scanning with default settings or did/do you have Header input vectors enabled?

[Vincent Yang]

Yes, just default settings. Actually I checked Windows ZAP log, has the same error message.

I changed my test cmd in step 3 to:

zap-full-scan.py -t https://192.168.1.1 -g gen.conf -r testreport.html -n /zap/wrk/testcontext -a -j -U xxxx

The scan result looks good, Number of Alerts are not less than the Manual scan report on Windows now.