Bad Format when running second spider scan

[Brendan Farrell]

I am using OWASP Zap v2.5.  I installed it on a Mac using Homebrew.  I have a simple app that I have been testing against to understand how it works.  If i run two spider scans against the same target in the same session, it will hang and I see the below exception in the log.  If I delete the site and clean up the old scans, it will work. 

2017-01-26 09:04:26,721 [pool-3-thread-1] WARN  URLCanonicalizer - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://getbootstrap.com)

2017-01-26 09:04:26,781 [pool-3-thread-2] WARN  URLCanonicalizer - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://getbootstrap.com)

2017-01-26 09:04:26,822 [pool-3-thread-1] WARN  URLCanonicalizer - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: https://lodash%5C.com/%5C)

2017-01-26 09:04:26,996 [ZAP-ProxyThread-1] WARN  API - ApiException while handling API request:

Bad Format (bad_format)

at org.zaproxy.zap.extension.api.API.handleApiRequest(Unknown Source)

at org.parosproxy.paros.core.proxy.ProxyThread.processHttp(Unknown Source)

at org.parosproxy.paros.core.proxy.ProxyThread.run(Unknown Source)

at java.lang.Thread.run(Thread.java:745)

The 'error while processing url' seems like red herrings because this is a successful scan

2017-01-26 09:03:23,703 [Thread-8] INFO  Spider - Starting spider...

2017-01-26 09:03:23,749 [pool-1-thread-2] WARN  URLCanonicalizer - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: http://getbootstrap.com)

2017-01-26 09:03:23,828 [pool-1-thread-1] WARN  URLCanonicalizer - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: https://),url(https://),red

2017-01-26 09:03:23,828 [pool-1-thread-1] WARN  URLCanonicalizer - Error while Processing URL in the spidering process (on base ): Host could not be reliably evaluated from: https://)

2017-01-26 09:03:23,906 [pool-1-thread-2] INFO  Spider - Spidering process is complete. Shutting down...

[thc...@gmail.com]

Hi.

Are you using the ZAP API to start the scans?

What do you mean by hang?


Those warnings mean that the spider was not able to successfully extract
some URLs (weekly release should work better and provide more details
than that [1]), that might be normal depending on the parsers selected
(e.g. "Parse HTML comments").


[1] https://github.com/zaproxy/zaproxy/issues/2898

Best regards.

[Brendan Farrell]

It happens both running the Quick start from the UI or spider scans from the API.  It just sits at 68% on the second run and never completes.  

[thc...@gmail.com]

That's odd, can you still pause/stop the spider scan?

It would be great to see a thread dump with the spider stuck.

BTW, which client are you using to access the ZAP API? (bad format means
the requested URL is not properly formed)

Best regards.