Passive Scan Threading

[psiinon]

We are in the process of changing the ZAP Passive Scanner to support threading.

Currently there is only one passive scan thread and with big sites, or when using some add-ons like Wappalyzer, the passive scan thread queue can grow quite large and take a non trivial time to clear.

The changes will allow you to configure the number of passive scan threads.

Using a suitable number of threads does seem to significantly speed up the time taken to clear the passive scan queue and therefore reduce the overall scan time.

One of the side effects of this change is that all of the tasks run by the Passive Scanner will now need to be thread safe.

We have found that while most of the existing ZAP passive scan tasks have not needed to be changed there are some which did/do need changing, eg:

• Wappalyzer: #3723 #3767
• JsFunctionScanRule.java (changes in progress)

If you have written any code that is invoked by the Passive Scanner then you will need to change your code to be thread safe.

If you have any questions or concerns then please ask here..

Cheers,

Simon

[kingthorin+owaspzap]

Have you tested passive scan scripts w/ threading changes? (In particular with the graal engine, which we've had other reports of threading challenges with.)

[psiinon]

On Thu, May 5, 2022 at 4:20 PM kingthorin+owaspzap <kingt...@gmail.com> wrote:

Have you tested passive scan scripts w/ threading changes? (In particular with the graal engine, which we've had other reports of threading challenges with.)

No, thats now on the TODO list :D

As you mentioned we are aware of existing problems with the graal engine which we do plan to fix, but I definitely need to make sure these changes dont make anything worse..

Cheers,

Simon

 

On Thursday, May 5, 2022 at 10:55:27 AM UTC-4 psiinon wrote:

We are in the process of changing the ZAP Passive Scanner to support threading.

Currently there is only one passive scan thread and with big sites, or when using some add-ons like Wappalyzer, the passive scan thread queue can grow quite large and take a non trivial time to clear.

The changes will allow you to configure the number of passive scan threads.

Using a suitable number of threads does seem to significantly speed up the time taken to clear the passive scan queue and therefore reduce the overall scan time.

One of the side effects of this change is that all of the tasks run by the Passive Scanner will now need to be thread safe.

We have found that while most of the existing ZAP passive scan tasks have not needed to be changed there are some which did/do need changing, eg:

• Wappalyzer: #3723 #3767
• JsFunctionScanRule.java (changes in progress)

If you have written any code that is invoked by the Passive Scanner then you will need to change your code to be thread safe.

If you have any questions or concerns then please ask here..

Cheers,

Simon

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP Developer Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-devel...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/zaproxy-develop/4f3f1475-324d-4335-8526-9adac99ca91fn%40googlegroups.com.

--

OWASP ZAP Project leader

[psiinon]

For those of you who are interested:

• https://github.com/zaproxy/zaproxy/pull/7270 Support passive scan threading
• https://github.com/zaproxy/zap-extensions/pull/3770 PscanrulesAlpha - JS Function threading fix

[kingthorin+owaspzap]

How about the fuzzer? Have you enabled passive scanning for it and made sure it behaves? 😀

[psiinon]

Good suggestion!

It looks like it is not picking up changes to that option, and I've hit one ConcurrentModificationException so I'll look into those asap...

Cheers,

Simon

[kingthorin+owaspzap]

I did some testing with it this morning. Seems like it's in good shape.