ZAP returned false positive once, didn't show up again

[C Batt]

Hi all, 

TLDR - Ran scan on two identical environments, one had a SQL Injection vulnerability.  Re-ran the scan without changing anything, SQL Injection no longer appeared.  Validity of Zap now being questioned.

I'm the admin for a third-party open-source via licence (only approved sites get the application but once you have it, you've got full access to the code) and we just had our 5th emergency upgrade in 3 months.  I did my usual full end-to-end testing (we're regulated so have to) and then once I was comfortable with the functionality, ran the ZAP Automated scan.  There's two production environments - let's call them "staff" and "public" - and on the "public", I had a number of medium / low vulnerabilities but nothing my InfoSec Office was concerned about.  On the "staff" version, I had the same vulnerabilities plus a high risk SQL Injection on some obscure page.  I flagged it to my InfoSec Office and they were concerned but asked me to post to the Application's Support Forum.  The suggestion was made to re-run the scan, which I did - not having changed anything, now the SQL Injection doesn't show up.  

This has thrown the confidence of Zap into question, and as our environment is regulated it's going to likely result in a pile of paperwork for me.  Any idea why this would have happened and how I can ensure the results I get are valid?

Chris

[psiinon]

TLDR - Dynamic scanning is hard ;) Also see https://www.zaproxy.org/faq/why-can-zap-scans-be-inconsistent/

I suspect the SQL Injection issue was due to a timing based attack - these are known to produce false positives (FPs) as systems under stess (eg ones being scanned by ZAP) tend to be slower than usual.

You should always double check that any vulnerability raised by an automated tool is valid and not an FP (even from tools that "claim" not to have FPs;).

FYI work is undergoing to rewrite the SQL timing attacks to make FPs much less likely.

If the alert was _not_ raised by a timing attack (that should be clear from the details) then let us know more.

Cheers,

Simon

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/10f5ab03-71a1-4092-9261-b12a2e086832n%40googlegroups.com.

--

OWASP ZAP Project leader

[C Batt]

Ahh - that makes sense!  Thanks so much, really appreciate your time.