ZAP Automation Framework - fail on given risk level

[Maciej Bała]

Hi There,

I was wondering if there is a way of using e.g. Job outcome tests, to make automation job fail if vulnerability with given level is found. I know I can use internat statistics to fail on certain vulnerability, but I was thinking of something more general.

I want to use it in my CI/CD pipeline so I can see that something regarding ZAP test has changed. Is there any other way to achieve this?

[Simon Bennetts]

Surprisingly there isn't an easy option right now.

The baseline script uses the outputSummary job - this is not really documented as its an 'internal' job just there for the packaged scan migration.

The alert tests can fail on specific alerts, but not on the presence of any alert at a given level.

What would a good job to handle this use case look like?

I think it should be different to the outputSummary job, so probably a new one?

One of the problems is how to define if "something has changed".

The baseline migration means we will need to support progress files, so the new job could handle those as well?

Suggestions appreciated :)

Many thanks,

Simon

[Maciej Bała]

Yeah, I've been looking for outputSummary documentation :) It's a bit misleading that I can add it to my Automation Framework job yet it seems to be useless atm ;)

Me and my devs have analyzed all alerts raised by ZAP and have either implemented solutions in app or flag alerts as FPs. I want my report to be clean - FPs in FP category, 0 HIGH/MED/LOW alerts.
So ideally I would like my job to inform me/fail when there is anything other than FP reported.
I would suggest something like stats.pscan.<rule-id>.alerts but with alert levels :)

I understand that as for now there is nothing I can do but verify the report manually.

Thank you for your feedback Simon, I can work with that. I mean at least I know where I am. Thanks

[Nicolò Mendola]

Hi,

we got the same "issue". We have build a CI/CD Pipeline with OWASP Docker Image and the Automation Feature. 

We would like to mark the Jenkins "Build" as Failed/Unstable if some Mid or High Alerts where found. Low and Informational are "ok" and build should be marked as succesfull.

First we hat the workarround with the outputSummary Feature, but the problem is that the low and informational Alerts are also counted as "WARN", so we cant easily parse the WARN field from json.

Next i try to make a html report AND a json report (not outputSummary), and try to parse the complete JSON report if there are some mid or higher alerts and mark the Jenkins build as unustable.

I think for the CI/CD pipline It would be easier , if the outputSummary would additionally count the summary of informational, low, medium and high alerts.

best regards

[Maciej Bała]

Thanks for heads up Nicolò, please let me know if it worked :)

--
You received this message because you are subscribed to a topic in the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zaproxy-users/Vdan6QRHfRM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/dc62ea41-9176-4687-81c8-8ea5b04a27f9n%40googlegroups.com.

--

Maciej Bała

[Nicolò Mendola]

Hi Maciej,

we got it running!

This is our Automation yaml Configuration for the json report (pretty standard):

- type: "report"
parameters:
template: "traditional-json"
reportDir: "/var/jenkins_home/workspace/SON_OWASP_Scan"
reportFile: "ZAP-JSON-Report"
reportTitle: "ZAP JSON Scanning Report"
reportDescription: "OWASP JSON Report"
displayReport: false
risks:
- "info"
- "low"
- "medium"
- "high"
confidences:
- "low"
- "medium"
- "high"
- "confirmed"

After that, we have following jenkins pipeline Snippet to parse the json result:

def resultsAsJson = new JsonSlurper().parseText(readFile('ZAP-JSON-Report.json'))
assert resultsAsJson instanceof Map
assert 'site' in resultsAsJson
def site = resultsAsJson.site
assert site instanceof List
assert site.size() == 1
def sonsite = site.get(0)
assert sonsite instanceof Map
def alerts = sonsite.alerts
assert alerts instanceof List

boolean highAlerts = alerts.any {it.riskdesc.startsWith('High')}
boolean mediumAlerts = alerts.any {it.riskdesc.startsWith('Medium')}

if (highAlerts || mediumAlerts){
currentBuild.result = 'UNSTABLE'
}

It's written a little bit properly to understand the single steps. I think the pipeline code would be shorter/easier if the outputSummary would be extended with more informations :)

Best Regards

[Gary Guo (Gary Guo)]

Hi  Nicolò , 

From your message , I learnt that  you have build a CI/CD Pipeline with OWASP Docker Image and the Automation Feature. Here is a problem I met with ZAP Automation Framework, how can I deal with the web authentication automatically in CI/CD pipeline ? 

  

[Nicolò Mendola]

Hi,

our Developer wrote an Authentication Java Script File which we defined in the context section under authentication in our yaml.

[Gary Guo (Gary Guo)]

Hi  Nicolò ,

 Could you share more details on the yaml or java script file ? how we configure the authentication parameters securely ? any guideline to follow ? 

[Simon Bennetts]

Authentication can be tricky to configure, but we are working on making that easier: https://www.zaproxy.org/blog/2023-01-19-authentication-help/

Right now my recommendation is to initially configure and test authentication in the ZAP desktop, even though you want to use ZAP in CI/CD.

Once you have it working create an AF plan in the desktop using the context you have configured.

When I do this I usually have a simple plan that sets up the environment and makes just one authenticated request.

Run the plan in the desktop - you should see the authentication requests and be able to confirm that it is working correctly.

Saving the plan in the desktop will generate a YAML file that you can use from the commandline.

AF plans support env vars so you do not need to hard code credentials.

This blog post: https://www.zaproxy.org/blog/2023-02-01-authenticating-using-selenium/ describes how to create an AF plan with complex authentication and uses env vars for the credentials.

If you need more help then just say - the more specific the questions the better :)

Cheers,

Simon

[phil young]

I did similar to this for ADO and Jenks, although I built a containered jar/jar respectively that parsed the output from the MD report Zap can produce. I was then able to have Expected vs Actuals and make a determination based on policy. My enhancement will be to add in specific alerts we want to fail or pass on and use this in conjunction with alerts marked as FP's in Zap.

[Gary Guo (Gary Guo)]

Hi  Nicolò  ,  how you make automation job fail in your CICD pipeline if vulnerability with given level is found, I mean how you get the scanning result from zap and pass the result to your CICD pipeline ?

I am working on this , but have no idea to figure it out .

在2023年2月24日星期五 UTC+8 04:24:09<nicolo....@gmail.com> 写道：

[Nicolò Mendola]

Hi ,

take a look at the complete conversation https://groups.google.com/g/zaproxy-users/c/Vdan6QRHfRM

On the 22 February 12 PM i wrote our Solution to parse the ZAP result.

[Gary Guo (Gary Guo)]

Hi  Nicolò  ,

 is your jenkins pipeline and zap installed on the same server ? 

[Nicolò Mendola]

Hi Gary, 

we are using the Docker Images for Zap and Jenkins. 

All configurations (automation yaml, authentication js) are commited in our git and will be pulled when owasp starts. 

Best regards

Nicolo

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/a761618d-85ce-46ad-8f1c-2aea913f9729n%40googlegroups.com.