Automation Framework in Azure Pipelines

[Philipp Riedel]

Hey everyone,

I am currently integrating ZAP into an azure Pipeline for automated scanning. I created with the desktop version of zap a .yml file (Baseline_Scan_URL.yaml) and I do want to start this automated framework in the azure pipeline. 

This command is working:
docker run --rm -v $(System.DefaultWorkingDirectory):/zap/wrk/:rw -v $(System.DefaultWorkingDirectory):/zap/report/ -t softwaresecurityproject/zap-stable zap.sh -cmd -autorun /zap/wrk/Baseline_Scan_URL.yaml

Now I do have two problems: 

- this PowerShell script is giving me an error and i dont know why

- With the desktop version i am able to find over 100 URLs, but with this script I only find 3

Thanks in advance for your help

[Simon Bennetts]

We're going to need a bit more info if we're going to be able to help you ;)

What error are you getting from powershell?

What does the plan do?

Are there any errors in the zap.log file?

https://www.zaproxy.org/faq/somethings-not-working-what-should-i-do/#check-the-log-file

Cheers,

Simon

[Philipp Riedel]

Thanks for your response.

The error: PowerShell exited with code '1' - there are no more informations about that :/ the problem with that is just that the overall pipeline fails and for example the later tasks are not running.

The plan is just a baseline scan.

The overall script:

- powershell: |
              docker pull ghcr.io/zaproxy/zaproxy:stable
              docker pull softwaresecurityproject/zap-stable
            displayName: 'Docker pull and run for OWASP ZAP'        
- powershell: |

              docker run --rm -v $(System.DefaultWorkingDirectory):/zap/wrk/:rw -v $(System.DefaultWorkingDirectory):/zap/report/ -t softwaresecurityproject/zap-stable zap.sh -cmd -autorun /zap/wrk/Baseline_Scan_URL.yaml

            displayName: 'OWASP ZAP Baseline Scan with automation framework'
 - task: PublishPipelineArtifact@1
            inputs:
              targetPath: '$(Build.ArtifactStagingDirectory)/report.html'
              artifactName: OWASPZAPREPORT
            displayName: 'Publish OWASP ZAP Report'  

Thanks in advance,

Philipp

[Simon Bennetts]

Can you see any errors on the command line?

Can you access the zap.log file?

https://www.zaproxy.org/faq/somethings-not-working-what-should-i-do/#check-the-log-file

Cheers,

Simon

[Philipp Riedel]

There are no errors beside the PowerShell Error and right know i'm not able to access the log-file

[Simon Bennetts]

Thats going to be a tough one to diagnose then.

I would start with a really simple plan that does pretty much nothing.

If that doesnt work then you have much fewer working parts to worry about.

When it does work then slowly add functionality until something breaks.

Cheers,

Simon

[Philipp Riedel]

Well, there is one more error (automation plan warnings): Job spider error accessing URL https://{my.website} status code returned : 404 expected 200

Is that the possible that this is the reason for the powershell error?

[Simon Bennetts]

Quite possibly yes - the plan will exit with an error code if it errors.

[Philipp Riedel]

okay thank you, this is the reason :) it it possible to deactivate this feature?

[Simon Bennetts]

The spider returning a 404 from the initial request is a pretty good indication that something is not right :)

Why would you want to ignore this?

The spider is not going to do anything useful if it hits a 404 page except in very unusual cases.

[Philipp Riedel]

Hi Simon,

thank you so much for your input. I was able to figure this out.

Next problem: i am running an active scan in the azure pipeline now. I do have everything configurated in the automation framework. Nonetheless I am getting different results when i am running the scan in the pipeline and in ZAP GUI.

Is it possible that the active scan is not working correctly in a pipeline with docker? Or are there any other possible mistakes i could have done in configurating?

Thanks in advance for your help

[Simon Bennetts]

How different are the results, and how do the times compare?

ZAP is running in a different environment so of course its possible it might be working in place and not in another.

The AF supports Statistics Job Tests so you can check that ZAP is doing what you expect

https://www.zaproxy.org/docs/desktop/addons/automation-framework/test-stats/ 

Cheers,

Simon

[Philipp Riedel]

It is basically just one external redirect that ZAP is not able to identify in the pipeline. Compared to the ZAP GUI, the active scan in the pipeline requires at least twice as much time.

Thanks,

Philipp

[Simon Bennetts]

That could be a scope problem?

Theres a good chance you are just reporting the issues on the target site, not external sites?

As for taking much longer, there are so many things that can affect this: processing power, network speed, number of threads etc etc

See https://www.zaproxy.org/faq/how-can-you-speed-up-scans/

Cheers,

Simon