Automation Active Scan different to Docker Full Scan

[dneece]

using the stable image.

Need help with configuring the Automation Framework to match the Docker full scan output. 

Automation active scan shows more warnings compared to the docker full scan and the number of vulnerabilities it's looking for doesn't match

Full Scan

FAIL-NEW: 0     FAIL-INPROG: 0  WARN-NEW: 9     WARN-INPROG: 0  INFO: 0 IGNORE: 0       PASS: 105

Automation Framework

FAIL-NEW: 0     FAIL-INPROG: 0  WARN-NEW: 15    WARN-INPROG: 0  INFO: 0 IGNORE: 0       PASS: 50

warnings and passes don't match

have attached (FYI - have altered the website with dummy words): 

• output for both scans
• config file for docker fullscan
• yaml file for automation framework - attempt at matching it with fullscan

[Simon Bennetts]

In the Automation Framework (AF) scan you are only installing the pscanrulesBeta add-on.

The Docker full scan includes the pscanrulesBeta add-on but also the ascanrulesBeta add-on (the Active Scan Beta Quality rules).

Include those in your AF plan and see what difference that makes.

Cheers,

Simon

[dneece]

I did do that, I seem to have sent you the yaml for my baseline scan.

here is the fullscan yaml file

[Simon Bennetts]

Well, now you are install pscanrulesAlpha :)

The docker full scan will only use the alpha scan rules if you specify the "-a" flag, and if you do that then it will also use the ascanrulesAlpha add-on.

If its not a "simple" config issue then we probably wont be able to help you without more info - the summary lines dont give enough info.

Find you which alerts are being raised in one of the scans and not the other - that may allow us to help you more...

Cheers,

Simon

[dneece]

I'm trying to recreate the docker full scan using the automation framework as docker full scan doesn't create an example yaml like how the baseline scan does.

I don't remember there being an option to turn alpha on for the automation framework scan.

Ok I have tried to run it again and now it seems clear the extra vulnerabilities was from the alpha active rules that I installed.

BUT now the results are the same as my baseline scan it is only showing passive warns.

Cross-referencing the output files that I gave you it seems the active rules are not being scanned in the automation framework setup

[Simon Bennetts]

Are any errors reported?

Either on the command line or in the zap.log file?

https://www.zaproxy.org/faq/somethings-not-working-what-should-i-do/#check-the-log-file

Cheers,

Simon

[dneece]

sorry replying again with same files, just removing some parts of the site being scanned

Here is the output on command line and zap.log

seems to show an error in command line but I don't understand it

I don't see any errors on zap log

[Simon Bennetts]

Some of those errors are from Firefox, so I dont understand them either :P

It looks like the domxss rule might be having problems, but that wouldnt explain a significant number of differences between the 2 scans.

How long is the AF scan taking? Is it significantly quicker than the packaged full scan?

Can you run the AF plan from the ZAP GUI?

If not remove _all_ of the parameters from the activeScan job - just to see if that makes a difference.

Cheers,

Simon

[dneece]

The AF scan is taking around 6-8 minutes (could be because it's taking a while to run through those errors)

The docker packaged scan is taking around the same time

I tried removing all the parameters from the active scan job and it remained the same.

Guess I will have to opt for using the docker package scan when running full scans.

However the docker packaged full scan doesn't detect the passive/beta vulnerabilities that the baseline scan detect, how do I fix that?

[dneece]

Ok got an update, I added the Generate report job to my automation framework yaml file to see if the issue was with the outputsummary job.

I was correct. the generated report using `traditional-html` template shows that the passive scans and the active scans are working as expected and matches the desktop GUI automated scan. 

not sure why the output summary works for passive but not active scans.

[Simon Bennetts]

The probable explanation that the output summary was specifically implemented to support the packaged baseline scan.

The plan is to migrate the API and full scans to use the Automation Framework as well, but it looks like the output summary job will need to be enhanced to support them :)

Thanks for letting us know!

Simon