Active Scan Authentication via Automation Framework

[Illia]

Hi there,

I am using ZAP 2.13.0 and running active scan through automation framework. 

During the scan, in the console I can see that ZAP is authenticating by using the user that I set up. Please refer to SS2.png

I tried using the Authentication Tester tool and it shows that it can verify all four elements. 

Please refer to SS1.png 

However, when I check my site's audit log, it is not accessing using the site as I can't see any login activities and the username is blank. It is also not accessing all modules and repeating the few same modules over and over again.

How to solve this? Any advice/help is much appreciated!

[Simon Bennetts]

Hiya,

Can you share your AF plan, obfuscating anything sensitive like URLs.

Cheers,

Simon

[Illia]

Hi Simon,

Sure, here it is. Just for your info, I run by using this command in CMD: .\zap.bat -daemon -config connection.timeoutInSecs=60  -autorun TestDAST.yaml

---
env:
  contexts:
  - name: "TestDAST"
    urls:
    - "https://example.com"
    includePaths:
    - "https://example.com.*"
    excludePaths:
    - "https://example.com/logout.html"
    authentication:
      method: "form"
      parameters:
        loginPageUrl: "https://example.com/login.html"
        loginRequestUrl: "https://example.com/login.html"
        loginRequestBody: "obscuring_sensitive_info"
      verification:
        method: "response"
        loggedInRegex: "body_onload();submitForm('1');UserLogin('','/index.html','1');"
        loggedOutRegex: "<a href=\"JavaScript:Navigate('Login')\"> <span id=\"Label4\"\
          >Click Here to Login</span></a>"
        pollFrequency: 60
        pollUnits: "requests"
        pollUrl: ""
        pollPostData: ""
    sessionManagement:
      method: "cookie"
      parameters: {}
    technology:
      exclude: []
    users:
    - name: "ADMIN"
      credentials:
        password: "password"
        username: "ADMIN"
  parameters:
    failOnError: true
    failOnWarning: false
    progressToStdout: true
  vars: {}
jobs:
- parameters:
    type: "url"
    fileName: "C:\\Program Files\\OWASP\\Zed Attack Proxy\\URL_List.txt"
  name: "import"
  type: "import"
- parameters:
    context: "TestDAST"
    user: "ADMIN"
    policy: ""
    maxRuleDurationInMins: 0
    maxScanDurationInMins: 0
    addQueryParam: true
    delayInMs: 0
    handleAntiCSRFTokens: true
    injectPluginIdInHeader: false
    scanHeadersAllRequests: true
    threadPerHost: 16
  policyDefinition:
    defaultStrength: "high"
    defaultThreshold: "medium"
    rules: []
  name: "activeScan"
  type: "activeScan"
- parameters:
    template: "traditional-pdf"
    reportDir: "C:\\Users\\Downloads"
    reportFile: "ZAP_Report"
    reportTitle: "ZAP Scanning Report"
    reportDescription: ""
    displayReport: false
  risks:
  - "info"
  - "low"
  - "medium"
  - "high"
  confidences:
  - "low"
  - "medium"
  - "high"
  - "confirmed"
  sections:
  - "instancecount"
  - "alertdetails"
  - "alertcount"
  sites:
  - "https://example.com/"
  name: "report"
  type: "report"

[Simon Bennetts]

If you want to do the same things as the Authentication Tester dialog (which I recommend) then you should configure your AF plan context as per https://www.zaproxy.org/blog/2023-05-02-authentication-auto-detection/#an-example-automation-plan

ie using browser based authentication and auto-detect for everything else.

If you want to use your own config then you need to check again that its really working for you.

My recommendation is to create a plan which has all of your auth settings but which then just makes one authenticated request via the requester, eg like this one:

https://github.com/zaproxy/zaproxy/blob/main/docker/integration_tests/configs/plans/auth-simple-json.yaml

Run that in the ZAP desktop (if you can) and check to see what happens.

Cheers,

Simon

[Illia]

Heya Simon,

Thanks for the reply!

Unfortunately I don't think that my site can use browser based authentication for active scan as the site is not using token.

For the recommendation that you gave, do you think I will be able to run it with form based authentication? Because I saw in the link you gave that it is using browser based authentication.

[psiinon]

OK, then use the AF config you already have.

But dont do anything else other than try to make one authenticated request.

Then look at the requests and responses and try to see whats going on.

Its easier using the ZAP dektop, but if you cant use that then see this FAQ: https://www.zaproxy.org/docs/docker/diagnosing-problems/#investigating-non-trivial-issues

Cheers,

Simon

--
You received this message because you are subscribed to the Google Groups "ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/83c10484-516a-4fdb-824d-4c0abb7dc270n%40googlegroups.com.

--

ZAP Project leader

[Illia]

Hi there Simon,

I tried changing the command line to this:

.\zap.bat -daemon -config connection.timeoutInSecs=120 -autorun Example.yaml -session "path\to\session\file\test1.session"

I noticed that ZAP is now scanning all modules and it seems to be authenticated as the audit log is showing the username which I have set in the context.

However, when I tried to create another session file by repeating the same steps I used to create test1.session, and running the same command in CMD, the audit log is not showing all modules scanned.

I have a few questions regarding this:

1.  May I know your thoughts on using -daemon, -session and -autorun parameter in the same command? 
   
   
2. When I checked the zap.log file, I saw this output. Why is it saying null when I have specified the time out = 120 secs by using the -config parameter in the command line?
   
   [main ] INFO  AbstractParam - Setting config connection.timeoutInSecs = 120 was null
   
   
3. Why is the session files which I created by using similar steps are producing different outputs when I checked via my site's audit log?

I am new to ZAP so I really appreciate your replies. 

Thanks!

[Simon Bennetts]

I would have expected you to use "-cmd" instead of "-daemon".

The "-cmd" option runs ZAP from the command line (no UI) and exits once the other options have been applied, in this case after the AF plan has run.

The "-daemon" option puts ZAP into the background - it will keep running until you stop it, and will not exit after the plan has completed.

For more details see https://www.zaproxy.org/docs/desktop/cmdline/

Ideally the AF should support all of the session options meaning that the "-session" option becomes unnecessary with "-autorun". But we are not there yet, so combining them makes sence.

The line "Setting config connection.timeoutInSecs = 120 was null" means that the option was previously not set, but now is set to 120.

Thats an informational line, but the fact is was "null" before is an indication you may have got something wrong.

I think you should be setting "network.connection.timeoutInSecs". We should support the old values (which you have used) but its best to use the new ones.

Why are you getting different outputs when using a different session file?

Becuase theres some subtle but key different maybe?

Diagnosing such problems without the UI is hard.

We have some suggestions here: https://www.zaproxy.org/docs/docker/diagnosing-problems/#investigating-non-trivial-issues

They are in the Docker docs but the principals will work when running ZAP headless from the commandline as well.

Cheers,

Simon

[Illia]

Hi Simon,

Thanks for the replies!

Now it make sense why the command prompt does not stop after the scan is done, instead I have to use "Ctrl+C" to stop it.

However, the -daemon parameter allows me to see the same output as the zap.log file in the commandline.

By using the -cmd option, I can't see the details of the whole ZAP process. 

Is there any way of using the -cmd option but still viewing the whole ZAP process via CMD?

I see. So I should be using -config network.connection.timeoutInSecs=120 then. 

Maybe you can consider updating the ZAP documentation here? 

https://www.zaproxy.org/faq/how-do-you-find-out-what-key-to-use-to-set-a-config-value-on-the-command-line/

Thank you, I'll take a look at the link you send me.

[Simon Bennetts]

You could run ZAP with -cmd as a background process and then 'tail' the zap.log file? :)

Thanks for pointing out the old config value in that FAW - we'll get that fixed!

Cheers,

Simon

[Illia]

That's what I'm doing now actually.

But the goal is to integrate ZAP scan in CICD pipeline, the ZAP tool will be installed in a VM which I won't have access to hence I could not check the zap.log file while the scan is running.

If there is any way I can view the ZAP process via console would be really helpful, or not I guess the only choice to wait for the pipeline to run finish and download the log file.

Sure, no problem!

Thanks.

[Simon Bennetts]

What are you checking the log for - anything specific?

Have you looked at the AF Job Outcome Tests? https://www.zaproxy.org/docs/desktop/addons/automation-framework/tests/

If you use those then you can terminate the job early if they detect problems.

I think this is much more reliable than manually watching the log ;)

Cheers,

Simon

[Illia]

Hey Simon,

Nothing specific, just would like to monitor the whole ZAP progress.

Unfortunately, terminating the job is not an option as I need the ZAP report generated after the active scan is completed.

I guess I will just get the zap.log file after the pipeline has done running then. Thanks for the suggestion.

Another thing that I notice in the zap.log file is that I saw this message

at org.zaproxy.zap.ZAP.main(ZAP.java:94) [zap-2.13.0.jar:2.13.0]
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Do you have any idea on what is causing this issue? 

I am currently using JDK 11 and might be planning to change to JDK 20 later for latest update.