ZAP Automation through Keycloack

[Pepe Kalley]

Hello Everyone,

I'm happy to join this group.

I'm using ZAP for my WEB Applications scanning but I'm going to need your help in this Channel.

I have to automate ZAP using this script:

and this plan file below: ./zap.sh -cmd -autorun config_plan.yaml

Config_plan.yaml:

---
env:
  contexts:
  - name: "some_name Context"
    urls:
    - "URL"
    - "URL"
    includePaths:
    - "URL.*"
    - "URL.*"
    excludePaths: []
    authentication:
      method: "form"
      parameters:
        loginPageUrl: "URL/auth/realms/dome/login-actions/authenticate?session_code=5VqQzm5kBNDun67mQVHHWOqEoyHt8zPiFLWJo-5AxQk&execution=39639581-fedb-482d-b267-6064f18efd33&client_id=some_name&tab_id=kHwnr4vzVuY"
        loginRequestUrl: "URL/auth/realms/dome/login-actions/authenticate?session_code=5VqQzm5kBNDun67mQVHHWOqEoyHt8zPiFLWJo-5AxQk&execution=39639581-fedb-482d-b267-6064f18efd33&client_id=some_name&tab_id=kHwnr4vzVuY"
        loginRequestBody: "username={%username%}&password={%password%}&credentialId="
      verification:
        method: "response"
        pollFrequency: 60
        pollUnits: "requests"
        pollUrl: ""
        pollPostData: ""
    sessionManagement:
      method: "cookie"
      parameters: {}
    technology:
      exclude: []
    users:
    - name: "admin"
      credentials:
        password: "password"
        username: "username"
  parameters:
    failOnError: true
    failOnWarning: false
    progressToStdout: true
  vars: {}
jobs:
- parameters: {}
  policyDefinition:
    rules: []
  name: "activeScan"
  type: "activeScan"

My Applications use Keycloak for authentication and receive a token every time  when they try to authenticate users. My Script can't work well beacause of token's expiration.

Can you help me please with  this issue? More informations if needed.

Thanks in adavance. 

[psiinon]

See https://www.zaproxy.org/docs/authentication/auto-detection/

If that doesnt work then let us know the details.

Auto-detection will make your life much easier, as long as it works!

Cheers,

Simon

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/942f486e-4d53-4bb1-bb30-4d95c52fa773n%40googlegroups.com.

--

OWASP ZAP Project leader

[Pepe Kalley]

Hello Simon,

Thank you for your reply. I'm going to try it and back to you.

Cheers,

[Pepe Kalley]

Hello Simon,

I've tested and I think it'll be difficult to scan with this kind of application. Using Keycloak for authentication, it gives a token each time we want to get to the application. 

Best reagrds,

[psiinon]

The fact that ZAP has identified the session handling is good news.

It failed to identify a verification URL, so you will need to do that yourself.

See https://www.zaproxy.org/docs/authentication/finding-a-verification-url/

Browser based auth will generate a new token every time it is used, so is ideal for these sort of apps.

Cheers,

Simon

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/b77247f9-5d44-4196-bcab-fa23f66d2718n%40googlegroups.com.

[Pepe Kalley]

Thanks for your reply. I'm going to test with these advicies.

[Pepe Kalley]

Hello Simon,

I've tested Browser-Based Authentication but I don't know how to handle the code_challenge with my authentication application in order to keep the same in my plan file.

The scan is working but just for an unauthenticate mode.

I've noticed that even the passive scans like spider in this type of scans are very slow. 

Cheers,

[psiinon]

Have you tried the Authentication Tester Dialog?

https://www.zaproxy.org/blog/2023-05-23-authentication-tester/

How far did it get?

I dont know what the "code_challenge" is, so we'll need much more info to help with this.

If you're trying to perform an authenticated spider but the authentication isnt working then ZAP will keep trying to re-authenticate.

This will make it very slow.

To solve this we need to get authentication working :)

Cheers,

Simon

You received this message because you are subscribed to the Google Groups "ZAP User Group" group.

To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/ed1a0835-e0cc-4d33-9a1c-e0c4ccce7777n%40googlegroups.com.

--

ZAP Project leader

[Pepe Kalley]

Simon,

Thank you for your reply.

Yes, I tried the authentication tester Dialog: https://03365941431588780809.googlegroups.com/attach/5fbea5cc7d5d8/2.PNG?part=0.1&view=1&vt=ANaJVrFNFzeMAWfBzaoXJlayC9VS-52zdtk8NgWR4bHs0G1t4qXfBu7PXDfVxfaJ9RBy9YtTuzcykaiki2h1l04wf0WdsFl6Rtv6VSsKTgpGyCLzzd-M7Ek

Just I can't send more information about our application. My compagny won't allow me to give more than that. Sorry for that!

I think I have to undersdand before the mecanism of authentication with keycloak deeply. 

Thanks for your help.

Best regards,

[psiinon]

OK, that actually looks really good.

Can you identify a suitable "verification URL"?

https://www.zaproxy.org/docs/authentication/finding-a-verification-url/

If you can then you can specify that and use auto detection for everything else.

Cheers,

Simon

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/275b8917-b1e9-47fa-8196-6f3ebbc3ac47n%40googlegroups.com.

--

ZAP Project leader

[Pepe Kalley]

Thanks for your reply again.

You can find some pictures of my authentication flow and see if it's possible to use it with Automated scan (with plan file).

Thanks in advance!

Cheers,

[psiinon]

The flow doesnt make any difference tbh.

The important things are:

• Can ZAP authenticate? (apparently so)
• Can ZAP identify the session handling? (apparently so)
• Can ZAP identify a suitable verification URL?

Can you identify a suitable verification URL?

If so we can explain how you can try all of these things out together.

Cheers,

Simon

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/5cefb1d0-e461-4b21-a6ef-4a44862f761fn%40googlegroups.com.

--

ZAP Project leader

[Pepe Kalley]

Simon,

the problem is I can't yet identify the suitable URL beacause of plan file. Maybe I'm not explaining very well.

If you look at the previous messages, I gave the plan file and each time we launch ZAP, the application ask another code to Keycloak which can be used once. The plan is the static file that's mean automation won't work with this kind of applications which are using another system of authentication and autorization.

If it's possible to look in this way: when we launch ZAP for automated scan, it should be possible to give some parameters in command line I think. Because even we handle URL in this plan file, we cannot use it for another scan in few hours or few days.

./zap.sh -cmd -autorun config_plan.yaml

Cheers,

[psiinon]

Ah.

ZAP authentication relies on us being able to identify a URL we can use for verification.

If we cant identify one, then ZAP wont be able to handle authentication.

Try the Authentication Tester Dialog again.

Then look through all of the requests and responses to see if you can identify one that ZAP should be able to use.

If we can work out how ZAP could identify this URL then we're good. We'll need to make some code changes, but it should all work.

If we cannot identify how ZAP can tell whether its logged in or not then you'll have to handle authentication yourself :/

Cheers,

Simon

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/b0dc8c8a-132a-440e-ae69-a62625d99227n%40googlegroups.com.

--

ZAP Project leader

[Pepe Kalley]

Simon,

Ok cool, I understand . Thank you again.

I'm going to look at authentication tester Dialog again if Needed.

Cheers,

[psiinon]

I wish more SSO providers would make their solutions "testable" :P

https://www.zaproxy.org/docs/authentication/documented-sso-solutions/

I'm happy to talk to anyone from SSO companies to discuss this...

Cheers,

Simon

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/1e195abb-cc4f-4ef1-82a6-dbfc9cb71badn%40googlegroups.com.

--

ZAP Project leader

[Pepe Kalley]

Simon,

I wish too ...

In my compagny, we use open source products such as Keycloak.

I hope we are going to find a good solution.

Best regards,

[psiinon]

I've asked on the #keycloak channel on https://slack.cncf.io/

The responses were:

I think it’s all written in documentation, but the easiest way to obtain token (session) would be using the Direct grant like this:

curl \
  -d "client_id=myclient" \
  -d "client_secret=40cc097b-2a57-4c17-b36a-8fdf3fc2d578" \
  -d "username=user" \
  -d "password=password" \
  -d "grant_type=password" \
  "http://localhost:8080/realms/master/protocol/openid-connect/token"

Keep in mind that the client should be confidential and not public.
More info here https://www.keycloak.org/docs/latest/securing_apps/#_resource_owner_password_credentials_flow

Also you can check if the user has a valid session with the /userinfo endpoint for example:

http://keycloakAddressAndPort/auth/realms/develop/protocol/openid-connect/userinfo

if it doesn’t you would probably receive 401 with something like:

{
    "error": "invalid_request",
    "error_description": "User session not found or doesn't have client attached on it"
}

or 200 with session attached to it.

Does that help at all?

I'll update https://www.zaproxy.org/docs/authentication/documented-sso-solutions/ but also v happy to see if we can get something more out-of-the box working for keycloak (and other SSO providers)...

Cheers,

Simon

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/a23916a5-cf3c-4fa1-a004-668b42f84d35n%40googlegroups.com.

--

ZAP Project leader

[Pepe Kalley]

Hello Simon,

Thank you for this reply. I'm going to test it.

Sincerely,

You received this message because you are subscribed to a topic in the Google Groups "ZAP User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zaproxy-users/kve79tGesQs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zaproxy-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/CAORxfg76zUPhF5hV%2BcnFNb4c_Bx4cevDD4L-FV1F7RWg_jvrfQ%40mail.gmail.com.

[Erwin Rooijakkers]

It's possible to obtain a session token (OAuth access token) using the client credentials grant as described above. But where do we need to add this session token to ZAP? Is that the env var ZAP_AUTH_HEADER_VALUE? And what if the session token is short lived? Can it be refreshed while running?

[Simon Bennetts]

See https://www.zaproxy.org/docs/authentication/

Thats the place to start for any questions related to authentication.

If you have more soecific questions then make sure that you say you have gone through that decision tree and why it didnt work for you.

Cheers,

Simon

[Erwin Rooijakkers]

I tried that approach and get advised to "Action: Run the Authentication Tester" which does not work for me. When I enable a proxy in Firefox I get the error "Could not start a new session". 

Then I gave up since I don't think the approach will work anyway, because Keycloak has unique state parameters (authorization code and PKCE parameters like code_challenge) for every call and requests cannot be replayed with the same query string. This is the same issue described above in this thread and that's why the people in the #keycloak channel advised to use the client credentials direct access grant.

I figured that without the authentication tester and just passing the token obtained using the direct grant to the ZAP_AUTH_HEADER_VALUE as described in https://www.zaproxy.org/docs/desktop/start/features/authentication/ and https://www.zaproxy.org/docs/getting-further/authentication/documented-sso-solutions/ (for Keycloak specifically) would work as well, but if the access token is short lived I don't think there's a way to refresh the token once ZAP is running if it can only be passed using an environment variable, or is there?

[Simon Bennetts]

If you cannot launch browsers from ZAP then you will be very limited.

Have a look at https://www.zaproxy.org/faq/how-can-i-fix-browser-was-not-found/ and let us know how you get on.

Cheers,

Simon

[Erwin Rooijakkers]

It does work when running on the remote machine using zap-cli. Thanks for the link.

--
For commercial support options see https://www.zaproxy.org/support/
ZAP is supported by the Crash Override Open Source Fellowship https://crashoverride.com/open-source?zap=user
---

You received this message because you are subscribed to a topic in the Google Groups "ZAP User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zaproxy-users/kve79tGesQs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zaproxy-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/zaproxy-users/12ccff59-c503-40b7-96d1-93426a5d4dc2n%40googlegroups.com.

[Simon Bennetts]

Thanks for letting us know.

If you are refering to the "zap-cli" tool please be aware that this is no longer supported and we do not recommend using it.

For the recommended ways to automated ZAP see https://www.zaproxy.org/docs/automate/

Cheers,

Simon