Authenticated Scan with Nondeterministic Login URLs

[Charles Williams]

Hello,

I am testing OWASP ZAP locally to determine if it is suitable for one of my tasks, and I'm running into a bit of an issue regarding authenticating a user in an active scan. The web app is running in Docker and I am running ZAP using the Desktop UI. The web app follows these redirection steps when a user attempts to login:

1. GET http://localhost:8000 (302)
   1. The initial access link the user starts from
2. GET http://localhost:8000/<further authentication link> (302)
   1. A redirection that leads to authentication
3. GET http://host.docker.internal:8080/<link with non-deterministic key/value pairs> (200)
   1. This is the landing zone for the login page within Docker
4. POST http://host.docker.internal:8080/<link now including a unique session code and id> (302)
   1. This is what is sent when the user logs in
5. GET http://localhost:8000/<authentication with unique state key/value pairs> (302)
   1. One last redirect where the app confirms a valid user ID
6. GET http://localhost:8000/<home page> (200)
   1. Finally landing at the home page of the app, and ready to be scanned

I believe my primary issue has come from the redirect links being non-deterministic due to the generated IDs on each login. I can't do form-based authentication since the target URL expires after each use, so I can't create a user and then send a POST request to the same location every time. I've also tried working with Zest scripts that follow redirects, but I haven't been having much success there either. I recorded the login process and set that as the authentication method, but since Zest appears to just send whatever requests it recorded, I run into similar issues where the link is expired.

How can I run an authenticated scan on this kind of unique login environment? I'm very new to ZAP so I'm sure there is some obvious solution that I'm missing somewhere, or I made some mistake in one of the approaches I'd tried. Once I get past the login section, the app links are all deterministic and have been spidered successfully, so the scan should proceed as intended from there. Let me know if I can provide any further information that would help, and I'll see what I can do.

Thank you!

[kingthorin+owaspzap]

You should be able to do a scripted login, you may have to create it manually following the redirects and submitting the correct details in the correct places. But it's doable, that's kinda why scripted login is an option.

You can try to do it with a recorded Zest script, but I'm not sure how well it'll handle non-deterministic URLs.

You can probably get some ideas from: https://github.com/zaproxy/community-scripts/tree/main/authentication

[Charles Williams]

Thanks for the link! I'm working on a script heavily inspired by the "multiple redirects from GET then POST" and "2FA" community scripts - essentially I want the script to start from the initial link, follow redirects to the login page, retrieve the link to POST to from the response body, then follow redirects from that POST to the final home page (where the scan should begin). I thought I had something that worked, but I'm having trouble debugging my script to see if it works or not.

I added my script into ZAP as an authentication script of type ECMAScript : Oracle Nashorn. I've loaded it into my context and configured the fields and users. However, part of my script has a debug log where it prints information so I can track if it is working. However, I'm not able to see any output when active scans run (and the 302 responses I get there are telling me that the authentication step isn't working). I'm also not able to press the "Run" button on my script from the script console. I've attached a safe-to-share version of my script - code design aside, my main concern is being able to see my logging messages when I run my scan so I can see if the authentication works or not (I'll be able to debug from there once I'm seeing these messages). How can I get these types of messages to be displayed when I run my scan? And if this method does not work, what can I do to see the logs from my script when running the authenticated scan?

Thank you!

[ricekot]

Did you create a context and configure script based authentication for it? See https://www.zaproxy.org/docs/desktop/ui/dialogs/session/context-auth/#script-based-authentication.

You must configure a context and set the scope, authentication method and verification strategy, etc. before starting an active scan against it.

The "Run" button is available only for Standalone scripts. All other script types are invoked by ZAP as appropriate (e.g. authentication scripts are invoked when ZAP determines that the user is logged out of the application based on your configured auth verification strategy). 

See https://www.zaproxy.org/docs/desktop/start/features/scripts/#script-types and https://www.zaproxy.org/docs/desktop/start/features/authstrategies/ for more details.

Best regards.

[Charles Williams]

Yes, I've created a context for my app, set include and exclude scopes, loaded a script based authentication with my required fields, and set the verification strategy to check every response for the logged out regex. I'll do some more digging in my requests to make sure I've configured that all correctly.

In theory, if this was all set up correctly, where would the debug messages in my script be printed to (so I could see that the script is at least running)? And would there be any further requirements beyond correctly configuring the context and script based authentication?

Thank you!

[thc...@gmail.com]

They are printed under the Script Console tab.

Yes, worth checking:
https://www.zaproxy.org/faq/how-can-zap-automatically-authenticate-via-forms/#diagnosing-problems

Best regards.

>>>>> 1. *GET http://localhost:8000 <http://localhost:8000> (302)*
>>>>> 1. The initial access link the user starts from
>>>>> 2. *GET http://localhost:8000/ <http://localhost:8000/><further
>>>>> authentication link> (302)*
>>>>> 1. A redirection that leads to authentication
>>>>> 3. *GET http://host.docker.internal:8080/

>>>>> <http://host.docker.internal:8080/><link with non-deterministic key/value

>>>>> pairs> (200)*
>>>>> 1. This is the landing zone for the login page within Docker
>>>>> 4. *POST http://host.docker.internal:8080/

>>>>> <http://host.docker.internal:8080/><link now including a unique session

>>>>> code and id> (302)*
>>>>> 1. This is what is sent when the user logs in
>>>>> 5. *GET http://localhost:8000/

>>>>> <http://localhost:8000/><authentication with unique state key/value pairs>

>>>>> (302)*
>>>>> 1. One last redirect where the app confirms a valid user ID
>>>>> 6. *GET http://localhost:8000/ <http://localhost:8000/><home page>
>>>>> (200)*
>>>>> 1. Finally landing at the home page of the app, and ready to be

[Charles Williams]

Hello,

I think I figured out my issue, but I have one last question regarding login. So I've created my test user with username Testusercw and password Password1!, the POST message body looks like this on a successful login (note the blank credentialId is intentional):

username=Testusercw&password=Password1%21&credentialId=

However, my debug messages are showing me that the following body is being used to login:

username=Testusercw&password=Password1!&credentialId=

It looks like the HTML URL encoding is not being applied, so I tried changing the test user's password to Password1%21, but got this in my debug log instead:

username=Testusercw&password=Password1%2521&credentialId=

So it appears then the that URL encoding is working for some characters but not others - how can I resolve this to get the proper message body?

Thank you!

[thc...@gmail.com]

Use `URLEncoder#encode` instead of `encodeURIComponent`, e.g.:

var URLEncoder = Java.type('java.net.URLEncoder');

print(URLEncoder.encode("Password1!", "UTF8"))


Best regards.

On 02/03/2022 13:46, Charles Williams wrote:
> Hello,
>
> I think I figured out my issue, but I have one last question regarding
> login. So I've created my test user with username Testusercw and password
> Password1!, the POST message body looks like this on a successful login
> (note the blank credentialId is intentional):
>

> *username=Testusercw&password=Password1%21&credentialId=*

>
> However, my debug messages are showing me that the following body is being
> used to login:
>

> *username=Testusercw&password=Password1!&credentialId=*

>
> It looks like the HTML URL encoding is not being applied, so I tried
> changing the test user's password to Password1%21, but got this in my debug
> log instead:
>

> *username=Testusercw&password=Password1%2521&credentialId=*