Ajax Spider for authentication

[chi]

Hi everyone! 

I am very new to ZAP and I have a question about authentication when trying to crawl a login page.

I assume I can crawl a given URL of my web app by using the UI. If I manually  hit the Login button on the web app, the login request node is found under Sites tab. So I can see the Login POST data information and I can flag it as context for Login request. However when I tried to use spider instead of manually hitting the Login button on the same page without logged in, the spider couldn't find the login request node. I used spider first and then I used the AJAX spider for all other possible nodes, but both of them couldn't find the login request node. Therefore I can't make it login to the page automatically. 

I also read this post https://groups.google.com/forum/#!searchin/zaproxy-develop/authentication/zaproxy-develop/CJcZ3IWsFdU/iWEwNgg_DNcJ , but I am not sure what else do I need to setup via the UI.

I use AngularJS for my app, I was wondering does the Ajax Spider in ZAP support AngularJS framework? Or is there anything I missed or misunderstood? Thanks!

 

[Cosmin Stefan-Dobrin]

Hi,

The post you linked to is describing a new feature of ZAP that allows users to customize and automate authentication. However, that feature has not yet been made available in a major release of ZAP and is planned to be available in the next release (which should be relatively soon). Right now, one way to use it is if you try one of the weekly releases of ZAP [1].

There is also a way/hack to make your scenario work using one of the full releases:

1. authenticate using the web browser, while proxying through ZAP
2. go to the Http Sessions tab and identify the session on which you have just authenticated (normally it should be just one so far if you haven't done anything else)
3. mark the session as Active (using the right-click menu)
4. make sure you exclude the logout URL from the Spider
5. spider normally using any of the spiders. The requests it sends should be all modified to match the authenticated session.

Please let us know how it works and if you need more details regarding any of the methods.

[1] - http://code.google.com/p/zaproxy/downloads/list

Cheers,

Cosmin

--
You received this message because you are subscribed to the Google Groups "OWASP ZAP User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zaproxy-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[chi]

Hi Cosmin, Thank you for the help! I tried what you mentioned about using the authenticated Http session which works great, so I can use spider to explore my web application. However I tried to use the weekly release version to setup those authentication settings, the spider still couldn't login to the application. I setup the url to be included in Context, authentication section, logout url to exclude from spider, and I added user information in the session properties. 

I tried to use the Login POST data I got from the authenticated session, and it still couldn't sign in to the application.  

I also noticed that if I Flag as Context for one page and I also fill out the Login POST data section for it, then it won't show the authentication icon (the little green arrow and open door picture). If I just Flag as Context without setting the Login POST data, it will show the icon. I am not sure about the difference. 

So my question will be does the authentication feature support AngularJS framework currently? Or is there any other settings I missed? Thanks!

[Cosmin Stefan-Dobrin]

Hi,

Whenever you set a POST data for a page in the Authentication tab, ZAP assumes that the request needed to be sent is an HTTP POST one. If no POST data is set, then ZAP assumes the login request needs to be sent as a HTTP GET. Probably, what is happening is that you are flagging a GET request as the login one and then, when you are adding the POST data ZAP considers them different so the icon is not displayed. However, whether the icon is displayed or not should not affect whether the authentication is done correctly, if what is filled in the Session Properties is correct. How is the authentication done in your tested app? Users fill in a form and fill in their authentication credentials (username/password)? Is the authentication request being sent as an HTTP POST or as an HTTP GET?

The easiest way to set up authentication is as follows:

1. make sure your browser proxies everything through ZAP and log into your application using the browser
2. go to ZAP and identify the request that was done for the login (most usually it's a HTTP POST request containing the username and the password and possibly other elements).
3. right click on the request and Flag as Context... Form-based Auth Login Request
4. a window will be opened already containing the request URL and the parameters (if any). Use the dropdown options to select which of the parameters correspond to the username and to the password
5. then you need to tell ZAP how to identify whether an authentication succeeded or not. You can do this by setting logged in or logged indicators. These are regex patterns which, if found in a response, tell ZAP whether it's authenticated or not (e.g. presence of a http://example.com/logout link or the presence of a 'Welcome, User X'). Only one of them is necessary. To set one of them, either type the regex directly in the Session Properties -> Authentication -> Logged In Indicator, either find an authenticated message in the Sites Tree, select it, open the Response View and select the text you wish to define as the indicator using the mouse and select the Flag as Context... Logged in indicator right-click menu option.
   
6. Define as many users as you need in the Session Properties -> Users section.
7. After this step, various actions are available in ZAP. For example, you now have a new right click option: Attack -> Spider Context As User. Or, using the Forced User Mode, you can force all the interactions that go through ZAP for a given Context to be from the perspective of a User. The User Forced Mode is enabled via the previous-to-last button in the toolbar (the one with the user and the lock) and is configured via Session Properties -> Forced User Mode.

You could also find more details in the ZAP help (of the weekly release):

• In Getting Started -> Features -> Authentication
  
• In User Interface -> The Dialogs -> Session -> Session Contexts
  

In your situation, you could use either of the options presented at point 7. However, the simple Spider, even though it does a great job at identifying links in normal websites, it does not process Javascript so it is not working with Ajax-based apps. For this, you would need to use the Ajax Spider (that you have already mentioned). The downside is that is does not yet have a 'Ajax Spider Context as User X' functionality. So the only way to use it would be with the Forced User mode.

Regarding the Angular JS framework, I personally have no experience with testing apps which work with it. But the Ajax Spider should not have any problems with it as it uses a browser for the JS processing and they do a great job. So, you could try the steps 1-7 from above and use the Forced User Mode and, although I have to admit I have not tested it, it should work.

Please, let us know if there's anything we can help you with or you need more details about anything. Also, it would be great if you could let us know if everything worked ok.

Cheers,

Cosmin

--

[chi]

Hi Cosmin,

I noticed that last time I already tried everything from 1 - 7 you mentioned.
However I always logout the app when I start AJAX spider since I thought it will login to the app automatically by using the authentication information I set up. Then it didn't work.

So I found out actually my question is ...
Is it possible to use Spider for a New session without logging to the app, but just flag the necessary context and fill in the data needed for authentication? (Which I tried and it didn't work.)

(It is form-base authentication and it is sending as an HTTP POST)

Other than that, it works great!!! Also thank you for the explanation.

[Cosmin Stefan-Dobrin]

Hi,

ZAP should be able to log you in automatically, if you have set up a everything that is needed in the Session Properties, even if it's a completely new browser session or you haven't logged in already or have never proxied a Login Request through ZAP (it's possible to just fill in the fields in the Sesion Properties -> Authentication Panel). A common scenario could be to do the ZAP authentication configuration once, save the ZAP session and anytime later load it and enable Force User Mode or Spider again as a User and ZAP should automatically log in.

Regarding it not working, can you give us a few more details? Was it when you used it with the Ajax Spider? Or using the normal Spider? Did you use an action such as Spider URL as User X or the Forced User Mode? What are the actions that you did after setting up?

Thanks,

Cosmin

[chi]

Hi,

It is the situation when I restart ZAP, and I haven't proxied a Login Request through ZAP. I typed all the information needed, and then I tried all options I can use for spider (Spider URL, spider site, spider in all scope, and spider as user) each time. I tried both spiders, but I think only AJAX spider can get the link to login. I also tried to use Forced User with different spider option.

I can see the popup window after I use AJAX spider, but it is just trying to type random string in the username text input area.

Thank you!

[Cosmin Stefan-Dobrin]

Hmm... Well probably it's an issue related to the Ajax nature of the site. Since you mentioned something about a popup for login and since, as far as I'm aware, the Ajax Spider still does not directly support the new authentication features, probably it's not going to work until some changes are made. I've raised an issue [1] for this, so feel free to add details and flag it so you can keep a track of it's implementation progress.

Thank you for reporting this and feel free to ask us any other questions you might have.

[1] - code.google.com/p/zaproxy/issues/detail?id=906

Thank you,

Cosmin

--

[chi]

Hi Cosmin,

I think is it possible this is due to that the POST login request url is not the same as the login page url?

Also the popup windows I meant the windows for running  AJAX spider, so I can see the spider is trying to type random strings in the input area, but it couldn't get to the POST login request url.

Thanks for helping!

[chi]

Also it seems like it is not hitting the Login button. (The Login button would be enabled once there are something in both username & password input areas. I always see the spider is typing in just the username section) So is it possible the spider is not considering it's a login page? Since the setup POST request url is different from the login page. I stepped through this and I saw the POST url appears after hitting the Login button on Login page.

[Cosmin Stefan-Dobrin]

Hi,

As far as I know, authentication is not integrated into the Spider so it's normal to not fill in the fields with the proper credentials. I've created the issue above and we'll try to take a look at it and implement it sometimes in the near future.

Also, just a small clarification for the Authentication configuration (in case you want to use it with the normal spider, for example), the login URL that needs to be filled in the Authentication panel should the final POST request URL (the form's target) and not the URL of the login page. Actually, I've just realized that the name might be a bit misleading.

Cosmin

On Mon, Nov 18, 2013 at 9:39 PM, chi <chima...@gmail.com> wrote:

Also it seems like it is not hitting the Login button. (The Login button would be enabled once there are something in both username & password input areas. I always see the spider is typing in just the username section) So is it possible the spider is not considering it's a login page? Since the setup POST request url is different from the login page. I stepped through this and I saw the POST url appears after hitting the Login button on Login page.

[chi]

Hi Cosmin,

Yes I actually filled in that section with POST request url. Still Thank you for the help! I will try to add any detail I know to the issue.

[thc202]

Hi.

As far as I know, authentication is not integrated into the Spider so it's normal to not fill in the fields with the proper credentials.

It's not integrated that way (fill the forms) and it doesn't/shouldn't need to since the authentication is already handled by ZAP.
The Ajax spider is already using the authentication functionalities currently available in ZAP (except HTTP authentication for current release [1]), that is, ZAP authenticates when Ajax requests are sent through ZAP. The problem (IMHO) is that the login requests are not handled correctly by the authentication extension as it should return the response of the valid authentication request (done with the credentials set in ZAP) instead of sending, after successfully authenticate, the (proxied) authentication request containing the invalid credentials.
The same thing happens with the normal spider when it submits a login form.


[1] https://code.google.com/p/zaproxy/issues/detail?id=584

Best regards.