Docker with Authentication

[Stéphanie Rochelemagne]

Hello,

I am new to ZAP, and I have to run it against my application. I tried to read as much documentation and see the videos, but I am kind of stuck.

I have set up a context file with the Authentication based on form. I have entered the URLs on login helped by the proxy browser. The parameters are filled and I have selected the email and password field for the authentication. I have created a user.

Then I have exported the context file and tried to run the docker command with the files. When the docker finally connect to the app, he doesn't pass the sessions URL as thr csrf token is invalid.

I tried to remove it from the application, just to see if I have a report. It seems to go a little further but it's just stop at 26% with the error:

ERROR HTTPConnectionPool(host='localhost', port=58706): Max retries exceeded with url: http://zap/JSON/ascan/view/status/?scanId=0 (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer')))

I don't know what to do, even trying to scan in from the desktop application it get stuck at a point. And yes I checked the application is still up so I don't know what I am doing wrong.

The command I run on docker is:
docker run -p 8080:8080 -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-full-scan.py -d -t https://MY-APPPLICATION-URL -c zap-casa-config.conf -x results-full.xml -n DefaultContext.context -U Admin-User

I cannot find either where I can import the zap-casa-config file in the UI as I am fine running it through the desktop app as the csrf token seems to be better handled there.

Thanks for any help

Stephanie

[Simon Bennetts]

Hi Stepanie,

Authentication is hard, but It sounds like you've already made a lot of progress.

Can you confirm that ZAP can authenticate to your app when you run tools like the spider and active scanner using the context you have created?

When are the CSRF tokens created?

Its always best to test the configs in the ZAP desktop as its much easier to debug problem there.

Cheers,

Simon

[Stéphanie Rochelemagne]

Hi Simon,

Thank you very much for taking the time to answer.

Actually no the authentication doesn't work even on the desktop application.

I think I have followed the doc and tutoriels:

- open OWASP  ZAP app

- open the browser from the application

- navigate to my application

- login to the application

- go back to the ZAP app

- add the login and sessions URLs in the context

- edit the context, in the Authentication, enter the POST url and the GET url from the URLs browsed

- it is filling up automatically the parameters (POST data)

- then I try to select the username and pwd field which are tricky as I have the authenticity_token in the parameters and the application always want to select this param as username

- go to the Users and add the user which I logged with by entering the email and password

- then I right click on the POST URL and select Active scan

=> it does thing but none are passing the login as the CSRF token cannot be verified.

Any "idea" or fixed will be welcome at this point.

Thanks again

Stephanie

[Simon Bennetts]

Hi Stephanie,

OK, so the first thing to do is to get authentication working in the ZAP desktop.

Do you know _exactly_ how authentication (and session handling) works in your app?

If not we'll need to work that out :)

The first thing you need to do is to find a "verification URL" - https://www.zaproxy.org/docs/authentication/finding-a-verification-url/

Next, authenticate to your app via the browser and open the verification URL in a new tab.

Check the request and response in ZAP - it should indicate that you are authenticated.

If it doesnt then maybe its not a good verification URL?

Once you've got a verification URL that works in the browser, then replay the request in ZAP.

It should show that you are authenticated.

If it doesnt then let us know.

If it does then look at the request - there will be one or more things in there which contain the key session information.

Typically the data will be in one or more headers (esp "Authorization" and "Cookie") but sometimes there will also be key info in the URL as well.

Remove the headers (and possibly URL elements) one at a time, make the request again and see if the response shows you are still authenticated.

If it doesnt then those elements are needed for authentication.

Try all of that and let us know how you get on.

When you get to this stage then we can walk you through the next steps.

Cheers,

Simon

[Stéphanie Rochelemagne]

Hi Simon,

Thank you very much for your detailed response.

Yes I know how the authentication works, the user do a GET /login which displays the login form with the CSRF token from the server. 

Once the user enters his credentials and submit them it submits a POST /sessions which verifies the credentials of the user of course but also the csrf token, if all is good the user is redirected to the account otherwise the login page is redisplayed.

So I followed your steps:

- find the verification_url => done

- open the browser app from ZAP

- go to the /login page

- enter the credentials and logged my user => POST /sessions

- opened a new tab and use the verification url => all good, user connected

- opened this verification url in the Manual Request Editor and run it => all good the user is connected

- from this same editor removed the session cookie, and re-run the command and the user is not connected anymore :-) I am redirected to the login page

Let me know what are the next steps now and thanks again for your help!

Sincerely,

Stephanie

[Simon Bennetts]

Great!

OK, the next step is to set up ZAP authentication, which to be fair you've probably already done.

But its worth sanity checking.

You should have configured the following Context tabs:

• Authentication
• Users
• Forced User
• Session Management

If you look at the main ZAP toolbar then you should see that the "Forced User Mode" button is _not_ disabled.

If it is disabled then you have not configured all of the above tabs correctly.

If you've got that far then enable the Forced User Mode via the toolbar button.

Open the verification request in the Manual Request Editor, and make sure the session cookie is _not_ present.

Now send the request.

Doest it indicate that the user is connected?

Can you see an authentication request in the History tab?

Cheers,

Simon

[Stéphanie Rochelemagne]

Hi Simon,

Just for the record because I filled up my context from different posts I just resume what I did.

Open the Default Context:

- click on Authentication from the left menu:

  - select Form-based Authentication

  - in the Login Form Target URL, I select the POST /sessions URL

  - in the URL to GET Login Page, I select the GET /login URL

  - in the Login Request POST Data, it is already filled up with the sessions URL

  - in the Username Parameter et Password parameter, I just selected the parameters that correspond to the email and password

  - in the Verification Strategy, I select Poll the Specified URL

  - in the URL to Poll for Verification, I select my verification URL

- click on Users from the left menu

  - create a user with the credentials

- click on Forced User, the user created is already selected

- click on Session Management, and make sure the option selected is Cookie-based

Then I clicked on OK

In the Toolbar the Forced User Mode was not enabled, so I enabled it

I came back to the Manual Request Editor with my verification URL, I removed session cookie from the request and send it.

The request went through and my user was connected. When I came back to the request tab, the session cookie was present.

In the History tab, I can see the verification URL request with several tags but I don't see any authentication requests with the login or sessions if that supposed to happen.

For me it's like it re-uses the session that was initially created. It is maybe what it should do but I am not sure it should happen this way...

Sincerely,

Stephanie

[Simon Bennetts]

OK, try this:

• Turn off Forced User Mode
  
• Access your app via a browser
• If you are logged in, then log out
• Open the verification URL in your browser - this should show you are logged out
• Turn on Forced User Mode
• Open the verification URL in your browser - this should show you are logged in, and there should be an authentication request in the History
  

Let us know what happens :)

Cheers,

Simon

[Stéphanie Rochelemagne]

Hi Simon,

Ok I did the steps and I couldn't see the authentication processed kicked :-(

So I remember of one video or a post where they "include" a url in the context file.

I right click on my verification url and click on "Include Site in Context" and it added the base URL of the site in the Include in Context of the context file. Sorry it might have been very obvious for you :-(

After doing that, I enabled the Force User mode, re-open the Manual Request Editor on my verification URL, removed the cookie and click on Send. This time I could see on the History tab the Authentication processed and then I was logged in :-)

Now, my context file with the authentication should be good, right?

My next questions would be:

- with a full scan, would the application will discover "all" the URLs of the application or should I add them "manually" in the Include in Context?

- if I export the context file and I run the docker command with the full scan script, will the authentication will work?

Thanks again for all your help so far!

Stephanie

[Simon Bennetts]

Hi Stephanie,

No, I didnt think about the verification URL needing to be in the context - we should add that to our docs.

But v pleased to hear you have got authentication working now :)

Exploring apps is another can of worms :/

ZAP supports a range of options:

• The standard spider (good for traditional apps)
• The ajax spider (good for modern apps)
• Importing API definitions:
  • OpenAPI
  • SOAP
  • GraphQL
• Proxying integration tests through ZAP

Which of these options is best for you will depend on your app.

In all cases you will want to make sure that your context definition includes all of the relevant URLs, but you shouldnt need to add them all to the context.

ZAP contexts are defined by regex expressions.

An include regex like:

• http://www.example.com.*

Will cover all URLs that start with "http://www.example.com" e.g. "http://www.example.com/", "http://www.example.com/page1", "http://www.example.com/dir/page2?aaa=bbb" etc

Just define the include regexes to cover all of the URLs that make up your app.

What we do recommend is that you check the ZAP auth stats to make sure that authentication keeps working.

How are you automating ZAP?

Via the docker packaged full scan or another way?

Cheers,

Simon

[Stéphanie Rochelemagne]

Hi Simon,

ok the goal of all of that is for a security audit that we have to do and they were explicit with 4 steps:

1. Download the zap-casa config file

2. Generate the context file by filling up the Authentication tab

3. run the docker image with full scan

4. Send us the report generated

So that was the theory where 4 little steps will be enough :-)

Now in my context file I added https://exmpale.com/* like you mentionned.

The thing is with the docker command, it is kind of a black box. I don't know what they are doing, I can only check the log in my application to make sure it is still working.

Now to have a little more log I added the -d option to have more debug and after running the command, it tries to connect to the local host but without really understanding what exactly it is waiting for, sometimes it work after 1 min and sometimes never. I have to stop the docker instance and try it again.

By trying again the docker command I got this error now:

81258 [ZAP-NewsFetcher] ERROR org.zaproxy.zap.ZAP.UncaughtExceptionLogger - Exception in thread "ZAP-NewsFetcher"
java.lang.NoClassDefFoundError: org/zaproxy/zap/extension/quickstart/NewsItem
    at org.zaproxy.zap.extension.quickstart.ExtensionQuickStart$1.run(ExtensionQuickStart.java:172) ~[?:?]
Caused by: java.lang.ClassNotFoundException
    at org.zaproxy.zap.control.AddOnClassLoader.findClass(AddOnClassLoader.java:330) ~[zap-2.12.0.jar:2.12.0]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]

I don't know what does it mean and how I can fix it.

Do you recommend to do the full scan on the desktop app and have the same kind of result as the docker command?

Thanks in advance for your help.

Sincerely,

Stephanie

[Stéphanie Rochelemagne]

Hi Simon,

One more question about the Authentication, so I made it work when sending a manual request, I got the Auth process triggered.

But now when I retry, it doesn't trigger all the time the Auth process, in order to have it triggered I need to update the URLs in the context file in the section Include in Context.

So I am not sure how the active scan or any san is working if to trigger the authentication I have to update the URLs.

Also from the context file, I tried to run an "Active Scan" and even if I have the root of the website in the Include Context, it doesn't discover the URLs of my application so I have to actually add them either manually or by clicking from the browser and then add them. Not sure to understand how it works.

Thanks again for your help

Stephanie