Possible to scan IIS website with Windows Authentication enabled?

[Daniel O'Brien]

I have searched the posts on this group as well as issues on GitHub, and from what I can tell it is not possible to use ZAP to scan a website hosted in IIS that uses Windows Authentication, despite the fact that the documentation suggests that NTLM authentication is supported.  All the posts / issues on this topic seem to be left hanging without any definitive answers.  Can anyone confirm - is this not possible? 

If it is possible I would greatly appreciate if someone would post the necessary steps. Manual authentication would be fine for my purposes.  I just want to some way to scan the localhost site I am working on.

Thanks, Dan

[kingthorin+owaspzap]

You can setup ZAP to do NTLM authentication.
https://github.com/zaproxy/zap-core-help/wiki/HelpStartConceptsAuthentication#httpntlm-authentication

There have been multiple threads and issue raised related to NTLM authentication, sadly these usually endup dropped by the original user and us either assuming their issue is resolved or that they gave up. Unfortunately in both those scenarios no improvement occurs (if necessary).

[Daniel O'Brien]

I read that article before posting to the group, but it really doesn't offer much information about NTLM authentication, and none specifically integrated Windows Authentication in IIS.  The only example provided is for forms authentication / cookie based session management - would be great if there was an example for each of the supported use cases.  

I would be happy to use manual authentication in ZAP, but it is not working - the scan returns 401 errors on all pages.  I read somewhere that after authenticating in a browser you need to open the HTTP Session tab in ZAP and set the session as active, but when I open this tab there are no sessions listed for the site.

If you want to reproduce:

1) Create a site in IIS 8.5 with a single default.htm page and enable Windows Authentication on the site.  

2) After starting up ZAP / configuring the proxy, navigate to the default.htm in a browser to ensure that you have an active authenticated session.

3) In ZAP, set the site as the Default Context.

3) Open the HTTP Sessions tab - there will be no sessions in the list for the site.

4) If you try to run a scan, all responses will return a 401.  

If there is a way to scan an IIS website that has Windows Authentication enabled, I would greatly appreciate if someone would post the steps.  

Thanks, Dan

[kingthorin+owaspzap]

You probably need to enable Forced User mode https://github.com/zaproxy/zap-core-help/wiki/HelpUiTltoolbar#--force-user-mode-on--off

This unresolved issue might give some points that help move things along for you: https://github.com/zaproxy/zaproxy/issues/1602

Wish I could provide more but I don't have a place to test an ntlm example.

[Daniel O'Brien]

That is one of the threads I was referring to in my initial post.  I have tried all those steps, and got the same results.  Note that the post was not abandoned by the user, but by the responders, which is why I have concluded ZAP simply does not support scanning a site using Windows Authentication.

If you don't have access to a Windows machine, you can download a Windows 10 VM (for evaluation) here, and see for yourself:

https://developer.microsoft.com/en-us/windows/downloads/virtual-machines

-Dan

[kingthorin+owaspzap]

Hey Dan, the ticket is still open. Unfortunately it stalled out with the ZAP Team asking for info the reporter was unable to provide. If you can provide diffs of failing vs successful authentications that'd really help.

Thanks for pointing out the VMs, unfortunately I don't need client VMs I'd need an IIS and AD server VM to recreate the server scenario and test ZAP against it.

[Daniel O'Brien]

You don't need a server version of Windows in order to run IIS and host a localhost site, which is all I am doing.  I am simply want to pentest a local development site that requires Windows Authentication.

-Dan

[kingthorin+owaspzap]

K, will see what I can get setup. Though I didn't think ms still bundled iis with any client versions of windows.

[thc...@gmail.com]

Hi.

Are you able to provide the (working) NTLM messages? You can use fake
credentials/domain, just want to do a comparison between the messages
sent by ZAP and other application (that works).

Otherwise, could you provide the steps to set up IIS/Windows so I can
reproduce that locally?

Thanks!
Best regards.

[Stephen Heeps]

Hi there.

I am also having the same issue and since this is the most up-to-date thread, I thought I'd respond to this.  If this is not the correct procedure, please let me know and I can create a new topic.

We are having issues using ZAP against a .NET webapp configured with Integrated Windows Authentication.  We are using IIS 8.5 on a WS2012R2 server and using our AD creds to log into said webapp.  We configure ZAP to use the HTTP/NTLM auth. schema and we configure the host, realm, Logged In indicator, AD user (domain\username), HTTP session management, and forced user mode is enabled.  When attempting to run a scan/spider on the site, we keep getting 401s with the following:

2017-04-20 18:36:01,540 [ZAP-ActiveScanner-0] INFO  User - Authenticating user: domain\username

2017-04-20 18:36:01,540 [ZAP-ActiveScanner-0] DEBUG DefaultHttpParams - Set parameter http.protocol.version = HTTP/1.0

2017-04-20 18:36:01,540 [ZAP-ActiveScanner-0] DEBUG DefaultHttpParams - Set parameter http.protocol.version = HTTP/1.1

2017-04-20 18:36:01,540 [ZAP-ActiveScanner-0] DEBUG DefaultHttpParams - Set parameter http.protocol.cookie-policy = compatibility

2017-04-20 18:36:01,540 [ZAP-ActiveScanner-0] DEBUG MultiThreadedHttpConnectionManager - HttpConnectionManager.getConnection:  config = HostConfiguration[host=http://webapp], timeout = 0

2017-04-20 18:36:01,540 [ZAP-ActiveScanner-0] DEBUG MultiThreadedHttpConnectionManager - Getting free connection, hostConfig=HostConfiguration[host=http://webapp]

2017-04-20 18:36:01,541 [ZAP-ActiveScanner-0] DEBUG HttpMethodDirector - User defined 'Authorization' headers present in the request.

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "GET /About HTTP/1.1[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG HttpMethodBase - Adding Host request header

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "Accept: application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "Referer: http://webapp/[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "Accept-Language: en-US[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "Cookie: __AntiXsrfToken=cc5491bd15834e00838fe8f5d7823edc[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "Proxy-Connection: Keep-Alive[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAHgAAABWAVYBkAAAAAgACABYAAAADAAMAGAAAAAMAAwAbAAAABAAEADmAQAAFYKI4gYBsR0AAAAPJDZ9BhEkrSdPMdaP+L06/0UAVgBOAFQAUwBIAGUAZQBwAHMAUwBIAEUARQBQAFMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAuZDSVsRSnHd7dMD/KlSP0AEBAAAAAAAATnHp4SK60gGxjp1dx+ltxgAAAAACAAgARQBWAE4AVAABABQAQgBFAEQALQA5ADkANQAtADIAMAAEABwAZQBhAHQAbwBuAHYAYQBuAGMAZQAuAGMAbwBtAAMAMgBCAEUARAAtADkAOQA1AC0AMgAwAC4AZQBhAHQAbwBuAHYAYQBuAGMAZQAuAGMAbwBtAAUAGgB3AGkAbgByAG8AbwB0AC4AbABvAGMAYQBsAAcACABOcenhIrrSAQYABAACAAAACAAwADAAAAAAAAAAAAAAAAAgAACv66rciabIkmMUgUXV/VU1TYq681UkJsPW0D6V+58qUgoAEAAAAAAAAAAAAAAAAAAAAAAACQAmAEgAVABUAFAALwBkAGUAdgAtAHAAbwBjAC0AdwBlAGIAYQBwAHAAAAAAAAAAAAAAAAAATReUsf+B/yWYX6CaCZ/CLQ==[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "Content-Length: 0[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "Host: webapp[\r][\n]"

2017-04-20 18:36:01,542 [ZAP-ActiveScanner-0] DEBUG header - >> "[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "HTTP/1.1 401 Unauthorized[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "Content-Type: text/html[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "Server: Microsoft-IIS/8.5[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "WWW-Authenticate: Negotiate[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "WWW-Authenticate: NTLM[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "X-Powered-By: ASP.NET[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "Date: Thu, 20 Apr 2017 22:36:01 GMT[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "Content-Length: 1293[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG header - << "[\r][\n]"

2017-04-20 18:36:01,544 [ZAP-ActiveScanner-0] DEBUG HttpMethodDirector - Authorization required

2017-04-20 18:36:01,545 [ZAP-ActiveScanner-0] DEBUG AuthChallengeProcessor - Supported authentication schemes in the order of preference: [ntlm, digest, basic, ntlm]

2017-04-20 18:36:01,545 [ZAP-ActiveScanner-0] INFO  AuthChallengeProcessor - ntlm authentication scheme selected

2017-04-20 18:36:01,545 [ZAP-ActiveScanner-0] DEBUG AuthChallengeProcessor - Using authentication scheme: ntlm

2017-04-20 18:36:01,545 [ZAP-ActiveScanner-0] DEBUG AuthChallengeProcessor - Authorization challenge processed

2017-04-20 18:36:01,545 [ZAP-ActiveScanner-0] DEBUG HttpMethodDirector - Authentication scope: NTLM <any realm>@webapp:80

2017-04-20 18:36:01,545 [ZAP-ActiveScanner-0] DEBUG HttpMethodDirector - Credentials required

2017-04-20 18:36:01,545 [ZAP-ActiveScanner-0] DEBUG HttpMethodDirector - Credentials provider not available

2017-04-20 18:36:01,545 [ZAP-ActiveScanner-0] INFO  HttpMethodDirector - No credentials available for NTLM <any realm>@webapp:80

The output file spits out this line: 

No indicators have been set for identifying authentication. Assuming response is authenticated for http://webapp.

The above line dictates (from research) that we do not have proper Logged In/Logged Out indicators. I've tried regex samples from various posts and keep getting this output.  The log tells me otherwise basically saying it cannot send the credentials.  However, I can provide you any steps/procedures that you need in able to reproduce this issue.  This issue has been present and numerous posts have been created with no avail (as mentioned above).  I think it's time we put this to bed, it will help all of us that have to do scans on webapps using Integrated Windows Auth.

1. Get a WS2012R2 VM
1. https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2012-r2 - if you need an evaluation license for the machine
   
2. Install IIS 8.5
1. Navigate to server manager
2. Click the Manage button in the top-right corner
3. Click Add Roles and Features from the context menu
4. On the "Before you Begin section", click Next
5. On the Installation Type, click Next
6. On Server Selection, click Next
7. On Server Roles, click Web Server (IIS) and click Next
8. Continue with all of the defaults until the installation begins
3. Create/use a webapp with Integrated Windows Auth
1. Verify AD creds work fine within the app manually
4. Open ZAP and configure browser to proxy through ZAP
5. Configure context to use HTTP/NTLM auth
1. Enter all relevant information regarding host, realm, etc.
6. Run an Active Scan/spider against the site

• Expected result
• Scan/spider successfully scans/crawls through web page with 200 response codes
• Actual result
• Scan/spider unsuccessfully scans/crawls through web pages and returns a 401 for every page found

IIS logs just show a bunch of 401s trying to access any page within the app.  Event viewer isn't giving me anything.

If there is anything else I can assist with or provide, please let me know!  This will not only benefit me, but everyone who needs to use this.

Best regards,

Steve

[thc...@gmail.com]

Hi.

Thanks for the thorough steps! I'll give that a try. (I also want to see
that fixed once and for all!)

One of the problems is that ZAP is not sending the domain correctly when
trying to authenticate with NTLM. (Not sure what the other problems are...)

Best regards.

> 1. Get a WS2012R2 VM
> 1. https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-2012-r2

> - if you need an evaluation license for the machine

> 2. Install IIS 8.5
> 1. Navigate to server manager
> 2. Click the Manage button in the top-right corner
> 3. Click Add Roles and Features from the context menu
> 4. On the "Before you Begin section", click Next
> 5. On the Installation Type, click Next
> 6. On Server Selection, click Next
> 7. On Server Roles, click Web Server (IIS) and click Next
> 8. Continue with all of the defaults until the installation begins
> 3. Create/use a webapp with Integrated Windows Auth
> 1. Verify AD creds work fine within the app manually
> 4. Open ZAP and configure browser to proxy through ZAP
> 5. Configure context to use HTTP/NTLM auth
> 1. Enter all relevant information regarding host, realm, etc.
> 6. Run an Active Scan/spider against the site
>
>
> - Expected result
> - Scan/spider successfully scans/crawls through web page with 200
> response codes
> - Actual result
> - Scan/spider unsuccessfully scans/crawls through web pages and

[Stephen Heeps]

Hi.

Thanks for the quick response!  It's great to see it and I can't wait to tackle this issue.

FWIW: 

• Host OS: Windows 7
• Browser: Firefox 47.0.1
• IE 11
• Version of ZAP: 2.5.0
• Remote OS: Windows Server 2012 R2
• Remote IIS Version: 8.5

I already said the last 2 pieces before, but just to reiterate in a list.

Please let me know if there's anything else I can assist with or provide!

Best Regards,

Steve

[Stephen Heeps]

Hi there.

I'm sure you're busy and have a life outside of this, but I wanted to follow up, keep this post active, and offer my assistance in any way possible.  When I get some spare cycles, I'll try to create this environment and bundle it up/host it for you if that would make things easier.

Best Regards,

Steve

[thc...@gmail.com]

Hi.

Thanks, I now have Windows Server 2012 R2 up and running.

I'll follow the steps you provided earlier to set up the authentication.
I'll let you know when done... (hopefully will also have news about the
issue.)

Thank you!
Best regards.

[Gustav Boström]

Hi,

I also have the same issue. I've read through all the mentioned posts.

I am trying to set up authentication to an IIS server. It works fine when I just work through the Proxy, but fails when I try to do a scan or resend requests.

I'm using Forced User mode and have user setup. To me this seems like bug in that the credentials are not found for the required authentication scheme. I will have a look through the source code.

The fact that the realm is not properly returned is suspicious.

The server is returning the following header: Server: Microsoft-IIS/8.5

Here is my wire log when resending a request:

2017-05-08 09:33:34,555 [Thread-67] DEBUG header - >> "GET /Fooo HTTP/1.1[\r][\n]"
2017-05-08 09:33:34,555 [Thread-67] DEBUG HttpMethodBase - Adding Host request header
2017-05-08 09:33:34,555 [Thread-67] DEBUG header - >> "Proxy-Connection: keep-alive[\r][\n]"
2017-05-08 09:33:34,555 [Thread-67] DEBUG header - >> "Cache-Control: max-age=0[\r][\n]"
2017-05-08 09:33:34,555 [Thread-67] DEBUG header - >> "UpFooo-Insecure-Requests: 1[\r][\n]"
2017-05-08 09:33:34,555 [Thread-67] DEBUG header - >> "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36[\r][\n]"
2017-05-08 09:33:34,555 [Thread-67] DEBUG header - >> "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8[\r][\n]"
2017-05-08 09:33:34,555 [Thread-67] DEBUG header - >> "Referer: http://acc1.fooadmin.example.com/Fooo[\r][\n]"
2017-05-08 09:33:34,555 [Thread-67] DEBUG header - >> "Accept-Language: sv-SE,sv;q=0.8,en-US;q=0.6,en;q=0.4[\r][\n]"
2017-05-08 09:33:34,556 [Thread-67] DEBUG header - >> "Cookie: ASP.NET_SessionId=bnogjqf4ajyd3ednyft2r5gb[\r][\n]"
2017-05-08 09:33:34,556 [Thread-67] DEBUG header - >> "Content-Length: 0[\r][\n]"
2017-05-08 09:33:34,556 [Thread-67] DEBUG header - >> "Host:acc1.fooadmin.example.com[\r][\n]"
2017-05-08 09:33:34,556 [Thread-67] DEBUG header - >> "[\r][\n]"
2017-05-08 09:33:37,141 [Thread-67] DEBUG header - << "HTTP/1.1 401 Unauthorized[\r][\n]"
2017-05-08 09:33:37,142 [Thread-67] DEBUG header - << "Content-Type: text/html[\r][\n]"
2017-05-08 09:33:37,143 [Thread-67] DEBUG header - << "Server: Microsoft-IIS/8.5[\r][\n]"
2017-05-08 09:33:37,143 [Thread-67] DEBUG header - << "WWW-Authenticate: Negotiate[\r][\n]"
2017-05-08 09:33:37,143 [Thread-67] DEBUG header - << "WWW-Authenticate: NTLM[\r][\n]"
2017-05-08 09:33:37,143 [Thread-67] DEBUG header - << "X-Powered-By: ASP.NET[\r][\n]"
2017-05-08 09:33:37,144 [Thread-67] DEBUG header - << "Date: Mon, 08 May 2017 07:32:45 GMT[\r][\n]"
2017-05-08 09:33:37,144 [Thread-67] DEBUG header - << "Content-Length: 1318[\r][\n]"
2017-05-08 09:33:37,144 [Thread-67] DEBUG header - << "[\r][\n]"
2017-05-08 09:33:37,145 [Thread-67] DEBUG HttpMethodDirector - Authorization required
2017-05-08 09:33:37,145 [Thread-67] DEBUG AuthChallengeProcessor - Supported authentication schemes in the order of preference: [ntlm, digest, basic, ntlm]
2017-05-08 09:33:37,146 [Thread-67] INFO  AuthChallengeProcessor - ntlm authentication scheme selected
2017-05-08 09:33:37,146 [Thread-67] DEBUG AuthChallengeProcessor - Using authentication scheme: ntlm
2017-05-08 09:33:37,146 [Thread-67] DEBUG AuthChallengeProcessor - Authorization challenge processed
2017-05-08 09:33:37,147 [Thread-67] DEBUG HttpMethodDirector - Authentication scope: NTLM <any realm>@acc1.fooadmin.example.com:80
2017-05-08 09:33:37,147 [Thread-67] DEBUG HttpMethodDirector - Credentials required
2017-05-08 09:33:37,147 [Thread-67] DEBUG HttpMethodDirector - Credentials provider not available
2017-05-08 09:33:37,147 [Thread-67] INFO  HttpMethodDirector - No credentials available for NTLM <any realm>@acc1.fooadmin.example.com:80
2017-05-08 09:33:37,150 [Thread-67] DEBUG HttpMethodBase - Buffering response body
2017-05-08 09:33:37,150 [Thread-67] DEBUG HttpMethodBase - Resorting to protocol version default close connection policy
2017-05-08 09:33:37,151 [Thread-67] DEBUG HttpMethodBase - Should NOT close connection, using HTTP/1.1
2017-05-08 09:33:37,163 [Thread-67] DEBUG HttpConnection - Releasing connection back to connection manager.
2017-05-08 09:33:37,164 [Thread-67] DEBUG MultiThreadedHttpConnectionManager - Freeing connection, hostConfig=HostConfiguration[host=http://acc1.fooadmin.example.com]
2017-05-08 09:33:37,164 [Thread-67] DEBUG IdleConnectionHandler - Adding connection at: 1494228817164
2017-05-08 09:33:37,164 [Thread-67] DEBUG MultiThreadedHttpConnectionManager - Notifying no-one, there are no waiting threads

[thc...@gmail.com]

Hi.

Could you start the scan by selecting the User instead of using Forced
User mode?

(I'll check the code maybe there's an issue with Forced User mode.)

Best regards.

> *2017-05-08 09:33:37,147 [Thread-67] INFO HttpMethodDirector - No
> credentials available for NTLM <any realm>@acc1.fooadmin.example.com:80*

[thc...@gmail.com]

Hi.

I enabled "Windows Authentication" in the "Default Web Site" and ZAP was
able to successfully authenticate, when proxying and active scanning as
a user.

Any ideas on what needs to be changed to reproduce the issue you
mentioned earlier?

Thanks.
Best regards.

[Gustav Boström]

Thanks,

Tried to do a scan without forced user. But get only 401 unauthorized users.

I specified the user manually for the scan.

Looking at this source code it seems NTLM does not handle a domain/realm:

https://github.com/zaproxy/zaproxy/blob/develop/src/org/zaproxy/zap/network/ZapNTLMScheme.java

Is that really correct?

[Gustav Boström]

Not sure what you mean with this config. The default context?

With Windows authentication do you mean NTLM?

Den måndag 8 maj 2017 kl. 10:15:47 UTC+2 skrev thc202:

[Stephen Heeps]

Hi,

What does your environment look like?  Do you have ZAP running on the server hosting IIS?  Browser?

I believe Gustav and I might be using the same environment.  We have our client (Windows 7-10) that has ZAP (2.5.0) installed, using Firefox or I.E. as our proxy and over on a different box we have WS2012R2 hosting IIS and our site.  I wonder if ZAP will let you authenticate if you're on the box hosting the server versus using a client and trying to hit it through a proxy on two machines?

A few more steps in reproduction (I am easily able to repro this issue)

1. Start up ZAP
2. Start up browser (I use I.E. 11 currently)
1. Proxy browser to localhost:8080 where ZAP's default proxy is configured
3. Navigate to site on IIS (WS2012) from client (W7)
4. Create context
1. Authentication
1. HTTP/NTLM Authentication
2. Hostname: hostname of application
3. Realm: domain I am on from Active Directory
4. Regex pattern identified in Logged In response messages: I have pointing to an element on the page
2. Users
1. AD user (no domain)
3. Forced User set to my account but disabled
4. Session Management
1. HTTP
5. Begin scan

Expected: Scan returns 200 and scans application

Actual: Scan returns 401 on every page it hits.  Wire log is shown in an above post and Gustav's post shows the same results.

Are you using a dev build of ZAP?  I am running on ZAP 2.5.0.  

[thc...@gmail.com]

Yes, NTLM/Negotiate, I was referring to the steps provided by Stephen in
an earlier post.

Best regards.

On 08/05/17 09:35, Gustav Boström wrote:
> Not sure what you mean with this config. The default context?
> With Windows authentication do you mean NTLM?
>
>
> Den måndag 8 maj 2017 kl. 10:15:47 UTC+2 skrev thc202:
>
>> Hi.
>>
>> I enabled "Windows Authentication" in the "Default Web Site" and ZAP was
>> able to successfully authenticate, when proxying and active scanning as
>> a user.
>>
>> Any ideas on what needs to be changed to reproduce the issue you
>> mentioned earlier?
>>
>> Thanks.
>> Best regards.
>>

[thc...@gmail.com]

The realm (not domain) in that class is referring to:
https://tools.ietf.org/html/rfc7235#section-2.2

which NTLM does not have/use.

Best regards.

On 08/05/17 09:33, Gustav Boström wrote:
> Thanks,
>
> Tried to do a scan without forced user. But get only 401 unauthorized users.
> I specified the user manually for the scan.
>
>
> Looking at this source code it seems NTLM does not handle a domain/realm:
> https://github.com/zaproxy/zaproxy/blob/develop/src/org/zaproxy/zap/network/ZapNTLMScheme.java
>
> Is that really correct?
>
>
> Den måndag 8 maj 2017 kl. 10:15:47 UTC+2 skrev thc202:
>
>> Hi.
>>
>> I enabled "Windows Authentication" in the "Default Web Site" and ZAP was
>> able to successfully authenticate, when proxying and active scanning as
>> a user.
>>
>> Any ideas on what needs to be changed to reproduce the issue you
>> mentioned earlier?
>>
>> Thanks.
>> Best regards.
>>

[thc...@gmail.com]

Hi.

> What does your environment look like? Do you have ZAP running on the
> server hosting IIS? Browser?

Running Windows Server in a virtual machine and using Linux to run ZAP
(I'm not using a browser, all setup done with ZAP).

> I wonder if ZAP will let you authenticate if you're on the box
> hosting the server versus using a client and trying to hit it through a
> proxy on two machines?

ZAP should authenticate always (that's how it should be, at least).

> 5. Begin scan

How are you starting the scan? Are you specifying the context/user? Or,
using Forced User mode as Gustav?

Could you share the log of the scan's first failed authentication?

> Are you using a dev build of ZAP? I am running on ZAP 2.5.0.

I was running a dev build (which would be similar to 2.6.0), but just
tested with ZAP 2.5.0 and the behaviour is the same.

Here is the log of the successful authentication (this one using ZAP
2.5.0) while active scanning with a context/user:
> 321230 [Thread-116] DEBUG httpclient.wire.header - >> "GET /6752984764206071014 HTTP/1.1[\r][\n]"
> 321230 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodBase - Adding Host request header
> 321230 [Thread-116] DEBUG httpclient.wire.header - >> "User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0[\r][\n]"
> 321230 [Thread-116] DEBUG httpclient.wire.header - >> "Pragma: no-cache[\r][\n]"
> 321230 [Thread-116] DEBUG httpclient.wire.header - >> "Cache-Control: no-cache[\r][\n]"
> 321231 [Thread-116] DEBUG httpclient.wire.header - >> "Content-Length: 0[\r][\n]"
> 321231 [Thread-116] DEBUG httpclient.wire.header - >> "Authorization: NTLM TlRMTVNTUAADAAAAGAAYAEgAAADIAMgAYAAAAAYABgAoAQAAGgAaAC4BAAAIAAgASAEAAAAAAABQAQAABYKIogUBKAoAAAAPaO2WU/Yp6iRGVXI6xoDtLA2o3oC+G9r6wu4vL2jRBaWZbxPNHCOXAwEBAAAAAAAAQGsLyAzI0gGZXj65GbmA8wAAAAACAB4AVwBJAE4ALQBOAEgAQwBNAEkATgBEAEYATABSAE0AAQAeAFcASQBOAC0ATgBIAEMATQBJAE4ARABGAEwAUgBNAAQAHgBXAEkATgAtAE4ASABDAE0ASQBOAEQARgBMAFIATQADAB4AVwBJAE4ALQBOAEgAQwBNAEkATgBEAEYATABSAE0ABwAIAHzAndVPyNIBAAAAAAAAAAAxADkAMgBBAGQAbQBpAG4AaQBzAHQAcgBhAHQAbwByAE0AYQByAHMA[\r][\n]"
> 321231 [Thread-116] DEBUG httpclient.wire.header - >> "Host: 192.168.122.96[\r][\n]"
> 321231 [Thread-116] DEBUG httpclient.wire.header - >> "[\r][\n]"
> 321234 [Thread-116] DEBUG httpclient.wire.header - << "HTTP/1.1 401 Unauthorized[\r][\n]"
> 321234 [Thread-116] DEBUG httpclient.wire.header - << "Content-Type: text/html[\r][\n]"
> 321234 [Thread-116] DEBUG httpclient.wire.header - << "Server: Microsoft-IIS/8.5[\r][\n]"
> 321234 [Thread-116] DEBUG httpclient.wire.header - << "WWW-Authenticate: Negotiate[\r][\n]"
> 321234 [Thread-116] DEBUG httpclient.wire.header - << "WWW-Authenticate: NTLM[\r][\n]"
> 321234 [Thread-116] DEBUG httpclient.wire.header - << "Date: Mon, 08 May 2017 23:10:16 GMT[\r][\n]"
> 321234 [Thread-116] DEBUG httpclient.wire.header - << "Content-Length: 1293[\r][\n]"
> 321235 [Thread-116] DEBUG httpclient.wire.header - << "[\r][\n]"
> 321235 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Authorization required
> 321236 [Thread-116] DEBUG org.apache.commons.httpclient.auth.AuthChallengeProcessor - Supported authentication schemes in the order of preference: [ntlm, digest, basic, ntlm]
> 321237 [Thread-116] INFO org.apache.commons.httpclient.auth.AuthChallengeProcessor - ntlm authentication scheme selected
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.auth.AuthChallengeProcessor - Using authentication scheme: ntlm
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.auth.AuthChallengeProcessor - Authorization challenge processed
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Authentication scope: NTLM <any realm>@192.168.122.96:80
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Retry authentication
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodBase - Resorting to protocol version default close connection policy
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodBase - Should NOT close connection, using HTTP/1.1
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - User defined 'Authorization' headers present in the request.
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Authenticating with NTLM <any realm>@192.168.122.96:80
> 321238 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Removed user defined 'Authorization' headers.
> 321240 [Thread-116] DEBUG httpclient.wire.header - >> "GET /6752984764206071014 HTTP/1.1[\r][\n]"
> 321241 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodBase - Adding Host request header
> 321241 [Thread-116] DEBUG httpclient.wire.header - >> "User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0[\r][\n]"
> 321241 [Thread-116] DEBUG httpclient.wire.header - >> "Pragma: no-cache[\r][\n]"
> 321241 [Thread-116] DEBUG httpclient.wire.header - >> "Cache-Control: no-cache[\r][\n]"
> 321241 [Thread-116] DEBUG httpclient.wire.header - >> "Content-Length: 0[\r][\n]"
> 321241 [Thread-116] DEBUG httpclient.wire.header - >> "Authorization: NTLM TlRMTVNTUAABAAAAAYIIogAAAAAoAAAAAAAAACgAAAAFASgKAAAADw==[\r][\n]"
> 321241 [Thread-116] DEBUG httpclient.wire.header - >> "Host: 192.168.122.96[\r][\n]"
> 321241 [Thread-116] DEBUG httpclient.wire.header - >> "[\r][\n]"
> 321243 [Thread-116] DEBUG httpclient.wire.header - << "HTTP/1.1 401 Unauthorized[\r][\n]"
> 321244 [Thread-116] DEBUG httpclient.wire.header - << "Content-Type: text/html; charset=us-ascii[\r][\n]"
> 321244 [Thread-116] DEBUG httpclient.wire.header - << "Server: Microsoft-HTTPAPI/2.0[\r][\n]"
> 321244 [Thread-116] DEBUG httpclient.wire.header - << "WWW-Authenticate: NTLM TlRMTVNTUAACAAAAHgAeADgAAAAFgoqix10Afc1NHz8AAAAAAAAAAJgAmABWAAAABgOAJQAAAA9XAEkATgAtAE4ASABDAE0ASQBOAEQARgBMAFIATQACAB4AVwBJAE4ALQBOAEgAQwBNAEkATgBEAEYATABSAE0AAQAeAFcASQBOAC0ATgBIAEMATQBJAE4ARABGAEwAUgBNAAQAHgBXAEkATgAtAE4ASABDAE0ASQBOAEQARgBMAFIATQADAB4AVwBJAE4ALQBOAEgAQwBNAEkATgBEAEYATABSAE0ABwAIABU8dEFQyNIBAAAAAA==[\r][\n]"
> 321244 [Thread-116] DEBUG httpclient.wire.header - << "Date: Mon, 08 May 2017 23:10:16 GMT[\r][\n]"
> 321244 [Thread-116] DEBUG httpclient.wire.header - << "Content-Length: 341[\r][\n]"
> 321244 [Thread-116] DEBUG httpclient.wire.header - << "[\r][\n]"
> 321244 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Authorization required
> 321244 [Thread-116] DEBUG org.apache.commons.httpclient.auth.AuthChallengeProcessor - Using authentication scheme: ntlm
> 321244 [Thread-116] DEBUG org.apache.commons.httpclient.auth.AuthChallengeProcessor - Authorization challenge processed
> 321244 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Authentication scope: NTLM <any realm>@192.168.122.96:80
> 321244 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Retry authentication
> 321245 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodBase - Resorting to protocol version default close connection policy
> 321245 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodBase - Should NOT close connection, using HTTP/1.1
> 321245 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodDirector - Authenticating with NTLM <any realm>@192.168.122.96:80
> 321248 [Thread-116] DEBUG httpclient.wire.header - >> "GET /6752984764206071014 HTTP/1.1[\r][\n]"
> 321248 [Thread-116] DEBUG org.apache.commons.httpclient.HttpMethodBase - Adding Host request header
> 321248 [Thread-116] DEBUG httpclient.wire.header - >> "User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:39.0) Gecko/20100101 Firefox/39.0[\r][\n]"
> 321248 [Thread-116] DEBUG httpclient.wire.header - >> "Pragma: no-cache[\r][\n]"
> 321248 [Thread-116] DEBUG httpclient.wire.header - >> "Cache-Control: no-cache[\r][\n]"
> 321248 [Thread-116] DEBUG httpclient.wire.header - >> "Content-Length: 0[\r][\n]"
> 321248 [Thread-116] DEBUG httpclient.wire.header - >> "Authorization: NTLM TlRMTVNTUAADAAAAGAAYAEgAAADIAMgAYAAAAAYABgAoAQAAGgAaAC4BAAAIAAgASAEAAAAAAABQAQAABYKIogUBKAoAAAAPCN/t2lHRfL67BFlttkLeZVNmasWHsj3iSVQfFd4Y23GIc2PQSl7kxAEBAAAAAAAA4HvfMw3I0gGk2LVrMwNtegAAAAACAB4AVwBJAE4ALQBOAEgAQwBNAEkATgBEAEYATABSAE0AAQAeAFcASQBOAC0ATgBIAEMATQBJAE4ARABGAEwAUgBNAAQAHgBXAEkATgAtAE4ASABDAE0ASQBOAEQARgBMAFIATQADAB4AVwBJAE4ALQBOAEgAQwBNAEkATgBEAEYATABSAE0ABwAIABU8dEFQyNIBAAAAAAAAAAAxADkAMgBBAGQAbQBpAG4AaQBzAHQAcgBhAHQAbwByAE0AYQByAHMA[\r][\n]"
> 321248 [Thread-116] DEBUG httpclient.wire.header - >> "Host: 192.168.122.96[\r][\n]"
> 321249 [Thread-116] DEBUG httpclient.wire.header - >> "[\r][\n]"
> 321251 [Thread-116] DEBUG httpclient.wire.header - << "HTTP/1.1 404 Not Found[\r][\n]"
> 321251 [Thread-116] DEBUG httpclient.wire.header - << "Content-Type: text/html[\r][\n]"
> 321251 [Thread-116] DEBUG httpclient.wire.header - << "Server: Microsoft-IIS/8.5[\r][\n]"
> 321251 [Thread-116] DEBUG httpclient.wire.header - << "Persistent-Auth: true[\r][\n]"
> 321251 [Thread-116] DEBUG httpclient.wire.header - << "Date: Mon, 08 May 2017 23:10:16 GMT[\r][\n]"
> 321252 [Thread-116] DEBUG httpclient.wire.header - << "Content-Length: 1245[\r][\n]"
> 321252 [Thread-116] DEBUG httpclient.wire.header - << "[\r][\n]"


Best regards.

On 08/05/17 15:40, Stephen Heeps wrote:
> Hi,
>
> What does your environment look like? Do you have ZAP running on the
> server hosting IIS? Browser?
>
> I believe Gustav and I might be using the same environment. We have our
> client (Windows 7-10) that has ZAP (2.5.0) installed, using Firefox or I.E.
> as our proxy and over on a different box we have WS2012R2 hosting IIS and
> our site. I wonder if ZAP will let you authenticate if you're on the box
> hosting the server versus using a client and trying to hit it through a
> proxy on two machines?
>
> A few more steps in reproduction (I am easily able to repro this issue)
>
>

> 1. Start up ZAP
> 2. Start up browser (I use I.E. 11 currently)
> 1. Proxy browser to localhost:8080 where ZAP's default proxy is
> configured
> 3. Navigate to site on IIS (WS2012) from client (W7)
> 4. Create context
> 1. Authentication
> 1. HTTP/NTLM Authentication
> 2. Hostname: hostname of application
> 3. Realm: domain I am on from Active Directory
> 4. Regex pattern identified in Logged In response messages: I have

> pointing to an element on the page

> 2. Users
> 1. AD user (no domain)
> 3. Forced User set to my account but disabled
> 4. Session Management
> 1. HTTP
> 5. Begin scan

>
> Expected: Scan returns 200 and scans application
>
> Actual: Scan returns 401 on every page it hits. Wire log is shown in an
> above post and Gustav's post shows the same results.
>
> Are you using a dev build of ZAP? I am running on ZAP 2.5.0.
>
> On Monday, May 8, 2017 at 4:15:47 AM UTC-4, thc202 wrote:
>>
>> Hi.
>>
>> I enabled "Windows Authentication" in the "Default Web Site" and ZAP was
>> able to successfully authenticate, when proxying and active scanning as
>> a user.
>>
>> Any ideas on what needs to be changed to reproduce the issue you
>> mentioned earlier?
>>
>> Thanks.
>> Best regards.
>>

[Stephen Heeps]

Hi,

> How are you starting the scan? Are you specifying the context/user? Or, 
> using Forced User mode as Gustav? 

I am specifying the context/user.  I include my site into a new context and under the Authentication section I use HTTP/NTLM and specify the Hostname and realm.  I go to users and put in my credentials (no domain here).  When I attempt to scan with context/user I get the following:

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "GET /Scripts/js.js HTTP/1.1[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG HttpMethodBase - Adding Host request header

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "Accept: */*[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "Referer: http://website/[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "Accept-Language: en-US[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "Cookie: __AntiXsrfToken=7c6d99760df54b609acbd788b9cf127b[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "If-Modified-Since: Thu, 30 Mar 2017 18:55:58 GMT[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "If-None-Match: "033604487a9d21:0"[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "Proxy-Connection: Keep-Alive[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAHgAAABWAVYBkAAAAAgACABYAAAADAAMAGAAAAAMAAwAbAAAABAAEADmAQAAFYKI4gYBsR0AAAAPanhwtWqcI2UiH93PH0ErnUUAVgBOAFQAUwBIAGUAZQBwAHMAUwBIAEUARQBQAFMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABI/U/mp3suEtOgZ38WY6OwEBAAAAAAAAlxbMYQnI0gE0yuI9SpiRPwAAAAACAAgARQBWAE4AVAABABQAQgBFAEQALQA5ADkANQAtADIAMAAEABwAZQBhAHQAbwBuAHYAYQBuAGMAZQAuAGMAbwBtAAMAMgBCAEUARAAtADkAOQA1AC0AMgAwAC4AZQBhAHQAbwBuAHYAYQBuAGMAZQAuAGMAbwBtAAUAGgB3AGkAbgByAG8AbwB0AC4AbABvAGMAYQBsAAcACACXFsxhCcjSAQYABAACAAAACAAwADAAAAAAAAAAAAAAAAAgAAAd+Ah56AZgKBF9eCcDNEX1/S4pWG8IpJg182u5Ppq3RgoAEAAAAAAAAAAAAAAAAAAAAAAACQAmAEgAVABUAFAALwBkAGUAdgAtAHAAbwBjAC0AdwBlAGIAYQBwAHAAAAAAAAAAAAAAAAAAm+E5yHf/4lXnhiS6QQcR/Q==[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "Host: website[\r][\n]"

2017-05-08 10:42:58,096 [ZAP-ProxyThread-964] DEBUG header - >> "[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "HTTP/1.1 401 Unauthorized[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "Content-Type: text/html[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "Server: Microsoft-IIS/8.5[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "WWW-Authenticate: Negotiate[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "WWW-Authenticate: NTLM[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "X-Powered-By: ASP.NET[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "Date: Mon, 08 May 2017 14:42:56 GMT[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "Content-Length: 1293[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG header - << "[\r][\n]"

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG HttpMethodDirector - Authorization required

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG AuthChallengeProcessor - Using authentication scheme: ntlm

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG AuthChallengeProcessor - Authorization challenge processed

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG HttpMethodDirector - Authentication scope: NTLM <any realm>@website:80

2017-05-08 10:42:58,098 [ZAP-ProxyThread-964] DEBUG HttpMethodDirector - Credentials required

2017-05-08 10:42:58,099 [ZAP-ProxyThread-964] DEBUG HttpMethodDirector - Credentials provider not available

2017-05-08 10:42:58,099 [ZAP-ProxyThread-964] INFO  HttpMethodDirector - Failure authenticating with NTLM <any realm>@website:80

My confusion lies here:

Failure authenticating with NTLM <any realm>@website:80

Why do we say <any realm> instead of the realm given?  I have the realm configured to be the domain we use.  I am definitely willing to chalk this up to being a configuration issue if that is the case.  Can you give me the configuration that you have so I can do a comparison to see if I am missing anything?

Best regards 

[Gustav Boström]

Hi,

My setup i similar, except that I am using ZAP 2.6.0 and IIS is running on a different machine.

I have tried both Forced User Mode and specifying a user for the context when starting a scan.

My trace looks the same as Stephens. I don't have  a Java dev Environment right now, otherwise I would do a debug sessions to see what happens.

My brief reading of the source code gives me a hypothesis that ZAP is not picking up the credentials and setting them for some reason. The log message we get , "Credentials provider not available",   happens when there are no CredentialProvider set on the HttpParams given to HttpClient.

Cheers,

[Gustav Boström]

We have a difference in config here. You are using the Default Context and not  a created context.

Try creating a context with NTML Authentication and Authorization setup and Forced User.

It could be that there is a bug in created contexts, but not the default.

Den måndag 8 maj 2017 kl. 10:15:47 UTC+2 skrev thc202:

[thc...@gmail.com]

Hi.

> The log message we get , "Credentials provider not available", happens when there
> are no CredentialProvider set on the HttpParams given to HttpClient.

Not necessarily, that also happens when the authentication credentials
set do not produce a successful authentication (e.g. wrong password).

Do you not see any authentication being attempted at all?

Best regards.

[thc...@gmail.com]

Same thing with a new/created context.

Could you (Gustav and Stephen) provide more details on the Windows
Authentication of your servers? I'd like to replicate that to reproduce
the issue. As mentioned before I'm just using the default configurations
in Windows Server, maybe there's some difference causing the issue?

Best regards.

On 09/05/17 08:34, Gustav Boström wrote:
> We have a difference in config here. You are using the Default Context and
> not a created context.
> Try creating a context with NTML Authentication and Authorization setup and
> Forced User.
>
> It could be that there is a bug in created contexts, but not the default.
>
>
>
> Den måndag 8 maj 2017 kl. 10:15:47 UTC+2 skrev thc202:
>
>> Hi.
>>
>> I enabled "Windows Authentication" in the "Default Web Site" and ZAP was
>> able to successfully authenticate, when proxying and active scanning as
>> a user.
>>
>> Any ideas on what needs to be changed to reproduce the issue you
>> mentioned earlier?
>>
>> Thanks.
>> Best regards.
>>

[thc...@gmail.com]

Hi.

> Why do we say <any realm> instead of the realm given?

Ignore that, the realm here is the authentication realm (as defined in
https://tools.ietf.org/html/rfc7235#section-2.2) not the domain, it does
not apply to NTLM.

> Can you give me the configuration that you have so I can do a comparison to
> see if I am missing anything?

Of course:

Context Name:
WS2012
Include in Context:
http://192.168.122.96.*

Authentication:
HTTP/NTLM Authentication
Hostname: 192.168.122.96
Port: 80
Realm: (leave empty)

Session Management
HTTP Authentication Session Management


Any guidance on the configurations (server side) to reproduce the issue
would be greatly appreciated.

Best regards.

[Gustav Boström]

Are you sure about that? The class: 

• org.apache.commons.httpclient.HttpMethodDirector
  

Only has two places with the log message  "Credentials provider not available" and they are both in methods that tries to get the credentials from the parameters.

How does your successful wire log  look like? In ours it doesn't look like any more authentication happens at all.

I'm afraid I don't have any more info on the server I'm running against at the moment. I'll try to get it.

[Stephen Heeps]

Hi,

I'd love to give you more information.  I'm not entirely sure how relevant most of the following information will be:

• OS: WS2012R2
• IIS: 8.5
• AppPools
• We created a separate one for our application
• .NET CLR version: .NET CLR VERSION v4.0.30319
• Managed Pipeline mode: Classic
• Identity: NetworkService
• Enable 32-Bit Applications: True
• IIS Site
• We created a new one for our application
• ASP.NET Impersonation: Enabled
• Forms Authentication: Enabled
• Response Type: HTTP 302 Login/Redirect
• Windows Authentication: Enabled
• Response Type: HTTP 401 Challenge
• Site is using Pass-through Authentication

WS2012R2 is connected to a DC (domain controller) our domain <domain>.com which hosts our users and policies.

This is as much information as I can find out about our environment server-side.  I wish I could be of more help there.

I have tried Default Context and created Context and both give me the same results.  However, I tried using a user vs. forced user and got different results.  Here are the wire logs:

Forced User:

2017-05-09 09:27:31,248 [Thread-344] INFO  User - Authenticating user: domain\user

2017-05-09 09:27:31,248 [Thread-344] DEBUG DefaultHttpParams - Set parameter http.protocol.version = HTTP/1.0

2017-05-09 09:27:31,248 [Thread-344] DEBUG DefaultHttpParams - Set parameter http.protocol.version = HTTP/1.1

2017-05-09 09:27:31,248 [Thread-344] DEBUG DefaultHttpParams - Set parameter http.protocol.cookie-policy = compatibility

2017-05-09 09:27:31,249 [Thread-344] DEBUG MultiThreadedHttpConnectionManager - HttpConnectionManager.getConnection:  config = HostConfiguration[host=http://website], timeout = 0

2017-05-09 09:27:31,249 [Thread-344] DEBUG MultiThreadedHttpConnectionManager - Getting free connection, hostConfig=HostConfiguration[host=http://website]

2017-05-09 09:27:31,249 [Thread-344] DEBUG HttpMethodDirector - User defined 'Authorization' headers present in the request.

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "GET /*/5641872227961592590 HTTP/1.1[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG HttpMethodBase - Adding Host request header

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "Accept: */*[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "Referer: http://website/[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "Accept-Language: en-US[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "Proxy-Connection: Keep-Alive[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "Cookie: __AntiXsrfToken=7c6d99760df54b609acbd788b9cf127b[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "Authorization: Negotiate TlRMTVNTUAABAAAAl4II4gAAAAAAAAAAAAAAAAAAAAAGAbEdAAAADw==[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "Host: website[\r][\n]"

2017-05-09 09:27:31,250 [Thread-344] DEBUG header - >> "[\r][\n]"

2017-05-09 09:27:31,252 [Thread-344] DEBUG header - << "HTTP/1.1 401 Unauthorized[\r][\n]"

2017-05-09 09:27:31,252 [Thread-344] DEBUG header - << "Content-Type: text/html; charset=us-ascii[\r][\n]"

2017-05-09 09:27:31,252 [Thread-344] DEBUG header - << "Server: Microsoft-HTTPAPI/2.0[\r][\n]"

2017-05-09 09:27:31,252 [Thread-344] DEBUG header - << "WWW-Authenticate: Negotiate TlRMTVNTUAACAAAACAAIADgAAAAVgoniGMoCQLWWIkYAAAAAAAAAAKgAqABAAAAABgOAJQAAAA9FAFYATgBUAAIACABFAFYATgBUAAEAFABCAEUARAAtADkAOQA1AC0AMgAwAAQAHABlAGEAdABvAG4AdgBhAG4AYwBlAC4AYwBvAG0AAwAyAEIARQBEAC0AOQA5ADUALQAyADAALgBlAGEAdABvAG4AdgBhAG4AYwBlAC4AYwBvAG0ABQAaAHcAaQBuAHIAbwBvAHQALgBsAG8AYwBhAGwABwAIAJQLcwLIyNIBAAAAAA==[\r][\n]"

2017-05-09 09:27:31,252 [Thread-344] DEBUG header - << "Date: Tue, 09 May 2017 13:27:30 GMT[\r][\n]"

2017-05-09 09:27:31,252 [Thread-344] DEBUG header - << "Content-Length: 341[\r][\n]"

2017-05-09 09:27:31,252 [Thread-344] DEBUG header - << "[\r][\n]"

2017-05-09 09:27:31,252 [Thread-344] DEBUG HttpMethodDirector - Authorization required

2017-05-09 09:27:31,252 [Thread-344] DEBUG AuthChallengeProcessor - Supported authentication schemes in the order of preference: [ntlm, digest, basic, ntlm]

2017-05-09 09:27:31,252 [Thread-344] DEBUG AuthChallengeProcessor - Challenge for ntlm authentication scheme not available

2017-05-09 09:27:31,252 [Thread-344] DEBUG AuthChallengeProcessor - Challenge for digest authentication scheme not available

2017-05-09 09:27:31,252 [Thread-344] DEBUG AuthChallengeProcessor - Challenge for basic authentication scheme not available

2017-05-09 09:27:31,252 [Thread-344] DEBUG AuthChallengeProcessor - Challenge for ntlm authentication scheme not available

2017-05-09 09:27:31,252 [Thread-344] WARN  HttpMethodDirector - Unable to respond to any of these challenges: {negotiate=Negotiate TlRMTVNTUAACAAAACAAIADgAAAAVgoniGMoCQLWWIkYAAAAAAAAAAKgAqABAAAAABgOAJQAAAA9FAFYATgBUAAIACABFAFYATgBUAAEAFABCAEUARAAtADkAOQA1AC0AMgAwAAQAHABlAGEAdABvAG4AdgBhAG4AYwBlAC4AYwBvAG0AAwAyAEIARQBEAC0AOQA5ADUALQAyADAALgBlAGEAdABvAG4AdgBhAG4AYwBlAC4AYwBvAG0ABQAaAHcAaQBuAHIAbwBvAHQALgBsAG8AYwBhAGwABwAIAJQLcwLIyNIBAAAAAA==}

2017-05-09 09:27:31,253 [Thread-344] DEBUG HttpMethodBase - Buffering response body

2017-05-09 09:27:31,253 [Thread-344] DEBUG HttpMethodBase - Resorting to protocol version default close connection policy

2017-05-09 09:27:31,253 [Thread-344] DEBUG HttpMethodBase - Should NOT close connection, using HTTP/1.1

2017-05-09 09:27:31,253 [Thread-344] DEBUG HttpConnection - Releasing connection back to connection manager.

2017-05-09 09:27:31,253 [Thread-344] DEBUG MultiThreadedHttpConnectionManager - Freeing connection, hostConfig=HostConfiguration[host=http://website]

Note: I did get the same results when using user vs forced user.

I did some more testing with HTTP sessions and different formats for username and got some interesting results:

2017-05-09 09:52:12,938 [ZAP-ProxyThread-100] DEBUG HttpMethodDirector - User defined 'Authorization' headers present in the request.

2017-05-09 09:52:12,938 [ZAP-ProxyThread-100] DEBUG HttpMethodDirector - Authenticating with NTLM <any realm>@website:80

2017-05-09 09:52:12,938 [ZAP-ProxyThread-100] DEBUG HttpMethodDirector - Ignoring authentication, user defined 'Authorization' headers present in the request.

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "GET /bundles/modernizr?v=Vd40cG5fYxxjdknf_y9ilK-zi7pnjL35tk9IAsOQgQc1 HTTP/1.1[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG HttpMethodBase - Adding Host request header

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "Accept: */*[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "Referer: http://website/[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "Accept-Language: en-US[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "Proxy-Connection: Keep-Alive[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "If-Modified-Since: Tue, 09 May 2017 13:49:33 GMT[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "Cookie: __AntiXsrfToken=7c6d99760df54b609acbd788b9cf127b[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "Pragma: no-cache[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAHgAAABWAVYBkAAAAAgACABYAAAADAAMAGAAAAAMAAwAbAAAABAAEADmAQAAFYKI4gYBsR0AAAAPd8QGk/LV3PGG6bwqLNmYFUUAVgBOAFQAUwBIAGUAZQBwAHMAUwBIAEUARQBQAFMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAifng/EN0U/VxudBOpCMJdwEBAAAAAAAAg3KLdcvI0gGREeyd7M77YAAAAAACAAgARQBWAE4AVAABABQAQgBFAEQALQA5ADkANQAtADIAMAAEABwAZQBhAHQAbwBuAHYAYQBuAGMAZQAuAGMAbwBtAAMAMgBCAEUARAAtADkAOQA1AC0AMgAwAC4AZQBhAHQAbwBuAHYAYQBuAGMAZQAuAGMAbwBtAAUAGgB3AGkAbgByAG8AbwB0AC4AbABvAGMAYQBsAAcACACDcot1y8jSAQYABAACAAAACAAwADAAAAAAAAAAAAAAAAAgAAAd+Ah56AZgKBF9eCcDNEX1/S4pWG8IpJg182u5Ppq3RgoAEAAAAAAAAAAAAAAAAAAAAAAACQAmAEgAVABUAFAALwBkAGUAdgAtAHAAbwBjAC0AdwBlAGIAYQBwAHAAAAAAAAAAAAAAAAAADiPlkuREP8n4EMzfzqT42w==[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "Host: website[\r][\n]"

2017-05-09 09:52:12,939 [ZAP-ProxyThread-100] DEBUG header - >> "[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "HTTP/1.1 401 Unauthorized[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "Content-Type: text/html[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "Server: Microsoft-IIS/8.5[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "WWW-Authenticate: Negotiate[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "WWW-Authenticate: NTLM[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "X-Powered-By: ASP.NET[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "Date: Tue, 09 May 2017 13:52:12 GMT[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "Content-Length: 1293[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG header - << "[\r][\n]"

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG HttpMethodDirector - Authorization required

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG AuthChallengeProcessor - Using authentication scheme: ntlm

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG AuthChallengeProcessor - Authorization challenge processed

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG HttpMethodDirector - Authentication scope: NTLM <any realm>@website:80

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG HttpMethodDirector - Credentials required

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] DEBUG HttpMethodDirector - Credentials provider not available

2017-05-09 09:52:12,941 [ZAP-ProxyThread-100] INFO  HttpMethodDirector - Failure authenticating with NTLM <any realm>@website:80

These results give me some sort of promise that it is, in fact, a configuration issue.  I was doing some experimentation and got lost in some of the steps so I'm going through some more ad hoc'ing to figure out what's going on.  I removed anything from the realm field and I seem to be getting better results.  And by better, I mean different failures.  Hopefully some of this points you in the right direction @thc and Gustav.

Best regards. 

[Gustav Boström]

Sorry, just reread your earlier post with a successful trace from a scan.

The log message  "Credentials provider not available" is not present. Neither is "Credentials required".

I don't know if that makes me much wiser. With some time I will setup a development environment and start to debug.

[Stephen Heeps]

Hi,

A follow up question I have to this is how do we pass a domain or do we need to pass it at all?  When we use the user context, do we pass it domain\user or just user?  I've been testing on both 2.5.0 as well as the latest dev version (using intellij to build/run it) and get the same results ending with a Failure of Authencation:

| 2017-05-09 12:51:00,233 [Thread-49] INFO  HttpMethodDirector - Failure authenticating with NTLM <any realm>@website:80

Can ZAP successfully authenticate a web application that is hooked up to an Active Directory domain?  If so, is there any documentation regarding this?  The NTLM documentation seems a little lackluster imho.

Do you have a Windows 7 client that you could test with against a WS2012R2 server?  Could it be the host OS that's causing the issue?

As a side note, I appreciate everything you've been doing to repro this issue with us @thc!

Best Regards.

[thc...@gmail.com]

> Are you sure about that?

Yes, if for whatever reason "ZAP" is not able to authenticate (with
existing credentials) it ends up saying "Credentials provider not
available" since it no longer has anymore to try.

> In ours it doesn't look like any more authentication happens at all.

Right, so, what happens when you try to access a page [1] that requires
authentication with Forced User mode enabled? Still no attempts?

> I don't know if that makes me much wiser. With some time I will setup a
> development environment and start to debug.

That's why I'm here ;) but I'm not able to reproduce the issue...


[1] https://github.com/zaproxy/zap-core-help/wiki/HelpUiDialogsMan_req

Best regards.

On 09/05/17 15:05, Gustav Boström wrote:
> Sorry, just reread your earlier post with a successful trace from a scan.
>
> The log message "Credentials provider not available" is not present.
> Neither is "Credentials required".
>
> I don't know if that makes me much wiser. With some time I will setup a
> development environment and start to debug.
>
>
> Den tisdag 9 maj 2017 kl. 13:38:00 UTC+2 skrev Gustav Boström:
>
>> Are you sure about that? The class:

>> - org.apache.commons.httpclient.HttpMethodDirector

>>
>> Only has two places with the log message "Credentials provider not
>> available" and they are both in methods that tries to get the credentials
>> from the parameters.
>> How does your successful wire log look like? In ours it doesn't look like
>> any more authentication happens at all.
>>
>> I'm afraid I don't have any more info on the server I'm running against at
>> the moment. I'll try to get it.
>>

>> -

[thc...@gmail.com]

Thanks for the detailed configurations posted earlier, unfortunately I'm
not able to replicate that :/

> A follow up question I have to this is how do we pass a domain or do we
> need to pass it at all?

The current code is not passing the domain, but it might be needed.

I did some changes in the past to allow to pass the domain although I
had reports that didn't help much:
https://github.com/thc202/zaproxy/commit/bf310eb49e19c57496937300bdf1b2a7a2d7e472

In any case I think those changes should be applied to main repo, I'll
also try to backport the NTLM implementation (more enhancements upstream
which might help in this case).

> When we use the user context, do we pass it domain\user or just user?

For the username field it should be just the user name (i.e. without
"domain\").

> Can ZAP successfully authenticate a web application that is hooked up to an
> Active Directory domain?

It should, that's transparent to the client, it just needs to provide
the correct data.

> If so, is there any documentation regarding this?

Not in ZAP side, but there's:
https://msdn.microsoft.com/en-us/library/cc236621.aspx
https://msdn.microsoft.com/en-us/library/cc237488.aspx

Although that's useful only when inspecting/comparing the NTLM messages
sent by ZAP and other client (e.g. browser).

> Do you have a Windows 7 client that you could test with against a WS2012R2
> server?

I have a virtual image, I can try that.

> Could it be the host OS that's causing the issue?

I wouldn't expect that to be the problem.

Best regards.

On 09/05/17 18:58, Stephen Heeps wrote:
> Hi,
>
> A follow up question I have to this is how do we pass a domain or do we
> need to pass it at all? When we use the user context, do we pass it
> domain\user or just user? I've been testing on both 2.5.0 as well as the
> latest dev version (using intellij to build/run it) and get the same
> results ending with a Failure of Authencation:
>
> | 2017-05-09 12:51:00,233 [Thread-49] INFO HttpMethodDirector - Failure
> authenticating with NTLM <any realm>@website:80
>
> Can ZAP successfully authenticate a web application that is hooked up to an
> Active Directory domain? If so, is there any documentation regarding this?
> The NTLM documentation seems a little lackluster imho.
>
> Do you have a Windows 7 client that you could test with against a WS2012R2
> server? Could it be the host OS that's causing the issue?
>
> As a side note, I appreciate everything you've been doing to repro this
> issue with us @thc!
>
> Best Regards.
>
> On Tuesday, May 9, 2017 at 10:02:24 AM UTC-4, Stephen Heeps wrote:
>>
>> Hi,
>>
>> I'd love to give you more information. I'm not entirely sure how relevant
>> most of the following information will be:
>>

>> - OS: WS2012R2
>> - IIS: 8.5
>> - AppPools
>> - We created a separate one for our application
>> - .NET CLR version: .NET CLR VERSION v4.0.30319
>> - Managed Pipeline mode: Classic
>> - Identity: NetworkService
>> - Enable 32-Bit Applications: True
>> - IIS Site
>> - We created a new one for our application
>> - ASP.NET Impersonation: Enabled
>> - Forms Authentication: Enabled
>> - Response Type: HTTP 302 Login/Redirect
>> - Windows Authentication: Enabled
>> - Response Type: HTTP 401 Challenge
>> - Site is using Pass-through Authentication

[Stephen Heeps]

Hi,

Thank you very much for the answers!  I have a dev environment currently configured so I'll take your NTLM changes and see where it takes me.  If there are any changes that you think might be of assistance with this issue, let me know and I can apply the changes.  Since I am able to repro the issue consistency, I thought it would be useful to get a dev environment set up and try debugging on my side :)

I do feel like we're not sending the domain, which could be our main issue.  I have two accounts that can access the web application we have set up and both are giving my a failure in authentication with the domain (improper way) and without (proper way).  I can see in the logs the authentication attempting and eventually failing, so I can see that some credentials are being passed.  I thought it shouldn't matter with my machine being on the domain and with ZAP being just a middle man regarding passing credentials.  I thought I shouldn't have to worry about the domain, but maybe there's some issue where ZAP is just trying to pass <credentials> with no concept of domain and the application just says "If you're giving me <credentials>, that's wrong.  I need <domain>\<credentials> in order to authenticate you properly".  Even with I.E. being able to pass wincreds built-in, I still have issues as we discussed.

Correct me if I'm wrong in any of my statements.  I am still rather new to ZAP.

Best Regards.

[Stephen Heeps]

Hi,

@THC, I have created a WS2012R2 VM that contains the following:

• IIS 8.5
• Active Directory installed and configured properly with users
• ASP.NET application with integration authentication (works with only Administrator, other users get 401 and this is easily configurable)
• ZAP 2.6.0
• ZAP Source code with Intellij for debugging + Git  (if you want eclipse or something else, let me know so I can bundle that up tonight)
• ZAP scan was against IE11, but Firefox is installed if you rather that.

I am able to reproduce the issue on that VM, so hopefully you can take a look at it and see if it's a configuration issue or something else.  I will persist a session so it will be easier for you to jump into it.  Once I get home after work, I will create a link and publish this VM for you.  I will also attach all credentials and anything you need on it.  I'm not sure what timezone you're in (I'm UTC -5), but we can jump on discord if you needed to discuss anything with me.  Hopefully, this will give us the momentum to find out what this issue is.

Best Regards.

[thc...@gmail.com]

That sounds great! UTC here but I'm available anyway.

Looking forward to check the VM. Thank you!

Best regards.

On 16/05/17 14:19, Stephen Heeps wrote:
> Hi,
>
> @THC, I have created a WS2012R2 VM that contains the following:
>
>

> - IIS 8.5
> - Active Directory installed and configured properly with users
> - ASP.NET application with integration authentication (works with only

> Administrator, other users get 401 and this is easily configurable)

> - ZAP 2.6.0
> - ZAP Source code with Intellij for debugging + Git (if you want

> eclipse or something else, let me know so I can bundle that up tonight)

> - ZAP scan was against IE11, but Firefox is installed if you rather that.

>
>
> I am able to reproduce the issue on that VM, so hopefully you can take a
> look at it and see if it's a configuration issue or something else. I will
> persist a session so it will be easier for you to jump into it. Once I get
> home after work, I will create a link and publish this VM for you. I will
> also attach all credentials and anything you need on it. I'm not sure what
> timezone you're in (I'm UTC -5), but we can jump on discord if you needed
> to discuss anything with me. Hopefully, this will give us the momentum to
> find out what this issue is.
>
> Best Regards.
>
> On Tuesday, May 9, 2017 at 3:09:24 PM UTC-4, Stephen Heeps wrote:
>>
>> Hi,
>>
>> Thank you very much for the answers! I have a dev environment currently
>> configured so I'll take your NTLM changes and see where it takes me. If
>> there are any changes that you think might be of assistance with this
>> issue, let me know and I can apply the changes. Since I am able to repro
>> the issue consistency, I thought it would be useful to get a dev
>> environment set up and try debugging on my side :)
>>
>> I do feel like we're not sending the domain, which could be our main
>> issue. I have two accounts that can access the web application we have set
>> up and both are giving my a failure in authentication with the domain
>> (improper way) and without (proper way). I can see in the logs the
>> authentication attempting and eventually failing, so I can see that some

>> credentials are being passed. I *thought* it shouldn't matter with my

>> machine being on the domain and with ZAP being just a middle man regarding

>> passing credentials. I *thought* I shouldn't have to worry about the

[Stephen Heeps]

Hi,

Please find the VM in the google link below:

https://drive.google.com/open?id=0B0UQimNyHaPpWXl1SjBPTmwwRmc

It's a 7GB zip file that contains the VM in VDI format.  If there's anything else I can do, let me know!

Best Regards.

[Stephen Heeps]

Hi,

Apologies.  Creds are as follows:

Administrator/Start123!

Best Regards.

[thc...@gmail.com]

Thank you! Will take a look at that along the day.

Best regards.

[Stephen Heeps]

Hi!

Just wanted to follow up on this issue.  Any progress, thc?  Is there anything I can do to help in resolving this?

BR

[thc...@gmail.com]

Hi.

No further progress since last time I checked the image you provided.
I'll tackle that this week...

Thanks!
Best regards.

[Stephen Heeps]

Hi,

No worries!  I just wanted to make sure this issue didn't lost in the deep blue sea :)

If there's anything I can do to assist, please let me know!

Best Regards

[thc...@gmail.com]

Hi.

FYI and for the record, the image you provided helped a lot, thank you! :)

So, in your set up the problem was caused by the wrong domain being set
by ZAP (the issue mentioned earlier where the hostname was being used as
realm/domain). Fixing that and the authentication is successful.
(Now it works when the realm/domain is left empty or when set the
correct one e.g. ZAPTEST)

There's another issue, that prevents the HTTP/NTLM authentication
configuration from being changed after its first use (which makes it a
lot harder to fix any issues! Any further change requires changing the
ZAP session). This one is not yet fixed though, I'll raised an issue to
track that.

Best regards.