NTLM authentication

[Abhinav Gogna]

I am using gatling for stress testing a web app that uses NTLM authentication. When i recorded the scripts I got a NTLM authentication in the header something like the following

   authorizationHeader("""NTLM TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAGAbEdAAAADw==""")

I ran the same test script after few hours and got 401 error. I believe NTLM authentication expired. Is there a way I can handle this in gatling?
or
Is there a way I can generate this authentication and replace it in the test scripts?

Thanks,
Abhi

[Stéphane Landelle]

Hi Abhi,

Yes, there should be a way, as underlying AsyncHttpClient is supposed to support NTLM.

Honestly, I personally never had a chance to get my hands on such an application and test it.

In your HttpProtocol, you can pass a com.ning.http.client.Realm to the authRealm.

You can build a Realm from a RealmBuilder.

I can try and help you debug this, but you'd have to dig too.

Typical problem for open source projects with Microsoft enterprise only protocols. :( 

Would I be able to work on such a system, I'm pretty confident I could have it running in a matter of minutes-hours, but until then...

--
You received this message because you are subscribed to the Google Groups "Gatling User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gatling+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Abhinav Gogna]

Thank you Stephane! I will try to figure out how to integrate it. May need your help on it later.

[Abhinav Gogna]

I was able to get to past NTLM using following HttpClient class but can't figure out how to get the authorization header with ntlm token/String out of it. Since gatling does get info out when using recorder, can you point me into some direction as to what can be done.

Here is the code I used:

DefaultHttpClient httpclient = new DefaultHttpClient();
        List<String> authpref = new ArrayList<String>();
        authpref.add(AuthPolicy.NTLM);
        httpclient.getParams().setParameter(AuthPNames.TARGET_AUTH_PREF, authpref);
        NTCredentials creds = new NTCredentials("userid", "Password", "", "corp");
        httpclient.getCredentialsProvider().setCredentials(AuthScope.ANY, creds);

        HttpHost target = new HttpHost("work.org");

        HttpContext localContext = new BasicHttpContext();

        HttpGet httpget = new HttpGet("http://work.org/web");
        HttpResponse response = httpclient.execute(target, httpget, localContext);
        int statusCode = response.getStatusLine().getStatusCode();
        System.out.println("Status Code:" + statusCode);

[Stéphane Landelle]

No, you're confusing Apache HttpComponent and AsyncHttpClient.

Gatling is built on top of the latter.

AsyncHttpClient has NTLM support, though I never used it myself.

Check out Realm and RealmBuilder links from my previous message.

[Abhinav Gogna]

Ok Stephane. So I did some digging and came up with the following code. Current problem is I don't know how to pass the realm back into the httpProtocol. When I append it to httpProtocol variable, I get the following compiler error "value realm is not a member of io.gatling.http.config.HttpProtocolBuilder"
Here is the code:

val builder = new AsyncHttpClientConfig.Builder();

    val realm = new Realm.RealmBuilder().setUsePreemptiveAuth(true)
                                            .setPrincipal("gognaab")
                                            .setPassword("password")
                                            .setNtlmDomain("corp")
                                            .setNtlmHost("http://work.org")
                                            .setScheme(AuthScheme.NTLM).build()

       builder.setRealm(realm).build();

val httpProtocol = http
        .baseURL("https://esp.qa.finra.org")
        .inferHtmlResources()
        .acceptHeader("""*/*""")
        .acceptEncodingHeader("""gzip, deflate""")
        .acceptLanguageHeader("""en-US,en;q=0.5""")
        .authorizationHeader("""fasfsafksjkl2342xxx""")//changed
        .connection("""keep-alive""")
        .userAgentHeader("""Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0""")
        .realm(realm)

    val scn = scenario("SearchSimulation")
        // Search
        .exec(http("request_90")
            .get("""/test/Search?q=test&rpp=20""")
            .check(status.is(200)))

    setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)

What am I doing wrong here?

Thanks,
Abhinav

[Stéphane Landelle]

Method is named authRealm, not realm:

val realm = new Realm.RealmBuilder().setUsePreemptiveAuth(true)

.setPrincipal("gognaab")

.setPassword("password")

.setNtlmDomain("corp")

.setNtlmHost("http://work.org")

.setScheme(AuthScheme.NTLM).build

val httpProtocol = http

.baseURL("https://esp.qa.finra.org")

.inferHtmlResources()

.acceptHeader("""*/*""")

.acceptEncodingHeader("""gzip, deflate""")

.acceptLanguageHeader("""en-US,en;q=0.5""")

.connection("""keep-alive""")

.userAgentHeader("""Mozilla/5.0 (Windows NT 6.1; WOW64; rv:32.0) Gecko/20100101 Firefox/32.0""")

.authRealm(realm)

Also:

• don't set the authorizationHeader, it will be generated by the Realm
• don't try to build an AHC config, Gatling will build one underneath.

[Abhinav Gogna]

Thanks. Now I am not getting 401 error but 400.

Problem is authRealm sets a header Realm, which for some reason creates an invalid header name. Is there a way I can remove this from request header?

Here is the realm header info:
realm=Realm{principal='yyy', password='xxx', scheme=NTLM, realmName=
'', nonce='', algorithm='MD5', response='', qop='auth', nc='00000001', cnonce=''
, uri='null', methodName='GET', useAbsoluteURI='true', omitQuery='false'}           

[Stéphane Landelle]

Sorry, which header name/value do you get exactly?

[Abhinav Gogna]

The following string was in the request header:

realm=Realm{principal='yyy', password='xxx', scheme=NTLM, realmName='', nonce='', algorithm='MD5', response='', qop='auth', nc='00000001', cnonce='', uri='null', methodName='GET', useAbsoluteURI='true', omitQuery='false'}

[Stéphane Landelle]

No, that's just some logging, not the header that was sent.

Could you upgrade to RC6 (release today), please?

You should see the exact headers that were sent: https://github.com/gatling/gatling/issues/2131

[Abhinav Gogna]

From RC6, I get the following log info about http request.

HTTP request:
GET http://xxx.org/xxx/xxx?q=test&rpp=20
headers=
Authorization: NTLM TlRMTVNTUAABAAAANQIIIBAAEAAmAAAABgAGACAAAABlAHMAcABOAEEAUwBE
AEMATwBSAFAA
Connection: keep-alive
Host: xxxxxxxx
Authorization: NTLM TlRMTVNTUAABAAAANQIIIBoAGgAmAAAABgAGACAAAABlAHMAcABOAFQATABN
ACAATgBBAFMARABDAE8AUgBQAA==
Accept: */*
realm=Realm{principal='yyy', password='xxxx', scheme=NTLM, realmName=

'', nonce='', algorithm='MD5', response='', qop='auth', nc='00000001', cnonce=''
, uri='null', methodName='GET', useAbsoluteURI='true', omitQuery='false'}

[Stéphane Landelle]

How come is there 2 Authorization headers?

Did you forgot to remove authorizationHeader?

[Abhinav Gogna]

I didn't add the extra authorization header. Here is the code that I have now

val httpProtocol = http
        .baseURL("xxx")
        .authRealm(realm)

val scn = scenario("SearchSimulation")
        // Search

        .exec(http("test")
            .get("""espweb/""")
            .check(status.is(200)))

[Abhinav Gogna]

So I compiled AsyncHttp client code separately to try to pinpoint the issue. It looks like Netty jar creates the problem. As soon as I remove it, I get Response code 200. Is there a workaround to this?

Here is the code

def main(args:Array[String])
    {

        val builder = new AsyncHttpClientConfig.Builder();
        val realm = new Realm.RealmBuilder()

            .setPrincipal("xxx")
            .setPassword("xxx")//put password here
            .setUsePreemptiveAuth(true)
            .setScheme(AuthScheme.NTLM)
            .build();
       
       
        builder.setRealm(realm).build()
       
        val client = new AsyncHttpClient(builder.build())
        println("before response")
       
        val response = client.prepareGet("yyy")
                            .execute().get();
       
        println(response.getStatusCode());

[Abhinav Gogna]

Another update. Looks like Netty 4.0.23 jar works fine with standalone code but execution breaks Gatling since there are some dependencies( I got logger error, don't of any other). If we just copy over those dependencies to netty 4.jar, we should be all set.

Thanks,
Abhinav

[Stéphane Landelle]

I've found 2 issues in the Netty provider code:

1. the Type1 NTLM Authentication header is still there when sending the Type3 header
2. preemptive NTLM handshake should only happen when connecting, not for every request

Then, I'm very surprised that you got something running without setting the NTLM domain and host.

Are you sure you got the expected page, even though you got a 200 status?

[Abhinav Gogna]

You are right. The code just checks response code and I assumed when its 200 then response should be ok. Looks like its not.

Its a bummer netty breaks this stuff. Any chance we can workaround it?

[Stéphane Landelle]

Actually, my second point was invalid.

The first one was, and I fixed it (but you weren't impacted as you didn't make it this far).

You can upgrade your AsyncHttpClient source code, but make sure you build the 1.9.x branch, not master!

Then, IMHO, the first thing to fix is on YOUR side: NTLM domain and host (I think it's the workstation) are/should be mandatory.

val realm = new Realm.RealmBuilder()
            .setPrincipal("xxx")
            .setPassword("xxx")//put password here
            .setUsePreemptiveAuth(true)
            .setScheme(AuthScheme.NTLM)

            .setNtlmDomain("YOUR_NTLM_DOMAIN_HERE")

            .setNtlmHost("YOUR_NTLM_HOST_HERE")
            .build();

[Abhinav Gogna]

I downloaded and compiled asyncHttp client from https://github.com/AsyncHttpClient/async-http-client/tree/1.9.x (branch). It still has the same problem (double authentication). As soon as I remove netty jar it works fine. Also, I added back ntlmDomain and ntlmHost in realm builder and I got response body back with 200.

So far I have only seen logger error during gatling run with netty 4. I will try to compile netty4 with netty 3 logging to see if it works.

[Abhinav Gogna]

Netty 4 and 3 don't look compatible. Not sure what else to do.

[Excilys]

I'll try to mock an NTLM auth and reproduce the header duplicates you observe.

I will ping you on Thursday.

[Stéphane Landelle]

I've fixed quite a few things: https://github.com/AsyncHttpClient/async-http-client/issues/730

At least I think I have, as I don't have a real NTLM system...

Could you give it a try please?

[Abhinav Gogna]

Now I get unauthorized access 401 error (checked userid/pwd at least 5 times).

I ran this without gatling libs but with netty 3.9.2. final sources.jar and new async client and it worked perfectly.

Here is the request from logs:

HTTP request:
GET http://xxxx
headers=
Authorization: NTLM TlRMTVNTUAADAAAAGAAYAEgAAADmAOYAYAAAABAAEABGAQAADgAOAFYBAAAG
AAYAZAEAABAAEABqAQAAFbKI4gUBKAoAAAAPq3GWgRCEBNkUc//IJ5ljiXHeDxGScgAUyqugKCyPHBQ2
hJkLGdupxAEBAAAAAAAAEMhAzc7dzwFx3g8RknIAFAAAAAACAAgAQwBPAFIAUAABABoASwBXAEEAVwBO
AEEAUABQAFQAUQAwADEAMQAEABwAYwBvAHIAcAAuAGYAaQBuAHIAYQAuAG8AcgBnAAMAOABLAFcAQQBX
AE4AQQBQAFAAVABRADAAMQAxAC4AYwBvAHIAcAAuAGYAaQBuAHIAYQAuAG8AcgBnAAUAHABjAG8AcgBw
AC4AZgBpAG4AcgBhAC4AbwByAGcABwAIADe3xczO3c8BAAAAAAAAAABOAEEAUwBEAEMATwBSAFAAZwBv
AGcAbgBhAGEAYgBlAHMAcACJOk4t+BRukQwI8ffVqpBU
Connection: keep-alive
Host: xxxx
Accept: */*

On a side note - I really appreciate your help and support with gatling. I have been reading on its documentation and it seems very promising, especially scaling part. I am actually trying to do a proof of concept, so, we can adopt it in our organization as load testing tool. So please feel free to let me know if I can help you with debugging.

Abhinav

[Stéphane Landelle]

Sorry, I don't get it.

Now I get unauthorized access 401 error (checked userid/pwd at least 5 times). 

I ran this without gatling libs but with netty 3.9.2. final sources.jar and new async client and it worked perfectly.

When you got 401, was it with Gatling, or directly with AsyncHttpClient compiled from sources?

When trying with Gatling, did you replace the AsyncHttpClient jar with the one you compiled yourself (you have to remove the old one, otherwise it might be picked up instead of your new one)?

[Abhinav Gogna]

After compiling the new async client, I replaced the old async client jar with this one in gatling lib and got 401 error - unauthorized access.

I used it with standalone code, it worked fine. Got response body back.

Then I added netty 3.9.2 jar to standalone - I got the 401 error unauthorized access.

Then I removed the netty 3.9.2 jar and replaced it with netty 3.9.2 final sources jar - it worked.

[Abhinav Gogna]

When I said it worked I meant on standalone code. It didn't work with gatling libs.

"Then I removed the netty 3.9.2 jar and replaced it with netty 3.9.2 final sources jar - it worked."

[Stéphane Landelle]

OK, so this procedure is wrong.

If you remove the netty jar, you ended using a JDK based implementation which is just a toy, not intended for production.

A source jar is just a zip with the sources, dropping it instead of the binary jar is just the same as removing.

As I don't have a NTLM env, what I really need is:

• the information pour pass to the Realm: login/password + NTLM domain and host if you set them
  
• the expected NTLM handshake = the WWW-Authenticate and Authentication headers chain:
  
• client sends bare request
• server answers WWW-Authenticate: NTLM
• client sends Type1 message: Authentication: NTLM something1
• server replies with Type2 message: WWW-Authenticate: NTLM something2
• clients then sends Type3 message: Authentication: NTLM something3
• server replies 200 OK with expected resource

Without this, I'm just shooting in the dark.

[Abhinav Gogna]

When I use the recorder, initially, I did get all client response with ntlm authentication headers. Will that be enough. I can do another record and send you the code.

If not, what would be the best way to get those headers chain?

[Stéphane Landelle]

With the sole recorder, you'll only get the requests, but not the responses.

What you can do however is use the recorder and enable DEBUG logging (in logback.xml) so you'll get the full requests and responses in your console.

[Abhinav Gogna]

Ok, as soon as I get to work, I will record it with debug and send it you.

[Stéphane Landelle]

Would be great, thanks a lot!

[Abhinav Gogna]

One more thing. Can I send the console output to log file. Recorder creates lot of data, it will be easy to just capture it in a log file.

[Stéphane Landelle]

Yes, you have to replace the ConsoleAppender by a FileAppender in logback.xml.

http://logback.qos.ch/manual/appenders.html

[Abhinav Gogna]

I have attached the recorder logs and run logs. Let me know if you need anything else.

[Abhinav Gogna]

I have attached the recorder logs and run logs. Let me know if you need anything else.

[Abhinav Gogna]

Adding my modified script and server logs.

<b

...

[Stéphane Landelle]

Sorry, that's not sufficient :(

In your logs, there seems to be only the Type3 message, and it resulted in a 401, not 200.

And you didn't provide your credentials, domain and host.

I understand that those information are private, but I can't really do anything without those.

I fear we've reached a dead end.

[Abhinav Gogna]

The problem that you don't have a way to test your code. What if we get a environment. Microsoft Azure provides iis and AD services. Will that help?

[Stéphane Landelle]

I guess so, yes. If you can help with the Windows set up too.

2014-10-03 13:39 GMT+02:00 Abhinav Gogna <abhi...@gmail.com>:

The problem that you don't have a way to test your code. What if we get a environment. Microsoft Azure provides iis and AD services. Will that help?

[Abhinav Gogna]

Great. I will do the best I can for windows setup. I will respond back with user information to azure account.

[Stéphane Landelle]

Great! Thanks a lot for your help with this.

2014-10-03 13:56 GMT+02:00 Abhinav Gogna <abhi...@gmail.com>:

Great. I will do the best I can for windows setup. I will respond back with user information to azure account.

[Abhinav Gogna]

I have finally figured out creating VM with AD and IIS on Azure. Anyway will send connection description in a private message.

This is what I have so far:
1) Domain Controller with AD and IIS
2) New AD forest
3) Windows Authentication enabled (NTLM)

What else is needed
1) If you need then a web app with ntlm challenge
2) Import Server certificate to the webapp to start using NTLM handshake

I used the following documentation to generate client certificates (c:/client.cert). For server certificates I am somewhat lost.
https://confluence.atlassian.com/display/CROWD/Configuring+an+SSL+Certificate+for+Microsoft+Active+Directory

It took me a while to figure out setting up windows server since it was my first time setting up a windows server. I hope it works! Let me know.

Thanks,
Abhinav

[Stéphane Landelle]

Hi Abhinav,

I'm able to connect, tahnks!

Then, yes, could you figure out how to set up a webapp with NTLM challenge, please?

I'll be out of office today, so I'll only be able to work on this tomorrow.

Cheers,

Stéphane

[Abhinav Gogna]

I have setup an app on the server. <authentication = "windows"> in web.config is all its needed for it.

You can access it by http://localhost:8080.

Thanks,
Abhinav

[Stéphane Landelle]

The issue is probably that you're not setting the NTLM domain and host. Those are actually mandatory: https://github.com/gatling/gatling/issues/2297

[Abhinav Gogna]

For anyone who is interested in NTLM, Stephane has resolved the issue in Async Http Client. It now works. So far, at least !

[Wicem Kerkeni]

Hello, I'am interested in NTML authentification. So what is the final solution to work?

[Stéphane LANDELLE]

http://gatling.io/docs/2.1.6/http/http_protocol.html#authentication

http://gatling.io/docs/2.1.6/http/http_request.html#authentication

Stéphane Landelle

Lead developer

slan...@gatling.io

On Wed, May 13, 2015 at 3:36 PM, Wicem Kerkeni <kerk...@gmail.com> wrote:

Hello, I'am interested in NTML authentification. So what is the final solution to work?

--

[hakan turğay]

Hi everyone .. final solution ?????

25 Eylül 2014 Perşembe 01:02:56 UTC+3 tarihinde Abhinav Gogna yazdı:

[hakan turğay]

Please somebody help me Please .

I try to solve this problem 2 weeks but i can't.

[Stéphane LANDELLE]

https://github.com/gatling/gatling/issues/3525

Stéphane Landelle

GatlingCorp CTO

slan...@gatling.io

--