SPNEGO NTLM / Kerberos over HTTP (aka RFC4559) confusion
jo...@feith.com
I wasn't quite sure of the best group for a HTTP protocol question.
We have a web application which needs to log into a backend data
service as the user who accessed our application. Now the standards
compliant way to accomplish this would seem to be to return a 401
WWW-Authenticate: Negotiate which should result in the browser using
SPNEGO to send the server a list of the supported authentication
mechanisms along with the optimistic mechanism token (as per RFC4559
and RFC4178). As long as one of the listed mechanisms is Kerberos
we should be able to use SPNEGO to get the Kerberos token which will
allow us to log into our data service.
This appears to work for the most part (tested with Firefox and IE),
however in some cases our server receives a NTLM token instead of a
SPNEGO in response to the 401. This problem occurs with both Firefox
and IE. A packet trace shows Negotiate was the only WWW-Authenticate
header we supplied when we responded. The configuration is correct
and Kerberos credentials are available. Microsoft has looked into
the issue and has stated that the Microsoft SSPI correctly implements
RFC4559. Specifically they've stated that in some cases the SSPI will
return a NTLM token instead of a SPNEGO token causing the browser to
return Authorization: Negotiate with a non-SPNEGO token and that this
behavior complies with RFC4559.
Now my understanding of RFC4559 is that SPNEGO must always be used by
the client when responding. The NTLM is allowed to be * inside * of
the
SPNEGO.
Anyone care to clear up the confusion, or suggest how to go about
resolving
the issue?
-- John
-------------------------------------------------------------------------
| Feith Systems | Voice: 1-215-646-8000 | Email:
jo...@feith.com |
| John Wehle | Fax: 1-215-540-5495
| |
-------------------------------------------------------------------------
Michael B Allen
Hi John,
The problem is that the client will not or cannot initiate Kerberos.
There could be many reasons for this:
1) The client cannot obtain a Kerberos ticket for the desired service.
There can be many reasons for this too:
1a) If a MS client is not "joined" to a domain it cannot participate
in Kebreros authentication and thus the client has no choice but to
fall-back to using NTLM.
1b) A temporary network failure has prevented the client from
acquiring the necessary ticket and thus the client has no choice but
to fall-back to using NTLM.
1c) The XP credential cache bug [1] is preventing the client from
renewing tickets causing the client to fall-back to NTLM.
2) The browser must be configured to trust the target server. In IE
you must add the target server or it's domain under Tools > Internet
Options > Security > IntrAnet Zone.
3) In some browsers automatic authentication must be enabled. For
example, in IE you must turn on Tools > Internet Options > Advanced >
Enabled Integrated Windows Authentication.
4) The user entered and saved credentials into a Network Password
Dialog which preempts the client to initiate NTLM rather than
Kerberos.
5) There is a problem with the service principal name on the service
account preventing the client from initiating Kerbeors authentication
and thus the client has no choice but to fall-back to using NTLM.
I recommend using wfetch.exe (available on MS's website) to narrow
down where the problem is. You can also look at our product manual at
http://www.ioplex.com/support.html. In particular see "Issue 3" and
"Issue 5" in the "Possible Issues" section. That document details all
of the conditions described here and much more. Even though our code
is completely different from what you're using the condition is the
same (described as the "GSS_S_BAD_MECH" error in our document).
Mike
PS: Also note that the HTTP service account must be set to permit
delegation. Otherwise, even if you get the client to do Kerberos you
won't get the delegated ticket necessary to impersonate the user when
initiating auth with the next tier.
[1] http://support.microsoft.com/kb/885887
http://support.microsoft.com/kb/906524/en-us
--
Michael B Allen
PHP Active Directory SPNEGO SSO
http://www.ioplex.com/
jo...@feith.com
> The problem is that the client will not or cannot initiate Kerberos.
Nice try, however no. The client has no problems using Kerberos.
There are credentials in the cache for user. There's no problem
fetching credentials for the webserver. The problem has specifically
been traced by Microsoft to a bit of code in the Negotiate SSPI
which causes a raw NTLM to be returned instead of a SPNEGO
in some situations even though Kerberos is available / working.
The issue is whether RFC4559 allows a raw NTLM to be returned.
My read of the RFC is SPNEGO is always required ... so let's
say that there was something interfering with Kerberos and
Windows dropped back to NTLM. What it should send is a
SPNEGO encapsulating a NTLM. What it actually sent was
a raw NTLM.
Microsoft's take is that a raw NTLM is a completely compliant
RFC4559 response.
I'm looking for someone clarifying the issue or suggest how
to resolve the issue.
-- John
jo...@feith.com
Michael B Allen
> > The problem is that the client will not or cannot initiate Kerberos.
>
>
> Nice try, however no. The client has no problems using Kerberos.
> There are credentials in the cache for user. There's no problem
> fetching credentials for the webserver. The problem has specifically
> been traced by Microsoft to a bit of code in the Negotiate SSPI
> which causes a raw NTLM to be returned instead of a SPNEGO
> in some situations even though Kerberos is available / working.
If the HTTP server returns "WWW-Authenticate: NTLM" then the client
must use NTLMSSP tokens. If it returns "WWW-Authenticate: Negotiate"
then the tokens must be SPNEGO. If it returns both, then the client
can pick.
Otherwise, you need to explain the point of failure in more detail. Is
it gss_accept_sec_context that is returning a token you didn't expect,
or InitializeSecurityContext, or what? If you're not sure then provide
an HTTP client / server call sequence w/ headers that illustrates the
point of failure.
Mike
jo...@feith.com
> If the HTTP server returns "WWW-Authenticate: NTLM" then the client
> must use NTLMSSP tokens. If it returns "WWW-Authenticate: Negotiate"
> then the tokens must be SPNEGO. If it returns both, then the client
> can pick.
Yep ... that's pretty much how I understand things. In our case we
are
only returning "WWW-Authenticate: Negotiate".
> Otherwise, you need to explain the point of failure in more detail.
The IE client is responding to "WWW-Authenticate: Negotiate" with
a raw NTLM instead of SPNEGO.
> If you're not sure then provide an HTTP client / server call sequence
We're sure ... it's all been check / doubled checked using pack
sniffers,
etc. Microsoft has also confirmed it and looked at their code. They
say that it's intentional to return a raw NTLM instead of SPNEGO
regardless of the availability of Kerberos in some situations when
responding to "WWW-Authenticate: Negotiate".
> the point of failure.
The real problem is that Microsoft admits that this is intentionally
and claims that it is RFC4559 compliant. I'm having great difficulty
in getting them to understand that RFC4559 * requires * that SPNEGO
be used. I'm open to suggestions.
-- John
jo...@feith.com
Michael B Allen
> On Mar 18, 12:59 am, "Michael B Allen" <iop...@gmail.com> wrote:
> > If the HTTP server returns "WWW-Authenticate: NTLM" then the client
> > must use NTLMSSP tokens. If it returns "WWW-Authenticate: Negotiate"
> > then the tokens must be SPNEGO. If it returns both, then the client
> > can pick.
>
>
> Yep ... that's pretty much how I understand things. In our case we
> are
> only returning "WWW-Authenticate: Negotiate".
>
>
> > Otherwise, you need to explain the point of failure in more detail.
>
>
> The IE client is responding to "WWW-Authenticate: Negotiate" with
> a raw NTLM instead of SPNEGO.
Well I just looked at some of my captures and I can see that raw
NTLMSSP tokens can be sent in response to "WWW-Authenticate:
Negotiate". And I now recall that raw Kerberos tokens can be used as
well.
> > If you're not sure then provide an HTTP client / server call sequence
>
>
> We're sure ... it's all been check / doubled checked using pack
> sniffers,
Well I'm glad you're sure but I'm not clear on what the point of
failure is so if you want my input you'll have to explain the exact
exchange. If you're seeing an XP client send a raw NTLM token in
response to sending it "WWW-Authenticate: Negotiate" then I refer you
to my original response.
> etc. Microsoft has also confirmed it and looked at their code. They
> say that it's intentional to return a raw NTLM instead of SPNEGO
> regardless of the availability of Kerberos in some situations when
> responding to "WWW-Authenticate: Negotiate".
>
> > the point of failure.
>
> The real problem is that Microsoft admits that this is intentionally
> and claims that it is RFC4559 compliant. I'm having great difficulty
> in getting them to understand that RFC4559 * requires * that SPNEGO
> be used. I'm open to suggestions.
>From glancing at RFC 4559 it indeed does not seem to include any
mention that raw NTLM and Kerberos tokens are accepted. But I learned
a long time ago that RFCs are not the law. What you see on the wire is
the law.
Todd Stecher
I'm guessing that your workflow / product / code requires Kerberos (in
the absence of a good open source NTLM story), and you're trying to
figure out how to get SPNEGO wrapped kerberos tokens all of the time?
Its also extremely difficult to *always* get kerberos authentication
(wrapped or unwrapped).
For example, if the target HTTP server is in another forest, then
chances are the client will end up using NTLM due to SPN location
issues. Also, some IE policies will not even allow a kerberos or NTLM
authentication over the HTTP transport to various zones.
My reading of the RFC is that it is truly "informational," describing
how clients and servers use SPNEGO + HTTP, but not specifying every
possible HTTP auth scheme. Chances are the answer you got about raw
NTLM being "OK" was passed through various layers of Microsoft from
Larry Zhu, the author of the RFC itself, and based on not on
"correctness" but rather on the behavior of millions of deployed
clients and servers. Even if you could get MS to change the behavior
to your interpretation of the RFC, its not going to help much until
every machine out there is updated.
Todd
On Mar 18, 2008, at 8:14 AM, Michael B Allen wrote:
> On 3/18/08, jo...@feith.com <jo...@feith.com> wrote:
>> On Mar 18, 12:59 am, "Michael B Allen" <iop...@gmail.com> wrote:
>>> If the HTTP server returns "WWW-Authenticate: NTLM" then the client
>>> must use NTLMSSP tokens. If it returns "WWW-Authenticate: Negotiate"
>>> then the tokens must be SPNEGO. If it returns both, then the client
>>> can pick.
>>
>>
>> Yep ... that's pretty much how I understand things. In our case we
>> are
>> only returning "WWW-Authenticate: Negotiate".
>>
>>
>>> Otherwise, you need to explain the point of failure in more detail.
>>
>>
>> The IE client is responding to "WWW-Authenticate: Negotiate" with
>> a raw NTLM instead of SPNEGO.
>
> Well I just looked at some of my captures and I can see that raw
> NTLMSSP tokens can be sent in response to "WWW-Authenticate:
> Negotiate". And I now recall that raw Kerberos tokens can be used as
> well.
>
>>> If you're not sure then provide an HTTP client / server call
>>> sequence
>>
>>
>> We're sure ... it's all been check / doubled checked using pack
>> sniffers,
>
> Well I'm glad you're sure but I'm not clear on what the point of
> failure is so if you want my input you'll have to explain the exact
> exchange. If you're seeing an XP client send a raw NTLM token in
> response to sending it "WWW-Authenticate: Negotiate" then I refer you
> to my original response.
>
>> etc. Microsoft has also confirmed it and looked at their code. They
>> say that it's intentional to return a raw NTLM instead of SPNEGO
>> regardless of the availability of Kerberos in some situations when
>> responding to "WWW-Authenticate: Negotiate".
>>
>>> the point of failure.
>>
>> The real problem is that Microsoft admits that this is intentionally
>> and claims that it is RFC4559 compliant. I'm having great difficulty
>> in getting them to understand that RFC4559 * requires * that SPNEGO
>> be used. I'm open to suggestions.
>
>> From glancing at RFC 4559 it indeed does not seem to include any
> mention that raw NTLM and Kerberos tokens are accepted. But I learned
> a long time ago that RFCs are not the law. What you see on the wire is
> the law.
>
> Mike
>
> --
> Michael B Allen
> PHP Active Directory SPNEGO SSO
> http://www.ioplex.com/
> ________________________________________________
> Kerberos mailing list Kerb...@mit.edu
> https://mailman.mit.edu/mailman/listinfo/kerberos
>
Michael B Allen
> My reading of the RFC is that it is truly "informational," describing
> how clients and servers use SPNEGO + HTTP, but not specifying every
> possible HTTP auth scheme. Chances are the answer you got about raw
> NTLM being "OK" was passed through various layers of Microsoft from
> Larry Zhu, the author of the RFC itself, and based on not on
> "correctness" but rather on the behavior of millions of deployed
> clients and servers. Even if you could get MS to change the behavior
> to your interpretation of the RFC, its not going to help much until
> every machine out there is updated.
I would hope that they do NOT change the existing behavior. I consider
accepting "raw" NTLM and Kerberos tokens to be a feature. In fact,
SPNEGO is largely dead weight - I don't recall seeing it ever
"negotiate" much of anything. It's just one of those things that
sounded nice in theory but in practice it didn't really help anyone.
But MS clients send SPNEGO tokens so we need to accept them.
Note that accepting raw tokens is not terribly hard considering SPNEGO
is largely a wrapper for the raw tokens. It's an extra condition in
your code. Or just use a GSSAPI implementation that supports SPNEGO
and you're done.
jo...@feith.com
> I would hope that they do NOT change the existing behavior. I consider
> accepting "raw" NTLM and Kerberos tokens to be a feature.
I have no problems with them * accepting * "raw" NTLM and Kerberos
tokens.
I am merely talking about them * sending * "raw" tokens when RFC4559
calls
for SPNEGO.
> Note that accepting raw tokens is not terribly hard considering SPNEGO
> is largely a wrapper for the raw tokens.
Agreed and if I was getting a raw Kerberos token I'd probably just
deal.
However handling a NTLM token in place of a Kerberos token is a little
more complicated.
In our situation the Microsoft SSPI has decided that since there are
NTLM
credentials available due to an interactive logon to the same machine
that happens to run our application it's going to send the NTLM
credentials
instead of using the Kerberos credentials which are also available.
This
is due to special case code in the SSPI which prefers NTLM over
Kerberos
in this situation. Now if they actually implemented SPNEGO as
required by
the RFC we would be able to respond with accept_incomplete and request
that the Kerberos token be used.
-- John
jo...@feith.com
John Wehle
> I'm guessing that your workflow / product / code requires Kerberos
Yes.
> and you're trying to figure out how to get SPNEGO wrapped kerberos
> tokens all of the time?
That would be nice, though as you mention there are many things that
get in the way.
> Chances are the answer you got about raw NTLM being "OK" was passed
> through various layers of Microsoft from Larry Zhu, the author of
> the RFC itself, and based on not on "correctness" but rather on
> the behavior of millions of deployed clients and servers.
I'd be impressed if they actually checked with Larry Zhu. I do suspect
that the answer is, as you said, based on how their product has always
functioned rather than "correctness".
> Even if you could get MS to change the behavior to your interpretation
> of the RFC, its not going to help much until every machine out there
> is updated.
I don't need every machine ... just my customers' machines. Our release
notes can indicate what versions / hotfixes are required for proper
operation. My approach might be different if I was designing a SSO
toolkit to be included in other people's products.
Michael B Allen
> > is largely a wrapper for the raw tokens.
>
> NTLM
> credentials available due to an interactive logon to the same machine
> that happens to run our application it's going to send the NTLM
> credentials
> instead of using the Kerberos credentials which are also available.
> This
> is due to special case code in the SSPI which prefers NTLM over
> Kerberos
> in this situation.
That problem doesn't really have anything to do with SPNEGO. The SSPI
layer knows nothing about interactive logons. The problem is that some
application has acquired and inserted an NTLM credential into the
credential cache so naturally the InitializeSecurityContext function
as called by IE is going to pick that. That may not be optimal but it
really has nothing to do with SPNEGO. The behavior you want would
require that IE specify that it wants the SPNEGO mechanism and not the
NTLM mechanism (not sure if SSPI supports the specification of a
mechanism like GSSAPI does - it may simply infer the mechanism from
the credential).
> Now if they actually implemented SPNEGO as
> required by
> the RFC we would be able to respond with accept_incomplete and request
> that the Kerberos token be used.
I wouldn't be super confident that that would actually work. Again,
just because you see something in an RFC doesn't mean that it actually
works like that in practice.
Our product provides a logon routine that allows authenticating
clients using a traditional username+password method. That handles all
of the "client will not or cannot do kerberos" scenarios and not just
the one NTLM case.
Note that NTLM doesn't support delegation so if I remember your
original post correctly, implementing NTLM with pass-through
authentication would not help your particular scenario.
jo...@feith.com
> That problem doesn't really have anything to do with SPNEGO. The SSPI
> layer knows nothing about interactive logons. The problem is that some
> application has acquired and inserted an NTLM credential into the
> credential cache so naturally the InitializeSecurityContext function
> as called by IE is going to pick that. That may not be optimal but it
> really has nothing to do with SPNEGO. The behavior you want would
> require that IE specify that it wants the SPNEGO mechanism and not the
> NTLM mechanism (not sure if SSPI supports the specification of a
> mechanism like GSSAPI does - it may simply infer the mechanism from
> the credential).
All I can tell you is Microsoft who recreated this problem in their
lab and who looked at their code indicated that IE * is * asking for
SPNEGO and special case code in the SSPI choose to return
a NTLM token because of an interactive logon session. I don't
have access to their source code so it hard for me to comment
on whether that is in fact how their code works.
I will note that the problem also occurs using Firefox when configured
to use the Microsoft SSPI and I have looked at that code which does
seem to explicitly request the Negotiate SSPI package.
-- John
jo...@feith.com