IAS suddenly stopped working for some time and then magically worked again
Rainer Sinsch
I have a problem with IAS that gives me headaches and I'd like to get some
opinions:
I'm running a large enterprise wireless lan using a lot of distributed
Microsoft IAS RADIUS servers (w2k3) with EAP-TLS and PEAP authentication.
The setup works fine for many years now, but somehow I experienced a strange
dropout last week:
Microsoft IAS suddenly stopped processing authentication requests on last
sunday between 1 am and 4 pm. I have to point out that we have a test
authentication setup running which sends authentication requests every
minute to check the radius service availability and I can see IAS logs of
that before 1 am and after 4 pm, but not in between. And there is nothing of
interest as far as IAS or other services are concerned. Of course users
reported problems, so the authentication was really down. The interesting
thing is that this happened on many locations on different authentication
servers at the same time.
What could be the reason for this?
Here are some facts:
- Normally IAS runs fine (no problems at all)
- The service was not stopped or restarted
- SQL server logging is active, the SQL server was available and I can see
no logs that SQL logging was unsuccessful
- I can see no IAS authentication reports in the System event log at the
specific time
- I can see sporadic authentication reports in the Security event log (but
only sporadic)
- The error occurs on many IAS servers spread over various locations
And to be more specific:What kind of dependencies does IAS have?
- Obviously Active Directory: So what happens if there are problems with AD,
could this have such an impact?
- The certificate authority: I'm pretty sure that a connection loss to the
CA does not interfere with the authentication process, but can someone
confirm this?
- Any other dependencies?
Thanks in advance
Rainer
James McIllece [MS]
Hi there --
I asked a team member in PSS about this and here is their response:
"Based on experience, this is probably a network outage of some sort. IAS
is not aware of other IAS servers so there is no coordinated event within
IAS that would knock out all servers at the same time. Further, IAS is
only dependant on the RPC service on the local box.
If the IAS service were still running and, for example, AD was offline,
IAS would still note that it had received requests, but could not service
them. There would be a similar event for CRL checking should the CA go
offline. For there to be no IAS events for 3 hours indicates that none of
the IAS servers were receiving requests. I would look at the wireless
infrastructure for clues as to what the issue may have been, but I highly
doubt it was the IAS servers themselves.
"
--
James McIllece, Microsoft
Please do not send email directly to this alias. This is my online account
name for newsgroup participation only.
This posting is provided "AS IS" with no warranties, and confers no rights.
Rainer Sinsch
first of all I'd like to thank you for your highly valued input.
> "Based on experience, this is probably a network outage of some sort.
It was definately not a network outage, because
a) it was on different locations and
b) I'm part of the network team responsible for these locations and we did
not suffer any other problems at that time
After I have now double-checked all configurations and participating
services I found one clue: All servers that did not work made use of SQL
server logging. Interestingly, I forgot to activate the logging on one
single machine and this was the only machine that did not experience any
outage! This leads me to the conclusion that we had some kind of deadlock
situation between our SQL logging server and our IAS machines that made the
whole IAS service hang.
I know that the current IAS behaviour is that authentications will fail if
the SQL logging server has problems (although you can see that in the logs,
which was not the case during our problems). I have heard that it is not
possible to configure this behaviour so I'd like to ask again, why is this
so and are there any thoughts to change this in the future? Things of such
importance should really be configurable.
> For there to be no IAS events for 3 hours indicates that none of
> the IAS servers were receiving requests.
They were definately receiving requests by
a) our monitoring system
b) our colleagues with wireless handhelds calling for help
> I would look at the wireless
> infrastructure for clues as to what the issue may have been, but I highly
> doubt it was the IAS servers themselves.
The wireless infrastructure sent a syslog message of "dead radius servers".
As stated above in my eyes this was an internal bug in the sql logging
procedure. Three servers not working with SQL logging and one server working
without SQL logging is too much of a coincidence.
Thanks again.
/rainer
James McIllece [MS]
Hi Rainier --
I agree that the SQL outage caused the problem. I have never discussed with
the devs why IAS is configured to stop operation if logging fails, but I
assume that it is a security precaution.
I believe that you can create a failover of sorts by also enabling logging
to a local file, so that IAS is logging both to a file AND to the SQL
server; that way if SQL fails, IAS can still log to file and IAS continues
to work.
Of course this might not be practical in all situations, but I thought I
would mention it as it is an option.
Rainer Sinsch
> I agree that the SQL outage caused the problem. I have never discussed
> with
> the devs why IAS is configured to stop operation if logging fails, but I
> assume that it is a security precaution.
Do me a favour and tell the dev-guys they should make this "feature"
configurable. There are users out there who use sql for logging purposes
only. And logging is not as important as application availability. I can not
argue with our colleages from the shop floor that their barcode scanners do
not work just because our logging is out of order.
> I believe that you can create a failover of sorts by also enabling logging
> to a local file, so that IAS is logging both to a file AND to the SQL
> server; that way if SQL fails, IAS can still log to file and IAS continues
> to work.
Unfortunately this does not work. If SQL fails the whole authentication
process fails. I know that one can argue with security precautions - but it
is ridiculous that this one is not configurable.
Thanks anyway.
Regards
Rainer
James McIllece [MS]
> If SQL fails the whole authentication
> process fails. I know that one can argue with security precautions -
> but it is ridiculous that this one is not configurable.
Hi Rainer --
Thanks for providing this feedback, I have forwarded your feature request
to the devs and other team members. Just thought I would let you know.
James McIllece [MS]
news:Xns994C8ADFC82F3ja...@207.46.248.16:
> "Rainer Sinsch" <n...@spam.de> wrote in news:f4jqbd$20p$1...@mail1.sbs.de:
>
>> If SQL fails the whole authentication
>> process fails. I know that one can argue with security precautions -
>> but it is ridiculous that this one is not configurable.
>
> Hi Rainer --
>
> Thanks for providing this feedback, I have forwarded your feature
> request to the devs and other team members. Just thought I would let
> you know.
>
Hi again --
The product team would like to discuss your idea and get additional
feedback from you, if possible. Can you please email me at wsdocs@no-
spam.microsoft.com and I will send your email address to the product
manager for IAS (who requested it)? Thanks much Rainer.
Rainer Sinsch
thanks for forwarding the feature request! I have written an email to your
provided email-address (removed the no-spam) with all my detailed contact
information.
/Rainer
"James McIllece [MS]" <jame...@online.microsoft.com> schrieb im Newsbeitrag
news:Xns994C8CFD113F6ja...@207.46.248.16...
Brocker@discussions.microsoft.com Brian Brocker
I would like to add that I've noted the same behavour as Rainer - related to
the failing of authentication if the SQL transaction fails. I can understand
the reasoning from a security standpoint in a highly secure environment.
You've indicated that if both text logging and sql logging are enabled, then
authentication will not fail upon loss of sql logging alone. Other MS
documents also support this statement. However, from what I can tell, a loss
of sql logging will cause a failure in authentication - regardless if text
logging is enabled or not. I'm just in the process of setting our IAS up
(still testing) and don't have any previous experience with it, so it may be
possible that I have something configured incorrectly. I found this thread
after trying to test the very feature (using both sql and text logging - then
'breaking it's sql logging capability). It appears that a successful
authentication is logged in the text file even with the sql log failure -
however the connection actually fails authentication. The IAS Server System
Log also indicates authentication failure in this scenario for me.
Like Rainer, I would also like to see the SQL Logging be configurable - with
a choice to allow authentication even if the logging failed. Additionally,
I'm not sure if there is a documentation error or I have a configuration
issue related to the above authentication failure even with the dual logging
configuation.
Thanks,
Brian Brocker
James McIllece [MS]
Thanks very much for this information, I have forwarded your comments to
the product team as well. When I have time I will investigate this further
with an eye to updating the documentation if I can reproduce the behavior
you have experienced.
Again I really appreciate your feedback.
James
=?Utf-8?B?QnJpYW4gQnJvY2tlcg==?= <Brian
Bro...@discussions.microsoft.com> wrote in
news:59FE26A9-51BE-4BA6...@microsoft.com:
James McIllece [MS]
"Rainer Sinsch" <n...@spam.de> wrote in news:f6t3km$a1m$1...@mail1.sbs.de: