Event Id 6005 Winlogon
Mireille Kreines
When new sessions are started, either via Microsoft RDP of Citrix ICA, they are disconnected within seconds. This applies to normal users and users with administrative privileges. This problem is caused by a chain of events. One components crash leads to an ungraceful shutdown of other components leaving a garbage configuration, preventing new connections.
Users connect to a Citrix XenApp server and are immediately disconnected. No sessions are visible on the server (no disconnected sessions either) and the same symptoms apply for sessions via Microsoft RDP. There are no events logged in the event log that indicate a potential problem.In some occasions the first RDP/ICA session is successful and subsequent sessions fail.
In the Application log we can see an event is raised by SceCli (Security Configuration Editor Client for Windows) with ID 1704 informing us that a new security policy is applied successfully.
When the Remote Desktop service restarts it unloads and reloads the module winsta.dll (the module that handles the WinStations), but reloads it incorrectly. As a result the Remote Desktop services will crash.
First an event is raised by Application Error with ID 1000 informing us that the application svchost.exe_TermService (the process of the Remote Desktop service) has an error. The faulting module is WINSTA.dll_unloaded.
90 seconds after the Remote Desktop Service stops working the winlogon notification subscriber notices the is taking a long to handle notification events, as can be seen in an event from Winlogon with ID 6005.
Sure enough, not much later the Citrix Health monitor was able to successfully run and pass the Terminal Services test and the state and failure threshold has been reset. Am information event is logged from CitrixHealthMon with ID 2006.
Although the service seemed to start working correctly again (the notification events are received and the health checks are passed) the Service Control Manager detects that the Remote Desktop Service is terminated unexpectedly and it will perform a correction action (Restart the service) in 60000 milliseconds, or 1 minute. Notice this happens 1 hour after the initial problem started.
Since the Remote Desktop Service is restarted the active (and disconnected) sessions are reset. In a normal situation an event is sent to all depending services such as Citrix IMA (used in Citrix XenApp 6.5) to inform when sessions are ended. When Citrix receives receives an event it can do some housekeeping such as cleaning up the Sessions key in the registry. HKLM\SOFTWARE\Citrix\Ica\Session
Since this event is never sent (the Remote Desktop Services service is in an inconsistent state and restarting) no housekeeping takes places and garbage remains in the registry. In this example three sessions where active. ID 3, 4 and 14.
Citrix XenApp detects that Remote Desktop Services is restarted and will act accordingly. For instance it will contact the Citrix license server to verify if it is available. An informational event is raised by MetaFrame with ID 9019.
So what happens if a new session is initiated? Since Citrix has hooked into Remote Desktop Service it is informed immediately when new Winstations are created (which happens for both RDP and ICA). Citrix is kind enough to keep track of all sessions in the HKLM\SOFTWARE\Citrix\Ica\Sessions key, regardless of the protocol used. In other words, RDP sessions are stored in the Ica sessions key as well.
In a virtualized environment, it can be difficult to differentiate between an error with the virtual machine or the VMkernel of the ESX. It is important to determine whether a halted or unresponsive virtual machine is a symptom of an error with Windows (or the applications running on the virtual machine) or the ESX host on which it resides. The following is intended as a set of general guidelines for diagnosing and identifying typical errors associated with a Windows failure.
Windows uses event codes that identify the issue encountered within the guest guest operating system. These codes are standardized and are useful in identifying the cause of issues on the guest. The Resolution section outlines a few common event codes, and provides information about each code. These codes are useful for determining the way in which the guest was made to go offline.
First, identify the approximate time the error occurred. Check the Application, Security and System logs for any items logged in this time period. If the guest has experienced a failure or unscheduled shutdown, this information is listed in the System log. Identify the error code in the System log to determine the type of shutdown encountered.
To identify the duration of a downtime, compare the time at which the 6006 or 6008 event occurred to the time when the 6005 event occurred. This amount of time is the duration of the downtime on this Windows guest machine.
When the type and time of interruption are identified, you can review the Application and Security logs to determine additional information about the cause can be identified. For example, a system failure may have related errors in the Application log that can explain what contributing factors lead to the failure. A 6006 shutdown event may have information regarding who initiated the command (or at least who was logged into the system) at the time of the incident.
After reviewing the Event Viewer, it may be necessary to identify more specific information regarding a Windows failure incident. At the time of a failure, Windows automatically dumps all of the data contained in its memory for diagnostic purposes. Windows does not save memory dumps by default. The guest operating system needs to be configured to do so. To configure the guest to do so, reference the Microsoft support and documentation resources to identify the correct method to enable this feature on the specific version of Windows that you are running.
If it has been configured, this memory information is stored as a file on the hard drive named memory.dmp . Typically, this file is saved to C:\windows\system and can usually be found with a simple search.
Group Policy is comprised of many client side extensions (CSEs). A few examples include Group Policy Software Installation,Folder Redirection, and Security Settings. When a machine is starting or being logged in, the Group Policy service has to process all of the settings for each CSE. By default, Windows does not show you which CSE or GPO is currently being processed. When a user reports that her machine is stuck at Please Wait or Applying Personal settings, you almost want to curse Windows for being so vague.
Once enable, reboot the machine to see the detailed startup/login messages. Now, you will see messages like Installing Managed Software: Microsoft Office instead of Please Wait. Personally, I find this setting so helpful that I keep it enabled across my domain. You can read more about Verbose Mode here.
Along with the specific problems above, your startup/logon times can be slowed by domain wide Group Policy settings and configurations. A common issue is the overuse of WMI filters (or time consuming WMI filters). WMI filters are extremely powerful because they allow your GPOs to process dynamically. Their downside is they have to be re-evaluated constantly. When you are able, avoid WMI filters.
A second widespread issue is the use of Loopback policy processing where it is not needed. Loopback allows you to apply user side settings, like a home page, to a group of computers. If you use Loopback Policy Processing in Merge mode, clients have to evaluate Group Policy twice. This can essentially double your processing time. A better solution, where possible, is to examine why you need loopback and if Active Directory can be reorganized to meet your need. For example, all of your users might be in a single OU. You might use loopback to set user homepages. The better solution is to divide your users into sub OUs and configure the homepage that way.
Our final issue is the abundant use of expensive Item Level Targetings. For example, an ILT that queries for multiple group memberships. These type of queries can slow processing because the computer has to query over the network to retrieve this information. If you absolutely must use security group memberships for Group Policy Preferences, read this Hotfix article from Microsoft.
With all of these problems, performance comes at the expense of complexity. The fastest GPO to evaluate will be the simplest. If you have followed these tips but are still experiencing slow downs, take a look at the Windows Performance Toolkit and run a trace.
Windows Server with Terminal Services TS / Remote Desktop Services RDS role installed takes up to an hour to allow logon ( stuck at Please wait for Group Policy Client ) then takes upto 90 minutes for logon ( stuck at Applying User Settings ) logging Winlogon 6005 Warnings with the following text:- The winlogon notification subscriber is taking a long time to handle the notification event (CreateSession) was solved by Microsoft KB2655998
93ddb68554