Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

csrss.exe High I/O in combination with Terminal service

2,590 views
Skip to first unread message

Vincent

unread,
Oct 17, 2008, 10:56:00 AM10/17/08
to
Hi,

Currently I’m running into some issues which occur on each and every
windows 2003 server that I’m managing:

When someone logs in on a server the csrss.exe process starts to use up a
lot of disk I/O. It seems to have a minimum of 10 MB/s with a maximum of 80
MB/s although the average seems to be around 30 MB/s. The high I/O continues
until the user logs off. The high I/O also stops completely when (and this is
especially weird) the Terminal service client screen is minimized, but high
disk activity will resume as soon as the screen is restored.

I am aware of knowledge base article 934330, and even though the article in
question states the fix is foor the issue that crss.exe utilizes a lot of
cpu instead of high I/O, I decided to install the hotfix on a server to see
if the problems are related. Unfortunately it seems they’re not.

As I already said I'm experiencing this issue on multiple windows 2k3
server versions (web, standard, standard R2). They are all running service
pack 2 and are fully patched. The hardware differs as well. I've seen this
problem occur on Dell and HP servers so it is practically impossible that its
hardware related.

I'm at loss as to what the cause is. Does anybody have a clue?

Thanks in advance,
Vincent

rpremuz

unread,
Oct 31, 2008, 4:49:29 PM10/31/08
to
On Oct 17, 3:56 pm, Vincent <Vinc...@discussions.microsoft.com> wrote:
>
> Currently I’m running into some issues which occur on each and every
> windows 2003 server that I’m managing:
>
> When someone logs in on a server the csrss.exe process starts to use up a
> lot of disk I/O. It seems to have a minimum of 10 MB/s with a maximum of 80
> MB/s although the average seems to be around 30 MB/s. The high I/O continues
> until the user logs off. The high I/O also stops completely when (and this is
> especially weird) the Terminal service client screen is minimized, but high
> disk activity will resume as soon as the screen is restored.

Vincent, I don't know the solution to the problem but I can confirm
that I experience the same on two MS Windows Server 2003 SP2 (Standard
Edition) with all the latest patches (the servers don't have the same
hardware or software configuration).

Here is how I can get extremely high I/O for the csrss.exe: when I'm
connected to the server through the Remote Desktop I press and hold a
key (e.g. Shift or Alt) on the keyboard: the Process Explorer shows
that Other Bytes Delta is about 200 MB in that case. See picture on
http://www.hotshare.net/image/91284-64755441d6.html

This is really a major issue. It there any MSMVP who has seen that
before and knows the solution?

-- rpr.

Robert Premuž

unread,
Nov 3, 2008, 7:22:13 AM11/3/08
to

I installed the 934330 hotfix received from MS Support (see
http://support.microsoft.com/kb/934330 that explains a similar problem
with csrss.exe) but it didn't solved the problem.

-- rpr.

jolteroli

unread,
Nov 3, 2008, 1:49:40 PM11/3/08
to
Do the session, the "freaking csrss" belongs to, also have a process named
"ntvdm" running?

-jolt

"Robert Premuž" <rpr...@yahoo.com> schrieb im Newsbeitrag
news:C715B684-C619-434F...@microsoft.com...

rpremuz

unread,
Nov 4, 2008, 4:23:11 AM11/4/08
to
On Nov 3, 7:49 pm, "jolteroli" <jolt1...@gmx.net> wrote:
> Do the session, the "freaking csrss" belongs to, also have a process named
> "ntvdm" running?

There is no ntvdm process running on the server.

-- rpr.

jolteroli

unread,
Nov 4, 2008, 4:52:44 PM11/4/08
to
Not sure if this will help...

If you look in proc-explorer on the threads-tab in the csrss process, you
should find a thread causing all this IO and having a pretty high
cpu/context-switch-delta value.

If you suspend this thread does the IO drop? (Do this on the csrss of a
test-user session)

What is the "starting address" and the call [Stack] of the thread?

-jolt

"rpremuz" <rpr.n...@gmail.com> schrieb im Newsbeitrag
news:d584c509-0cb1-4bb5...@p31g2000prf.googlegroups.com...

rpremuz

unread,
Nov 5, 2008, 8:27:06 AM11/5/08
to
On Nov 4, 10:52 pm, "jolteroli" <jolt1...@gmx.net> wrote:
>
> If you look in proc-explorer on the threads-tab in the csrss process, you
> should find a thread causing all this IO and having a pretty high
> cpu/context-switch-delta value.
>
> If you suspend this thread does the IO drop? (Do this on the csrss of a
> test-user session)
>
> What is the "starting address" and the call [Stack] of the thread?

The thread of csrss.exe with the highest CSwitch Delta has the
following start address: winsrv.dll!ConServerDllInitialization+0x5866

If the thread is suspended, the I/O drops but the terminal session
screen freezes (no animation) and it becomes unresponsive to keyboard
and mouse inputs. If the thread is resumed (from the Process Explorer
running in another terminal session), the terminal session unfreezes
and high I/O problem starts again.

The call stack of the thread is the following:
0 ntoskrnl.exe+0x397ea
1 ntoskrnl.exe+0x3df9e
2 ntoskrnl.exe+0x50675
3 ntoskrnl.exe+0x40720
4 ntoskrnl.exe+0x3f326
5 win32k.sys+0x15287
6 win32k.sys+0x6fd25
7 win32k.sys+0x98a52
8 ntoskrnl.exe+0x33bef
9 ntdll.dll!KiFastSystemCallRet

The high I/O can be induced by keyboard or mouse activity.
Here is what a little testing on a server showed for the csrss.exe
corresponding to a terminal session:
-- the terminal session window restored or maximized
+ no mouse or keyboard activity -- Other Bytes Delta: 7 MB
+ mouse activity -- Other Bytes Delta: 50 - 70 MB
+ keyboard activity -- Other Bytes Delta: 150 - 200 MB
-- the terminal session window minimized -- Other Bytes Delta: 0

If the server console is used for log on, there is no high I/O for the
corresponding csrss.exe process.

-- rpr.

andrea...@gmail.com

unread,
Nov 8, 2008, 10:11:14 AM11/8/08
to
Hi.

I have no solution, but same issue on three w2k3 enterprise R2 sp2.
I try patch from microsoft kb 934330 without success.

Some ideas?

Andrea

jolteroli

unread,
Nov 9, 2008, 6:36:04 AM11/9/08
to
Hey Robert,

I don't want bother you, but you've not configured symbols. If you want to
have them:

1.) Install latest windbg.
2.) Copy latest dbghelp.dll (in the windbg directory) to system32
3.) Create a directory for symbol files, e.g.: f:\symbols
4.) Set system-wide environment variable:
_NT_SYMBOL_PATH=SRV*f:\symbols*http://msdl.microsoft.com/download/symbols
5.) Download symbols for a single image by:
symchk c:\winnt\system32\winsrv.dll
symchk c:\winnt\system32\csrss.exe
symchk c:\winnt\system32\win32k.sys
symchk c:\winnt\system32\ntkrnpa.exe
symchk c:\winnt\system32\ntdll.dll
... and so on...
Or download any image in system32 recursively:
symchk /r c:\winnt\system32\*
6.) In the Process Explorer, configure the symbol path, to f:\symbols

You can put
dbgeng.dll
dbghelp.dll
SymbolCheck.dll
symchk.exe
symsrv.dll
on a USB pen. So you don't need the Debugging Tools any time.

If you now view the threads and their stack trace, it might look like:

winsrv.dll!StartCreateSystemThreads:
ntkrnlpa.exe!KiSwapContext+0x2f
ntkrnlpa.exe!KiSwapThread+0x8a
ntkrnlpa.exe!KeWaitForMultipleObjects+0x284
win32k.sys!RawInputThread+0x4f3
win32k.sys!xxxCreateSystemThreads+0x60
win32k.sys!NtUserCallOneParam+0x23
ntkrnlpa.exe!KiFastCallEntry+0xfc
ntdll.dll!KiFastSystemCallRet
winsrv.dll!NtUserCallOneParam+0xc

And you can see which functions are involved in the freaking thread. I would
start and view the stack traces

Once, when the RDP window is minimized has no IO
Once, when the RDP window is maximized and produce IO

Any difference?

-jolt

rpr

unread,
Nov 12, 2008, 2:30:11 PM11/12/08
to
Hi, jolteroli!

As you suggested I installed Debugging Tools for Windows 32-bit
Version
(from http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx
).
To replace "%SystemRoot%\system32\dbghelp.dll", which is in use by
some running processes, with the newer version in
"%ProgramFiles%\Debugging Tools for Windows (x86)\" you need to use
one of the methods explained at http://support.microsoft.com/kb/181345

Then I run the following commands:
mkdir c:\windows\symbols

reg add "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager
\Environment" /v _NT_SYMBOL_PATH /d "SRV*c:\windows\symbols*http://
msdl.microsoft.com/download/symbols" /f

"%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
"%SystemRoot%\system32\winsrv.dll"

"%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
"%SystemRoot%\system32\csrss.exe"

"%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
"%SystemRoot%\system32\ntoskrnl.exe"

"%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
"%SystemRoot%\system32\win32k.sys"

"%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
"%SystemRoot%\system32\ntdll.dll"

After restarting the server I got the following data from
Process Explorer regarding my problem:

When the RDP window is minimized, the I/O is normal.


The thread of csrss.exe with the highest CSwitch Delta has the

following start address: winsrv.dll!StartCreateSystemThreads

The call stack of the thread is the following:

ntoskrnl.exe!KiSwapContext+0x26
ntoskrnl.exe!KiSwapThread+0x2e5
ntoskrnl.exe!KeWaitForSingleObject+0x346
ntoskrnl.exe!KiSuspendThread+0x18
ntoskrnl.exe!KiDeliverApc+0x117
ntoskrnl.exe!KiSwapThread+0x300
ntoskrnl.exe!KeWaitForMultipleObjects+0x3d7
win32k.sys!RawInputThread+0x4e0
win32k.sys!xxxCreateSystemThreads+0x60
win32k.sys!NtUserCallOneParam+0x23
ntoskrnl.exe!KiFastCallEntry+0xfc
ntdll.dll!KiFastSystemCallRet
winsrv.dll!NtUserCallOneParam+0xc

When the RDP window is restored or maximized, the I/O is very high.
But the thread of csrss.exe with the highest CSwitch Delta has the
start address and call stack as above (no difference when compared to
the normal state).

Does this tell you anything? To me it tells nothing as I don't have
the sources :-| (Almost a complete waste of time.)

-- rpr.

jolteroli

unread,
Nov 13, 2008, 3:07:03 PM11/13/08
to
Hee Rob,

> reg add "HKLM\SYSTEM\CurrentControlSet\Control\Session
> Manager \Environment" /v _NT_SYMBOL_PATH /d
> "SRV*c:\windows\symbols*http://msdl.microsoft.com/download/symbols" /f

Real men don't click, cool... :)

> ntoskrnl.exe!KiSwapContext+0x26
> ntoskrnl.exe!KiSwapThread+0x2e5
> ntoskrnl.exe!KeWaitForSingleObject+0x346
> ntoskrnl.exe!KiSuspendThread+0x18
> ntoskrnl.exe!KiDeliverApc+0x117
> ntoskrnl.exe!KiSwapThread+0x300
> ntoskrnl.exe!KeWaitForMultipleObjects+0x3d7
> win32k.sys!RawInputThread+0x4e0
> win32k.sys!xxxCreateSystemThreads+0x60
> win32k.sys!NtUserCallOneParam+0x23
> ntoskrnl.exe!KiFastCallEntry+0xfc
> ntdll.dll!KiFastSystemCallRet
> winsrv.dll!NtUserCallOneParam+0xc
>

> Does this tell you anything?

Since our servers (not having this issue) doesn't fork to KiDeliverApc, I
guess there is constantly arriving keyb data (that is what RawInputThread
RIT "polls" for) and that's causing the IO. I have no clue what is causing
the keyb data...

In german we have the Systemcontrol "Eingabehilfen" with the symbol of Steve
Balmers wheelchair. Check if any enhanced services is turned on there...

ATI's stupid HotKeyPolling service comes to mind too...

May be something in this direction. For me it seems the data comes from the
keyb/driver.

> To me it tells nothing as I don't have the sources :-| (Almost a complete
> waste of time.)

Any additional help would need a debugger, so it was a waste of time. Sorry
Rob.

-jolt

TP

unread,
Nov 13, 2008, 3:27:21 PM11/13/08
to
Robert and Vincent,

What are the exact problem symptoms you are experiencing? Is
the server running slow? Are users complaining? If yes, what
lead you to the conclusion that this particular reading is the
source of the problems you are having?

I have looked at the numbers you are seeing and this must be
a bug. I see this on every 2003 server I have examined so far
(since reading your post). If these numbers were true the
servers would grind to a halt.

Thanks.

-TP

rpr

unread,
Nov 14, 2008, 5:31:04 AM11/14/08
to
On Nov 13, 9:07 pm, "jolteroli" <jolt1...@gmx.net> wrote:
>
> > reg add "HKLM\SYSTEM\CurrentControlSet\Control\Session
> > Manager \Environment" /v _NT_SYMBOL_PATH /d
> > "SRV*c:\windows\symbols*http://msdl.microsoft.com/download/symbols" /f
>
> Real men don't click, cool... :)

I also live in Unix world and hence I'm familiar with the command
line. Unfortunately MS KBAs rarely suggest using command line tools.
Fortunately we have this groups for writing our own knowledge base
articles :-)

> Since our servers (not having this issue) doesn't fork to KiDeliverApc, I
> guess there is constantly arriving keyb data (that is what RawInputThread
> RIT "polls" for) and that's causing the IO. I have no clue what is causing
> the keyb data...
>
> In german we have the Systemcontrol "Eingabehilfen" with the symbol of Steve
> Balmers wheelchair. Check if any enhanced services is turned on there...
>
> ATI's stupid HotKeyPolling service comes to mind too...
>
> May be something in this direction. For me it seems the data comes from the
> keyb/driver.

Any of the Accessibility Options is not enabled on the server.
Also, no ATI services.

-- rpr.

rpr

unread,
Nov 14, 2008, 5:36:09 AM11/14/08
to
I discovered the high I/O problem in csrss.exe while investigating a
problem with the system clock on one of DCs in the domain. The
server's system clock is quite inaccurate.
I tried to solve that by reconfiguring the Windows Time Service
(w32time) using the procedures suggested at
http://technet.microsoft.com/en-us/library/cc773061.aspx
but it was not satisfactory as the Windows Time Service cannot take
into account the clock drift (as a good NTP service should).

At http://answers.google.com/answers/threadview?id=438195
you can find complains that suggest this high I/O slows down the
Windows Server 2003. Note that the question is written on 2006-01-26.

At http://forum.sysinternals.com/forum_posts.asp?TID=12894
Mark Russinovich said the following regarding this issue on
2008-03-06:
"This behavior is a known bug in the w2k3 terminal server I/O
counters."
But it was not explained any further.

Even if this bug causes no real harm it should have been solved long
ago. It certainly creates confusion and increases dissatisfaction with
MS software.

-- rpr.

mrmmills

unread,
Nov 19, 2008, 9:44:17 AM11/19/08
to
Found your post while trying to understand Microsoft Office Server 2007 consumption of resources (MOSS 2007\SharePoint).

My SharePoint Central Administration Server is a W2k3 R2 SP2 Std Ed X64 server with 8GB of memory.
My IO Delta climbs at times from 64KB to 84GB (not a typo ...84GB!)this is for the CRSS.exe (Client Server Runtime Process) when I view it in the latest version of Process Explorer.

My "I\O Other Bytes" is reading 19 Trillion+, equivelant to 19 Tereabytes????. The server itself is on a 30GB and 40GB partitions so the 19Trillion Terabytes may be I\O's from reading the SQL DB's its attached to which reside on another server. But the SQL db's its talking to total less than 200GB?

dumb founded....this server is showing very few eventlog errors and was only rebooted 12 hours ago.

In Process Explorer I'm also seeing a lot (as in hundreds) of "Contentions" for different instancesof the MS Seach Component mssdmn.exe

MSSearch.exe is also racking up a significant amount or PageFaults??

Will be openign a case with MS as soon as schedule permits.

kC C

unread,
Apr 6, 2011, 11:45:08 AM4/6/11
to
hi Guys just a quick question to you all suffering this issue, wheres your Page filing pointing to and whats it set to?


> On Friday, October 17, 2008 10:56 AM Vincen wrote:

> Hi,


>
> Currently I’m running into some issues which occur on each and every
> windows 2003 server that I’m managing:
>
> When someone logs in on a server the csrss.exe process starts to use up a
> lot of disk I/O. It seems to have a minimum of 10 MB/s with a maximum of 80
> MB/s although the average seems to be around 30 MB/s. The high I/O continues
> until the user logs off. The high I/O also stops completely when (and this is
> especially weird) the Terminal service client screen is minimized, but high
> disk activity will resume as soon as the screen is restored.
>

> I am aware of knowledge base article 934330, and even though the article in
> question states the fix is foor the issue that crss.exe utilizes a lot of
> cpu instead of high I/O, I decided to install the hotfix on a server to see
> if the problems are related. Unfortunately it seems they’re not.
>
> As I already said I'm experiencing this issue on multiple windows 2k3
> server versions (web, standard, standard R2). They are all running service
> pack 2 and are fully patched. The hardware differs as well. I've seen this
> problem occur on Dell and HP servers so it is practically impossible that its
> hardware related.
>
> I'm at loss as to what the cause is. Does anybody have a clue?
>
> Thanks in advance,
> Vincent


>> On Saturday, November 01, 2008 7:30 AM rpremuz wrote:

>> On Oct 17, 3:56 pm, Vincent <Vinc...@discussions.microsoft.com> wrote:

>> a
>> 80
>> ues
>> s is
>> gh


>>
>> Vincent, I don't know the solution to the problem but I can confirm
>> that I experience the same on two MS Windows Server 2003 SP2 (Standard
>> Edition) with all the latest patches (the servers don't have the same
>> hardware or software configuration).
>>
>> Here is how I can get extremely high I/O for the csrss.exe: when I'm
>> connected to the server through the Remote Desktop I press and hold a
>> key (e.g. Shift or Alt) on the keyboard: the Process Explorer shows
>> that Other Bytes Delta is about 200 MB in that case. See picture on
>> http://www.hotshare.net/image/91284-64755441d6.html
>>
>> This is really a major issue. It there any MSMVP who has seen that
>> before and knows the solution?
>>

>> -- rpr.


>>> On Monday, November 03, 2008 7:22 AM rpremu wrote:

>>> On 2008-10-31 I wrote:
>>>
>>> I installed the 934330 hotfix received from MS Support (see
>>> http://support.microsoft.com/kb/934330 that explains a similar problem
>>> with csrss.exe) but it didn't solved the problem.
>>>
>>> -- rpr.


>>>> On Monday, November 03, 2008 1:49 PM jolteroli wrote:

>>>> Do the session, the "freaking csrss" belongs to, also have a process named
>>>> "ntvdm" running?
>>>>

>>>> -jolt


>>>>> On Tuesday, November 04, 2008 4:52 PM jolteroli wrote:

>>>>> Not sure if this will help...
>>>>>

>>>>> If you look in proc-explorer on the threads-tab in the csrss process, you
>>>>> should find a thread causing all this IO and having a pretty high
>>>>> cpu/context-switch-delta value.
>>>>>
>>>>> If you suspend this thread does the IO drop? (Do this on the csrss of a
>>>>> test-user session)
>>>>>
>>>>> What is the "starting address" and the call [Stack] of the thread?
>>>>>

>>>>> -jolt
>>>>>
>>>>> "rpremuz" <rpr.n...@gmail.com> schrieb im Newsbeitrag
>>>>> news:d584c509-0cb1-4bb5...@p31g2000prf.googlegroups.com...
>>>>> On Nov 3, 7:49 pm, "jolteroli" <jolt1...@gmx.net> wrote:
>>>>>
>>>>> There is no ntvdm process running on the server.
>>>>>
>>>>> -- rpr.


>>>>>> On Thursday, November 06, 2008 1:34 AM rpremuz wrote:

>>>>>> d


>>>>>>
>>>>>> There is no ntvdm process running on the server.
>>>>>>
>>>>>> -- rpr.


>>>>>>> On Thursday, November 06, 2008 1:34 AM rpremuz wrote:

>>>>>>> On Nov 4, 10:52=A0pm, "jolteroli" <jolt1...@gmx.net> wrote:
>>>>>>>
>>>>>>> The thread of csrss.exe with the highest CSwitch Delta has the

>>>>>>> following start address: winsrv.dll!ConServerDllInitialization+0x5866
>>>>>>>
>>>>>>> If the thread is suspended, the I/O drops but the terminal session
>>>>>>> screen freezes (no animation) and it becomes unresponsive to keyboard
>>>>>>> and mouse inputs. If the thread is resumed (from the Process Explorer
>>>>>>> running in another terminal session), the terminal session unfreezes
>>>>>>> and high I/O problem starts again.
>>>>>>>

>>>>>>> The call stack of the thread is the following:

>>>>>>> 0 ntoskrnl.exe+0x397ea
>>>>>>> 1 ntoskrnl.exe+0x3df9e
>>>>>>> 2 ntoskrnl.exe+0x50675
>>>>>>> 3 ntoskrnl.exe+0x40720
>>>>>>> 4 ntoskrnl.exe+0x3f326
>>>>>>> 5 win32k.sys+0x15287
>>>>>>> 6 win32k.sys+0x6fd25
>>>>>>> 7 win32k.sys+0x98a52
>>>>>>> 8 ntoskrnl.exe+0x33bef
>>>>>>> 9 ntdll.dll!KiFastSystemCallRet
>>>>>>>
>>>>>>> The high I/O can be induced by keyboard or mouse activity.
>>>>>>> Here is what a little testing on a server showed for the csrss.exe
>>>>>>> corresponding to a terminal session:
>>>>>>> -- the terminal session window restored or maximized
>>>>>>> + no mouse or keyboard activity -- Other Bytes Delta: 7 MB
>>>>>>> + mouse activity -- Other Bytes Delta: 50 - 70 MB
>>>>>>> + keyboard activity -- Other Bytes Delta: 150 - 200 MB
>>>>>>> -- the terminal session window minimized -- Other Bytes Delta: 0
>>>>>>>
>>>>>>> If the server console is used for log on, there is no high I/O for the
>>>>>>> corresponding csrss.exe process.
>>>>>>>
>>>>>>> -- rpr.


>>>>>>>> On Saturday, November 08, 2008 5:07 PM andrea.tiron wrote:

>>>>>>>> Hi.
>>>>>>>>
>>>>>>>> I have no solution, but same issue on three w2k3 enterprise R2 sp2.
>>>>>>>> I try patch from microsoft kb 934330 without success.
>>>>>>>>
>>>>>>>> Some ideas?
>>>>>>>>
>>>>>>>> Andrea
>>>>>>>>
>>>>>>>>

>>>>>>>> ou


>>>>>>>>>> On Thursday, November 13, 2008 3:07 PM jolteroli wrote:

>>>>>>>>>> Hee Rob,


>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Real men don't click, cool... :)
>>>>>>>>>>
>>>>>>>>>>

>>>>>>>>>> Since our servers (not having this issue) doesn't fork to KiDeliverApc, I
>>>>>>>>>> guess there is constantly arriving keyb data (that is what RawInputThread
>>>>>>>>>> RIT "polls" for) and that's causing the IO. I have no clue what is causing
>>>>>>>>>> the keyb data...
>>>>>>>>>>
>>>>>>>>>> In german we have the Systemcontrol "Eingabehilfen" with the symbol of Steve
>>>>>>>>>> Balmers wheelchair. Check if any enhanced services is turned on there...
>>>>>>>>>>
>>>>>>>>>> ATI's stupid HotKeyPolling service comes to mind too...
>>>>>>>>>>
>>>>>>>>>> May be something in this direction. For me it seems the data comes from the
>>>>>>>>>> keyb/driver.
>>>>>>>>>>
>>>>>>>>>>

>>>>>>>>>> Any additional help would need a debugger, so it was a waste of time. Sorry
>>>>>>>>>> Rob.
>>>>>>>>>>
>>>>>>>>>> -jolt


>>>>>>>>>>> On Thursday, November 13, 2008 3:27 PM TP wrote:

>>>>>>>>>>> Robert and Vincent,
>>>>>>>>>>>
>>>>>>>>>>> What are the exact problem symptoms you are experiencing? Is
>>>>>>>>>>> the server running slow? Are users complaining? If yes, what
>>>>>>>>>>> lead you to the conclusion that this particular reading is the
>>>>>>>>>>> source of the problems you are having?
>>>>>>>>>>>
>>>>>>>>>>> I have looked at the numbers you are seeing and this must be
>>>>>>>>>>> a bug. I see this on every 2003 server I have examined so far
>>>>>>>>>>> (since reading your post). If these numbers were true the
>>>>>>>>>>> servers would grind to a halt.
>>>>>>>>>>>
>>>>>>>>>>> Thanks.
>>>>>>>>>>>
>>>>>>>>>>> -TP
>>>>>>>>>>>

>>>>>>>>>>> Robert Premuž wrote:


>>>>>>>>>>>> On Thursday, November 13, 2008 11:11 PM rpr wrote:

>>>>>>>>>>>> Hi, jolteroli!
>>>>>>>>>>>>
>>>>>>>>>>>> As you suggested I installed Debugging Tools for Windows 32-bit
>>>>>>>>>>>> Version
>>>>>>>>>>>> (from http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx
>>>>>>>>>>>> ).
>>>>>>>>>>>> To replace "%SystemRoot%\system32\dbghelp.dll", which is in use by
>>>>>>>>>>>> some running processes, with the newer version in
>>>>>>>>>>>> "%ProgramFiles%\Debugging Tools for Windows (x86)\" you need to use
>>>>>>>>>>>> one of the methods explained at http://support.microsoft.com/kb/181345
>>>>>>>>>>>>
>>>>>>>>>>>> Then I run the following commands:
>>>>>>>>>>>> mkdir c:\windows\symbols
>>>>>>>>>>>>

>>>>>>>>>>>> reg add "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager

>>>>>>>>>>>> \Environment" /v _NT_SYMBOL_PATH /d "SRV*c:\windows\symbols*http://
>>>>>>>>>>>> msdl.microsoft.com/download/symbols" /f
>>>>>>>>>>>>
>>>>>>>>>>>> "%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
>>>>>>>>>>>> "%SystemRoot%\system32\winsrv.dll"
>>>>>>>>>>>>
>>>>>>>>>>>> "%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
>>>>>>>>>>>> "%SystemRoot%\system32\csrss.exe"
>>>>>>>>>>>>
>>>>>>>>>>>> "%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
>>>>>>>>>>>> "%SystemRoot%\system32\ntoskrnl.exe"
>>>>>>>>>>>>
>>>>>>>>>>>> "%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
>>>>>>>>>>>> "%SystemRoot%\system32\win32k.sys"
>>>>>>>>>>>>
>>>>>>>>>>>> "%ProgramFiles%\Debugging Tools for Windows (x86)\symchk.exe"
>>>>>>>>>>>> "%SystemRoot%\system32\ntdll.dll"
>>>>>>>>>>>>
>>>>>>>>>>>> After restarting the server I got the following data from
>>>>>>>>>>>> Process Explorer regarding my problem:
>>>>>>>>>>>>
>>>>>>>>>>>> When the RDP window is minimized, the I/O is normal.
>>>>>>>>>>>> The thread of csrss.exe with the highest CSwitch Delta has the
>>>>>>>>>>>> following start address: winsrv.dll!StartCreateSystemThreads
>>>>>>>>>>>>
>>>>>>>>>>>> The call stack of the thread is the following:

>>>>>>>>>>>> ntoskrnl.exe!KiSwapContext+0x26
>>>>>>>>>>>> ntoskrnl.exe!KiSwapThread+0x2e5
>>>>>>>>>>>> ntoskrnl.exe!KeWaitForSingleObject+0x346
>>>>>>>>>>>> ntoskrnl.exe!KiSuspendThread+0x18
>>>>>>>>>>>> ntoskrnl.exe!KiDeliverApc+0x117
>>>>>>>>>>>> ntoskrnl.exe!KiSwapThread+0x300
>>>>>>>>>>>> ntoskrnl.exe!KeWaitForMultipleObjects+0x3d7
>>>>>>>>>>>> win32k.sys!RawInputThread+0x4e0
>>>>>>>>>>>> win32k.sys!xxxCreateSystemThreads+0x60
>>>>>>>>>>>> win32k.sys!NtUserCallOneParam+0x23
>>>>>>>>>>>> ntoskrnl.exe!KiFastCallEntry+0xfc
>>>>>>>>>>>> ntdll.dll!KiFastSystemCallRet
>>>>>>>>>>>> winsrv.dll!NtUserCallOneParam+0xc
>>>>>>>>>>>>

>>>>>>>>>>>> When the RDP window is restored or maximized, the I/O is very high.
>>>>>>>>>>>> But the thread of csrss.exe with the highest CSwitch Delta has the
>>>>>>>>>>>> start address and call stack as above (no difference when compared to
>>>>>>>>>>>> the normal state).
>>>>>>>>>>>>

>>>>>>>>>>>> Does this tell you anything? To me it tells nothing as I don't have


>>>>>>>>>>>> the sources :-| (Almost a complete waste of time.)
>>>>>>>>>>>>

>>>>>>>>>>>> -- rpr.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Nov 9, 12:36=A0pm, "jolteroli" <jolt1...@gmx.net> wrote:
>>>>>>>>>>>> o
>>>>>>>>>>>> ad/symbols
>>>>>>>>>>>> uld


>>>>>>>>>>>>> On Sunday, November 16, 2008 2:03 AM rpr wrote:

>>>>>>>>>>>>> On Nov 13, 9:07=A0pm, "jolteroli" <jolt1...@gmx.net> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I also live in Unix world and hence I'm familiar with the command
>>>>>>>>>>>>> line. Unfortunately MS KBAs rarely suggest using command line tools.
>>>>>>>>>>>>> Fortunately we have this groups for writing our own knowledge base
>>>>>>>>>>>>> articles :-)
>>>>>>>>>>>>>

>>>>>>>>>>>>> g
>>>>>>>>>>>>> eve
>>>>>>>>>>>>> he


>>>>>>>>>>>>>
>>>>>>>>>>>>> Any of the Accessibility Options is not enabled on the server.
>>>>>>>>>>>>> Also, no ATI services.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -- rpr.


>>>>>>>>>>>>>> On Sunday, November 16, 2008 2:03 AM rpr wrote:

>>>>>>>>>>>>>> I discovered the high I/O problem in csrss.exe while investigating a
>>>>>>>>>>>>>> problem with the system clock on one of DCs in the domain. The
>>>>>>>>>>>>>> server's system clock is quite inaccurate.
>>>>>>>>>>>>>> I tried to solve that by reconfiguring the Windows Time Service
>>>>>>>>>>>>>> (w32time) using the procedures suggested at
>>>>>>>>>>>>>> http://technet.microsoft.com/en-us/library/cc773061.aspx
>>>>>>>>>>>>>> but it was not satisfactory as the Windows Time Service cannot take
>>>>>>>>>>>>>> into account the clock drift (as a good NTP service should).
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> At http://answers.google.com/answers/threadview?id=3D438195


>>>>>>>>>>>>>> you can find complains that suggest this high I/O slows down the
>>>>>>>>>>>>>> Windows Server 2003. Note that the question is written on 2006-01-26.
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> At http://forum.sysinternals.com/forum_posts.asp?TID=3D12894


>>>>>>>>>>>>>> Mark Russinovich said the following regarding this issue on
>>>>>>>>>>>>>> 2008-03-06:
>>>>>>>>>>>>>> "This behavior is a known bug in the w2k3 terminal server I/O
>>>>>>>>>>>>>> counters."
>>>>>>>>>>>>>> But it was not explained any further.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Even if this bug causes no real harm it should have been solved long
>>>>>>>>>>>>>> ago. It certainly creates confusion and increases dissatisfaction with
>>>>>>>>>>>>>> MS software.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -- rpr.
>>>>>>>>>>>>>>

>>>>>>>>>>>>>> On Nov 13, 9:27=A0pm, "TP" <tperson.knowsp...@mailandnews.com> wrote:

David Ludvik

unread,
May 3, 2011, 10:04:28 PM5/3/11
to
Hi Guys,
this hot-fix should fix the issue
http://support.microsoft.com/kb/956438


> On Friday, October 17, 2008 10:56 AM Vincen wrote:

David Ludvik

unread,
May 3, 2011, 10:05:42 PM5/3/11
to
Hi Guys,
This should fix the issue.

http://support.microsoft.com/kb/956438

> On Friday, October 17, 2008 10:56 AM Vincen wrote:

rpr.n...@gmail.com

unread,
Jul 24, 2013, 6:32:41 PM7/24/13
to
On Wednesday, May 4, 2011 4:04:28 AM UTC+2, David Ludvik wrote:
> Hi Guys,
> this hot-fix should fix the issue
> http://support.microsoft.com/kb/956438

Thanks. I can confirm that the 956438 hot-fix has fixed the issue.

-- rpr.
0 new messages