Tasks launched by service terminating with exit code 128
Morten Damgaard
I have a set (41) of tasks running on a Win2000 Advanced Server.
The tasks are launched by a task controller which is running as a service
under a specific user account.
The tasks are launched using CreateProcess.
Both the task controller and the tasks are written in C++ and build using
Visual C++ 6.0.
The "interact with desktop" flag is not set for the service.
None of the launched tasks or the service itself contains any GUI elements.
The tasks accept socket connections and telegrams or/and perform database
operations on the basis of the received telegrams.
Other read and writes telegram from files or sends telegrams to subscribers.
All theads are using exception handling and SetUnhandledExceptionFilter is
used to catch any unhandled exceptions. If any exceptions are caught a line
will be written in the log file.
Sometimes (after running for days/weeks) one/more of the tasks spontaneously
terminates with exit code 128 (0x80).
I have the task controller waiting for process handles to get signaled, and
then call GetExitCodeProcess which returns 128.
This lead me to this article
http://support.microsoft.com/default.aspx?scid=kb;en-us;184802 which
indicates that low desktop heap may be the problem.
It is not a problem when launching the task but only when the task has been
running for a while. This tells me that it may be desktop heap leak problem
that I'm facing.
Which Win32 API functions allocates memory on the desktop heap? or what is
stored on the desktop heap?
The above article says "menus, hooks, strings, windows" are stored in the
desktop heap, but what about file handles,
socket handles, etc.? None of the tasks are displaying any GUI or allocating
GUI object, but they do use a lot of file handles, and socket handles, and
threads, and mutex'.
Since the task controller is running as a service under a specific user a
new desktop is created when the service is started, and all tasks are using
of this desktop heap. The size of this desktop heap is 512K since I haven't
change
the default values in the registry.
I have tried to have the service running under LocalSystem account and the
tasks seems to terminate the same way - one of them still after 7 days even
though the size of the desktop heap is different. To me, this indicates that
it is not a desktop heap problem that I'm facing.
One more thing is that I have been monitoring a task using the Desktop Heap
Monitor tool from Microsoft which shows the size (total and free)
of the different desktop heaps in the system, and it doesn't indicate a
leak. Starting and stopping a task changes the amount of free memory on the
dedicated desktop heap, but having a task running and processing messages
etc. for several days doesn't result in increasing desktop heap usage.
So right now I have almost discarded the possibility that the problems are
caused by low desktop heap memory.
Any help will be most appreciated.
Thanks,
Morten Damgaard
Kellie Fitton
What happens when you use the API CreateProcessAsUser()
to launch the child process (tasks) ?
Kellie.
Morten Damgaard
"Kellie Fitton" <KELLIE...@YAHOO.COM> wrote in message
news:1140645201....@z14g2000cwz.googlegroups.com...
Hi Kellie
Thank you for the very quick answer!
I haven't tryed the CreateProcessAsUser instead of CreateProcess.
I want the tasks that are launched by the Task Controller (the one running
as a service under a specific account) to run as the same user as the
service. So I cannot see what difference it would make to use
CreateProcessAsUser.
From MSDN:
CreateProcess: "The new process runs the specified executable file in the
security context of the calling process"
CreateProcessAsUser: "the new process runs in the security context of the
user represented by the hToken parameter. ".
This tells me that CreateProcess and CreateProcessAsUser with the current
security context as argument (ImpersonateLoggedOnUser) would be the same.
But please, I'm not familiar with CreateProcessAsUser (I have never used
it), so I could easily be wrong. I'm just trying to understand the reason
for using the function instead of CreateProcess in this case.
Furthermore, the tasks terminates after have been running for many days, not
when calling CreateProcess.
Regards,
Morten
Kellie Fitton
Well, the functions CreateProcess() and CreateProcessAsUser()
are NOT the same, the latter provides one Important Difference,
the newly created process runs in a context in which the system
sees the user represented by the hToken parameter, as if that user
had logged on to the system and then launched the new process.
Moreover... the default security descriptors of disks are very
restrictive towards non-admin users, admins and system are allowed
full control, so, if your tasks processes are trying to access any
dataBase system to peform file IO operations as you have mentioned,
or trying to access any remote shares on a remote server machine,
then the proper approach to launch the task process is as follows:
LogonUserEx()
ImpersonateLoggedOnUser()
CreateEnvironmentBlock()
GetUserProfileDirectory()
LoadUserProfile()
CreateProcessAsUser()
Then,
DestroyEnvironmentBlock()
UnloadUserProfile()
RevertToSelf()
CloseHandle()
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/secauthn/security/logonuserex.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/policy/policy/loaduserprofile.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/policy/policy/unloaduserprofile.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/secauthz/security/reverttoself.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sysinfo/base/closehandle.asp
Hope these information helps,
Kellie.
Grzegorz Wróbel
> Hi,
>
> Well, the functions CreateProcess() and CreateProcessAsUser()
> Moreover... the default security descriptors of disks are very
> restrictive towards non-admin users, admins and system are allowed
Kelly, Morten already specified in his first post that he tried to run his service under localsystem account and the same behaviour occured. Localsystem processes run in ring0 and have highest possible priviledges (unless some security priviledges has been removed from process token), higher even than those of administrative accounts. So CreateProcessAsUser() rather won't help him here, but of course if he is patient enough to wait few days, he can always try to confirm that.
Microsoft specified clearly there are only 2 possibilities to get ERROR_WAIT_NO_CHILDREN. Since CreateProcess succeeded (ie the executed process does have proper security access to the window station and desktop associated with the process) then the reason must be the second case: The system has run out of desktop heap.
Morten, what winstation/desktop your service is run on? If there were other processes services running on the same desktop they could lead to either leak or consumption of all desktop heap memory. I would suggest to create dedicated desktop or winstation/desktop for this service and check if this behaviour happens as well. If so this would mean you have the leak within your service and searching there is the way to go.
--
677265676F727940346E6575726F6E732E636F6D
Kellie Fitton
My reply was referring to admins and system administrator are allowed
full control,
not the LocalSystem account.
Kellie. :--)
David Jones
> Localsystem processes run in ring0 and have highest possible priviledges
> (unless some security priviledges has been removed from process token),
> higher even than those of administrative accounts.
Actually, those processes still run in ring 3. Drivers run in
ring 0. Normally, I wouldn't quibble, but that's a *huge*
difference -- the difference between ring 3 and ring 0 is far
greater than the difference between SYSTEM and a normal user.
David
Grzegorz Wróbel
I know Kelly that your reply was reffering to admins. However, my reply to your reply was reffering to the fact that services which are run under local system account have unlimited priviledges on the local computer. Thus using CreateProcessAsUser() with administrative account in a service won't give you any benefit comparing to running the same service under local system account and using CreateProcess() instead. :-)
Am I correct?
--
677265676F727940346E6575726F6E732E636F6D
Grzegorz Wróbel
You're right of course, I simplified it too much (system wide priviledges to processor priviledges). Since sevice is still an application not a driver it doesn't need to access hardware directly. So local system processes run in ring3.
--
677265676F727940346E6575726F6E732E636F6D
Kellie Fitton
The service process is launching the child processes (tasks)
to peroform the following jobs:
* Accept socket connections
* Database access operations
* File read writes operations
* Send teleGrams to subscribers
Now, as I have mentioned before, the default security descriptors
of hardDisks are very restrictive towards NON-admin users, admins
and system administrator are allowed full control of the file I/O
operation, and only privileged users like administrators can read
or modify files created by other users, moreover, the LocalSystem
account does not have netWork credentials, and explicitly denied
any access to the UNC names.
Kellie. :--)
Grzegorz Wróbel
Hi Kelly,
I always belived that set of local system account's privileges is a superset of administartive privileges with exception of network credentials, since those for obvious reasons are not required for a local account. Now you made me to check it :). According to MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/localsystem_account.asp) privileges of local system account are (pluses indicate additional privileges comparing to administrative users):
SE_ASSIGNPRIMARYTOKEN_NAME (SeAssignPrimaryTokenPrivilege) +
SE_AUDIT_NAME (SeAuditPrivilege) +
SE_BACKUP_NAME (SeBackupPrivilege)
SE_CHANGE_NOTIFY_NAME (SeChangeNotifyPrivilege)
SE_CREATE_PAGEFILE_NAME (SeCreatePagefilePrivilege)
SE_CREATE_PERMANENT_NAME (SeCreatePermanentPrivilege) +
SE_CREATE_TOKEN_NAME (SeCreateTokenPrivilege) +
SE_DEBUG_NAME (SeDebugPrivilege)
SE_INC_BASE_PRIORITY_NAME (SeIncreaseBasePriorityPrivilege)
SE_INCREASE_QUOTA_NAME (SeIncreaseQuotaPrivilege)
SE_LOAD_DRIVER_NAME (SeLoadDriverPrivilege)
SE_LOCK_MEMORY_NAME (SeLockMemoryPrivilege) +
SE_PROF_SINGLE_PROCESS_NAME (SeProfileSingleProcessPrivilege)
SE_RESTORE_NAME (SeRestorePrivilege)
SE_SECURITY_NAME (SeSecurityPrivilege)
SE_SHUTDOWN_NAME (SeShutdownPrivilege)
SE_SYSTEM_ENVIRONMENT_NAME (SeSystemEnvironmentPrivilege)
SE_SYSTEM_PROFILE_NAME (SeSystemProfilePrivilege)
SE_SYSTEMTIME_NAME (SeSystemtimePrivilege)
SE_TAKE_OWNERSHIP_NAME (SeTakeOwnershipPrivilege)
SE_TCB_NAME (SeTcbPrivilege) +
SE_UNDOCK_NAME (SeUndockPrivilege)
A process running on WinXP SP2 machine under administrative account shows to hold following privileges (pluses indicated additional privileges to comparing local system account):
SeChangeNotifyPrivilege
SeSecurityPrivilege
SeBackupPrivilege
SeRestorePrivilege
SeSystemtimePrivilege
SeShutdownPrivilege
SeRemoteShutdownPrivilege +
SeTakeOwnershipPrivilege
SeDebugPrivilege
SeSystemEnviromentPrivilege
SeSystemProfilePrivilege
SeProfileSingleProcessPrivilege
SeIncreaseBasePriorityPrivilege
SeLoadDriverPrivilege
SeCreatePageFilePrivilege
SeIncreaseQuotaPrivilege
SeUndockPrivilege
SeManageVolumePrivilege +
SeCreateGlobalPrivilege +
SeImpersonatePrivilege +
Since SeRemoteShutdownPrivilege and SeImpersonatePrivilege are not required for local system account (it's run only locally and impersonated already), there are two privileges in question: SeCreateGlobalPrivilege and SeManageVolumePrivilege. I think SeCreateGlobalPrivilege is not required for anything in case of local system account, since service applications in a Terminal Services environment use the global namespace by default anyway.
It would seem however that lack of SeManageVolumePrivilege could constite a problem in case service tried to do some file management on a volume. So trying CreateProcessAsUser() instead CreateProcess() would be worth checking indeed, although both me and Morten ignored that idea at the very beggining. However it would be easier for Morten to simply run his service under an administrative account instead.
--
677265676F727940346E6575726F6E732E636F6D
Morten Damgaard
> Morten, what winstation/desktop your service is run on? If there were
other processes services running on the same desktop they could lead to
either leak or consumption of all desktop heap memory. I would suggest to
create dedicated desktop or winstation/desktop for this service and check if
this behaviour happens as well. If so this would mean you have the leak
within your service and searching there is the way to go.
>
The service run in its own desktop/window station created by the SCM. The
service itself does not use any CreateWindowStation/CreateDesktop functions
explicit.
From MSDN: "Every service process executed under a user account will receive
a new desktop in a noninteractive window station created by the Service
Control Manager (SCM). "
I have used the Desktop Heap Monitor tool from MS to monitor the desktop
usage of my tasks. According to this tool there is no leaks in the tasks
which are causing the amount of allocated desktop heap space to grow.
When I start the controlling service I can see that a new desktop is
created, and every time I start a new task from the controlling service I
can see that memory is allocated on the desktop heap associated with the
controlling service.
Furthermore; if I stop one or more tasks again the Used Rate of the desktop
heap will decrement to the originating level.
Output from MS heap monitor:
Desktop Heap Information Monitor Tool (Version 8.0.2823.0)
Copyright (c) 2003-2005 Microsoft Corporation. All rights reserved.
-------------------------------------------------------------
Session ID: 0 Total Desktop: ( 9344 KB - 8 desktops)
WinStation\Desktop Heap Size(KB) Used Rate(%)
-------------------------------------------------------------
WinSta0\Default 3072 3.7
WinSta0\Winlogon 128 5.0
Service-0x0-3e7$\Default 1024 13.1
Service-0x0-8c82$\Default 1024 0.7
SAWinSta\SADesktop 1024 0.2
TelnetSrvWinSta\TelnetSrvDesktop 1024 0.2
__X78B95_89_IW\__A8D9S1_42_ID 1024 0.2
Service-0x0-aa58e$\Default 1024 19.6
-------------------------------------------------------------
The last entry is the WinStation/Desktop used by my service and tasks.
If I let a task run for a couple of days the Used Rate of that desktop is
still the same.
Right now I have doubled the size of the desktop (through a Registry change)
and is testing the tasks again. One of the tasks exists when it has been
running for 7 days, and I certainly expect this to happend again with the
new size of the desktop - but time will tell. It started it yesterday, so
there are still 6 days to go....
But, to be honest, is don't see this exit code 128 as a desktop heap problem
anymore, because chaning the size of the desktop heap by moving the
controlling task from LocalSystem to a specific user didn't affect the
number of days to task could run.
But now I'm doing a final test by changing the size of the default desktop
heap for noninteractive desktops, and see if that effect the number of days
my tasks will run.
/Morten
Grzegorz Wróbel
>
> Output from MS heap monitor:
>
> Desktop Heap Information Monitor Tool (Version 8.0.2823.0)
> Copyright (c) 2003-2005 Microsoft Corporation. All rights reserved.
> -------------------------------------------------------------
> Session ID: 0 Total Desktop: ( 9344 KB - 8 desktops)
>
> WinStation\Desktop Heap Size(KB) Used Rate(%)
> -------------------------------------------------------------
> WinSta0\Default 3072 3.7
> WinSta0\Winlogon 128 5.0
> Service-0x0-3e7$\Default 1024 13.1
> Service-0x0-8c82$\Default 1024 0.7
> SAWinSta\SADesktop 1024 0.2
> TelnetSrvWinSta\TelnetSrvDesktop 1024 0.2
> __X78B95_89_IW\__A8D9S1_42_ID 1024 0.2
> Service-0x0-aa58e$\Default 1024 19.6
> -------------------------------------------------------------
>
> The last entry is the WinStation/Desktop used by my service and tasks.
> If I let a task run for a couple of days the Used Rate of that desktop is
> still the same.
>
> Right now I have doubled the size of the desktop (through a Registry change)
> and is testing the tasks again. One of the tasks exists when it has been
> running for 7 days, and I certainly expect this to happend again with the
> new size of the desktop - but time will tell. It started it yesterday, so
> there are still 6 days to go....
Is it always the same task/process that exits? What it does exactly? If it something unusual then it is possible that it tries to do something that requires lot of heap space after that 7 days.
--
677265676F727940346E6575726F6E732E636F6D
Morten Damgaard
> Is it always the same task/process that exits? What it does exactly? If it
something unusual then it is possible that it tries to do something that
requires lot of heap space after that 7 days.
>
I have seen the problem in several tasks. In one of the tasks the problem is
reoccuring every week, not excatly after 7 days, but 7 days +/- a couple of
hours.
The task is reading xml telegrams from a text files and inserting/updating
into a Oracle 9 database. Telegrams are sent over socket connection every
time information is updated in the database.
The runs 24/7 and inserts/updates data approx. every 10-20 seconds
What API functions are causing desktop heap to be allocated? I'm not sure
that the task in question uses any of them. To me it is mustly GUI functions
which does that.
Process exit code 128 does this mean for sure that it is either a
initialization problem during CreateProcess or a desktop heap problem, or
can it be something else?
Well, yes it can. Some memory is overwritten in the task and it end up
returning 128 to the OS.
To me it seems to be something else but I'm not sure yet. Right now I'm
waiting for the task to exit again after I've changed the size of the
desktop heap.
Morten