Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Problem using DF_ALLOWOTHERACCOUNTHOOK and SetWindowsHookEx

146 views
Skip to first unread message

Nobody

unread,
Oct 13, 2010, 3:37:11 PM10/13/10
to
I have a problem with SetWindowsHookEx() failing with error code
ERROR_ACCESS_DENIED(5) when the target process is running as another user.
In this case, both users are members of the Administrators group. The OS is
XP Pro+SP2, and I am running it on the console on the same desktop and using
right-click "Run as" to run the target process.

I am using thread specific hooks(WH_GETMESSAGE) and the hook procedure
resides in a multithreaded DLL written in VC6. The target process is
enabling DF_ALLOWOTHERACCOUNTHOOK flag on the desktop and all calls succeed
when enabling that flag.

So what could be causing SetWindowsHookEx() to fail?

If I start the target process as the same user, it works fine, and I see
messages. The target process is a simple single threaded application that I
have written for testing. The real application is what I want to automate
for a customer, and since I don't have the source code for the real process,
I would use AppInit_DLLs to enable that flag on the real target process.
Needless to say I have already written that DLL and it loads fine and calls
to set the flag succeed, but SetWindowsHookEx() still fails, so I am using
my own simple target process until I found the real cause. Here is the
output from the code below using DebugView:

[3528] OpenInputDesktop succeeded.
[3528] SetUserObjectInformation succeeded

I already tried GetThreadDesktop, and OpenDesktop("Default") with the same
result. Here is the code in the target process which I run the first thing
when WinMain is called:

// Allow other hooks code
USEROBJECTFLAGS uof;
BOOL bRet;
char szDebug[200];

HDESK hDesktop = OpenInputDesktop(DF_ALLOWOTHERACCOUNTHOOK, FALSE,
DESKTOP_HOOKCONTROL|DESKTOP_READOBJECTS|DESKTOP_WRITEOBJECTS);
// HDESK hDesktop = GetThreadDesktop(GetCurrentThreadId());
if (hDesktop!=0) {
sprintf(szDebug, "OpenInputDesktop succeeded.\n");
OutputDebugString(szDebug);
} else {
sprintf(szDebug, "OpenInputDesktop failed, GetLastError = %u\n",
GetLastError());
OutputDebugString(szDebug);
}
uof.fInherit = TRUE;
uof.fReserved = 0;
uof.dwFlags = DF_ALLOWOTHERACCOUNTHOOK;
bRet = SetUserObjectInformation(hDesktop, UOI_FLAGS, &uof,
sizeof(USEROBJECTFLAGS));
if (bRet!=0) {
sprintf(szDebug, "SetUserObjectInformation succeeded\n");
OutputDebugString(szDebug);
} else {
sprintf(szDebug, "SetUserObjectInformation failed, GetLastError = %u\n",
GetLastError());
OutputDebugString(szDebug);
}

if (hDesktop) {
CloseDesktop(hDesktop);
}

Thanks in advance for any help...

Joseph M. Newcomer

unread,
Oct 13, 2010, 4:37:24 PM10/13/10
to
See below...

On Wed, 13 Oct 2010 15:37:11 -0400, "Nobody" <nob...@nobody.com> wrote:

>I have a problem with SetWindowsHookEx() failing with error code
>ERROR_ACCESS_DENIED(5) when the target process is running as another user.
>In this case, both users are members of the Administrators group. The OS is
>XP Pro+SP2, and I am running it on the console on the same desktop and using
>right-click "Run as" to run the target process.

***
But what are you running as? Administrator? What account is running the hooking process?
****


>
>I am using thread specific hooks(WH_GETMESSAGE) and the hook procedure
>resides in a multithreaded DLL written in VC6. The target process is
>enabling DF_ALLOWOTHERACCOUNTHOOK flag on the desktop and all calls succeed
>when enabling that flag.

****
So you're saying that it works if you set this flag?
****


>
>So what could be causing SetWindowsHookEx() to fail?
>
>If I start the target process as the same user, it works fine, and I see
>messages. The target process is a simple single threaded application that I
>have written for testing. The real application is what I want to automate
>for a customer, and since I don't have the source code for the real process,
>I would use AppInit_DLLs to enable that flag on the real target process.
>Needless to say I have already written that DLL and it loads fine and calls
>to set the flag succeed, but SetWindowsHookEx() still fails, so I am using
>my own simple target process until I found the real cause. Here is the
>output from the code below using DebugView:
>
>[3528] OpenInputDesktop succeeded.
>[3528] SetUserObjectInformation succeeded
>
>I already tried GetThreadDesktop, and OpenDesktop("Default") with the same
>result. Here is the code in the target process which I run the first thing
>when WinMain is called:
>
>// Allow other hooks code
>USEROBJECTFLAGS uof;
>BOOL bRet;
>char szDebug[200];

***
Even in VS6, char was dead as a way of life. You should be using TCHAR and working as if
you might have to compile it as Unicode with no warning whatsoever.
****


>
>HDESK hDesktop = OpenInputDesktop(DF_ALLOWOTHERACCOUNTHOOK, FALSE,
>DESKTOP_HOOKCONTROL|DESKTOP_READOBJECTS|DESKTOP_WRITEOBJECTS);

****
It reads better if you put spaces around the | operators
****


>// HDESK hDesktop = GetThreadDesktop(GetCurrentThreadId());
>if (hDesktop!=0) {
> sprintf(szDebug, "OpenInputDesktop succeeded.\n");

****
You should not be using sprintf, even in VS6. Look at strsafe.h, which works with vs6
****


> OutputDebugString(szDebug);
>} else {
> sprintf(szDebug, "OpenInputDesktop failed, GetLastError = %u\n",
>GetLastError());
> OutputDebugString(szDebug);
>}
>uof.fInherit = TRUE;
>uof.fReserved = 0;
>uof.dwFlags = DF_ALLOWOTHERACCOUNTHOOK;
>bRet = SetUserObjectInformation(hDesktop, UOI_FLAGS, &uof,
>sizeof(USEROBJECTFLAGS));
>if (bRet!=0) {
> sprintf(szDebug, "SetUserObjectInformation succeeded\n");
> OutputDebugString(szDebug);
>} else {
> sprintf(szDebug, "SetUserObjectInformation failed, GetLastError = %u\n",
>GetLastError());
> OutputDebugString(szDebug);
>}
>
>if (hDesktop) {
> CloseDesktop(hDesktop);
>}

****
I'm still confused; does this work, or not work? Given the processes are in the same
desktop, have you tried just opening the process handle?
joe
****


>
>
>
>Thanks in advance for any help...
>
>

Joseph M. Newcomer [MVP]
email: newc...@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

Nobody

unread,
Oct 13, 2010, 5:15:34 PM10/13/10
to
"Joseph M. Newcomer" <newc...@flounder.com> wrote in message
news:pm5cb6h9ci09tr10l...@4ax.com...

> But what are you running as? Administrator? What account is running the
> hooking process?

The hooking process is running as me, and I am a member of the
"Administrators" group. The target process is running as "Administrator"
user, and I just tried a limited user. In both cases, the calls to enable
the flag succeed, but the hooking process gets ERROR_ACCESS_DENIED.

> So you're saying that it works if you set this flag?

No, the hooking process gets ERROR_ACCESS_DENIED whether the target process
sets this flag or not. The hooking process retries every second, and all
attempts get the same error.

> Even in VS6, char was dead as a way of life. You should be using TCHAR
> and working as if
> you might have to compile it as Unicode with no warning whatsoever.

Okay.

> You should not be using sprintf, even in VS6. Look at strsafe.h, which
> works with vs6

Okay.

> I'm still confused; does this work, or not work? Given the processes are
> in the same
> desktop, have you tried just opening the process handle?

See above. I tried OpenProcess(PROCESS_QUERY_INFORMATION) from the hooking
process, and it failed with ERROR_ACCESS_DENIED. I tried running the target
process as "Administrator" user, and limited user. In both cases,
OpenProcess fails with ERROR_ACCESS_DENIED. Strangely though, I have a
service that I made that is running as SYSTEM, and I can open a handle to
it, but this is not the target process that I am talking about, as this
target process would use a dedicated account, not SYSTEM.

Nobody

unread,
Oct 13, 2010, 5:38:09 PM10/13/10
to
Also, I just used Process Explorer to verify that both processes are on the
same desktop, and they are both in "WinSta0\Default", and the same session
ID, Session 0.


Joseph M. Newcomer

unread,
Oct 13, 2010, 6:09:48 PM10/13/10
to
On Wed, 13 Oct 2010 17:15:34 -0400, "Nobody" <nob...@nobody.com> wrote:

>"Joseph M. Newcomer" <newc...@flounder.com> wrote in message
>news:pm5cb6h9ci09tr10l...@4ax.com...
>> But what are you running as? Administrator? What account is running the
>> hooking process?
>
>The hooking process is running as me, and I am a member of the
>"Administrators" group. The target process is running as "Administrator"
>user, and I just tried a limited user. In both cases, the calls to enable
>the flag succeed, but the hooking process gets ERROR_ACCESS_DENIED.
>
>> So you're saying that it works if you set this flag?
>
>No, the hooking process gets ERROR_ACCESS_DENIED whether the target process
>sets this flag or not. The hooking process retries every second, and all
>attempts get the same error.

****
Thanks, that clarifies it. I don't know why it should fail, sadly.
****


>
>> Even in VS6, char was dead as a way of life. You should be using TCHAR
>> and working as if
>> you might have to compile it as Unicode with no warning whatsoever.
>
>Okay.
>
>> You should not be using sprintf, even in VS6. Look at strsafe.h, which
>> works with vs6
>
>Okay.
>
>> I'm still confused; does this work, or not work? Given the processes are
>> in the same
>> desktop, have you tried just opening the process handle?
>
>See above. I tried OpenProcess(PROCESS_QUERY_INFORMATION) from the hooking
>process, and it failed with ERROR_ACCESS_DENIED. I tried running the target
>process as "Administrator" user, and limited user. In both cases,
>OpenProcess fails with ERROR_ACCESS_DENIED. Strangely though, I have a
>service that I made that is running as SYSTEM, and I can open a handle to
>it, but this is not the target process that I am talking about, as this
>target process would use a dedicated account, not SYSTEM.

****
That's odd. You should not be able to hook something in another desktop. But what I'm
missing here is that in your code, you do not show the SetWIndowsHookEx call at all, and
therefore I know no way to tell what is going wrong. SetWindowsHookEx has several
parameters, and you have not shown us any of them. In particular, you did not show how
you obtained the thread ID for the thread you want to hook.
joe
****

Nobody

unread,
Oct 15, 2010, 4:42:52 PM10/15/10
to
"Joseph M. Newcomer" <newc...@flounder.com> wrote in message
news:84bcb6hld237bc46f...@4ax.com...

> On Wed, 13 Oct 2010 17:15:34 -0400, "Nobody" <nob...@nobody.com> wrote:
>>See above. I tried OpenProcess(PROCESS_QUERY_INFORMATION) from the hooking
>>process, and it failed with ERROR_ACCESS_DENIED. I tried running the
>>target
>>process as "Administrator" user, and limited user. In both cases,
>>OpenProcess fails with ERROR_ACCESS_DENIED. Strangely though, I have a
>>service that I made that is running as SYSTEM, and I can open a handle to
>>it, but this is not the target process that I am talking about, as this
>>target process would use a dedicated account, not SYSTEM.
> ****
> That's odd. You should not be able to hook something in another desktop.

Sorry for not replying sooner, the news server I was using wasn't available
yesterday...

I am not hooking the SYSTEM process, just using OpenProcess like you
requested with PROCESS_QUERY_INFORMATION only access, and it succeeds. The
SYSTEM process is a non-interactive service that I made. It's running in
"Service-0x0-3e7$\Default", session 0, while the target process is in
session 0, "WinSta0\Default", and OpenProcess errors out for it. In the
hooking process I don't need to use OpenProcess, SetWindowsHookEx doesn't
require it, just a thread ID.

> But what I'm
> missing here is that in your code, you do not show the SetWIndowsHookEx
> call at all, and
> therefore I know no way to tell what is going wrong. SetWindowsHookEx has
> several
> parameters, and you have not shown us any of them. In particular, you did
> not show how
> you obtained the thread ID for the thread you want to hook.

I am using EnumWindows(), then GetWindowThreadProcessId() on the window I
want. The hook DLL exports an InstallHook function that I call. I made the
DLL print out the ThreadID that it's passing to SetWindowsHookEx, and it's
the correct ThreadID as shown by Spy++(after converting to decimal of
course).

Nobody

unread,
Oct 15, 2010, 4:43:31 PM10/15/10
to
I have found the cause of the problem, and it turned out that I wasn't
enabling DF_ALLOWOTHERACCOUNTHOOK properly. I needed to call
SetThreadDesktop() for some reason for the changes to take effect. Below is
a function that one could call to allow other processes to hook your
process.

// Returns TRUE when successful, FALSE otherwsie
BOOL AllowOthersToHookMe()
{
USEROBJECTFLAGS uof;
BOOL bRet;

HDESK hDesktop = GetThreadDesktop(GetCurrentThreadId());
if (hDesktop==0) {
// GetThreadDesktop failed
return FALSE;


}
uof.fInherit = TRUE;
uof.fReserved = 0;
uof.dwFlags = DF_ALLOWOTHERACCOUNTHOOK;
bRet = SetUserObjectInformation(hDesktop, UOI_FLAGS, &uof,
sizeof(USEROBJECTFLAGS));

if (bRet==0) {
// SetUserObjectInformation failed
CloseDesktop(hDesktop);
return FALSE;
}

bRet = SetThreadDesktop(hDesktop);
if (bRet==0) {
// SetThreadDesktop failed
CloseDesktop(hDesktop);
return FALSE;
}
CloseDesktop(hDesktop);
return TRUE;
}

0 new messages