How to set Windows Hooks?

[Lennart P.]

Hello,

I am confused about how to use the SetWindowsHookEx function through JNA. I am trying to get visible text on an external Windows application through the WH_CALLWNDPROC or WH_CALLWNDPROC hook chains and I would like to do that through JNA and use C++ only as a last resort. 

My thought process so far..

From msdn doc I read that "If the application installs a hook procedure for a thread of a different application, the procedure must be in a DLL.", ok so I must have a DLL which contains the hook procedure. I also found this, a DLLCallback JNA interface, which Javadoc reads "Indicate that the callback needs to appear to be within a DLL. The effective DLL module handle may be obtained by TODO. Use this interface when your callback must reside within a DLL (hooks set via SetWindowsHook, and certain service handlers, for example)." This is where it gets confusing. MSDN says the callback needs to reside in a DLL and the DLLCallback says that if you implement me, the callback will APPEAR to be inside a DLL, but what DLL?

If we look at the SetWindowsHookEx JNA interface there's 4 parameters, 2 of which are

1. HOOKPROC lpfn - Pointer to the hook procedure.
   
2. HINSTANCE hMod - Handle to the DLL containing the hook procedure pointed to by the lpfn parameter.

So If I were to implement the DLLCallback (lpfn) in Java (see code below), what would the hMod parameter point to? It needs to point to a DLL, If i leave it NULL or use "Kernel32.INSTANCE.GetModuleHandle(null);" for hMod I get a 1428 error: ERROR_HOOK_NEEDS_HMOD. 

My attempt to implement the hook procedure: (I am not sure this is 100% correct, especially the )

public static class SWPRETStruct extends Structure {

  public WinDef.LRESULT lResult;
  public WinDef.LPARAM lParam;
  public WinDef.WPARAM wParam;
  public WinDef.UINT message;
  public WinDef.HWND hwnd;

  @Override
  protected List getFieldOrder() {
    return Arrays.asList("lResult", "lParam", "wParam", "message", "hwnd");
  }
}

public interface WinHookProc extends WinUser.HOOKPROC, DLLCallback {
  WinDef.LRESULT callback(int nCode, WinDef.WPARAM wParam, SWPRETStruct hookProcStruct); // how do i know what the signature has to be?
}

public final class EventHookProcedure implements WinHookProc {

  public WinUser.HHOOK hhook;

  @Override
  public WinDef.LRESULT callback(int nCode, WinDef.WPARAM wParam, SWPRETSTRUCT hookProcStruct) {
    // process message

    return user32.CallNextHookEx(hhook, nCode, wParam, hookProcStruct.lParam); // call next hook
  }
}

And initializing the hook

EventHookProcedure proc = new EventHookProcedure();
proc.hhook = user32.SetWindowsHookEx(4 /*WH_CALLWNDPROC*/, proc, Kernel32.INSTANCE.GetModuleHandle(null) /*needs to point to DLL, but what DLL?*/, theThreadIdOfTheHwndIWantToHook);

if (proc.hhook == null) {
  int x = Native.getLastError();
  System.out.println("error: " + x); // prints 1428
}

So to recap. I quess my question is how would I go about using DLLCallback interface to implement WH_CALLWNDPROC or WH_CALLWNDPROC hooks to an external application? Do I need to create custom DLL? If I need a custom DLL what would be in the DLL and how would it be best communicated to Java?

Kind regards,

Lennart

[Timothy Wall]

The DLLCallback functionality was never completely implemented, since it would require the DLL-resident callback to include code to initialize the JVM prior to actually calling the function

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.