JNA Com support.

[David Akehurst]

Hi,

I have been doing quite a lot to improve the COM support in JNA.

Is anyone else working on this?

cheers

[wolf....@gmx.net]

Hi David,

I´ve introduced the COM support initially into JNA. 

Good to have you on board!

cheers

Tobi

[David Akehurst]

It's on a github fork at present. 

Once I've tested it a bit more i'll issue a pull request.

If you want an early look check my fork. 

--
You received this message because you are subscribed to a topic in the Google Groups "Java Native Access" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jna-users/Ub1uEiMgIFA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jna-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Doubrovkine]

That'd be great! I am happy to spend some time doing code reviews and such.

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

dB. | Moscow - Geneva - Seattle - New York
code.dblock.org - @dblockdotorg - artsy.net - github/dblock

[David Akehurst]

here is my fork

https://github.com/dhakehurst/jna

please feel free to email me and thoughts, suggestions, or improvements.

We will be using this quite a lot in the future, so I hope that my modifications can get back into the original code base.

[Daniel Doubrovkine]

Start making small-ish pull requests out of this. We'll help you through the process.

[Tobias Wolf]

Hi David,

 

I has a quick check to your fork. Great work! 

Just am idea came out of me ;-). Would you be so kind to change the MS Office example to reflect your imrovements?

What do you think?

 

regards

Tobi

 

Gesendet: Mittwoch, 29. Oktober 2014 um 11:34 Uhr
Von: "David Akehurst" <da...@akehurst.net>
An: "jna-...@googlegroups.com" <jna-...@googlegroups.com>
Betreff: Re: JNA Com support.

[David Akehurst]

Done for MsWord, will do Excel bit later....unless you fancy having a go at it :-)

[David Akehurst]

What is a 'small pull request' ?
I can create a "pull request", not sure or how to change the size of it.


Also, I see that JNA is released under dual licence, LGPG and Apache from version 4.

The copyright headers in the code files only mentions LGPL,
- Is this simply legacy and will be changed over time?
- how should the individual headers of new files mentions copyright? just LGPL or both?


thanks

[wolf....@gmx.net]

Hi David,

I think your work to add annotations is a good improvement!

But as far as I know the support for ROT is for getObject to get a already running instance for e.g. MS Word to use that instance. 

These things you need to write a new COM object. The current COM implementation so far was only for using existing COM objects.

Are you planning to provide support for writing COM objects in JNA/Java?

I provided a java main class for generating Java code out of a typelib. Is is possible for you to add the annotation support also to that generator (TlbImp.java)?

To unsubscribe from this group and all its topics, send an email to jna-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

dB. | Moscow - Geneva - Seattle - New York
code.dblock.org - @dblockdotorg - artsy.net - github/dblock

--
You received this message because you are subscribed to a topic in the Google Groups "Java Native Access" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jna-users/Ub1uEiMgIFA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jna-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Akehurst]

Hi,

I am not planing to provide support for writing a COM object in JNA/Java, but we do need to interact with various COM objects.

We need the following 6 things,

1) get an existing running instance of a COM application/Object

2) create/start a new instance of a COM application/object

3) look at all the currently running instances of COM objects and pick one based on various properties of the COM object.

4) Call/invoke various COM properties and methods.

5) Support for receiving COM events.

6) An active OSS community

We looked at a number of different java COM libraries (Com4J, Jacob, J-interop and JNA). None of them gave all these 6 things. We picked JNA because it seemed to have the most active user/developer community, and liked the way that it has a minimal native code part.

However JNA didn't support all these three things, hence I have been implementing what is missing, both the low level calls, and a nicer 'utils' more Java like way to use COM via JNA.

Out requirements are (will be) answered by JNA as follows

1) 'GetActiveObject' is a low level C call that get a pointer to an active running instance of a COM object. If there is more than one object running, it will arbitrarily give one of them, you need to iterate over the RunningObjectTable if you need a specific one. 

2) 'CoCreateInstance' is a low level C call to create a new COM object. The ....COM.utils.Factory class wraps the GetActiveObject and CoCreateInstance calls so that they work nicely with the annotated interfaces approach.

3) The RunningObjectTable support I have added, starts on the low level C call 'GetRunningObjectTable'. The interfaces and object added to the ...COM package implement the low level interface functions require to iterate over the running objects. There are also nicer java wrappers around this in the ....COM.util package.

4) The call/invoke properties was already provided by the existing ...COM.Dispatch class and the ....COM.COMBindingBaseObject. I have used these as a basis for the annotation and ProxyObject classes in the COM.utils package. This makes it very easy to implement Java based COM interfaces.

5) Support for receiving COM events is more complex, other Java COM libraries do this in native code. I think I have provided a way to do this based on JNA, but am still working on it / testing it.

6) The JNA community appears to be more active than the others

(sorry I do not at present have time to improve the TlbImp part, the event callbacks is more important to our work, and is my current focus.)

[David Akehurst]

On Tuesday, 4 November 2014 13:32:30 UTC+1, David Akehurst wrote:

5) Support for receiving COM events is more complex, other Java COM libraries do this in native code. I think I have provided a way to do this based on JNA, but am still working on it / testing it.

OK, so I think I'm going to need some advice about how the callback mechanism works.
What I have read about JNA callbacks implies that they occur on their own thread, however, in what I am doing, the thread is not visible in the Java debugger, and they appear to occur on the main thread.

I will explain what I am trying to achieve.

To get a callback event from a COM object we have to make a COM call ConnectionPoint::Advise(IUnknown* pUnkSink, DWORD* pdwCookie).
Using JNA, I can get a ConnectionPoint object, and can call this method.
The pointer to an IUnknown object, pUnkSink, is an object that will be called back by the COM object with event notifications.
In reality one can pass a pointer to an IDispatch object (which inherits from IUnknown) and this same object is used to handle the event callbacks.
I can construct the equivalent of a C++ object, in C, by using a struct with a vtable pointer to function pointers.
I appear to be able to do this using JNA also, (see code below).

The IUnknown methods ("QueryInterface", "AddRef", "Release") are all successfully calledback as part of the initial Advise call (on the same thread according to the Java debugger),
however, at the end of the sequence of QueryInterface calls that are made, Word appears to 'hang' as though its main thread is busy.

The Java program is happy to continue, however, it of course doesn't get any events, because Word is "on hold".
When the Java program exits, Word happily continues with no issue.

I can replicate the same program using C/C++, including artificial construction of the IDispatch object using structs.
Everything works fine, but the calls to QueryInterface (after Advise), appear to be on different threads,
and COM events are successfully received - on a different thread.

Any help or explanation about callback threads or anything else gratefully received :-)

DispatchListerner and Vtable "Structures" shown below:


public class DispatchListener extends Structure {
    public DispatchListener(IDispatchCallback callback) {
        this.vtbl = this.constructVTable();
        this.initVTable(callback);
        super.write();
    }
    public DispatchVTable.ByReference vtbl;
   
    @Override
    protected List<String> getFieldOrder() {
        return Arrays.asList(new String[] { "vtbl" });
    }
   
    protected DispatchVTable.ByReference constructVTable() {
        return new DispatchVTable.ByReference();
    }
   
    protected void initVTable(final IDispatchCallback callback) {
        this.vtbl.QueryInterfaceCallback = new DispatchVTable.QueryInterfaceCallback() {
            @Override
            public HRESULT invoke(Pointer thisPointer, REFIID.ByValue refid, PointerByReference ppvObject) {
                return callback.QueryInterface(refid, ppvObject);
            }
        };
        this.vtbl.AddRefCallback = new DispatchVTable.AddRefCallback() {
            @Override
            public int invoke(Pointer thisPointer) {
                return callback.AddRef();
            }
        };
        this.vtbl.ReleaseCallback = new DispatchVTable.ReleaseCallback() {
            @Override
            public int invoke(Pointer thisPointer) {
                return callback.Release();
            }
        };
        this.vtbl.GetTypeInfoCountCallback = new DispatchVTable.GetTypeInfoCountCallback() {
            @Override
            public HRESULT invoke(Pointer thisPointer, UINTByReference pctinfo) {
                return callback.GetTypeInfoCount(pctinfo);
            }
        };
        this.vtbl.GetTypeInfoCallback = new DispatchVTable.GetTypeInfoCallback() {
            @Override
            public HRESULT invoke(Pointer thisPointer, UINT iTInfo, LCID lcid, PointerByReference ppTInfo) {
                return callback.GetTypeInfo(iTInfo, lcid, ppTInfo);
            }
        };
        this.vtbl.GetIDsOfNamesCallback = new DispatchVTable.GetIDsOfNamesCallback() {
            @Override
            public HRESULT invoke(Pointer thisPointer, REFIID.ByValue riid, WString[] rgszNames, int cNames, LCID lcid,
                    DISPIDByReference rgDispId) {
                return callback.GetIDsOfNames(riid, rgszNames, cNames, lcid, rgDispId);
            }
        };
        this.vtbl.InvokeCallback = new DispatchVTable.InvokeCallback() {
            @Override
            public HRESULT invoke(Pointer thisPointer, DISPID dispIdMember, REFIID.ByValue riid, LCID lcid, WORD wFlags,
                    DISPPARAMS.ByReference pDispParams, VARIANT.ByReference pVarResult, EXCEPINFO.ByReference pExcepInfo,
                    IntByReference puArgErr) {

                return callback.Invoke(dispIdMember, riid, lcid, wFlags, pDispParams, pVarResult, pExcepInfo, puArgErr);
            }
        };
    }

}

public class DispatchVTable extends Structure {
    public static class ByReference extends DispatchVTable implements Structure.ByReference {
    }

    public QueryInterfaceCallback QueryInterfaceCallback;
    public AddRefCallback AddRefCallback;
    public ReleaseCallback ReleaseCallback;
    public GetTypeInfoCountCallback GetTypeInfoCountCallback;
    public GetTypeInfoCallback GetTypeInfoCallback;
    public GetIDsOfNamesCallback GetIDsOfNamesCallback;
    public InvokeCallback InvokeCallback;

    @Override
    protected List<String> getFieldOrder() {
        return Arrays.asList(new String[] { "QueryInterfaceCallback", "AddRefCallback", "ReleaseCallback","GetTypeInfoCountCallback", "GetTypeInfoCallback",
                "GetIDsOfNamesCallback", "InvokeCallback" });
    }

    public static interface QueryInterfaceCallback extends StdCallLibrary.StdCallCallback {
        WinNT.HRESULT invoke(Pointer thisPointer, REFIID.ByValue refid, PointerByReference ppvObject);
    }

    public static interface AddRefCallback extends StdCallLibrary.StdCallCallback {
        int invoke(Pointer thisPointer);
    }

    public static interface ReleaseCallback extends StdCallLibrary.StdCallCallback {
        int invoke(Pointer thisPointer);
    }
   
    public static interface GetTypeInfoCountCallback extends StdCallLibrary.StdCallCallback {
        WinNT.HRESULT invoke(Pointer thisPointer, UINTByReference pctinfo);
    }

    public static interface GetTypeInfoCallback extends StdCallLibrary.StdCallCallback {
        WinNT.HRESULT invoke(Pointer thisPointer, UINT iTInfo, LCID lcid, PointerByReference ppTInfo);
    }

    public static interface GetIDsOfNamesCallback extends StdCallLibrary.StdCallCallback {
        WinNT.HRESULT invoke(Pointer thisPointer, REFIID.ByValue riid, WString[] rgszNames, int cNames, LCID lcid,
                DISPIDByReference rgDispId);
    }

    public static interface InvokeCallback extends StdCallLibrary.StdCallCallback {
        WinNT.HRESULT invoke(Pointer thisPointer, DISPID dispIdMember, REFIID.ByValue riid, LCID lcid, WORD wFlags,
                DISPPARAMS.ByReference pDispParams, VARIANT.ByReference pVarResult, EXCEPINFO.ByReference pExcepInfo,
                IntByReference puArgErr);
    }
}

[Timothy Wall]

Here there be dragons :) Having worked a fair bit with COM event notifications myself way back, it’s not for the faint of heart.

When a JNA callback gets invoked, it checks whether the current thread is attached to the JVM and attaches it if it isn’t (later detaching it if it wasn’t originally attached). There are some callback thread handling options you can set (daemon, whether to detach on exit, etc) that you can set per-callback or a few other ways.

You might consider having the callbacks post their to some sort of queue and immediately return, to avoid tying up whatever thread the events are being delivered on.

You might also need to explicitly run an event pump (see the contrib examples of low-level hooks).

Just educated guessing here, YMMV.

[David Akehurst]

Thanks.

After a little more investigation, namely attaching Visual Studio debugger to Word.

It seems that Word has hung on a call to  something like Dispatch::RemoteInvoke.

which says to me that the JNA code is not 'receiving' processing the call but somehow is blocking it!

Should a native thread create a Java one, and use that to process the JNA callback? 

You received this message because you are subscribed to a topic in the Google Groups "Java Native Access" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jna-users/Ub1uEiMgIFA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jna-users+...@googlegroups.com.

[David Akehurst]

Event Pump needed !

as described in this post https://groups.google.com/forum/#!searchin/jna-users/User32.INSTANCE.TranslateMessage/jna-users/pQYazrJn2Dg/AXRla7cGFW0J
(also something similar in the contrib\native_window_msg example)

it seems to be necessary to have this code running,

            WinUser.MSG msg = new WinUser.MSG();
            while (((User32.INSTANCE.GetMessage(msg, null, 0, 0)) != 0)) {
                User32.INSTANCE.TranslateMessage(msg);
                User32.INSTANCE.DispatchMessage(msg);
            }

I guess this is what you mean by an 'event pump'.

I don't really understand why! and why it is unnecessary in the C version.
Though perhaps in the C version, this pump is automatically started by one of the windows DLLs - maybe ntdll.dll, which is running some threads.

A step forwards anyhow, thanks for the event pump tip off.

[Timothy Wall]

I would suggest having the Java callback do as little as possible (basically forward whatever data it receives onto a processing queue) and return as quickly as possible.

[Timothy Wall]

Yeah, I’ve never been clear why an additional pump is needed. Likely something to do with the context in which the native thread is running, you need the explicit call from the JVM context in order for things to get moving.

I’ve never been able to find sufficient documentation for how the DLL context is managed. The most docs I’ve found are for hook callbacks requiring the callback code to reside within a DLL (which is why you can’t use JNA callbacks for regular hooks).

Note that the pump thread never actually returns from the first GetMessage call.

[Daniel Doubrovkine]

By small pull request I mean something digestable to the reviewer and valuable to everyone without the whole massive code change that you have there, I would start with new Win32 mappings that you have that work and have tests.

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Akehurst]

Sorry, I can't see how to get github to produce a pull request of only some of the files that have been committed?

please explain how to do this,
(apologies if I have missed something obvious)

[Daniel Doubrovkine]

I tend to cherry pick things by hand. 

I know it's a lot of work, but it's otherwise a lot of work for the reviewers. Either way, do the best you can and we'll do the best on our side.

[David Akehurst]

This article

http://www.codeguru.com/cpp/com-tech/activex/apts/article.php/c5533/Understanding-COM-Apartments-Part-II.htm

I think explains why the event pump is needed.