BSTRByReference: Invalid memory access

[Markus Karg]

I have trouble with one particular COM invocation (for demonstration purposes solely, here I do run it in an infinite loop):

while (true) {
    var versionRef = new BSTRByReference();
    comObject.qsstatVersion(handle, versionRef);
    var version = versionRef.getString();
    System.out.println(version);
}

This loop works perfectly and prints the always same String on screen for several iterations, then suddenly throws an exception instead:

Exception in thread "main" java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
        at com.sun.jna.platform.win32.COM.util.Factory.runInComThread(Factory.java:187)
        at com.sun.jna.platform.win32.COM.util.Factory.access$100(Factory.java:56)
        at com.sun.jna.platform.win32.COM.util.Factory$ProxyObject2.invoke(Factory.java:95)
        at jdk.proxy1/jdk.proxy1.$Proxy6.qsstatVersion(Unknown Source)
        at de.quipsy.sandbox.qcom.Main.main(Main.java:46)
Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
        at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
        at com.sun.jna.platform.win32.COM.util.ComThread.execute(ComThread.java:155)
        at com.sun.jna.platform.win32.COM.util.Factory.runInComThread(Factory.java:172)
        ... 4 more
Caused by: java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:119)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at com.sun.jna.platform.win32.COM.util.Factory$ProxyObject2$1.call(Factory.java:98)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1589)
Caused by: java.lang.Error: Invalid memory access
        at com.sun.jna.Native.invokeInt(Native Method)
        at com.sun.jna.Function.invoke(Function.java:429)
        at com.sun.jna.Function.invoke(Function.java:364)
        at com.sun.jna.Function.invoke(Function.java:318)
        at com.sun.jna.Function.invoke(Function.java:309)
        at com.sun.jna.platform.win32.COM.COMInvoker._invokeNativeObject(COMInvoker.java:48)
        at com.sun.jna.platform.win32.COM.Dispatch.Invoke(Dispatch.java:145)
        at com.sun.jna.platform.win32.COM.util.ProxyObject.oleMethod(ProxyObject.java:726)
        at com.sun.jna.platform.win32.COM.util.ProxyObject.invokeMethod(ProxyObject.java:450)
        at com.sun.jna.platform.win32.COM.util.ProxyObject.invokeMethod(ProxyObject.java:433)
        at com.sun.jna.platform.win32.COM.util.ProxyObject.invoke(ProxyObject.java:259)
        at jdk.proxy1/jdk.proxy1.$Proxy6.qsstatVersion(Unknown Source)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        ... 6 more

As the COM server implements this method by returning a static C string value always, I wonder if my program is incorrect, if the COM server has a bug, or if JNA has a bug?! 🤔

Any ideas welcome!

[Markus Karg]

Update: I did the very same using JACOB instead of JNA, but there it works like a charm! Hence I assume a bug in JNA. Can somebody please confirm this?

[Matthias Bläsing]

Hi,

Am Donnerstag, dem 25.01.2024 um 08:03 -0800 schrieb Markus Karg:

This loop works perfectly and prints the always same String on screen for several iterations, then suddenly throws an exception instead:

you might see GC effects. Normally JNA using code is not called so hot, that it will be compiled/inlined the JVM might inline the Function pointer and release the surrounding object, which will be a problem.

Greetings

Matthias

[Markus Karg]

The original code has no loop at all, and I discovered the problem just by incident still. So I doubt it is a GC effect, as in some of my test runs it actually failed already at iteration #1, also (so nothing "hot" here, so JVM would not even try to inline), and I even had test runs where after ten iterations the JVM process itself crashed (Azul OpenJDK v19). In fact I solely added the loop to enforce the crash as a demonstration here in the user forum.

What do you need from me to further investigate? Again, on JACOB the same is rock solid, so I am pretty sure it is a bug in JNA.

[Matthias Bläsing]

Hi,

Am Donnerstag, dem 25.01.2024 um 10:15 -0800 schrieb Markus Karg:

What do you need from me to further investigate? Again, on JACOB the same is rock solid, so I am pretty sure it is a bug in JNA.

I'd need a standalone reproducer. Then I can take a look.

Greetings

Matthias

[Markus Karg]

I do understand your claim very well as randomly happening bugs are always a nightmare to track, and I am willing to help with everything I can. But the problem is that the reproducer would imply installing the called native Win32 application (multi GB suite) on your 64 Bit Windows desktop. Is that the road you want to go down, or can we work together to boil this down to the actual cause, as I do have it installed on my development laptop already? If this helps, I am a professional software developer for 30+ years, and I am an open source committer for 20+ years, including being a contributor to OpenJDK for approx. 5 years, so "we speak the same language"). We also can share my screen (we can offer Teams, we can offer German if this helps). If this helps, we could also offer a bug bounty for this. WDYT?

For the problem itself, some more (possibly helpful) facts:

* It works like a charm on JACOB, even in "hot" loops. Hence I tend to assume that neither the calling application nor the called application are the problem here, but "something inside JNA" is.

* It fails not only for this single method, it apparently fails for all methods using VT_BSTR | VT_BYVAR variant parameters (non-byvar BSTR parameters do work pretty well).

* It even fails if we explicitly set the type and object of the VARIANT.

* It even fails when we put a BSTR instance as an initialization value into BSTRByReference, initially holding 16k of free space.

* The problem typically happens within 1...5 iterations already (hence even in non-optimized execution).

[Timothy Wall]

I would suggest doing everything you can to remove GC as a culprit.  That would mean keeping hard references to anything involved in the operation (com object - including checking com object ref count, byref object, byref object pointer, etc).

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/7b3e0427-2260-43bc-8bfb-76f5ac4ac95bn%40googlegroups.com.

[Support]

On Fri, 2024-01-26 at 09:15 -0500, 'Timothy Wall' via Java Native Access wrote:

I would suggest doing everything you can to remove GC as a culprit.  That would mean keeping hard references to anything involved in the operation (com object - including checking com object ref count, byref object, byref object pointer, etc).

And or disabling/delaying GC as much as you can via JVM options.

[Matthias Bläsing]

Hi,

Am Freitag, dem 26.01.2024 um 00:53 -0800 schrieb Markus Karg:

* It fails not only for this single method, it apparently fails for all methods using VT_BSTR | VT_BYVAR variant parameters (non-byvar BSTR parameters do work pretty well).

ok, the lets have a look. The interesting bits are in com.sun.jna.platform.win32.COM.util.ProxyObject.invokeMethod(Class<T>, DISPID, Object[])#invokeMethod. In that method the java arguments are wrapped into VARIANTS as required by the COM dispatch interface.

The wrapping itself happens in com.sun.jna.platform.win32.COM.util.Convert#toVariant and for the "VT_BSTR | VT_BYVAR" this directly invokes the corresponding VARIANT constructor. _VARIANT#pbstrVal  is a pointer to a BSTR and in the empty case just null. After the call that member is filled with the pointer to the resulting BSTR. After the native call the allocations for the method need to be potentially freed. That is handled in Convert#free/Convert#toJavaObject. pbstrVal should not be modified by this as the only data "owned" by the VARIANT is the pointer and that is inline in the structure.

It might be worth checking what happens in the cleanup path. BSTR values are special cased as they can be unwrapped to String objects and then they need to be freed using SysFreeString, if they are returned as raw BSTR, they are not freed as the callsite now owns the value and need to clear it. If that clearing path is hit in your case, this might explain the problem as the loop will basicly result in a use-after-free.

Maybe single stepping through this, will reveal strange invocations. That would be my approach when running locally.

Maybe this helps to get started with debugging.

Matthias

[Daniel Widdis]

There are only two pointer values involved here: the pointer to the JNA-side memory allocation in BSTRByReference, and the pointer to the native-side string which is placed in this allocation.

The JNA-side pointer is part of versionRef which is accessed after the COM invocation, so there shouldn't be any issues with reachability or early GC.  This narrows down the issue to the pointer to the native BSTR.

In the intial post you said this native return value was a static C string value.  BSTR assumes the value is a UTF-16LE value with the length in the 4 bytes preceding the pointer value. . Can you confirm that the value assigned to the second argument of comObject.qsstatVersion() is a pointer to a UTF-16LE string whose length is in the preceding 4 bytes?

[Daniel Widdis]

Another thought: I see another thread on this mailing list with the subject "32 Bit COM Server called by 64 Bit Java".   Is this the server in question?  In this case, the value set in the ByReference pointer would only write 32 bits of its 64-bit allocation.  Is this the case here?  

On Friday, January 26, 2024 at 10:52:22 AM UTC-8 mbla...@doppel-helix.eu wrote:

[Markus Karg]

Matthias,

thank you for this first steps and sorry for the delay. I am quite busy in a different project currently, but I found the time to setup a build environment (which builds but unfortunately fails the Unicode Load Test -- see separate topic).

Amending your idea, and basing on the comment below found in the source code of BSTRByReference...

        /**
         * Store a reference to the specified {@link BSTR}. This method does not
         * maintain a reference to the object passed as an argument. The user is
         * responsible for allocating and freeing the memory associated with this
         * {@link BSTR}.
         *
         * @param value
         *            The BSTR to be referenced. Only the pointer is stored as a
         *            reference.
         */

(https://github.com/java-native-access/jna/blob/7da005e8f2ddb7bbe6488418a1bc302c0c67b4e5/contrib/platform/src/com/sun/jna/platform/win32/WTypes.java#L176-L185)

...I actually do assume (I can't proof it as the COM server is a closed-source product, so I can't check its source code) that this particular COM server fills in his BSTR into the by-ref VARIANT it received, then he immediately frees his BSTR again (i. e. the BSTR only exists in the COM server for the short time of passing it into the by-ref VARIANT). In that case, JNA will definitively use an invalid pointer from this very instant, because the BSTR code in JNA is not storing the string content, but just the pointer to the contained wchars, and that pointer is pointing to free'd memory now! So whether or not my assumption over the COM server's behavior actually is true or not, that will be definitively wrong in JNA IMHO.

JNA's test suite cannot detect this situation, because it keeps the BSTR valid for the full time of the test (not just for the time of passing its value into the BSTRByReference):

  if (Platform.isWindows()) {
      BSTR b = OleAuto.INSTANCE.SysAllocString("bstr");
      BSTRByReference bstrbr = new BSTRByReference(b);
      parseAndTest(bstrbr.toString(), "BSTR", "bstr");
      OleAuto.INSTANCE.SysFreeString(b); // <-- IMHO if this happens BEFORE parseAndTest, eventually the same crash might happen
  }

(https://github.com/java-native-access/jna/blob/7da005e8f2ddb7bbe6488418a1bc302c0c67b4e5/contrib/platform/test/com/sun/jna/platform/ByReferencePlatformToStringTest.java#L90-L95)

(Even if we would move the SysFreeString before the parseAndTest, Windows eventually would allow reading the memory, at least as long as nobody else claimed it IMHO.)

Could you please confirm this conclusion?

[Markus Karg]

Yes, this is a 32 Bit COM Server (EXE file running as a Windows Service) called from a 64 Bit Java JNA application. IIUC then you assume that Java is reading 64 Bits of which 32 Bit are correctly sent by the client, but the other 32 Bit are pointing to random bits?

[Markus Karg]

Just noticed that I cannot reproduce the problem anymore once I changed the following in my calling application:

var bstr = OleAuto.INSTANCE.SysAllocString(""); <-- Added this one...
try {
    var ref = new BSTRByReference(bstr); <-- ...and gave it into the By-Ref, even if this is an OUT (not an IN-OUT) COM parameter...
    tQsstatRemoteControl.qsstatVersion(handle, ref);
    return ref.getString();
} finally {
    // OleAuto.INSTANCE.SysFreeString(bstr); <-- ...but NOT doing this one (as it would reproduce the crash again!)
}

While I am happy that it does not crash anymore, I wonder why this is working?

• Why do I have to provide an empty BSTR to an OUT (not IN-OUT) parameter (or in other words, for what does BSTRByRef have an empty constructor if an init value is always needed)?
  
• Who is freeing that BSTR actually? Apparently someone IS freeing it, as the crash produced by SysFreeString proofs (that's why I commented it out). The called server definitively is NOT doing that, because this is an OUT (not an IN-OUT) parameter!

I also wonder if I should call SysFreeString(ref.getValue()) to prevent memory leaks?

On Friday, January 26, 2024 at 7:52:22 PM UTC+1 mbla...@doppel-helix.eu wrote:

[Matthias Bläsing]

Hi Markus,

I think this might help to clear the rules:

https://learn.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2012/xda6xzx7(v=vs.110)?redirectedfrom=MSDN

For the why is it freed? I have to speculate:

You have multi wrappings, that might indicate an empty value:

1. A VARIANT can carry null (VT_NULL) or mark itself as empty (VT_EMPTY)

2. In case of a VT_BYREF the wrapped pointer can be a null pointer

So with your observations, what might work better is:

1. Allocate a VARIANT with VT_EMPTY yourself

2. Pass that to qsstatVersion

3. Now the runtime has no chance to free anything, because it would violate the VARIANT contract

4. The result should be placed in the VARIANT

5. Now you retrieve the value with `stringValue`

6. Finally you clean up after yourself using VariantClear

Greetings

Matthias

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/5fb673a7-f8c7-4bd1-8923-114f34b01cb7n%40googlegroups.com.

[Daniel B. Widdis]

IIUC then you assume that Java is reading 64 Bits of which 32 Bit are correctly sent by the client, but the other 32 Bit are pointing to random bits?

 

Precisely.  Specifically the “ByReference” in JNA assumes the pointer size of the JVM on the JNA side (8 bytes).  It is passing an 8-byte allocation to the COM server.  Here is the allocation in the ByReference superclass that BSTRByReference extends:

 

protected ByReference(int dataSize) {

    setPointer(new Memory(dataSize));

}

 

Of note, new Memory(size) does not clear those bytes so they can contain anything.

 

The COM server is faithfully filling 4 of those bytes.

 

Ultimately you shouldn’t be using BSTRByReference on a 64 bit JVM to map this data type.  You can simply use an IntByReference to fetch the BSTR pointer, then use similar methods to the BSTR internals to fetch the string from that pointer value (get the length from pointer-4 and read it from pointer value).

--

You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/32ccc10f-0e63-4c97-80bd-d02d18d82d2an%40googlegroups.com.

[Markus Karg]

I see what you mean (you imply that the COM server actually is not compliant with Microsoft's rules, right?). I think this is what you propose:

var ref = new VARIANT();
try {
    ref.setVarType((short) VT_EMPTY);
    tQsstatRemoteControl.qsstatVersion(handle, ref);
    return ref.stringValue();
} finally {
    OleAuto.INSTANCE.VariantClear(ref);
}

Unfortunately ref.stringValue() always returns null then. If I instead use VT_EMPTY | BY_REF then JNA says: com.sun.jna.platform.win32.COM.COMInvokeException: An den Stub wurde ein Nullzeiger übergeben.(HRESULT: 800706f4)

[Daniel Widdis]

The COM server is compliant.  It’s just 32-bit. 

You could run a 32-bit JVM and JNA would work perfectly as well. 

The issue is that the ByReference classes use pointer size to allocate memory and if that differs you have to account for it. 

Sent from my iPhone

On Feb 8, 2024, at 12:36 AM, Markus Karg <ka...@quipsy.de> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/6959a1b0-29bd-4151-b269-c55c1cc43b83n%40googlegroups.com.

[Matthias Bläsing]

Hi,

Am Donnerstag, dem 08.02.2024 um 00:36 -0800 schrieb Markus Karg:

I see what you mean (you imply that the COM server actually is not compliant with Microsoft's rules, right?). I think this is what you propose:

var ref = new VARIANT();
try {
    ref.setVarType((short) VT_EMPTY);
    tQsstatRemoteControl.qsstatVersion(handle, ref);
    return ref.stringValue();
} finally {
    OleAuto.INSTANCE.VariantClear(ref);
}

Unfortunately ref.stringValue() always returns null then. If I instead use VT_EMPTY | BY_REF then JNA says: com.sun.jna.platform.win32.COM.COMInvokeException: An den Stub wurde ein Nullzeiger übergeben.(HRESULT: 800706f4)

- what does "System.out.println(ref.getValue())" show?

- what does "System.out.println(ref.getVarType())" show?

If I remember correctly, there were cases, where VT_EMPTY failed, but VT_NULL worked, would be worth testing.

Greetings

Matthias

[Markus Karg]

IIUC this is what you want me to do:

var ref = new VARIANT();
try {
    ref.setVarType((short) VT_EMPTY);
    tQsstatRemoteControl.qsstatVersion(handle, ref);

    System.out.println(ref.getValue());
    System.out.println(ref.getVarType());
    return ref.stringValue();
} finally {
    OleAuto.INSTANCE.VariantClear(ref);

}

Using VT_EMPTY the result is:

null

0

Using VT_NULL the result is:

com.sun.jna.platform.win32.COM.COMInvokeException: Typenkonflikt.(HRESULT: 80020005) (puArgErr=0)
        at com.sun.jna.platform.win32.COM.COMUtils.checkRC(COMUtils.java:187)

In both cases (VT_NULL just as VT_EMPTY) or-ing VT_BYREF produces this:

com.sun.jna.platform.win32.COM.COMInvokeException: An den Stub wurde ein Nullzeiger übergeben.(HRESULT: 800706f4)

        at com.sun.jna.platform.win32.COM.COMUtils.checkRC(COMUtils.java:187)

(NB: Calling ref.getVarType() before calling qsstatVersion in fact sets the VARIANT's type to VT_EMPTY always, as it internally performs this.read(). If this is wanted behavior, it should be documented in the JavaDocs, as it is rather unexpected that a getter modifies internal state.).

[Matthias Bläsing]

Ok, to get this more open. I used my google foo and finally found the software we are talking about:

Hexagon Q-DAS qs-STAT

That leads me to the download site:

https://training.q-das.de/de/service/software-downloads

And there I find:

ComClient-Examples (ZIP 5.37 MB)

That actually might help because

a) there is a .tlb inside, that can be used with oleview to actually see the definitions

b) there is a C++ sample. C would be nicer, but this might already help

In the samples I find VCPPCOMClient1View.cpp, lines 2445-2460. In contrast to the second example, I think line 2448 does the sane thing: It initalizes the BSTR to the null pointer.

That might be helpful.

new BSTRByRef()

creates an an object holding a pointer that points to a pointer sized memory region.

https://github.com/java-native-access/jna/blob/7da005e8f2ddb7bbe6488418a1bc302c0c67b4e5/contrib/platform/src/com/sun/jna/platform/win32/WTypes.java#L173

https://github.com/java-native-access/jna/blob/7da005e8f2ddb7bbe6488418a1bc302c0c67b4e5/src/com/sun/jna/ptr/ByReference.java#L56-L58

The ByReference constructor allocates memory, but it does not zero it. I don't know why the COM environment should do this, but it could interpret the non-zero value as a string to be freed. So you could try to explicitly zero it.

while (true) {
    var versionRef = new BSTRByReference(new BSTR());

    comObject.qsstatVersion(handle, versionRef);
    var version = versionRef.getString();
    System.out.println(version);
}

The parameterless BSTR constructor wraps a null pointer. The code just switches from "random value" initialization to zero initialization of the BSTRByReference.

Given, that the VCPP samples do not free the value passed in and given that a version string is constant, it seems sensible, that the result value should not be freed.

Greetings

Matthias

[Markus Karg]

First of all, thank you so much Matthias, for investing your personal time into our troubles with QS-STAT! Your support is definitively outstanding and we really do appreciate your help! 🦸‍♂️

I did not even know that there still is a public download site of this legacy closed-source product, but yes, that is exactly the product we have to "talk" to from our 64 Bit Java application. Regarding the TLB and C++ source code we need to see that the binary product itself is 32 Bit only, so I assume that COM internally is using DCOM to connect an auto-created 64 Bit stub with the actual 32 Bit product. Hence long in the TLB actually means "32 Bit" (I only have a 32 Bit TLB viewer so I cannot see whether a 64 Bit TLB viewer would actually learn anything from this truth).

I can confirm that the proposed code change indeed works like a charm! I modified all uses of BSTRByReference and and none of them is failing! 🚀  So apparently it seems that BSTRByReference(new BSTR()) is what the QS-STAT product expects us to do in the VT_BSTR | VT_BYREF case always, instead of simply BSTRBYReference(). So the question is: Is the difference between BSTRByReference(new BSTR()) and BSTRByReference() a bug in JNA, or is QS-STAT's expectation a violation of Microsoft's COM rules? 🤔

Anyways, we still have to find a solution for DoubleByReference always returning zero... 😉

[Matthias Bläsing]

Am Sonntag, dem 11.02.2024 um 23:48 -0800 schrieb Markus Karg:

Hence long in the TLB actually means "32 Bit" (I only have a 32 Bit TLB viewer so I cannot see whether a 64 Bit TLB viewer would actually learn anything from this truth).

long in the Win32 API is 32bit. It does not matter which architecture is involved:

https://en.wikipedia.org/wiki/64-bit_computing#64-bit_data_models

See difference between LLP64 and LP64.

I can confirm that the proposed code change indeed works like a charm! I modified all uses of BSTRByReference and and none of them is failing! 🚀  So apparently it seems that BSTRByReference(new BSTR()) is what the QS-STAT product expects us to do in the VT_BSTR | VT_BYREF case always, instead of simply BSTRBYReference(). So the question is: Is the difference between BSTRByReference(new BSTR()) and BSTRByReference() a bug in JNA, or is QS-STAT's expectation a violation of Microsoft's COM rules? 🤔

This is C so most probably someone will invoke "undefined behavior" and that ends the matter.

Anyways, we still have to find a solution for DoubleByReference always returning zero... 😉

You never showed the invocation or what method is invoked. Please do so.

Greetings

Matthisa

[Daniel B. Widdis]

Is the difference between BSTRByReference(new BSTR()) and BSTRByReference() a bug in JNA, or is QS-STAT's expectation a violation of Microsoft's COM rules

Option C: None of the above.

JNA is not designed to use different pointer size than the operating system, and when interacting with another program it is up to the user to identify appropriate requirements for using its API; in fact, there's no way for JNA to know (without some help from you) that the COM server is 32-bit.

The workaround clears all 64 bits of memory, which lets the 32-bit pointer keep its same 32-bit value when JNA sees it as 64 bits.  Without clearing the memory, the behavior (as Matthias said someone would say, and I'll be that someone) is undefined.

Certainly JNA could be modified to defensively clear all memory it allocates, but that would incur a performance hit for every such allocation.

--

You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/1ff347414dcbb85d91173dbc2f70a027203096c9.camel%40doppel-helix.eu.

--

Dan Widdis

[Matthias Bläsing]

Hi,

Am Montag, dem 12.02.2024 um 09:35 -0800 schrieb Daniel B. Widdis:

Is the difference between BSTRByReference(new BSTR()) and BSTRByReference() a bug in JNA, or is QS-STAT's expectation a violation of Microsoft's COM rules

Option C: None of the above.

JNA is not designed to use different pointer size than the operating system, and when interacting with another program it is up to the user to identify appropriate requirements for using its API; in fact, there's no way for JNA to know (without some help from you) that the COM server is 32-bit.

do you have documentation on this? I highly doubt, that a COM call is that simple. As the caller you don't know the bitness of your target and thus you can't and don't need to expect, that your datatype is only partially written/read.

I assume, that the call goes through a proxy, that marshalls the parameters going out and in and takes care of pointer mapping. No magic involved.

Even with a perfect mapping, if the assumptions of the proxy are not held, bad things can happen.

Greetings

Matthias

[Markus Karg]

Actually I am not convinced that the problem really is related to the 32/64 Bit mismatch, as IMHO what JNA actually talks to is not the 32 Bit COM server itself but a 64 Bit COM stub created by the operating system on the fly, hence there is no pointer mismatch (at least not on the client side of DCOM). I cannot proof it but I would rather say the problem is that it is never a good idea to provide a random pointer to any system function (here: to the OS-generated sub), as we do not know what the OS does with that pointer (I assume it reads the memory as it does not know that this is "just" an OUT parameter, not an IN-OUT parameter, so the OS sends -or tries to send- the referenced data to the actual 32 Bit COM server by DCOM). Also, JACOB works well out of the box without any 32/64 Bit mismatch problems or manual adjustments on the client application, which would be impossible in your paradigma (JACOB must fail too, but it succeeds, without any "tricks").

Regarding performance, I doubt that the one or two machine code instructions that are needed to zero the pointer would provide any measurable performance impact, as both, JACOB and Panama do perform such upfront zeroing always, and both are way faster than JNA.

So IMHO I would plea for always zeroing the pointer, hence let people simply use new BSTRByRef() without new BSTR() in future -- and have much less headache.

[Markus Karg]

Matthias,

in any case, I would plea for zeroing the pointer in the BSTRByReference() default constructor, so in future people have no problem with BSTRByReference. :-)

Regarding DoubleByReference, the use case is the Q-DAS function GetStatResult(handle, ..., BSTRByReference, DoubleByReference). I will post the full example tomorrow, the outcome was posted already in a separate thread. :-)

[Matthias Bläsing]

Hi,

Am Montag, dem 12.02.2024 um 10:14 -0800 schrieb Markus Karg:

in any case, I would plea for zeroing the pointer in the BSTRByReference() default constructor, so in future people have no problem with BSTRByReference. :-)

Regarding DoubleByReference, the use case is the Q-DAS function GetStatResult(handle, ..., BSTRByReference, DoubleByReference). I will post the full example tomorrow, the outcome was posted already in a separate thread. :-)

you wrote:

 VARIANT(VT_R4 | VT_BYREF, new DoubleByReference()),

that can't work. A C double ist 64bit wide.  Indeed this code runs without problems. ForGetStatResult I see a VT_R8|VT_BYREF.

import com.sun.jna.platform.win32.Variant;

import com.sun.jna.platform.win32.Variant.VARIANT;

import com.sun.jna.ptr.DoubleByReference;

import com.sun.jna.ptr.FloatByReference;

public class FloatingpointVariantByRef {

    public static void main(String[] args) {

        FloatByReference floatResult = new FloatByReference();

        DoubleByReference doubleResult = new DoubleByReference();

        VARIANT v = new Variant.VARIANT();

        v.setValue(Variant.VT_R4 |Variant.VT_BYREF, floatResult);

        v.setValue(Variant.VT_R8 |Variant.VT_BYREF, doubleResult);

        // Broken: Mismatch width

//         v.setValue(Variant.VT_R4 |Variant.VT_BYREF, doubleResult);

//         v.setValue(Variant.VT_R8 |Variant.VT_BYREF, floatResult);

    }

}

Greetings

Matthias

[Markus Karg]

Matthias,

thanks again, you are so amazing! 🦸‍♂️

Using setValue(VT_R8 | VT_BYREF, dblByRef) it works like a charm!

In fact I was under the impression to have tested this already, but you do proof me wrong again! 😅

What I do wonder about still is:

• Wouldn't it make sense to provide VARIANT(DoubleByReference) and VARIANT(int vt, Object value) constructors in a future JNA release, so one could leave out the subsequent setValue(int vt, Object value) call?
• Frankly spoken I have not quite understood why I have to manually tell JNA that a DoubleByReference actually should be a VT_R8 | VT_BYREF variant. Shouldn't it simply work out of the box to directly use DoubleByReference with an @ComMethod mapping? That would make things easier for application programmers.

If you could elaborate on that, I would really appreciate, as I will better understand JNA then. ☺️

[Matthias Bläsing]

Hi,

Am Dienstag, dem 13.02.2024 um 01:20 -0800 schrieb Markus Karg:

What I do wonder about still is:

• Wouldn't it make sense to provide VARIANT(DoubleByReference) and VARIANT(int vt, Object value) constructors in a future JNA release, so one could leave out the subsequent setValue(int vt, Object value) call?

I think I considered this in the past, but if I remember correctly I discarded that because there is already a huge number of constructors and did not want to further that chaos.

• Frankly spoken I have not quite understood why I have to manually tell JNA that a DoubleByReference actually should be a VT_R8 | VT_BYREF variant. Shouldn't it simply work out of the box to directly use DoubleByReference with an @ComMethod mapping? That would make things easier for application programmers.

Don't know why. It was as it is today when I first came to JNA (also because I was interested in COM automation) and in review it is easy to judge.

Greetings

Matthias

[Markus Karg]

If I find the time, I will look into both and maybe provide a PR, as I think it would be very useful. It would have prevented this hole discussion possibly.