[BUG?] WMI: java.lang.Error: Invalid memory access

[Darius Juodokas]

This looks like a potential bug, but just in case it's not, I'm posting my problem here.

I'm trying to utilize WMI using JNA, using only JNA-provided classes and methods, nothing custom yet. I'm basing my example on @matthiasblaesing snippet here: https://github.com/java-native-access/jna/issues/1083#issuecomment-482720999 . 

I'm also pasting the code that's failing below. I'm trying to fetch all the properties of Network Adapter Config and print them all one-by-one. The code fails at result[0].Get(/**/) for properties: "DHCPEnabled", "IPConnectionMetric", "IPEnabled", "InterfaceIndex", "Index".

Please mind the commented out line. It contains the properties that JNA failed to fetch for me. But if I only select those properties (i.e. uncomment that line), JNA succeeds and the program exits successfully.

If I shuffle the initial property set, I get failures at different properties.

Am I using this Matthias' example wrong, or is there in fact a bug?

P.S. I got the list of failing properties by wrapping result[0].Get(/**/) inside try-catch, not by removing them from the set.

```

import com.sun.jna.platform.win32.COM.COMUtils;

import com.sun.jna.platform.win32.COM.Wbemcli;

import com.sun.jna.platform.win32.COM.WbemcliUtil;

import com.sun.jna.platform.win32.Ole32;

import com.sun.jna.platform.win32.OleAuto;

import com.sun.jna.platform.win32.Variant;

import com.sun.jna.ptr.IntByReference;

public class SOExample {

    public static void main(String[] args) {

        Ole32.INSTANCE.CoInitializeEx(null, Ole32.COINIT_MULTITHREADED);

        Wbemcli.IWbemServices svc = WbemcliUtil.connectServer("ROOT\\CIMV2");

        try {

            String[] props = {"Caption", "Description", "SettingID", "ArpAlwaysSourceRoute", "ArpUseEtherSNAP", "DatabasePath", "DeadGWDetectEnabled", "DefaultIPGateway", "DefaultTOS", "DefaultTTL", "DHCPEnabled", "DHCPLeaseExpires", "DHCPLeaseObtained", "DHCPServer", "DNSDomain", "DNSDomainSuffixSearchOrder", "DNSEnabledForWINSResolution", "DNSHostName", "DNSServerSearchOrder", "DomainDNSRegistrationEnabled", "ForwardBufferMemory", "FullDNSRegistrationEnabled", "GatewayCostMetric", "IGMPLevel", "Index", "InterfaceIndex", "IPAddress", "IPConnectionMetric", "IPEnabled", "IPFilterSecurityEnabled", "IPPortSecurityEnabled", "IPSecPermitIPProtocols", "IPSecPermitTCPPorts", "IPSecPermitUDPPorts", "IPSubnet", "IPUseZeroBroadcast", "IPXAddress", "IPXEnabled", "IPXFrameType", "IPXMediaType", "IPXNetworkNumber", "IPXVirtualNetNumber", "KeepAliveInterval", "KeepAliveTime", "MACAddress", "MTU", "NumForwardPackets", "PMTUBHDetectEnabled", "PMTUDiscoveryEnabled", "ServiceName", "TcpipNetbiosOptions", "TcpMaxConnectRetransmissions", "TcpMaxDataRetransmissions", "TcpNumConnections", "TcpUseRFC1122UrgentPointer", "TcpWindowSize", "WINSEnableLMHostsLookup", "WINSHostLookupFile", "WINSPrimaryServer", "WINSScopeID", "WINSSecondaryServer"};

//            props = new String[]{"DHCPEnabled", "IPConnectionMetric", "IPEnabled", "InterfaceIndex", "Index"}; // REMOVE THIS LINE TO MAKE IT FAIL

            Wbemcli.IEnumWbemClassObject enumerator = svc.ExecQuery("WQL",

                    "SELECT " + String.join(",", props) + " FROM Win32_NetworkAdapterConfiguration",

                    Wbemcli.WBEM_FLAG_FORWARD_ONLY | Wbemcli.WBEM_FLAG_RETURN_IMMEDIATELY, null);

            try {

                Wbemcli.IWbemClassObject[] result;

                Variant.VARIANT.ByReference pVal = new Variant.VARIANT.ByReference();

                IntByReference pType = new IntByReference();

                IntByReference plFlavor = new IntByReference();

                while ((result = enumerator.Next(1000, 1)).length > 0) {

                    System.out.println("-------------RESULT_ENTRY------------");

                    for (String prop : props) {

                        System.out.print(prop + "> ");

                        COMUtils.checkRC(result[0].Get(prop, 0, pVal, pType, plFlavor));

                        System.out.println(pVal.getValue());

                        OleAuto.INSTANCE.VariantClear(pVal);

                    }

                    result[0].Release();

                }

            } catch (Exception e) {

                throw new RuntimeException(e);

            } finally {

                enumerator.Release();

            }

        } finally {

            svc.Release();

            Ole32.INSTANCE.CoUninitialize();

        }

    }

}

```

[Darius Juodokas]

Forgot to mention JNA version:

implementation group: 'net.java.dev.jna', name: 'jna-platform', version: '5.8.0'

[Darius Juodokas]

java: OpenJDK 11.

OS: Windows 10 (downloaded yesterday), x64.

Platform: libvirt (hosted on LinuxMint)

[Daniel B. Widdis]

Haven't found the culprit yet, but offering some debug info:

- Reproduced with user code

- Does not happen with every interface, just one.

- Simplified reproduction with shorter list.  The problem occurs with the first array member, "DefaultIPGateway".  When it is in the list, the crash occurs a few entries later.

- When adding in a delay between reading members, the problem still occurs a few entries later.

Problematic SAFEARRAY causing the crash -- clearly valid here.   Something appears to happen to this Oaldl.SAFEARRAY.ByReference() pointer between here and a later call; the invalid memory access is attempting to retrieve the cDims member later on a completely unrelated call.

DefaultIPGateway> OaIdl$SAFEARRAY$ByReference(native@0x1f4f4193980) (32 bytes) {

  WinDef$USHORT cDims@0x0=1

  WinDef$USHORT fFeatures@0x2=384

  WinDef$ULONG cbElements@0x4=8

  WinDef$ULONG cLocks@0x8=0

  WinDef$PVOID pvData@0x10=native@0x1f4f4365a20 (com.sun.jna.platform.win32.WinDef$PVOID@f4365c14)

  OaIdl$SAFEARRAYBOUND rgsabound[1]@0x18=[Lcom.sun.jna.platform.win32.OaIdl$SAFEARRAYBOUND;@14a2f921

}

My suspicion is the pointer to this isn't being cleared out and is being called again when it's not the right type.

Will keep looking in a few hours.

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/47f1b556-5f7a-449d-91d4-4baa0085f7dcn%40googlegroups.com.

--

Dan Widdis

[Daniel B. Widdis]

The pointer re-use theory is getting stronger.  Can reproduce with only the two Strings "DefaultIPGateway", "DHCPEnabled".  The latter is a boolean.

Tried with "IPAddress" as the second string and got a Segfault.

I suspect the problem is with pVal which is defined outside the loop but then cleared in the loop.

OleAuto.INSTANCE.VariantClear(pVal); 

--

Dan Widdis

[Daniel B. Widdis]

Still don't know the root cause, but:

 - the "Type" of the results is not changing when called with the SAFEARRAY argument, so the next non-null result tries to parse a structure.

 - the problem can be fixed by moving the pVal declaration inside the loop.

                Wbemcli.IWbemClassObject[] result;

                // REMOVE FROM HERE

                // Variant.VARIANT.ByReference pVal = new Variant.VARIANT.ByReference();

                IntByReference pType = new IntByReference();

                IntByReference plFlavor = new IntByReference();

                while ((result = enumerator.Next(1000, 1)).length > 0) {

                    System.out.println("-------------RESULT_ENTRY------------");

                    for (String prop : props) {

                        // ADD HERE

                        Variant.VARIANT.ByReference pVal = new Variant.VARIANT.ByReference();

                        System.out.print(prop + "> ");

                        COMUtils.checkRC(result[0].Get(prop, 0, pVal, pType, plFlavor));

                        System.out.println(pVal.getValue());

                        OleAuto.INSTANCE.VariantClear(pVal);

                        Util.sleep(1000);

                    }

                    result[0].Release();

                }

Not sure yet whether there's an underlying bug or if this is intentional.

--

Dan Widdis

[Daniel B. Widdis]

This is really interesting (and/or confusing).

The documentation for IWbemClassObject::Get states:

When successful, this parameter is assigned the correct type and value for the qualifier, and the VariantInit function is called on pVal. It is the responsibility of the caller to call VariantClear on pVal when the value is not needed. If there is an error, the value that pVal points to is not modified. If an uninitialized pVal value is passed to the method, then the caller must check the return value of the method, and call VariantClear only when the method succeeds.

We are hitting the "not modified" case here.  Unfortunately, the unmodified value causes the type of the object to remain the same (Structure) and the invalid memory access occurs internally to the Get() before we have a chance to check the HRESULT.

Skipping the "DHCPEnabled" field skips that particular error, but it fails later on the "Index" field shortly after a successful return from "GatewayCostMetric".  Skipping that field errors on "InterfaceIndex" after the same array.  Skipping that one successfully grabs the next SAFEARRAY but fails on "IPConnectionMetric"

The commonality here is:

 - Getting an array result (SAFEARRAY)

 - Passing the re-used VARIANT.ByReference to another call

 - Attempting to fetch a non-null, non-array, non-String value (boolean, uint32) which causes the call to fail, with the result object type (SAFEARRAY) unmodified, and the error occurs

I think I'm at the limit of my expertise here.   Some thoughts:

 - I still don't know why the calls are failing for the boolean/uint32 types if pVal is previously populated with array types.  This seems to be the direction to keep going to figure out the root cause.

 - Is there a way inside JNA's implementation to check the (failed) result before trying to read invalid pointers in an "unchanged" structure type?  

--

Dan Widdis

[Matthias Bläsing]

Hi,

based on Daniels suggestion I modified the original sample and pushed it here:

https://github.com/matthiasblaesing/JNA-Demos/blob/master/WMIQuery/src/main/java/eu/doppel_helix/dev/wmiquery/NetworkAdapterConfigurationQuery.java

I think the core change is this:

https://github.com/matthiasblaesing/JNA-Demos/blob/master/WMIQuery/src/main/java/eu/doppel_helix/dev/wmiquery/NetworkAdapterConfigurationQuery.java#L65-L67

The VARIANT.ByReference is created on each iteration.

With that change I ran this multiple times and got no errors. The access to the SAFEARRAYs also works without a problem, giving me for example lists of ip addresses.

HTH

Matthias

[Daniel B. Widdis]

To give a bit of justification to Matthias's change, a lot more type checking needs to be done when changing variant types, as referenced here:  Variant Manipulation Functions | Microsoft Docs 

I got far enough troubleshooting to determine that the active field of the _VARIANT union remaining on the old value (combined with autoRead) was the root of the problem, and the order in which types are checked and read on retrieval causes the issues.  I could manually change the pVal type (by setting a new value) before the call to work around it, but just creating a new variant is probably a better option!

As an aside, I've implemented a slightly higher level abstraction of WMI code in my own project here that you might find useful.  I don't think I handle arrays but you could copy and extend my code to do so.

oshi/WmiQueryHandler.java at master · oshi/oshi (github.com)

Calling examples here.

oshi/oshi-core/src/main/java/oshi/driver/windows/wmi at master · oshi/oshi (github.com) 

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/6e1be855e6098c73413108ff6e21f402f8e8464b.camel%40doppel-helix.eu.

--

Dan Widdis

[Darius Juodokas]

Thank you lads for looking into this. I'll adjust my code according to your recommendations.

As for the Oshi - thank you for the suggestion, I'll definitely look into it. I'd been building my own monstrosity for easier WMI querying (when ran into that weird memory access Error) that operates with primitive datatypes: Strings and doubles, Maps and Lists. Yes, very javascript-like, but I find it easier to extend than fixed datatypes, e.g. enums, which are a headache when it comes to backwards and forwards compatibility. However, I was scratching my head around extracting native values to Java ones, and this is where your examples will come in handy. Thank you!

Thanks, guys! That was some A+ class fast support! 

FTR: the monstrosity I was building, usage is in the main() method at the top. It's a WIP, still buggy and not all tested, esp structs→Map; conversion. I think I saw some .Net developers using similar WQL builders and I found this approach very intuitive. Probably that's just me :) 

P.S. that clunky Map could be later used to fill in fields should I ever decide to create class Win32_DiskDrive or similar (e.g. new Win32_DiskDrive(result)).

import com.fasterxml.jackson.databind.ObjectMapper;

import com.sun.jna.Structure;

import com.sun.jna.platform.win32.COM.COMUtils;

import com.sun.jna.platform.win32.COM.Wbemcli;

import com.sun.jna.platform.win32.COM.WbemcliUtil;

import com.sun.jna.platform.win32.OaIdl;

import com.sun.jna.platform.win32.Ole32;

import com.sun.jna.platform.win32.OleAuto;

import com.sun.jna.platform.win32.Variant;

import com.sun.jna.ptr.ByReference;

import com.sun.jna.ptr.IntByReference;

import java.io.Closeable;

import java.io.IOException;

import java.lang.reflect.Method;

import java.util.*;

public class JNAApp {

    

    public static void main(String[] args) throws IOException {

        WMI.Connection connection = new WMI.Connection("ROOT\\CIMV2");

        List<Map<String, Object>> result = connection.runQuery(new WMI.WQL()

                .Select("Caption", "Capabilities", "CapabilityDescriptions")

                .From("Win32_DiskDrive"));

        System.out.println(new ObjectMapper().writerWithDefaultPrettyPrinter().writeValueAsString(result));

    }

    public static class WMI {

        public static class Connection implements Closeable, AutoCloseable {

            private final String namespace;

            private final Wbemcli.IWbemServices svc;

            private final Map<String, List<String>> ddl;

            public Connection(String namespace) throws IOException {

                this.namespace = namespace;

                Ole32.INSTANCE.CoInitializeEx(null, Ole32.COINIT_MULTITHREADED);

                svc = WbemcliUtil.connectServer(namespace);

                ddl = new HashMap<>();

            }

            public List<Map<String, Object>> runQuery(WQL qry) {

                Wbemcli.IEnumWbemClassObject enumerator = svc.ExecQuery("WQL", qry.build(),

                        Wbemcli.WBEM_FLAG_FORWARD_ONLY | Wbemcli.WBEM_FLAG_RETURN_IMMEDIATELY, null);

                List<Map<String, Object>> result = new ArrayList<>();

                try {

                    List<String> properties = new ArrayList<>();

                    for (String field : qry.getFields()) {

                        if (field.equals("*")) {

                            properties.addAll(getAllProperties(qry.getTable()));

                        } else {

                            properties.add(field);

                        }

                    }

                    Wbemcli.IWbemClassObject[] resultObject;

                    IntByReference pType = new IntByReference();

                    IntByReference plFlavor = new IntByReference();

                    while ((resultObject = enumerator.Next(1_000, 1)).length != 0) {

                        Map<String, Object> resultItem = new HashMap<>();

                        for (String field : properties) {

                            Variant.VARIANT.ByReference pVal = new Variant.VARIANT.ByReference();

                            try {

                                COMUtils.checkRC(resultObject[0].Get(field, 0, pVal, pType, plFlavor));

                                resultItem.put(field, getValue(field, pVal.getValue()));

                            } catch (Throwable e) {

                                resultItem.put(field, e);

                            } finally {

                                OleAuto.INSTANCE.VariantClear(pVal);

                            }

                        }

                        result.add(resultItem);

                        resultObject[0].Release();

                    }

                } finally {

                    enumerator.Release();

                }

                return result;

            }

            private Object getValue(String property, Object value) {

//                System.out.println("Type[" + property + "]: " + (value == null ? null : value.getClass()));

                if (value == null) {

                    return null;

                } else if (value instanceof OaIdl.SAFEARRAY) {

                    OaIdl.SAFEARRAY safeArray = (OaIdl.SAFEARRAY) value;

                    List<Object> array = new ArrayList<>();

                    for (int i = safeArray.getLBound(0); i <= safeArray.getUBound(0); i++) {

                        array.add(getValue(property, safeArray.getElement(i)));

                    }

                    return array;

                } else if (value instanceof Number) {

                    return ((Number) value).doubleValue();

                } else if (value.getClass().getSimpleName().endsWith("STR")) {

                    return value.toString();

                } else if (value instanceof ByReference) {

                    try {

                        Method getter = value.getClass().getMethod("getValue");

                        getter.setAccessible(true);

                        return getValue(property, getter.invoke(this));

                    } catch (Exception e) {

                        e.printStackTrace();

                    }

                } else if (value instanceof Structure) {

                    try {

                        Method getter = value.getClass().getMethod("fields");

                        getter.setAccessible(true);

                        Map<String, Object> fields = (Map<String, Object>) getter.invoke(this);

                        Map<String, Object> struct = new HashMap<>();

                        fields.forEach((fName, field) -> {

                            try {

                                Method fGetter = value.getClass().getMethod("readField");

                                fGetter.setAccessible(true);

                                Object fValue = fGetter.invoke(value, field);

                                struct.put(fName, getValue(fName, fValue));

                            } catch (Exception e) {

                                e.printStackTrace();

                            }

                        });

                        return struct;

                    } catch (Exception e) {

                        e.printStackTrace();

                    }

                } else {

//                    System.out.println("RAW Type[" + property + "]: " + (value == null ? null : value.getClass()));

                }

                return value;

            }

            public Collection<String> getAllProperties(String table) {

                if (!this.ddl.containsKey(table)) {

                    Wbemcli.IEnumWbemClassObject enumerator = svc.ExecQuery("WQL",

                            "SELECT * FROM " + table,

                            Wbemcli.WBEM_FLAG_FORWARD_ONLY | Wbemcli.WBEM_FLAG_RETURN_IMMEDIATELY, null);

                    try {

                        Wbemcli.IWbemClassObject[] resultObject;

                        if ((resultObject = enumerator.Next(1_000, 1)).length > 0) {

                            Variant.VARIANT.ByReference pNames = new Variant.VARIANT.ByReference();

                            String[] names = resultObject[0].GetNames(null, 0, pNames);

                            OleAuto.INSTANCE.VariantClear(pNames);

                            if (names != null) {

                                this.ddl.put(table, List.of(names));

                            } else {

                                System.err.println("Failed to get properties of table " + table);

                            }

                            resultObject[0].Release();

                        } else {

                            System.err.println("Failed to get any results from table " + table);

                        }

                    } finally {

                        enumerator.Release();

                    }

                }

                return this.ddl.getOrDefault(table, new ArrayList<>());

            }

            @Override

            public void close() throws IOException {

                svc.Release();

                Ole32.INSTANCE.CoUninitialize();

            }

        }

        public static class WQL {

            private String[] fields;

            private String table;

            private String filter;

            public WQL() {

                this.fields = new String[]{"*"};

            }

            public WQL Select(String prop1, String... properties) throws IllegalArgumentException {

                prop1 = assertGoodProperty(prop1);

                this.fields = new String[properties.length + 1];

                this.fields[0] = prop1;

                for (int i = 0; i < properties.length; i++) {

                    String prop = properties[i];

                    prop = assertGoodProperty(prop);

                    this.fields[i + 1] = prop;

                }

                return this;

            }

            public WQL From(String table) throws IllegalArgumentException {

                this.table = assertGoodProperty(table);

                return this;

            }

            public WQL Filter(String filter) throws IllegalArgumentException {

                this.filter = filter;

                return this;

            }

            public String[] getFields() {

                return fields;

            }

            public String getTable() {

                return table;

            }

            protected String build() throws IllegalStateException {

                if (this.fields == null || this.fields.length == 0) {

                    throw new IllegalStateException("Missing properties in the WQL");

                }

                if (this.table == null || this.table.isBlank()) {

                    throw new IllegalStateException("Missing table in the WQL");

                }

                StringBuilder sb = new StringBuilder("SELECT ")

                        .append(String.join(", ", this.fields))

                        .append(" FROM ").append(this.table);

                if (this.filter != null) {

                    sb.append(" ").append(this.filter);

                }

                return sb.toString();

            }

            private String assertGoodProperty(String prop) throws IllegalArgumentException {

                if (prop == null || (prop = prop.trim()).isEmpty()) {

                    throw new IllegalArgumentException("Missing property: " + prop);

                }

                return prop;

            }

        }

    }

}

[Daniel Widdis]

At the end of the day, Java has single dimensional structures and WMI results have two dimensions.

 

You either have a map/structure of lists as I’ve implemented, or your suggestion of a list of map/structure.

 

I do like your structure approach and if I were starting from scratch I might do it that way.  At the moment, I’ve got a battle-tested conglomeration that works and is self-documenting and I don’t have the mental energy to start over 😊

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/e019cb0b-2e1c-4b30-856d-1da128bce86dn%40googlegroups.com.