Byte array to Structure without native memory

[Daniel Widdis]

This isn't strictly a JNA question, because I'm asking how to do something JNA does very well ... without JNA. :)

I have a use case to turn a byte array into a structure.  Specifically, I am trying to import the /proc/[pid]/psinfo file available on Solaris and AIX proc filesystems and map it to a Java class implementing the C struct.

For example, Solaris/Illumos has the psinfo structure defined here:

https://github.com/joyent/illumos-joyent/blob/1c32b49d7fb5206ef37ab8485bc1eca433781374/usr/src/uts/common/sys/procfs.h#L281-L321

I could map this in JNA with the appropriate types (many of which are variable sized with 32- or 64-bits), read in the file to a byte array, write() the values to the memory backing a Structure, and read() that memory into the Structure.  A few lines of code and JNA's magic does everything I need.

But there's no need to create a native memory allocation for this processing magic. I just want JNA to do its magic on a Java byte[].

Are there any shortcuts (JNA utility methods) to do what I want?  Is copying the reflection code from Structure not a good idea?  Are there better core java methods for this use case?    

[L Will Ahonen]

Hi,

Not actually answering your question, just questioning if you need to worry about it at all. Does this bit of code get called in a hot loop?

If you really do need to worry about the extra memory allocation, I guess you could call libc.read() to get the data directly into a Memory object instead of going through a byte[], and then call the Structure constructor using that backing memory buffer. Probably even faster to call libc.malloc() and use that pointer instead of a Memory, and keeping that pointer around to reuse the native memory block for subsequent reads.

I guess what I'm trying to say is: If performance matters, using the byte[] directly won't save you many cycles - and if performance doesn't matter, just allocate a Memory, write the byte[] into it and call it a day.

Cheers,

Will

[Daniel Widdis]

Thanks for the suggestion, Will. 

Don't think I'd call it a hot loop.  It is called once for every process on a system as often as the user polls.  If you're thinking something like "top" that'd be every few seconds.  So the process iteration is the bigger impact, and leaving open the opportunity for multithreading would challenge memory re-use.  I suspect all the Reflection overhead of JNA (and multiple reads to fetch each field) would be a bigger impact.

I ended up starting down a different path before seeing this reply and it seemed to work well, so I'll probably leave it.  My approach:

 - created an enum with the values representing the structure fields.  Each enum also carried an integer of the size in bytes.

 - manually adjusted padding for 8-byte alignment boundary

 - created a static EnumMap to iterate the enum and store offsets based on the byte sizes

Then, for each process when queried:

 - To query the values, read a RandomAccessFile into a ByteBuffer via a FileChannel/ByteArrayOutputStream

 - Used the ByteBuffer getters (using the enum size to know which one) at the appropriate offsets from the EnumMap to fetch the values.

Related to your reply, I did end up using libc.pread() later, to read strings from the process address space into memory for a different purpose.  Perhaps my followup question on that should be a new thread, though.

Dan