java and pmem?

[Nicholas Murphy]

Does anyone know of any efforts to enable Java access to pmem in any form?

Thanks,
Nick

[Andy Rudoff]

Hi Nick,

Sorry, almost forgot to reply to this.

I'm aware of several efforts around Java, happening in several different companies.  As an Intel employee, I can only speak for Intel but perhaps other companies represented on this list will chime in about their efforts.

At Intel we're looking at Java in multiple ways.  First, there's the most basic, least transparent way, where Java applications use persistent memory by calling into libraries to get the features we've implemented in the NVM Libraries.  This might give applications access to persistent memory quickly, but is the least Java-like and therefor not that friendly to Java application programmers.  So the next step is to use the NVM Libraries to create Java libraries that fit much better into the Java ecosystem.  I'm not able to point to public information about that work yet, but there's lots of interesting work going on and as I find public information I'll post it here (please anyone else also post what you find in this space).

Finally, there are several efforts to leverage persistent memory in a way that's transparent to applications.  This is obviously the easiest to adopt, since it does not require application changes, but instead it requires changes to some layer of software that the application uses.  The benefit might not be as high as an application being modified to leverage persistent memory directly, but the low barrier to adoption is very compelling.

Okay, so that wasn't a very detailed answer because I don't want to publicly announce what people are doing before they're ready.  But I'd be very interested to hear ideas from others on this list on how they think Java could best leverage persistent memory and I bet the discussion will draw out comments from people doing work in this area...

-andy

[Francesco Nigro]

HI Andy!

As a Java dev, I'm a lot interested to hear them too :)

I've worked on a memory mapped version of a journal for an open source broker and I was always asking myself: it wouldn't be awesome if when the JVM asks to memory map a file that is mounted on a persistent storage then every writes will avoid of being managed by the virtual memory system of the OS and hit directly the storage itself?

Hence, IMHO that would be the best way to integrate the persistent storage solutions transparently to any Java dev, but I suppose that It couldn't be so simple :)

Or it is something that could be feasable?wdyt?

Regards,

Francesco

[Andy Rudoff]

Hi Francesco,

What you outline sounds interesting to me, but I admit I'm no Java expert.  Also, in the time since the last post in this thread, a group in Intel has created an open source project called "Persistent Collections for Java" and you might find that work interesting: https://github.com/pmem/pcj

-andy

[Francesco Nigro]

Thank you Andy!

Seems interesting, I've to dig a lil' more around the ByteBuffer implementation on that project, but seems that they haven't subclassed ByteBuffer, hence reusing existing code would be not so plain and simple :(

Anyway I'll try to ping them regarding it!

Thanks for pointing me that project anyway!

Francesco

[Lei Fan]

Hi Francesco,

This is Lei from the Persistent Collections for Java project. We are so glad that you took the time to look into our library, and hope that you found it interesting and useful!

Indeed, we have not subclassed ByteBuffer. The issue is that, although the Javadoc for java.nio.ByteBuffer does not indicate that it is final, the abstract ByteBuffer class contains two package private abstract methods that we are not able to override legally, as we are not part of the package. Since no one outside of the java.nio package can override the two package private abstract methods, the java.nio.ByteBuffer is, effectively, "final". Instead, we attempted to mirror (almost) every functionality that a java.nio.ByteBuffer offers, in the hope that this would be sufficient, though we understand (and tried very hard to get around, but to no avail) the difficulties in migrating code from using a java.nio.ByteBuffer to using a lib.persistent.PersistentByteBuffer.

We would absolutely love to hear more from you regarding this issue or anything else that you see as you explore our library - your feedback would be greatly appreciated as we continue our active development. Please feel free to reach out to us at the email addresses listed on the PCJ GitHub page.

Thank you very much,

Lei